8
It is not necessary but good and helpful to run a separate process to run tasks in the background.
When you run a server, a process is created – run ps aux | grep runserver – which is responsible for serving web requests. When you say that you want to run certain tasks in the background, it implicitly means that you want a separate process to execute those tasks. This is where asynchronous task tools like celery come in.
You can also spawn a separate process yourself – as you said – by doing:
import subprocess
process = subprocess.Popen(['python', 'manage.py','process_tasks'], stdout=subprocess.PIPE, stderr=subprocess.PIPE
This method is also completely fine if you have just one or two small tasks that you want to run in parallel. However, when you have tons of complicated tasks that you are running in the background, you would want to manage them properly. Also, you need to be able to debug those tasks, if something goes wrong. Later, you will need more visibility into what is happening in all the background tasks, their status, etc. This is where celery will help you. It will give you decorated methods which will handle all those things for you. You just have to worry about your business logic then
