[Django]-Django: start a process in a background thread?

If you really need a quick hack, simply start a process using subprocess.

But I would not recommend spawning a process (or even a thread), especially if your web site is public: in case of high load (which could be “natural” or the result of a trivial DoS attack), you would be spawning many processes or threads, which would end up using up all your system resources and killing your server.

I would instead recommend using a job server: I use Celery (with Redis as the backend), it’s very simple and works just great. You can check out many other job servers, such as RabbitMQ or Gearman. In your case, a job server might be overkill: you could simply run Redis and use it as a light-weight message server. Here is an example of how to do this.

Cheers

In case someone really wants to run another thread

def background_process():
    import time
    print("process started")
    time.sleep(100)
    print("process finished")

def index(request):
    import threading
    t = threading.Thread(target=background_process, args=(), kwargs={})
    t.setDaemon(True)
    t.start()
    return HttpResponse("main thread content")

This will return response first, then print “process finished” to console. So user will not face any delay.

Using Celery is definitely a better solution. However, installing Celery could be unnecessary for a very small project with a limited server etc.

You may also need to use threads in a big project. Because running Celery in all your servers is not a good idea. Then there won’t be a way to run a separate process in each server. You may need threads to handle this case. File system operations might be an example. It’s not very likely though and it is still better to use Celery with long running processes.

Use wisely.

IIUC, The problem here is that the webserver process might not like extra long-running threads, it might kill/spawn server processes as demand go up and down, etc etc.

You’re probably better of by communicating to an external service process for this type of processing, instead of embedding it in in the webserver’s wsgi/fastcgi process.

If the only thing you’re sending over is the filepath, it ought to be pretty easy to write that service app.