[Answered ]-Django concurrency with celery

First make sure that ‘inconcurrency’ is actually caused by your heavy task. If you’re using only one worker for django, you will be able to process only one request at a time, no matter what it will be. Consider having more workers for some concurrency, because it will affect also short requests.

For returning some information when task is done, you can do it in at least two ways:

• sending AJAX requests periodicaly to fetch status of your task
• using SSE or websocket to subscribe for actual result

Both of them will require to write some more JavaScript code for handling it. First one is really easy achievable, for second one you can use uWSGI capabilities, as described here. It can be handled asynchronously that way, independently of your django workers (django will just create connection and start task in celery, checking status and sending it to client will be handled by gevent.

To follow up on GwynBliedD’s answer:

celery is commonly used to process tasks, it has very simple django integration. @GwynBlieD’s first suggestion is very commonly implemented using celery and a celery result backend.

https://www.reddit.com/r/django/comments/1wx587/how_do_i_return_the_result_of_a_celery_task_to/

A common workflow Using celery is:

1. client hits heavy()
2. heavy() queues heavy() task asynchronously
3. heavy() returns future task ID to client (view returns very quickly because little work was actually performed)
4. client starts polling a status endpoint using the task ID
5. when task completes status returns result to client