[Django]-Does restarting celery cause duplicate tasks?

3👍

Task duplicating is possible if worker/beat processes had not stopped correctly. How do you restart celery workers/beat? Check server for zombie celery worker and beat processes. Try to stop all celery processes, check no processes of celery exist and start it again. After all check that ps ax | grep celery shows fresh workers and only one beat.

👤tug

2👍

Tasks won’t restart in case of incorrect worker stop if you set CELERY_ACKS_LATE = False. In this case the task marked as acknowledged immediately after consuming. See docs.

Also make sure that your tasks have no retry enabled. If any exception happens inside task – they might retry with the same input arguments.

Another possible case – your tasks are written wrong and each run selects the same recipients set.

👤baldr

Leave a comment