[Django]-Celery Async Tasks and Periodic Tasks together

by

4👍

Here are a few observations, resulted from multiple trial and errors, and diving into celery’s source code.

  1. @periodic_task is deprecated. Hence it would not work.

from their source code: 

#venv36/lib/python3.6/site-packages/celery/task/base.py
def periodic_task(*args, **options):
    """Deprecated decorator, please use :setting:`beat_schedule`."""
    return task(**dict({'base': PeriodicTask}, **options))

  1. Use UTC as base timezone, to avoid timezone related confusions later on. Configure periodic task to fire on calculated times with respect to UTC. e.g. for ‘Asia/Calcutta’ reduce the time by 5hours 30mins.

  2. Create a celery.py as follows:

celery.py

import django
import os 

from celery import Celery
# set the default Django settings module for the 'celery' program.
from celery.schedules import crontab

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
# Setup django project
django.setup()

app = Celery('proj')

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
app.conf.beat_schedule = {
    'test_task': {
        'task': 'test_task',
        'schedule': crontab(hour=2,minute=0),
    }
}

and task could be in tasks.py under any app, as follows

@shared_task(name="test_task")
def test_add():
    print("Testing beat service")

Use celery worker -A proj -l info and celery beat -A proj -l info for worker and beat, along with a broker e.g. redis. and this setup should work fine.

👤anurag

Leave a comment