[Django]-Django celery and celery-beat daemonizing script error

by

2👍

✅

As django celery tasks aren’t properly a proyect module,we need to call the django.setup() method in our respective tasks modules.

In my case the solution was to set this lines inside task.py file before any other project related import:

import django
django.setup()
👤Brian Ocampo

1👍

First thing… Are you married to this project structure? 😀

Because you are just starting with celery, I’m gonna go out on a limb and say that you can move celery.py.

You could try exporting your settings module in your celeryd file if it’s in a custom path.

# regular celeryd config stuff...

# projects settings module.
export DJANGO_SETTINGS_MODULE=my_project.settings.production
export PYTHONPATH=$PYTHONPATH:/path/to/my_project/

And use exactly 'indicator_celery.settings' in your celery.py file, not config.settings.local

# celery.py file

from __future__ import absolute_import
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'indicator_celery.settings')

from django.conf import settings  # noqa

app = Celery('indicator_celery')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

also indicator_celery.__init__ file shoudl be like this:

# celery app init file
# -*- coding: utf-8 -*-

from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app  # noqa

final directory tree:

indicator_ms (project root)
├── config
│   ├── settings
│   │    ├── __init__.py
│   │    ├── local.py
│   │    ├── test.py
│   │    └── production.py
│   ├── __init__.py
│   ├── urls.py
│   └── wsgi.py    
├── indicator_celery
│   ├── __init__.py
│   └── celery.py
└── indicators (app)
    ├── __init__.py
    └── tasks.py
👤mislavcimpersak

1👍

To further @mislav’s point, there is an important caveat in the celery docs about configuration of your celery app instance:

Django users now uses the exact same template as above, but make sure that the module that defines your Celery app instance also sets a default value for DJANGO_SETTINGS_MODULE as shown in the example Django project in First steps with Django.

(emphasis added)

So make sure something similar to the following line(s) appear in your celery.py

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'my_app.settings')

from django.conf import settings  # noqa

app = Celery('my_project_name')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

If all else fails, update the initd script to source your virtual environment before you invoke celery.

👤2ps

Leave a comment