[Answer]-Django Celerybeat PeriodicTask running far more than expected

by

1👍

✅

Consider creating a separate queue with one worker process and fixed rate for this type of tasks and just add the tasks in this new queue instead of running them in directly from celerybeat. I hope that could help you to figure out what is wrong with your code, is it problem with celerybeat or your tasks are running longer than expected.

@task(queue='create_report', rate_limit='0.5/m')
def create_report():
    scraper = Scraper()
    scraper.get_report()

class GetReportTask(PeriodicTask):
    run_every = datetime.timedelta(minutes=2)

    def run(self, *args, **kwargs):
        create_report.delay()

in settings.py

   CELERY_ROUTES = {
     'myapp.tasks.create_report': {'queue': 'create_report'},
   }

start additional celery worker with that would handle tasks in your queue

celery worker -c 1 -Q create_report -n create_report.local

Problem 2. Your YESTERDAY and TODAY variables are set at class level, so within one thread they are set only once.

👤singer

Leave a comment