[Django]-Detect whether Celery is Available/Running

66πŸ‘

βœ…

Here’s the code I’ve been using. celery.task.control.Inspect.stats() returns a dict containing lots of details about the currently available workers, None if there are no workers running, or raises an IOError if it can’t connect to the message broker. I’m using RabbitMQ – it’s possible that other messaging systems might behave slightly differently. This worked in Celery 2.3.x and 2.4.x; I’m not sure how far back it goes.

def get_celery_worker_status():
    ERROR_KEY = "ERROR"
    try:
        from celery.task.control import inspect
        insp = inspect()
        d = insp.stats()
        if not d:
            d = { ERROR_KEY: 'No running Celery workers were found.' }
    except IOError as e:
        from errno import errorcode
        msg = "Error connecting to the backend: " + str(e)
        if len(e.args) > 0 and errorcode.get(e.args[0]) == 'ECONNREFUSED':
            msg += ' Check that the RabbitMQ server is running.'
        d = { ERROR_KEY: msg }
    except ImportError as e:
        d = { ERROR_KEY: str(e)}
    return d

22πŸ‘

From the documentation of celery 4.2:

from your_celery_app import app


def get_celery_worker_status():
    i = app.control.inspect()
    availability = i.ping()
    stats = i.stats()
    registered_tasks = i.registered()
    active_tasks = i.active()
    scheduled_tasks = i.scheduled()
    result = {
        'availability': availability,
        'stats': stats,
        'registered_tasks': registered_tasks,
        'active_tasks': active_tasks,
        'scheduled_tasks': scheduled_tasks
    }
    return result

of course you could/should improve the code with error handling…

πŸ‘€Ouss

13πŸ‘

To check the same using command line in case celery is running as daemon,

  • Activate virtualenv and go to the dir where the β€˜app’ is
  • Now run : celery -A [app_name] status
  • It will show if celery is up or not plus no. of nodes online

Source:
http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/

πŸ‘€akashbw

7πŸ‘

The following worked for me:

import socket
from kombu import Connection

celery_broker_url = "amqp://localhost"

try:
    conn = Connection(celery_broker_url)
    conn.ensure_connection(max_retries=3)
except socket.error:
    raise RuntimeError("Failed to connect to RabbitMQ instance at {}".format(celery_broker_url))
πŸ‘€Lukas S.

6πŸ‘

One method to test if any worker is responding is to send out a β€˜ping’ broadcast and return with a successful result on the first response.

from .celery import app  # the celery 'app' created in your project

def is_celery_working():
    result = app.control.broadcast('ping', reply=True, limit=1)
    return bool(result)  # True if at least one result

This broadcasts a β€˜ping’ and will wait up to one second for responses. As soon as the first response comes in, it will return a result. If you want a False result faster, you can add a timeout argument to reduce how long it waits before giving up.

πŸ‘€Tim Tisdall

3πŸ‘

I found an elegant solution:

from .celery import app
try:
    app.broker_connection().ensure_connection(max_retries=3)
except Exception as ex:
    raise RuntimeError("Failed to connect to celery broker, {}".format(str(ex)))
πŸ‘€Sri

2πŸ‘

You can use ping method to check whether any worker (or specific worker) is alive or not
https://docs.celeryproject.org/en/latest/_modules/celery/app/control.html#Control.ping

celey_app.control.ping()

πŸ‘€mirhossein

1πŸ‘

The below script is worked for me.

    #Import the celery app from project
    from application_package import app as celery_app
    def get_celery_worker_status():
        insp = celery_app.control.inspect()
        nodes = insp.stats()
        if not nodes:
            raise Exception("celery is not running.")
        logger.error("celery workers are: {}".format(nodes))
        return nodes
πŸ‘€bSr

1πŸ‘

You can test on your terminal by running the following command.

celery -A proj_name worker -l INFO

You can review every time your celery runs.

πŸ‘€Sohail Ahmed

1πŸ‘

Run celery status to get the status.

When celery is running,

(venv) ubuntu@server1:~/project-dir$ celery status
->  celery@server1: OK

1 node online.

When no celery worker is running, you get the below information displayed in terminal.

(venv) ubuntu@server1:~/project-dir$ celery status
Error: No nodes replied within time constraint
πŸ‘€SuperNova

Leave a comment