Django Celery

Django is a python-based web framework. It is an open-source and free-to-use framework. Django uses the Model View Template architecture. Django provides many functionalities to web developers. One of these functionalities is celery, and it is an asynchronous job queue used to run tasks asynchronously, i.e., in the background.

Introduction to Celery

Celery is a simple asynchronous job queue distributed system to process a large number of messages. Celery mainly works with real-time processing, but we can also use task scheduling.

Task queues are used to distribute work across the machine. It gives work based on the task unit. Task Queues are being constantly monitored to look for new tasks. The new task is initiated by the client when he sends a message to the queue. After this process, the celery, which acts as a mediator, delivers the message to the worker. Similar to Django, celery is also written in Python, but we can write the protocols in multiple languages.

Features

• Celery makes scheduling tasks very easier.
• With the help of celery, we can run applications in the background. Asynchronous tasks also improve the websites by loading the data in the foreground and running the long task in the background.
• Celery is very simple and easy to use. It does not require much setup to install. The community is also large if you need any help.
• Celery can process tasks with very quick speed.
• We can also control the number of tasks to be done in a given time.

Setup

Like Django, we will also install celery in a virtual environment. Before actually installing the celery, run the Django server. We will use the python package manager called pip to install celery. Use the below command to install:

pip install celery

Installing Development Version

We can also install the development version if we want to use the latest features of celery. To install the development version of celery, we also have to install the development version of kombu, amqp, billiard, and vine. We can use the following commands to install celery and other libraries:

pip install https://github.com/celery/celery/zipball/master#egg=celery
pip install https://github.com/celery/billiard/zipball/master#egg=billiard
pip install https://github.com/celery/py-amqp/zipball/master#egg=amqp
pip install https://github.com/celery/kombu/zipball/master#egg=kombu
pip install https://github.com/celery/vine/zipball/master#egg=vine

Implementation

To implement celery in our app, we need to create an instance of celery first. Suppose we have a project called myproject, then the structure would be:

-myproject/
-manage.py
-myproject/
-__init__.py
-settings.py
-urls.py

We have to create a file called celery.py inside the second myproject folder. So the location will be myproject/myproject/celery.py.

The celery should be like:

Celery.py file

importos


from celery import Celery




app= Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()




@app.task(bind=True)
defdebug_task(self):
print(f'Request: {self.request!r}')

After saving the celery.py file, we need to import it into our __init__.py file to make sure that Django loads it every time we run the server.

__init__.py

from .celery import app as celery_app


__all__ = ('celery_app',)

Starting Worker Process

To run the celery worker, use the below command:

celery -A tasks worker --loglevel=INFO