Intermediate · 4h 45m · 18 lessons
Celery
Run slow work in the background — tasks, retries, workflows, and scaling.
What you'll learn
- ✦Explain why moving slow work into background tasks keeps a web app fast for users
- ✦Define Celery tasks and call them with delay() or apply_async() and the right options
- ✦Explain how the broker, the workers and the result backend work together to run a task
- ✦Add retries with backoff and write idempotent tasks that are safe to run twice
- ✦Build workflows that run tasks in order and in parallel with chains, groups and chords
- ✦Scale with more workers, concurrency and queues, and monitor them in production
Part of these career paths
- Backend Developer path · course 11 of 14
Certification projects
Build 4 projects to earn your certificate
Hands-on work that proves you can apply what you learned — part of the certificate requirements.
Syllabus
Module 1 · 51m
Why background tasks Free
What you'll be able to do
- Can work out how long a user waits when slow work runs inside the request, and after it moves to a task
- Can explain what Celery is and which process actually runs a task
- Can say what the broker, the worker and the result backend each do, and start a broker and a worker
Module 2 · 51m
Tasks Locked
What you'll be able to do
- Can turn a Python function into a Celery task with @app.task or @shared_task
- Can queue a task with delay(), or with apply_async() and options such as countdown, eta and queue
- Can decide when a task needs a result backend and check a result without making the user wait
Module 3 · 47m
Reliability Locked
What you'll be able to do
- Can add automatic retries with backoff and a retry cap to a task that calls a flaky service
- Can make a task idempotent so a second run causes no extra effect
- Can decide when to turn on acks_late and explain what it does and doesn't protect against
Module 4 · 46m
Scaling Locked
What you'll be able to do
- Can work out how many workers and how much concurrency a backlog needs, and pick the right pool
- Can route slow and fast tasks to separate queues, each with its own workers
- Can set a rate limit that keeps several workers under an outside API's cap
Module 5 · 45m
Workflows Locked
What you'll be able to do
- Can build a chain and predict which arguments each step receives
- Can choose between a chain, a group and a chord, and write a chord that combines results
- Can schedule tasks with Beat using crontab or seconds, with one Beat and the right time zone
Module 6 · 46m
Production Locked
What you'll be able to do
- Can read Flower and queue numbers to tell whether workers are keeping up
- Can decide whether a job belongs in a Celery task, a lighter tool or the request itself
- Can take a task through a production checklist: idempotency, acks_late, retries, time limits, visibility timeout