Celery
Celery is a distributed task queue for background jobs. It is useful when API requests should enqueue work instead of performing heavy processing inline.
Where it fits
This pattern is a good match for document ingestion, enrichment pipelines, email sending, or other tasks that should not block a normal HTTP request.
Repeated project patterns
- MaRESS uses Celery for heavier processing workflows around document and metadata enrichment.
- Celery commonly appears beside FastAPI and Redis when application work needs a queue and worker model.
Trade-offs
- Background workers improve responsiveness, but they also add operational complexity, task observability, retry semantics, and failure modes.
- Teams still need explicit boundaries for what must remain synchronous and what can become eventual work.