Concepts

A task queue lets your app hand slow work — sending an email, processing an upload, calling a third-party API — to a background worker, so the request that triggered it can return right away. flexiq is a task queue for Python with one distinguishing trait: it needs no broker.

Why "no broker" matters

Most Python task queues (Celery, RQ, Dramatiq, Huey) split the job into separate services you have to run and operate:

• a broker (Redis or RabbitMQ) — a standalone server that holds the list of pending jobs and hands them out to workers;
• a result backend (often another Redis, or a database) — where return values are stored; and
• the workers themselves.

flexiq collapses the broker and the result backend into a single queue database — an embedded SQLite file by default, or a Postgres server for shared deployments. Enqueued jobs and their results live in the same place. There is no separate message broker to install, secure, or monitor: with SQLite it's pip install flexiq, point a queue at a path, and start a worker; Postgres still runs its own database server, but that's the only external service.

The mental model

A queue is a SQLite (or Postgres) database that holds enqueued jobs. You instantiate one in your application code:

from flexiq import Queue
queue = Queue(db_path="tasks.db")

A task is any function decorated with @queue.task(). Enqueuing a task — calling .delay(...) on it — does not run the function. It writes a job row to the database and immediately returns a JobResult handle.

A worker is a process (or thread, or async task) that pulls pending jobs from the queue and runs the matching task function. The Rust scheduler handles dispatch, retries, rate limits, and cleanup; your Python code runs only during actual task execution. If no worker is running, enqueued jobs simply wait.

A result — the task's return value, or the exception it raised — is written back to the same database. Calling job.result(timeout=30) blocks (or await job.aresult(...) yields) until the job finishes, then returns the value or re-raises the error.

When a task fails, flexiq retries it with backoff; once it exhausts max_retries the job moves to the dead letter queue (DLQ) — a holding area for permanently-failed jobs that you can inspect and replay.

Next