Changelog

All notable changes to FlexiQ are documented here. The format is based on Keep a Changelog; the project follows Semantic Versioning. All SDKs (Python, Node, Java) and the underlying Rust crates are released together, in lock-step.

Releases up to and including 0.23.0 were published under the project's former name, Taskito, and their entries below keep that name.

2.1.0

Namespaces become a tenancy boundary with their own quotas, pauses and periodic schedules; tokens narrow to queues and tasks, and token-authorised writes leave an audit trail. Two new clients (Go, Ruby), a Rust SDK and the fq command line, trace context from enqueue to execute, push dispatch, triggers and event egress. Read Upgrading before rolling: ten core migrations and one workflow migration run on first start.

Breaking (Rust crates)

Nothing here reaches the wire, the contract level (still 2), or the Python, Node or Java API. It reaches only a Rust program depending on flexiq-core or flexiq-workflows directly, and that is why this is a minor: the crates' own semver check is waived for 2.1.0 alone.

• flexiq-core: Storage methods take a namespace. pause_queue, resume_queue, list_paused_queues, list_workers, purge_dead, purge_dead_by_task, get_due_periodic, update_periodic_schedule, list_periodic, delete_periodic and set_periodic_enabled gain a namespace: Option<&str> parameter, on the trait and on every backend's inherent method (#836, #918). The trait also gains methods for audit, settle, quotas and draining; an out-of-tree Storage implementation must add them.
• flexiq-workflows: submit and run types carry trace context. SubmitStaticWorkflowRequest gains enqueued_by and trace, WorkflowRun gains traceparent and tracestate, and lifecycle::build_metadata_json takes a third argument (#992, #1047). A struct literal of either type must name the new fields.

Added

• SDK-served dashboards record changes in the audit trail (#1020). A dashboard the Python, Node or Java SDK serves now records every state-changing request as the flexiq-server dashboard does — signed-in user (or anonymous with auth off or the shared token), refusals included, same operation, targets and outcome — because the rule and the off-path writer moved into the shared core (flexiq_core::audit). Each prunes its namespace hourly: audit_retention_days / auditRetentionDays, else FLEXIQ_AUDIT_RETENTION_DAYS, else 90 days. The Helm chart sets FLEXIQ_AUDIT_RETENTION_DAYS whenever the dashboard or gRPC door runs, from the new audit.retentionDays (falling back to grpc.auditRetentionDays), plus audit.readsRetentionDays.

• Shorter retention for audited reads (#1018). FLEXIQ_AUDIT_READS_RETENTION_DAYS keeps read records for less than writes (default: the write window; longer is refused at startup). A new access column (migration 0027_audit_access; existing rows read write) says whether a record is a write or a read, and the hourly prune drops reads at their own window.

• Audit trail page on the dashboard (#995). The flexiq-server dashboard lists the trail under Configuration → Audit trail, newest first, filtered by principal kind, token id or username, target kind, target and a time range, paged by cursor; targets link to the job, workflow run, topic, webhook, queue, worker or token page that shows them, and a gRPC token's row menu opens what it did, revoked tokens included. Served as GET /api/audit-records, admin-only like the token list; SDK-served dashboards have no such route and hide the page.

• Dashboard changes and token lifecycle in the audit trail (#994). Every POST, PUT, PATCH or DELETE the flexiq-server dashboard answers is recorded, attributed to the signed-in user (or anonymous with auth off), refusals included, with the route as the operation and its path values as targets; minting a token names the new id. flexiq-server token create|revoke record too, as cli. A new principal_kind column (migration 0026_audit_principal_kind; existing rows read token) says who a record's token_id names, on the wire as AuditRecord.principal_kind and filterable with principalKind= / fq audit list --principal-kind; fq audit list shows the caller as kind:id in a by column, which replaces token. The window is now FLEXIQ_AUDIT_RETENTION_DAYS (the FLEXIQ_GRPC_ name is still read), and a dashboard with no gRPC listener prunes it itself.

• Optional read auditing (#993). FLEXIQ_GRPC_AUDIT_READS=on (Helm grpc.auditReads) records calls needing read or inspect in the audit trail too, each naming what it asked for — a job, run, dead letter or periodic task id, a listing's queue and task filters, the ids or queue a WatchJobs stream opened on (recorded once, at open). A read repeating one recorded within FLEXIQ_GRPC_AUDIT_READS_WINDOW seconds (default 60; 0 records every read; Helm grpc.auditReadsWindow) by the same token, RPC, target and outcome is folded into it. Off by default; an unrecognised value refuses to start.

• An audit trail of token-authorised writes (#840). Every call on the gRPC door or its JSON facade that needs produce or admin is recorded in a new audit_log table (migration 0022_audit_log, on all three backends), one record per thing the call acted on — a job, queue, worker, dead letter, periodic task, task or workflow run — or one untargeted record for a call refused before naming any: the token's public id and name, the namespace, the RPC, the target, the time and the google.rpc.Code it ended with (CANCELLED for a call that never answered). Refusals of a valid token are recorded; reads, the executor door and unauthenticated calls are not, and no record ever carries the token or its digest. Read it back with AdminService.ListAuditRecords (inspect), GET /v1/admin/auditRecords, or fq audit list --job … | --token-id … | --target kind:id. Records are written off the request path — a record the buffer or the table cannot take is logged as JSON under flexiq::audit and counted in flexiq_audit_records_total — and pruned after FLEXIQ_GRPC_AUDIT_RETENTION_DAYS (default 90; Helm grpc.auditRetentionDays).

• Token grants narrowed to queues and tasks (#839, #988, #989, #990). A new read scope opens the producer door's reads — GetJob, ListJobs, QueueStats, GetWorkflowRun, WatchJobs, and GET on the JSON facade; produce still implies it. produce, read, execute, inspect and admin grants take queue= / task= patterns (an exact name, a prefix ending in *, or *), e.g. produce:queue=emails-*,task=send_receipt, and a token's grants are a union. A job, dead letter, periodic task or workflow run outside them reads as not found; an enqueue, batch or workflow node outside them is refused whole with SCOPE_DENIED before anything is written; an executor advertising a task beyond them is refused at Attach, and a job on a queue it is not granted goes to a peer that is. Listings must name a reachable queue or task and are never filtered after the page is cut. A narrowed token cannot debounce, and a unique_key hit on another queue's job is refused rather than handed back. Workers, quotas and the audit trail take whole grants only. Grants live in the existing scopes array, so there is no migration; an older build drops a grant it cannot read, so a rollback only narrows a token.

• API tokens managed over the admin door (#1042). A new tokens scope — not implied by admin, never narrowable — opens CreateToken, GetToken, ListTokens and RevokeToken on flexiq.admin.v1 (JSON under /v1/admin/tokens) and fq tokens create|list|show|revoke. No call escalates: every requested grant must be covered by one of the caller's, a new token may not outlive the caller's, every call acts on the caller's namespace, and revoking needs grants covering the target's. The secret is returned once by CreateToken and never recorded; the calls are audited as target kind token.

• The submitting token is recorded on the job (#992). enqueued_by (migration 0025_job_enqueued_by, on live, archived and dead-letter rows) holds the public id of the token that authorised an Enqueue, EnqueueBatch, SubmitWorkflow or TriggerPeriodic — the id the audit trail stores — so it outlives the audit window. Only the server sets it; it is null when no token was involved, and a dead-letter replay keeps the original. On the wire as Job.enqueued_by, in Python as enqueued_by, in Node and Java as enqueuedBy, and as an "Enqueued by" row on every dashboard.

• Per-namespace quotas (#841). Max pending jobs, enqueue rate, max running jobs, and row ceilings on the archive and dead-letter queue, enforced in the core so every producer path shares them. reject answers the existing QueueFull / RateLimitExceeded; drop dead-letters the excess without it ever going live. Read and set with GetNamespaceQuota, SetNamespaceQuota and ClearNamespaceQuota on the admin door or fq quota get|set|clear. An unset quota is unlimited, one that cannot be read fails closed, and running processes see a change within about 2 seconds. quota: is now a reserved settings prefix, so a tenant cannot raise its own limit through the settings API.

• Watching jobs instead of polling (#837, #977, #978). WatchJobs, a server stream on flexiq.v1.ProducerService, follows up to 100 job ids (a snapshot each, then transitions, ending once every one is terminal) or a queue (live transitions, each with a resume cursor). Decide completion by an item's terminal field, not its status. Bounded by FLEXIQ_GRPC_WATCH_MAX_PER_TOKEN (default 16), …_STALL, …_BUFFER and …_RECONCILE (default 5 s), which the Helm chart exposes as grpc.watch.*. The JSON facade serves the same stream as Server-Sent Events at GET /v1/jobs:watch (read scope, Last-Event-ID resume, a final event: end). fq tail and the Go client's Wait / EnqueueAndWait are built on it. A queue watch sees only the transitions of the replica serving it.

• In-process TLS and mTLS on the server's credentialled listeners (#838). The gRPC door (FLEXIQ_GRPC_TLS_CERT, …_KEY, …_CLIENT_CA) and the attach listener (FLEXIQ_LISTEN_TLS_CERT, …_KEY, …_CLIENT_CA) terminate TLS themselves when configured, and verify client certificates against a CA bundle when one is set — beside the API token or the attach token, never instead of it. Certificates reload without a restart: the files are re-read when their content changes, and a broken renewal keeps the pair in force. Executors dial tls://host:port from every SDK (--tls-ca/--tls-cert/--tls-key, or FLEXIQ_ATTACH_TLS_*; tls options on the Node and Java executor APIs), fq takes the same three flags, and the Helm chart mounts attach.tls / grpc.tls Secrets. Unset, both listeners stay plaintext, so a mesh deployment configures TLS once, in the mesh. TLS material beside a plaintext or Unix address is refused on both sides rather than ignored.

• More than one push target per process. FLEXIQ_PUSH_TARGETS names several targets, each with its own URL and queues (FLEXIQ_PUSH_<NAME>_URL, FLEXIQ_PUSH_<NAME>_QUEUES) and any push setting of its own (FLEXIQ_PUSH_<NAME>_CAPACITY, …_OIDC_AUDIENCE, …), falling back to the shared FLEXIQ_PUSH_TARGET_* value. Each target runs its own scheduler over its own queues, sized from its own capacity, so a full target never holds up another; shutdown drains them concurrently, and the executor door hands a Settle, ExtendLease or report to the target that made the dispatch. A queue named by two targets, FLEXIQ_PUSH_TARGET_URL beside the list, and FLEXIQ_QUEUES or FLEXIQ_WORKERS beside named targets are refused at boot. The Helm chart still configures one target.

• Kafka and NATS event sinks (#971). Two more kinds for the event egress document. kafka produces each event as a structured-mode CloudEvent record keyed by job id, placed with Kafka's own murmur2 partitioner and acknowledged by every in-sync replica, over TLS and SASL PLAIN or SCRAM when configured; its client is pure Rust, so no build gains librdkafka. nats publishes to a subject templated from {namespace}, {queue}, {task} and {type}, waiting for JetStream acknowledgements by default — with the CloudEvents id as Nats-Msg-Id, so a stream drops retried duplicates — or plain core NATS with mode: "core". Each retries or drops by its client's own error taxonomy, names its secrets by *_env, and sits behind a cargo feature (events-kafka, events-nats) compiled into the published image and every SDK build; a build without one refuses a document naming it.

• Event egress (#848). Job lifecycle transitions — job.enqueued, job.started, job.completed, job.failed, job.retrying, job.dead, job.cancelled, job.sleeping — can leave the process as a structured-mode CloudEvent, POSTed to an HTTP endpoint or appended to a Redis stream, so a warehouse, an audit log or an alerting pipeline follows jobs without polling storage. One JSON document configures it everywhere: flexiq-server reads FLEXIQ_EVENTS_FILE (drain budget FLEXIQ_EVENTS_DRAIN, default 5 seconds) and emits job.enqueued and pre-run job.cancelled from its doors, cascading a door cancel to every dependent with reason: "dependency cancelled"; each SDK's worker takes the same document as a worker option — Queue(event_sinks=...), runWorker({ eventSinks }), Worker.Builder.eventSinks(...), WorkerBuilder::events(...) — and emits everything from job.started on. Every scheduler, embedded or in flexiq-server, also reports a pending job's expiry (reason: "expired" from the reaper, "expired before execution" from the poller) and a dead-letter cascade (reason: "dependency failed") as job.cancelled, and a periodic firing or a DLQ auto-retry as job.enqueued with attempt 0 — the only enqueues an embedded worker reports on its own. A scheduler announces only its own namespace's jobs — the default namespace's when it has none — even where its maintenance reaches other namespaces, so a namespace's maintenance events come only from a scheduler of that namespace. Delivery is at-most-once overall and may repeat an event, so consumers dedupe on the CloudEvents id; emitting never blocks the queue. The HTTP sink goes through push dispatch's egress guard (a required allowlist, no proxy, no redirects) and signs with x-flexiq-event-signature when given a secret; job payloads are sent only by a sink that sets include_payload. Per-sink counters appear on the server's /metrics as flexiq_events_* and on every worker's event-sink stats. The HTTP sink is behind the events-http cargo feature, compiled into the published image and SDK builds; the Helm chart gains events.*. The contract a consumer implements against is contracts/EVENT_EGRESS_CONTRACT.md; the stream is not a complete audit, and its What is not seen section lists the transitions that emit nothing.

• The admin door (#836). flexiq.admin.v1.AdminService puts the dashboard's operator actions on the gRPC listener, and as JSON under /v1/admin/: list, pause and resume queues; per-queue throughput over a window; list, read, replay, delete and purge dead letters; list and drain workers; create, replace, pause, resume, trigger and delete periodic tasks; set and clear task and queue overrides. Two new token scopes split it — inspect for the read-only methods, admin for the rest — and a produce token reaches neither. Every call acts on the credential's namespace only. Cron over the network, which #847 left to this service, is PutPeriodicTask.

• Draining a worker from outside (#836). DrainWorker (and the dashboards' equivalent) marks a registered worker's row; every SDK's worker reads it back on its next heartbeat and stops as it would on SIGTERM — it claims nothing new, finishes what it holds, and unregisters. Storage::heartbeat now answers the row's status, and Storage::request_worker_drain is new.

• Triggers (#847). FLEXIQ_TRIGGER_LISTEN turns flexiq-server into the thing a webhook sender or an object-store eventing platform calls: a URL that maps an inbound request to an enqueue per event it carries, with no service of your own in between. Triggers are configuration, not code — a JSON file (FLEXIQ_TRIGGERS_FILE) fixes each trigger's task, queue and rate limit, and a request supplies argument values through JSON Pointer, header, query and constant selectors, never a destination. Every trigger verifies its sender (GitHub, Stripe, Standard Webhooks, Twilio, a generic HMAC-SHA256 header, or a shared secret) against the raw body before anything else, then draws from a storage-backed rate-limit bucket that holds across replicas, so forged requests cannot spend a real sender's budget. A delivery id deduplicates redeliveries. kind: object_store unwraps S3 (EventBridge, or SNS with its RSA signature verified and the subscription confirmed), GCS (Pub/Sub push, optionally proven by its Google-signed OIDC token) and Azure (Event Grid, handshake included) events into one common shape. The listener is plain HTTP on its own port, and the Helm chart gains triggers.* with its own Service and Ingress. Cron over the network is left to the admin service (#836).

• Push dispatch (#843, #844). FLEXIQ_PUSH_TARGET_URL adds a second way for the scheduler to reach an executor: instead of waiting for one to attach, it POSTs each claimed job straight to an operator-configured HTTP endpoint — for a platform that starts a container from an inbound request rather than running one continuously. Behind the http-target cargo feature, which the published flexiq-server image compiles in alongside postgres, redis and grpc. Every dispatch goes through an egress guard: a deny-by-default allowlist, DNS vetted inside resolution so the address dialled is the address checked, loopback/link-local/cloud metadata refused whatever the allowlist says, and no redirects. Four outbound auth schemes prove the scheduler's identity to the target: a static bearer token, HMAC-SHA256 with replay defence, OIDC identity tokens (Cloud Run, Azure IMDS, Azure App Service), and AWS SigV4 (Lambda, API Gateway). The wire contract a target implements against is contracts/PUSH_DISPATCH_CONTRACT.md. Three limits worth knowing before deploying it: a job has to finish inside one HTTP request — Cloud Run caps that at 60 minutes, Lambda at 15 (#845, below, lifts this); cancel() abandons the request and fences the target's answer out rather than stopping the target's own work (#846, below, lets a target learn of it); and push dispatch does not honour HTTP_PROXY / HTTPS_PROXY / ALL_PROXY, since a proxy would resolve the target outside the egress guard.

• Push work can outlive the request deadline (#845). With FLEXIQ_PUSH_TARGET_SETTLE=grpc (which requires FLEXIQ_GRPC_LISTEN; the Helm chart supports it), a target answers 202 Accepted, ends the request, and reports afterwards through four new flexiq.executor.v1 RPCs — ExtendLease, ReportProgress, WriteTaskLog and Settle — under the dispatch's lease. Exactly one of a Settle, the settle deadline or the stale-job reaper settles the attempt. Migration 0019_claim_settle_deadline adds the marker, and a claim holding a live one now survives the hourly claim purge. Off by default: without it a 202 still dead-letters.

• DebounceOptions and TaskHandler are reachable from the crate root (#921). Both belong on flexiq-core's curated root re-export — the blessed import path new code is meant to prefer — and both were missing: DebounceOptions is a parameter of Storage::enqueue_debounced, and TaskHandler is the enum TaskRegistry stores and NativeDispatcher matches on. A consumer had to name each through its module. flexiq re-exports core's root wholesale, so they are flexiq::DebounceOptions and flexiq::TaskHandler too. Purely additive; the module paths still resolve.

• The last six records reach the crate root, completing the list (#948). The entry above said every record beside DebounceOptions was already re-exported; six were not. SubscriptionMode, Topic, TopicMessage, TopicLogStats, WorkerRegistration and WorkerStatus are now on flexiq-core's root re-export, so all 25 of storage::records' public types are reachable as flexiq_core::X — and, through flexiq's wholesale re-export, as flexiq::X. The pub/sub four were the conspicuous gap: Subscription and NewSubscription were on the list but the Topic trio and the SubscriptionMode that travels with them were not, so converting one subscription meant naming both paths in one file. Purely additive; the module paths still resolve. The list has now been declared complete twice while it was not, so a test diffs the root re-export group against records.rs and fails on the next record that misses it.

• Trace context carried from enqueue to execute (#854, #1047). W3C traceparent / tracestate ride as top-level keys of a job's metadata, merged by one rule in every SDK (flexiq_core::trace), and the gRPC door and JSON facade merge the request headers the same way. The Python and Node OpenTelemetry middlewares inject at enqueue and extract at execution, Java gains contrib.FlexiQOtel, and Python and Node job contexts now expose the job's metadata. Workflow node jobs — fan-out, fan-in, deferred and compensation included — carry the submitter's context too, stored on the run (workflow migration 0005_workflow_run_trace_context) and inherited by sub-workflows; each SDK takes an explicit carrier on submit, and a new on_workflow_submit / onWorkflowSubmit middleware hook lets the OpenTelemetry integration fill it.

• A Rust SDK (#830, #920). flexiq grows from a re-export into an SDK, additively: #[flexiq::task] from the new published flexiq-macros crate, a queue handle (FlexiQ::open, enqueue, batch, cancel, reads, stats), a worker, durable steps, periodic tasks, and the option surface the other SDKs expose — retries, priority, unique keys, debounce, on_excess, concurrency and rate limits, retry budgets, circuit breakers. The engine re-export stays as the escape hatch. The docs site gains a Rust tier.

• fq, a standalone command line (#832). One binary over the gRPC door, linking no SDK and no storage layer: fq enqueue, fq jobs list|get|cancel and fq queues, plus fq tail, fq quota, fq tokens and fq audit as the doors grew. --endpoint picks the transport by scheme (http://, https://, unix:); --json prints machine-readable output. Arguments are read as JSON, falling back to a string, and sent as the wire's structured arm, so keyword arguments encode in sorted key order and integers cap at ±(2⁵³−1) — pass --unique-key where deduplication must match an SDK's auto: key.

• A Go client (#829, #907, #908, #929, #1043). github.com/ByteVeda/flexiq/sdks/go/v2 links nothing from the Rust crates. The root package covers every producer RPC — Enqueue, EnqueueBatch, GetJob, ListJobs / AllJobs, CancelJob, QueueStats, SubmitWorkflow, GetWorkflowRun — with refusals matched by reason through errors.Is; a graph using a construct only a live SDK process can advance is refused by the server with WORKFLOW_CONSTRUCT_UNSUPPORTED. executor attaches to the executor door and runs handlers, with lease, side_channel and durable steps (executor.Step, job.Sleep). admin covers queues, overrides, periodic tasks, the namespace quota and tokens; CreateToken is never retried automatically.

• Ruby gems (#834, #1024, #1025, #1026, #1028, #1027). flexiq is a producer client over the JSON facade with no runtime dependency and no native extension: enqueue and batch (through a pure-Ruby CBOR codec that passes every wire vector), read, cancel, queue counts, list_jobs / each_job with cursor paging, watch_jobs / watch_queue / wait over the SSE stream, and submit_workflow / get_workflow_run for static graphs. TLS is on by default; plain http:// needs insecure: true, and the client never retries. flexiq-executor, a separate gem, attaches to the executor door over gRPC with side_channel and lease (no durable steps yet). A ruby-v* tag publishes both to RubyGems through trusted publishing.

• An OpenAPI document for the JSON facade (#831). contracts/openapi.json (OpenAPI 3.1) is generated from google.api.http annotations on the producer RPCs and gated, beside the descriptor, against the facade's route table, so the two cannot drift.

• A KEDA external scaler (#850). flexiq-server serves KEDA's externalscaler.ExternalScaler on the gRPC listener, so a ScaledObject can use type: external or external-push in place of the metrics-api shim, which stays. The token rides in scalerMetadata (token, or tokenFromEnv) and needs inspect. IsActive, GetMetricSpec, GetMetrics and StreamIsActive count live jobs through a new Storage::live_counts, never a full stats scan. Example manifests are in deploy/keda/.

• A Terraform provider (#851). deploy/terraform/provider, built on the admin door alone: flexiq_queue (override and pause), flexiq_namespace (quota), flexiq_periodic_task and flexiq_token (secret kept in state as sensitive, rotation through rotate_before_days, revoked on destroy). Not yet on the Terraform Registry.

• The Helm chart is published as an OCI artifact (#872). flexiq-server is pushed to oci://ghcr.io/byteveda/charts, version-aligned with the server image, once that image passes verification.

• The sidecar webhook mounts attach TLS Secrets (#984). flexiq.dev/tls-ca-secret (key from flexiq.dev/tls-ca-key, default ca.crt) and flexiq.dev/tls-client-secret mount Secrets read-only into the injected sidecar and set FLEXIQ_ATTACH_TLS_*; the pod is refused unless flexiq.dev/attach is a tls:// address. Mounts are whole directories, so a rotated Secret is read on the next attach.

• A reproducible benchmark harness (#824). bench/ measures enqueue throughput, end-to-end latency and idle cost for the Python and Node SDKs beside Celery, Dramatiq, RQ and BullMQ, every entrant on its own defaults; the docs publish the results, losses included.

• Documentation. Migration guides from BullMQ (#863), Cloud Tasks and EventBridge (#974), and Sidekiq (#865); a comparison with Temporal, Cloud Tasks and BullMQ (#866); a Kubernetes Helm guide (#853); the published server image's contract — tags, architectures, digest pinning (#852); an Operate section and a server-and-wire tier for flexiq-server; and the contract a gRPC-only client owes.

Changed

• Queue pauses are per namespace (#836). A pause was one row per queue name for the whole database, and every scheduler read it unscoped, so pausing emails in one tenant stopped emails in every tenant. Migration 0020_queue_state_namespace makes a pause (namespace, queue); every SDK's pause/resume/paused_queues, the dashboards and the scheduler use the handle's namespace. On upgrade, an existing pause belongs to the default namespace — a namespaced deployment that had a queue paused should pause it again.
• Workers record their namespace (#836). Migration 0021_worker_namespace adds it to the registry, and a worker listing — the dashboards', the SDKs' workers(), /metrics and the readiness probe's worker count — shows one namespace's workers. A worker that registered before the upgrade reads as the default namespace's until it restarts.
• Task and queue overrides are per namespace (#836). A namespaced handle reads and writes overrides:ns:<len>:<ns>:task:<name> (and …:queue:…) instead of sharing overrides:task:<name> with every tenant; the default namespace keeps the old keys, so a single-tenant deployment changes nothing. An override a namespaced dashboard stored before the upgrade is no longer read — set it again. The layout is in BINDING_CONTRACT.md.
• Dead-letter purges honour the namespace (#836). purge_dead and purge_dead_by_task take one, so an SDK or dashboard purge from a namespaced handle no longer deletes other tenants' entries.
• Every executor attach is offered lease (#947). The lease book belonged to the scheduler, which flexiq-server starts on the first attach, so that attach — and any other before the scheduler role started — was acknowledged without lease and fenced on (owner, attempt) alone until its stream rotated. RemoteDispatcher now mints the book itself and always acknowledges lease, and the scheduler adopts it through the new WorkerDispatcher::own_lease_book; a pool that owns no book is handed the scheduler's through set_lease_book, as before — and a RemoteDispatcher handed a book that way still uses it. An executor that does not advertise lease is dispatched none, as before.
• Redis claims in one round trip and wakes on enqueue (#959, #960, #961). One Lua script selects and claims up to N ready jobs, and Redis claims 8 per cycle by default (SchedulerConfig::batch_size is now Option<usize>; SQLite and Postgres keep 1). Push dispatch is compiled into every shipped artifact and on by default for Redis only, woken by PUBLISH/SUBSCRIBE per namespace and queue inside each enqueue's existing round trip; opt out with push_dispatch=False, pushDispatch: false, pushDispatch(false), Worker::push_dispatch(false) or FLEXIQ_PUSH_DISPATCH=false. Storage calls draw from a connection pool instead of dialling per call. Against a Redis about 33 ms away, enqueue rose from 8.4 to 30.6 jobs/s and p50 latency fell from 195 s to 119 s. Work no wake announces — a delayed job coming due, a resumed queue, a freed slot — can take up to 2 seconds to be picked up, and stopping a Redis worker can take about a second longer. Postgres no longer sends a pg_notify per enqueue.
• Execute spans are CONSUMER spans (#854). With a traceparent on the job, the Python and Node execute span has a remote parent, so a parent-based sampler follows the enqueuer's decision. Java's EnqueueContext.metadata() is layered over a JSON-object EnqueueOptions.metadata instead of replacing it. The Java SPI's QueueBackend.submitWorkflow gains an overload taking trace context (#1047); a backend overriding the old one still works, without trace context, and that overload is deprecated.
• GET /v1/stats?queue=x honours the filter (#831). It answered namespace-wide counts and ignored the query; it now means GET /v1/queues/x/stats.

Fixed

• flexiq-server's scheduler applies task and queue overrides. It registered none, so under attach and push alike every job retried on the core's default backoff, and an override's rate limit or concurrency cap — set on the dashboard, over the admin door or with the CLI — was stored and never enforced; for a push deployment that was every task it ran. The scheduler now reads them when it starts, as an SDK worker does: a task's retry_backoff, rate_limit and max_concurrent, and a queue's rate_limit and max_concurrent. A field that cannot be read is logged and skipped rather than failing the start.
• A Python worker start no longer resumes a paused schedule. Every start re-registered each @queue.periodic with enabled: true, so a restart silently undid an operator's pause. It now declares the schedule, which keeps the pause and the last run (#919's conditional write).
• The Java worker applies task and queue overrides. Overrides set from a dashboard reached every worker but a Java one. It now reads them at start — rate limit, concurrency cap and retry backoff for a task; rate limit and concurrency cap for a queue — like the other SDKs.
• cancel() reaches attached executors and push targets behind flexiq-server (#846). A cancel only ever set the storage flag, and only the native pool reads it — so under flexiq-server neither an attached executor nor a push target heard of one, and the job ran to the end while the API reported success. Worker now relays the flag to any dispatcher it was handed, once a second, through one batch read (Storage::cancel_requested_among). An attached executor gets its cancel frame; a push dispatch settles Cancelled. A push target that answered 202 learns of it from its next ExtendLease, ReportProgress, WriteTaskLog or Settle, which is refused with the new ErrorInfo reason JOB_CANCELLED (a lost race now carries CLAIM_LOST, rather than no reason). What cancel() promises under each topology is written down in the push contract and the server docs: under push it stops the result from landing and tells a target that asks, but does not stop the work. The Rust SDK's pool had no cancel at all; a task now calls flexiq::check_cancelled()? (or reads flexiq::cancel_requested()), which returns the new Abort::Cancelled once the relay has delivered a cancel, and the job settles Cancelled without a retry.
• A float's width on the wire is pinned, so auto: keys no longer diverge over one (#905). contracts/wire-vectors.json left float width free — a writer could emit half or single precision for 1.5 — and pinned the float case's decoded value instead of its bytes. Both widths decode to the same number, so a narrowing writer interoperated fine and silently stopped idempotent enqueues deduping: the auto: key is a sha256 over the serialized payload, so two conforming runtimes derived different keys for the same float argument and each enqueued its own active job. The same class of bug as the indefinite-length maps of #638. A finite float now carries the 64-bit head fb, never f9 or fa; readers still accept the narrower widths, and the float case moved into encode where its bytes are asserted by every SDK suite, beside two new float-narrow-* cases pinning the half- and single-precision payloads a reader must still accept and a writer must not produce. Two encoders needed fixing to comply, both over a 32-bit float. A Java float argument reached the wire as a 4-byte CBOR float where the same value passed as a double took 8, at any depth in a payload; a Go float32 did the same, because ShortestFloatNone governs a float64 only and there is no encoder option for the narrower kind — so the Go client widens the encoded bytes on the way out. A non-finite float is a stated exemption rather than part of the rule — CBOR libraries hard-code RFC 8949's two-byte spelling of an infinity and a NaN, some without a way to write one wide and others without a way to write one narrow — so a payload carrying one still wants unique_key. Neither value can reach the structured gRPC door, which refuses both.
• A failed Rust task records an empty traceback, not a null one (#930). flexiq's failure formatter wrote "traceback": null, a third shape the binding contract does not describe — it specifies an array of strings, [] when the language or runtime has no frames to offer. Rust was the only shell writing null, and a reader that had to special-case it is a reader that would eventually not.
• An executor that honoured the lease acknowledgement lost every frame (#932). The scheduler decided whether to check an attached executor's frames for a lease from its hello, but whether to advertise lease back in hello_ack from whether it held a lease book at that instant — and the book is installed when the scheduler role starts, which can be after an executor has attached. An executor acknowledged in that window sends no lease, exactly as the contract tells it to, and had every frame about every job read as a superseded attempt and dropped: successes, failures, progress, logs. The job was never settled and waited for the reaper. Both halves now come from one decision — a capability is in force only where the executor advertised it and that attach's acknowledgement carried it — so a dispatch never requires a lease the acknowledgement did not promise. An executor attaching before the role starts is fenced on (owner, attempt) alone for that connection, which is the documented give-up for a peer without lease, and renegotiates on its next attach.
• The refusal these frames logged asserted a re-dispatch it had not checked for, sending whoever read it looking for a second execution that never happened. A frame carrying no lease and one carrying a superseded lease are different faults and now read differently.
• A step commit from an executor that never negotiated steps was applied (#932). The same asymmetry, one capability over: the scheduler checked that it had a step store but not that the executor had claimed the capability. Such a peer is sent no job_steps snapshot, so it runs every step un-memoized — and storing its commit left a memo it would never be handed back, so the next attempt re-ran the side effect with a record on file saying it was already done. steps refuses rather than degrades, so the commit is now refused permanently.
• A rate limit that can never release a job is refused (#917). "0/s", "-5/s", "0.5/s" and "NaN/s" were accepted, and the task then never dispatched with nothing saying why. A count that is not finite or is below 1 is now rejected, and every SDK turns that into an error at worker start. A Python queue-level rate that fails to parse is an error too, rather than leaving the queue unthrottled.
• Periodic tasks are scoped by namespace (#918). periodic_tasks was keyed by name alone, so on a shared database two tenants registering one task name overwrote each other's row, a listing returned every tenant's schedules, and a delete or a pause reached rows the caller did not own. Identity is now (namespace, name) (migration 0018_periodic_task_namespace). A scheduler with no namespace still fires every namespace's due schedules, each job inheriting its row's namespace.
• A redeploy no longer undoes a pause or repeats a firing (#919). Declaring a periodic task from code read the row and then replaced it, so a pause or a scheduler's next_run committed in between was written back stale. Storage::declare_periodic is a conditional write: it never touches enabled or last_run, and rewrites next_run only when the cron expression or timezone changed.
• A periodic task's kwargs is refused rather than silently dropped (#922). Every backend stored it and nothing read it — a fired job's payload comes from args alone. Registering or declaring one with kwargs set now fails with a configuration error; rows written earlier still load. The Python binding's register_periodic loses its unused kwargs= parameter.
• A swallowed step.sleep is reported as a sleep (#891). A task body that caught the sleep signal could report a late failure under the woken attempt's claim and dead-letter a job that was sleeping correctly. In Python, Node and Java an attempt that committed a sleep now reports the sleep, whatever the body did next; a swallowed divergence or refusal still fails the attempt.
• An empty workflow node queue means default (#990). A graph submitted over the gRPC door with no queue on a node put that node's job in a queue literally named "".
• The JSON facade's error body no longer reads as success (#831). An error carrying no ErrorInfo — only the workflow RPCs produced one — rendered {"code": 200, "status": "OK"} under a non-2xx HTTP status.
• The Go client reads a task error with a null sibling as structured (#931). A failure with "traceback": null, as a Rust worker wrote one, came back unstructured with its errtype lost. Only a string message is required now; a missing or null errtype or traceback takes its default, as every other reader does.
• Push dispatch name lookups no longer hold shutdown open (#945). A wedged DNS lookup ran on the runtime's blocking pool, which shutdown waits for, and could get a pod killed with leases still open. Lookups run on detached threads, at most 32 at once per resolver.
• OIDC dashboard login pins id_token verification to the issuer's key (#885). The algorithm came from the token being verified; it now derives from the published JWK and is never HMAC, and the four claims OIDC Core makes mandatory are required. jsonwebtoken moves to 10.3, and request-supplied values in the OAuth and scaler logs are escaped against log injection.
• The published flexiq-core crate's tests compile (#901). The wire-vector test read a file outside the package, so cargo test on the crate from crates.io failed to build; that one test target is now excluded from the package.
• The Python worker's startup banner spells flexiq (#1014), not taskito.

Upgrading to 2.1.0

Eleven migrations run on first start, across the two ledgers — ten core (m0018 through m0027, tracked in schema_migrations) and one workflow (m0005, tracked in workflow_schema_migrations). The contract level stays 2.

All but m0018 are expand-only. m0018 rebuilds periodic_tasks inside the migration's own transaction — create, copy, drop, rename — so there is no partial state, but its name primary key is gone: on Postgres a 2.0.0 process's periodic registration upserts on that key, so upgrade every process that declares schedules together. A process with a namespace then registers its schedules as new (namespace, name) rows beside the migrated default-namespace copies, and a scheduler with no namespace fires both; once those processes have re-registered, delete the default-namespace copies they replace. m0023 and m0024 build indexes over jobs, archived_jobs and dead_letter, so a large database pauses on first start while they build. Redis runs no migrations; it backfills its per-namespace quota index sets, and during a rolling upgrade jobs enqueued by a not-yet-upgraded process are not counted against a quota until they finish.

Behaviour to check before rolling: queue pauses, task and queue overrides, dead-letter purges and the worker registry are now per namespace (see Changed) — a namespaced deployment re-applies a pause or an override set before the upgrade. A periodic task registered with kwargs, and a rate limit whose count is below 1, are now refused at start rather than ignored, and Redis workers wake on enqueue by default.

2.0.0

Three things a queue could not do before: checkpoint work inside a job, be reached by a process that has no FlexiQ binding at all, and shed or debounce work it should not queue. Plus the removal of two public items in flexiq-workflows that were documented, exported, and dead.

This is the first release since 1.0.0. There is no 1.1.0 on any registry — one was prepared and merged, but the version was never tagged and everything in it ships here instead. Read Upgrading before rolling: eight migrations run on first start, one changes how unique_key deduplicates, and one drops a column, which is what makes this a major rather than a minor.

Breaking

Most of this reaches only a Rust program depending on flexiq-core or flexiq-workflows directly. Two items do not, and they are why the version number moves: the Python and Java NodeStatus enums lose a member, and the wire loses an enum value.

SDKs and the wire:

• NodeStatus.READY is gone from the Python and Java enums, and "ready" from the dashboard's WorkflowNodeStatus union. See Removed for why it never meant anything.
• WORKFLOW_NODE_STATUS_READY is gone from flexiq.v1.WorkflowNodeStatus. Number 2 is reserved rather than reused, so nothing a client already holds changes meaning — but a proto3 JSON client that sends that name by hand now gets a parse error. No server ever emitted it.
• workflow_nodes.fan_in_data is dropped by a migration. A 1.x process cannot read the table afterwards; see Upgrading.
• CONTRACT_VERSION and MIN_CONTRACT_VERSION are both 2, so set_min_contract(1) is now refused.

Storage trait:

• enqueue_unique_reporting and enqueue_unique_batch_reporting are new required methods. They return whether the enqueue deduplicated, which the producer door has to answer on the wire and which is not computable above the backend — the job's id is generated inside the insert, so a caller has no candidate to compare against. The existing names stay as one-line wrappers, so a backend still has one implementation.
• reschedule gained a third parameter, namespace: Option<&str>. A step sleep reschedules with an id that reached the queue through task code, so the call had to become scoped.

New public fields, which break an exhaustive struct literal:

• Dispatch.lease, DeadJob.origin_job_id, DebounceOptions.max_pending, ExecutorConfig.capabilities, ExecutorConfig.step_ack_timeout, RemoteConfig.step_workers, RemoteConfig.step_capacity.
• lease on ExecutorMessage::{Progress, TaskLog, Success, Failure, Cancelled} and on SchedulerMessage::Job.

Newly #[non_exhaustive]. The consequence differs by what was marked:

• Enums ExecutorMessage, SchedulerMessage, JobResult, ResultOutcome — a match on one now needs a _ arm.
• Structs AttachedExecutor, WorkerInfo, WorkerRegistration — no longer constructible with a struct literal outside the crate, and destructuring one needs a trailing ...
• Variant ExecutorMessage::Hello — same two rules as a struct, applied to that variant alone; the other variants still construct and destructure as they did.

New variants on enums that were exhaustive in 1.0.0:

• QueueError::{QueueFull, ClaimLost, StepDiverged, StepSequenceDiverged, StepLimitExceeded, StepRefused}.
• WorkflowError::InvalidStepMetadata, split out so a malformed step payload from a caller no longer reports as a storage fault.

Changed discriminants, which reach only a numeric cast:

• SchedulerMessage::Cancel 2 → 4 and SchedulerMessage::Shutdown 3 → 5, from two variants landing mid-enum.

Removed

• WorkflowNode::fan_in_data, and the workflow_nodes.fan_in_data column behind it. Nothing ever wrote either: collected fan-in results reach the fan-in job as its arguments, so the payload travels with the job rather than through a node row. Unrelated to StepMetadata::fan_in, the definition-time {from} marker, which is unaffected.
• WorkflowNodeStatus::Ready, and the READY member mirroring it in the Python and Java NodeStatus enums, the dashboard's WorkflowNodeStatus union, and the documented node-status tables. No build persisted it — a runnable node stays pending, because readiness is a predicate over the DAG evaluated by get_ready_workflow_nodes, not a stored status. Code that grouped ready with pending (the only correct reading) needs no change; code that matched ready alone was already dead.

Added

• flexiq_core::storage::migrate::drop_column, the mirror of add_column: DROP COLUMN IF EXISTS on Postgres, and a plain DROP COLUMN on SQLite whose missing-column error is tolerated.
• Durable steps. ctx.step.run / ctx.step.sleep in Python and Node, ctx.step().run / ctx.step().sleep in Java, all backed by a job_steps table. A retried job replays the steps that already committed instead of re-running them, which is what makes a task that charges a card safe to retry. Every write is fenced on (owner, attempt, epoch): a step commit from a worker that has lost its claim is refused rather than applied, because the alternative to failing closed here is a second charge. step.sleep ends the attempt and reschedules rather than blocking a worker slot. Steps work on an attached executor too, over the frame protocol and under the in-memory harness.
• Server mode. FLEXIQ_GRPC_LISTEN adds a fourth role to flexiq-server, serving two independent doors: flexiq.v1.ProducerService — enqueue, batch, read, cancel, count, and submit or read a workflow — and flexiq.executor.v1.ExecutorService, an attach stream that reaches the same dispatcher a socket-attached executor does. The producer door is also served as ordinary HTTP with proto3 JSON bodies on the same port, so a shell script can enqueue with no protobuf toolchain and no CBOR library. The wire contract is contracts/proto, gated by buf and stable from this release; server-v* releases carry it as an asset.
• Scoped API tokens (flexiq-server token create|list|revoke, and a dashboard page). A credential is minted per client, carries produce or execute and never both, expires, and is revoked without restarting a pod or re-issuing to anyone else. A revoked token fails on its next call: the check is not cached, because a cached allow is a revocation that never landed.
• Task discovery. @task without a queue plus Queue.autodiscover in Python, async discovery in Node, and a compile-time registry via the annotation processor in Java. The scheduler now records each worker's registry and warns when one advertises a set no live peer has — the safety net for a worker that imported half its task tree and would otherwise dead-letter the rest in silence.
• Debounce and shedding on every SDK. debounce options on the task decorator, builder and registration API; on_excess="drop" for a queue that should shed rather than queue.
• A dispatch lease on every job. The scheduler mints an opaque token when it wins an execution claim and requires it back on every frame that settles or advances that attempt, so a frame from a superseded attempt cannot record a result.
• Bounded middleware hooks. Every before now pairs with exactly one after or on_sleep, under a deadline, so a hook that hangs cannot hold a slot indefinitely.
• Documentation is restructured around Guides, Modules, Operate and About, with a beginner track that ends with something running, an API reference generated from the SDK declarations and gated for coverage, and full-text search.

Changed

• The contract level is 2, for both CONTRACT_VERSION and MIN_CONTRACT_VERSION. Level 1 was every schema change up to now, all of them expand-only; dropping a column is the first that is not, so level 1 is no longer a level this build interoperates with rather than merely an older one. Both constants move together — the floor a fresh deployment reports has to be a level the build actually speaks.
• The .proto files are stable. They carried an UNSTABLE banner until the release that first shipped the gRPC role. Field numbers, field names and RPC shapes are permanent from here.
• The gRPC listener has keepalive, a request deadline and a per-connection concurrency cap (FLEXIQ_GRPC_KEEPALIVE_INTERVAL, FLEXIQ_GRPC_REQUEST_TIMEOUT, FLEXIQ_GRPC_MAX_CONCURRENT_REQUESTS). The deadline bounds a response head, so it closes no long-lived stream; use FLEXIQ_GRPC_EXECUTOR_STREAM_MAX_AGE for an attach stream.
• GET /metrics is served on the gRPC listener, so a release that enables only the gRPC role has a scrape target. It carries the dashboard's storage gauges plus per-RPC call counts and latencies.

Fixed

• unique_key deduplicated across namespaces. A tenant enqueueing a key another tenant had already used got that tenant's job back and never created its own. The index is rebuilt over (namespace IS NULL, COALESCE(namespace, ''), unique_key) by migration m0017, and the lookups are scoped to match. The leading discriminator is not redundant: COALESCE alone would fold an unnamespaced job and one in a namespace literally named "" into the same group. Debounce was already namespaced; unique_key was the outlier.
• A queue's admission cap was not applied inside a debounced enqueue, so a debounce window could admit work past a full queue.
• An empty debounce placeholder could key a window, collapsing unrelated jobs into one.
• Shed dead-letter entries were picked up by auto-retry, which retried work the queue had deliberately dropped.
• One executor connection could be handed the same job id twice.
• A degraded readiness probe answered 200. It answers 503 now, so an orchestrator takes the replica out of rotation.
• Dead-letter rows lost a run's origin and a job's metadata, which are the two fields a replay needs.
• A Python step session could be fenced against another worker's claim, and a Node executor could drop the wrong dispatch from its step registry.
• Deferred Python tasks were not claimed on every path that loads an app.
• The dashboard compared worker registries across queue groups rather than within one, so unrelated workers were reported as divergent.

Upgrading to 2.0.0

Eight migrations run on first start, across the two ledgers. Seven are core schema (m0011 through m0017, tracked in schema_migrations); the eighth is workflow schema (m0004, tracked in workflow_schema_migrations):

The first seven are expand-only — they add columns, tables and indexes and drop no data. The eighth is not, and it is the one that decides your rollout order.

m0004 is one-way and not backwards compatible. A 1.x process names fan_in_data in every workflow-node SELECT, so once a database has been migrated a 1.x worker fails on every read of workflow_nodes, not only on rows the new build wrote. Do not run 1.x and 2.0.0 workers against the same database. Upgrade every process, then write the floor, so a stray old one refuses to start rather than failing mid-read — it reads the key, sees a level it does not speak, and exits, whereas leaving the key unset means it starts and fails on its first node read:

queue.set_min_contract(2)

What the drop itself costs depends on the backend. Postgres only marks the column dropped in the catalog, so it returns at once, though it takes an ACCESS EXCLUSIVE lock on workflow_nodes while it does. SQLite has no such shortcut: its DROP COLUMN rewrites the table's content row by row, so on a large workflow_nodes the first start pauses for as long as that takes. Neither backend has an index on the column to rebuild. Redis is schemaless and runs nothing — a stale fan_in_data hash field on an existing node key is simply ignored.

m0017 changes observable behaviour. It drops and recreates idx_jobs_unique_key over (namespace IS NULL, COALESCE(namespace, ''), unique_key), so a unique_key now deduplicates within one namespace instead of across all of them. That was a cross-tenant bug — a tenant enqueueing a key another tenant had used got that tenant's job back and never created its own — but a deployment that relied on the old reach has to key on something namespace-independent instead. A deployment with a single namespace, or none, sees no change. Recreating the index takes a table scan; on a large jobs table the first start pauses for it.

Payload markers and the worker frame protocol version are unchanged from 1.0.0. The contract floor is not: it moves to 2, which is the mechanism above.

If you consume the crates directly: WorkflowNode has one fewer field, so struct literals over it need one fewer initializer, and a match over WorkflowNodeStatus needs one fewer arm. WorkflowNodeStatus::from_str_val("ready") now returns None, which the storage layer already resolves to Pending. Pin the exact version rather than a 1.0 range.

1.0.0

The project is now FlexiQ. This is a rename, not a redesign — no API was added, removed, or changed in behaviour beyond its name — but it is a hard break with no compatibility shims.

Published under new names on every registry: flexiq (PyPI), @byteveda/flexiq (npm), org.byteveda:flexiq (Maven), and flexiq/flexiq-core (crates.io). The former packages receive no further releases.

• Imports move to flexiq / @byteveda/flexiq / org.byteveda.flexiq / flexiq_core, and Java types follow: Taskito → FlexiQ, TaskitoException → FlexiQException.
• Every TASKITO_* environment variable is now FLEXIQ_*, with no fallback.
• The default database moves to ~/.flexiq/flexiq.db; the schema is unchanged.
• Webhook signatures ride in X-Flexiq-Signature, the dashboard token in X-Flexiq-Token, and the session cookie is flexiq_session. The spelling is X-Flexiq-, not X-FlexiQ-, so the name survives the title-casing HTTP clients apply to outgoing headers.
• Django commands become flexiq_worker, flexiq_dashboard, flexiq_info.
• The Helm chart is flexiq-server.

Drain your queue before upgrading. Payload markers changed from __taskito_*__ to __flexiq_*__, so 0.23.x and 1.0.0 workers must not share a database. Full instructions: Migrating to FlexiQ.

0.23.0

Version-skew release. A polyglot deployment can now say which SDK and release each worker runs, and the wire format the SDKs exchange is pinned by golden vectors instead of by convention.

Changed

• BREAKING (Rust core): Storage::register_worker takes a WorkerRegistration struct, and WorkerInfo carries two more fields. Registration was already nine positional arguments under a too_many_arguments allow, so the SDK identity below would have made it eleven. Only direct users of the taskito-core crate are affected — the Python, Node and Java APIs are unchanged.

Added

• Workers record the SDK and release that registered them. sdk and sdk_version land on the workers table (migration 0009_worker_sdk), reach list_workers and /api/workers in every SDK, and render as an SDK column in the dashboard. In a polyglot deployment the registry is the only place an operator can tell a stale worker from a current one without going host by host. A worker registered by an older shell reports null rather than a wrong version.
• Golden wire vectors. contracts/wire-vectors.json pins the payload envelope byte for byte — nine encode vectors and three decode-only ones — and every SDK suite asserts against the same file, so a serializer that drifts fails in its own tests rather than in a mixed deployment.
• A polyglot example. examples/polyglot/ runs a Python producer with Node and Java workers against one jobs table, documenting the two footguns that setup has: every SDK's default serializer is same-language-only, and all three processes must opt into the same codec.
• The taskito facade re-exports the workflow and mesh crates, so a Rust consumer depends on one crate instead of three.

Fixed

• An unparseable per-task rate_limit now raises instead of silently disabling throttling. A typo like "100/mn" parsed to None and dropped the limit with no error, which is precisely the failure the limit exists to prevent; retry_budget already rejected the same mistake.
• CBOR payloads are encoded with definite-length containers in every SDK. One SDK emitted indefinite-length maps, so the same arguments hashed differently across languages and automatic idempotency keys did not match between a producer and a worker written in different languages.

0.22.0

Attach and executor release. taskito-server becomes a standalone binary that schedules for an app written against any SDK, workers attach to it over a shared worker protocol, and an executor side-channel carries progress, logs and middleware toggles back. Namespace scoping is completed across the id-addressed storage APIs, and cross-SDK parity lands the operator admin surface, predicates, retry policy and the full event taxonomy in Node and Java.

Changed

• BREAKING (Python): resource scopes renamed to match the other SDKs. ResourceScope.TASK meant checked out of a bounded pool and REQUEST meant fresh per task — the reverse of what Java and Node call those names, so the same scope="task" code behaved differently per SDK. Python now uses POOLED (was TASK) and TASK (was REQUEST); REQUEST is gone, and scope="request" raises. scope="task" still resolves but now builds per task instead of pooling, so pooled resources must move to scope="pooled" — passing pool_size/pool_min with any other scope now raises, which catches that case.
• Closed-set parameters across the core and all three SDKs take enums instead of bare strings: built-in proxy ids, interception mode, predicate on_false, log-consumer on_error, gate on_timeout, workflow diagram format, fan-out strategy, dispatch order, task-log level, workflow run state, dashboard role, subscription mode, and worker status. Wire and stored values are unchanged, and existing string callers keep working (Java keeps its String overloads, deprecated).
• More closed-set surfaces typed, finishing the sweep above: workflow step condition (WorkflowCondition, Python + Java), storage backend (StorageBackend, Python + Java), interception register_type(strategy=…) (Strategy, Python), and the webhook delivery-log status filter (DeliveryStatus, Python). Same non-breaking contract — a string still works. Internally, Python's workflow tracker and saga orchestrator now compare node/run status against the enums rather than string literals.
• The Node native binding is built on napi 3. An internal upgrade from napi 2 with no change to the published API; the minimum supported Node version stays 20.

Added

• taskito-server: a standalone scheduler, attach listener and dashboard. One Rust binary runs the scheduler loop and serves the dashboard for an app written against any SDK, so the fat application image no longer has to be redeployed as a worker. Workers attach to it over a shared worker protocol — a JSON header line plus raw payload bytes, one format for both the prefork pipe and the attach socket — dispatched through a RemoteDispatcher in the core. Database credentials stay in the server rather than the app image.

• A taskito executor subcommand in every SDK. Python, Node and Java each ship an executor that attaches to a running server, advertises the tasks it can run, and executes only those — the process side of the attach topology above.

• An executor side-channel for progress, logs and middleware toggles. An attached executor reports update_progress, task logs, and middleware enable/disable back over the same connection, so an attached worker is observable and controllable exactly like an in-process one. Implemented in the core and wired through all three SDKs.

• A Helm chart for taskito-server, with annotation-driven sidecar injection. deploy/helm/taskito-server installs the scheduler, dashboard and injector as one release, and refuses to render the combinations the server rejects at boot — an attach listener with no token, an unauthenticated dashboard — naming the value to fix instead of leaving a CrashLoopBackOff. With webhook.enabled=true, annotating a pod with taskito.dev/inject adds an executor sidecar built from the pod's own image, so nothing new is pulled and the pattern works for any language. The webhook serves a chart-generated certificate by default, or cert-manager's.

• taskito-server ships as a container image. docker/scheduler.Dockerfile builds a distroless image around a static binary — no libc and no interpreter, with Postgres and Redis compiled in — published as a linux/amd64 + linux/arm64 manifest at ghcr.io/byteveda/taskito-server. One image schedules for apps written against any SDK, so the fat application image no longer has to be duplicated for a worker deployment.

• Java operator admin surface. taskito.requeueJob(id) forces a stuck running job back to pending and releases its execution claim, preserving the retry budget — the only recovery path for a job that neither the timeout reaper nor orphan recovery will touch (use it only once the owning worker is confirmed dead or hung, or the job can run twice). Alongside it: taskito.reloadResources(names) hot-reloads resources built with withReloadable(true), rebuilding dependencies before the resources that used them; taskito.predicateStats() reports what the enqueue gates decided; taskito.analyzeArguments(taskName, payload) dry-runs the interceptor chain without enqueuing; and taskito.shutdown() closes every worker started from the client, so a shutdown hook no longer has to hold each Worker.

• Node operator admin surface. queue.requeueJob(id) forces a stuck running job back to pending and releases its execution claim, preserving the retry budget — the only recovery path for a job that neither the timeout reaper nor orphan recovery will touch (use it only once the owning worker is confirmed dead or hung, or the job can run twice). Alongside it: queue.reloadResources(names?) hot-reloads resources marked reloadable, rebuilding dependencies before the resources that used them; queue.analyzeArguments(taskName, args) dry-runs the interceptor chain without enqueuing or moving interceptionStats(); and queue.shutdown() stops every worker started from the queue, so a SIGTERM handler no longer has to hold each Worker.

• Node predicate recipes, decisions, and registry. A gate could only allow or reject; it can now return a Decision that also skips (no job — queue.tryEnqueue() reports null, enqueue throws the new EnqueueSkippedError) or defers (job created with the gate's delay). Nine ready-made gates ship as Recipes: businessHours, timeWindow, dayOfWeek, isWeekend, after, before, payloadMatches, featureFlag, envVarTruthy — time zones resolved through Intl, so no new dependency, and DST-correct. Gates can be registered by name (registerPredicate) and referenced as gate(task, "name") for config-driven gating, queue.predicateStats() reports what they decided, and predicate.skipped / predicate.deferred join predicate.rejected on the event bus.

• Node producer-side batch accumulator. queue.batcher(task, { maxSize, maxWaitMs }) buffers one-at-a-time enqueues and flushes them as a single enqueueMany call on a size or time trigger — the counterpart of Python's BatchAccumulator and Java's Batcher<T>. Entries keep their own args and EnqueueOptions, a failed flush keeps them buffered (and a timed one retries, reporting to onError), and Symbol.dispose means using flushes the tail at block exit.

• Node bare-metal autoscaler. serveAutoscaler(queue, { app }) and taskito autoscale <app> spawn and drain worker processes to track queue depth on hosts without Kubernetes, closing the last scaling gap against Python's AutoscaleController. Same HPA-shaped formula — depth and utilisation signals, per-direction stabilisation windows, a tolerance band — plus crash replacement, and concurrencyPerWorker is applied to the workers it spawns rather than merely declared. Autoscaler, computeDesiredWorkers, and WorkerProcessManager are exported for embedding.

• Node request resource scope. A fresh instance on every useResource() call, each disposed when the task ends — matching the Java scope of the same name.

• Pub/sub backlog stats in the Node and Java SDKs. queue.topicStats(topic?) (Node) and topicStats() / topicStats(topic) (Java) return the per-subscription snapshot Python has as topic_stats(): pending, running, dead, and oldestPendingAgeMs, with a row for every registered subscription even at zero backlog.

• Job outcome events report how long the task ran: durationMs() on Java's OutcomeEvent, durationMs on Node's, and duration_ms on Python's job event payloads. Java also gains NodeSnapshot.durationMs() / compensationDurationMs() and TaskContext.elapsedMs().

• Full lifecycle event taxonomy in the Node and Java SDKs. Both grow from the 4 job-outcome events to the 29-event cross-SDK taxonomy: enqueue, per-attempt failure (job.failed), worker lifecycle, queue pause/resume, workflow submit/terminal/gate, saga compensation, and predicate rejection. Node types queue.on() per event name (EventMap); Java adds queue-level taskito.onEvent(EventName, Consumer<TaskitoEvent>) with typed event records alongside the unchanged worker-scoped Worker.Builder.on(...). Webhooks can subscribe to every event name. predicate.cancelled is reserved in all three — subscribable, typed and webhook-matchable — but only Python emits it: Node and Java gate at enqueue, where a terminal skip is predicate.skipped because no job exists yet.

• Java webhook deliveries use the dotted wire names. Bodies that previously said "event": "success" now say "event": "job.completed"; stored subscriptions with the legacy four names keep matching, and job-outcome bodies gain duration_ms.

• Retention dry-run. dry_run_retention() reports what a purge would delete right now — per table and in total, against the windows the elected cleaner actually published — so a window can be sized before it deletes anything. The dashboard echoes those live windows rather than guessing at the defaults.

• Retry predicates in the Node and Java SDKs. A per-task retryOn classifies a thrown error: return false and the job dead-letters immediately, whatever retry budget is left. One predicate covers both directions — test for the errors worth retrying, or negate a test for the permanent ones. Java can signal the same intent by exception type instead, via RetryableException / NonRetryableException, so a domain hierarchy doubles as the policy.

• Push-dispatch in the Node and Java shells. Both accept the pushDispatch option, so an enqueue wakes the scheduler immediately instead of waiting out the poll interval.

• Node standalone health and readiness helpers. Liveness and readiness checks that run without the dashboard, for a worker deployment that serves no HTTP of its own.

• Python periodic-schedule catalog management. list_periodic(), delete_periodic(name), pause_periodic(name) and resume_periodic(name) operate the cron catalog at runtime, so a schedule can be suspended or dropped without a redeploy.

• Java classifier jars and GraalVM metadata. Per-platform classifier artifacts publish alongside the fat jar, and bundled reachability metadata makes the SDK usable from a GraalVM native image.

Fixed

• A readiness probe that can actually run. /readiness is gated alongside /metrics, so a Kubernetes probe — which carries no credential, and cannot be given one, since a probe header is a literal string in the manifest — got 401 and the pod never went Ready. TASKITO_DASHBOARD_PUBLIC_READINESS=1 answers that one route without a credential; /metrics stays gated. The Helm chart sets it by default and probes /readiness; turn it off and readiness falls back to /health.

• The attach socket admits a shared group. bind left it at the umask-derived 0755, which denies the group write — and connect(2) needs write — so only the binding uid could attach. It is now chmodded to 0660, which is what makes a same-pod attach configurable: two containers can be given a shared fsGroup, where matching uids is rarely practical.

• Node worker.stop() is awaitable. It now returns a promise that resolves once worker-scoped resources have been disposed, instead of firing that teardown and returning. Dispatch, the heartbeat and log consumers still halt synchronously, so callers that ignore the result behave exactly as before.

• Node event listeners that throw or reject are logged at debug rather than silently swallowed; sibling listeners stay isolated from the failure as before.

• Postgres segfault on process exit. The connection pool opens connections on background threads that can still be running libpq/OpenSSL when the process exits, racing OpenSSL's default atexit teardown. Taskito now initializes OpenSSL itself with that teardown suppressed (OPENSSL_INIT_NO_ATEXIT) before the first Postgres connection.

• Namespace scoping is complete across the id-addressed storage APIs. Every method that addresses a row by id now takes the namespace it must match, so one tenant can no longer read, mutate or complete another's job by guessing its id. Cross-namespace dependencies are rejected at enqueue, and workflow runs are scoped through the store handle.

• Dashboard settings are compare-and-set in every SDK. Two dashboards editing the same JSON settings document no longer silently overwrite each other; a stale write is rejected instead of clobbering the newer one.

• The predicate event taxonomy is complete across SDKs. The gate outcome names each SDK recognized diverged, so a webhook subscribed to one could not be registered against another. All three now carry the same set. Separately, workflow.submitted fires on run submission rather than staying dormant.

• Java null-safety contract and native platform coverage. Nullability is declared across the public API, and the published artifact carries every supported platform's native library rather than only the build host's. JobRec reads are guarded by the backend monitor, and the build no longer silently disables every compile task.

• The Redis task-metric retention purge is bounded. It read its whole below-cutoff window in one ZRANGEBYSCORE and issued a GET per id, so the first sweep after a long retention gap stalled the maintenance tick. It now drains in batches like every other purge, with one MGET per batch.

• Batch enqueue applies unique_key dedup. A batch carrying any keyed entry now routes through the unique path rather than the plain multi-row insert, which would hit the partial unique index and fail the whole batch the moment a key duplicated an active job. Shared by the Node and Java batch enqueues; a keyless batch keeps the chunked throughput path, and Python still batch-inserts without dedup.

• Version drift in the Node CLI banner and the Java installation docs.

0.21.0

Overload-controls and retention release. The queue gains admission and load-shedding controls, retention runs by default with elected cleaners, an opt-in log/cursor pub/sub model lands, and taskito-core is prepared for standalone publication. Cross-SDK parity work and Node webhook hardening round it out.

Added

• Log + cursor pub/sub. An opt-in log topic mode writes one message row per publish (O(1)) and is consumed by pull — read, process, then ack — with per-message acknowledgement, a first-class topic registry, and a managed consumer that tracks cursors for you.
• Retention, on by default. Per-table retention windows purge aged job, dead-letter, and archived rows. A single reaper and retention cleaner are elected across workers so the sweep runs once, and every purge is bounded on all three backends.
• Overload controls. An opt-in max_pending admission cap rejects enqueues past a ceiling, opt-in CoDel load shedding drops work that has dwelled too long, and an opt-in LIFO dispatch order serves newest-first.
• Retry budget. A per-task token-bucket retry_budget caps how fast a task may retry, independently of max_retries.
• Native async dispatch. Coroutine tasks run on a native async executor, with worker drain decoupled from Python reference counts.
• taskito-tui. A headless terminal UI crate inspects and operates a queue directly against its database, with no changes to the core.
• Native Rust worker. A native worker with a task registry and dispatcher, plus typed Redis and JSON error variants, as part of readying taskito-core for standalone use.
• Cross-SDK parity. Node and Java gain per-task policy and batching parity; the remaining parity gaps are tracked explicitly.
• Single-source release version. One version literal in the root manifest drives every SDK, checked in CI so a mismatched manifest fails the build.

Changed

• Scale to millions of rows. Keyset pagination and bounded scans replace offset paging and full scans across listings and mass mutations.
• Async backpressure. In-flight work is bounded to the pool's worker and async concurrency, so async pools no longer over-claim.
• One task lifecycle. Synchronous and native-async dispatch share a single task-lifecycle path in Python.
• Retention tuning. PostgreSQL fillfactor and autovacuum are tuned for the retention workload, and the Redis archived-retention purge is bounded.

Fixed

• Node webhook SSRF guard. Webhook delivery screens loopback, private, link-local, and cloud-metadata targets at registration and re-checks every attempt; redirects are not followed.
• Node webhook store. Subscriptions persist in the shared cross-SDK layout, so a hook registered by any SDK is visible to all of them.
• Node log consumers flush when subscriptions are declared and stop earlier on worker shutdown.
• Negative result_ttl is rejected rather than silently accepted (Python).
• Documentation is served under /taskito without a doubled path prefix.

0.20.0

Scaling release: the queue survives higher task ingest/dispatch and pub/sub fan-out rates, and schema management moves to code-first migrations. Worker dispatch now respects each pool's execution capacity, so a worker no longer claims more than it can run and starve its peers.

Added

• Code-first migrations. All schema DDL is built programmatically with sea-query (no raw SQL) and applied by a versioned migrator with path-based auto-discovery — dropping a migrations/mXXXX_*.rs file registers it automatically. Migration recording is atomic and idempotent, so concurrent startups and interruptions are safe. New scaling indexes ship as part of the baseline.
• Batching. Keyed pub/sub fan-out is committed in a single transaction, Redis subscription reads are pipelined to constant round trips, execution claims are taken per batch, and successful results are finalized together — cutting round trips under load across all SDKs.
• Bounded in-flight dispatch. A scheduler now caps how many jobs it keeps in flight at the worker pool's execution parallelism, freeing a slot (and refilling immediately) as each job finishes.

Changed

• Drain-until-empty polling. Each poll wake dispatches all ready work instead of one job per tick, removing the throughput ceiling imposed by the poll interval.
• Jittered backoff. Retry backoff uses full jitter, and the poller's gate-reschedule delays are jittered, to avoid retry stampedes.
• Topic fan-out releases the GIL during publish/reap/backlog work (Python).

Fixed

• Worker fairness. Under drain-until-empty, one worker could claim a whole backlog and strand it Running on itself while peers sharing the database sat idle; in-flight dispatch is now bounded to the pool size.
• Workflow step double-execution. A step now honors the task's max_retries, and the tracker reacts only to a node's terminal job failure — closing an exactly-once violation under load.
• Jobs stranded by a transient claim error return to Pending immediately rather than waiting out the stale-job reaper.
• Postgres batch claims run in one transaction (no partially-committed claims on error).

0.19.0

Parity release: the Java SDK reaches feature parity with the other SDKs (and its baseline moves to Java 17), payload codecs land as a cross-SDK wire contract, and the resource system gains a bounded pooled scope plus health-check-driven recreation. One breaking change in the Node SDK: scan-heavy queue methods now return Promises.

Added

• Java SDK full parity. Workflows grow gates, conditional steps, sub-workflows, sagas, incremental caching, analysis/visualization, and canvas shortcuts. New: worker resource dependency injection with five scopes (worker / thread / task / request / pooled), enqueue predicates and interception, two-sided payload codecs with per-task annotations, signed resource proxies with expiry/purpose binding and sessions, producer-side batching, in-process autoscaling plus a scaler HTTP endpoint, an optional MessagePack serializer, observability middleware, a Spring Boot 3 starter, a mesh scheduling worker, and a Panama FFM fast-path for hot byte operations. Maven Central publishing is automated end to end.
• Payload codecs (all SDKs). A codec chain (gzip, AES-GCM, HMAC) that wraps the queue serializer on both the enqueue and result paths — configurable globally or per task via a named registry. The wire format is a cross-SDK contract: payloads encoded by one SDK decode in any other. Dedup keys hash pre-codec bytes, so encryption does not break idempotency.
• Pooled resource scope (Node + Java). A bounded checkout/return pool per resource: each task checks out one instance for its duration and returns it at task end. Capacity, prewarm, acquire timeout, and max lifetime are configurable; instances are disposed at worker shutdown or lifetime expiry.
• Node resource health checks. Per-resource healthCheck with automatic recreation and a terminal unhealthy state after the recreation budget is spent. The worker heartbeat now carries per-resource health, surfaced through listWorkers().
• Node proxy system. Signed proxy references with a file handler (HMAC-SHA256 over a canonical JSON form, path allowlists, TTL and purpose binding), plus proxy sessions with identity dedup on deconstruct, signature-keyed memoization on reconstruct (re-verified on every resolve), and LIFO cleanup on close.
• Node enqueue interceptors. Pass / convert / redirect / reject rules that run before defaults, middleware, and serialization — same model as the other SDKs.
• Java THREAD and REQUEST resource scopes, completing the scope matrix alongside worker, task, and pooled.

Changed

• Node (breaking): scan-heavy queue methods are now async. stats*, listJobs, deadLetters*, purge*, listWorkers, getMetrics, and getJobErrors return Promises and no longer block the event loop; webhook Delivery was reshaped to the dashboard contract.
• Java baseline is now 17 (previously 11).

Fixed

• Python results always use the queue serializer. Task results were serialized with raw cloudpickle on three of the four write paths while reads went through the configured serializer; all result writes now route through the queue serializer (with a legacy fallback so old results stay readable).
• Node workflow tracker correctness (static-successor cascade on managed runs, fan-out sibling successors — both also fixed in Python), plus cursor-based task-log streaming.

0.18.0

Reliability and storage release: Postgres dequeues with SKIP LOCKED, crashed workers' in-flight jobs are recovered within ~30s instead of waiting their full timeout, and the dead-letter and filter paths are indexed. The internal job_payloads side table was removed (payloads inline again). No public API or wire-format change.

Changed

• Storage dequeue hardening. Postgres now dequeues with FOR UPDATE SKIP LOCKED, so concurrent workers claim disjoint jobs instead of all scanning the same candidates and racing on the claim. dequeue_batch no longer reads payload blobs during its candidate scan, and expired batch candidates are archived instead of left stranded in the live table. No API or wire change.

Added

• Fast in-flight recovery on worker death. When a worker process crashes mid-job, a surviving worker now requeues its Running jobs within ~30s of the missed heartbeat (via retry / dead-letter) instead of waiting the job's full timeout. The scheduler claims execution under its real worker_id and a new maintenance step reclaims orphaned claims atomically, so concurrent survivors never double-rescue. Works across Python, Node, and Java; SQLite/Postgres/Redis. No API change.
• Indexes for dead-letter and filter paths. dead_letter (previously unindexed) gains indexes on failed_at and task_name; jobs gains (task_name, status) and partial indexes on expires_at / namespace; archived_jobs gains created_at and namespace. Keeps DLQ, listing, and maintenance queries flat as tables grow.

Removed

• job_payloads side table. Reverted the 0.15 payload side table — payload and result live inline on jobs/archived_jobs again. The narrow dequeue scan already excludes the blobs by column selection (Postgres TOAST / SQLite overflow keep them off the scanned pages), so the side table bought nothing. It is dropped on migration; the inline columns were never removed, so no data moves.

0.17.0

Feature release across the SDKs: periodic-task management, task-scoped dead-letter queries, and the first Java SDK release.

Added

• Java SDK. First release, published to Maven Central as org.byteveda:taskito. A JNI binding over the Taskito core with the full producer / worker / inspection / admin surface, workflows, distributed locks, periodic tasks, an in-memory taskito-test backend, and a @TaskHandler annotation processor. Verified under GraalVM native-image.
• Periodic task management. List, unschedule, pause, and resume registered cron tasks — listPeriodic / deletePeriodic / pausePeriodic / resumePeriodic (Java, Node) and the equivalent native bindings (Python). Pause toggles the task's enabled flag without removing it (#313).
• Per-task dead-letter queries. List and purge dead-letter entries scoped to a single task — listDeadByTask / purgeDeadByTask (Java), deadLettersByTask / purgeDeadByTask (Node), and native bindings (Python). Filtering happens server-side so pagination stays correct (#314).
• Per-task retry backoff (Java). Task.retryPolicy(...) surfaces the core retry engine's backoff curve — exponential with jitter, or exact explicit delays — with the retry budget still set per enqueue (#311).

Changed

• Node now requires Node.js 20+. Node 18 reached end-of-life; the test toolchain (vitest 4) requires Node 20.12+. The engines floor and tsup target were raised accordingly (#312).

0.16.4

Maintenance release. Upgrades the Rust↔Python binding layer and refreshes project docs; no public API changes.

Changed

• PyO3 0.29. Upgraded the PyO3 binding layer from 0.22 to 0.29 (and pyo3-log to 0.13). Internal only — no Python API change. Migrates to the Bound / IntoPyObject APIs and declares gil_used, laying the groundwork for Python 3.14 and free-threaded builds.
• Node SDK default SQLite settings. Opening a SQLite-backed queue now applies sensible defaults out of the box (#292).
• Documentation. Restructured the READMEs into a project-level root README plus self-contained per-SDK pages, and introduced this root CHANGELOG.md — now also rendered on the documentation site.

0.16.3

Reliability patch — correctness fixes across the scheduler and storage layer, plus the missing Django admin templates. No API changes.

Fixed

• Django admin templates. The taskito.contrib.django admin views shipped without their templates, so every admin page raised TemplateDoesNotExist. The templates are now authored and packaged, and the integration guide documents the required taskito.contrib.django entry in INSTALLED_APPS.
• At-most-once retries. An explicit per-call max_retries=0 is no longer overridden by the task-policy default, so a no-retry job is no longer re-executed up to the policy limit.
• Unique enqueue. enqueue_unique now validates dependencies inside the transaction — a missing dependency no longer runs the job and a dead/cancelled dependency no longer strands it Pending — and never returns a phantom (rolled-back) job under unique-key contention.
• Dead-letter policy. DLQ auto-retry is scoped to the worker's namespace and served queues (no longer resurrecting other workers' entries); per-entry and per-job result TTLs are purged even without a global result_ttl; and job metadata survives the dead-letter round trip.
• SQLite transaction safety. Read-then-write operations (dequeue, enqueue, rate-limit) use immediate write-priority transactions, avoiding SQLITE_BUSY deadlocks; migration ALTER TABLE failures now propagate instead of being downgraded to warnings.
• Postgres rate limiting. The rate-limit row is seeded and locked FOR UPDATE, preventing concurrent workers from over-admitting beyond the configured limit.

0.16.2

Reliability patch — correctness fixes across the scheduler, dashboard, and Redis backend. No API changes.

Fixed

• Scheduler retry budget. Soft-gate reschedules (rate limit, circuit breaker, concurrency cap, channel backpressure) no longer consume a job's retry budget; a job that never executed is rescheduled without incrementing retry_count, so it is no longer dead-lettered early on its first real failure.
• Webhook secrets. Webhook signing secrets and delivery logs (stored under webhooks: keys) are no longer readable through GET /api/settings.
• Structured notes. The dashboard job detail page no longer crashes on jobs with notes (the API now emits notes as a canonical JSON string), and non-finite floats (NaN/Infinity) are rejected at enqueue instead of producing invalid JSON that broke the jobs API.
• Redis write-path atomicity. Closed a cluster of races in the Redis backend: the dequeue claim is now an atomic compare-and-set (a cancelled/expired job can no longer be resurrected and executed); retry/reschedule requeue atomically (no stranded jobs); complete and purge release the unique-key pointer with a compare-and-delete (no clobbering a reused key); progress/cancel writes are guarded against resurrecting an archived job; and the dead-letter move commits the DLQ entry and the archive together.

0.16.1

Patch release with no user-facing changes.

0.16.0

Feature release: mesh scheduling, DLQ policies, and a full dashboard redesign.

Added

• Mesh scheduling. Decentralized worker overlay network for task affinity and work-stealing. SWIM gossip protocol for peer discovery, consistent-hash ring for affinity routing, TCP work-stealing for load balancing. New taskito-mesh crate (feature-gated via mesh). Python API: MeshWorker class passed to queue.run_worker(mesh=...). The database remains source of truth — mesh is a dispatch optimization only.
• DLQ policies. Dead-letter queue entries now support auto-retry, per-entry TTL, and explicit delete. New Queue params: dlq_auto_retry_delay and dlq_auto_retry_max. New methods: delete_dead() / adelete_dead(). Storage trait gains delete_dead, purge_dead_with_ttl, and list_dead_for_retry across all three backends. Dashboard adds a Discard button and retry-count badge on the Dead Letters page.
• Dashboard "Friendly ops" redesign. Full re-skin with warm-paper neutrals, emerald accent, and IBM Plex typography. New shared primitives: Switch, QueueBar, MeterBar, Segmented, Stepper, KvList, Callout, LiveDot, StatusBadge. Overview page gains a health pulse banner, workers card, and busiest-queues table. All pages restyled. Light/dark toggle preserved; custom accent override via useApplyAccent.
• Dashboard workflows module. Workflows page with routes and nav entry; command palette entries for Tasks, Webhooks, and Settings.
• Dashboard change-password dialog in the user menu.

Fixed

• OAuth callback errors redirect to login instead of showing a raw error page. Missing joserfc dependency added to the oauth extras group.
• Dashboard timestamp contract enforced. Webhook store, overrides store, and auth endpoints now emit Unix milliseconds (matching the documented API contract) instead of seconds — fixes "56 years ago" display bugs.
• Dashboard sidebar health derived from stats query, no longer hardcoded green.
• Dashboard sidebar polls at configured interval on all pages, preventing stale "Updated Xs ago" timers.
• Fixed sidebar scrolls independently of page content.
• Dead code removed from dashboard (QueueBreakdown, formatBytes, unused webhook queries).
• Consistent vertical spacing across all dashboard pages.
• Workers card shows overflow count ("+N more" link when >6 workers).
• Accessibility improvements. prefers-reduced-motion honored for animations; aria-live on Stepper value changes; aria-pressed on live-tail toggle; accessible name on tasks search input.

Changed

• Dashboard dependencies updated (22 packages in the dashboard-deps group).

Notes

• jobs.payload and jobs.result columns (dual-written since 0.15.0 for rollback safety) are not dropped in this release. They remain for one more cycle to give any lagging workers time to upgrade. Expect removal in 0.17.

0.15.2

Security-hardening release. No breaking changes.

Added

• SignedSerializer. Opt-in HMAC-SHA256 integrity wrapper for task payloads and results — a worker refuses to deserialize bytes not produced with the shared key, closing the untrusted-storage code-execution path. Compose with EncryptedSerializer for confidentiality + integrity.
• Queue.max_payload_bytes. Per-queue cap (default 1 MiB) that rejects oversized serialized payloads at enqueue time.
• Security guide. New Operations → Security page documenting the trust model and a production hardening checklist.

Changed

• Dashboard authorization is enforced server-side. Mutating API routes now require the admin role; viewer sessions are read-only. Session and CSRF cookies carry Secure, and /metrics / /readiness can be gated behind TASKITO_DASHBOARD_METRICS_TOKEN.
• Scaler binds to 127.0.0.1 by default. Use --host 0.0.0.0 only behind a trusted network boundary.

Fixed

• Settings API no longer leaks auth data. /api/settings hides and rejects internal auth: keys (password hashes, sessions, CSRF secret).
• Webhook SSRF closed. Target URLs are re-validated against private/loopback/metadata addresses at delivery time (defeating DNS rebinding) and redirects are no longer followed.
• Proxy hardening. File-proxy allowlist prefix/symlink bypass closed; argument reconstruction restricted to safe type kinds; a warning is emitted when proxy recipes are reconstructed without a signing key.
• Lock owner token no longer disclosed. DistributedLock.info() masks another holder's owner_id; Redis locks set a native TTL with an atomic reap.
• No duplicate execution on claim errors. The scheduler skips dispatch when a claim attempt errors instead of running the job unguarded.
• Job-list API summarizes tracebacks. Only the final exception line is shown in the jobs list; the full traceback stays on the per-job errors endpoint.

0.15.1

Fixed

• Mobile navigation drawer closes on navigation. The drawer previously stayed open after tapping a link; it now closes on every navigation, including re-selecting the current route.
• Mobile menu shows the full navigation. The Tasks and Webhooks links were missing from the mobile menu. The sidebar and mobile menu now share a single navigation definition so the two can no longer drift.
• Job logs tab virtualized. The job detail logs view renders only the visible rows, keeping jobs with hundreds of log lines smooth to scroll.
• Resources table loading state matches its layout. The placeholder now uses the table-shaped skeleton instead of a single opaque block.

0.15.0

See the Upgrading to 0.15 guide for migration steps, rolling-upgrade notes, and the downgrade floor.

Breaking

• Serializer default changed to SmartSerializer. Payloads written by 0.15 carry a 1-byte codec tag and cannot be read by pre-0.15 workers. Upgrade all workers together; if rolling deployment is required, pin Queue(serializer=CloudpickleSerializer()) until every worker is on 0.15. Reading old (untagged cloudpickle) payloads is fully backward compatible.
• Terminal jobs archived immediately. Completed, failed, dead-lettered, and cancelled jobs move to archived_jobs on completion. Pre-0.15 binaries only look in jobs and will not see archived rows — treat 0.15 as a minimum-version floor for any upgraded database.

Added

• SmartSerializer (new default). msgpack for plain-data payloads (faster, smaller); cloudpickle fallback for lambdas, closures, and custom classes. msgpack is now a base dependency. Tuples are preserved; namedtuples fall back to cloudpickle.
• Queue(scheduler_batch_size=N). New optional kwarg; defaults to 1 (unchanged behaviour). Raise it to claim up to N jobs per scheduler round-trip for higher throughput under sustained load.
• Opt-in push-based dispatch (push_dispatch=True). Event-driven wakeups instead of polling — removes the dispatch latency floor and idle DB load. Off by default; requires building the extension with --features push-dispatch (not included in the default wheel). SQLite and Redis are fully event-driven; Postgres falls back to a faster poll pending a native LISTEN listener.

Changed

• Per-task / per-queue stats computed server-side. Replaces O(N) full scans with SINTERCARD. No API change, no data migration.
• Job payloads moved to job_payloads side table. The dequeue path no longer loads payload BLOBs for candidates it does not claim. Transparent — no API change. jobs.payload / jobs.result are dual-written in 0.15 for rollback safety and will be dropped in 0.16. Redis is unaffected.

0.14.2

Fixed

• wait() no longer returns transient FAILED during saga startup. WorkflowRun.wait()'s poll safety-net could observe the run in Failed state before the saga orchestrator transitioned it to Compensating. When a tracker event is registered but has not yet fired, wait() now defers to the event instead of returning the possibly-transient terminal state. The event is set only after the saga decision is fully resolved, so wait() can never prematurely return Failed for a run that is about to compensate.
• _mark_run_compensating retries on transient SQLite lock. The Failed → Compensating storage write could silently fail under SQLite file contention, leaving the run stuck in Failed. The method now retries up to 5 times with backoff, matching the existing retry pattern in compensation job dispatch.

0.14.1

Fixed

• Saga compensation race on Windows SQLite. start_compensation transitioned the workflow run from Failed to Compensating only after a storage I/O call (get_base_run_node_data) that could be slow under SQLite file contention — especially on Windows. During that gap, WorkflowRun.wait()'s poll loop could observe the transient Failed state (which is terminal) and return early, before the saga had a chance to run. The Compensating transition now happens immediately after the cheap in-memory eligibility check, closing the race window. A vacuous-compensation edge case (eligible but no nodes to undo) is handled by finalising as Compensated immediately and resolving any pending parent saga node inline.

0.14.0

Added

• Bare-metal autoscaler. taskito.autoscale provides an in-process HPA-style control loop that spawns and drains taskito worker subprocesses based on queue depth and utilisation. Mirrors the Kubernetes HPA dual-signal formula (depth_desired + util_desired). Configurable stabilisation windows (default: scale-up immediate, scale-down 5 minutes), tolerance band (10%), overload override, and crash recovery. Use serve_autoscaler(queue, AutoscaleConfig(...)) as the entry point. See the Bare-Metal Autoscaler guide.
• Dashboard /api/workflows backend. Four new REST endpoints expose workflow run data: GET /api/workflows/runs (paginated list, filterable by definition name and state), GET /api/workflows/runs/{id} (run header + per-node detail with compensation fields), GET /api/workflows/runs/{id}/dag (DAG JSON), and GET /api/workflows/runs/{id}/children (sub-workflow runs). All timestamps are Unix milliseconds.
• Canvas saga compensation. chain, group, and chord support .with_compensation([...]). On failure, compensators for completed steps are enqueued with a deterministic canvas_compensation:{run_id}:{slot} idempotency key. Dispatch order: chain → reverse-sequential, group → parallel, chord → callback compensator first then group members. Signature.with_compensation(compensator) lets individual signatures carry their own compensator.
• Per-item batch results. @queue.task(batch={"per_item_results": True}) enables per-item tracking for batched tasks. The task returns list[BatchItemResult]; each caller's BatchedJobResult.result() returns only that caller's value. BatchItemResult.success(item_index, result) and BatchItemResult.failure(item_index, error) are the constructors. partial_failures() returns failed items after a successful batch. BatchPartialFailureError and BatchResultTypeError are the new exception types. Cannot be combined with idempotent=True.
• Saga compensate_on_continue. Workflow(on_failure="continue", compensate_on_continue=True) triggers compensation after a continue-mode run terminates with failures. The run transitions through CompletedWithFailures before entering Compensating, then ends in Compensated or CompensationFailed. Without the flag, continue-mode runs end in CompletedWithFailures with no compensation.
• COMPLETED_WITH_FAILURES workflow state. New WorkflowState variant for on_failure="continue" runs that finished with at least one step failure. Emits WORKFLOW_COMPLETED_WITH_FAILURES event. Accessible as "completed_with_failures" in the REST API state field.

0.13.0

Added

• Workflow PostgreSQL backend. WorkflowPostgresStorage mirrors the SQLite implementation through the shared impl_workflow_diesel_ops! macro. Workflows now run on Postgres-backed queues — the previous PyRuntimeError on non-SQLite backends is gone.
• Workflow Redis backend. WorkflowRedisStorage (in crates/taskito-workflows/src/redis_store.rs) stores definitions, runs, and nodes as hashes under a wf: prefix, with sorted-set indexes for state/definition lookup and a Lua FINALIZE_FAN_OUT script for atomic fan-in CAS.
• Multi-backend workflow contract suite (crates/taskito-workflows/tests/storage_contract.rs). 27 cases run against every workflow backend. CI now verifies parity on every PR for SQLite, PostgreSQL, and Redis.
• @queue.task(batch=...). Producer-side task batching collects per-call items in memory and dispatches them as a single job whose payload is the list. Tunable max_size / max_wait_ms. See /docs/guides/core/batching.
• Saga compensation. @queue.task(compensates=...) and Workflow.step(..., compensates=...) declare a compensator. On forward failure (in fail_fast mode) taskito runs the registered compensators in reverse topological order, with idempotency guaranteed via compensation:{run_id}:{node_name} dedup keys. New workflow states: Compensating, Compensated, CompensationFailed. See /docs/guides/workflows/sagas.
• Compensator contract fully wired. Compensators now receive populated (forward_args, forward_kwargs, forward_result) for both deferred and static workflow nodes. current_compensation_context() returns a populated CompensationContext inside compensator bodies (workflow run id, node name, forward job id, forward result).
• Sub-workflow saga propagation. When a parent workflow's saga reaches a sub-workflow node whose child run has its own compensators, compensation propagates into the child. Child saga runs to completion, then the parent node is marked accordingly and the parent's wave advances. Depth-capped at 10 levels to prevent runaway recursion.
• Saga examples + API reference. /docs/more/examples/saga-checkout, /docs/more/examples/batch-emails, /docs/api-reference/saga, /docs/api-reference/batching.

Fixed

• Atomic fan-out node creation. expand_fan_out now uses the transactional create_workflow_nodes_batch to insert all child workflow nodes in one transaction. A crash partway through fan-out expansion no longer leaves half-tracked children.
• Diesel ?-placeholder rewriting for Postgres. sql_query does not auto-rewrite ? → $N for PostgreSQL; the workflow crate's macro now does it explicitly via the pg_rewrite helper.

0.12.3

Added

• Dashboard SSO (OAuth & OIDC). Native sign-in for Google, GitHub, and any OIDC-compliant provider (Okta, Auth0, Keycloak, Microsoft Entra) sits alongside the existing password login. Multiple named OIDC providers run side-by-side, each rendered as its own button on the login screen. Configuration is env-var driven (TASKITO_DASHBOARD_OAUTH_*); the oauth extra (pip install 'taskito[oauth]') pulls in authlib, joserfc, and requests. Security: PKCE S256, single-use server-side state (5-min TTL), nonce verification, JWKS-validated ID tokens, issuer/audience/expiry checks, open-redirect protection on the post-login next URL, HTTPS-only redirect URIs outside localhost. Allowlists by Google Workspace domain, GitHub org, or OIDC email domain. Promote OAuth users to admin via an explicit TASKITO_DASHBOARD_OAUTH_ADMIN_EMAILS list, with a first-user-wins fallback for empty deployments. Password login can be disabled entirely with TASKITO_DASHBOARD_PASSWORD_AUTH_ENABLED=false.
• Dashboard SSO operator guide. New Dashboard › SSO (OAuth & OIDC) doc walks through registering OAuth clients with Google, GitHub, and generic OIDC providers, plus the full env-var reference, allowlist semantics, security model, and troubleshooting cookbook. Includes a Mermaid sequence diagram of the end-to-end flow.

Changed

• Docs nav: dedicated Dashboard section. The dashboard documentation outgrew Observability — five pages (overview, authentication, SSO, task overrides, REST API) versus three actual observability topics. Moved them into their own top-level Guides section and stripped redundant prefixes from page titles (Dashboard Authentication → Authentication, etc.). All cross-section links updated.

Fixed

• WebhookManager delivery thread leak. reload() unconditionally spawned a daemon thread on every Queue construction. With ~800 tests each creating a fresh Queue, macOS CI runners blew through the per-process thread limit and panicked in r2d2's reaper / tokio's worker-thread spawn (Resource temporarily unavailable). The thread now starts only when at least one subscription exists, matching the pre-0.12.2 behaviour.
• EncryptedSerializer.loads exception handling. A blanket except Exception re-wrapped every failure as ValueError, including programmer errors like MemoryError that should propagate untouched. The catch is now narrowed to cryptography.exceptions.InvalidTag (the one expected failure mode); the original exception is preserved on __cause__ for debugging. The InvalidTag class is also pre-cached on __init__ so loads avoids a per-call import. This also fixed two latent test failures (test_wrong_key_fails, test_tampered_ciphertext_fails) that only surfaced once a release pulled in cryptography via the OAuth extra.

Internal

• HttpClient Protocol for OAuth providers. GoogleProvider / GitHubProvider / GenericOIDCProvider previously typed their http parameter as requests.Session, forcing every test to use # type: ignore[arg-type] to inject a stub. The Protocol captures the small subset of Session actually used (one get method) so production code passes a requests.Session and tests pass an in-memory stub — no nominal-type fight, no runtime casts.
• CI installs the oauth extra. uv sync --extra dev was leaving authlib / joserfc / requests uninstalled, so the OAuth test modules failed collection with ModuleNotFoundError once they shipped. Both lint and test jobs now sync --extra dev --extra oauth. requests is also pinned explicitly in the oauth extra (Authlib does not declare it as a hard dep).
• Cross-version mypy compatibility on JWKS decoding. joserfc.jwk.KeySet.import_key_set was widened to accept dict-shaped JWKS in mypy 2.x; mypy 1.x still requires the KeySetSerialization TypedDict. Suppressed with the standard # type: ignore[arg-type, unused-ignore] dual pattern so the file lints under both versions.

Test counts at release

• Rust: 95 tests (default), 107 with --features workflows
• Python: 896 collected across 74 files (up from 592 / 58 at 0.12.2)
• Dashboard (vitest): 106 tests across 10 files

0.12.2

Fixed

• Inline imports inside the predicate boolean combinators. AndPredicate.evaluate, OrPredicate.evaluate, and NotPredicate.evaluate were each importing _resolve_outcome from taskito.predicates.evaluate inside the method body. The cycle they were defending against doesn't exist at runtime — evaluate.py only references core under TYPE_CHECKING — so the import was hoisted to module scope, satisfying the project's no-inline-imports rule.

Internal

• QueuePredicateMixin extracted from app.py. Predicate state (six instance dicts and PredicateMetrics) plus the three gating methods (_apply_enqueue_predicate, _apply_dispatch_predicate, _reenqueue_after_defer) and the public inspection / registration API (list_predicates, predicate_for, register_predicate) now live on a dedicated mixin under py_src/taskito/mixins/predicates.py. Repeated _emit_event(...) blocks consolidated into small helpers. app.py drops from 901 to 619 LOC.
• QueueRuntimeConfigMixin split from mixins/decorators.py. register_type, set_queue_rate_limit, and set_queue_concurrency are runtime configuration knobs, not decorator surface — they moved to a new mixin alongside the existing QueueSettingsMixin (which manages dashboard key/value state). mixins/decorators.py drops from 597 to 533 LOC.
• Migration DDL deduplicated via diesel_common/migrations.rs. The run_migrations() methods on SqliteStorage and PostgresStorage shared ~750 LOC of nearly-identical CREATE TABLE / CREATE INDEX / ALTER TABLE statements. The shared module now exposes create_tables(&dialect), create_indexes(), and alter_statements(&dialect); a Dialect struct holds per-backend type substitutions (BLOB/BYTEA, INTEGER/BIGINT, REAL/DOUBLE PRECISION, boolean defaults, and the IF NOT EXISTS prefix on ALTER). Each backend's run_migrations() is now a ~10-line driver. sqlite/mod.rs drops 502 → 126 LOC; postgres/mod.rs drops 508 → 130 LOC.
• tests/core/test_predicates.py — 30 focused tests covering AST short-circuit semantics, fail-closed evaluation + metric recording, JSON and string-DSL round-trips, recipe behaviour (after/before/in_time_window/payload_matches/env_var_truthy), the callable adapter, custom predicate registration, and three queue integration tests for enqueue-time cancel/defer.

Test counts at release

• Rust: 95 tests (default), 107 with --features workflows
• Python: 592 passed, 9 skipped across 58 files
• Dashboard (vitest): 121 tests across 9 files

0.12.1

Fixed

• PyPI wheels and sdist now bundle the compiled dashboard. A regression in 0.12.0 shipped wheels with an empty taskito/static/ directory because the release pipeline ran maturin build without first building the dashboard frontend. End users hit Dashboard assets not bundled when running taskito dashboard. The release pipeline now builds the dashboard once in a dedicated job, distributes it to every wheel/sdist job as a build artifact, and verifies static/dashboard/index.html is present before each maturin build invocation — preventing the regression class entirely.

Internal

• .github/actions/dashboard-build/ — composite action centralizing pnpm/Node toolchain versions and the dashboard build steps. Consumed by both publish.yml (release) and dashboard.yml (CI), so the assets shipped in a wheel are produced by the exact same build path that CI tests on every PR.