Trace context across SDKs
One trace from the request that enqueues a job to the worker that runs it, even when the two are different SDKs.
One trace from the request that enqueues a job to the worker that runs it, even when the two are different SDKs.
A trace that starts in one SDK's request, enqueues a job, and executes in another SDK's worker is one trace. flexiq carries the W3C Trace Context in the job itself, so the producer and the worker need no shared process, broker or sidecar — only an OpenTelemetry integration on each side.
The repo's polyglot example (examples/polyglot) runs a Python producer, a Node
worker and a Java worker; CI asserts that each order yields exactly one trace.
The carrier is the job's metadata JSON object. Two keys at its top level use the W3C header names:
{
"traceparent": "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01",
"tracestate": "vendor=value"
}At enqueue, the OpenTelemetry integration injects the current context into the job's metadata. At execution, the worker extracts it and starts the execute span as a child of the enqueuer's span.
Injection never clobbers metadata you set yourself:
| Metadata at enqueue | Result |
|---|---|
| Absent or empty | Becomes the carrier object |
| A JSON object | Gains traceparent / tracestate, unless it already has either key |
A JSON object with traceparent or tracestate | Left as is — the caller's own context wins whole |
| Not a JSON object (array, string, non-JSON text) | Left untouched; the job starts a new trace |
When keys are added, the existing bytes are kept as they were: the new keys are spliced in after the opening brace, and the caller's keys are never overwritten.
The worker extracts the context from the job's metadata and creates the
execute span flexiq.execute.<task> (span kind CONSUMER) as a child of the
enqueuer's span. A job enqueued from inside a task continues that task's
execute span, so a chain of jobs stays one trace.
| Path | Behaviour |
|---|---|
| Retry | Same job row, so every attempt is a child of the original enqueuer |
step.sleep resume | Same row; the resumed attempt is again a child of the enqueuer |
| Pub/sub fan-out | Each subscriber's delivery job copies the message's metadata, so a carrier in the published metadata reaches every subscriber |
| Dead-letter replay | The replayed job keeps its metadata, continuing the original trace |
| Attached executor and gRPC executor door | Metadata rides the dispatch frame |
| gRPC producer door and JSON facade | Put traceparent / tracestate in EnqueueOptions.metadata, or send them as traceparent / tracestate request headers (see below) |
| Periodic jobs | A new trace per firing — there is no caller to continue |
| Workflow node jobs | Carried from the submit — see Workflows below |
A workflow submit stores the submitter's context on the run. Every node job
carries it beside the run's routing keys (workflow_run_id,
workflow_node_name), which it never displaces:
{
"workflow_run_id": "0199…",
"workflow_node_name": "charge",
"traceparent": "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01"
}Every step's execute span is a child of the submit span, not of the step before it — flat, the way pub/sub fan-out already is. That holds for steps enqueued at submit and for those the tracker enqueues later: fan-out children, fan-in collectors, successors of a condition or gate, and compensation jobs. A sub-workflow continues its parent run's trace.
With the OpenTelemetry integration registered, the submit picks up the current context on its own. You can also pass a carrier explicitly; it wins over the middleware's.
run = queue.submit_workflow(wf, trace_context={"traceparent": traceparent})The integration fills it through the on_workflow_submit middleware hook,
which any middleware can implement.
const handle = queue.workflows.submit(builder, { traceContext: { traceparent } });The integration fills it through the onWorkflowSubmit middleware hook, which
any middleware can implement.
WorkflowRun run = flexiq.submitWorkflow(wf, Map.of(), Map.of("traceparent", traceparent));The integration fills it through Middleware.onWorkflowSubmit, which any
middleware can implement.
The server merges traceparent / tracestate request headers into the job's
metadata by the same rule as above. SubmitWorkflow takes the same headers
and stores them on the run, as described in Workflows.
traceparent or tracestate already in the metadata wins over the headers.traceparent header is ignored, together with its tracestate.tracestate over 512 bytes is dropped.Each side needs its own OpenTelemetry integration and a W3C propagator.
Add OpenTelemetryMiddleware to the queue. Python's OpenTelemetry API uses the
W3C propagator by default, so nothing else is needed.
from flexiq import Queue
from flexiq.contrib.otel import OpenTelemetryMiddleware
queue = Queue(middleware=[OpenTelemetryMiddleware()])Details and options: OpenTelemetry Integration.
Register the middleware from the @byteveda/flexiq/contrib/otel subpath.
import { propagation } from "@opentelemetry/api";
import { W3CTraceContextPropagator } from "@opentelemetry/core";
import { otelMiddleware } from "@byteveda/flexiq/contrib/otel";
propagation.setGlobalPropagator(new W3CTraceContextPropagator());
queue.use(otelMiddleware());The Node OpenTelemetry API registers no propagator by default, so register one as above, or use the OpenTelemetry Node SDK, which does it for you.
Middleware hooks cannot wrap the handler, so the execute span is not made active for the handler's own spans. An enqueue inside the handler still continues it.
Details and options: OpenTelemetry Integration.
Register FlexiQOtel with the OpenTelemetry instance your app configures.
Propagators come from that instance.
import org.byteveda.flexiq.contrib.FlexiQOtel;
flexiq.use(new FlexiQOtel(openTelemetry));The execute span is current for the handler, so the handler's own spans and enqueues are its children.
Details and options: OpenTelemetry Integration.
The Rust SDK ships no OpenTelemetry integration and has no middleware to hang
one on. A Rust producer can still start a trace that another SDK's worker
continues: inject your context into a map with your OpenTelemetry crate's
propagator and pass it, JSON-encoded, to the enqueue options' metadata. A
Rust task does not see its job's metadata, so a job a Rust worker runs starts a
trace of its own.