Attached Executors

An attached executor splits a worker in two. The scheduler — flexiq-server — holds the database connection, claims jobs, and owns retries, the dead-letter queue and retention. Your app container runs flexiq executor, which dials the scheduler, announces the tasks it can run, and executes whatever it is sent.

The app image needs no database credentials and no inbound port.

For the deployment shape — a Compose stack, a Kubernetes sidecar over a shared Unix socket, and the security requirements that go with the attach port — see Deployment.

Running the scheduler

flexiq-server is configured entirely through the environment — there are no flags. At minimum it needs a DSN and an attach address:

FLEXIQ_DSN=postgres://user:pass@db/flexiq \
FLEXIQ_LISTEN=0.0.0.0:7777 \
FLEXIQ_ATTACH_TOKEN=$(openssl rand -hex 32) \
flexiq-server

The scheduler starts lazily, on the first attach. With nothing attached it would claim jobs no executor advertises, and each would fail retryably once the placement timeout elapsed — a retry storm against an idle deployment.

Running an executor

FLEXIQ_ATTACH=scheduler:7777 \
FLEXIQ_ATTACH_TOKEN=... \
flexiq executor --app myapp:queue --slots 4

FLEXIQ_ATTACH=scheduler:7777 \
FLEXIQ_ATTACH_TOKEN=... \
flexiq executor ./app.js --slots 4

FLEXIQ_ATTACH=scheduler:7777 \
FLEXIQ_ATTACH_TOKEN=... \
java -cp app.jar org.byteveda.flexiq.cli.Cli executor --slots 4

--slots is how many jobs run at once; the scheduler dispatches a task name only to executors that advertised it, so a handler missing from an executor is a job that never reaches it.

Security

The attach port dispatches code, so the defaults are strict:

• A non-loopback FLEXIQ_LISTEN refuses to start without FLEXIQ_ATTACH_TOKEN.
• The shared secret is compared in constant time, and never logged — an attach failure is logged with the executor id and peer only.
• hello must arrive before any other frame; an unauthenticated socket is dropped rather than queued, and never receives an acknowledgement.
• Use a Unix socket (unix:/run/flexiq.sock) for a same-pod sidecar; it needs no token because the filesystem permissions are the boundary.

What a task can do on an executor

A task body behaves the same as it does on an in-process worker. Progress, log lines and published partials all work — the executor has no storage, so it reports them to the scheduler, which applies them:

@queue.task()
def resize(path: str) -> str:
    current_job.update_progress(50)
    current_job.log("halfway")
    current_job.publish({"stage": "halfway"})
    return "done"

queue.task("resize", async (path: string) => {
  const job = currentJob();
  job?.setProgress(50);
  job?.log("halfway");
  job?.publish({ stage: "halfway" });
  return "done";
});

@TaskHandler("resize")
public String resize(String path) {
    JobContext job = JobContext.current();
    job.setProgress(50);
    job.log("halfway");
    job.publish(Map.of("stage", "halfway"));
    return "done";
}

Middleware disabled from the dashboard is honoured too. The executor cannot read settings, so the scheduler resolves the list and attaches it to each dispatch — a toggle still takes effect on the next job, with nothing to restart.

What is not available

Anything that needs the database directly. An enqueue, a workflow submission or a queue inspection raises rather than silently doing nothing, because an enqueue that quietly vanished would be worse than one that failed.

A handful of job fields also arrive as zeros and nulls, because a dispatch frame carries what running the task needs rather than the whole row: created_at, scheduled_at, priority, unique_key and notes. metadata is the exception — middleware reads it, so it rides the frame. A task that needs the rest wants a worker, not an executor.

Version skew

The scheduler and its executors upgrade independently. The handshake carries a capability list rather than a version both sides must match, so a newer executor attached to an older scheduler never sends a frame that scheduler could not parse — it degrades instead.

Concretely: a scheduler that does not advertise side_channel gets no progress or task-log frames, and those calls become no-ops (logged once). Everything else — dispatch, results, retries, cancellation — is unaffected. The executor logs which capabilities it negotiated at attach.

Skew in the other direction is covered too. A frame type the receiving side has never heard of is logged once and skipped rather than treated as a corrupt stream: a header declares its own payload length, so the reader stays aligned on the next frame. That holds both ways, so neither an older scheduler nor an older executor drops a live session — and its in-flight jobs — over a frame it did not need.

Trade-offs