Streaming partial results
Publish intermediate values from a task via task logs and consume them live.
Publish intermediate values from a task via task logs and consume them live.
A long-running task can publish intermediate results as it works; a consumer polls them as they arrive — useful for ETL, ML training steps, or batch progress. In Java, partials ride on task logs: durable, per-job log rows in shared storage.
From inside the handler, write a log row against the job (get the id across with the middleware pattern):
for (int epoch = 1; epoch <= epochs; epoch++) {
double loss = trainEpoch(epoch);
flexiq.writeTaskLog(jobId, "train", "result",
"{\"epoch\": " + epoch + ", \"loss\": " + loss + "}");
}writeTaskLog(jobId, taskName, level, message) takes any level string; an
overload adds an extra payload. By the cross-SDK contract, published partials
use level "result" — workers in other processes that publish partials store
them the same way, so they are readable here.
getTaskLogs(jobId) returns every row for the job, oldest first; poll it until
the job is terminal:
This loop assumes a worker is already running the train handler
elsewhere — e.g. flexiq.worker().handle(train, p -> runTraining(p)) .start() in another process or thread. Without one, the job stays
PENDING and the loop polls forever.
String id = flexiq.enqueue(train, config);
int seen = 0;
while (flexiq.getJob(id).map(j -> j.status == JobStatus.PENDING || j.status == JobStatus.RUNNING).orElse(false)) {
List<TaskLog> logs = flexiq.getTaskLogs(id);
for (TaskLog log : logs.subList(seen, logs.size())) {
if ("result".equals(log.level)) {
render(log.message);
}
}
seen = logs.size();
Thread.sleep(200);
}
int result = flexiq.getResult(id, Integer.class).orElseThrow();It works from any process sharing the storage, and a late consumer still reads
every value already published. Each TaskLog carries jobId, taskName,
level, message, extra, and loggedAt (Unix ms).
Partials are stored as task-log rows, so they survive until the log is
purged. Pipe the poll loop straight to an SSE / WebSocket endpoint to push
progress to a browser. For a single scalar 0–100, prefer
setProgress.