Serialization
How task arguments and results are encoded — per-SDK codecs, the cross-SDK CBOR envelope, and what lands in the database.
How task arguments and results are encoded — per-SDK codecs, the cross-SDK CBOR envelope, and what lands in the database.
SmartSerializer, which uses MessagePack with a cloudpickle fallback for lambdas, closures, and complex objects.flexiq uses a pluggable serializer for task arguments and results. The default is JsonSerializer, which emits human-readable JSON bytes.flexiq uses a pluggable serializer for task arguments and results. The default is JsonSerializer, which emits human-readable JSON bytes via Jackson.The Rust SDK has one codec and no way to swap it. Task arguments and results travel in the cross-SDK CBOR envelope — codec tag 0x02, the same bytes every other FlexiQ SDK reads.
Coming from Celery? Celery defaults to JSON; flexiq's Python default is
SmartSerializer(MessagePack with a cloudpickle fallback). The fallback executes code on load, and it's Python-only — useJsonSerializerfor cross-language or untrusted payloads.
from flexiq import Queue, JsonSerializer
# Use JSON for simpler, cross-language payloads
queue = Queue(serializer=JsonSerializer())import { Queue, MsgpackSerializer } from "@byteveda/flexiq";
// Use msgpack for compact, binary payloads
new Queue({ dbPath: "flexiq.db", serializer: new MsgpackSerializer() });import org.byteveda.flexiq.FlexiQ;
import org.byteveda.flexiq.serialization.MsgpackSerializer;
// Use msgpack for compact, binary payloads
FlexiQ queue = FlexiQ.builder()
.sqlite("flexiq.db")
.serializer(new MsgpackSerializer())
.open();An argument is anything that implements Serialize on the producer side and
Deserialize on the worker side:
#[derive(serde::Serialize, serde::Deserialize)]
struct Order {
order_id: String,
amount_cents: i64,
}
#[flexiq::task]
fn charge(order: Order) -> flexiq::Outcome<i64> {
Ok(order.amount_cents)
}Encoding goes through flexiq_core::wire::encode_call, the engine's writer —
contracts/wire-vectors.json pins its output byte for byte. Decoding is this
crate's own ciborium reader, because the engine ships none: a Rust worker is
the first thing that has to read one of these envelopes back.
Four consequences, all load-bearing:
order_id precedes amount_cents
on the wire because it precedes it in the struct. A codec that sorted keys
would still interoperate — and would silently stop auto: idempotency keys
from deduplicating across languages, since the key hashes the encoded bytes.is_human_readable() is false,
matching the reader. A type that branches on it — uuid::Uuid among them —
round-trips because of that, and travels as a byte string; send a String
when the far side is a worker in another language.u64 past i64::MAX has no representation in
the envelope's signed-integer arm, so it is refused at enqueue rather than
wrapped into a negative.Results go out through the same writer. Reading one back is not wrapped:
job.result is Option<Vec<u8>>, the encoded bytes exactly as written.
| Serializer | Format | Best for |
|---|---|---|
SmartSerializer (default) | MessagePack + cloudpickle fallback | Mixed payloads; falls back to pickle for complex objects |
CloudpickleSerializer | Binary (pickle) | Complex Python objects, lambdas, closures |
JsonSerializer | JSON | Simple types, cross-language interop, debugging |
CborSerializer | CBOR (wire-envelope tag 0x02) | Cross-SDK tasks — big integers, datetimes, bytes round-trip losslessly |
| Serializer | Format | Best for |
|---|---|---|
JsonSerializer (default) | JSON | Simple types, cross-language interop, debugging |
MsgpackSerializer | Binary (msgpack) | Compact payloads, smaller storage |
CborSerializer | Binary (CBOR, wire-envelope tag 0x02) | Cross-SDK tasks — big integers, Date, bytes round-trip losslessly |
SignedSerializer | Wraps a codec + HMAC-SHA256 | Tamper detection |
EncryptedSerializer | Wraps a codec + AES-256-GCM | Encrypted, authenticated payloads |
| Serializer | Format | Best for |
|---|---|---|
JsonSerializer (default) | JSON (Jackson) | Simple types, cross-language interop, debugging |
MsgpackSerializer | Binary (msgpack) | Compact payloads, smaller storage |
CborSerializer | Binary (CBOR, wire-envelope tag 0x02) | Cross-SDK tasks — big integers, decimals, bytes round-trip losslessly |
SignedSerializer | Wraps a serializer + HMAC-SHA256 | Tamper detection |
EncryptedSerializer | Wraps a serializer + AES-GCM | Encrypted, authenticated payloads |
One codec, and it is not selectable: the CBOR wire envelope, codec tag 0x02.
There is no serializer type to choose between.
Serializer interface with dumps(obj) / loads(data) methods.Implement the Serializer interface with serialize(value) / deserialize(bytes) methods.Implement the Serializer interface with serialize(value) / deserialize(bytes, type) methods.There is no serializer interface to implement — the payload codec is fixed. A type that needs a representation of its own implements Serialize and Deserialize for itself, and the envelope carries whatever that produces.
payloadresultperiodic_tasks.argsA Rust periodic takes no parameters — cron on a function with an argument is a
compile error — so its periodic_tasks.args row holds the empty argument list.
A durable step's memoized value goes through the same writer a result does, so
a replay reads it back with exactly the codec it was written with.