Resource System
How external dependencies are registered, injected into tasks, and reconstructed on the worker.
How external dependencies are registered, injected into tasks, and reconstructed on the worker.
The resource system injects external dependencies into tasks and runs entirely outside Rust. It is a three-layer PythonNode.jsJava pipeline:
ArgumentInterceptor walks every argument before serialization. CONVERT types become JSON-safe markers, REDIRECT types become DI placeholders, PROXY types are deconstructed, REJECT types raise in strict mode.ResourceRuntime initializes resources at worker startup in topological order, then injects requested ones (via inject= or Inject["name"]inject: or useResource()Resources.use()). Task-scoped resources come from a semaphore pool.ProxyHandlers deconstruct live objects (file handles, HTTP sessions, cloud clients) into a JSON recipe and reconstruct them on the worker. Recipes are optionally HMAC-signed for tamper detection.Layer 1 — Argument Interception: the ArgumentInterceptor walks every argument
before serialization, applying the strategy registered for its type. CONVERT types
are transformed to JSON-safe markers. REDIRECT types are replaced with a DI placeholder.
PROXY types are deconstructed by their handler. REJECT types raise an error in strict
mode.
Layer 2 — Worker Resource Runtime: ResourceRuntime initializes all registered
resources at worker startup in topological dependency order. At task dispatch time it
injects the requested resources (via inject= or Inject["name"] annotation) as
keyword arguments. Task-scoped resources are acquired from a semaphore pool and
returned after the task finishes.
Layer 3 — Resource Proxies: ProxyHandler implementations know how to deconstruct
live objects (file handles, HTTP sessions, cloud clients) into a JSON-serializable
recipe, and how to reconstruct them on the worker before the task function is called.
Recipes are optionally HMAC-signed for tamper detection.
Layer 1 — Registration: resources are registered once with
queue.resource(name, factory, { scope }). A factory may depend on another
resource through the context it receives (ctx.use("other")), so the runtime
builds them in topological dependency order.
Layer 2 — Worker Resource Runtime: at worker startup the runtime initializes
every registered resource. At dispatch time it resolves the ones a task asked
for — declaratively via inject: (received as a trailing deps object) or
imperatively via useResource() inside the handler. The scope
(worker / task / …) decides each resource's lifetime.
Layer 3 — Resource Proxies: handlers deconstruct live objects (file handles, HTTP sessions, cloud clients) into a serializable recipe and reconstruct them on the worker before the handler runs. Recipes are optionally HMAC-signed for tamper detection.
Layer 1 — Registration: resources are registered once with
queue.resource(name, factory) (plus scope and pool-config overloads). A
factory may depend on another resource through the context it receives
(ctx.use("other")), and a cycle fails fast instead of recursing.
Layer 2 — Worker Resource Runtime: each worker leases a ResourceRuntime
that builds resources on first use and disposes them LIFO at shutdown. Handlers
reach them with Resources.use("name") (or generated @Resource bindings); the
scope — worker / thread / task / request / pooled — decides each resource's
lifetime, with pooled instances checked out per task and returned at task end.
Layer 3 — Resource Proxies: ProxyHandler implementations deconstruct live
objects (file handles and similar) into a serializable reference and
reconstruct them before use. References are HMAC-SHA256-signed, with optional
expiry and purpose binding folded into the signature.