# Cancellation (/concepts/cancellation)



Cancellation is the semantic model for operator or system requests that ask a run to stop. Any runtime or operator surface that supports cancellation must preserve audit history and use cooperative stopping instead of pretending it can hard-kill arbitrary user code safely.

Runlane cancellation is durable and cooperative. The cancellation request is written to run history before task code is notified, and user code decides where it is safe to stop. Runlane does not kill JavaScript execution, cancel network calls that were not passed an abort signal, roll back side effects, or erase the run.

Task handlers receive both `context.signal` and `context.isCancellationRequested()`. Pass the signal to cancellable I/O and check it inside long loops or cleanup boundaries.

```ts
const importContacts = task({
  id: 'contacts.import',
  schema: importContactsSchema,
  async run(payload, context) {
    for await (const contact of crm.streamContacts(payload.accountId, { signal: context.signal })) {
      if (context.isCancellationRequested()) {
        return
      }

      await upsertContact(contact, { signal: context.signal })
    }
  },
})

const { run } = await runlane.trigger(importContacts, { accountId: 'acct_123' })

await runlane.runs.cancel(run.id, {
  actor: { type: ActorType.Operator, id: 'ops@example.com' },
  reason: 'operator_requested',
})
```

## Waiting runs [#waiting-runs]

A run that is waiting in `queued`, `scheduled`, `released`, or `retrying` state has no active user code to interrupt. Cancelling it appends `run.cancelled` immediately and moves the run to terminal `cancelled`.

The run remains in append-only history with the actor, reason, trace carrier, and metadata supplied to `runs.cancel()`. Terminal runs cannot be cancelled again or moved back to active state.

## Running runs [#running-runs]

Running runs have an active lease and may already be inside user code, so cancellation happens in two phases:

1. `runs.cancel()` appends `run.cancellation_requested` and the run becomes active `cancellation_requested`.
2. The worker stops the attempt cooperatively. When the attempt returns successfully after observing cancellation, core appends `run.cancelled`.

`cancellation_requested` is not runnable. Workers do not acquire it as fresh work, and delivery recovery does not make it executable again. It remains active only so the current owner can stop cleanly or so maintenance can finalize it after the lease expires.

Same-process cancellation is fast. If the operator API and the running attempt share a runtime instance, an in-memory registry matches the run id, worker id, and lease token, then aborts the task's `context.signal` immediately after the durable cancellation request is appended.

Cross-process cancellation is still durable. A different worker process observes the `cancellation_requested` state through lease monitoring. Once observed, it aborts its local `context.signal` and stops extending the lease so an uncooperative attempt cannot keep the run alive forever.

Repeated `runs.cancel()` calls on a `cancellation_requested` run do not append duplicate cancellation-requested events. They return the current run and re-notify same-process task code when possible.

## Completion after cancellation [#completion-after-cancellation]

Cancellation changes how successful completion is interpreted. If user code returns while the run is `cancellation_requested`, core records terminal `cancelled` instead of `succeeded` or `released`.

Cancellation does not hide failures. If task code throws after seeing `context.signal.aborted`, or if a retryable failure races with cancellation, core records terminal `failed`. A cleanup failure is still a task failure, not a successful cancellation, and retry scheduling is not used once cancellation has been requested.

If maintenance finalizes the run before a cooperative completion append lands, execution returns the stored terminal cancellation when it can prove the stored run is already `cancelled`. Persistence conflicts that cannot be reconciled still surface as framework errors instead of being converted into task failures.

## Maintenance finalization [#maintenance-finalization]

`runlane.tick()` finalizes stale cancellation requests. It scans `cancellation_requested` runs whose current lease has expired, re-reads each run, and appends `run.cancelled` with the system actor. The default tick bound is controlled by the maintenance cancellation-finalization limit.

This is the recovery path for workers that crash, lose their process, ignore the signal, or never reach a cooperative stop point. Maintenance waits for lease expiry so a live owner has a chance to clean up before the system records terminal cancellation.

## Limits [#limits]

Cancellation is not preemption. Code that ignores `context.signal`, CPU-bound loops that never yield, and third-party calls that do not accept an `AbortSignal` may continue running until they return, throw, or the process exits. Runlane can stop renewing the lease and finalize durable state, but it cannot undo side effects already performed by user code.

Worker shutdown uses the same task-context signal but is not the same as an operator cancellation request. Calling `worker.stop()` or aborting the worker signal aborts `context.signal`; if the task returns normally and no durable cancellation was requested, the run can still succeed.

## Invariants [#invariants]

Cancellation is an operator control surface, not a data-erasing shortcut. Cancelling a run must preserve audit history, respect active leases, and avoid mutating terminal runs back into active states.


# Current-Process Execution (/concepts/current-process-execution)



Current-process execution creates a durable run, claims its lease, and executes one attempt in the caller's process. Use it when application code intentionally wants the result of the first attempt now, but still wants Runlane's payload validation, run history, leases, retry/release outcomes, singleton enforcement, and operator visibility.

This is not a sync lane. `runNow()` is a core runtime method over the same storage contract as `trigger()`, workers, and delivered wakeups. Task handlers can still `await` I/O, observe `context.signal`, return JSON output, return `context.release(...)`, throw retryable errors, or be cancelled by operators.

```ts
const run = await runlane.runNow(syncQuickBooksInvoices, { userId: 'user_123' })
```

With explicit creation and lease controls:

```ts
import { ActorType, asId } from '@runlane/core'

const run = await runlane.runNow(
  syncQuickBooksInvoices,
  { userId: 'user_123' },
  {
    actor: { type: ActorType.Operator, id: 'user_123' },
    idempotencyKey: asId<'idempotency_key'>('quickbooks_sync_button_user_123'),
    singletonKey: asId<'singleton_key'>('quickbooks_invoices_user_123'),
    traceCarrier,
  },
)
```

## When To Use It [#when-to-use-it]

Use `runNow()` for request-path or operator-path work where the caller wants immediate durable execution and can afford to wait for one attempt. Common examples are admin buttons, explicit user sync buttons, local development flows, and small workflows where the first attempt should complete before the HTTP response is chosen.

Use `trigger()` when the caller only needs to persist work and return. Use workers, `executeNext()`, `executeDelivery()`, or SQS consumers when another process or provider message owns execution.

`runNow()` executes exactly one attempt. If that attempt returns JSON output, the returned `RunRecord` includes it on `output`. If that attempt returns `context.release(...)`, the run becomes `released`. If it fails retryably with retry budget left, the run becomes `retrying`. Later `tick()` writes a fresh delivery request through the normal maintenance path and requests an outbox row only for transport-delivery lanes. `runNow()` does not loop through retries or releases inline.

Do not use `runNow()` as a way to bypass background execution policy. It still resolves the task through the runtime's authoritative task registry, validates payload before durable creation, resolves queues and creation keys like `trigger()`, requires the selected lane capabilities for idempotency, singleton, and concurrency keys, and validates the persisted payload again before user code runs.

## API [#api]

`runlane.runNow(task, payload?, options?)` accepts the same payload rules as `trigger()`: the payload is validated before durable creation and again before user code runs from the persisted payload.

Creation options:

| Option              | Default                                    | What it controls                                                                                                                          |
| ------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `actor`             | `{ type: ActorType.System }`               | Actor recorded on the initial `run.created` event.                                                                                        |
| `concurrencyKey`    | Task-level `concurrencyKey`                | Per-run bounded-queue partition override.                                                                                                 |
| `idempotencyKey`    | Task-level `idempotencyKey`                | Per-run idempotency owner override. Duplicate active or retained owners return the existing run without executing another inline attempt. |
| `idempotencyKeyTTL` | Task-level `idempotencyKeyTTL`, then `30d` | Retention for the selected idempotency owner. Requires a task or explicit idempotency key.                                                |
| `meta`              | None                                       | JSON object stored on the initial run event and run projection for operator context.                                                      |
| `queue`             | Task queue or runtime default queue        | Per-run queue definition override. Recovered, retried, or released inline runs keep this queue for future workers.                        |
| `runId`             | Generated `run_${uuid}`                    | Caller-supplied run id for external correlation. It should be rare.                                                                       |
| `singletonKey`      | Task-level `singletonKey`                  | Per-run active-resource lock override.                                                                                                    |
| `traceCarrier`      | None                                       | Small string map for trace headers or request correlation.                                                                                |

Execution options:

| Option               | Default                                  | What it controls                                                                                                                                                                                             |
| -------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `leaseDuration`      | `contractDefaults.lease.duration` (`5m`) | Durable ownership window written with the initial inline lease and each heartbeat.                                                                                                                           |
| `heartbeatInterval`  | Half of `leaseDuration`                  | How often core writes `run.lease_heartbeat` while the inline attempt is still running. Must be shorter than `leaseDuration`.                                                                                 |
| `maxAttemptDuration` | Task-level `maxAttemptDuration`          | Maximum wall-clock duration for this attempt. The fixed deadline is written on the lease claim and is not extended by heartbeats.                                                                            |
| `signal`             | None                                     | Local abort signal linked into `TaskContext.signal`. It does not create a durable cancellation request by itself, and an already-aborted signal still allows creation before the handler observes the abort. |
| `workerId`           | Runtime worker id                        | Diagnostic owner recorded on the inline lease. It does not route work.                                                                                                                                       |

`runNow()` returns a `RunRecord`. For a new inline run, the record is returned after the first attempt outcome is durably persisted. If storage idempotency returns an existing owner, no inline attempt runs and the existing active or retained terminal run is returned as-is.

For a new inline attempt, the returned status can be:

| Status      | Meaning                                                                                                                                            |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `succeeded` | The handler completed without returning a release. JSON output, including explicit `null`, appears on `run.output`; void success leaves it absent. |
| `failed`    | The handler failed and no retry is scheduled. Unknown thrown errors are persisted as `ErrorCode.TaskFailed` with a stable public message.          |
| `retrying`  | The handler failed retryably and retry budget remains. A later `tick()` requests delivery.                                                         |
| `released`  | The handler returned `context.release(...)`. A later `tick()` requests delivery when the release is due.                                           |
| `cancelled` | A durable operator cancellation request was observed and the attempt completed cancellation.                                                       |

Validation and configuration errors reject before durable creation. Storage capability errors for selected keys reject before durable creation. If storage, projection, heartbeat, or outcome persistence fails after creation, `runNow()` rejects with a structured `RunlaneError`; it does not convert framework failures into task failures.

Unknown handler errors are caught inside the task-attempt boundary and persisted as `ErrorCode.TaskFailed` with the stable public message `Task failed.` Structured `RunlaneError` failures preserve their code, metadata, and retryability, but user-facing failure messages are still normalized. Non-JSON handler output is also caught inside the task-attempt boundary and persisted as `ErrorCode.TaskOutputInvalid`.

If an inline heartbeat loses lease ownership with `StorageConflict`, core aborts `TaskContext.signal`, abandons the task result, and `runNow()` rejects with `StorageConflict` because it cannot prove the current process still owns the run. Other heartbeat or persistence failures reject for the same reason: ownership or durable outcome is not trustworthy.

## Attempt Deadlines [#attempt-deadlines]

`maxAttemptDuration` bounds one attempt. Configure it on the task definition for the default policy, or pass it to `runNow()`, `executeNext()`, workers, or `executeDelivery()` for a narrower runtime path override.

When a process claims a lease, core computes a fixed `attemptDeadlineAt` from the claim time and writes it on `run.lease_claimed` and the materialized `RunRecord`. Heartbeats extend lease ownership, but they do not move `attemptDeadlineAt`.

At the deadline, the runtime aborts `TaskContext.signal`. If the handler cooperates and returns or throws after the deadline, core persists a task attempt failure with `ErrorCode.TaskTimedOut`. If the process exits, loses the lease, or user code never returns after the attempt has started, maintenance later scans expired running attempts and appends either `run.retry_scheduled` or `run.failed` according to the task retry policy. A claimed lease that never records `run.started` is recovered through normal lease-expiry delivery instead of timeout finalization.

The deadline is about Runlane's durable attempt state, not a JavaScript hard kill. Task code should pass `context.signal` to provider SDKs and check it around long loops so timed-out attempts can close promptly.

## Event Model [#event-model]

The initial inline append writes the created event and the lease claim atomically:

```text
run.created
run.lease_claimed (+ attemptDeadlineAt when configured)
run.started
run.lease_heartbeat...
run.succeeded | run.failed | run.retry_scheduled | run.released | run.cancelled
```

There is no initial `run.delivery_requested` event and no initial outbox row. `runNow()` does not call transport for the first attempt, even when the runtime's trigger dispatch policy is eager.

That atomic lease matters. A storage-polling worker scanning immediately after creation sees an already leased running run, not a queued run it can claim before the caller starts the inline attempt.

If the process dies after `run.created + run.lease_claimed` but before an outcome is persisted, the run remains `running` until the lease expires. A later `tick()` appends `run.delivery_requested`; transport-delivery lanes also create an outbox row. Normal workers or transport consumers recover the run.

Local caller abort and durable operator cancellation are different. Passing `signal` only affects the local task context. If the handler reacts by throwing, that is persisted as a task failure according to the task's retry policy. Durable cancellation comes from the operator API; when the inline attempt observes `cancellation_requested`, core aborts the task context and persists `run.cancelled` if the run is still owned.

## Choosing An Execution Path [#choosing-an-execution-path]

| API                                 | Creates a run | Executes user code               | How it chooses work                                                                              | Use when                                                                   |
| ----------------------------------- | ------------- | -------------------------------- | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------- |
| `runNow(task, payload, options)`    | Yes           | Yes, one current-process attempt | The task and payload supplied by the caller                                                      | The caller wants one durable attempt to start immediately.                 |
| `trigger(task, payload, options)`   | Yes           | No                               | The task and payload supplied by the caller                                                      | The caller wants durable background work and can return after persistence. |
| `executeNext(options)`              | No            | Yes, at most one attempt         | Scans storage for one due run in the allowed queues                                              | A process intentionally owns storage polling for one unit of work.         |
| `worker(options)`                   | No            | Yes, repeated attempts           | Loops over `executeNext()` in poll or drain mode                                                 | A process should continuously or boundedly drain storage-acquired work.    |
| `executeDelivery(message, options)` | No            | Yes, at most one delivered run   | Reads the specific run named by a transport message                                              | A transport already acquired a provider message, such as SQS Lambda.       |
| SQS consumers                       | No            | Through `executeDelivery()`      | Receives provider messages, parses Runlane delivery envelopes, and acknowledges handled messages | AWS SQS owns wakeup delivery and redrive behavior.                         |

Do not implement current-process execution by calling `trigger()` followed by `executeNext()`. That path creates a delivery request, may create an outbox row for transport-delivery lanes, and under concurrency `executeNext()` can execute a different due run. `runNow()` creates the run and its lease in one storage append, then executes that exact run.

## Singleton Example [#singleton-example]

Use the same singleton key for the protected resource everywhere the work can be created. A UI button and a cron path that both sync QuickBooks invoices for one user should resolve the same singleton key.

```ts
import { asId, queue, task } from '@runlane/core'

const quickBooksQueue = queue({
  name: asId<'queue'>('quickbooks'),
  concurrencyLimit: 2,
})

const syncQuickBooksInvoices = task({
  id: asId<'task'>('quickbooks.invoices.sync'),
  queue: quickBooksQueue,
  schema: quickBooksSyncSchema,
  schedule: [
    {
      id: asId<'schedule'>('quickbooks.invoices.hourly.user_123'),
      cron: '0 * * * *',
      timeZone: 'UTC',
      payload: { userId: 'user_123' },
    },
  ],
  singletonKey: (payload) => asId<'singleton_key'>(`quickbooks_invoices_${payload.userId}`),
  async run(payload, context) {
    await syncInvoicesForUser(payload.userId, { signal: context.signal })
  },
})

await runlane.runNow(syncQuickBooksInvoices, { userId: 'user_123' })
```

If a schedule has already created an active invoice sync for `user_123`, the inline UI path fails with the same storage-enforced singleton conflict instead of overlapping the sync. The local lane enforces singleton keys in memory so app tests can exercise this definition, but that does not prove production locking. Production singleton correctness requires a lane whose storage reports and implements `enforcesSingleton`.


# Durable Steps (/concepts/durable-steps)



Durable steps let a task checkpoint the output of a named operation before the whole attempt finishes. Use them when retry, release, timeout, or process loss could otherwise repeat an external side effect such as submitting an AWS Batch job, creating a provider export, starting a media transcode, or opening a payment-session-like operation.

They are per run, not per task. The key `submit-render-job` names one checkpoint inside one run. A different run with the same task and payload gets its own step row.

```ts
import { task } from '@runlane/core'
import * as z from 'zod'

const batchSubmissionSchema = z.object({
  jobId: z.string().min(1),
})

const renderPdf = task({
  id: 'pdf.render',
  schema: renderPdfPayloadSchema,
  async run(payload, context) {
    const submission = await context.step.run(
      'submit-render-job',
      { output: batchSubmissionSchema },
      async ({ token }) => {
        const job = await batch.submitRenderJob({
          clientToken: token,
          documentId: payload.documentId,
          signal: context.signal,
        })

        return { jobId: job.id }
      },
    )

    const job = await batch.describeJob(submission.jobId, { signal: context.signal })

    if (job.status !== 'SUCCEEDED') {
      return context.release('30s', {
        meta: { jobId: submission.jobId },
        reason: 'batch_not_finished',
      })
    }

    await saveRenderedPdf(job)
  },
})
```

On the first attempt, `context.step.run()` executes the callback, validates its output with the supplied Standard Schema-compatible `output` schema, verifies the parsed value is JSON-compatible, and asks storage to complete the step. On later attempts, the same call reads the stored step output and skips the callback.

If the callback throws or returns invalid output, the step is not checkpointed. The attempt follows normal task failure and retry rules, and a later attempt can call the callback again.

The same pattern applies outside external compute. This task starts a provider export once, then polls that export on later attempts:

```ts
const exportSubmissionSchema = z.object({
  exportId: z.string().min(1),
})

const syncInvoices = task({
  id: 'invoices.sync',
  schema: syncInvoicesPayloadSchema,
  async run(payload, context) {
    const exportSubmission = await context.step.run(
      'create-invoice-export',
      { output: exportSubmissionSchema },
      ({ token }) =>
        billingProvider.createInvoiceExport({
          accountId: payload.accountId,
          idempotencyKey: token,
          signal: context.signal,
        }),
    )
    const providerExport = await billingProvider.getInvoiceExport(exportSubmission.exportId, {
      signal: context.signal,
    })

    if (providerExport.status !== 'complete') {
      return context.release('2m', { reason: 'invoice_export_pending' })
    }

    await importInvoiceExport(providerExport.downloadUrl)
  },
})
```

## What To Store [#what-to-store]

Store the smallest stable value needed to continue without repeating the one-time operation:

| Good step output          | Why                                                                                   |
| ------------------------- | ------------------------------------------------------------------------------------- |
| `{ jobId }`               | Lets later attempts poll the existing provider job instead of submitting another one. |
| `{ exportId }`            | Lets later attempts check or download the existing export.                            |
| `{ providerOperationId }` | Lets later attempts reconcile the existing operation with application state.          |

Do not store domain state, provider response bodies, large payloads, secrets, or application records in step output. Domain state belongs in your application database. Release `meta` is operator context, not state. Successful task output is the final run result, not a scratchpad for intermediate provider identifiers.

When the output schema transforms, the callback returns the schema input shape and `context.step.run()` returns the parsed output shape.

## Crash Windows [#crash-windows]

Durable steps narrow the crash window, but they do not make a non-idempotent provider call safe by themselves:

| Window                                                                        | Result                                                                     | Required design                                                                                                          |
| ----------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Process crashes before the callback side effect                               | No step exists, so the callback can run later.                             | Safe.                                                                                                                    |
| Provider accepts the side effect but storage completion crashes before commit | No step exists, so the callback can run again.                             | Pass the step `token` as the provider idempotency token when possible, or keep an application ledger keyed by the token. |
| Storage completes the step but the attempt crashes before release or success  | Later attempts read the step and skip the callback.                        | Safe; continue from stored output.                                                                                       |
| The attempt releases after the step completes                                 | `tick()` later requests delivery and the next attempt reads the same step. | Safe; release is still the control-flow primitive.                                                                       |

The callback receives a deterministic `token` for `environment + runId + stepKey`. Use it as the provider idempotency key or as the key for your own submission ledger. Do not derive behavior by parsing the token.

## Step Keys [#step-keys]

Step keys use the same public id rules as other Runlane keys: non-empty string, no `:`, opaque to storage, and stable across deployments. Name the operation, not the resource:

| Good                     | Avoid                                    |
| ------------------------ | ---------------------------------------- |
| `submit-render-job`      | `submit-render-job-user_123`             |
| `create-provider-export` | `export:${payload.exportId}`             |
| `open-transcode-job`     | Raw provider ids or unbounded user input |

The run already scopes the step by payload, resource, queue, environment, and attempt history. Put resource ids in task payload, application state, singleton keys, or step output as appropriate; do not hide them in the step key.

## Storage And Observability [#storage-and-observability]

Durable steps require storage that reports `durableSteps: true`. The local lane supports them in memory for development and tests. Postgres storage persists them in `runlane_run_steps`. A lane without support fails the attempt with `ErrorCode.CapabilityUnsupported` when task code calls `context.step.run()`.

Storage completes a step only for the currently owned run lease and attempt. Duplicate completion with the same output is idempotent. Duplicate completion with different output is an invariant violation because the same named operation cannot have two durable results in one run.

Runtime observers and durable observation exporters receive `RunlaneObservationType.RunStep` after storage confirms a new step completion. The observation includes environment, run id, step key, step token, attempt, and completion time. It intentionally does not include step output, task payload, release metadata, or provider detail.

## Relationship To Release, Retry, And Output [#relationship-to-release-retry-and-output]

Durable steps are checkpoints, not lifecycle outcomes. A task still returns a value to succeed, throws or returns invalid output to fail, or returns `context.release(...)` / `context.waitForSignal(...)` / `context.waitForRun(...)` to wait.

Use release for business waiting after the checkpointed operation starts. Use retry for failure pressure. Use task output for the final public result of the run. Use step output only for the intermediate pointer needed to avoid repeating the external operation.


# Primitives (/concepts)



Runlane is built from a small set of durable primitives. Every feature, adapter, worker, and operator action is a composition of these ideas.

The core idea is simple: user code is not the durable thing. A run is.

```text
task + payload
  -> durable run
  -> append-only events
  -> materialized run state
  -> worker or delivered wakeup executes one attempt
```

A [task](/concepts/tasks-and-runs) defines user code, a payload schema, and execution policy. A run is one durable execution record for that task and payload. Events record what happened. Storage persists events and the current run state atomically. Transport only wakes workers; it never decides whether a run exists or what state it is in.

## One model, three job shapes [#one-model-three-job-shapes]

Runlane supports three product shapes, but they all create or continue the same run model:

* **Background jobs**: application code triggers durable work that can run outside the request path.
* **Scheduled jobs**: task-colocated schedules materialize durable runs at a time or cadence.
* **Polling jobs**: a task releases its run to continue later without counting the wait as a failure.

See [Job Shapes](/concepts/job-shapes) for how these map to real workloads. The important part is that scheduled work and polling work are not separate execution engines. They create or resume durable runs, and workers execute those runs through the same lifecycle as any other work.

## The normal lifecycle [#the-normal-lifecycle]

A typical background run looks like this:

```text
trigger()
  -> run.created
  -> run.delivery_requested
  -> worker claims lease
  -> task handler runs
  -> run.succeeded | run.failed | run.retry_scheduled | run.released | run.cancelled
```

`runNow()` is the current-process exception. It creates the run, claims the first lease, and executes exactly that run inline. It still writes durable history and uses the same task execution path after the lease is claimed. See [Current-Process Execution](/concepts/current-process-execution).

## Primitive map [#primitive-map]

Read the concepts in nav order when you are learning Runlane for the first time:

* [Job Shapes](/concepts/job-shapes) explains background, scheduled, and polling work.
* [Tasks And Runs](/concepts/tasks-and-runs) defines user code, payload validation, ids, idempotency, singleton keys, concurrency keys, runs, and events.
* [Queues](/concepts/queues) explains provider-neutral routing and durable queue capacity.
* [Workers](/concepts/workers) explains polling workers, drain workers, delivered wakeups, leases, and heartbeats.
* [Current-Process Execution](/concepts/current-process-execution) explains `runNow()` and when inline durable execution is the right path.
* [Schedules](/concepts/schedules) explains once, interval, and cron schedules that materialize ordinary runs.
* [Retry Vs Release](/concepts/retry-vs-release) explains the difference between failure pressure and business waiting.
* [Observability](/concepts/observability) explains live observations and durable observation export.
* [Operator APIs](/concepts/operator-apis) explains run reads, attempts, events, cancellation, rerun, manual retry, idempotency reset, and pruning.
* [Cancellation](/concepts/cancellation) explains cooperative stop requests for queued and running work.
* [Rerun And Manual Retry](/concepts/rerun-and-manual-retry) explains linked recovery runs without mutating terminal history.
* [Pruning](/concepts/pruning) explains retention for old terminal run data.
* [Lanes, Storage, And Transport](/concepts/lanes-storage-and-transport) explains why storage owns truth and transport only publishes wakeups.
* [Local Development](/concepts/local-development) explains the in-memory local lane for development and tests.

## Boundaries that matter [#boundaries-that-matter]

Runlane keeps product behavior in core and durable truth in storage:

```text
@runlane/core
  task registration, triggering, schedules, retry, release, workers, operators

storage adapter
  events, materialized runs, leases, schedules, outbox rows, indexes, pruning

transport adapter
  wakeup publishing and provider acknowledgements
```

This split is why delayed, duplicate, or missing transport messages do not decide run state. A worker or transport consumer always reads the current run from storage before executing. If the run is terminal, not due, on the wrong queue, or already leased, the wakeup is ignored safely.

It also means `tick()` is part of the runtime model. Maintenance materializes due schedules, requests delivery for released or retrying runs, recovers expired leases, finalizes abandoned cancellations, and flushes pending outbox rows.

## Picking the right first page [#picking-the-right-first-page]

If you are defining work, start with [Tasks And Runs](/concepts/tasks-and-runs). If you are deciding how work gets executed, read [Queues](/concepts/queues) and [Workers](/concepts/workers). If you are modeling "try again later," read [Retry Vs Release](/concepts/retry-vs-release) before writing task code. If you are wiring infrastructure, read [Lanes, Storage, And Transport](/concepts/lanes-storage-and-transport) before choosing or authoring adapters.


# Job Shapes (/concepts/job-shapes)



Runlane has three product shapes, but only one execution record: the run.

```text
application request -> trigger() -> run
schedule fire time -> tick() -> run
external submission -> context.step.run() -> step checkpoint
business wait point -> context.release() -> same run continues later
```

The shape changes how a run is created or resumed. It does not change what operators inspect. Background, scheduled, and polling work all move through the same run statuses, event history, leases, queues, retry policy, release policy, cancellation, and operator APIs.

| Shape          | What starts or resumes it                                                        | What executes user code                                                                | Operator view                                                      |
| -------------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| Background job | Application code calls `trigger()` or `runNow()`                                 | `worker()`, `executeNext()`, `executeDelivery()`, or the one inline `runNow()` attempt | One durable run created from a caller request                      |
| Scheduled job  | `tick()` materializes a registered task schedule                                 | The normal worker or delivery path                                                     | One durable run with `source.type = "schedule"`                    |
| Polling job    | A task returns `context.release(...)`; later `tick()` requests delivery when due | The normal worker or delivery path                                                     | The same durable run, with release events instead of failure noise |

## Background jobs [#background-jobs]

A background job starts when application code asks Runlane to do work durably.

```ts
const { run } = await runlane.trigger(sendWelcomeEmail, { userId: 'user_123' })
```

`trigger()` validates the payload, persists a run, records a delivery request, and returns a `TriggerRunResult`. With a transport-delivery lane and the default dispatch policy, it also tries to publish the new wakeup immediately. If dispatch is deferred, or if a retryable transport publish fails, `tick()` can publish pending outbox rows later.

Workers then claim and execute the run. A process can do that by polling storage with `executeNext()` or `worker()`, or by handling a delivered transport message with `executeDelivery()`.

Use background jobs for work that should survive process restarts, be retried on transient failure, and remain visible to operators. Common examples include sending email, syncing external systems, processing media, or updating search indexes.

When the application wants to start one durable attempt immediately in the current process, use `runNow()`.

```ts
const run = await runlane.runNow(syncQuickBooksInvoices, { userId: 'user_123' })
```

`runNow()` is still a background-job run with durable history and leases. It creates the run and claims the first attempt inline, then returns after that one attempt records an outcome. It does not loop through retries or releases inline. See [Current-Process Execution](/concepts/current-process-execution).

## Scheduled jobs [#scheduled-jobs]

A scheduled job starts from a `task({ schedule: ... })` entry instead of an application request.

```ts
const sendDigest = task({
  id: 'digests.send',
  schema: digestPayloadSchema,
  schedule: {
    id: 'digests.daily',
    cron: '0 9 * * *',
    timeZone: 'America/New_York',
    payload: { userId: 'user_123' },
  },
  async run(payload) {
    await sendDigestEmail(payload.userId)
  },
})

await runlane.tick()
```

`tick()` claims due schedule occurrences from the runtime's registered task set. For each due occurrence, it materializes an ordinary durable run and flushes the run's wakeup through the outbox. User task code does not run inside `tick()`. Workers or delivered transport messages execute the generated runs later.

Schedules are not a second execution engine. Operators should inspect generated runs the same way they inspect manually triggered runs.

Runlane supports one-time, interval, and cron schedules. Interval and cron schedules materialize the latest due fire time in a tick, not every missed boundary in one call. See [Schedules](/concepts/schedules) for schedule options and occurrence behavior.

## Polling jobs [#polling-jobs]

A polling job is a durable run that reaches a business wait point and asks to continue later.

```ts
const pollReport = task({
  id: 'reports.poll',
  schema: reportPayloadSchema,
  async run(payload, context) {
    const report = await reports.get(payload.reportId)

    if (report.status === 'processing') {
      return context.release('5m', { reason: 'provider_not_ready' })
    }

    await storeReport(report)
  },
})
```

The run is released with a resume time. When that time is due, `tick()` records a fresh delivery request and publishes a wakeup through the outbox. The next worker attempt loads the same persisted payload and run history, then continues through the normal task lifecycle.

When polling starts a one-time external operation, checkpoint the submission with `context.step.run()` before releasing. The later attempt can read the stored job id, export id, or provider operation id and poll that existing operation instead of submitting a duplicate. See [Durable Steps](/concepts/durable-steps).

Polling jobs should use release semantics, not retry semantics. Waiting for a third-party report, payment, invoice, or export is not the same as failing to execute the task.

Use retry when the attempt failed and should count as failure pressure. Use release when the task made progress but must wait for external business state. See [Retry Vs Release](/concepts/retry-vs-release).


# Lanes, Storage, And Transport (/concepts/lanes-storage-and-transport)



A lane is the infrastructure boundary Runlane uses at runtime. It combines one storage adapter with an explicit delivery mode:

* storage owns durable run truth
* storage leases let polling workers scan and lease due runs directly
* `lane.delivery.mode` selects storage polling or transport-backed wakeups
* transport delivery publishes provider messages that call back into storage-verified execution
* core owns task behavior, reducer semantics, retries, releases, schedules, cancellation, workers, and operator APIs

That split is intentional. A Postgres-backed runtime and an in-memory local runtime should run the same core behavior; only the persistence and acquisition mechanisms should change.

## The Lane Boundary [#the-lane-boundary]

Lane packages use `createLane()` from `@runlane/contracts` to compose compatible adapters:

```ts
import { createLane, LaneDeliveryMode } from '@runlane/contracts'

export function createProductionLane() {
  const storage = postgresStorage({ connectionString, schema: 'runlane' })
  const transport = sqsTransport({ client, queues })

  return createLane({
    delivery: { mode: LaneDeliveryMode.Transport, transport },
    name: 'postgres-sqs',
    productionDurable: true,
    storage,
  })
}
```

`createLane()` validates the adapter shapes and delivery mode, keeps the original adapter instances, exposes storage and lane capability flags through `lane.capabilities`, and standardizes lifecycle order:

1. `lane.start()` starts storage first.
2. `lane.start()` starts the transport second when `lane.delivery.mode` is `transport`.
3. If transport startup fails, storage is closed.
4. `lane.close()` closes the transport first when `lane.delivery.mode` is `transport`.
5. `lane.close()` closes storage second.

It is not a capability shim. It does not make storage durable, add operator reads, provide missing queue semantics, or emulate unsupported transport behavior.

`createLane()` requires an explicit `delivery` mode. It still defaults `operatorReads` to `true`, `productionDurable` to `false`, and `name` to `lane`. Lane packages should override those defaults only when the composed adapters actually provide the advertised behavior. Malformed lane options and incomplete adapters fail fast with `RunlaneError` and `ErrorCode.ConfigurationInvalid`.

## Storage [#storage]

Storage persists append-only run events and the current materialized run state. Core supplies the projected run state when it appends events; storage verifies the command, checks optimistic sequence ownership, and commits the events, projection, storage-owned ownership rows, and any requested derived outbox rows atomically.

Storage also owns the durable concurrency decisions that cannot live in memory in a multi-process system:

| Storage responsibility     | Why it belongs in storage                                                        |
| -------------------------- | -------------------------------------------------------------------------------- |
| Event sequence checks      | Competing writers need one authoritative compare-and-append boundary.            |
| Idempotency ownership      | Concurrent duplicate triggers must converge on one owner.                        |
| Singleton ownership        | Competing runs for the same singleton key must serialize.                        |
| Bounded queue capacity     | Multiple dispatchers and workers can race for the same queue partition.          |
| Run leases and heartbeats  | Workers need durable execution ownership.                                        |
| Durable step completion    | A stale worker must not checkpoint output after losing the active attempt lease. |
| Schedule occurrence claims | Multiple maintenance processes may observe the same due fire time.               |
| Outbox persistence         | Transport wakeups must be recoverable after process or provider failure.         |
| Operator reads and pruning | Run history, summaries, and retention need backend-owned pagination and indexes. |

Storage capabilities describe which of those guarantees an adapter implements. Unsupported feature methods must fail with `ErrorCode.CapabilityUnsupported`; they must not silently return an empty result or a fake success.

Storage-polling workers use storage directly. `executeNext()` and `worker({ mode: 'poll' | 'drain' })` ask storage for due run candidates, then try to claim a lease before executing.

See [Adapter Authoring](../contracts/adapter-authoring.mdx) for the full storage contract and [Postgres Storage](../contracts/postgres-storage.mdx) for the first-party durable implementation.

## Transport [#transport]

Transport publishes minimal wakeups. A wakeup identifies the environment, run id, logical queue, request time, and optional trace context. It must not carry task payloads, run projections, retry policy, schedule state, or operator data.

Duplicate, delayed, or stale wakeups are safe because a worker does not trust the message as truth. It passes the wakeup to `runlane.executeDelivery(message)`, and core re-reads storage before execution. Delivery proceeds only when storage still says that exact run is due for that queue and can be leased. Terminal, not-due, wrong-queue, already-leased, claim-lost, or otherwise stale wakeups are ack-safe ignored results.

Transport-driven execution is different from storage polling:

| Path                          | Acquisition source                      | Use when                                                     |
| ----------------------------- | --------------------------------------- | ------------------------------------------------------------ |
| `executeDelivery(message)`    | A provider-delivered wakeup such as SQS | A transport consumer already received a message for one run. |
| `executeNext()` or `worker()` | Storage scans                           | A process intentionally polls storage for due work.          |

A custom deployment may deliberately run storage-polling workers alongside a transport consumer during migration or as a polling backstop. The lane contract only models the optional external transport; polling is possible when storage supports run leases and workers are configured to poll. Deployment still needs one acquisition owner per logical queue. Polling workers can execute durable runs that also had provider wakeups published, but they cannot acknowledge provider messages. Without a consumer for that provider queue, messages can redrive or land in a provider DLQ.

Do not publish provider messages for a queue and then only run a storage-polling worker for that same queue. The worker may still execute the durable runs, but it will not receive or delete provider messages, so those messages can redrive or land in a provider DLQ.

See [SQS Transport](../contracts/sqs-transport.mdx) for the first-party wakeup transport and its Lambda and long-running consumer helpers.

## Lane Matrix [#lane-matrix]

| Lane                             | Storage                       | Delivery topology                    | Acquisition path                                   | Use when                                                                                                       |
| -------------------------------- | ----------------------------- | ------------------------------------ | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `@runlane/lane-local`            | In-memory process-local state | Storage polling                      | Storage scan in the owning process                 | Development, demos, and application tests.                                                                     |
| `@runlane/lane-postgres-polling` | Durable Postgres state        | Storage polling                      | `worker({ mode: WorkerMode.Poll })` scans Postgres | Production jobs with one operational dependency and long-running worker processes.                             |
| `@runlane/lane-postgres-sqs`     | Durable Postgres state        | SQS wakeups, `durableDelivery: true` | SQS consumers call `executeDelivery(message)`      | AWS event-driven wakeups, Lambda consumers, or workloads where polling pressure should not drive wake latency. |

## Choosing Between Postgres Polling And Postgres/SQS [#choosing-between-postgres-polling-and-postgressqs]

Postgres polling keeps the production surface to Postgres plus worker processes. Postgres/SQS keeps Postgres as durable truth but adds provider wakeups and provider message ownership.

| Choose           | When                                                                                                                                      |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Postgres polling | You want one operational dependency, can keep long-running workers online, and accept database scan-and-claim latency.                    |
| Postgres/SQS     | You want provider-delivered wakeups, Lambda or SQS consumer scale-out, or workloads where polling pressure should not drive wake latency. |

The migration path keeps task definitions, Postgres storage semantics, and operator APIs stable:

1. Add SQS queues and bind each Runlane queue with `sqsQueue()`.
2. Switch the lane factory from `postgresPollingLane({ connectionString, schema })` to `postgresSqsLane({ postgres: { connectionString, schema }, sqs })`.
3. Replace polling workers for those queues with SQS consumers that call `runlane.executeDelivery(message)`.
4. Keep `tick()` running as separate maintenance infrastructure.

Do not run SQS consumers and polling workers for the same logical queue unless the migration intentionally accepts duplicate acquisition attempts. Runlane storage leases protect durable execution, but provider messages still need one clear owner.

## Local Lane [#local-lane]

`@runlane/lane-local` provides `createLocalLane()` for development, examples, and tests. It composes `createLocalStorage()` from `@runlane/local-adapters` without an external transport.

The local lane is real Runlane behavior over process-local memory:

* run state, event history, durable step completions, observation records, exporter checkpoints, leases, schedule occurrences, idempotency owners, singleton owners, and queue capacity state live in memory
* local storage reports `processLocalState: true` and `durableState: false`
* `lane.delivery.mode` is `storage_polling`
* the composed lane reports `productionDurable: false`

Use it when you want a complete local backend without Postgres, SQS, or cloud credentials. Do not use it to prove production crash recovery, database isolation, provider delivery, or cross-process state sharing. A second process that constructs `createLocalLane()` gets a different in-memory store unless it talks to the process that owns the local runtime.

## Postgres Storage [#postgres-storage]

`@runlane/postgres-storage` is the first-party durable storage adapter. It stores run history, materialized run state, durable step completions, observation records, exporter checkpoints, leases, schedule occurrences, idempotency and singleton ownership, bounded queue reservations, outbox rows, operator read state, and pruning state in Postgres.

It reports:

* `durableState: true`
* `durableSteps: true`
* `exportsObservations: true`
* `processLocalState: false`
* idempotency, singleton, queue concurrency, leasing, schedule claims, outbox persistence, run-history reads, and pruning support as enabled

`postgresStorage({ connectionString, schema })` resolves the schema from the explicit `schema` option, then from a Prisma-style `?schema=` query parameter, then from `public`. It removes that query parameter before creating the `pg` pool. Runlane does not auto-migrate at adapter startup; apply `getPostgresStorageMigrationSql()` through your normal migration system before starting workers. `postgresStorage().start()` probes the migrated runs table so missing schema or migration problems fail during `lane.start()`.

## Postgres Polling Lane [#postgres-polling-lane]

`@runlane/lane-postgres-polling` is the first-party production Postgres-only composition:

1. Validate the lane-owned options: required `connectionString`, optional `schema`, optional `name`.
2. Create Postgres storage from those options.
3. Return `createLane({ name: 'postgres-polling', productionDurable: true, storage, delivery: { mode: LaneDeliveryMode.StoragePolling } })`.

There is no fake polling transport. Core records delivery intent in Postgres run state and workers acquire due work by polling storage. Because `lane.delivery.mode` is `storage_polling`, core does not create or publish delivery outbox rows for this lane.

This is the default production path when teams want durable jobs, schedules, retries, releases, idempotency, singleton enforcement, operator reads, and worker processes backed by Postgres only. Run maintenance with `tick()` and run workers with `worker({ mode: WorkerMode.Poll })`. If all workers are down, no external system wakes work; due runs remain durable in Postgres until a polling worker starts.

See [Postgres Polling Lane](../contracts/postgres-polling-lane.mdx) for setup, worker, and maintenance guidance.

## SQS Transport [#sqs-transport]

`@runlane/transport-sqs` is the first-party production wakeup transport. It maps logical Runlane queues to SQS queues with `sqsQueue(queueDefinition, options)`, then publishes one SQS message per durable outbox row.

The SQS message body is a versioned `runlane.wakeup` envelope containing only the delivery message. The adapter may batch provider calls with `SendMessageBatch`, but it does not bundle multiple runs into one JSON body. `publishWakeups()` returns one indexed outcome for each attempted wakeup so core can mark each claimed outbox row published, failed, or dead-lettered.

Standard SQS queues are the default recommendation because Runlane correctness comes from storage leases and durable run state, not provider ordering. FIFO queues are supported through queue-level `fifo` options; the adapter reports `messageGrouping: true` and `orderedDelivery: true` only when every configured queue is FIFO. SQS native delay is not used for schedules, retries, releases, or outbox recovery; storage decides when work is due.

An SQS-consuming deployment should use one acquisition path per logical queue:

* Lambda with `createSqsLambdaHandler()`
* a long-running process with `createSqsDeliveryConsumer()`
* another deliberate SQS consumer that parses wakeups and calls `executeDelivery()`

## Postgres/SQS Lane [#postgressqs-lane]

`@runlane/lane-postgres-sqs` is the reference production composition:

1. Validate the lane-owned options: optional `name`, required `postgres`, required `sqs`.
2. Create SQS transport from the nested `sqs` options.
3. Create Postgres storage from the nested `postgres` options.
4. Return `createLane({ name: 'postgres-sqs', productionDurable: true, storage, delivery: { mode: LaneDeliveryMode.Transport, transport } })`.

The package does not import core and does not add runtime semantics. It does not implement worker loops, Lambda behavior, retries, releases, schedules, storage polling, or outbox recovery. Those remain core and transport-helper responsibilities.

In the normal production flow:

1. `trigger()` creates or finds a durable run in Postgres.
2. Core appends `run.delivery_requested` when delivery should be attempted.
3. Postgres writes the event, projected run, and outbox row atomically.
4. Eager trigger dispatch may claim and publish the just-created outbox row.
5. SQS receives one wakeup message for that outbox row.
6. A Lambda or long-running SQS consumer parses the wakeup and calls `runlane.executeDelivery(message)`.
7. Core re-reads Postgres, claims a run lease if the run is still executable for the delivered queue, and records the attempt outcome.
8. `tick()` handles recovery and maintenance: deferred outbox rows, failed eager publishes, due schedules, due releases and retries, expired leases, cancellation finalization, and stranded outbox rows.

Run `tick()` from separate maintenance infrastructure such as EventBridge, cron, a sidecar, or an operator command. Do not add a global `tick()` pass to every SQS delivery batch handler.

See [Postgres SQS Lane](../contracts/postgres-sqs-lane.mdx) for configuration and deployment details.

## Lanes Versus Queues [#lanes-versus-queues]

Use queues for workload routing inside one runtime boundary. Examples: `default`, `emails`, `billing`, `imports`, `media`, or `high-priority`. Queues decide which workers may claim work and where transport wakeups are published; they do not create a separate durable truth boundary.

Use separate lanes only when the infrastructure boundary is actually different:

| Choose        | When                                                                                                                                 |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Another queue | Same storage, same transport family, same environment, same durability policy, but different routing, concurrency, or worker fleet.  |
| Another lane  | Different database, schema, AWS account, region, tenant boundary, compliance boundary, durability policy, or transport/storage pair. |

Splitting lanes when a queue would do makes operator views, maintenance, migrations, and recovery harder. Overloading queues when the storage boundary is truly different makes isolation and failure handling unclear.


# Local Development (/concepts/local-development)



`@runlane/lane-local` is the local development backend for Runlane. Use it with `createRunlane()` when you want the full runtime experience in one process:

```ts
import { createRunlane, queue } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'

const emailQueue = queue({ name: 'emails', default: true })

const runlane = createRunlane({
  lane: createLocalLane(),
  queues: [emailQueue],
  tasks: { sendEmail },
})

await runlane.trigger(runlane.tasks.sendEmail, { userId: 'user_123' })
await runlane.executeNext()
```

The local lane stores runs, events, leases, durable step completions, observation records, exporter checkpoints, schedule occurrences, bounded-queue reservations, idempotency and singleton ownership in memory. It uses storage polling for delivery, so application tests exercise real Runlane behavior without a database, queue, or cloud account. It reports `processLocalState: true` and `productionDurable: false`; process exit loses all local state.

Process boundaries are real. A `runlane dev` process and a separate process that each construct `createLocalLane()` get different in-memory stores. The CLI handles the common two-terminal workflow by having the long-running `runlane dev` process own a loopback control server when the runtime lane reports `storage.capabilities.processLocalState: true`; stateful commands such as `runlane trigger`, `runlane tick`, `runlane runs list`, `runlane runs get`, `runlane retry`, `runlane rerun`, `runlane cancel`, and `runlane prune` proxy to the matching dev session, so they run inside the process that owns the local lane. Other application processes still need to call into the same process or use a shared lane when they need cross-process state.

`createLocalLane()` options:

| Option | Required | Default | What it controls                                                                        |
| ------ | -------- | ------- | --------------------------------------------------------------------------------------- |
| `name` | No       | `local` | Lane diagnostic name surfaced through the lane contract. It must be a non-empty string. |

## What It Provides [#what-it-provides]

From a user's point of view, `createLocalLane()` should feel like a complete local Runlane backend. You can trigger tasks, run `runNow()` for current-process execution, run `executeNext()`, start local workers, call `tick()` for schedules, delivery requests, and cancellation finalization, and inspect or control runs through `runlane.runs`.

Internally, the split stays simple:

```text
@runlane/core
  owns trigger, workers, schedules, retry, release, cancellation, and operator APIs

@runlane/local-adapters
  owns in-memory storage and optional in-memory wakeup publishing for adapter tests

@runlane/lane-local
  composes local storage without an external transport
```

That separation is the point. The same core runtime semantics run over local memory today and production storage or transport resources later.

The storage side is intentionally more realistic than a mock: it persists append-only run history, projected run records, durable step completions, observation records and checkpoints, leases, schedule occurrence claims, keyset pagination cursors, idempotency and singleton ownership, queue-capacity reservations, pruning, and storage outbox rows when a test explicitly exercises transport wakeup persistence. The ready-to-use local lane uses `lane.delivery.mode: 'storage_polling'`.

## CLI Development Loop [#cli-development-loop]

Use `runlane dev` when you want the local runtime to behave like a running backend process:

```sh
# Terminal 1
pnpm exec runlane dev

# Terminal 2, using the same resolved Runlane config path
pnpm exec runlane trigger emails.welcome '{"userId":"user_123"}'
pnpm exec runlane runs list
pnpm exec runlane runs get <runId> --events
```

The long-running dev command starts a polling worker and a maintenance loop. The maintenance loop calls `tick()` on the configured interval, so due schedules are materialized, deferred delivery requests are appended, and expired cancellations are finalized. When the lane is process-local, the dev command also writes a temporary session file keyed by the resolved config path and accepts authenticated loopback requests from matching stateful CLI commands.

`runlane dev --once` is different: it runs one maintenance pass, starts a drain worker, waits for due work to finish, and exits. Use it for scripts and tests that need deterministic local execution. Because it exits after the one-shot pass, it does not provide a long-lived bridge for a second terminal.

## Testing Applications [#testing-applications]

Prefer testing your app against a real local runtime instead of mocking Runlane. Mock your own side-effect dependencies, such as email providers, payment clients, or HTTP clients, and let Runlane create and execute real runs:

```ts
import { createRunlane, queue, task, WorkerMode } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'
import * as z from 'zod'

const sentEmails: string[] = []
const emailQueue = queue({ name: 'emails', default: true })

const sendWelcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema: z.object({ userId: z.string() }),
  async run(payload) {
    sentEmails.push(payload.userId)
  },
})

const runlane = createRunlane({
  lane: createLocalLane(),
  queues: [emailQueue],
  tasks: { sendWelcomeEmail },
})

await runlane.trigger(runlane.tasks.sendWelcomeEmail, { userId: 'user_123' })

const worker = runlane.worker({ mode: WorkerMode.Drain })
await worker.closed
```

Use `WorkerMode.Drain` in tests when you want deterministic worker execution. Drain mode processes currently due work and exits; it does not sleep or keep a polling loop alive. Use `executeNext()` when a test should execute exactly one available run.

For pure unit tests, put business logic behind ordinary functions and test those directly. Use the local lane for integration tests where you care about payload validation, durable run state, retry/release behavior, schedule materialization, worker acquisition, cancellation, or operator-visible history.

The local lane integration tests in Runlane exercise the same path application tests should use: `createRunlane({ lane: createLocalLane(), ... })`, then public runtime calls such as `trigger()`, `executeNext()`, `worker({ mode: WorkerMode.Drain })`, `tick()`, and `runlane.runs.*`. Adapter-level tests live lower, in `@runlane/local-adapters`, and are for proving the storage and optional transport contracts directly.

## Local Limits [#local-limits]

Local state is process memory, not production durability.

| Local behavior                                                                         | What it proves                                                         | What it does not prove                                             |
| -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------ |
| Runs, events, durable steps, observation records, leases, and schedules live in memory | Runtime behavior and task-contract drift inside one process            | Crash recovery or restart durability                               |
| Idempotency, singleton, and bounded queue policy run in memory                         | Application definitions use the right Runlane primitives               | Production locking or transaction isolation                        |
| `runlane dev` proxies stateful commands into a matching process-local dev session      | Two-terminal local workflows with the same config path                 | Cross-process shared storage outside the dev bridge                |
| Local lane uses storage polling                                                        | Worker acquisition and maintenance behavior run without provider setup | Provider delivery guarantees, ordering, grouping, or native delays |
| Pruning works against local history                                                    | Operator retention flows call the public API correctly                 | Production indexes, foreign keys, table bloat, or compaction cost  |

Use production adapter tests and storage conformance to prove database constraints, provider error mapping, migrations, and queue-specific delivery behavior.

## Conformance [#conformance]

Conformance is Runlane's shared contract test vocabulary:

```text
storage conformance
  proves a storage adapter obeys the run truth contract

transport conformance
  proves a transport adapter obeys the wakeup publish contract

lane composition conformance
  proves a lane wires compatible storage and delivery mode correctly
```

`@runlane/local-adapters` runs storage and transport conformance. `@runlane/lane-local` runs lane composition conformance and integration tests over `@runlane/core` to prove the local runtime experience. The local lane should be product-complete locally without becoming a second runtime implementation.


# Observability (/concepts/observability)



Runlane has three visibility surfaces:

* durable run history for audit, recovery, and operator workflows
* live observers for cheap in-process logs, counters, tests, and local handoff
* durable observation export for scalable out-of-band telemetry delivery

Use run history when correctness matters. Use live observers when the runtime process can do the work quickly after a durable write. Use durable export when telemetry provider latency, retries, or backpressure must stay outside task execution.

## Live Observers [#live-observers]

Configure observers on the runtime:

```ts
import { createRunlane, RunlaneObservationType } from '@runlane/core'

const runlane = createRunlane({
  lane,
  queues,
  tasks: { sendEmail },
  observability: {
    observers: [
      (observation) => {
        if (observation.type !== RunlaneObservationType.RunEvent) {
          return
        }

        metrics.increment('runlane.run_event', {
          environment: observation.environment.name,
          eventType: observation.eventType,
        })
      },
    ],
    onObserverError({ cause, observation }) {
      logger.warn({ cause, observation }, 'Runlane observer failed')
    },
  },
})
```

Observers run after storage confirms the event or step completion was persisted. A failed storage write does not emit an observation. A duplicate or stale race that storage rejects also does not emit an observation.

Observer failures do not roll back run state. Core catches observer rejections and reports them through `observability.onObserverError` when provided. If no error handler is configured, observer failures are ignored after the durable write.

Observers are live hooks, so slow observers add latency after the storage write that triggered them. Keep raw observers fast: increment a counter, write a local log line, enqueue to an application-owned bounded buffer, or hand off to an already-running telemetry loop. Do not perform one network export per observation in the observer callback.

## Durable Observation Export [#durable-observation-export]

Durable export is a storage-owned stream over committed Runlane facts. Storage writes an observation record in the same durable operation as the run event or durable step completion. Runtime execution does not call telemetry backends, exporter loops, or provider SDKs.

Run an exporter in a separate process:

```ts
import { createRunlaneObservationExporter } from '@runlane/observability'
import { otlpObservationSink } from '@runlane/observability/opentelemetry'

const exporter = createRunlaneObservationExporter({
  consumer: 'production_otel',
  environment: { name: 'production' },
  storage,
  sink: otlpObservationSink({
    endpoint: 'http://otel-collector:4318/v1/logs',
    timeout: '10s',
  }),
})

await exporter.start({ signal })
```

The exporter reads the last checkpoint for one `consumer`, scans the next batch of observation records, passes that batch to the sink, and advances the checkpoint only after the sink accepts the batch. If the process crashes or the sink rejects, the next run resumes from the previous checkpoint.

Delivery is at least once. Sinks must tolerate duplicate delivery by `record.id` when duplicate external telemetry matters. The same durable run event or step completion maps to the same observation record id across exporter restarts.

## Console Export [#console-export]

Use the console sink for local inspection, tests, or piping into a process supervisor:

```ts
import { consoleObservationSink, createRunlaneObservationExporter } from '@runlane/observability'

const exporter = createRunlaneObservationExporter({
  batchSize: 100,
  consumer: 'local_console',
  environment: { name: 'development' },
  storage,
  sink: consoleObservationSink(),
})

const result = await exporter.exportOnce()
```

`exportOnce()` performs one scan/export/checkpoint cycle and returns a structured tick result with current stats. Failed ticks include the original error so callers can inspect `RunlaneError` codes such as `ObservationExportFailed` or `CapabilityUnsupported`. `start()` keeps polling until its `AbortSignal` is aborted and rejects non-retryable `RunlaneError` failures instead of retrying bad configuration forever.

## OTLP To Collector [#otlp-to-collector]

Runlane's OTLP sink uses the official OpenTelemetry JavaScript OTLP/HTTP log exporter to send sanitized observation records to an OpenTelemetry Collector endpoint. Treat the Collector as the vendor boundary: it should receive, process, batch, retry, and export telemetry to Datadog, Honeycomb, Grafana, or another backend.

```ts
import { createRunlaneObservationExporter } from '@runlane/observability'
import { otlpObservationSink } from '@runlane/observability/opentelemetry'

const exporter = createRunlaneObservationExporter({
  batchSize: 500,
  consumer: 'production_otel',
  environment: { name: 'production' },
  idlePollInterval: '1s',
  retryDelay: '5s',
  storage,
  sink: otlpObservationSink({
    endpoint: process.env.OTLP_ENDPOINT ?? 'http://localhost:4318/v1/logs',
    headers: { 'x-runlane-source': 'production' },
    resourceAttributes: { 'service.namespace': 'jobs' },
  }),
})
```

Point the sink at the Collector logs endpoint, normally `/v1/logs`. Do not import OpenTelemetry SDKs into `@runlane/core` or runtime task execution to make export work.

If you need a different OpenTelemetry transport or a preconfigured exporter, adapt it with `openTelemetryObservationSink()` instead of rebuilding Runlane's observation-to-log-record mapping:

```ts
import { OTLPLogExporter } from '@opentelemetry/exporter-logs-otlp-http'
import { openTelemetryObservationSink } from '@runlane/observability/opentelemetry'

const sink = openTelemetryObservationSink({
  exporter: new OTLPLogExporter({ url: 'http://localhost:4318/v1/logs' }),
  resourceAttributes: { 'service.namespace': 'jobs' },
})
```

## Stream Semantics [#stream-semantics]

The storage stream is scoped by `environment` and ordered by a storage-assigned stream position. Observation records include:

| Field            | Meaning                                                           |
| ---------------- | ----------------------------------------------------------------- |
| `id`             | Stable duplicate-safe observation record id.                      |
| `streamPosition` | Storage-owned opaque string position used for forward cursors.    |
| `sourceKind`     | Durable fact kind: `run_event` or `run_step`.                     |
| `sourceId`       | The run event id or run step token that produced the observation. |
| `recordedAt`     | When the source fact occurred or completed.                       |
| `observation`    | Sanitized telemetry payload.                                      |

`scanObservationRecords({ environment, cursor, limit })` returns records after the supplied cursor in stable forward order. The returned `cursor` is the checkpoint value after the final returned record; store it only after the sink accepts the batch.

`getObservationCheckpoint()` reads the current checkpoint for one named consumer. `advanceObservationCheckpoint()` requires the expected previous cursor, only accepts cursors that reference a real stream record in the same environment, and rejects stale, missing-record, or non-monotonic advances with `StorageConflictKind.ObservationCheckpoint`.

## Observation Shape [#observation-shape]

The current observation types are:

| Type                                            | Emits after                                 | Fields                                                                                                  |
| ----------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `RunlaneObservationType.RunEvent` (`run_event`) | A durable run event is persisted.           | `environment`, `runId`, `eventId`, `eventSequence`, `eventType`, `occurredAt`, optional `traceCarrier`. |
| `RunlaneObservationType.RunStep` (`run_step`)   | A new durable step completion is persisted. | `environment`, `runId`, `stepKey`, `stepToken`, `attempt`, `occurredAt`.                                |

Observations intentionally do not include task payload, successful output, durable step output, failure detail, release metadata, or operator-expanded storage rows. Those values can contain customer data and can be large. Fetch durable run state through operator APIs only when an operator workflow actually needs detail.

## High-Scale Topology [#high-scale-topology]

The default exporter polls storage. That is enough for many deployments because task execution writes to primary storage while exporter reads happen out of band.

For higher volume:

* Run exporters from a read replica when replica lag is acceptable. Checkpoints still write to primary storage.
* Use one active exporter loop per `(environment, consumer)` checkpoint. Multiple active loops with the same consumer race by design and one loses the checkpoint advance.
* Give independent sinks independent consumer ids, such as `production_otel` and `production_audit_index`.
* When polling storage is no longer enough, stream the observation table through CDC or a message queue and preserve the same stable record ids and checkpoint semantics at the downstream boundary.

Primary storage writes should not wait on telemetry provider reads, Collector outages, vendor retries, or exporter backpressure. If storage itself is overloaded, that is a storage/runtime capacity issue, not a telemetry provider dependency.

## Footguns [#footguns]

Do not tag metrics with run ids:

```ts
if (observation.type !== RunlaneObservationType.RunEvent) {
  return
}

metrics.increment('runlane.run_event', {
  eventType: observation.eventType,
  runId: observation.runId,
})
```

`runId` is high-cardinality. It belongs in logs, traces, and operator links, not metric labels. Prefer low-cardinality dimensions such as `environment` and `eventType`.

Do not put provider I/O in the observer:

```ts
const runlane = createRunlane({
  lane,
  observability: {
    observers: [
      async (observation) => {
        await fetch('https://telemetry.example.com/events', {
          body: JSON.stringify(observation),
          method: 'POST',
        })
      },
    ],
  },
})
```

That makes every observed durable write wait on the telemetry provider. During a provider outage, task execution, maintenance, signal resume, and delivery paths all inherit that latency.

Do not use unbounded buffers:

```ts
import type { RunlaneObservation } from '@runlane/core'

const pending: RunlaneObservation[] = []

const runlane = createRunlane({
  lane,
  observability: {
    observers: [
      (observation) => {
        pending.push(observation)
      },
    ],
  },
})
```

An unbounded array only moves the outage from latency to memory growth. If the process emits observations faster than the exporter drains them, the process can run out of memory while run history remains healthy.

Do not fetch run details from inside the observer by default:

```ts
const runlane = createRunlane({
  lane,
  observability: {
    observers: [
      async (observation) => {
        const run = await runlane.runs.get(observation.runId)
        if (run !== undefined) {
          auditLog.write({ observation, output: run.output })
        }
      },
    ],
  },
})
```

That adds a read for every event, can leak payload/output/failure/step detail into telemetry, and turns a narrow signal into a broad data export path.

Do not drive workflow behavior from observers or exporters. Observability is telemetry over facts that are already durable. Use task handlers, `context.waitForRun(...)`, `context.waitForSignal(...)`, schedules, operator APIs, and storage-backed run state for behavior that must be correct.

Do not assume export is exactly once. Runlane provides stable records and at-least-once delivery; the sink owns dedupe if the external system needs it.


# Operator APIs (/concepts/operator-apis)



Operator APIs work on runs, not task definitions. They are the runtime surface for production maintenance tools, CLIs, dashboards, and support paths that need to inspect durable history or make an auditable control-plane change.

The core surface lives under `runlane.runs`:

```ts
const run = await runlane.runs.get(runId)
const page = await runlane.runs.list({ limit: 50, statuses: [RunStatus.Failed] })
const activeSync = await runlane.runs.findActive({ task: syncInvoices, singletonKey })
const currentRequest = await runlane.runs.findCurrent({ task: syncInvoices, idempotencyKey })
const attempts = await runlane.runs.attempts(runId)
const events = await runlane.runs.events(runId, {
  sortBy: RunEventSortField.Sequence,
  sortDirection: SortDirection.Asc,
})
```

Reads are scoped to the runtime environment and use adapter-owned cursors. Core validates filters and passes them to storage; storage owns efficient indexes and cursor encoding.

The read APIs require both `lane.capabilities.operatorReads` and `storage.capabilities.readsRunHistory`:

* `get()`
* `list()`
* `events()`
* `attempts()`
* `findActive()`
* `findCurrent()`
* read-backed control actions: `cancel()`, `rerun()`, and `retry()`

Core must inspect the current run before returning data or appending an operator-controlled change.

`findActive()` is an advisory UX helper over run filters. It requires a task plus an idempotency key or singleton key and returns an active run when one currently matches. It can race and should not be used as the correctness boundary; `trigger()` plus storage idempotency or singleton ownership is the correctness boundary.

`findCurrent()` is task-scoped idempotency lookup. It returns the active owner or the retained terminal owner for one idempotency key, following the same retention rules as duplicate `trigger()` calls.

`runlane.idempotencyKeys.reset(task, { key })` is the idempotency escape hatch, not a force-run option. It requires `storage.capabilities.enforcesIdempotency`, clears one retained terminal owner for the runtime environment and task id, and rejects active owners.

`attempts(runId)` returns derived attempt summaries for dashboards and support tools. Raw `events(runId)` remains canonical history. `RunRecord.output` exposes the current visible successful output, `RunSummary` intentionally omits output for compact list rows, and `RunSummary.failure` / `RunRecord.failure` expose current visible failure state. `RunSummary.wait` and `RunRecord.wait` expose the current business-wait condition for released runs. `attempts()` shows previous success output, retry, release, and failure outcomes without embedding attempt history into every run projection.

## Read Options [#read-options]

`runlane.runs.list(options)`:

| Option           | Default                                    | What it filters or controls                                                                                            |
| ---------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
| `cursor`         | None                                       | Continues a previous page from storage.                                                                                |
| `limit`          | `contractDefaults.pagination.defaultLimit` | Maximum summaries to return, capped by storage/default pagination policy.                                              |
| `statuses`       | All statuses                               | Durable run statuses to include.                                                                                       |
| `taskIds`        | All tasks                                  | Task ids to include.                                                                                                   |
| `queues`         | All queues                                 | Queue definitions to include; core resolves each against `runtime.queues`.                                             |
| `idempotencyKey` | None                                       | Runs with this task-scoped idempotency key.                                                                            |
| `singletonKey`   | None                                       | Runs with this singleton key.                                                                                          |
| `sourceRunId`    | None                                       | Linked children created from a source run, including operator reruns, manual retries, and task-context child triggers. |
| `createdAt`      | None                                       | `{ from?, to? }` inclusive created-time bounds.                                                                        |
| `updatedAt`      | None                                       | `{ from?, to? }` inclusive updated-time bounds.                                                                        |
| `runAt`          | None                                       | `{ from?, to? }` inclusive due-time bounds.                                                                            |
| `sortBy`         | `contractDefaults.sort.runs.field`         | `created_at`, `updated_at`, or `run_at`.                                                                               |
| `sortDirection`  | `contractDefaults.sort.runs.direction`     | `asc` or `desc`.                                                                                                       |

`runlane.runs.events(runId, options)`:

| Option          | Default                                     | What it filters or controls                                                 |
| --------------- | ------------------------------------------- | --------------------------------------------------------------------------- |
| `cursor`        | None                                        | Continues a previous event page.                                            |
| `limit`         | `contractDefaults.pagination.defaultLimit`  | Maximum events to return.                                                   |
| `types`         | All event types                             | Event types to include.                                                     |
| `occurredAt`    | None                                        | `{ from?, to? }` inclusive event-time bounds.                               |
| `sortBy`        | `contractDefaults.sort.runEvents.field`     | `occurred_at` or `sequence`. Sequence sorting is meaningful inside one run. |
| `sortDirection` | `contractDefaults.sort.runEvents.direction` | `asc` or `desc`.                                                            |

`findActive()` and `findCurrent()`:

| Method                                                              | Required options             | Meaning                                                                                       |
| ------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------- |
| `runlane.runs.findActive({ task, idempotencyKey?, singletonKey? })` | `task` plus at least one key | Advisory active-run lookup for UX. Correctness still belongs to storage-enforced `trigger()`. |
| `runlane.runs.findCurrent({ task, idempotencyKey })`                | `task`, `idempotencyKey`     | Reads the current active or retained terminal idempotency owner.                              |
| `runlane.idempotencyKeys.reset(task, { key })`                      | `key`                        | Clears one retained terminal idempotency owner. Active owners reject.                         |

## Action Options [#action-options]

Operator actions share audit fields:

| Option         | Default                                                     | Used by                             | What it records                                                 |
| -------------- | ----------------------------------------------------------- | ----------------------------------- | --------------------------------------------------------------- |
| `actor`        | `contractDefaults.actor.operator` where applicable          | `cancel`, `rerun`, `retry`, `prune` | Operator identity recorded on audit events or prune commands.   |
| `meta`         | None                                                        | `cancel`, `rerun`, `retry`          | JSON object with action context.                                |
| `traceCarrier` | None for `cancel`; source run trace carrier for linked runs | `cancel`, `rerun`, `retry`          | Trace context carried into linked run creation or audit events. |

Action-specific options:

| Method                   | Option      | Default                               | What it controls                                                     |
| ------------------------ | ----------- | ------------------------------------- | -------------------------------------------------------------------- |
| `cancel(runId, options)` | `reason`    | None                                  | Public cancellation reason stored on the cancellation event.         |
| `rerun(runId, options)`  | `queue`     | Source run queue                      | Queue definition for the linked child run.                           |
| `rerun(runId, options)`  | `runId`     | Generated child run id                | Caller-supplied id for the linked child.                             |
| `retry(runId, options)`  | `queue`     | Source run queue                      | Queue definition for the manual-retry child run.                     |
| `retry(runId, options)`  | `runId`     | Generated child run id                | Caller-supplied id for the manual-retry child.                       |
| `prune(options)`         | `olderThan` | Required                              | Duration string relative to runtime clock or explicit `Date` cutoff. |
| `prune(options)`         | `statuses`  | All terminal statuses                 | Terminal statuses to prune. Active statuses are rejected.            |
| `prune(options)`         | `limit`     | `contractDefaults.pruning.batchLimit` | Maximum terminal runs storage prunes in one call.                    |
| `prune(options)`         | `cursor`    | None                                  | Continues a previous prune with the same frozen retention scope.     |

## Cancellation [#cancellation]

`runlane.runs.cancel(runId, options)` preserves history.

| Run state                                | Cancellation behavior                                                                  |
| ---------------------------------------- | -------------------------------------------------------------------------------------- |
| Queued, released, retrying, or scheduled | The run becomes terminally cancelled immediately.                                      |
| Running in the same runtime              | Core writes `run.cancellation_requested`, then aborts the active task context signal.  |
| Running in another runtime               | The owner observes the request during lease monitoring and stops extending the lease.  |
| Owner died or never cooperates           | `tick()` records terminal cancellation after the cancellation-requested lease expires. |

If cooperative task code stops cleanly, core records `run.cancelled`. If task code throws during cancellation or cleanup, the attempt remains a failure.

```ts
await runlane.runs.cancel(runId, {
  actor: { type: ActorType.Operator, id: 'ops@example.com' },
  reason: 'operator_requested',
})
```

Task code must cooperate by checking or passing `context.signal`:

```ts
const syncAccount = task({
  id: 'accounts.sync',
  schema: accountSyncSchema,
  async run(payload, context) {
    for await (const page of provider.listPages(payload.accountId, { signal: context.signal })) {
      if (context.isCancellationRequested()) {
        return
      }

      await persistPage(page, { signal: context.signal })
    }
  },
})
```

Cancellation is not a hard process kill. If user code never observes the signal and never returns, the durable run remains `cancellation_requested` until the active lease expires and maintenance terminally cancels it.

Repeating `cancel()` for a `cancellation_requested` run re-notifies local user code when possible and returns the current run. It does not force an actively leased attempt into a terminal state. Terminal runs cannot be cancelled again.

## Rerun And Manual Retry [#rerun-and-manual-retry]

`runlane.runs.rerun(sourceRunId)` creates a new linked child run from a terminal source run. It requires the source task to be registered in the current runtime, uses the persisted payload, validates it against the currently registered task schema, and links the new run with `source.type = RunSourceType.Rerun`.

`runlane.runs.retry(sourceRunId)` is narrower: the source run must be failed. It creates a linked recovery child with `source.type = RunSourceType.ManualRetry`.

Neither operation mutates the source run.

```ts
const replay = await runlane.runs.rerun(sourceRunId)
const recovery = await runlane.runs.retry(failedRunId)
```

Linked child runs intentionally do not reuse task idempotency keys because the operator is asking for new work. Task singleton keys still apply when the lane can enforce them, so recovery actions do not overlap active work for the same logical resource.

Linked child runs use the runtime dispatch policy. With a transport-delivery lane and the default `dispatch.onTrigger: TriggerDispatchMode.Eager`, `rerun()` and `retry()` publish the child run's wakeup immediately. With `dispatch.onTrigger: TriggerDispatchMode.Deferred`, they persist any child outbox row, and `tick()` or another maintenance runner publishes later. Storage-polling lanes persist delivery intent for polling workers instead.

Task handlers can also create linked child runs with `context.trigger(...)`. Those children use `source.type = RunSourceType.Trigger` and the same `sourceRunId` filter, but they are application-created work rather than operator recovery actions.

Queue overrides use registered queue definitions:

```ts
await runlane.runs.retry(failedRunId, { queue: recoveryQueue })
```

## Pruning [#pruning]

`runlane.runs.prune()` delegates retention work to storage adapters that report `storage.prunesRuns: true`. Core always sends terminal statuses to storage, even when the caller omits `statuses`, and rejects active statuses before the adapter is called.

```ts
await runlane.runs.prune({
  actor: { type: ActorType.Operator, id: 'ops@example.com' },
  olderThan: '30d',
  statuses: [RunStatus.Succeeded, RunStatus.Cancelled],
})
```

`olderThan` may be a duration string relative to the runtime clock or an explicit `Date`. When `limit` is omitted, storage uses `contractDefaults.pruning.batchLimit`.

When pruning returns `nextCursor`, pass that cursor back with the same `olderThan` and `statuses` inputs. Duration cutoffs are frozen into the continuation cursor so the next call does not drift forward in time.

Pruning is irreversible through the public API. Use explicit bounded `limit` and `cursor` values when deleting large histories.

## CLI Commands [#cli-commands]

The CLI uses the same runtime-backed operator APIs after loading the configured runtime. See [CLI configuration](/configuration/cli) for runtime loading and JSON output, then use the command sections for the shipped operator surfaces:

* [`runlane runs list`](/configuration/cli#runlane-runs-list) maps to `runlane.runs.list()`.
* [`runlane runs get`](/configuration/cli#runlane-runs-get) maps to `runlane.runs.get()` and can include `runlane.runs.events()` with `--events`.
* [`runlane cancel`](/configuration/cli#runlane-cancel) maps to `runlane.runs.cancel()`.
* [`runlane retry`](/configuration/cli#runlane-retry) maps to `runlane.runs.retry()`.
* [`runlane rerun`](/configuration/cli#runlane-rerun) maps to `runlane.runs.rerun()`.
* [`runlane prune`](/configuration/cli#runlane-prune) maps to `runlane.runs.prune()`.

There are no CLI commands for `findActive()`, `findCurrent()`, `attempts()`, or `idempotencyKeys.reset()` yet; call the runtime APIs directly from an operator tool when those controls are needed.


# Pruning (/concepts/pruning)



Pruning is the retention semantic for removing old terminal run data without touching active work.

Only terminal runs are eligible: `succeeded`, `failed`, and `cancelled`. Active statuses such as `queued`, `running`, `retrying`, `released`, `scheduled`, and `cancellation_requested` are rejected before storage is called. A stuck active run is an operational problem to inspect or cancel, not data to delete silently.

## Retention scope [#retention-scope]

Prune filters are environment-scoped. This lets operators apply retention rules to one environment without touching another. Filters may also restrict eligible terminal statuses, such as pruning old succeeded and cancelled runs while keeping failed runs longer. When `statuses` is omitted, core sends the full terminal status set to storage.

`olderThan` is required. It may be a Runlane duration string such as `30d`, resolved relative to the runtime clock when `prune()` is called, or an explicit `Date` cutoff. Core resolves either form into a concrete `Date` before calling storage. First-party local and Postgres storage prune terminal runs whose materialized run `updatedAt` is before that cutoff.

`runlane.runs.prune()` is available only when the selected lane's storage reports `storage.prunesRuns: true`. Otherwise core rejects the call with `CapabilityUnsupported` and does not attempt a best-effort cleanup.

## Operator consequences [#operator-consequences]

Pruning is irreversible from the public API perspective. After old run history is pruned, operators should not expect to inspect the removed event history or payload through normal run reads.

Durable observation export records are not operator run history. First-party storage keeps observation records and exporter checkpoints separate so a slow exporter does not lose telemetry just because terminal run retention pruned the run projection and event history. Plan observation-stream retention at the telemetry/export boundary.

Use pruning only through an explicit maintenance surface with clear age and status boundaries. Pass a bounded `limit` for large histories and continue with `nextCursor` until no cursor is returned.

```ts
await runlane.runs.prune({
  actor: { type: ActorType.Operator, id: 'ops@example.com' },
  olderThan: '30d',
  statuses: [RunStatus.Succeeded, RunStatus.Cancelled],
  limit: 500,
})
```

The prune command sent to storage includes `prunedAt` from the runtime clock and `prunedBy` from `actor`. If `actor` is omitted, core uses `contractDefaults.actor.operator`.

When `limit` is omitted, storage prunes at most `contractDefaults.pruning.batchLimit` runs. Cursor continuations are tied to the same retention filter.

If pruning returns `nextCursor`, pass it back with the same environment, `olderThan`, and status set. Public prune cursors wrap the adapter cursor and the retention scope. Core rejects continuations when the cursor environment, `olderThan`, or statuses do not match. When `olderThan` is a duration string, core freezes the computed cutoff into the cursor so later continuation calls do not drift forward in time; pass the same duration string rather than recalculating a new date yourself.

```ts
const firstPage = await runlane.runs.prune({
  olderThan: '30d',
  statuses: [RunStatus.Succeeded, RunStatus.Cancelled],
  limit: 500,
})

if (firstPage.nextCursor !== undefined) {
  await runlane.runs.prune({
    cursor: firstPage.nextCursor,
    olderThan: '30d',
    statuses: [RunStatus.Succeeded, RunStatus.Cancelled],
    limit: 500,
  })
}
```

## CLI [#cli]

The CLI command maps directly to `runlane.runs.prune()` after loading the configured runtime:

```sh
runlane prune --older-than 30d --status succeeded cancelled --limit 500
```

`--older-than` is required and accepts either a Runlane duration or an ISO 8601 date/time. Ambiguous date strings such as `05/01/2026` are rejected before the runtime is loaded. `--status` accepts terminal status names only: `succeeded`, `failed`, and `cancelled`.

When CLI output includes `nextCursor`, continue with the same cutoff and statuses:

```sh
runlane prune --older-than 30d --status succeeded cancelled --cursor <nextCursor>
```

Use `--actor-id <id>` to pass an operator actor id. The CLI turns it into the prune `actor`; without it, core falls back to the default operator actor.


# Queues (/concepts/queues)



A queue is provider-neutral runtime infrastructure policy. It names where work is routed and, when configured, how much of that work may execute at once.

Queue names are persisted as stable strings on runs, events, outbox messages, cursors, and transport envelopes. TypeScript authoring APIs use queue definitions so task code, worker filters, trigger overrides, schedules, and provider bindings do not drift.

Use queues for workload routing such as `default`, `emails`, `billing`, `media`, and `imports`. Use separate lanes for real infrastructure boundaries such as a different database, AWS account, region, tenant boundary, compliance boundary, or durability policy.

Define queues once and export them as a catalog:

```ts
import { queue } from '@runlane/core'

export const mainQueue = queue({ name: 'main', default: true })

export const invoiceSyncQueue = queue({
  name: 'invoice_sync',
  concurrencyLimit: 1,
  dispatchTimeout: '2m',
})

export const queues = [mainQueue, invoiceSyncQueue]
```

Register the same definitions with the runtime:

```ts
const runlane = createRunlane({
  lane,
  queues,
  tasks: { syncInvoices },
})
```

`queue()` options:

| Option             | Required | Default   | What it controls                                                                                                                                             |
| ------------------ | -------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `name`             | Yes      | None      | Stable logical queue name persisted on runs, events, outbox rows, cursors, and transport messages. It must be a non-empty Runlane id and cannot contain `:`. |
| `default`          | No       | `false`   | Marks the queue selected when run creation has no task queue or trigger override. At most one runtime queue can be default.                                  |
| `concurrencyLimit` | No       | Unbounded | Durable max occupied slots per queue capacity partition when storage reports `enforcesQueueConcurrency`.                                                     |
| `dispatchTimeout`  | No       | None      | How long a bounded-queue dispatch reservation may sit unclaimed before maintenance can make that capacity available again. Requires `concurrencyLimit`.      |

Tasks, schedules, triggers, workers, and operator queue overrides should pass queue definitions:

```ts
const syncInvoices = task({
  id: 'invoices.sync',
  queue: invoiceSyncQueue,
  schema,
  async run(payload) {
    await syncProvider(payload)
  },
})

const worker = runlane.worker({
  queues: [invoiceSyncQueue],
  concurrency: 4,
})
```

Omitting `queues` on `runlane.worker()` or `executeNext()` means the worker can scan every registered runtime queue. Passing `queues: [invoiceSyncQueue]` is a filter: that worker only claims runs selected for the registered `invoice_sync` queue.

## Defaults And Drift [#defaults-and-drift]

`createRunlane({ queues })` is the authoritative queue registry. Queue names must be unique, and at most one queue may be marked `default: true`.

Tasks, schedules, and trigger calls may select a queue explicitly. When run creation has no task queue and no trigger override, runtime resolution uses the default queue. If the runtime has no default queue, that path fails fast with a configuration error.

There is no lane-invented fallback queue. If a runtime has no default queue, callers must select a queue explicitly.

Runtime APIs reject queue definitions whose name is missing from the registry or whose registered policy differs. This catches drift between shared queue constants, task definitions, worker filters, operator filters, and provider bindings.

## Durable Capacity [#durable-capacity]

`concurrencyLimit` is durable queue policy enforced by storage. A bounded queue occupies capacity with queued runs that still have an active dispatch reservation and running or cancellation-requested runs that still have an active lease. Storage uses that occupied count before appending new bounded-queue delivery requests or returning runnable candidates.

`dispatchTimeout` controls how long a dispatch reservation may sit unclaimed before maintenance can make capacity available again. It is only valid when `concurrencyLimit` is set. If omitted on a bounded queue, dispatch recovery uses the contract lease duration as the reservation window.

Worker `concurrency` is local process capacity. Queue `concurrencyLimit` is durable storage capacity. Use both when appropriate: a worker may have 10 local slots while a queue allows only 2 active attempts for a resource partition.

`concurrencyKey` partitions a bounded queue. Without a `concurrencyKey`, the whole queue is one capacity partition. With a key, each distinct key gets its own `concurrencyLimit`.

## Provider Bindings [#provider-bindings]

Provider-specific settings live at lane setup, not in task definitions. For SQS:

```ts
import { sqsQueue } from '@runlane/transport-sqs'

const lane = postgresSqsLane({
  postgres,
  sqs: {
    client,
    queues: [
      sqsQueue(invoiceSyncQueue, {
        queueUrl: process.env.RUNLANE_INVOICE_SYNC_QUEUE_URL,
      }),
    ],
  },
})
```

Each SQS binding provides exactly one of `queueUrl` or `queueName`. `queueOwnerAWSAccountId` is only valid with `queueName`. FIFO queues are configured on the SQS binding with `fifo` options, not on the provider-neutral queue definition.

The SQS transport exposes the bound queue definitions, so `createRunlane()` can reject missing provider bindings or queue-policy drift before the first wakeup publish. SQS still carries wakeups only: standard and FIFO settings affect provider delivery fields, not Runlane's durable run truth, lease ownership, retry policy, schedule policy, or task execution semantics.

See [Workers](./workers.mdx) for storage-polling worker filters and [SQS Transport](../contracts/sqs-transport.mdx) for SQS publish, FIFO, Lambda, and long-running consumer behavior.

## What Queues Are Not [#what-queues-are-not]

A queue is not durable truth for run state. A delayed, duplicated, or missing transport message must not decide whether a run exists or what status it has. Workers always load the current run from storage before execution.


# Rerun And Manual Retry (/concepts/rerun-and-manual-retry)



Rerun and manual retry are operator semantics for creating a new linked run. Neither operation should reactivate the original run.

## Rerun [#rerun]

Rerun means "run this task again." Call `runlane.runs.rerun(sourceRunId)` when an operator intentionally wants another execution of a terminal run's task and payload.

Rerun accepts any terminal source status: `succeeded`, `failed`, or `cancelled`. It rejects active source runs because those runs can still move forward through their own lifecycle.

Use rerun when an operator wants to repeat completed work, replay a task for investigation, or run the same payload again after external conditions changed.

## Manual retry [#manual-retry]

Manual retry means "recover this failed run." Call `runlane.runs.retry(sourceRunId)` when an operator wants a new recovery attempt for a failed run after automatic retry is exhausted, disabled, or no longer appropriate.

Manual retry is intentionally narrower than rerun. The source run must be `failed`; retrying a `succeeded`, `cancelled`, or still-active source run is rejected. The API name is `retry()` because it is an operator action, while the persisted child source is `RunSourceType.ManualRetry`.

## Linked child semantics [#linked-child-semantics]

Both operations create a new child run with its own id, counters, event history, lease lifecycle, delivery intent, and terminal outcome. Transport-delivery lanes may also create outbox rows for that child. The child records `source.sourceRunId` and either `source.type = RunSourceType.Rerun` or `source.type = RunSourceType.ManualRetry`, so operator tools can list the recovery history without rewriting the source.

The source run must be readable by the current lane and must belong to the runtime environment. Its task must also be registered in the current runtime. If the task is missing, Runlane cannot safely execute the persisted payload and returns `TaskNotFound`.

Runlane does not trust the old payload blindly. Before creating the child, core revalidates the source run's persisted payload against the currently registered task schema. If the task schema changed incompatibly, linked creation fails instead of enqueueing a run that workers would reject later.

```ts
const replay = await runlane.runs.rerun(sourceRunId)
const recovery = await runlane.runs.retry(failedRunId)
```

## Queue and dispatch [#queue-and-dispatch]

By default, the child uses the source run's queue. You may override it with a registered queue definition:

```ts
const recovery = await runlane.runs.retry(failedRunId, {
  queue: recoveryQueue,
})
```

Queue overrides go through the same queue registry validation as task registration, triggering, and workers. Passing an unknown queue, or a definition that conflicts with the registered policy for that name, is a configuration error.

Linked runs use the runtime trigger dispatch policy. With `dispatch.onTrigger: TriggerDispatchMode.Eager`, transport-delivery `rerun()` and `retry()` calls persist the child run and immediately flush matching outbox wakeups. With `TriggerDispatchMode.Deferred`, they persist any child outbox rows and leave publication to `tick()` or another maintenance runner. Storage-polling lanes only persist the child run's delivery intent.

## Audit options [#audit-options]

The child creation event uses the operator actor by default. Pass `actor` to record a specific operator identity, `meta` to attach safe JSON context to the child creation event, and `traceCarrier` to override trace propagation:

```ts
await runlane.runs.rerun(sourceRunId, {
  actor: { type: ActorType.Operator, id: 'ops@example.com' },
  meta: { reason: 'provider_replay' },
  traceCarrier: { traceparent },
})
```

When `traceCarrier` is omitted, the child inherits the source run's trace carrier. When `actor` is omitted, core records `ActorType.Operator`.

Use `runId` only when an operator process needs a deterministic child id for external correlation:

```ts
await runlane.runs.retry(failedRunId, {
  runId: asId<'run'>('run_retry_2026_05_23_user_123'),
})
```

If that run id already exists, storage reports the conflict. Linked creation does not use idempotency-key recovery to turn duplicate deterministic ids into a successful replay.

## Idempotency, singleton, and concurrency [#idempotency-singleton-and-concurrency]

Linked child runs intentionally do not reuse task idempotency keys. An operator rerun or manual retry means "do new work," not "return the old idempotency owner." The `CreateLinkedRunOptions` surface therefore has no `idempotencyKey` option, and task-level idempotency functions are not applied.

Task singleton and concurrency keys still apply when the lane can enforce them. This preserves resource protection for recovery work: linked children can be new work while still avoiding overlap with active work for the same singleton resource or queue concurrency slot.

## Why terminal runs stay terminal [#why-terminal-runs-stay-terminal]

Terminal runs are facts. Reactivating them makes history harder to reason about, creates awkward event replay semantics, and can hide the difference between the original failure and the recovery attempt. Linked child runs preserve both stories.

The source run is not mutated. Its status, counters, payload, trace carrier, queue, and event history remain the same before and after linked child creation. Operator UIs should show reruns and manual retries as related children, not as extra attempts on the source run.


# Retry Vs Release (/concepts/retry-vs-release)



Retry and release both make a run continue later, but they mean different things.

## Retry [#retry]

Retry means the task failed transiently and should attempt again. It is failure semantics.

Use retry for infrastructure or dependency failures such as timeouts, deadlocks, rate limits, and temporary provider outages. Retries increment failure and retry counters. Operators should see retry pressure because it indicates work is failing before eventually succeeding or exhausting its budget.

Retry policy controls the retry budget and backoff timing. `maxAttempts` counts total attempts, including the first execution. If a retryable failure happens before the budget is exhausted, core records `run.retry_scheduled`, persists the sanitized `RunFailure`, increments failure and retry counters, clears the lease, sets status to `retrying`, and stores the next `runAt`. When the retry budget is exhausted, core records `run.failed` and the run becomes terminally `failed`.

Failure records use stable `ErrorCode` values.

| Failure source              | Durable code behavior                                                             |
| --------------------------- | --------------------------------------------------------------------------------- |
| Unknown user-code throw     | Becomes `ErrorCode.TaskFailed` with the generic message `Task failed.`            |
| Structured `RunlaneError`   | Keeps its code while core maps the message to task-facing text.                   |
| Provider or business detail | Belongs in failure `meta`; keep `code` reserved for Runlane-owned classification. |

```ts
import { RetryBackoffType, task } from '@runlane/core'

const sendEmail = task({
  id: 'emails.send',
  retry: {
    maxAttempts: 3,
    backoff: { type: RetryBackoffType.Exponential, delay: '30s', maxDelay: '5m' },
  },
  schema,
  async run(payload) {
    await emailProvider.send(payload)
  },
})
```

An omitted `backoff` uses `contractDefaults.retry.backoff`. A string backoff is fixed. Exponential backoff doubles from the base delay for each retry and respects `maxDelay` when present.

Retry options:

| Option             | Required                    | Default                                  | What it controls                                                                 |
| ------------------ | --------------------------- | ---------------------------------------- | -------------------------------------------------------------------------------- |
| `maxAttempts`      | Yes                         | None                                     | Total attempt budget, including the first execution.                             |
| `backoff`          | No                          | `contractDefaults.retry.backoff` (`30s`) | Delay before the next retry. A string means fixed backoff.                       |
| `backoff.type`     | When `backoff` is an object | None                                     | `RetryBackoffType.Fixed` or `RetryBackoffType.Exponential`.                      |
| `backoff.delay`    | When `backoff` is an object | None                                     | Base delay as a Runlane duration string.                                         |
| `backoff.maxDelay` | No                          | No cap                                   | Maximum delay for exponential backoff. Must be greater than or equal to `delay`. |

Retry outcomes are durable attempt outcomes:

| Attempt outcome                       | Status after the attempt | Counter changes                               | Event                 |
| ------------------------------------- | ------------------------ | --------------------------------------------- | --------------------- |
| Retryable failure with budget left    | `retrying`               | `attempts + 1`, `failures + 1`, `retries + 1` | `run.retry_scheduled` |
| Retryable failure with no budget left | `failed`                 | `attempts + 1`, `failures + 1`                | `run.failed`          |
| Non-retryable structured failure      | `failed`                 | `attempts + 1`, `failures + 1`                | `run.failed`          |

`runNow()` follows the same outcome model. It executes exactly one inline attempt, then returns the persisted run. If that attempt schedules a retry, `runNow()` returns a `retrying` run; it does not sleep and execute the retry inline.

## Release [#release]

Release means the task is not done yet, but nothing failed. It is business waiting.

Use release when the next correct action is to wait for external state: a report is still processing, a payment is pending, an invoice is not ready, another run must finish first, or a provider tells you to check again later.

Releases increment release counters, not failure counters. Core records `run.released`, clears the lease, sets status to `released`, and stores a durable wait condition. A time release stores `runAt`; signal and run-completion releases wait for the matching event and may also carry a timeout fallback.

```ts
import { task } from '@runlane/core'

const pollReport = task({
  id: 'reports.poll',
  schema,
  async run(payload, context) {
    const report = await reports.get(payload.reportId)

    if (report.status === 'processing') {
      return context.release('5m', { reason: 'provider_not_ready' })
    }

    await storeReport(report)
  },
})
```

If the first polling attempt starts a one-time external operation, wrap that submission in `context.step.run()` and then release while the provider is still working. Release controls when the run wakes up again; the durable step prevents the submission callback from running again after a retry, release, or crash. See [Durable Steps](/concepts/durable-steps).

`context.release(delay, options)` returns a `TaskReleaseType.Release` result. Task handlers must return that result to release the run. Core reserves `type: 'task_release'` as the release discriminant, so successful output objects must not use that exact `type` value.

`context.release(delay, options)` fields:

| Value            | Required | Default | What it records                                                       |
| ---------------- | -------- | ------- | --------------------------------------------------------------------- |
| `delay`          | Yes      | None    | Runlane duration string until the released run becomes due again.     |
| `options.reason` | No       | None    | Short public reason for operator views, such as `provider_not_ready`. |
| `options.meta`   | No       | None    | JSON object with structured domain detail.                            |

Handlers can also release until a signal key is sent or another run reaches terminal state:

```ts
const waitForWebhook = task({
  id: 'reports.wait_for_webhook',
  schema,
  async run(payload, context) {
    if (!(await reportIsReady(payload.reportId))) {
      return context.waitForSignal(`report_ready_${payload.reportId}`, {
        reason: 'provider_webhook_pending',
        timeout: '1h',
      })
    }
  },
})
```

`context.waitForSignal(signalKey, options)` and `context.waitForRun(runId, options)` return the same `TaskReleaseType.Release` result. `options.reason` and `options.meta` are recorded for operators. `options.timeout` is optional; when supplied, the released run is also eligible for normal maintenance delivery after that time if the signal or source run completion never arrives.

A wait timeout is not an attempt timeout. It does not record `ErrorCode.TaskTimedOut`, does not increment failure counters, and does not consume retry budget. It only gives the released run a fallback wakeup time. When the fallback fires, the next attempt reruns the handler against the same payload and current application state.

Applications resume signal waits with `runlane.signals.send(signalKey, { limit? })`. Run-completion waits resume automatically when the source run reaches a terminal state through the same runtime; if the source run is already terminal when the waiter releases, core appends the resume delivery request with the release. Duplicate signal sends are safe: once the first delivery request clears the wait condition, later sends do not target that run again.

## Signal Resume API [#signal-resume-api]

Use signal waits when an external fact decides when the run should continue: a webhook arrives, a human approves work, a provider callback completes, or an application event says a resource is ready. The signal key names that fact. It is not a pub/sub topic and it does not carry a payload.

```ts
const waitForReport = task({
  id: 'reports.wait',
  schema,
  async run(payload, context) {
    if (!(await reportIsReady(payload.reportId))) {
      return context.waitForSignal(`report_ready_${payload.reportId}`, {
        reason: 'provider_webhook_pending',
        timeout: '1h',
      })
    }

    await storeCompletedReport(payload.reportId)
  },
})

async function handleReportWebhook(request: Request) {
  const webhook = await request.json()

  await persistWebhookResult(webhook)

  const resumedRuns = await runlane.signals.send(`report_ready_${webhook.reportId}`)

  return Response.json({ resumed: resumedRuns.length })
}
```

`runlane.signals.send(signalKey, options)`:

| Field           | Required | Meaning                                                                                                                           |
| --------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `signalKey`     | Yes      | Runlane id value for the external fact. It must be a non-empty string and must not contain `:`.                                   |
| `options.limit` | No       | Maximum matching released runs to resume in one scan. Defaults to the runtime maintenance delivery-request limit.                 |
| return value    | N/A      | `Promise<readonly RunRecord[]>` for runs where core durably appended a resume delivery request. These runs have not executed yet. |

`send()` appends `run.delivery_requested` to each still-released run waiting on that signal. Projection clears `run.wait`, sets the run back to queued, and records the delivery as available now. A worker, delivered-wakeup consumer, or later execution call runs the next attempt through the normal lease path.

Duplicate sends are safe. If the first send already cleared the wait, later sends return no run for that completed wait. If two callers race to send the same signal, event-sequence fencing lets one append win and the loser skips that run.

Signal data belongs in application storage. Persist webhook payloads, approval records, provider job results, or domain facts before calling `runlane.signals.send(...)`; the resumed handler should reread the authoritative application state. Do not encode unbounded provider payloads in the signal key, and do not expect Runlane to pass signal payloads to the handler.

Bounded queues keep their capacity rules. A signal resume records durable intent to continue, but it does not steal capacity from currently active bounded-queue work. The queued run waits for the normal dispatch reservation path.

Signal waits can have timeout fallbacks. If the signal arrives before the timeout, the signal clears the wait and the timeout later has nothing to resume. If the signal never arrives, maintenance can request delivery after the timeout so the handler can decide whether to keep waiting, succeed, fail, or retry.

Release outcomes are durable attempt outcomes:

| Attempt outcome                        | Status after the attempt | Counter changes                         | Event           |
| -------------------------------------- | ------------------------ | --------------------------------------- | --------------- |
| Handler returns a release result       | `released`               | `attempts + 1`, `releases + 1`          | `run.released`  |
| Handler later completes after resuming | `succeeded`              | `attempts + 1` for that resumed attempt | `run.succeeded` |

`runNow()` also executes only one release attempt inline. If the inline handler returns `context.release(...)`, `runNow()` returns a `released` run; the later attempt is driven by maintenance and workers.

## Maintenance wakeups [#maintenance-wakeups]

Retry and time-based release rely on maintenance after the run is waiting. When a `retrying` or time-released run's `runAt` is due, `tick()` appends `run.delivery_requested`. Transport-delivery lanes also request a matching outbox row for publication; storage-polling lanes keep the delivery intent in run state for polling workers. A worker or delivered-wakeup handler executes the next attempt.

Signal and run-completion waits are event-driven. `runlane.signals.send()` and terminal source-run completion append the next durable delivery request directly. That clears `run.wait` and records the run as queued. For bounded queues, resume does not steal capacity from already active work; the queued run waits for the normal capacity-reserving dispatch path. Timeout fallbacks still rely on `tick()` once the timeout is due.

`tick()` does not run user task code. It only materializes schedules, finalizes abandoned cancellations and timed-out attempts, requests delivery for due waiting or expired-lease runs, and flushes due outbox rows. For bounded queues, maintenance first reserves queue capacity through the dispatch path before appending the delivery request.

## Why the distinction matters [#why-the-distinction-matters]

Using retry for polling makes operator data noisy. It turns normal business waiting into apparent system failure and spends retry budget on expected provider state. Using release for actual failures is also wrong: it hides failure pressure from counters, alerts, and manual recovery paths.

Runlane keeps these outcomes separate so dashboards, alerts, and recovery actions can tell the difference between broken execution and expected continuation.


# Schedules (/concepts/schedules)



A schedule is deployment-time task configuration that creates runs automatically at a specific time or cadence. A schedule does not execute user code directly. It materializes durable runs, and workers execute those runs through the normal task lifecycle.

Runlane supports three schedule shapes:

* **once**: materialize a run at one specific time
* **interval**: materialize the latest due run on a fixed duration cadence
* **cron**: materialize the latest due run from a cron expression, optionally interpreted in a time zone

Schedules are colocated on the task they create. The authoring shape does not include a `type` or `taskId` field; core infers the schedule kind from exactly one of `runAt`, `every`, or `cron` and attaches the task id during registration.

```ts
import { createRunlane, queue, task, WorkerMode } from '@runlane/core'

const digestQueue = queue({ name: 'digests', default: true })

const sendDigest = task({
  id: 'digests.send',
  queue: digestQueue,
  schema: digestPayloadSchema,
  schedule: [
    {
      id: 'digests.daily',
      cron: '0 9 * * *',
      timeZone: 'America/New_York',
      payload: { userId: 'user_123' },
    },
  ],
  async run(payload) {
    await sendDigestEmail(payload.userId)
  },
})

const runlane = createRunlane({
  lane,
  queues: [digestQueue],
  tasks: { sendDigest },
})

await runlane.tick()
await runlane.worker({ mode: WorkerMode.Drain }).closed
```

Schedule options:

| Option     | Applies to    | Required                                   | Default                                          | What it controls                                                                                                                                    |
| ---------- | ------------- | ------------------------------------------ | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`       | all schedules | Yes                                        | None                                             | Stable schedule id. It is part of deterministic occurrence identity, must be unique across the runtime's task catalog, and cannot contain `:`.      |
| `queue`    | all schedules | No                                         | Task queue or runtime default queue              | Queue definition used by materialized runs. The queue must be registered on the runtime.                                                            |
| `enabled`  | all schedules | No                                         | `true`                                           | Whether the runtime materializer should consider this schedule.                                                                                     |
| `payload`  | all schedules | Required when task schema requires payload | `undefined` only when the task schema accepts it | Static JSON payload validated synchronously when the task is registered, then validated again before materialization. There is no payload callback. |
| `runAt`    | once          | Yes                                        | None                                             | Exact `Date` when one run should be materialized.                                                                                                   |
| `every`    | interval      | Yes                                        | None                                             | Runlane duration string cadence, such as `15m` or `1h`.                                                                                             |
| `startsAt` | interval      | No                                         | Unix epoch anchor                                | Alignment point for interval cadence.                                                                                                               |
| `endsAt`   | interval      | No                                         | No end bound                                     | Stops materializing interval occurrences after this time.                                                                                           |
| `cron`     | cron          | Yes                                        | None                                             | Cron expression used to find the latest due fire time.                                                                                              |
| `timeZone` | cron          | No                                         | Cron parser default                              | IANA time zone used to interpret the cron expression.                                                                                               |

Call `runlane.tick()` from maintenance infrastructure. It claims due occurrences from the runtime's registered task schedules and writes ordinary durable runs. It is safe to call from a cron process, a serverless scheduled function, or a maintenance worker. It does not execute task code; workers still claim and execute the materialized runs.

One tick is bounded maintenance. By default, `contractDefaults.maintenance` allows one pass to:

* materialize up to 100 due schedules
* terminally cancel up to 100 abandoned cancellation requests
* request delivery for up to 100 due waiting or expired-lease runs
* claim up to 100 outbox rows

Call `tick()` repeatedly when draining a large backlog. Each call uses the runtime clock once, then runs schedule materialization, cancellation finalization, delivery recovery, and outbox publishing in that order. Keep maintenance separate from transport delivery handlers; a delivery handler should process the wakeups it received, while `tick()` is a bounded global pass.

Materialization is durable per occurrence, not transactional across the whole registered schedule set. If one schedule fails, earlier schedules in the same tick remain materialized. The failing tick rejects and later maintenance phases do not run in that call; rerun `tick()` after fixing the failure to advance the rest and flush pending outbox rows.

## Occurrences [#occurrences]

A schedule occurrence is the durable claim for one scheduled fire time. Occurrences prevent duplicate materialization when more than one scheduler process is running.

Storage must support schedule occurrence claims for due registered schedules. The scheduler claims an occurrence, creates or recovers the generated run, and marks the occurrence complete with the run ids it produced. If another process already owns or completed the occurrence, storage returns no claim and the materializer skips it.

Occurrence ids are deterministic for `environment + scheduleId + fireAt`. The generated id is an opaque bounded hash, not a parseable key. The scheduled run id is derived from that occurrence id, so all materializers race on the same durable identities.

If a process creates the scheduled run and crashes before completing the occurrence, the next materializer can reclaim the same occurrence after claim expiry, see the already-created scheduled run, and complete the occurrence with that run id instead of creating a duplicate.

Interval and cron materialization intentionally emits at most one occurrence per schedule per tick: the latest fire time due at or before the maintenance time. Run `tick()` repeatedly from maintenance infrastructure to advance recurring schedules over time. Missed intermediate interval or cron boundaries are not backfilled in the same call.

Schedule timing rules:

| Shape      | Timing behavior                                                                                                                                                                                 |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `once`     | Materializes at the original `runAt` once it is due, even if the materializer runs late.                                                                                                        |
| `interval` | Materializes the latest interval boundary at or before the maintenance time. The cadence anchors to Unix epoch unless `startsAt` is set, and `endsAt` caps the latest eligible boundary.        |
| `cron`     | Materializes the most recent matching fire time at or before the maintenance time, using `timeZone` when supplied. Create or enable cron schedules only when that catch-up behavior is desired. |

## Run Creation Semantics [#run-creation-semantics]

Scheduled runs are ordinary durable runs with `source.type = "schedule"` and `source.scheduleId` set. They use the occurrence id as their dedupe identity. Queue capacity does not block materialization: due runs are created first, then dispatch and lease acquisition wait behind bounded queue capacity.

Schedule queues override the task queue for materialized runs. If a schedule does not specify `queue`, runtime resolution falls back to the task queue, then the runtime default queue. Missing or unregistered queues fail runtime registration or dynamic task registration before a scheduled run is created.

Task-level singleton keys still apply, so recurring schedules cannot overlap work for the same resource when the lane enforces singleton keys. Task-level idempotency keys do not apply to scheduled runs; reusing producer idempotency across recurring occurrences would suppress legitimate future runs.

Task-level concurrency keys also apply to scheduled runs and require bounded-queue support just like triggered runs. Core resolves singleton and concurrency keys before claiming the occurrence, so capability or key-validation failures do not consume an occurrence claim.

Each schedule entry carries static payload data when the task schema requires payload. Omitted `payload` is allowed only when the task schema accepts `undefined`; omission passes `undefined` through the same schema validation before registration and materialization. Because schedules are deployment-time configuration, schedule payload schemas must validate synchronously during registration.

Runlane does not support payload builder callbacks or implicit trigger-time payload inheritance in the task-colocated schedule surface. Model dynamic multi-tenant recurring work through a separate application API rather than hiding it inside static schedule definitions.

## Invariants [#invariants]

Schedules are definitions. Occurrences are durable claims. Runs are executable work.

This separation keeps scheduled jobs observable. Operators inspect the generated runs, not a separate schedule execution system.


# Tasks And Runs (/concepts/tasks-and-runs)



A task is user code plus the payload schema and execution policy around that code. A run is one durable execution record for a task and payload.

This separation is important. A task can be triggered many times. Each trigger creates or reuses a run according to idempotency and singleton rules. Operators inspect runs, not task definitions, because runs are where attempts, failures, releases, leases, and final outcomes live.

Use this page for the task/run mental model. For deeper surfaces, see [Schedules](/concepts/schedules), [Durable Steps](/concepts/durable-steps), [Retry Vs Release](/concepts/retry-vs-release), [Observability](/concepts/observability), [Operator APIs](/concepts/operator-apis), [Rerun And Manual Retry](/concepts/rerun-and-manual-retry), [Current-Process Execution](/concepts/current-process-execution), and the [Run Lifecycle vocabulary](/contracts/vocabulary/run-lifecycle).

## Task [#task]

A task definition describes:

* the stable task id
* the payload schema
* the default queue definition
* the retry budget and backoff policy
* optional idempotency and singleton keys
* optional colocated schedule entries with static payloads
* the handler that executes user code

The payload is validated before user code runs. Public task schemas use the Standard Schema contract so runtime packages can accept Zod and other compatible validators without coupling the public API to one validation library.

In `@runlane/core`, `task()` is the authoring helper for this definition. It accepts a task id, a Standard Schema-compatible schema, and a handler:

```ts
import { task } from '@runlane/core'

const sendEmail = task({
  id: 'emails.send',
  schema,
  async run(payload, context) {
    await sendEmailToUser(payload.userId)
  },
})
```

`task()` validates and normalizes the definition, then returns an immutable task handle. Treat that handle as deployment-time configuration:

* register it with a runtime
* trigger it through that runtime
* do not mutate it to change queues, retry policy, schedules, schema, or handlers

To change task behavior, create a new task definition and deploy a runtime with that definition.

`task()` options:

| Option               | Required | Default                              | What it controls                                                                                                                       |
| -------------------- | -------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| `id`                 | Yes      | None                                 | Stable task identity. Use names such as `emails.welcome`; do not include tenant, user, or resource ids.                                |
| `schema`             | Yes      | None                                 | Standard Schema-compatible payload validator. Runtime validates before trigger creation and again before handler execution.            |
| `run`                | Yes      | None                                 | Async or sync task handler. It receives the validated payload and `TaskContext`.                                                       |
| `queue`              | No       | Runtime default queue                | Default queue definition for runs of this task.                                                                                        |
| `retry`              | No       | No automatic retries                 | Retry budget and backoff for retryable task failures.                                                                                  |
| `maxAttemptDuration` | No       | No attempt deadline                  | Maximum wall-clock duration for one attempt. The fixed deadline is written when a lease is claimed.                                    |
| `schedule`           | No       | No schedule                          | One schedule entry or an array of schedule entries colocated with the task.                                                            |
| `idempotencyKey`     | No       | None                                 | Static key or payload resolver for producer dedupe. Duplicate active or retained terminal owners return the original run.              |
| `idempotencyKeyTTL`  | No       | `30d` when an idempotency key exists | Retention for successful or cancelled terminal idempotency owners. Use `'active'` to release at terminal. Requires an idempotency key. |
| `singletonKey`       | No       | None                                 | Static key or payload resolver for one active protected resource. Requires a lane whose storage reports `enforcesSingleton`.           |
| `concurrencyKey`     | No       | None                                 | Static key or payload resolver for bounded-queue capacity partitioning. Requires a queue with `concurrencyLimit`.                      |

Runlane ids, queue names, idempotency keys, singleton keys, and concurrency keys are opaque strings. Values must be non-empty and must not contain `:`. Queue definitions are created with `queue()` and registered on the runtime; core still brands and revalidates untrusted task definitions at runtime before writing durable run, schedule, lease, and outbox records.

Register tasks when creating the runtime:

```ts
const runlane = createRunlane({
  lane,
  queues,
  tasks: { sendEmail },
})
```

When `tasks` is supplied as a named catalog, that catalog is authoritative for triggering, executing, and maintaining work, and the same task handles are available through `runlane.tasks`. Triggering a task outside the catalog fails before a run is created, and workers can only execute task ids registered in that runtime.

Arrays are still accepted for registration-only runtimes, but application factories should prefer named catalogs so task registration and task lookup stay in one place.

Runtime observers can subscribe to durable observations:

```ts
import { RunlaneObservationType } from '@runlane/core'

const runlane = createRunlane({
  lane,
  queues,
  tasks: { sendEmail },
  observability: {
    observers: [
      (observation) => {
        if (observation.type !== RunlaneObservationType.RunEvent) {
          return
        }

        metrics.increment('runlane.run_event', {
          environment: observation.environment.name,
          eventType: observation.eventType,
        })
      },
    ],
  },
})
```

Observers run after storage confirms a durable event or step completion. Run-event observations receive `type: RunlaneObservationType.RunEvent`, environment, run id, event id, event sequence, event type, occurred time, and trace carrier when present. Step observations receive `type: RunlaneObservationType.RunStep`, environment, run id, step key, step token, attempt, and completion time. Observations do not receive payloads, successful output, step output, or failure detail. Observer failures are isolated from runtime behavior and can be collected with `observability.onObserverError`. See [Observability](/concepts/observability) for the runtime hook contract and exporter guidance.

`trigger(task, payload, options)` options:

| Option              | Default                                    | What it controls                                                                                            |
| ------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| `actor`             | `{ type: ActorType.System }`               | Actor recorded on the initial `run.created` event and on immediate unbounded-queue delivery requests.       |
| `concurrencyKey`    | Task-level `concurrencyKey`                | Per-run bounded-queue partition override.                                                                   |
| `idempotencyKey`    | Task-level `idempotencyKey`                | Per-run idempotency owner override.                                                                         |
| `idempotencyKeyTTL` | Task-level `idempotencyKeyTTL`, then `30d` | Retention for the selected idempotency owner. Requires a task or trigger idempotency key.                   |
| `meta`              | None                                       | JSON object stored on the initial run event and run projection for operator context.                        |
| `queue`             | Task queue or runtime default queue        | Per-run queue definition override. It must match a registered runtime queue definition.                     |
| `runId`             | Generated `run_${uuid}`                    | Caller-supplied run id for external correlation. It participates in optimistic creation and should be rare. |
| `singletonKey`      | Task-level `singletonKey`                  | Per-run active-resource lock override.                                                                      |
| `traceCarrier`      | None                                       | Small string map for trace headers or request correlation.                                                  |

`trigger()` returns a `TriggerRunResult`:

| Outcome                               | Meaning                                                                                                                                       |
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `TriggerOutcomeType.Created`          | This call persisted a new run. Unbounded queues also get an immediate delivery request; bounded queues wait for durable capacity reservation. |
| `TriggerOutcomeType.ReturnedExisting` | Storage idempotency returned the active or retained run that already owns the selected idempotency key.                                       |

Use the `run` field for the durable record:

```ts
const { run, outcome } = await runlane.trigger(runlane.tasks.sendWelcomeEmail, {
  userId: 'user_123',
})
```

`runNow(task, payload, options)` uses the same durable creation options, then adds lease execution controls such as `leaseDuration`, `heartbeatInterval`, `signal`, and `workerId`. It creates `run.created + run.lease_claimed` atomically and executes exactly one current-process attempt without creating an initial delivery request. Duplicate idempotency owners return the existing run without executing another inline attempt. See [Current-Process Execution](/concepts/current-process-execution) for the full API reference and failure modes.

The runtime validates payloads twice: once when `trigger()` creates new work, and again when `executeNext()` reads the persisted payload before calling user code. The second validation catches schema drift in old runs before the handler sees data it no longer accepts.

Tasks may omit payload when their schema accepts `undefined`. Callers may use `runlane.trigger(task)` and task-colocated schedules may omit `payload` only for those no-payload task schemas; that omission is not inheritance or a hidden callback, and no synthetic payload is written into run history.

Retry policies are part of the task definition. `maxAttempts` is the total attempt budget, including the first attempt. `backoff` may be a fixed duration string or a fixed/exponential backoff object; when omitted, core uses `contractDefaults.retry.backoff`.

`maxAttemptDuration` is also part of the task execution policy. It bounds one attempt, not the whole run. Core writes a fixed `attemptDeadlineAt` when the attempt lease is claimed; heartbeats do not extend that deadline. If the handler cooperates with `context.signal`, the attempt fails with `ErrorCode.TaskTimedOut`. If the process dies or user code never returns after the attempt has started, maintenance can finalize the expired running attempt later. A claim that dies before `run.started` is recorded is lease-recovered for another attempt instead.

Handlers can return JSON-compatible output when the attempt succeeds. `undefined` means void success and leaves `run.output` unset; `null` is an explicit JSON output and is persisted. Objects and arrays must be JSON-compatible because storage adapters persist output in run history and the materialized `RunRecord`.

```ts
const fetchInvoice = task({
  id: 'stripe.invoice.fetch',
  schema: invoiceFetchSchema,
  async run(payload, context) {
    const invoice = await stripe.invoices.retrieve(payload.invoiceId, {
      signal: context.signal,
    })

    return {
      invoiceId: invoice.id,
      status: invoice.status,
    }
  },
})
```

Handlers can checkpoint one-time intermediate operations with `context.step.run(stepKey, { output }, callback)`. A completed step stores the parsed JSON-safe output in storage; later attempts for the same run and step key return that stored output and skip the callback. Use it for external submissions that should not be repeated after release, retry, timeout, or process loss. See [Durable Steps](/concepts/durable-steps).

Handlers can return `context.release(delay, options)` when the correct outcome is to continue later without recording a failure. Release is for business waiting, not exceptions. `type: 'task_release'` is a reserved handler-result discriminant; do not use that exact `type` value for successful output objects. See [Retry Vs Release](/concepts/retry-vs-release) for the retry budget, backoff, and release semantics.

Handlers can create child work with `context.trigger(task, payload, options)`. This uses the same validation, queue, idempotency, singleton, dispatch, and operator-read path as public `runlane.trigger()`, but records the child with `source.type = RunSourceType.Trigger` and `source.sourceRunId` set to the parent run. The child inherits the parent trace carrier unless the trigger options supply a new one.

```ts
const importAccount = task({
  id: 'accounts.import',
  schema: accountImportSchema,
  async run(payload, context) {
    await context.trigger(syncAccountContacts, {
      accountId: payload.accountId,
    })
  },
})
```

## Outputs, Causation, Timeouts, And Resume [#outputs-causation-timeouts-and-resume]

These primitives intentionally describe different facts:

| Primitive           | Stored on                                                               | What it means                                                                                                                                 |
| ------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Successful output   | `run.succeeded` and current `RunRecord.output`                          | The handler completed and returned JSON-compatible output. `undefined` is void success and leaves `output` absent; `null` is explicit output. |
| Durable step output | Step storage keyed by run and step key                                  | An intermediate JSON-compatible checkpoint used to avoid repeating one named operation inside the same run.                                   |
| Causation           | Child `RunRecord.source`                                                | This run was created by another durable run, an operator rerun, or a manual retry. Trace carrier is correlation data, not causation.          |
| Attempt timeout     | `run.lease_claimed.attemptDeadlineAt` and `RunRecord.attemptDeadlineAt` | One owned attempt has a fixed wall-clock deadline. Heartbeats keep lease ownership alive but never move this deadline.                        |
| Resume wait         | `run.released.wait` and current `RunRecord.wait`                        | The run is waiting for time, a signal, or another run to complete without counting that wait as a failure.                                    |

A handler can checkpoint durable steps during an attempt, but the attempt itself still either succeeds with output, fails, retries, releases, or is cancelled. Release results are explicit values returned by `context.release(...)`, `context.waitForSignal(...)`, or `context.waitForRun(...)`; core does not infer release from fields such as `delay`, `reason`, or `meta`, but it reserves the `type: 'task_release'` discriminant for release control flow.

`maxAttemptDuration` and release timeouts are different. `maxAttemptDuration` is failure pressure for a running attempt: core aborts `context.signal`, and the attempt eventually becomes `run.failed` or `run.retry_scheduled` with `ErrorCode.TaskTimedOut`. A wait timeout is a fallback for business waiting: if the signal or source run never arrives, maintenance queues the same released run for another attempt. That resumed attempt decides whether to keep waiting, succeed, fail, or retry.

## Task Ids, Idempotency, Singleton, And Concurrency [#task-ids-idempotency-singleton-and-concurrency]

Use the task `id` to name the kind of work. Use `singletonKey` to name the active resource that must not overlap.

```ts
import { queue, task } from '@runlane/core'

const quickBooksQueue = queue({
  name: 'quickbooks',
  concurrencyLimit: 2,
  dispatchTimeout: '2m',
})

const syncQuickBooksInvoices = task({
  id: 'quickbooks.invoices.sync',
  queue: quickBooksQueue,
  schema: quickBooksSyncSchema,
  concurrencyKey: (payload) => `quickbooks_${payload.userId}`,
  singletonKey: (payload) => `quickbooks_invoices_${payload.userId}`,
  async run(payload, context) {
    await syncInvoicesForUser(payload.userId, { signal: context.signal })
  },
})
```

In this example, `quickbooks.invoices.sync` identifies the task definition. Every user uses the same task id because every run executes the same handler.

`quickbooks_invoices_user_123` identifies the protected resource. If a cron schedule and a UI button both trigger invoice sync for `user_123`, they should resolve the same singleton key so storage can reject overlapping active runs for that user.

Idempotency keys answer: "Is this the same logical trigger request as before?"

| State                                         | Idempotency behavior                                                    |
| --------------------------------------------- | ----------------------------------------------------------------------- |
| Active owner exists                           | Repeated triggers return the original run.                              |
| Successful or cancelled terminal owner exists | The owner is retained for `idempotencyKeyTTL`, which defaults to `30d`. |
| Failed owner reaches terminal                 | The key is released automatically.                                      |
| `idempotencyKeyTTL: 'active'`                 | The key is released as soon as the run reaches any terminal state.      |

The scope is environment + task id + idempotency key. Runlane does not compare payloads for idempotency keys. Reusing the same key with different payloads returns the original run. If an integration needs payload-mismatch detection or permanent API response replay, keep that in an application request ledger.

Operators can clear one retained terminal owner with `runlane.idempotencyKeys.reset(task, { key })`. Reset rejects active owners because it is an escape hatch for retained terminal keys, not a force-run option.

Singleton keys are active-run locks. They prevent overlap while a matching run is queued, running, retrying, released, or otherwise active.

Terminal runs release the active singleton key when storage persists the terminal projection. Core validates and resolves the key, then requires the selected lane to report `storage.enforcesSingleton: true`; storage owns the uniqueness guarantee.

The local lane enforces singleton keys in memory so application tests can use the same task definitions as production. It does not prove production locking or transaction behavior.

Keep singleton keys stable and resource-oriented. Do not put user ids, provider account ids, or tenant ids into the task id. If external ids can contain reserved characters or unbounded text, encode or hash them into a Runlane-safe singleton key before returning it.

Concurrency keys partition a bounded queue. They answer: "Which queue-capacity partition does this run belong to?" They do not deduplicate triggers. Use them when every request should become a run but execution should be limited per tenant, account, or other partition.

Without a `concurrencyKey`, the whole bounded queue is one capacity partition. With a key, each distinct key gets its own `concurrencyLimit`. See [Queues](/concepts/queues) for the difference between durable queue capacity and local worker concurrency.

## Run [#run]

A run records:

* the task id and validated payload
* the environment namespace
* the queue
* current status
* attempt, failure, retry, and release counters
* append-only event history
* optional lease, source, wait condition, attempt deadline, idempotency, singleton, trace, and metadata fields
* optional concurrency key, dispatch reservation, current failure summary, and successful output

Run statuses are either active or terminal. Active runs can continue moving through the lifecycle. Terminal runs do not reactivate. Rerun and manual retry create new linked runs instead of mutating a terminal source run.

The stable status vocabulary lives in [Run Lifecycle](/contracts/vocabulary/run-lifecycle). The important split is:

| Class    | Statuses                                                                           | Meaning                                                                                |
| -------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Active   | `queued`, `running`, `retrying`, `released`, `scheduled`, `cancellation_requested` | The run can still be claimed, resumed, retried, cancelled, or completed.               |
| Terminal | `succeeded`, `failed`, `cancelled`                                                 | The run is historical fact. Operator rerun or manual retry creates a linked child run. |

Failed and retry-scheduled events include a `RunFailure` summary. Its `code` is always an `ErrorCode`: unstructured handler failures become `ErrorCode.TaskFailed`, while structured runtime failures preserve their `RunlaneError.code`.

`run.failure` exposes the current visible failure on failed and retrying runs. Full failure history remains in raw events and `runlane.runs.attempts(runId)`.

Business or provider-specific values belong in `meta`, not in `code`, because Runlane owns the failure classification vocabulary used by retries, operators, and adapters.

`run.output` exposes the current successful output only for succeeded runs. Failed and retrying runs use `failure`; released and cancelled runs do not have final output. Invalid handler output is recorded as a task attempt failure with `ErrorCode.TaskOutputInvalid`, not as a storage or transport failure.

`run.wait` exposes the current business-wait condition for released runs. Time releases carry a resume time; signal and run-completion waits carry the durable external condition plus an optional timeout fallback. `run.attemptDeadlineAt` is present only while a running attempt has a fixed max-attempt deadline.

Operator reads, cancellation, rerun, manual retry, idempotency-key reset, and pruning are covered in [Operator APIs](/concepts/operator-apis). Rerun and manual retry are intentionally linked-run creation operations; see [Rerun And Manual Retry](/concepts/rerun-and-manual-retry).

## Events [#events]

Run events are the durable history of a run. Core owns the reducer semantics that turn events into materialized run state. Storage persists both the events and the core-projected state atomically.

Existing event shapes are replay contracts. New required event data should be introduced with a new event type instead of changing old history in place.

Unbounded-queue `trigger()` runs normally start with `run.created` and `run.delivery_requested`. Bounded-queue trigger creation starts with `run.created`; `tick()` later reserves queue capacity and appends `run.delivery_requested` when the run is due and capacity is available. That split is what keeps bounded queue capacity durable instead of being decided by transport delivery.

After delivery is requested, a worker or delivered wakeup records `run.lease_claimed`, `run.started`, optional `run.lease_heartbeat` events during long attempts, and finally `run.succeeded`, `run.failed`, `run.retry_scheduled`, `run.released`, or `run.cancelled` depending on the attempt outcome. `run.succeeded` may include JSON output; failure, retry, release, and cancellation events keep output absent.

Current-process runs created by `runNow()` skip the initial delivery request and instead start with `run.created`, `run.lease_claimed`, and `run.started`. If the first inline attempt releases, retries, or loses process ownership before an outcome, later `tick()` recovery uses the normal delivery request path.


# Workers (/concepts/workers)



A worker is the runtime loop that turns queued durable runs into task attempts. Core has two storage-acquisition modes, `poll` and `drain`, both built on `executeNext(options)`, plus a separate transport-driven `executeDelivery(message, options)` path.

`runNow()` is the current-process exception: it creates and leases one new run atomically, then reuses the same post-lease execution path workers use. It is covered separately because it must not scan storage or execute an unrelated queued run. See [Current-Process Execution](/concepts/current-process-execution).

Use polling workers in application processes that should execute background work continuously:

```ts
import { WorkerMode } from '@runlane/core'

import { emailQueue } from './queues'

const worker = runlane.worker({
  mode: WorkerMode.Poll,
  queues: [emailQueue],
  concurrency: 4,
})

// During process shutdown:
await worker.stop()
```

`runlane.worker()` starts immediately and returns a handle. Omitting `mode` preserves the default polling behavior.

For production deployments that should use only Postgres, compose `postgresPollingLane()` from `@runlane/lane-postgres-polling` and run `WorkerMode.Poll` workers. That is the prescribed EC2, ECS, Fargate, Kubernetes, or systemd path when you do not want SQS. The lane uses `lane.delivery.mode: 'storage_polling'`, so the worker acquires runnable work by scanning and leasing runs from Postgres.

## Worker Handle [#worker-handle]

The worker handle separates passive state from actions:

```ts
const worker = runlane.worker({ mode: WorkerMode.Poll })

await worker.closed
```

`worker.closed` is a `Promise<void>` property, not a function. It is the same terminal promise every time it is read, and it settles when every worker slot has exited. Await it when the process should stay alive until the worker exits naturally or fails.

`worker.stop()` is the action. It aborts the worker signal, waits for active attempts to finish cooperatively, and then returns the same terminal result represented by `worker.closed`.

```ts
process.once('SIGTERM', () => {
  void worker.stop()
})
```

Await `worker.closed` or `worker.stop()` to observe worker loop failures. The handle attaches an internal rejection handler only to prevent unhandled promise warnings.

## Execution Options [#execution-options]

`runlane.worker()` options:

| Option               | Default                                  | Valid with              | What it controls                                                                                                                                                                                                                        |
| -------------------- | ---------------------------------------- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mode`               | `WorkerMode.Poll`                        | worker                  | `Poll` keeps scanning until stopped; `Drain` exits when current due work is gone or `maxRuns` is reached.                                                                                                                               |
| `queues`             | All registered runtime queues            | worker, `executeNext()` | Queue definitions this storage-acquisition path may claim from.                                                                                                                                                                         |
| `concurrency`        | `1`                                      | worker                  | Local process slots. This is not durable queue capacity.                                                                                                                                                                                |
| `emptyPollDelay`     | `100ms`                                  | poll worker only        | Sleep after an empty storage scan.                                                                                                                                                                                                      |
| `maxRuns`            | No limit                                 | drain worker only       | Maximum successfully executed runs before drain exits. Lost claims and abandoned attempts do not count.                                                                                                                                 |
| `leaseDuration`      | `contractDefaults.lease.duration` (`5m`) | worker, `executeNext()` | Durable ownership window written when a run is claimed and on each heartbeat.                                                                                                                                                           |
| `heartbeatInterval`  | Half of `leaseDuration`                  | worker, `executeNext()` | How often core writes `run.lease_heartbeat` while user code is still running. Must be shorter than `leaseDuration`.                                                                                                                     |
| `maxAttemptDuration` | Task-level `maxAttemptDuration`          | worker, `executeNext()` | Maximum wall-clock duration for one claimed attempt. Heartbeats do not extend this deadline.                                                                                                                                            |
| `onRunExecuted`      | None                                     | worker                  | Worker-local callback after a worker slot persists the run's final state for one attempt. Use runtime [Observability](/concepts/observability) for durable run-event telemetry across trigger, maintenance, delivery, and worker paths. |
| `signal`             | None                                     | worker, `executeNext()` | Cooperative stop/cancel signal linked into `TaskContext.signal`.                                                                                                                                                                        |
| `workerId`           | Runtime worker id                        | worker, `executeNext()` | Diagnostic owner recorded on leases. It does not route work.                                                                                                                                                                            |

`runlane.executeNext(options)` accepts the shared storage-acquisition options without worker loop controls:

| Option               | Default                                  | What it controls                                                                                                                           |
| -------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `queues`             | All registered runtime queues            | Queue definitions this one storage scan may claim from.                                                                                    |
| `leaseDuration`      | `contractDefaults.lease.duration` (`5m`) | Durable ownership window written when a run is claimed and on each heartbeat.                                                              |
| `heartbeatInterval`  | Half of `leaseDuration`                  | How often core writes `run.lease_heartbeat` while user code is still running. Must be shorter than `leaseDuration`.                        |
| `maxAttemptDuration` | Task-level `maxAttemptDuration`          | Maximum wall-clock duration for this attempt.                                                                                              |
| `signal`             | None                                     | Cooperative cancellation signal linked into `TaskContext.signal`. If already aborted before the scan, `executeNext()` returns `undefined`. |
| `workerId`           | Runtime worker id                        | Diagnostic owner recorded on leases.                                                                                                       |

`runlane.executeDelivery(message, options)` options:

| Option               | Default                                  | What it controls                                                                                                                                   |
| -------------------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `leaseDuration`      | `contractDefaults.lease.duration` (`5m`) | Durable ownership window for this delivered attempt.                                                                                               |
| `heartbeatInterval`  | Half of `leaseDuration`                  | Heartbeat cadence for this attempt. Must be shorter than `leaseDuration`.                                                                          |
| `maxAttemptDuration` | Task-level `maxAttemptDuration`          | Maximum wall-clock duration for this delivered attempt.                                                                                            |
| `signal`             | None                                     | Abort signal linked into the task context. If already aborted before claim, delivery rejects so the transport does not acknowledge unhandled work. |
| `workerId`           | Runtime worker id                        | Diagnostic owner recorded on the run lease.                                                                                                        |

## Acquisition Paths [#acquisition-paths]

Plain language:

`poll` means: "Keep looking for work until I'm told to stop."

* Good for EC2, containers, local dev.
* If storage has work, execute it.
* If storage has no work, wait `emptyPollDelay`, then check again.
* It is a long-running loop.

`drain` means: "Process whatever is available right now, then exit."

* Good for cron/serverless/bounded jobs.
* If storage has 10 due runs and concurrency is 2, process them until none are left.
* If storage is empty, return immediately.
* No sleeping, no idle loop.

`executeNext(options)` performs one storage scan, tries to claim one currently due run, executes at most one attempt, and returns the final `RunRecord` for that attempt. If storage has no due work, the signal is already aborted, the candidate run disappears, another worker wins the claim, or the attempt is abandoned after losing lease ownership, it returns `undefined`. Worker loops keep the reason internally so drain workers can distinguish idle storage from races that should not spend `maxRuns`.

`executeDelivery` is different. It should not use drain.

`drain` asks storage: "What work is due?"

`executeDelivery(message)` says: "A transport woke me for this specific `runId`; try to execute exactly this run if storage says it is still valid."

```text
poll
  -> storage scan
  -> execute next due run
  -> no work? sleep, repeat

drain
  -> storage scan
  -> execute due runs until empty or maxRuns reached
  -> exit

executeDelivery
  -> use delivered runId
  -> read that run
  -> ignore if terminal/not due/wrong queue/already leased
  -> otherwise claim and execute it
  -> exit
```

SQS/Lambda uses `executeDelivery`, not `drain`, because Lambda already received specific SQS messages. Scanning storage again would ignore the transport's work acquisition and can cause bad behavior around ack/retry.

## Queues [#queues]

Workers only claim runs from their configured queues. Omitting `queues` means the worker can process any registered runtime queue. Queue filters accept queue definitions from the runtime queue catalog; CLI text input such as `--queue emails` is resolved against `runtime.queues` before calling the runtime API.

Queue filters are routing, not ownership. Storage still owns run truth, and every worker reads the latest `RunRecord` before executing.

## Concurrency [#concurrency]

`concurrency` controls how many attempts one storage-acquisition worker process may run at the same time. Omitting it starts one slot.

Each polling concurrency slot runs the same storage scan, claim, and execute loop. If storage has no due run, the slot waits for `emptyPollDelay` before scanning again. Omitting `emptyPollDelay` uses `100ms`.

Worker `concurrency` and `emptyPollDelay` are local process controls. Queue `concurrencyLimit` is durable storage policy enforced across workers for bounded queues. Transport-driven SQS consumers use provider message delivery plus `executeDelivery(message)` and do not use the storage-polling worker's empty-poll loop.

Use the core polling worker for local development loops and deployments that intentionally poll storage, such as a simple EC2 or container process. Event-driven transports should use `executeDelivery(message)`.

## Drain Workers [#drain-workers]

Drain mode is still storage acquisition. It is useful when the process itself owns the decision to scan storage, but should not stay alive after the current backlog is gone:

```ts
const worker = runlane.worker({
  mode: WorkerMode.Drain,
  queues: [emailQueue],
  concurrency: 2,
  maxRuns: 100,
})

await worker.closed
```

`maxRuns` is optional. When omitted, drain mode keeps scanning until storage returns no due work. `emptyPollDelay` is not valid in drain mode because drain workers do not sleep on empty storage.

Lost lease-claim races and abandoned attempts do not count against `maxRuns`. In both cases storage had due work, but this worker did not execute a run, so drain workers keep scanning and exit only when storage is idle or the executed-run budget is reached.

## Delivered Wakeups [#delivered-wakeups]

`executeDelivery(message)` is the transport-driven execution primitive. It receives a `DeliveryMessage`, reads storage by `message.environment + message.runId`, verifies the queue, and then either claims and executes that run or returns an ignored result.

```ts
import { ExecuteDeliveryStatus } from '@runlane/core'

const result = await runlane.executeDelivery(message)

if (result.status === ExecuteDeliveryStatus.Ignored) {
  // Ack-safe non-execution: terminal, not due, wrong queue, already leased, claim lost, or abandoned.
}
```

Ignored delivery results are safe for transport adapters to acknowledge.

| Result                                                                                                           | Transport behavior                                                                                                       |
| ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Persisted task outcome: `run.succeeded`, `run.failed`, `run.retry_scheduled`, `run.released`, or `run.cancelled` | Acknowledge the provider message. Delivery handling completed.                                                           |
| Missing registered task or queue                                                                                 | Core persists a non-retryable `run.failed` outcome, then returns `Executed` to avoid poison-message redelivery loops.    |
| Storage, projection, cancellation-before-claim, or persistence failure                                           | Reject so the transport integration can retry or report a provider batch failure.                                        |
| Already-aborted delivered signal                                                                                 | Reject with `OperationCancelled` before claim. A delivered provider message should not be acknowledged without handling. |

This is stricter than storage polling: `executeNext({ signal })` returns `undefined` when the signal is already aborted, because no provider message has been acquired.

For SQS-to-Lambda, `@runlane/transport-sqs`:

* parses SQS records into `DeliveryMessage` values
* calls the supplied `executeDelivery()` callback for each parsed record
* returns batch item failures only for rejected framework or infrastructure failures

For standard SQS queues, the Lambda helper continues after a failed record and reports only that record as a partial batch failure. For FIFO queues, pass `fifo: true`; the helper stops after the first failure and returns that failed record plus every later unprocessed record so provider ordering is preserved.

For a long-running SQS consumer, `createSqsDeliveryConsumer()` blocks on SQS `ReceiveMessage`, calls `executeDelivery()` for each received message, and deletes only messages whose delivery resolves. It does not scan storage on `emptyPollDelay`. Messages whose provider shape, wakeup body, or execution path rejects are left undeleted so SQS visibility timeout and redrive policy can handle retry or DLQ movement.

| Path                                 | Acquisition owner           | Acknowledges provider messages      | Use when                                                                |
| ------------------------------------ | --------------------------- | ----------------------------------- | ----------------------------------------------------------------------- |
| `worker({ mode: WorkerMode.Poll })`  | Runlane storage scan        | No                                  | A long-running process intentionally polls storage.                     |
| `worker({ mode: WorkerMode.Drain })` | Runlane storage scan        | No                                  | A bounded process should execute due storage work and exit.             |
| `executeNext(options)`               | One Runlane storage scan    | No                                  | Tests, CLIs, or custom loops need one storage-acquired attempt.         |
| `createSqsLambdaHandler()`           | AWS Lambda SQS event source | Yes, through partial batch failures | Lambda receives SQS records and should invoke `executeDelivery()`.      |
| `createSqsDeliveryConsumer()`        | SQS `ReceiveMessage` loop   | Yes, through `DeleteMessageBatch`   | EC2, ECS, Fargate, or another Node process should consume SQS directly. |

Pick one production acquisition path per queue. For SQS-backed queues on EC2/ECS/Fargate, use `createSqsDeliveryConsumer()`. For polling deployments, use `runlane.worker({ mode: WorkerMode.Poll })` with a lane/config that does not publish SQS wakeups for that queue. A storage-polling worker can execute due runs from storage, but it will not receive or delete SQS records.

With `@runlane/lane-postgres-polling`, the polling worker is the production acquisition path. With `@runlane/lane-postgres-sqs`, an SQS consumer is the production acquisition path. Do not run SQS consumers and polling workers for the same logical queue unless you are deliberately testing a migration or backstop; storage leases prevent duplicate durable execution, but provider messages still need one owner that can acknowledge or delete them.

## Leases And Heartbeats [#leases-and-heartbeats]

Every attempt starts by claiming a run lease. The lease prevents two workers from executing the same run at the same time. While task code is running, core records `run.lease_heartbeat` events so storage can extend worker ownership.

`leaseDuration` controls how long each claim or heartbeat keeps ownership alive. In Lambda, `runlane.executeDelivery(message, { leaseDuration: '2m' })` writes the durable marker that means "this run is held by this worker invocation until time T."

If the Lambda crashes or times out before persisting an outcome, the lease eventually expires and `tick()` can request delivery again for another consumer. That lease is the durable safety net behind the provider timeout.

Set `leaseDuration` longer than the consumer's maximum execution window. For example, a Lambda with a 1-minute timeout commonly uses a 2-minute Runlane lease for provider shutdown, storage writes, and recovery races.

Set the SQS visibility timeout longer than the same handling window so the provider does not redeliver before Runlane's own lease can protect the run.

`heartbeatInterval` controls how often core heartbeats during an active attempt. When omitted, core heartbeats halfway through the lease duration. A custom `heartbeatInterval` must be shorter than `leaseDuration`.

`maxAttemptDuration` controls the maximum wall-clock duration of one attempt. Core writes the fixed deadline when the lease is claimed. When the deadline passes, the task context signal is aborted and a cooperative handler outcome becomes `ErrorCode.TaskTimedOut`; maintenance can finalize a non-cooperative or crashed owner after the deadline. This is separate from `context.waitForSignal(..., { timeout })`, which is a release fallback and does not count as failure pressure.

If a heartbeat observes `CancellationRequested`, core aborts the task context signal and stops scheduling further heartbeats for that attempt. The handler still exits cooperatively; core then persists the handler's outcome against the current owned run if ownership is still valid.

If a heartbeat fails with `StorageConflict`, the attempt has lost ownership. Core aborts the task context signal, abandons that attempt result, and lets the run become recoverable through normal lease expiry. Other heartbeat or persistence failures, such as `StorageUnavailable`, reject the worker loop or delivery call because the runtime cannot prove ownership or persist the outcome.

## Cooperative Stop [#cooperative-stop]

Runlane does not hard-kill user code. Worker stop and external worker signals abort the `TaskContext.signal` passed to the handler and make `context.isCancellationRequested()` return `true`.

Task code should check the signal at safe points and finish in a way that matches the domain: return successfully after cleanup, throw a structured non-retryable error, or release the run to continue later. Core waits for the handler to cooperate before the worker closes.

Cancellation before acquisition differs by path:

| Path                                    | Already-aborted signal before claim                                                                    |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| `executeNext({ signal })`               | Returns `undefined`; no provider message has been acquired.                                            |
| `worker({ signal })` or `worker.stop()` | Stops the local worker loop and waits for active attempts to cooperate.                                |
| `executeDelivery(message, { signal })`  | Rejects with `OperationCancelled` so the transport does not acknowledge an unhandled provider message. |


# Runlane CLI (/configuration/cli)



`@runlane/cli` is the operational command surface for a runtime you already own. It does not discover tasks, lanes, storage, or transport by itself; a config module exports the runtime instance or a factory, and every runtime-backed command calls public runtime APIs such as `runlane.trigger()`, `runlane.tick()`, `runlane.worker()`, and `runlane.runs.*`.

```sh
pnpm add @runlane/cli @runlane/core @runlane/lane-local
```

## Config Module [#config-module]

The packaged `runlane` binary loads `runlane.config.ts`, `runlane.config.mts`, `runlane.config.mjs`, `runlane.config.js`, or `runlane.config.cjs` from the current working directory. Pass `--config` when the file lives somewhere else.

Keep this file as a thin CLI adapter. Do not duplicate lane construction, task registration, or environment defaults here if the application already owns them. Put the real runtime construction in one shared module and have the CLI config import that module.

For example, the application can own a shared runtime factory:

```ts
import { createRunlane, queue } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'

import { sendEmail } from './tasks/send-email.js'

const emailQueue = queue({ name: 'emails', default: true })

export function createAppRunlane() {
  return createRunlane({
    lane: createLocalLane(),
    queues: [emailQueue],
    tasks: { sendEmail },
  })
}
```

Then `runlane.config.ts` stays boring:

```ts
import { type RunlaneCliConfig } from '@runlane/cli'

import { createAppRunlane } from './runtime/runlane.ts'

export default {
  runtime: createAppRunlane,
} satisfies RunlaneCliConfig
```

TypeScript config files are loaded through `tsx`, so local development does not require a precompile step. Plain JavaScript config files still load through Node's normal module loader.

The exported `runtime` value may be a runtime object or an async factory. The CLI starts the runtime before each runtime-backed command and closes it afterward. The runtime must expose `start()`, `close()`, `tick()`, `trigger()`, `worker()`, `queues`, and `runs.cancel()`, `runs.events()`, `runs.get()`, `runs.list()`, `runs.prune()`, `runs.rerun()`, and `runs.retry()`. The `trigger` command also needs `createRunlane({ tasks })` in the configured runtime so it can resolve the task id passed on the command line.

If you want the CLI inside an existing app script instead of a config file, call `runRunlaneCli({ loadRuntime })` directly:

```ts
import { runRunlaneCli } from '@runlane/cli'

import { runlane } from './runtime.js'

const result = await runRunlaneCli({
  argv: process.argv.slice(2),
  loadRuntime: () => runlane,
})

process.exitCode = result.exitCode
```

Use `--json` before the command to emit machine-readable output:

```sh
runlane --json runs list --status failed --limit 20
```

| Switch                | Behavior                                                                                                                                                               |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-c, --config <path>` | Loads a config module from a non-default path for runtime-backed commands. Relative paths resolve from the current working directory.                                  |
| `--json`              | Emits command results and Runlane errors as JSON objects. Long-running lifecycle output is newline-delimited JSON. Command output goes to stdout; errors go to stderr. |

CLI commands normally run in the current Node process. When `runlane dev` runs against a runtime whose storage reports `processLocalState: true`, it opens a loopback control server and writes a config-keyed session file under the OS temp directory. Stateful operator commands such as `runlane trigger`, `runlane tick`, `runlane runs list`, `runlane runs get`, `runlane retry`, `runlane rerun`, `runlane cancel`, and `runlane prune` check for that matching dev session before loading their own runtime; when the session exists, the request is proxied into the live `runlane dev` process. This is what makes in-memory local lanes work across two terminals.

Shared-process lanes do not need that bridge. If storage state is reachable from another process, the command loads the configured runtime directly and calls the same public APIs in the current process.

The control server is a local development bridge, not a transport adapter. It asks the dev runtime to call `runlane.trigger()`, and core still owns run creation, outbox persistence, and wakeup publishing through the configured lane.

The session file is keyed by the resolved config path. If two `runlane dev` processes use the same config path, the most recently written session wins. Proxied commands also run against the task code already loaded in the dev process, so restart `runlane dev` after changing task definitions or runtime wiring.

## `runlane init config` [#runlane-init-config]

`init config` writes the thin config adapter that the packaged binary loads. It does not load or start a runtime.

```sh
runlane init config --runtime ./runtime/runlane.ts
```

The generated file imports an application-owned runtime value or factory and exports the CLI config contract:

```ts
import { type RunlaneCliConfig } from '@runlane/cli'

import { createAppRunlane } from './runtime/runlane.ts'

export default {
  runtime: createAppRunlane,
} satisfies RunlaneCliConfig
```

Switches:

* `--runtime <specifier>` is required. It is the module specifier imported from the generated config file.
* `--path <path>` writes to a different config path. The default is `runlane.config.ts`.
* `--runtime-export <name>` selects the export to import from the runtime module. The default is `createAppRunlane`; pass `default` for a default export.
* `--overwrite` replaces an existing config file.

Existing config files are not overwritten by default. If the target file exists, choose a different `--path` or pass `--overwrite` deliberately.

## `runlane trigger` [#runlane-trigger]

`trigger` creates one durable run for a task registered in the configured runtime's `createRunlane({ tasks })` collection.

```sh
runlane trigger emails.send '{"userId":"user_123"}'
runlane trigger reports.daily '{"date":"2026-05-22"}' --idempotency-key daily_report_2026_05_22
```

The payload argument is JSON. Omit it only when the task schema accepts `undefined`.

Human output distinguishes a newly created run from an idempotency hit. Run detail output includes the run status, task, queue, environment, timestamps, counters, keys, source, payload, output, and metadata when those fields exist:

```text
✓ created run_123
  status          queued
  task            emails.send
  queue           emails
  environment     default
  event sequence  1
  created         2026-05-23T20:15:26.000Z
  updated         2026-05-23T20:15:26.000Z
  attempts        0
  failures        0
  releases        0
  retries         0
  payload         {"userId":"user_123"}

⊝ returned existing run_123 (idempotency)
  status          queued
  task            emails.send
  queue           emails
  environment     default
  event sequence  1
  created         2026-05-23T20:15:26.000Z
  updated         2026-05-23T20:15:26.000Z
  attempts        0
  failures        0
  releases        0
  retries         0

hint  use runlane rerun run_123 to execute this work again
```

`--json` emits the structured `TriggerRunResult` with `run` and `outcome`.

When a matching `runlane dev` session is active for the same config path and the dev runtime uses process-local storage, this command proxies to that process before loading a new runtime. That is what makes the two-terminal local lane flow work:

```sh
# terminal 1
runlane dev

# terminal 2
runlane trigger emails.welcome '{"userId":"usr_demo_alice"}'
```

Switches:

* `--queue <queue>` overrides the task queue.
* `--run-id <runId>` supplies the run id.
* `--idempotency-key <key>` deduplicates producer retries for the same logical work.
* `--idempotency-key-ttl <duration>` sets terminal idempotency owner retention. It accepts a Runlane duration such as `30d` or `active`.
* `--singleton-key <key>` prevents overlapping active runs for the same logical resource.
* `--concurrency-key <key>` partitions bounded queue capacity.
* `--meta <json>` stores a JSON object on the run creation event.
* `--actor-id <id>` records the operator actor id.

## `runlane dev` [#runlane-dev]

`dev` starts a storage-polling worker and a maintenance loop in one process. It is intended for local development against a configured runtime.

```sh
runlane dev --queue emails --concurrency 2 --tick-interval 5s
```

`--once` runs one maintenance pass, drains currently due work, and exits. That is useful for scripts and smoke tests:

```sh
runlane dev --once --max-runs 25
```

Long-running `dev` uses `runlane.worker({ mode: WorkerMode.Poll })` and periodically calls `runlane.tick()`. `--once` runs `tick()`, starts a drain worker with `WorkerMode.Drain`, waits for that worker to close, and prints the tick summary plus worker stop line.

Human output for long-running `dev` is append-only lifecycle output. Proxied commands, executed runs, and maintenance ticks are printed as timestamped one-line records so terminal logs, CI captures, and redirected output stay readable:

```text
2026-05-23T20:15:26.000Z command trigger ✓ created run_123
2026-05-23T20:15:26.050Z run succeeded run_123 task=emails.welcome queue=emails attempts=1
2026-05-23T20:15:30.123Z tick materialized=0 deliveryRequested=2 cancellationsFinalized=0 timeoutsFinalized=0 outboxPublished=2 outboxFailed=0 outboxDeadLettered=0
```

For shared lanes such as Postgres, `runlane dev` is still a storage-polling worker plus maintenance loop. With a Postgres+SQS lane, this is a local development convenience, not a production topology: it can execute runs from Postgres, but it does not receive or delete SQS messages. Production Postgres+SQS deployments need an SQS consumer such as the Lambda helper or long-running SQS consumer. Production polling deployments should use a polling lane/config that does not publish SQS wakeups for queues with no SQS consumer.

`--once` does not run a second maintenance tick after the drain worker exits. Run `runlane tick` explicitly afterward when a script intentionally wants a follow-up maintenance pass for newly due recovery work.

Switches:

* `--once` runs one maintenance tick, drains due work, and exits.
* `--queue <queues...>` limits worker acquisition to one or more queues.
* `--concurrency <count>` sets the number of worker slots. The default is `1`.
* `--max-runs <count>` limits executed runs in `--once` mode.
* `--empty-poll-delay <duration>` sets the polling worker sleep when no work is due. The default is `100ms`. It is only valid without `--once`.
* `--tick-interval <duration>` sets the maintenance loop interval. The default is `5s`.
* `--schedule-materialization-limit <count>` bounds due schedule occurrence materialization per tick.
* `--delivery-request-limit <count>` bounds due delivery requests appended per tick.
* `--cancellation-finalization-limit <count>` bounds expired cancellations finalized per tick.
* `--timeout-finalization-limit <count>` bounds expired attempt deadlines finalized per tick.
* `--outbox-claim-limit <count>` bounds outbox rows claimed for publishing per tick.

Advanced:

* `--worker-id <id>` overrides the generated worker identity recorded on leases and outbox claims. It is for diagnostics, not routing; use `--queue` to choose which work this process handles.

## `runlane work` [#runlane-work]

`work` starts a core storage-acquisition worker. Poll mode is the default and keeps running until the process receives a stop signal.

```sh
runlane work --queue emails --concurrency 4 --empty-poll-delay 250ms
```

Drain mode processes currently due work and exits when storage is idle or `--max-runs` is reached. Human output prints `worker started`, timestamped `run ...` lines for executed runs, and `worker stopped`.

```sh
runlane work --drain --queue emails --max-runs 100
```

This command scans storage. It is not an SQS consumer and does not delete SQS provider messages. Use the SQS transport Lambda helper or long-running SQS consumer for transport-delivered wakeups.

Switches:

* `--drain` processes currently due work and exits when storage is idle or `--max-runs` is reached.
* `--queue <queues...>` limits acquisition to one or more queues.
* `--concurrency <count>` sets the number of worker slots. The default is `1`.
* `--max-runs <count>` limits executed runs in drain mode.
* `--empty-poll-delay <duration>` sets poll sleep when no work is due. The default is `100ms`. It is only valid without `--drain`.
* `--lease-duration <duration>` overrides the run lease duration.
* `--heartbeat-interval <duration>` overrides the run lease heartbeat interval.

Advanced:

* `--worker-id <id>` overrides the generated worker identity recorded on leases. It is for diagnostics, not routing; use `--queue` to choose which work this process handles.

## `runlane tick` [#runlane-tick]

`tick` runs one maintenance pass over registered schedules, due released or retrying runs, cancellation finalization, attempt-timeout finalization, and durable outbox publishing. Human output prints a `tick complete` card with counts for materialized schedules, delivery requests, finalized cancellations, finalized timeouts, and outbox publish results.

```sh
runlane tick --schedule-materialization-limit 100 --outbox-claim-limit 100
```

Bounds are optional. Omitted values use core defaults.

Switches:

* `--schedule-materialization-limit <count>` bounds due schedule occurrence materialization.
* `--delivery-request-limit <count>` bounds due delivery requests appended.
* `--cancellation-finalization-limit <count>` bounds expired cancellations finalized.
* `--timeout-finalization-limit <count>` bounds expired attempt deadlines finalized.
* `--outbox-claim-limit <count>` bounds outbox rows claimed for publishing.

## `runlane runs list` [#runlane-runs-list]

`runs list` reads operator run summaries in the configured runtime environment.

```sh
runlane runs list --status failed --queue emails --limit 50
runlane runs list --task emails.send --created-from 2026-05-01T00:00:00.000Z
```

Human output is an aligned table for scanning ids, statuses, tasks, queues, and due times. Use `--json` when another program needs the raw page and cursor.

Switches:

* `--cursor <cursor>` continues a previous operator read page.
* `--limit <count>` bounds returned run summaries.
* `--status <statuses...>` filters by durable run status: `cancellation_requested`, `cancelled`, `failed`, `queued`, `released`, `retrying`, `running`, `scheduled`, or `succeeded`.
* `--task <taskIds...>` filters by task id.
* `--queue <queues...>` filters by queue.
* `--source-run <runId>` filters by linked source run id.
* `--created-from <date>` and `--created-to <date>` filter `createdAt` by inclusive ISO date/time bounds.
* `--updated-from <date>` and `--updated-to <date>` filter `updatedAt` by inclusive ISO date/time bounds.
* `--run-at-from <date>` and `--run-at-to <date>` filter `runAt` by inclusive ISO date/time bounds.
* `--sort-by <field>` sorts by `created_at`, `updated_at`, or `run_at`.
* `--sort-direction <direction>` sorts `asc` or `desc`.

## `runlane runs get` [#runlane-runs-get]

`runs get` reads one materialized run. Add `--events` to include append-only event history for that run. A missing run returns a structured `run_not_found` error.

```sh
runlane runs get run_123
runlane runs get run_123 --events --limit 100
```

Event history is sorted by sequence ascending so the output follows reducer order.

Human output renders the run as an indented detail card and appends an event timeline when `--events` is present. Use `--json` for the raw run and event page. Without `--events`, JSON output contains the run only because undefined fields are omitted.

Switches:

* `--events` includes append-only run event history.
* `--cursor <cursor>` continues a previous run event page.
* `--limit <count>` bounds returned events.

## `runlane retry` [#runlane-retry]

`retry` creates a manual retry child from a failed source run. The source run remains terminal and unchanged.

```sh
runlane retry run_failed_123 --queue emails --actor-id ops@example.com
```

Use `--run-id` only when an operator process needs to supply a deterministic child run id.

Switches:

* `--queue <queue>` overrides the child run queue.
* `--run-id <runId>` supplies the child run id.
* `--actor-id <id>` records the operator actor id.

## `runlane rerun` [#runlane-rerun]

`rerun` creates a linked child from any terminal source run. It is for replaying completed work, not recovering a failure specifically.

```sh
runlane rerun run_succeeded_123 --queue emails
```

Like manual retry, rerun creates a new run and preserves the source run.

Switches:

* `--queue <queue>` overrides the child run queue.
* `--run-id <runId>` supplies the child run id.
* `--actor-id <id>` records the operator actor id.

## `runlane cancel` [#runlane-cancel]

`cancel` writes a durable cancellation request. Waiting runs become terminally cancelled. Running runs move to `cancellation_requested` and rely on cooperative task code observing `context.signal`.

```sh
runlane cancel run_123 --reason operator_requested --actor-id ops@example.com
```

Cancellation is not a process kill. If the active owner does not cooperate, maintenance finalizes cancellation after the cancellation-requested lease expires.

Switches:

* `--reason <reason>` records an operator cancellation reason.
* `--actor-id <id>` records the operator actor id.

## `runlane prune` [#runlane-prune]

`prune` deletes old terminal run history through storage adapters that report pruning support.

```sh
runlane prune --older-than 30d --status succeeded cancelled --limit 500
```

`--older-than` accepts a Runlane duration such as `30d` or an ISO date/time. Status filters must be terminal statuses. When output includes a continuation cursor, pass it back with the same cutoff and statuses:

```sh
runlane prune --older-than 30d --status succeeded cancelled --cursor cursor_123
```

Switches:

* `--older-than <duration-or-date>` is required. It accepts a Runlane duration such as `30d` or an ISO date/time cutoff.
* `--status <statuses...>` filters terminal statuses. Valid values are `cancelled`, `failed`, and `succeeded`.
* `--limit <count>` bounds pruned runs.
* `--cursor <cursor>` continues a previous prune page using the same cutoff and statuses.
* `--actor-id <id>` records the operator actor id.


# Configuration (/configuration)



Runlane configuration is code. There is no project-wide YAML file and no hidden task discovery step. Your application creates a runtime with `createRunlane()`, wires a lane, registers queues and tasks, and exports that same runtime to processes that need to trigger, execute, maintain, or inspect runs.

Use this page to choose the right surface before changing config.

| Surface                | Where it lives                                                                                                 | Use it for                                                                                           |
| ---------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| Runtime construction   | `createRunlane()` from `@runlane/core`                                                                         | Queues, tasks, environment, dispatch policy, clock, and default worker id.                           |
| Lane construction      | Lane packages such as `@runlane/lane-local`, `@runlane/lane-postgres-polling`, or `@runlane/lane-postgres-sqs` | Storage and delivery resource composition.                                                           |
| Adapter options        | Packages such as `@runlane/postgres-storage` and `@runlane/transport-sqs`                                      | Database connection, schema, SQS client, SQS queue bindings, FIFO options, and adapter batch limits. |
| CLI config             | `runlane.config.ts` or another module passed with `--config`                                                   | A thin adapter that exports the runtime instance or runtime factory used by `runlane` commands.      |
| Deployment environment | Your app, process manager, IaC, and secret store                                                               | Values such as `DATABASE_URL`, SQS queue URLs, AWS region, and runtime environment name.             |

## Runtime Construction [#runtime-construction]

`createRunlane()` is the main application configuration boundary. It requires a lane and an explicit queue registry:

```ts
import { createRunlane, queue } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'

import { sendEmail } from './tasks/send-email.js'

const emailQueue = queue({ name: 'emails', default: true })

export function createAppRunlane() {
  return createRunlane({
    environment: { name: 'development' },
    lane: createLocalLane(),
    queues: [emailQueue],
    tasks: { sendEmail },
  })
}
```

Runtime options are validated when the runtime is created. Unsupported options fail with `ErrorCode.ConfigurationInvalid`.

| Option               | Required | Default                    | Notes                                                                                                                             |
| -------------------- | -------- | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `lane`               | Yes      | None                       | Storage and delivery resource composition.                                                                                        |
| `queues`             | Yes      | None                       | Authoritative queue registry. Queue names must be unique.                                                                         |
| `tasks`              | No       | Empty registry             | Task collection used by workers, schedules, and CLI `trigger`. Prefer a named object catalog.                                     |
| `environment`        | No       | `{ name: 'default' }`      | Durable namespace used by storage, transport, idempotency, singleton, schedules, and operator reads.                              |
| `dispatch.onTrigger` | No       | Eager dispatch             | Controls whether transport-delivery `trigger()` calls try to publish new outbox rows immediately or leave publishing to `tick()`. |
| `clock`              | No       | Wall clock                 | Deterministic time source for tests and maintenance.                                                                              |
| `workerId`           | No       | Generated `worker_${uuid}` | Default diagnostic worker identity for leases and outbox claims.                                                                  |

Queue declarations belong in shared application code, not in the CLI config file. The same queue objects should be passed to `createRunlane({ queues })`, task definitions, and provider bindings such as `sqsQueue(queue, options)`.

See [Quick Start](../introduction/quick-start.mdx) for the runtime lifecycle and [Queues](../concepts/queues.mdx) for queue policies.

## Lane And Adapter Options [#lane-and-adapter-options]

A lane is the storage plus delivery mode the runtime uses. Local development normally uses `createLocalLane()` from `@runlane/lane-local`; it accepts only an optional diagnostic `name` and keeps state in the current process.

Production code usually has more adapter configuration. Use Postgres polling when workers should poll Postgres directly:

```ts
import { postgresPollingLane } from '@runlane/lane-postgres-polling'

const lane = postgresPollingLane({
  connectionString: process.env.DATABASE_URL,
  schema: 'runlane',
})
```

Use Postgres/SQS when a provider should deliver wakeups:

```ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { postgresSqsLane } from '@runlane/lane-postgres-sqs'
import { sqsQueue } from '@runlane/transport-sqs'

const lane = postgresSqsLane({
  postgres: {
    connectionString: process.env.DATABASE_URL,
    schema: 'runlane',
  },
  sqs: {
    client: new SQSClient({ region: process.env.AWS_REGION }),
    queues: [
      sqsQueue(emailQueue, {
        queueUrl: process.env.RUNLANE_EMAILS_QUEUE_URL,
      }),
    ],
  },
})
```

`postgresSqsLane()` validates its top-level shape, then delegates nested validation to the adapter packages:

| Package                          | Function                | Important options                                                                                    |
| -------------------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------- |
| `@runlane/lane-postgres-polling` | `postgresPollingLane()` | `connectionString`, optional `schema`, optional lane `name`.                                         |
| `@runlane/lane-postgres-sqs`     | `postgresSqsLane()`     | `postgres`, `sqs`, optional lane `name`.                                                             |
| `@runlane/postgres-storage`      | `postgresStorage()`     | `connectionString`, optional `schema`.                                                               |
| `@runlane/transport-sqs`         | `sqsTransport()`        | `client`, `queues`, optional `batchSize`, optional transport `name`.                                 |
| `@runlane/transport-sqs`         | `sqsQueue()`            | Exactly one of `queueUrl` or `queueName`, optional `queueOwnerAWSAccountId`, optional FIFO settings. |

Run Postgres migrations before starting a Postgres-backed lane. `lane.start()` probes the migrated tables and fails when the database, schema, or migration is not ready.

See [Postgres Polling Lane](../contracts/postgres-polling-lane.mdx), [Postgres SQS Lane](../contracts/postgres-sqs-lane.mdx), [Postgres Storage](../contracts/postgres-storage.mdx), and [SQS Transport](../contracts/sqs-transport.mdx) for adapter contracts and deployment notes.

## CLI Config [#cli-config]

The packaged `runlane` binary loads a config module from the current working directory. The default search order is:

1. `runlane.config.ts`
2. `runlane.config.mts`
3. `runlane.config.mjs`
4. `runlane.config.js`
5. `runlane.config.cjs`

Pass `--config <path>` before the command when the module lives somewhere else.

The config module exports a `RunlaneCliConfig` object with a `runtime` value or factory:

```ts
import { type RunlaneCliConfig } from '@runlane/cli'

import { createAppRunlane } from './runtime/runlane.ts'

export default {
  runtime: createAppRunlane,
} satisfies RunlaneCliConfig
```

Keep this file thin. It should import the application-owned runtime factory rather than rebuilding tasks, queues, lanes, or environment defaults in a second place.

`runlane init config --runtime ./runtime/runlane.ts` writes this adapter for you. It does not start a runtime or generate lane configuration.

See [Runlane CLI](./cli.mdx) for command behavior, `--json`, `runlane dev`, worker options, and operator commands.

## Environment Values [#environment-values]

Runlane does not prescribe an environment variable format. Keep secrets and deployment-specific values in your app's normal configuration system, then pass typed values into the runtime and lane factory.

Common values include:

| Value                    | Passed to                                                                                                                | Notes                                                                                                           |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
| Database URL             | `postgresStorage({ connectionString })`, `postgresPollingLane({ connectionString })`, or `postgresSqsLane({ postgres })` | Must use `postgres://` or `postgresql://`. A Prisma-style `?schema=` is accepted and removed before connecting. |
| Postgres schema          | `postgresStorage({ schema })`                                                                                            | Optional. Overrides `?schema=` from the URL; defaults to `public`.                                              |
| AWS region               | `new SQSClient({ region })`                                                                                              | Owned by the AWS SDK client you pass to SQS transport.                                                          |
| SQS queue URL or name    | `sqsQueue(queue, options)`                                                                                               | Each Runlane queue must be bound to exactly one provider queue source.                                          |
| Runtime environment name | `createRunlane({ environment })`                                                                                         | Use separate names for durable namespaces that must not share runs, schedules, or idempotency ownership.        |

Do not split one logical runtime across mismatched environment names, queue definitions, or SQS bindings. `createRunlane()` and the transport adapters validate many of these mismatches at startup, but shared constants make them easier to avoid.


# Adapter Authoring (/contracts/adapter-authoring)



Runlane lanes are composed from one storage adapter and an explicit delivery mode. Storage owns durable truth. Polling workers discover runnable work through storage leases; transport delivery adds an external wakeup publisher. A lane package wires compatible pieces together, drives optional lifecycle hooks, and reports the composed capabilities.

{/* @skip-typecheck - contract excerpt, not runnable code */}

```ts
interface Lane {
  readonly name: string
  readonly capabilities: LaneCapabilities
  readonly delivery: LaneDelivery
  readonly storage: StorageAdapter
  start?(): Promise<void>
  close?(): Promise<void>
}

type LaneDelivery =
  | { readonly mode: 'storage_polling' }
  | { readonly mode: 'transport'; readonly transport: TransportAdapter }
```

## Storage [#storage]

Implement `StorageAdapter` when you are building a reusable durable backend such as Postgres, MySQL, DynamoDB, SQLite, or Redis.

```ts
import type { StorageCapabilities } from '@runlane/contracts'

export const postgresStorageCapabilities = {
  claimsScheduleOccurrences: true,
  durableState: true,
  durableSteps: true,
  exportsObservations: true,
  enforcesIdempotency: true,
  enforcesQueueConcurrency: true,
  enforcesSingleton: true,
  leasesRuns: true,
  persistsOutbox: true,
  processLocalState: false,
  prunesRuns: true,
  readsRunHistory: true,
} satisfies StorageCapabilities
```

The important rule is the append boundary: core owns reducer semantics; storage owns atomic persistence.

`durableState` and `processLocalState` answer different questions. `durableState: false` means process loss can lose committed state. `processLocalState: true` means another process cannot construct an equivalent adapter and reach the same state. Redis without persistence can be non-durable but shared across processes; in-memory local storage is both non-durable and process-local.

Report capability flags as public promises, not implementation wishes. Core and operator APIs use these flags to decide whether runtime construction, maintenance, and operator commands are valid.

Storage capability flags:

| Capability                  | Meaning                                                                                          |
| --------------------------- | ------------------------------------------------------------------------------------------------ |
| `durableState`              | Committed storage state survives adapter process loss.                                           |
| `durableSteps`              | Named step completions can be persisted and replayed inside one run.                             |
| `exportsObservations`       | Storage can scan sanitized durable observation records and maintain exporter checkpoints.        |
| `processLocalState`         | Storage state is only reachable from the process that owns this adapter instance.                |
| `readsRunHistory`           | Operator run and event reads are supported.                                                      |
| `prunesRuns`                | Terminal run pruning is supported.                                                               |
| `leasesRuns`                | Workers can claim and heartbeat run leases through storage.                                      |
| `claimsScheduleOccurrences` | Maintenance can claim schedule occurrences safely.                                               |
| `persistsOutbox`            | Storage can create storage-backed outbox rows when transport delivery requires external wakeups. |
| `enforcesIdempotency`       | Storage enforces task-scoped idempotency ownership.                                              |
| `enforcesSingleton`         | Storage enforces singleton ownership.                                                            |
| `enforcesQueueConcurrency`  | Storage enforces bounded queue capacity.                                                         |

`appendRunEvents()` receives append-only events and the core-projected `RunRecord`. It verifies `environment`, `runId`, and `expectedSequence`, persists the events and supplied projection, and creates outbox rows for any `run.delivery_requested` events in the same transaction when the command's `persistDeliveryOutbox` option is `true` or omitted. Storage-polling lanes pass `false` so delivery intent stays in run state without creating provider outbox rows.

Append commands include a required `expectedSequence`; callers use the current `RunRecord.eventSequence` or `contractDefaults.run.newEventSequence` for a new run.

Storage adapters may expose `start()` and `close()` hooks for pools, embedded engines, or external clients. Lane/runtime code starts storage before transport delivery and closes transport delivery before storage.

`StorageAdapter.start()` and `StorageAdapter.close()` are optional lifecycle hooks. Every other `StorageAdapter` method below is required by the contract and must exist on the adapter instance. When a required method is guarded by an unsupported capability, it should fail fast with `RunlaneError` and `ErrorCode.CapabilityUnsupported`; it must not silently no-op or return an empty success value.

| Method                                      | Contract                                                                                                                                                                                             |
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `appendRunEvents()`                         | Atomically persist supplied run events, the supplied projected run, and requested outbox rows implied by `run.delivery_requested` events after verifying environment, run id, and expected sequence. |
| `getRun()`                                  | Return the current materialized run for one environment/run id, or `undefined` when it is absent.                                                                                                    |
| `getRuns()`                                 | Batch-read current materialized runs for one environment in caller-requested order while omitting missing runs.                                                                                      |
| `getRunStep()`                              | Return one completed durable step for a run and step key, or `undefined` when it is absent.                                                                                                          |
| `completeRunStep()`                         | Atomically complete one durable step for the current run lease and attempt, returning the stored step record.                                                                                        |
| `getRunByIdempotencyKey()`                  | Return the active or retained terminal run owning one task-scoped idempotency key, or `undefined` when no retained owner exists.                                                                     |
| `resetIdempotencyKey()`                     | Clear one retained terminal idempotency-key owner, rejecting active owners.                                                                                                                          |
| `listRuns()`                                | Page operator run summaries with opaque cursor semantics, stable ordering, and filters scoped to the requested environment.                                                                          |
| `listRunEvents()`                           | Page durable run history records without losing event order; sequence sorting is meaningful only inside one run.                                                                                     |
| `listRunnableRuns()`                        | Return compact due-work candidates for storage polling; do not return payload-bearing run records.                                                                                                   |
| `listRunsNeedingDispatch()`                 | Return bounded-queue runs that are due and need a dispatch reservation.                                                                                                                              |
| `listRunsNeedingCancellationFinalization()` | Return cancellation-requested runs whose current lease has expired so maintenance can append terminal cancellation.                                                                                  |
| `listRunsNeedingTimeoutFinalization()`      | Return running runs whose current lease has recorded `run.started` and whose fixed attempt deadline has passed so maintenance can append timeout failure or retry.                                   |
| `listRunsNeedingDelivery()`                 | Return runs that need fresh delivery recovery, excluding queued runs that already have outbox-backed delivery intent.                                                                                |
| `listRunsWaitingForSignal()`                | Return released runs waiting on one signal key so `runlane.signals.send()` can request delivery.                                                                                                     |
| `listRunsWaitingForRunCompletion()`         | Return released runs waiting on one source run id so terminal source completion can request delivery.                                                                                                |
| `reserveRunDispatch()`                      | Atomically reserve bounded queue capacity, append the supplied `run.delivery_requested` event, and create the outbox row when requested.                                                             |
| `claimRunLease()`                           | Atomically claim an executable run attempt using the core-projected lease events and return the updated run only when ownership was acquired.                                                        |
| `heartbeatRunLease()`                       | Atomically append the core-projected heartbeat event for the current lease owner and return the updated run.                                                                                         |
| `releaseRunLease()`                         | Clear the current owner's lease after `appendRunEvents()` has already persisted the terminal or waiting lifecycle event and projected inactive run.                                                  |
| `claimScheduleOccurrence()`                 | Claim one schedule occurrence by deterministic occurrence id and fire time, returning `undefined` when another owner already holds or completed it.                                                  |
| `completeScheduleOccurrence()`              | Complete a claimed schedule occurrence with the runs it materialized, enforcing the claim token.                                                                                                     |
| `claimOutboxMessages()`                     | Claim due outbox rows for publishing, optionally restricted to specific message ids, and return only rows owned by the new claim.                                                                    |
| `markOutboxMessagesPublished()`             | Batch-ack successfully published claimed outbox rows.                                                                                                                                                |
| `markOutboxMessagesFailed()`                | Batch-record retryable publish failures and next availability for claimed outbox rows.                                                                                                               |
| `markOutboxMessagesDeadLettered()`          | Batch-record terminal publish failures for claimed outbox rows.                                                                                                                                      |
| `scanObservationRecords()`                  | Page sanitized durable observation records in stable forward order by environment and cursor.                                                                                                        |
| `getObservationCheckpoint()`                | Read one exporter consumer's durable checkpoint for an environment.                                                                                                                                  |
| `advanceObservationCheckpoint()`            | Advance one exporter consumer checkpoint only from the expected previous cursor and only forward.                                                                                                    |
| `pruneRuns()`                               | Remove terminal runs selected by the prune command and return bounded progress with an opaque continuation cursor when more work remains.                                                            |

### Storage concurrency guarantees [#storage-concurrency-guarantees]

Adapter correctness depends on the guarantees below, not on any particular database primitive. Postgres uses advisory locks, row locks, generated columns, and `ON CONFLICT`, but another backend can satisfy the same contract with conditional writes, compare-and-swap transactions, serializable partitions, leases, or single-writer streams. The important rule is the observable behavior under competing callers.

`appendRunEvents()` must compare the current persisted event sequence with `expectedSequence` before validating projected run invariants. If it does not match, reject with `RunlaneError`, `ErrorCode.StorageConflict`, and `StorageConflictKind.EventSequence`. This preserves optimistic-concurrency recovery: a stale append is a lost race, not an adapter contract violation.

When an append succeeds, these records commit atomically:

* event-history rows
* materialized projection, including `RunRecord.output` when `run.succeeded` carries JSON output, `RunRecord.wait` while released, and `RunRecord.attemptDeadlineAt` while a running attempt has a deadline
* storage-owned idempotency or singleton ownership rows
* derived outbox rows requested by the command
* derived observation records when `exportsObservations` is supported

The `RunEventRecord` values returned from `appendRunEvents()`, `claimRunLease()`, and `heartbeatRunLease()` must be the exact records persisted to history, including ids and sequence numbers. Do not generate one event id for storage and a second event id for the method result.

Idempotency owner creation is a serialized ownership decision for `(environment, taskId, idempotencyKey)`.

| Race outcome                             | Required behavior                                                                              |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------- |
| One run creates the live owner           | That owner wins.                                                                               |
| Another run attempts the same live owner | Reject with `StorageConflictKind.IdempotencyKey` so core can read and return the original run. |
| Adapter uses upsert                      | It must not overwrite a concurrent owner.                                                      |

A valid implementation may lock the partition, conditionally insert then read back the owner, or use a transaction primitive that guarantees one owner.

Idempotency retention uses the policy captured at owner creation:

* active owners never expire while active
* with a finite TTL, successful and cancelled terminal owners remain readable until their captured TTL expires or `resetIdempotencyKey()` clears them
* with `ttl: "active"`, ownership is released when the run reaches any terminal state
* failed owners are released

`getRunByIdempotencyKey()` is side-effect free: an expired owner returns `undefined`; cleanup can happen on a later write, reset, or prune path.

Bounded queue capacity is partitioned by `(environment, queue, concurrencyKey)`. Absent concurrency keys share one partition.

`listRunnableRuns()` and `listRunsNeedingDispatch()` should apply capacity predicates before returning candidates so a capacity-blocked early row does not starve runnable work in another partition.

`claimRunLease()` and `reserveRunDispatch()` must enforce the same capacity partition inside the write operation. A scan-time check alone is not enough, and row-locking only the candidate run is not enough when two different runs in the same partition can be claimed or reserved concurrently.

`reserveRunDispatch()` is the atomic bounded-queue delivery path. In one operation, it:

1. Checks the current sequence.
2. Confirms the run is still dispatchable.
3. Verifies partition capacity.
4. Appends the supplied `run.delivery_requested` event.
5. Writes the projected reservation.
6. Creates the outbox row when the command requests transport wakeup persistence.

A stale sequence, expired/ineligible dispatch state, or full capacity returns `undefined`; malformed command data remains an adapter contract violation.

`claimRunLease()` is a race-tolerant ownership operation. A stale sequence, expired dispatch reservation, or full bounded queue returns `undefined` so competing workers can keep polling without treating normal contention as a framework error. `heartbeatRunLease()` is stricter because the caller believes it already owns the run: stale sequence is `StorageConflictKind.EventSequence`, and lost lease ownership is `StorageConflictKind.LeaseOwnership`.

Outbox mutation methods are batch-only by contract.

`claimOutboxMessages()` atomically transitions only due rows whose status is `pending`, `failed`, or `claimed` with an expired claim into a new claim, and returns the exact rows now owned by that claim. When `outboxMessageIds` is provided, do not widen the claim into unrelated backlog rows.

`markOutboxMessagesPublished()`, `markOutboxMessagesFailed()`, and `markOutboxMessagesDeadLettered()` must only mutate messages whose current claim token still matches the command. A stale or missing claim rejects with `StorageConflictKind.OutboxClaim`.

Promise-returning adapter methods must report validation, contract, conflict, and backend failures as rejected promises. If an adapter implementation performs synchronous validation, wrap it so callers never observe a synchronous throw from a Promise-typed method.

Records returned from storage must be defensive copies. Mutable `Date` instances and nested objects must not share references with adapter-owned state or command input.

Successful task output is storage-owned durable run state, not transport state. Store it in canonical `run_json`, preserve `run.succeeded.output` in event history, and keep any indexed output column such as Postgres `output_json` synchronized from the same projection. Failed, retrying, released, and cancelled runs must not expose current success output.

Durable step output is storage-owned checkpoint state keyed by `(environment, runId, stepKey)`. `getRunStep()` is a side-effect-free read. `completeRunStep()` must validate the active lease token, the active attempt number, and the run status before inserting the step. Lost ownership rejects with `StorageConflictKind.LeaseOwnership`. Replaying completion with the same JSON output is idempotent and returns the stored row with `completed: false`; replaying the same step key with different output is `ErrorCode.InvariantViolation`. Do not infer steps from run events, release metadata, or task output.

### Observation export stream [#observation-export-stream]

The durable observation stream is a storage-owned export boundary over committed Runlane facts. Adapters that report `exportsObservations: true` must record sanitized observation records for persisted run events and newly completed durable steps. Adapters that report `exportsObservations: false` must still expose the required methods, but each method should reject with `ErrorCode.CapabilityUnsupported`.

Observation records must be committed with the source fact. A successful `appendRunEvents()`, `claimRunLease()`, `heartbeatRunLease()`, or `reserveRunDispatch()` write that returns run event records must also make those events visible to `scanObservationRecords()`. A successful first-time `completeRunStep()` must make the step visible to the stream; an idempotent replay returning `completed: false` must not create a second observation record.

Observation record ids are duplicate-safe. The same durable run event id maps to the same run-event observation record id, and the same durable step token maps to the same run-step observation record id. The stream payload must stay sanitized: no task payload, successful output, durable step output, failure detail, release metadata, or operator-expanded run rows by default.

`scanObservationRecords({ environment, cursor, limit })` returns records after the supplied cursor in stable forward order. Storage owns the cursor encoding and should clamp limits with `contractDefaults.pagination`. The returned `cursor` is the checkpoint value after the final returned record; exporters store it only after their sink accepts the batch. Observation `streamPosition` values are opaque strings at the contract boundary so adapters can use database-native cursor precision internally.

`getObservationCheckpoint()` and `advanceObservationCheckpoint()` are scoped by `(environment, consumer)`. `advanceObservationCheckpoint()` must compare the stored checkpoint with `expectedCursor` and reject stale writers with `StorageConflictKind.ObservationCheckpoint`. It must also reject non-monotonic advances and cursors that do not reference an existing stream record in the same environment, so a crashed, slow, or malformed exporter cannot move the checkpoint backward or skip unexported records.

Released run waits are storage-owned durable run state too. Preserve `RunRecord.wait` from the supplied projection, index the durable condition needed for signal and run-completion scans, and clear it only when the projected lifecycle event clears it. Do not infer signal or source-run waits from reason strings or metadata.

Attempt deadlines are fixed at lease claim. Preserve `RunLeaseClaimedEvent.attemptDeadlineAt` and the matching `RunRecord.attemptDeadlineAt`; heartbeat writes must not move it. Terminal, retry, release, cancellation, and delivery-recovery projections clear the field.

`listRunnableRuns()` returns compact `RunnableRunRef` candidates for worker acquisition. It must not return payload-bearing `RunRecord` values.

Workers use the refs to pick candidates, read the full run, and then supply core-projected lease events to `claimRunLease()`. Use `getRunRunnableAvailableAt()` for the due-work predicate and ordering timestamp, and use `runStatusValues.isRunnableCandidate()` when adapter code needs status narrowing.

`listRunsNeedingDelivery()` is the maintenance query for fresh delivery requests.

| Must include                                                               | Must exclude                                             | Predicate helper                      |
| -------------------------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------- |
| Due `scheduled`, `released`, `retrying`, plus expired-lease `running` runs | `queued` runs, because they already have delivery intent | `getRunDeliveryRecoveryAvailableAt()` |

Use `runStatusValues.isDeliveryRecoveryCandidate()` when adapter code needs to narrow the compact ref status. Maintenance then calls `getRuns()` once for the selected ids, appends `run.delivery_requested`, and asks storage to create a new outbox row only for transport delivery lanes.

`listRunsNeedingDispatch()` and `reserveRunDispatch()` are the bounded-queue dispatch path. Queue capacity must be enforced inside the storage transaction because multiple schedulers and workers can race.

Capacity counts currently running runs plus unexpired dispatch reservations in the same environment, queue, and `concurrencyKey` partition. When no `concurrencyKey` is present, the whole queue is one partition.

Dispatch reservations use `dispatchExpiresAt`; if no worker claims the run before that time, maintenance may reserve and publish it again.

`listRunsNeedingCancellationFinalization()` is the cancellation maintenance query. It returns only `cancellation_requested` runs whose current lease has expired.

Use `getRunCancellationFinalizationAvailableAt()` for the predicate and `runStatusValues.isCancellationFinalizationCandidate()` for status-only narrowing. Maintenance then reads current projections with `getRuns()` and appends `run.cancelled`; it does not wake workers for cancellation finalization.

`listRunsNeedingTimeoutFinalization()` is the attempt-deadline maintenance query. It returns only `running` runs whose `attemptDeadlineAt` is at or before the requested time.

Use `getRunAttemptTimeoutFinalizationAvailableAt()` for the predicate. Maintenance then reads current projections with `getRuns()` and appends either `run.retry_scheduled` or `run.failed` with `ErrorCode.TaskTimedOut` according to the registered task retry policy.

`listRunsWaitingForSignal()` and `listRunsWaitingForRunCompletion()` are event-driven resume scans. They return compact refs for released runs whose current `wait` condition exactly matches the requested signal key or source run id. They must not return runs that are no longer released, no longer have the matching wait condition, or are merely time-released.

`getRunByIdempotencyKey()` is the recovery read behind retained idempotent `trigger()` semantics. Idempotency ownership is scoped by environment, task id, and key.

Retention rules:

* active owners never expire while active
* successful and cancelled terminal owners with a finite captured `idempotencyKeyTTL` remain until that TTL expires or `resetIdempotencyKey()` clears them
* owners captured with `idempotencyKeyTTL: "active"` are released on terminal state
* failed owners are cleared automatically

Storage must use its authoritative owner table for this lookup, not operator pagination.

`pruneRuns()` receives terminal statuses and a concrete `olderThan: Date` from core and must never delete active runs. Core resolves public duration strings before calling storage and freezes that cutoff into public continuation cursors.

When the command omits `limit`, storage uses `contractDefaults.pruning.batchLimit`. When storage cannot finish in one bounded call, it returns `nextCursor`; core passes the adapter cursor back on the next prune command with the same retention filter.

Ownership tokens are part of the contract. Lease tokens are supplied in the core-projected lease events passed to `claimRunLease()` and `heartbeatRunLease()` so storage does not compute run projections. Schedule occurrence and outbox claim tokens are storage-generated, and callers return those tokens for schedule completion, publish, failure, or dead-letter updates.

Adapter indexes should treat Runlane ids as opaque strings. Public Runlane ids are non-empty and reserve `:` for backend-internal key composition; adapters should still avoid parsing prefixes or separators for behavior.

Schedule occurrence ids and generated schedule run ids use deterministic hashed scope data and may be longer than random run ids. Do not parse them to recover environment, schedule, or fire time.

Outbox mutations are batch-only so networked adapters can persist publish results without one round-trip per row.

| Method                                          | Required behavior                                                                    |
| ----------------------------------------------- | ------------------------------------------------------------------------------------ |
| `markOutboxMessagesPublished()`                 | Acknowledge successful publishes.                                                    |
| `markOutboxMessagesFailed()`                    | Record retryable failed attempts and optional `nextAvailableAt`.                     |
| `markOutboxMessagesDeadLettered()`              | Move poisoned rows to `OutboxMessageStatus.DeadLettered` with final failure records. |
| `claimOutboxMessages()` with `outboxMessageIds` | Claim only those rows; do not widen into an unrelated backlog sweep.                 |

`OutboxFailureRecord.code` must be an `ErrorCode`. Provider-specific response codes belong in `meta` so operator tooling can branch on Runlane's stable vocabulary.

Run events are durable replay records. Existing event types stay backward compatible: add optional fields with reducer defaults, or introduce a new `RunEventType` when new required data is needed.

Lease heartbeats are real events in v1. Adapters must preserve observable sequence and history semantics for `listRunEvents()`. Physical compaction is only valid if callers still see the same ordered event records, or if a future compaction contract explicitly changes that behavior.

Use `environmentKey(environment)` for durable scoping and uniqueness indexes. Do not build parallel environment key logic from `environment.name`; future environment identity fields must widen the same contract everywhere.

Operator list APIs use opaque cursors. Storage should apply `contractDefaults.pagination`, and include filter and sort state in cursor semantics so a cursor cannot silently resume a different ordering. Event `sequence` sorting is only valid inside one run, so adapters should require `runId` when callers request `RunEventSortField.Sequence`.

## Transport [#transport]

Implement `TransportAdapter` when you are building a reusable wakeup backend such as SQS, Redis, RabbitMQ, Pub/Sub, Kafka, or HTTP push.

Transport should carry the minimum needed to wake a worker. It does not own payloads, successful output, failure detail, or materialized run state.

```ts
import { asId, type DeliveryMessage } from '@runlane/contracts'

const wakeup = {
  environment: { name: 'production' },
  queue: asId<'queue'>('emails'),
  requestedAt: new Date(),
  runId: asId<'run'>('run_123'),
} satisfies DeliveryMessage
```

Workers use the wakeup `environment` and `runId` to read current durable state from storage before executing. Duplicate or delayed transport messages are safe only when storage remains the durable truth boundary.

Transport capability flags:

| Capability        | Meaning                                                                                                                                    |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `durableDelivery` | Provider-accepted wakeups survive adapter process loss until the provider delivers or redrives them.                                       |
| `messageGrouping` | The adapter can group wakeups for provider-level FIFO or partition semantics.                                                              |
| `nativeDelay`     | The provider can hold future wakeups natively; Runlane's first-party adapters currently use storage due times and outbox recovery instead. |
| `orderedDelivery` | The adapter can preserve provider-level order for the configured queue set.                                                                |

If a transport exposes `queues`, runtime construction verifies that every runtime queue has a matching provider binding and policy. Omit `queues` only for transports whose binding cannot be described as static Runlane queue definitions.

Transport-driven execution uses `runlane.executeDelivery(message)`, not a drain worker. Drain asks storage what work is due; delivery execution uses the transport-acquired `runId`, verifies the stored run is still runnable for that queue, and returns an ignored result for ack-safe stale wakeups such as terminal, not-due, wrong-queue, already-leased, claim-lost, or abandoned attempts.

Reusable transport adapters should stay contract-only and should not import core. Provider-specific runtime helpers, such as an SQS/Lambda bridge, can parse provider records into `DeliveryMessage` values and then call `executeDelivery()`.

Persisted user task outcomes are not transport failures. Storage, projection, registration, and persistence failures should propagate so the provider integration can retry or report batch item failure.

Transport publish commands receive claimed outbox attempts keyed by `outboxMessageId` and `claimToken`. A successful `publishWakeups()` return must include `outcomes[index]` for `command.attempts[index]`, with each outcome tagged by `WakeupPublishOutcomeType.Published` or `WakeupPublishOutcomeType.Failed`.

Throw `TransportUnavailable` or operation-level `TransportPublishFailed` when the adapter cannot produce trustworthy per-attempt outcomes; core records that operation-level failure against every claimed attempt. Do not echo ids or split successful and failed rows into parallel arrays.

Use `publishWakeupsCommandSchema` when an adapter or adapter test intentionally validates a publish command. Do not use provider serialization or clone heuristics as a substitute for the Runlane command contract.

Transport adapters may expose `start()` and `close()` hooks for clients, sockets, or subscriptions. They must not own durable run state even when the transport backend is itself durable.

## Lane Packages [#lane-packages]

A lane package composes storage and an explicit delivery mode with `createLane()` from `@runlane/contracts`. The helper validates the supplied adapters, reports composed capabilities, and uses the standard lifecycle order: start storage before transport delivery, close transport delivery before storage.

`createLane()` is a composition boundary, not a capability shim. It keeps the original adapter instances, exposes `storage.capabilities` through `lane.capabilities.storage`, and adds only lane-level metadata and lifecycle ordering. It does not make an adapter durable, add operator reads, emulate unsupported storage behavior, or invent a fake transport for polling lanes.

| Option              | Required | Default  | Meaning                                                                                                                                                                                                  |
| ------------------- | -------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `storage`           | Yes      | None     | Complete `StorageAdapter` instance used as the lane's durable truth boundary.                                                                                                                            |
| `delivery`          | Yes      | None     | Explicit delivery mode. Use `{ mode: LaneDeliveryMode.StoragePolling }` for storage-scanning workers or `{ mode: LaneDeliveryMode.Transport, transport }` when the lane should publish external wakeups. |
| `name`              | No       | `"lane"` | Human-readable lane name for diagnostics and operator surfaces.                                                                                                                                          |
| `operatorReads`     | No       | `true`   | Whether the composed lane exposes operator-facing reads through storage. Set this honestly for the packaged lane.                                                                                        |
| `productionDurable` | No       | `false`  | Whether the composed lane is safe as a production persistence and delivery boundary. Set `true` only when the selected storage and delivery mode actually provide that guarantee.                        |

Malformed options or incomplete adapters fail fast with `RunlaneError` and `ErrorCode.ConfigurationInvalid`. Unsupported option names are rejected instead of ignored so lane packages do not grow accidental shadow configuration.

```ts
import { createLane, LaneDeliveryMode, type Lane, type StorageAdapter, type TransportAdapter } from '@runlane/contracts'

export interface MyLaneOptions {
  readonly name?: string
  readonly operatorReads?: boolean
  readonly productionDurable?: boolean
  readonly storage: StorageAdapter
  readonly transport?: TransportAdapter
}

export function myLane(options: MyLaneOptions): Lane {
  return createLane({
    delivery:
      options.transport === undefined
        ? { mode: LaneDeliveryMode.StoragePolling }
        : { mode: LaneDeliveryMode.Transport, transport: options.transport },
    name: options.name ?? 'my-lane',
    operatorReads: options.operatorReads ?? options.storage.capabilities.readsRunHistory,
    productionDurable: options.productionDurable ?? false,
    storage: options.storage,
  })
}
```

## Errors [#errors]

Adapters throw `RunlaneError` with stable `ErrorCode` values. Raw driver errors can be attached as `cause` for logs, but public callers should branch on `error.code`.

Use the narrowest code that describes the boundary that failed:

| Code                                          | Use                                                                                                                                                                                       |
| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CapabilityUnsupported`                       | A method is not supported by the adapter's reported capabilities.                                                                                                                         |
| `ValidationFailed`                            | Public query input is invalid, such as malformed cursors, non-positive limits, or sequence sorting without a `runId` filter.                                                              |
| `StorageConflict`                             | Optimistic concurrency, idempotency ownership, singleton ownership, lease ownership, observation checkpoint ownership, outbox claim ownership, or schedule occurrence ownership is stale. |
| `AdapterContractViolation`                    | The caller supplied an internally inconsistent command, or the adapter returned data that violates its contract.                                                                          |
| `RunNotFound` / `ScheduleNotFound`            | A command requires an existing record that is missing.                                                                                                                                    |
| `StorageUnavailable` / `TransportUnavailable` | A backend outage prevents the operation. Attach the raw driver failure as `cause` for server-side logs.                                                                                   |
| `TransportPublishFailed`                      | The provider rejected a publish attempt but the transport backend is reachable. Include provider request ids or provider error codes in metadata.                                         |
| `ObservationExportFailed`                     | A durable observation sink rejected a batch or the telemetry export boundary failed. Include provider status or retry hints in metadata.                                                  |

Create storage conflicts with `createStorageConflictError({ kind: StorageConflictKind.* })`. Do not hand-roll bare `RunlaneError` storage conflicts; that drops the `storageConflictKind` metadata core and tooling use to distinguish expected races from real storage failures.

Do not expose raw driver messages as the public error contract. Callers should be able to handle adapter failures by code, retryability, and structured metadata without parsing provider text.

## Conformance [#conformance]

Adapter packages should import the shared conformance suites from `@runlane/testing` and provide a fresh adapter or lane resource per test. Vitest packages can use `@runlane/testing/vitest`; other runners can import `defineStorageConformanceSuite()`, `defineTransportConformanceSuite()`, or `defineLaneCompositionConformanceSuite()` from `@runlane/testing` and pass the resulting suite to `runConformanceSuite()` with a runner object that supplies `describe` and `test`.

Conformance covers one primitive boundary at a time: storage conformance for durable run truth, transport conformance for wakeup publishing, and lane composition conformance for wiring compatible delivery mode into a `Lane`. Keep backend-specific tests beside the adapter for migrations, SQLSTATE or SDK error mapping, connection lifecycle, and operational behavior the generic contract cannot observe.


# Core Reducer (/contracts/core-reducer)



`@runlane/core` owns run lifecycle semantics. The low-level public primitive is `projectRunEvents()`, a pure reducer that turns append-only `RunEvent` values into the `RunRecord` projection storage adapters persist.

Most application code should never call this directly. Runtime APIs such as `trigger()`, `runNow()`, `worker()`, `tick()`, and `runs.cancel()` call it before they ask storage to append lifecycle events. Adapter authors need the contract because storage must persist the supplied events and supplied projection atomically without reimplementing status, counter, lease, retry, release, or cancellation rules.

## Short example [#short-example]

The reducer can create a run from its first durable event:

```ts
import { asId, contractDefaults, RunEventType } from '@runlane/contracts'
import { projectRunEvents } from '@runlane/core'

const environment = { name: 'production' }
const runId = asId<'run'>('run_123')
const now = new Date()

const projectedRun = projectRunEvents({
  expectedSequence: contractDefaults.run.newEventSequence,
  events: [
    {
      actor: contractDefaults.actor.system,
      environment,
      occurredAt: now,
      payload: { userId: 'user_123' },
      queue: contractDefaults.queue,
      runId,
      taskId: asId<'task'>('emails.send'),
      type: RunEventType.Created,
    },
  ],
})
```

The returned `RunRecord` has `eventSequence: 1`, zeroed counters, `status: "queued"`, and the durable identity fields copied from the created event.

Appending later events starts from the current materialized run:

```ts
const updatedRun = projectRunEvents({
  currentRun: projectedRun,
  expectedSequence: projectedRun.eventSequence,
  events: [
    {
      actor: contractDefaults.actor.system,
      delivery: {
        availableAt: now,
        environment,
        queue: contractDefaults.queue,
        requestedAt: now,
        runId,
      },
      environment,
      occurredAt: now,
      runId,
      type: RunEventType.DeliveryRequested,
    },
  ],
})
```

## Sequence contract [#sequence-contract]

`expectedSequence` must match the current run sequence. Use `contractDefaults.run.newEventSequence` when creating a run and the current `RunRecord.eventSequence` when appending to an existing run. A stale sequence throws `RunlaneError` with `ErrorCode.StorageConflict` so callers can retry the read-project-append operation.

The reducer assigns projected sequences in memory as `expectedSequence + event index + 1`. Storage still owns durable event ids and must reject appends if its current sequence no longer matches the command.

Projection failures use two different error classes:

| Condition                                                                        | Error                                                          |
| -------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| `expectedSequence` is stale against `currentRun.eventSequence`                   | `RunlaneError`, `ErrorCode.StorageConflict`, `retryable: true` |
| `expectedSequence` is negative, non-integer, or unsafe                           | `RunlaneError`, `ErrorCode.InvariantViolation`                 |
| `events` is empty                                                                | `RunlaneError`, `ErrorCode.InvariantViolation`                 |
| Event order, status, lease, attempt, queue, run id, or environment is impossible | `RunlaneError`, `ErrorCode.InvariantViolation`                 |

Adapters should preserve this distinction. A stale sequence is a normal lost race. An impossible projection means core, storage, or persisted history is inconsistent.

## Lifecycle rules [#lifecycle-rules]

`run.created` is the only valid first event. It creates a queued run with zero attempts, failures, retries, and releases. The run projection copies durable identity and creation fields from this event: `runId`, `environment`, `taskId`, `queue`, `payload`, `runAt`, `concurrencyKey`, `idempotencyKey`, `singletonKey`, `source`, `traceCarrier`, `meta`, `createdAt`, and `updatedAt`.

Every later event must target the same `runId` and `environment` as the current projection. Terminal runs reject all later lifecycle events; rerun and manual retry create new source-linked runs instead of mutating terminal history.

| Event                        | Projection effect                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `run.delivery_requested`     | Records durable wakeup intent. It can wake queued runs, due scheduled/retrying/released runs, or running runs whose lease has expired. It sets `runAt` to `delivery.availableAt`, clears the lease, and sets `status` to `scheduled` when `availableAt` is later than `occurredAt`; otherwise it sets `status` to `queued`. The delivery intent must target the same environment, run id, and queue as the run.                                        |
| `run.lease_claimed`          | Moves runnable work to `running`, records the supplied lease and optional `attemptDeadlineAt`, and clears dispatch reservation and wait fields. Queued runs are runnable at `runAt ?? updatedAt`; scheduled, retrying, and released runs are runnable only once `runAt` is due; running runs are runnable again only after their lease expires.                                                                                                        |
| `run.lease_heartbeat`        | Replaces the current lease while keeping the status and counters unchanged. The run must be `running` or `cancellation_requested`, and the heartbeat must use the current lease token and worker id.                                                                                                                                                                                                                                                   |
| `run.started`                | Starts the next attempt. The run must be `running` with an active lease, and `event.attempt` must equal `counters.attempts + 1`. The reducer increments `attempts`, clears any previous `failure` and stale `output`, and records `startedAt`.                                                                                                                                                                                                         |
| `run.succeeded`              | Completes the current attempt and terminally sets `status` to `succeeded`. It clears lease and dispatch reservation fields, clears `failure`, records `finishedAt`, and projects optional JSON `output`.                                                                                                                                                                                                                                               |
| `run.failed`                 | Completes the current attempt and terminally sets `status` to `failed`. It increments `failures`, stores `failure`, clears output, lease, and dispatch reservation fields, and records `finishedAt`.                                                                                                                                                                                                                                                   |
| `run.retry_scheduled`        | Completes the current attempt as failure pressure without making the run terminal. It increments `failures` and `retries`, stores `failure`, clears output, lease, and dispatch reservation fields, sets `runAt` to `retryAt`, and sets `status` to `retrying`.                                                                                                                                                                                        |
| `run.released`               | Completes the current attempt as business waiting. It increments `releases`, clears output, failure, lease, dispatch reservation, and attempt-deadline fields, stores the wait condition, and sets `status` to `released` without incrementing failures or retries. Time releases must carry `delay` and `resumeAt`; signal and run-completion releases carry `wait` only, and projection derives `runAt` from `wait.timeoutAt` when a timeout exists. |
| `run.cancellation_requested` | Marks a running leased run as `cancellation_requested` while keeping the lease. This lets cooperative cancellation reach the handler through its abort signal and then persist a terminal or waiting outcome.                                                                                                                                                                                                                                          |
| `run.cancelled`              | Terminally cancels a queued, scheduled, retrying, released, or cancellation-requested run. Direct cancellation of a still-running run is rejected until cancellation has been requested. The reducer clears output, lease, dispatch reservation fields, and `failure`, then records `finishedAt`.                                                                                                                                                      |

Attempt completion events are `run.succeeded`, `run.failed`, `run.retry_scheduled`, and `run.released`. They require a `running` or `cancellation_requested` run, require at least one started attempt, and must reference the current attempt number. The reducer does not infer attempt numbers from event order.

## Projection Fields [#projection-fields]

`RunRecord` is the materialized view of durable history. These fields have reducer-owned semantics:

| Field                                | Reducer rule                                                                                                                                                                 |
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `eventSequence`                      | Final sequence after applying all events in the command.                                                                                                                     |
| `status`                             | Derived only from the event transition rules above.                                                                                                                          |
| `counters.attempts`                  | Set by `run.started`; must advance by exactly one.                                                                                                                           |
| `counters.failures`                  | Incremented by `run.failed` and `run.retry_scheduled`.                                                                                                                       |
| `counters.retries`                   | Incremented only by `run.retry_scheduled`.                                                                                                                                   |
| `counters.releases`                  | Incremented only by `run.released`.                                                                                                                                          |
| `runAt`                              | Copied from `run.created`, then updated by delivery, retry, and release events.                                                                                              |
| `wait`                               | Set by `run.released` for time, signal, or run-completion waits. Cleared when the run is claimed or otherwise leaves the released wait.                                      |
| `attemptDeadlineAt`                  | Set by `run.lease_claimed` when an attempt has a max-attempt deadline. Cleared when the run leaves active execution.                                                         |
| `startedAt`                          | Updated by the latest `run.started`.                                                                                                                                         |
| `finishedAt`                         | Set only by terminal success, failure, or cancellation. Retry and release leave it unset.                                                                                    |
| `failure`                            | Set by failed and retry-scheduled outcomes; cleared by started, succeeded, and cancelled transitions.                                                                        |
| `output`                             | Set only from JSON-compatible `run.succeeded.output`; omitted for void success and cleared by started, failed, retry, release, and cancellation transitions.                 |
| `lease`                              | Set by lease claim and heartbeat. Cleared when delivery recovery, terminal outcomes, retry, release, or cancellation move the run out of active execution.                   |
| `dispatchedAt` / `dispatchExpiresAt` | Set only when a delivery request carries `delivery.dispatchExpiresAt`, which represents a bounded-queue dispatch reservation. Unbounded delivery requests clear both fields. |
| `source`                             | Copied from `run.created`. Triggered, scheduled, rerun, and manual-retry lineage is not rewritten by later events.                                                           |
| `traceCarrier`                       | Copied from `run.created`, then refreshed by delivery as `delivery.traceCarrier ?? event.traceCarrier ?? currentRun.traceCarrier`.                                           |
| `meta`                               | Copied from `run.created`. Later event `meta` remains in event history; it does not rewrite `RunRecord.meta`.                                                                |

The reducer does not create outbox rows. It only projects fields that storage and runtime code use when deciding whether outbox, dispatch, lease, and operator behavior is valid.

## Adapter boundary [#adapter-boundary]

Adapters should treat the projected run as caller-supplied truth that must be checked, not silently trusted. A storage adapter should verify the command environment, run id, expected sequence, projected final sequence, and command-specific ownership before committing.

`appendRunEvents()` must compare the current persisted sequence with `expectedSequence` before validating projection invariants. If it does not match, reject with `ErrorCode.StorageConflict` and `StorageConflictKind.EventSequence`; do not report a stale append as a projection bug.

When an append succeeds, storage commits these records atomically:

* append-only event-history rows with storage-assigned ids and monotonic per-run sequences
* the supplied `projectedRun`
* storage-owned idempotency or singleton ownership updates
* requested outbox rows derived from `run.delivery_requested` events

`reserveRunDispatch()`, `claimRunLease()`, and `heartbeatRunLease()` also receive core-projected events and projected runs. Storage owns the atomic capacity, dispatch-reservation, lease-ownership, and sequence checks for those operations, but it should not recompute the materialized lifecycle projection.

The returned `RunEventRecord` values must be the exact rows persisted to history, including ids and sequence numbers. Do not generate one event id for storage and another for the API result.

See [Adapter Authoring](/contracts/adapter-authoring) for the full storage contract and [Run Lifecycle Vocabulary](/contracts/vocabulary/run-lifecycle) for stable status and event names.


# Contracts Reference (/contracts)



`@runlane/contracts` is the package every Runlane runtime, adapter, lane, and test helper builds on. It owns public shapes, stable enum values, validators, defaults, and adapter interfaces.

It does not own runtime behavior. Reducers, workers, schedules, retries, releases, cancellation, and task execution live in `@runlane/core`.

## When To Start Here [#when-to-start-here]

| You are building               | Start here                                                                                        | Then read                                                              |
| ------------------------------ | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| A storage or transport adapter | Capability types, adapter interfaces, conflict semantics, outbox publishing, and error vocabulary | [Adapter Authoring](/contracts/adapter-authoring)                      |
| A lane package                 | `Lane`, `LaneCapabilities`, `createLane()`, lifecycle ordering, and capability reporting          | [Lanes, Storage, And Transport](/concepts/lanes-storage-and-transport) |
| Operator tooling               | `RunFilter`, `RunEventFilter`, pagination, sort fields, pruning filters, and failure codes        | [Operator APIs](/concepts/operator-apis)                               |
| Runtime-facing code            | task, queue, schedule, delivery, duration, and id/key contracts re-exported by `@runlane/core`    | [Concepts](/concepts)                                                  |
| Adapter or lane tests          | reusable fixtures plus storage, transport, and lane conformance suites                            | [Testing Package](/contracts/testing)                                  |

## Page Map [#page-map]

| Page                                                      | Read it when                                                                                                            |
| --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| [Vocabulary](/contracts/vocabulary)                       | You need stable enum values, IDs, persisted status strings, error codes, sort fields, or worker/delivery result values. |
| [Core Reducer](/contracts/core-reducer)                   | You need to project append-only run events into the `RunRecord` that storage persists.                                  |
| [Adapter Authoring](/contracts/adapter-authoring)         | You are implementing `StorageAdapter`, `TransportAdapter`, or a reusable lane package from contract types alone.        |
| [Local Adapters](/contracts/local-adapters)               | You need direct in-memory storage or transport primitives for tests and local experiments.                              |
| [Postgres Storage](/contracts/postgres-storage)           | You are using or operating the first-party durable Postgres storage adapter.                                            |
| [Postgres Polling Lane](/contracts/postgres-polling-lane) | You want the first-party production lane that uses Postgres storage plus storage-polling workers without SQS.           |
| [Postgres SQS Lane](/contracts/postgres-sqs-lane)         | You want the first-party production lane composed from Postgres storage and SQS wakeups.                                |
| [SQS Transport](/contracts/sqs-transport)                 | You are binding Runlane queues to Amazon SQS and choosing a producer or consumer path.                                  |
| [Testing Package](/contracts/testing)                     | You need fixtures, deterministic clocks, observation helpers, or executable adapter conformance suites.                 |

## Small Example [#small-example]

Use contract types to describe capabilities honestly:

```ts
import type { StorageCapabilities } from '@runlane/contracts'

export const postgresStorageCapabilities = {
  claimsScheduleOccurrences: true,
  durableState: true,
  durableSteps: true,
  exportsObservations: true,
  enforcesIdempotency: true,
  enforcesQueueConcurrency: true,
  enforcesSingleton: true,
  leasesRuns: true,
  persistsOutbox: true,
  processLocalState: false,
  prunesRuns: true,
  readsRunHistory: true,
} satisfies StorageCapabilities
```

Use contract validators at boundaries:

```ts
import { ErrorCode, jsonValueSchema, RunlaneError, validateWithSchema } from '@runlane/contracts'

const validation = await validateWithSchema(jsonValueSchema, {
  reason: 'provider_not_ready',
})

if (validation.issues) {
  throw new RunlaneError({ code: ErrorCode.ValidationFailed })
}
```

## Export Map [#export-map]

| Area          | What it owns                                                                                                                                   | Primary exports                                                                                                                                  |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| JSON          | JSON-safe payload, metadata, and failure detail boundaries                                                                                     | `JsonValue`, `JsonObject`, `isJsonValue`, `jsonValueSchema`                                                                                      |
| Schemas       | Validator-agnostic task payload contracts and validation execution                                                                             | `Schema`, `SchemaInput`, `SchemaOutput`, `validateWithSchema`                                                                                    |
| Errors        | Stable framework, adapter, task, and transport failures                                                                                        | `ErrorCode`, `RunlaneError`, `StorageConflictKind`                                                                                               |
| IDs           | Opaque branded ids and public id/key validation                                                                                                | `Id`, `IdKind`, `RunlaneIdInput`, `asId`, `createRunlaneIdValueSchema`                                                                           |
| Identity      | Durable environment, actor, and trace context scoping                                                                                          | `Environment`, `environmentKey`, `Actor`, `ActorType`, `TraceCarrier`                                                                            |
| Defaults      | Shared defaults for queues, retry, leases, maintenance, pruning, pagination, and operators                                                     | `contractDefaults`                                                                                                                               |
| Queues        | Provider-neutral routing and durable capacity policy                                                                                           | `QueueDefinition`, `QueueName`, `queueDefinitionSchema`, `isBoundedQueue`                                                                        |
| Time          | Public duration strings and executable millisecond values                                                                                      | `DurationString`, `DurationUnit`, `DurationValue`, `durationSchema`, `Clock`                                                                     |
| Runs          | Durable run projections and visible run state                                                                                                  | `RunRecord`, `RunSummary`, `RunStatus`, `RunFailure`                                                                                             |
| Steps         | Named per-run checkpoints for external operations                                                                                              | `RunStepRecord`, `RunStepKey`, `RunStepToken`, `getRunStepToken`                                                                                 |
| Run scans     | Shared status groups and due-work predicates for adapters                                                                                      | `runStatusValues`, `getRunRunnableAvailableAt`, `getRunDispatchAvailableAt`, and related availability helpers                                    |
| Events        | Replayable run history                                                                                                                         | `RunEventType`, `RunEvent`, `RunEventRecord`                                                                                                     |
| Observability | Sanitized live observations, durable observation records, exporter stream queries, and checkpoint commands                                     | `RunlaneObservationType`, `RunlaneObservation`, `RunlaneObservationRecord`, `ScanObservationRecordsQuery`, `AdvanceObservationCheckpointCommand` |
| Tasks         | User work, retry policy, release values, and handler context                                                                                   | `TaskDefinition`, `TaskContext`, `TaskRelease`, `RetryPolicy`                                                                                    |
| Schedules     | Task-colocated schedules and durable occurrence state                                                                                          | `ScheduleDefinition`, `ScheduleOccurrence`, `ScheduleType`                                                                                       |
| Delivery      | Outbox rows and transport wakeups                                                                                                              | `DeliveryIntent`, `DeliveryMessage`, `deliveryMessageSchema`, `OutboxFailureRecord`, `OutboxMessage`, `OutboxMessageStatus`                      |
| Storage       | Atomic durable truth, operator reads, leases, durable steps, observation export, outbox mutation, idempotency, singleton, and pruning commands | `StorageAdapter`, `AppendRunEventsCommand`, `ClaimRunLeaseCommand`, `CompleteRunStepCommand`, `ScanObservationRecordsQuery`, `PruneRunsCommand`  |
| Transport     | Wakeup publishing, indexed publish outcomes, and provider acknowledgement data                                                                 | `TransportAdapter`, `publishWakeupsCommandSchema`, `parsePublishWakeupsResult`, `WakeupPublishOutcomeType`                                       |
| Lanes         | Contract-only storage and delivery-mode composition                                                                                            | `Lane`, `LaneDelivery`, `LaneDeliveryMode`, `LaneCapabilities`, `createLane`, `laneSchema`, `laneDeliverySchema`                                 |
| Operators     | Cursor-backed reads and retention commands                                                                                                     | `PaginationParams`, `Page`, `RunFilter`, `RunEventFilter`, `PruneRunsFilter`                                                                     |
| Capabilities  | Public promises made by storage, transport, and lanes                                                                                          | `StorageCapabilities`, `TransportCapabilities`, `LaneCapabilities`                                                                               |

## Boundary Map [#boundary-map]

| Boundary          | Owns                                                                                  | Must not own                                           |
| ----------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| Contracts         | Shapes, defaults, validators, enums, adapter interfaces                               | Worker loops, reducer transitions, task execution      |
| Core              | Runtime behavior, reducer semantics, event projection, acquisition paths              | Database-specific storage, provider-specific transport |
| Storage adapter   | Atomic durable truth, conflicts, indexes, leases, idempotency, singleton, outbox rows | Recomputed reducer behavior or wakeup delivery         |
| Transport adapter | Minimal wakeup publish and provider-specific delivery plumbing                        | Task payloads, run state, schedule truth               |
| Lane package      | Storage and delivery resource composition, capability reporting, lifecycle order      | New runtime semantics or capability emulation          |

## Non-Negotiables [#non-negotiables]

| Rule                                                                                          | Why it matters                                                                                 |
| --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Storage persists core-projected events and run state atomically.                              | Reducer behavior stays in core while storage remains the conflict authority.                   |
| `RunEventRecord` results must be the exact records persisted to history.                      | Callers cannot reconcile one event id in storage with another in memory.                       |
| Durable step completion is fenced by lease token and attempt.                                 | A stale worker cannot checkpoint external-operation output into a run it no longer owns.       |
| Observation export scans committed facts, not operator projections.                           | Telemetry export stays sanitized, stable, and independent of task execution latency.           |
| Transport messages carry only `environment`, `runId`, queue, request time, and trace context. | Duplicate or delayed wakeups stay safe because workers re-read storage.                        |
| Publish outcomes are indexed: `outcomes[index]` belongs to `command.attempts[index]`.         | Maintenance can mark claimed outbox rows published, failed, or dead-lettered without guessing. |
| Capabilities are promises.                                                                    | Unsupported methods fail fast instead of creating hidden no-ops.                               |
| Public IDs and keys are opaque and must not contain `:`.                                      | Adapters can compose internal keys without turning public ids into parse contracts.            |
| Cursors are opaque.                                                                           | Storage owns pagination encoding, filter state, and ordering semantics.                        |

## Defaults And Validation [#defaults-and-validation]

| Surface            | Contract rule                                                                                                       |
| ------------------ | ------------------------------------------------------------------------------------------------------------------- |
| Task schemas       | Runtime APIs accept Standard Schema-compatible validators through `Schema`.                                         |
| JSON metadata      | Use `jsonValueSchema` or `isJsonValue`; do not persist raw provider responses or cyclic objects.                    |
| Durations          | Use `durationSchema` or `createDurationSchema(message)`; valid values include `500ms`, `30s`, `5m`, `1h`, and `7d`. |
| Retry default      | `contractDefaults.retry.backoff` applies when `retry.maxAttempts` exists without an explicit backoff.               |
| Lease default      | `contractDefaults.lease.duration` applies to worker attempts and direct execution unless overridden.                |
| Maintenance bounds | `contractDefaults.maintenance` bounds each `tick()` phase.                                                          |
| Pruning batch      | `contractDefaults.pruning.batchLimit` applies when a prune command omits `limit`.                                   |

## Errors [#errors]

Runlane failures use `RunlaneError` with stable `ErrorCode` values. Public code should branch on `error.code`, not parse raw driver messages.

Persisted failure summaries follow the same rule:

| Field                      | Contract                                                                                  |
| -------------------------- | ----------------------------------------------------------------------------------------- |
| `RunFailure.code`          | Always a Runlane-owned `ErrorCode`. Unknown handler throws become `ErrorCode.TaskFailed`. |
| `RunFailure.meta`          | JSON-safe domain or provider detail, such as request ids or provider codes.               |
| `OutboxFailureRecord.code` | Always a Runlane-owned `ErrorCode`, not a provider namespace.                             |
| `cause`                    | Server-side diagnostic detail only; not the public contract.                              |

See [Error Codes](/contracts/vocabulary/error-codes) for the full vocabulary.


# Local Adapters (/contracts/local-adapters)



`@runlane/local-adapters` exports the in-memory storage and transport primitives used by local lanes and package tests. Use it when you need direct adapter access, not when you want the ready-to-use local lane.

```ts
import { createLocalStorage, createLocalTransport } from '@runlane/local-adapters'

const storage = createLocalStorage()
const transport = createLocalTransport()
```

Both factories take no options. Each call creates a new adapter instance; local storage owns fresh in-memory state, and local transport is stateless between publish calls.

## When To Use It [#when-to-use-it]

* core/runtime tests that need to compose a lane without importing `@runlane/lane-local`
* adapter contract tests that need a simple in-memory peer
* local experiments where you want to inspect storage and transport separately

For application code and examples, prefer `createLocalLane()` from `@runlane/lane-local`. The lane package wires local storage without an external transport and reports composed capabilities.

If you do need a lane while staying at the adapter layer, compose local storage with explicit transport delivery:

```ts
import { createLane, LaneDeliveryMode } from '@runlane/contracts'
import { createLocalStorage, createLocalTransport } from '@runlane/local-adapters'

const lane = createLane({
  delivery: { mode: LaneDeliveryMode.Transport, transport: createLocalTransport() },
  name: 'local-contract-test',
  storage: createLocalStorage(),
})
```

Pass `delivery: { mode: LaneDeliveryMode.StoragePolling }` when the test should exercise storage polling without a transport.

## Process-Local State [#process-local-state]

Local adapters are intentionally in-memory. They preserve Runlane storage and transport contracts inside one process, but they are not production durability:

* process exit loses all adapter state
* two calls to `createLocalStorage()` do not share runs, events, cursors, or outbox rows
* two processes cannot construct equivalent local adapters and reach the same state
* memory grows with retained history until you prune it or discard the process

`@runlane/lane-local` uses local storage to create a complete local lane. In a `runlane dev` workflow, the long-running dev process owns the process-local runtime and matching stateful CLI commands proxy into that process. Direct adapter instances do not provide that bridge.

The package depends on `@runlane/contracts`, not `@runlane/core`. Worker loops, schedules, retries, releases, cancellation, operator APIs, and task execution stay in core; local adapters only implement the storage and transport contracts.

## Storage Adapter [#storage-adapter]

`createLocalStorage()` returns a storage adapter named `local-memory-storage`.

| Capability                  | Value   | What that means locally                                                    |
| --------------------------- | ------- | -------------------------------------------------------------------------- |
| `claimsScheduleOccurrences` | `true`  | Due schedule fires are claimed in memory with expiring claim tokens.       |
| `durableState`              | `false` | Committed state does not survive process loss.                             |
| `durableSteps`              | `true`  | Durable step completions are checkpointed in the adapter instance.         |
| `exportsObservations`       | `true`  | Observation records and exporter checkpoints are kept in memory.           |
| `enforcesIdempotency`       | `true`  | Active and retained idempotency owners are tracked in memory.              |
| `enforcesQueueConcurrency`  | `true`  | Bounded queue capacity is checked against in-memory run state.             |
| `enforcesSingleton`         | `true`  | Active singleton keys are tracked in memory.                               |
| `leasesRuns`                | `true`  | Workers claim, heartbeat, and release leases through storage methods.      |
| `persistsOutbox`            | `true`  | Delivery requests can create outbox rows in memory when the command asks.  |
| `processLocalState`         | `true`  | Only the adapter instance's process can access its state.                  |
| `prunesRuns`                | `true`  | Terminal run retention can be pruned through `pruneRuns()`.                |
| `readsRunHistory`           | `true`  | Operator reads and run-event history are available with keyset pagination. |

Local storage keeps these in-memory indexes:

* materialized run records
* durable step completions
* durable observation records and exporter checkpoints
* append-only event history per run
* released wait conditions and running attempt deadlines
* retained idempotency owners
* active singleton owners
* schedule occurrences
* outbox messages
* pagination and prune cursors

Core supplies projected run records when it appends events. Local storage validates that the events, environment, queue, expected sequence, projected sequence, idempotency policy, and delivery intent agree before mutating state. The write path stores the event records, projected run, storage-owned key indexes, outbox rows, and observation records together; stale event sequences and stale lease or outbox ownership fail without partially mutating state.

The local observation stream mirrors the production storage contract for deterministic tests and local exporters. Run event records and first-time durable step completions become sanitized observation records with stable ids and opaque string stream positions. `scanObservationRecords()`, `getObservationCheckpoint()`, and `advanceObservationCheckpoint()` operate only inside the adapter instance; process exit loses both the stream and checkpoint state. Checkpoints can only advance to cursors returned from existing records in the same environment.

All records crossing the adapter boundary are defensively copied. Mutating a command object after a write, or mutating a returned run, event, durable step, lease, outbox message, schedule occurrence, page, or cursor input, must not mutate storage-owned state.

## Outbox And Pruning [#outbox-and-pruning]

Local storage persists an outbox row for accepted `run.delivery_requested` events when the command requests delivery outbox persistence. Transport delivery lanes request it; storage-polling lanes do not. `claimOutboxMessages()` returns due pending, failed, or expired-claim messages in deterministic publish order. `markOutboxMessagesPublished()`, `markOutboxMessagesFailed()`, and `markOutboxMessagesDeadLettered()` require the current claim token; stale publisher ownership fails with a storage conflict.

`pruneRuns()` deletes terminal runs older than the requested cutoff. It defaults to all terminal statuses when no status filter is supplied, accepts a bounded `limit`, and returns a cursor when more matching rows remain. Pruning removes the selected run records, event history, retained idempotency owners, and matching outbox rows. It does not delete active runs.

This is useful for exercising operator retention flows locally. It does not prove production index design, table compaction, migration behavior, or database locking.

## Transport Adapter [#transport-adapter]

`createLocalTransport()` returns a transport adapter named `local-memory-transport`.

| Capability        | Value   | What that means locally                                                            |
| ----------------- | ------- | ---------------------------------------------------------------------------------- |
| `durableDelivery` | `false` | Published wakeups are not stored by transport and do not survive process loss.     |
| `messageGrouping` | `false` | The transport does not group messages for provider FIFO semantics.                 |
| `nativeDelay`     | `false` | Delays are represented by storage due times and maintenance, not transport timers. |
| `orderedDelivery` | `false` | The transport does not provide ordering guarantees.                                |

`publishWakeups()` validates the publish command and immediately returns one `Published` outcome per attempted wakeup. It does not enqueue messages for a worker, does not preserve wakeups for later delivery, and does not model provider acknowledgments, redrive, grouping, ordering, or native delay. When a custom test lane includes this transport, recovery comes from storage outbox rows and the runtime `tick()` path, not from the in-memory transport.

## Local Lane Relationship [#local-lane-relationship]

`createLocalLane()` from `@runlane/lane-local` composes `createLocalStorage()` without a transport. Its only option is `name`, defaulting to `local`; it does not expose capability overrides such as `productionDurable`.

Use `createLocalLane()` with `createRunlane()` when you want a local backend for application tests or examples. Use `@runlane/local-adapters` directly when you are proving adapter behavior, composing a custom test lane, or exercising storage and transport methods without the core runtime.

## Conformance Role [#conformance-role]

`@runlane/local-adapters` is the reusable local primitive. Its storage adapter runs storage conformance, and its transport adapter runs transport conformance. `@runlane/lane-local` composes local storage with storage polling and runs lane composition conformance.

Core runtime behavior is tested in `@runlane/core` against these same local adapters. `@runlane/lane-local` also has integration tests that pair `createLocalLane()` with `createRunlane()` to prove the full local development experience.

Do not move worker, schedule, operator, retry, release, or cancellation implementations into local adapters or the local lane package. Those remain runtime concerns over any conforming lane.


# Postgres Polling Lane (/contracts/postgres-polling-lane)



`@runlane/lane-postgres-polling` is the prescribed production lane when you want Runlane backed by Postgres only. It composes `postgresStorage()` from `@runlane/postgres-storage` with `createLane()` from `@runlane/contracts` and no external transport.

Use it for ordinary worker-process deployments on EC2, ECS, Fargate, Kubernetes, systemd, or any platform where a long-running process can poll Postgres. Core still owns task execution, schedules, retries, releases, cancellation, workers, and operator APIs. Postgres owns durable truth.

```sh
pnpm add @runlane/core @runlane/lane-postgres-polling @runlane/postgres-storage zod
```

## When To Use This Lane [#when-to-use-this-lane]

Use this lane when one operational dependency is the right trade-off:

| Use it for                      | Why                                                                                                                      |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Postgres-only production jobs   | Runs, history, leases, schedules, idempotency, singleton ownership, operator reads, and pruning all live in Postgres.    |
| Long-running worker processes   | Workers call `runlane.worker({ mode: WorkerMode.Poll })` and claim due runs from storage.                                |
| Scheduled jobs and polling jobs | `tick()` materializes schedules, makes due retries and releases visible to polling workers, and recovers expired leases. |

Use `@runlane/lane-local` for in-memory development and tests. See [Lanes, Storage, And Transport](../concepts/lanes-storage-and-transport.mdx) when you are choosing between first-party lanes.

## Create The Lane [#create-the-lane]

```ts
import { createRunlane, queue, task, WorkerMode } from '@runlane/core'
import { postgresPollingLane } from '@runlane/lane-postgres-polling'
import * as z from 'zod'

const quickbooksQueue = queue({ name: 'quickbooks', default: true })

const syncQuickBooksInvoices = task({
  id: 'quickbooks.invoices.sync',
  queue: quickbooksQueue,
  schema: z.object({ userId: z.string().min(1) }),
  singletonKey: (payload) => `quickbooks_invoices_${payload.userId}`,
  async run(payload, context) {
    await syncInvoicesForUser(payload.userId, { signal: context.signal })
  },
})

const lane = postgresPollingLane({
  connectionString: process.env.DATABASE_URL,
  schema: 'runlane',
})

const runlane = createRunlane({
  lane,
  queues: [quickbooksQueue],
  tasks: { syncQuickBooksInvoices },
})

const worker = runlane.worker({
  mode: WorkerMode.Poll,
  queues: [quickbooksQueue],
  concurrency: 4,
})

process.once('SIGTERM', () => {
  void worker.stop()
})

await worker.closed
```

`postgresPollingLane()` accepts the same Postgres storage options users learn for `postgresStorage()`, plus an optional lane diagnostic `name`:

| Option             | Required | Default                       | What it controls                                                             |
| ------------------ | -------- | ----------------------------- | ---------------------------------------------------------------------------- |
| `connectionString` | Yes      | None                          | Forwarded to `postgresStorage()`. Must use `postgres://` or `postgresql://`. |
| `schema`           | No       | URL `?schema=`, then `public` | Forwarded to `postgresStorage()`. Must be an unquoted Postgres identifier.   |
| `name`             | No       | `postgres-polling`            | Lane diagnostic name.                                                        |

Run `getPostgresStorageMigrationSql()` from `@runlane/postgres-storage` in your normal migration system before starting the lane. The polling lane does not auto-migrate. `lane.start()` probes the migrated Postgres tables and fails fast when the database, schema, or migration is not ready.

## Capabilities [#capabilities]

`postgresPollingLane()` returns a standard `Lane` from `createLane()`:

| Boundary                              | Reported behavior                                                                                                                                                                                                                        |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Lane name                             | `name ?? 'postgres-polling'`                                                                                                                                                                                                             |
| `lane.capabilities.operatorReads`     | `true`, backed by Postgres operator reads                                                                                                                                                                                                |
| `lane.capabilities.productionDurable` | `true`                                                                                                                                                                                                                                   |
| `lane.capabilities.storage`           | Exact Postgres storage capabilities: durable state, process-shared state, durable steps, observation export, leases, idempotency, singleton, queue concurrency, schedule occurrence claims, persisted outbox, history reads, and pruning |
| `lane.delivery.mode`                  | `storage_polling`                                                                                                                                                                                                                        |

The polling lane has no external transport. Core does not create delivery outbox rows for this lane, and `tick()` does not publish provider wakeups. If no polling worker is running, no external provider will wake work; the run remains durable and due in Postgres until a worker starts.

## Worker Processes [#worker-processes]

Run a polling worker anywhere you can keep a process alive:

```ts
import { WorkerMode } from '@runlane/core'

const worker = runlane.worker({
  mode: WorkerMode.Poll,
  queues: [quickbooksQueue],
  concurrency: Number(process.env.RUNLANE_WORKER_CONCURRENCY ?? 4),
  emptyPollDelay: '100ms',
  leaseDuration: '5m',
})

process.once('SIGTERM', () => {
  void worker.stop()
})

process.once('SIGINT', () => {
  void worker.stop()
})

await worker.closed
```

`concurrency` is local process fan-out. Queue `concurrencyLimit` is durable Postgres policy across all polling workers. Multiple worker processes can run for the same queue; storage decides which run each process owns through leases.

## Maintenance [#maintenance]

Polling workers execute due runs, but they do not materialize schedules or recover every waiting state on their own. Run `tick()` from a separate maintenance loop:

```ts
import { setTimeout as sleep } from 'node:timers/promises'

while (!signal.aborted) {
  await runlane.tick()
  await sleep(30_000, undefined, { signal })
}
```

One maintenance pass handles:

| Tick phase                | Why it matters in Postgres polling                                                              |
| ------------------------- | ----------------------------------------------------------------------------------------------- |
| Schedules                 | Due task-colocated schedules become durable queued runs.                                        |
| Retries and releases      | Runs waiting for retry backoff or `ctx.release()` are re-queued when due.                       |
| Cancellation finalization | Abandoned cancellation requests become terminal when eligible.                                  |
| Delivery recovery         | Due waiting or expired leased runs get fresh delivery intent so polling workers can claim them. |
| Expired leases            | Crashed workers become recoverable after their lease expires.                                   |
| Bounded queue dispatch    | Postgres reserves durable queue capacity before dispatching bounded-queue runs.                 |

A common starting cadence is 30 seconds. Use a shorter cadence when schedule or retry latency matters; use a longer cadence when database polling pressure matters more. Worker `emptyPollDelay` controls how often idle workers scan for due runs after maintenance has made them runnable.

## Deployment Shapes [#deployment-shapes]

Use one shared runtime factory for producers, workers, maintenance, and operator commands:

| Role         | What it does                                                                                                         |
| ------------ | -------------------------------------------------------------------------------------------------------------------- |
| Producer     | Calls `runlane.start()` once per process, then `runlane.trigger()` from app code.                                    |
| Worker       | Runs `runlane.worker({ mode: WorkerMode.Poll })` for one or more queues.                                             |
| Maintenance  | Calls `runlane.tick()` on a cadence for schedules, retries, releases, lease recovery, and cancellation finalization. |
| Operator/CLI | Lists, inspects, cancels, retries, reruns, prunes, or triggers runs against the same Postgres lane.                  |

For ECS, Fargate, Kubernetes, or systemd, run the worker and maintenance roles as separate process types when possible. A small deployment can run both loops in one process, but keep the code paths separate so worker failures and maintenance failures are visible.


# Postgres SQS Lane (/contracts/postgres-sqs-lane)



`@runlane/lane-postgres-sqs` is the reference production lane. It composes `postgresStorage()` from `@runlane/postgres-storage`, `sqsTransport()` from `@runlane/transport-sqs`, and `createLane()` from `@runlane/contracts`.

The package does not own task execution, retries, releases, schedules, worker loops, Lambda parsing, or outbox semantics. Core owns behavior. Postgres owns durable truth. SQS carries wakeups.

```sh
pnpm add @runlane/core @runlane/lane-postgres-sqs @runlane/postgres-storage @runlane/transport-sqs @aws-sdk/client-sqs
```

## When To Use This Lane [#when-to-use-this-lane]

Use this lane when one Runlane runtime should share durable state through Postgres and wake workers through SQS:

| Use it for                        | Why                                                                                                                                       |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Multi-process production runtimes | Producers, Lambda consumers, long-running workers, maintenance, and CLI commands all see the same Postgres state.                         |
| SQS-based acquisition             | Each durable outbox row becomes one SQS wakeup, and consumers call `runlane.executeDelivery()` for the delivered run.                     |
| Operator history and retention    | Postgres serves run lists, event history, idempotency ownership, singleton ownership, queue capacity, and pruning.                        |
| AWS deployments                   | The lane fits RDS Postgres plus SQS, Lambda event source mappings, EventBridge maintenance, ECS/Fargate workers, and AWS SDK credentials. |

Use `createLocalLane()` instead when you want an in-memory local backend without Postgres or SQS. Use separate lanes only when the infrastructure boundary is actually different: another database or schema, AWS account, region, tenant boundary, compliance boundary, or storage/transport pair. Use queues for ordinary workload routing inside the same Postgres/SQS boundary.

## Create The Lane [#create-the-lane]

```ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { queue } from '@runlane/core'
import { postgresSqsLane } from '@runlane/lane-postgres-sqs'
import { sqsQueue } from '@runlane/transport-sqs'

export const defaultQueue = queue({ name: 'default', default: true })
export const emailsQueue = queue({ name: 'emails' })
export const queues = [defaultQueue, emailsQueue]

export const lane = postgresSqsLane({
  postgres: {
    connectionString: process.env.DATABASE_URL,
    schema: 'runlane',
  },
  sqs: {
    client: new SQSClient({ region: 'us-east-1' }),
    queues: [
      sqsQueue(defaultQueue, {
        queueUrl: process.env.RUNLANE_DEFAULT_QUEUE_URL,
      }),
      sqsQueue(emailsQueue, {
        queueName: 'runlane-emails-production',
      }),
    ],
  },
})
```

`postgres` accepts the same options as `postgresStorage()`. `sqs` accepts the same options as `sqsTransport()`. The lane package validates only its own top-level shape and delegates nested adapter validation to the packages that own those contracts.

| Option     | Required | Default        | What it controls                          |
| ---------- | -------- | -------------- | ----------------------------------------- |
| `postgres` | Yes      | None           | Options forwarded to `postgresStorage()`. |
| `sqs`      | Yes      | None           | Options forwarded to `sqsTransport()`.    |
| `name`     | No       | `postgres-sqs` | Lane diagnostic name.                     |

Unsupported top-level keys fail before adapters are constructed. Unsupported nested `postgres` or `sqs` keys fail in the owning adapter constructor. Because `sqsTransport()` exposes its bound queue definitions, `createRunlane({ queues })` also verifies that every runtime queue has a matching SQS provider binding with the same queue policy.

By default the lane is named `postgres-sqs`. Pass `name` only when you need a different diagnostic label:

```ts
postgresSqsLane({
  name: 'orders-production',
  postgres: { connectionString, schema: 'runlane' },
  sqs: { client, queues },
})
```

Run `getPostgresStorageMigrationSql()` from `@runlane/postgres-storage` in your normal migration system before starting the lane. `lane.start()` probes the migrated Postgres tables and fails fast when the database, schema, or migration is not ready.

## Capabilities And Lifecycle [#capabilities-and-lifecycle]

`postgresSqsLane()` returns a standard `Lane` from `createLane()`:

| Boundary                               | Reported behavior                                                                                                                                                                                                                        |
| -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Lane name                              | `name ?? 'postgres-sqs'`                                                                                                                                                                                                                 |
| `lane.capabilities.operatorReads`      | `true`, inherited from `createLane()` defaults and backed by Postgres operator reads                                                                                                                                                     |
| `lane.capabilities.productionDurable`  | `true`                                                                                                                                                                                                                                   |
| `lane.capabilities.storage`            | Exact `postgresStorageCapabilities`: durable state, process-shared state, durable steps, observation export, leases, idempotency, singleton, queue concurrency, schedule occurrence claims, persisted outbox, history reads, and pruning |
| `lane.delivery.mode`                   | `transport`                                                                                                                                                                                                                              |
| `lane.delivery.transport`              | SQS transport adapter                                                                                                                                                                                                                    |
| `lane.delivery.transport.capabilities` | Exact SQS transport capabilities: `durableDelivery: true`, `nativeDelay: false`, and adapter-wide FIFO grouping/ordering flags only when every configured queue is FIFO                                                                  |

Lifecycle order is the `createLane()` order:

1. `lane.start()` starts Postgres storage first, then SQS transport.
2. If transport startup fails, storage is closed.
3. `lane.close()` closes transport first, then Postgres storage.

The current SQS transport has no startup probe, so the meaningful production probe is Postgres storage startup. Queue URL resolution for `queueName` bindings happens on first publish and is cached by the transport.

## Dispatch And Recovery [#dispatch-and-recovery]

`trigger()` validates the task payload and creates or returns a durable run in Postgres. What happens next depends on queue policy and dispatch mode:

| Case                                  | Durable effect                                                                                                                                      |
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| Duplicate idempotency owner           | Returns the existing owner run; no new delivery request or SQS wakeup is created.                                                                   |
| Unbounded queue                       | Appends `run.created` and `run.delivery_requested` together, and Postgres writes the outbox row in the same transaction.                            |
| Bounded queue with `concurrencyLimit` | Appends `run.created` only. A later `tick()` reserves queue capacity through Postgres, appends `run.delivery_requested`, and writes the outbox row. |

With the default `dispatch.onTrigger: TriggerDispatchMode.Eager`, core claims and publishes only the outbox rows returned by that creation call. That means eager trigger dispatch can publish the initial wakeup for unbounded queues, but it does not bypass bounded queue capacity. Bounded queues still need maintenance to reserve capacity and publish wakeups.

`tick()` is the recovery and maintenance path. It reserves bounded-queue dispatch capacity, publishes deferred outbox rows, retries failed eager publish attempts after the outbox retry delay, materializes due schedules, wakes due released or retried runs, recovers expired leases, and finalizes eligible cancellations.

Run `tick()` from separate maintenance infrastructure:

* EventBridge scheduled Lambda
* cron process
* container sidecar
* operator command

Do not call `tick()` at the end of the SQS delivery Lambda batch. Delivery Lambdas should stay focused on the SQS records AWS already delivered; maintenance is a bounded global pass and can run concurrently across its own scheduler.

```ts
const runlane = createRunlane({
  lane,
  queues,
  tasks: { sendEmail },
})

await runlane.trigger(runlane.tasks.sendEmail, { userId: 'user_123' })
await runlane.tick()
```

Use `dispatch: { onTrigger: TriggerDispatchMode.Deferred }` when the producer process should persist work only and let a maintenance process publish SQS wakeups for rows that are already dispatchable:

```ts
import { TriggerDispatchMode } from '@runlane/core'

const runlane = createRunlane({
  dispatch: { onTrigger: TriggerDispatchMode.Deferred },
  lane,
  queues,
  tasks: { sendEmail },
})
```

## Deployment Shape [#deployment-shape]

A production deployment usually has four process roles sharing the same runtime factory:

| Role         | What it does                                                                                                       |
| ------------ | ------------------------------------------------------------------------------------------------------------------ |
| Producer     | Calls `runlane.start()` once per process, then `runlane.trigger()` from HTTP handlers, jobs, scripts, or app code. |
| SQS consumer | Drains exactly one SQS acquisition path per logical queue, then calls `runlane.executeDelivery(message, options)`. |
| Maintenance  | Runs `runlane.tick()` on a schedule such as EventBridge, cron, sidecar, or operator command.                       |
| Operator/CLI | Uses the same runtime to list, inspect, cancel, retry, rerun, prune, or trigger runs.                              |

Keep lane composition and task registration in one shared factory so every role registers the same queues and tasks. The [AWS example](../examples/postgres-sqs-aws.mdx) uses this shape: an HTTP trigger Lambda, two SQS-Lambda consumers, one Fargate SQS consumer, and an EventBridge tick Lambda all import the same runtime factory.

For example:

```ts
// src/runtime/runlane.ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { createRunlane, queue, task } from '@runlane/core'
import { postgresSqsLane } from '@runlane/lane-postgres-sqs'
import { sqsQueue } from '@runlane/transport-sqs'
import * as z from 'zod'

import { emailProvider } from '../email-provider.js'

const emailsQueue = queue({ name: 'emails', default: true })

const sendEmail = task({
  id: 'emails.send',
  queue: emailsQueue,
  schema: z.object({ userId: z.string().min(1) }),
  idempotencyKey: (payload) => `emails.send.${payload.userId}`,
  async run(payload, context) {
    await emailProvider.send(payload.userId, { signal: context.signal })
  },
})

export function createAppRunlane() {
  const sqsEndpoint = process.env.RUNLANE_SQS_ENDPOINT
  const sqsClient = new SQSClient({
    region: process.env.AWS_REGION ?? 'us-east-1',
    ...(sqsEndpoint === undefined ? {} : { endpoint: sqsEndpoint }),
  })
  const lane = postgresSqsLane({
    postgres: {
      connectionString: process.env.DATABASE_URL,
      schema: process.env.RUNLANE_POSTGRES_SCHEMA ?? 'public',
    },
    sqs: {
      client: sqsClient,
      queues: [
        sqsQueue(emailsQueue, {
          queueUrl: process.env.RUNLANE_QUEUE_EMAILS_URL,
        }),
      ],
    },
  })

  return createRunlane({
    environment: { name: process.env.RUNLANE_ENVIRONMENT ?? 'production' },
    lane,
    queues: [emailsQueue],
    tasks: { sendEmail },
  })
}
```

## Lambda SQS Handler [#lambda-sqs-handler]

The lane package intentionally does not wrap Lambda. Use the SQS transport helper and pass core's `executeDelivery()` callback:

```ts
import { createSqsLambdaHandler } from '@runlane/transport-sqs'

import { createAppRunlane } from '../runtime/runlane.js'

const runlane = createAppRunlane()

let startup: Promise<void> | undefined

async function ensureStarted() {
  startup ??= runlane.start()
  await startup
}

export const handler = createSqsLambdaHandler({
  deliveryOptions: {
    leaseDuration: '2m',
  },
  async executeDelivery(message, options) {
    await ensureStarted()
    return runlane.executeDelivery(message, options)
  },
})
```

Enable Lambda partial batch responses with `ReportBatchItemFailures` on the SQS event source mapping.

The handler acknowledges a provider message when Runlane delivery handling completed, even if the durable task attempt records `run.failed`, `run.retry_scheduled`, `run.released`, or `run.cancelled`. Framework, storage, registration, projection, parser, or persistence failures leave the SQS record unacknowledged so AWS can retry or redrive it.

Use one production acquisition path per logical queue. For a Postgres+SQS lane, that path is an SQS consumer: Lambda through `createSqsLambdaHandler()` or a long-running process through `createSqsDeliveryConsumer()`. `runlane dev` and `runlane.worker({ mode: WorkerMode.Poll })` are storage-polling workers; they can execute durable runs from Postgres, but they will not receive or delete provider messages from SQS. Use polling workers only with a polling lane/config that does not publish SQS wakeups for that queue. For local testing of SQS delivery behavior, run the SQS Lambda helper or long-running SQS consumer against LocalStack, Ministack, or AWS.

## Local SQS Path [#local-sqs-path]

Use a real SQS consumer when the local test is meant to prove the Postgres+SQS lane. Do not run `runlane dev` for the same queue during this test: the dev worker polls Postgres directly and can claim the run before the SQS consumer handles the wakeup.

The generic Runlane path is a small Node process using `createSqsDeliveryConsumer()`. Framework-specific queue subscribers are valid when the app already uses that framework, but they are not required.

| Path                          | What it proves                                                                              | What runs locally                                                                          |
| ----------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `createSqsDeliveryConsumer()` | SQS receive/delete behavior and `runlane.executeDelivery()` without a framework             | A long-running Node consumer receives from AWS SQS, LocalStack, or Ministack.              |
| Framework queue subscriber    | Lambda event-source wiring, partial batch response handling, and `createSqsLambdaHandler()` | The framework runs or proxies the subscriber handler while SQS remains the provider queue. |

For a fully local Ministack path, point both the lane transport and the consumer at the same local queue URL. `createAppRunlane()` below is an application-owned runtime factory; Runlane exports `createRunlane()`, not your app factory.

```ts
// scripts/run-sqs-consumer.ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { createSqsDeliveryConsumer } from '@runlane/transport-sqs'

import { createAppRunlane } from '../src/runtime/runlane.js'

const runlane = createAppRunlane()
await runlane.start()

const sqs = new SQSClient({
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID ?? 'test',
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY ?? 'test',
  },
  endpoint: process.env.RUNLANE_SQS_ENDPOINT,
  region: process.env.AWS_REGION ?? 'us-east-1',
})

const consumer = createSqsDeliveryConsumer({
  client: sqs,
  deliveryOptions: {
    leaseDuration: '2m',
  },
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
  queueUrl: process.env.RUNLANE_QUEUE_EMAILS_URL,
  visibilityTimeoutSeconds: 120,
  waitTimeSeconds: 2,
})

process.once('SIGINT', () => {
  void consumer.stop()
})
process.once('SIGTERM', () => {
  void consumer.stop()
})

await consumer.start()
await consumer.closed
await runlane.close()
```

Run the consumer beside Ministack and the producer command:

```sh
# terminal 1
ministack

# terminal 2
RUNLANE_SQS_ENDPOINT=http://127.0.0.1:4566 pnpm tsx scripts/run-sqs-consumer.ts

# terminal 3
RUNLANE_SQS_ENDPOINT=http://127.0.0.1:4566 runlane trigger emails.send '{"userId":"usr_demo_alice"}'
```

For an unbounded queue with eager dispatch, `trigger()` should persist the run in Postgres, publish the wakeup to the configured SQS queue, and the consumer should delete the message only after `executeDelivery()` resolves. Run `runlane tick` separately when testing bounded queue dispatch, schedules, releases, retries, lease recovery, or deferred outbox publishing.


# Postgres Storage (/contracts/postgres-storage)



`@runlane/postgres-storage` is the first-party durable storage adapter for production lanes. It stores run history, materialized run state, leases, durable step completions, durable observation export records, observation exporter checkpoints, schedule occurrence claims, idempotency and singleton ownership, outbox rows, operator read projections, and terminal-run pruning targets in Postgres.

Use it when Postgres should be the durable truth boundary. Pair it with `createLane()` or a lane package; storage does not execute tasks or publish wakeups by itself.

```ts
import { postgresStorage } from '@runlane/postgres-storage'

const storage = postgresStorage({
  connectionString: process.env.DATABASE_URL,
  schema: 'runlane',
})
```

The adapter reports `durableState: true` and `processLocalState: false`: state survives process loss and can be reached by any process using the same migrated database and schema.

## Capabilities [#capabilities]

The adapter reports the full first-party production storage capability set:

| Capability                  | Value   | What Postgres owns                                                                                              |
| --------------------------- | ------- | --------------------------------------------------------------------------------------------------------------- |
| `durableState`              | `true`  | Committed runs, events, outbox rows, ownership rows, and schedule occurrences survive process loss.             |
| `durableSteps`              | `true`  | `context.step.run()` completions are stored by run and step key so later attempts can skip completed callbacks. |
| `exportsObservations`       | `true`  | Sanitized observation records can be scanned by cursor and checkpointed per exporter consumer.                  |
| `processLocalState`         | `false` | App, worker, maintenance, and CLI processes can share state through the same database and schema.               |
| `readsRunHistory`           | `true`  | Operator run lists and event history are served from indexed Postgres rows.                                     |
| `prunesRuns`                | `true`  | Terminal runs can be deleted in bounded, cursor-resumable batches.                                              |
| `leasesRuns`                | `true`  | Worker lease claims and heartbeats are persisted with optimistic sequence checks.                               |
| `claimsScheduleOccurrences` | `true`  | Schedule fires are claimed by deterministic occurrence id.                                                      |
| `persistsOutbox`            | `true`  | Transport-wakeup delivery requests can create durable outbox rows in the same transaction as run events.        |
| `enforcesIdempotency`       | `true`  | Task-scoped idempotency ownership is serialized in Postgres.                                                    |
| `enforcesSingleton`         | `true`  | Active singleton ownership is serialized in Postgres.                                                           |
| `enforcesQueueConcurrency`  | `true`  | Bounded queue capacity is checked transactionally by environment, queue, and concurrency key.                   |

## Connection And Schema [#connection-and-schema]

`postgresStorage()` accepts one connection string and an optional schema:

```ts
postgresStorage({ connectionString, schema })
```

| Option             | Required | Default                                | What it controls                                                                                                                                                              |
| ------------------ | -------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `connectionString` | Yes      | None                                   | Postgres connection string passed to `pg` after Runlane removes the Prisma-style `schema` query parameter. Must be a non-empty string using `postgres://` or `postgresql://`. |
| `schema`           | No       | `?schema=` from the URL, then `public` | Postgres schema that contains Runlane tables, indexes, and constraints. Must be a valid Postgres identifier.                                                                  |

Schema resolution is deterministic:

* `schema` option wins when provided.
* `?schema=runlane` on the connection string is used when `schema` is omitted.
* `public` is used when neither source names a schema.

The adapter rejects unsupported option keys with `ConfigurationInvalid`. It removes the Prisma-style `schema` query parameter before creating the `pg` pool, so the same URL can be shared with tools that understand `?schema=` without sending an unsupported startup parameter to Postgres.

Connection strings may point at ordinary Postgres, RDS Proxy, PgBouncer, or another load-balanced endpoint supported by `node-postgres`. The adapter uses ordinary transactions, row locks, transaction-scoped advisory locks, and `FOR UPDATE SKIP LOCKED`. Named schemas are schema-qualified in adapter queries; the default `public` schema uses Postgres's default table resolution because Drizzle treats `public` as the unqualified schema.

## Migrations [#migrations]

Runlane does not auto-migrate at adapter startup. Apply the SQL migration in your normal migration system before starting producers, workers, maintenance, or CLI commands that use this storage.

```ts
import { getPostgresStorageMigrationSql } from '@runlane/postgres-storage'

const sql = getPostgresStorageMigrationSql({ schema: 'runlane' })
```

The package also ships `migrations/0001_initial.sql` for teams that want to vendor or review the public-schema SQL directly. The generated SQL schema-qualifies table references and creates all tables, constraints, generated columns, and indexes required by the storage contract, including the durable observation stream and checkpoint tables. The package test suite keeps the shipped migration file byte-for-byte aligned with `getPostgresStorageMigrationSql()`.

This is an initial schema generator, not an online migration runner. `postgresStorage().start()` probes the migrated `runlane_runs` table and fails fast when the database, schema, or migration is missing, but it does not create or alter tables for you. Malformed options fail earlier, when the adapter is constructed. Before a Runlane storage schema is applied to shared or production databases, a package upgrade may change the fresh-install baseline; after it is applied, treat the migration as immutable and add forward migrations instead of rewriting the original file.

## Migration Tooling [#migration-tooling]

Prisma teams have four reasonable ways to apply the Runlane storage schema. The right choice depends on the ownership boundary you want:

* Prisma owns the migration ledger.
* Runlane and app schema management stay separate.
* `schema.prisma` mirrors Runlane tables.
* Prisma is only a client generator while another migration tool owns DDL.

Application code should not query Runlane tables through Prisma in any of these layouts. Use the Runlane operator API (`runlane.runs.list`, `runlane.runs.get`, `runlane.runs.retry`, `runlane.runs.cancel`, etc.) for run history and state.

### Option 1: Prisma migrate runs both (recommended) [#option-1-prisma-migrate-runs-both-recommended]

Snapshot Runlane SQL into a Prisma migration directory and let `prisma migrate deploy` apply both Runlane and app migrations in one pass. Use a fixed all-zero timestamp prefix so the Runlane directory sorts before any app migration:

```sh
mkdir -p prisma/migrations/00000000000000_runlane_init
node -e "import('@runlane/postgres-storage').then(m => process.stdout.write(m.getPostgresStorageMigrationSql({ schema: 'public' })))" \
  > prisma/migrations/00000000000000_runlane_init/migration.sql
pnpm prisma migrate deploy
```

`schema.prisma` describes only app tables. One CLI applies both migration streams, while Runlane SQL remains owned by `@runlane/postgres-storage`.

Generate this snapshot when you create a new database baseline. After Prisma has applied a migration to any shared, staging, or production database, treat that migration directory as immutable; do not overwrite it during a package upgrade. If a future `@runlane/postgres-storage` release changes the storage schema, apply the new package guidance or add a new migration that performs the upgrade. The [`@runlane/example-aws`](../examples/postgres-sqs-aws.mdx) reference example follows the initial-snapshot pattern with `pnpm db:emit-runlane-sql`.

### Option 2: Bootstrap Runlane out of band [#option-2-bootstrap-runlane-out-of-band]

Apply the Runlane SQL once with a tool that is not tied to Prisma's `_prisma_migrations` ledger:

```sh
pnpm prisma db execute --schema prisma/schema.prisma \
  --file <(node -e "import('@runlane/postgres-storage').then(m => process.stdout.write(m.getPostgresStorageMigrationSql({ schema: 'public' })))")
pnpm prisma migrate deploy
```

Or run the SQL through `psql`, Atlas, or the package's vendored `migrations/0001_initial.sql`. Prisma's migration ledger never tracks the Runlane SQL; you own applying future Runlane storage upgrades through the same out-of-band path. Pick this when you want a strict boundary between Runlane storage and app schema management.

### Option 3: Mirror Runlane tables in `schema.prisma` [#option-3-mirror-runlane-tables-in-schemaprisma]

Possible, but Prisma's schema language cannot express Runlane's generated columns or `CHECK` constraints:

* `runnable_available_at`
* `delivery_recovery_available_at`
* storage contract `CHECK` clauses

You will hand-edit each generated migration file to add those clauses, and `schema.prisma` will lie about the column shape. Drift risk against `@runlane/postgres-storage` is severe: a future Runlane upgrade that changes the storage schema breaks the adapter probe at `lane.start()` while Prisma still thinks the schema is current.

If you accept those costs because your operator UI wants Prisma-typed reads of Runlane tables, model them read-oriented and never write through Prisma:

```prisma
model RunlaneRun {
  environmentKey String   @map("environment_key")
  runId          String   @map("run_id")
  eventSequence  Int      @map("event_sequence")
  runJson        Json     @map("run_json")
  // ...remaining fields...

  @@id([environmentKey, runId])
  @@map("runlane_runs")
}
```

You almost certainly don't need this. The Runlane operator API exposes the same projections without duplicating the schema in `schema.prisma`.

### Option 4: Skip Prisma's migration tool [#option-4-skip-prismas-migration-tool]

Use Drizzle, Atlas, or plain `psql` for Runlane SQL while Prisma owns only the app-table client (no `prisma/migrations/`). This is appropriate when your team already standardizes on a non-Prisma migration runner; the Prisma client is then purely a query interface for app tables.

```sh
psql "$DATABASE_URL" -f <(node -e "import('@runlane/postgres-storage').then(m => process.stdout.write(m.getPostgresStorageMigrationSql({ schema: 'public' })))")
```

### Connection strings across tools [#connection-strings-across-tools]

Use the same Postgres URL for Prisma and Runlane. If the URL includes `?schema=runlane`, Prisma will use it, and Runlane will resolve the same schema before passing the sanitized URL to `pg`. You do not need a separate `schema` key in application configuration unless you prefer the explicit option.

## Conformance Testing [#conformance-testing]

The package includes the shared storage conformance suite and lane composition conformance with an acknowledging test transport. They run only when `RUNLANE_POSTGRES_TEST_DATABASE_URL` is set:

```sh
RUNLANE_POSTGRES_TEST_DATABASE_URL='postgresql://postgres:secret@localhost:5432/runlane-test' pnpm --filter @runlane/postgres-storage test
```

The test creates a fresh schema named `runlane_test_<uuid>`, applies the package migration, runs the shared adapter and composition behavior tests, and drops that schema during cleanup. Point the variable at a disposable database where the test user can create and drop schemas.

For local development, use the package script:

```sh
pnpm --filter @runlane/postgres-storage test:postgres:local
```

It defaults to `postgresql://postgres:secret@localhost:5432/runlane-test`, removes leftover `runlane_test_%` schemas from failed prior runs, and then runs the package tests with `RUNLANE_POSTGRES_TEST_DATABASE_URL` set. CI can provide `RUNLANE_POSTGRES_TEST_DATABASE_URL` explicitly for its disposable Postgres service; the script still requires a database name containing `test` before it runs cleanup.

## Operational Notes [#operational-notes]

Use one schema per Runlane storage instance. The adapter scopes all data by `environmentKey(environment)`, but schema separation is the operational boundary for migrations, backups, and cleanup.

Runlane stores canonical run and event JSONB payloads and validates persisted rows with Zod-derived schemas at the adapter boundary. Successful handler output is stored in `run_json`, the `run.succeeded` event JSON, and the indexed `output_json` column on `runlane_runs`; transport outbox rows do not carry output.

Durable step completions live in `runlane_run_steps`, keyed by `environment_key`, `run_id`, and `step_key`. The row stores the deterministic step token, completing attempt number, completion time, parsed JSON output, and canonical step JSON. `completeRunStep()` locks the current run, verifies the active lease token and attempt, and treats same-output replay as idempotent while rejecting conflicting output as an invariant violation.

Durable observation export records live in `runlane_observation_records`. Run event appends and first-time durable step completions insert observation rows in the same transaction as the source fact. Each row stores a database-native `bigint` `stream_position` exposed through the public contract as an opaque string cursor position, a stable duplicate-safe `record_id`, the source kind/id, sanitized observation JSON, and environment identity. The hot exporter scan is indexed by `(environment_key, stream_position)`, and source uniqueness prevents the same event id or step token from producing duplicate observation records.

Exporter checkpoints live in `runlane_observation_checkpoints`, keyed by `(environment_key, consumer)`. `advanceObservationCheckpoint()` verifies that the target cursor references an existing observation record in the same environment, locks the checkpoint row, verifies the caller's `expectedCursor`, rejects stale writers with `StorageConflictKind.ObservationCheckpoint`, and rejects non-monotonic cursor moves. Exporters may read observation records from a read replica when replica lag is acceptable, but checkpoint writes must go to the primary database.

Released wait conditions are stored in `run_json` and indexed through `wait_type`, `wait_signal_key`, and `wait_source_run_id` columns so signal and run-completion resume scans do not page through unrelated released runs. Attempt deadlines are stored in `attempt_deadline_at`; the timeout finalization index covers running attempts whose fixed deadline has passed, and the adapter rechecks the current projection so pre-start lease claims are recovered by lease expiry instead of timeout finalization.

`environment_key` is the durable scope column and comes from `environmentKey(environment)`. If the `Environment` identity contract gains new fields, that helper and the corresponding storage migration are the only places that should redefine durable environment identity. SQL table definitions are maintained with Drizzle internally, but that is not part of the public API.

Postgres constraints and ownership tables enforce:

* task-scoped idempotency owners
* one idempotency owner per run
* one completed result per run step key
* one observation record per run event or durable step source
* active singleton owners
* immutable event sequence identity per run
* outbox and schedule occurrence row identity

`runlane_idempotency_keys` stores active owners and retained successful/cancelled terminal owners with the TTL captured at run creation. Failed runs clear their idempotency owner.

`getRunByIdempotencyKey()` reads that owner table directly so core can return the original active or retained terminal run after duplicate triggers or concurrent insert races. `resetIdempotencyKey()` deletes retained terminal owners and rejects active owners.

Bounded queue capacity is not a uniqueness constraint. The adapter serializes each capacity partition with `pg_advisory_xact_lock(hashtextextended(...))`, counts active occupants, and writes the reservation or lease inside the same transaction. Capacity is partitioned by `environment`, queue, and `concurrencyKey`; absent concurrency keys share one partition.

Bounded queues use `dispatched_at` and `dispatch_expires_at` on `runlane_runs` as durable dispatch reservations. `reserveRunDispatch()` appends `run.delivery_requested` and creates the outbox row in the same transaction when the caller requests transport wakeup persistence after checking queue capacity for the selected queue and `concurrency_key` partition. Capacity counts queued runs with unexpired dispatch reservations plus running or cancellation-requested runs with unexpired leases.

`listRunsNeedingDispatch()` uses the same partition predicate for recovery scans. Postgres-only conformance tests run `EXPLAIN (ANALYZE, BUFFERS)` against realistic queued/running/cancellation-requested rows to keep the `runlane_runs_queue_capacity_idx` predicate aligned with that hot path. If the reservation expires before a worker claims the run, maintenance can reserve and publish it again.

Outbox rows live in `runlane_outbox_messages`. `claimOutboxMessages()` uses `FOR UPDATE SKIP LOCKED` to claim due `pending`, `failed`, or expired `claimed` rows and returns only rows owned by the new claim. `markOutboxMessagesPublished()`, `markOutboxMessagesFailed()`, and `markOutboxMessagesDeadLettered()` batch-update only rows whose current claim token matches; stale claims reject as storage conflicts.

Operator run lists and event history read indexed columns but return contract projections parsed from canonical JSONB. Run-list cursors include filter and sort scope, and event sequence sorting requires a `runId` filter. `RunSortField.RunAt` uses the same generated availability policy as worker scans, so queued, scheduled, released, retrying, and running runs sort by generated availability while terminal runs sort after runnable work.

`listRunsNeedingTimeoutFinalization()` uses `attempt_deadline_at` and only returns currently running runs. `listRunsWaitingForSignal()` and `listRunsWaitingForRunCompletion()` use the wait indexes and only return released runs whose current wait condition still matches the requested key.

`pruneRuns()` deletes only terminal runs whose `updated_at` is older than the concrete cutoff core passes to storage. It orders by `(updated_at, created_at, run_id)`, deletes at most the requested limit, and returns an opaque cursor when more rows in the same environment/status/cutoff scope remain. Deleting a run cascades its event history, durable step rows, outbox messages, idempotency owner, and singleton owner rows through foreign keys; schedule occurrence rows retain their `run_ids` array because they are keyed by occurrence, not by run. Observation records and exporter checkpoints are separate from operator run retention so exporters can still catch up after terminal run pruning.

Postgres driver and SQLSTATE failures are mapped into structured Runlane errors. Bad URLs, missing schemas, missing tables, and privilege/auth failures become `ConfigurationInvalid`; connection/resource failures become retryable `StorageUnavailable`; unique, serialization, and deadlock races become retryable `StorageConflict`; unknown failures become `InternalError`.

`postgresStorage().start()` probes the migrated runs table. An unreachable database, missing schema, or unapplied migration fails during `lane.start()` instead of waiting for the first run operation. Malformed options such as an invalid URL, unsupported protocol, unsupported key, or invalid schema name fail when `postgresStorage()` is called.


# SQS Transport (/contracts/sqs-transport)



`@runlane/transport-sqs` publishes Runlane wakeups to Amazon SQS. It is a production transport adapter, not a storage layer and not a compute integration.

SQS carries only enough data to find a durable run again: environment, run id, logical queue, requested time, and optional trace context. Storage still owns payloads, successful output, run state, leases, retries, releases, schedules, idempotency, singleton enforcement, and operator truth.

Use it when your production lane should wake Lambda, EC2, ECS, Fargate, or another Node process through SQS. Do not use it to store task payloads or to model AWS Batch, browser workers, retries, or schedules.

```sh
pnpm add @runlane/transport-sqs @aws-sdk/client-sqs
```

## Producer Transport [#producer-transport]

Create one transport and bind registered Runlane queue definitions to AWS SQS resources:

```ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { queue } from '@runlane/core'
import { sqsQueue, sqsTransport } from '@runlane/transport-sqs'

const defaultQueue = queue({ name: 'default', default: true })
const emailsQueue = queue({ name: 'emails' })
const mediaQueue = queue({ name: 'media' })

const transport = sqsTransport({
  client: new SQSClient({ region: 'us-east-1' }),
  queues: [
    sqsQueue(defaultQueue, {
      queueUrl: process.env.RUNLANE_DEFAULT_QUEUE_URL,
    }),
    sqsQueue(emailsQueue, {
      queueName: 'runlane-emails-production',
    }),
    sqsQueue(mediaQueue, {
      queueUrl: process.env.RUNLANE_MEDIA_QUEUE_URL,
    }),
  ],
})
```

`sqsQueue(queueDefinition, options)` is the provider binding. The queue definition owns Runlane routing and durable capacity policy; the SQS options own provider URL/name and FIFO behavior.

Use queues for workload routing such as `default`, `emails`, `billing`, `media`, and `imports`. Use separate lanes only for real infrastructure boundaries such as a different database, AWS account, region, tenant boundary, compliance boundary, or durability policy.

Each SQS queue binding accepts exactly one of `queueUrl` or `queueName`. `queueUrl` skips provider lookup. `queueName` calls AWS `GetQueueUrlCommand` once and caches the resolved URL inside the adapter; add `queueOwnerAWSAccountId` when resolving a queue owned by another AWS account. If the configured queue name cannot be resolved, `publishWakeups()` rejects with `RunlaneError` and `ErrorCode.ConfigurationInvalid`.

Unsupported options are rejected when the transport or queue binding is constructed. The SQS transport exposes its bound queue definitions, so `createRunlane({ queues })` rejects missing provider bindings or queue-policy drift before the first wakeup publish.

| Option                            | Default                                    | Notes                                                                                                                     |
| --------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| `batchSize`                       | `10`                                       | `SendMessageBatch` chunk size. Must be between 1 and the SQS maximum of 10.                                               |
| `client`                          | Required                                   | AWS SDK v3 SQS client or compatible `send()` surface.                                                                     |
| `name`                            | `'sqs'`                                    | Adapter name reported through the transport contract.                                                                     |
| `queues`                          | Required                                   | Array of `sqsQueue(queueDefinition, options)` bindings. Must contain at least one entry.                                  |
| `queues[].queueUrl`               | None                                       | Direct SQS queue URL. Skips lookup. Must be an `http://` or `https://` URL.                                               |
| `queues[].queueName`              | None                                       | SQS queue name. Resolved with `GetQueueUrlCommand` and cached. Queue names must satisfy SQS naming rules.                 |
| `queues[].queueOwnerAWSAccountId` | None                                       | Only valid with `queueName`.                                                                                              |
| `queues[].fifo.messageGroup`      | `SqsFifoMessageGroup.Run` (`'run'`)        | FIFO only. `Run` isolates groups per run; `Queue` groups by logical queue.                                                |
| `queues[].fifo.deduplication`     | `SqsFifoDeduplication.Outbox` (`'outbox'`) | FIFO only. `Outbox` sends a dedupe id derived from the durable outbox row; `ContentBased` omits `MessageDeduplicationId`. |

The package exports enums for the option and failure discriminants consumers may branch on:

| Enum                      | Values                                  | Where it appears                                                                    |
| ------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------- |
| `SqsFifoMessageGroup`     | `Run`, `Queue`                          | `queues[].fifo.messageGroup`                                                        |
| `SqsFifoDeduplication`    | `Outbox`, `ContentBased`                | `queues[].fifo.deduplication`                                                       |
| `SqsQueueSourceType`      | `QueueUrl`, `QueueName`                 | Parsed `SqsTransportConfig.queues[].source.type` from `resolveSqsTransportConfig()` |
| `SqsDeliveryFailurePhase` | `Message`, `Parse`, `Execute`, `Delete` | `SqsDeliveryFailure.phase` from Lambda and long-running consumer failure observers  |

## Wakeup Body [#wakeup-body]

SQS `MessageBody` contains a versioned envelope:

```ts
type SqsWakeupEnvelopeV1 = {
  type: 'runlane.wakeup'
  version: 1
  message: {
    environment: { name: string }
    runId: string
    queue: string
    requestedAt?: string
    traceCarrier?: Record<string, string>
  }
}
```

One durable outbox row becomes one SQS message. The adapter batches provider calls with `SendMessageBatch`, but it does not bundle multiple run ids into one JSON body. That keeps SQS visibility, Lambda partial batch failure, FIFO dedupe, and Runlane leases aligned to the same unit: one run wakeup.

`parseSqsWakeupBody(body)` validates the envelope with Zod, converts `requestedAt` back to a `Date`, and validates the resulting value with the contract-owned `deliveryMessageSchema`.

## Publish Behavior [#publish-behavior]

`sqsTransport().publishWakeups(command)` validates `PublishWakeupsCommand` with `publishWakeupsCommandSchema`, groups attempts by resolved SQS queue URL, and sends `SendMessageBatch` chunks using `batchSize`, capped by the SQS maximum of 10 entries. `outcomes[index]` always describes `command.attempts[index]`, even when AWS returns a mixed success/failure batch response.

AWS `Successful[]` entries become `WakeupPublishOutcomeType.Published`. AWS `Failed[]` entries become `WakeupPublishOutcomeType.Failed` with `ErrorCode.TransportPublishFailed`; provider codes, sender-fault flags, request ids, and queue details go in structured `meta`. Raw AWS error text is not copied into public failure messages.

If an AWS response omits an entry, duplicates an entry id, or contains an unrecognized entry id, the adapter throws an operation-level `RunlaneError` because indexed outcomes are no longer trustworthy. If the SDK call rejects before per-entry outcomes exist, the adapter throws `TransportUnavailable` for retryable provider/network failures or `TransportPublishFailed` for non-retryable provider failures.

## Capabilities [#capabilities]

The SQS adapter reports:

* `durableDelivery: true`
* `nativeDelay: false`
* `messageGrouping: true` only when every configured queue is FIFO
* `orderedDelivery: true` only when every configured queue is FIFO

`sqsTransport()` derives the configured `messageGrouping` and `orderedDelivery` flags from the parsed queue map so mixed standard/FIFO adapters do not overstate adapter-wide ordering guarantees.

Runlane does not use SQS native delay for schedules, releases, retries, or outbox recovery. Storage availability and outbox state decide when work is due.

## Standard Queues [#standard-queues]

Standard SQS queues are the default recommendation. Runlane correctness does not require provider ordering because every wakeup re-reads durable storage and tries to claim a lease before executing. Duplicate, delayed, stale, wrong-queue, already-leased, not-due, and terminal wakeups are ack-safe ignored results from `executeDelivery(message, options)`.

Create a standard queue and optional dead-letter queue through your normal IaC system. A minimal AWS CLI sketch:

```sh
aws sqs create-queue --queue-name runlane-default-production
aws sqs create-queue --queue-name runlane-default-production-dlq
```

Attach a redrive policy to the source queue so poison messages land in the DLQ after a bounded number of receives.

Standard source queues can share a standard DLQ. FIFO source queues need a FIFO DLQ; the AWS example provisions `Dlq` for `Emails` and `Heavy`, and `DlqFifo` for `Ordered`.

Set the SQS visibility timeout longer than the expected `executeDelivery()` handling window for that consumer. For Lambda event source mappings, AWS recommends a queue visibility timeout of at least six times the Lambda function timeout plus `MaximumBatchingWindowInSeconds`.

Do not share a Runlane SQS queue with non-Runlane messages unless your consumer deliberately parses and failure-handles those messages. The default parser treats malformed bodies as delivery failures so the provider can retry and eventually redrive them.

## Choosing A Consumer [#choosing-a-consumer]

SQS is a transport choice. After SQS wakeups are published, exactly one SQS-consuming deployment path should drain each SQS queue:

| Deployment                  | Uses SQS? | Acquisition path                                                       | Use when                                                                                 |
| --------------------------- | --------- | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Lambda SQS consumer         | Yes       | AWS invokes Lambda with SQS records.                                   | Lambda can run the lightweight orchestration work within its timeout and runtime limits. |
| Long-running SQS consumer   | Yes       | EC2, ECS, Fargate, or another Node process calls SQS `ReceiveMessage`. | You want SQS wakeups with long-running process control.                                  |
| Core storage-polling worker | No        | `runlane.worker({ mode: WorkerMode.Poll })` scans storage.             | You intentionally do not want SQS as the acquisition path.                               |

Do not publish SQS wakeups for a queue and then only run a storage-polling worker for that same queue. The polling worker can execute the durable runs, but it does not receive or delete SQS messages, so those provider messages become stale and eventually redrive or move to the DLQ.

Running a polling worker and SQS consumers against the same queue is usually wasteful. Storage leases prevent double execution, but two acquisition paths race for the same durable runs and make provider behavior harder to reason about. Use both only as a deliberate temporary migration or emergency backstop.

## FIFO Queues [#fifo-queues]

FIFO support is a queue-level provider option, not a separate Runlane abstraction:

```ts
import { queue } from '@runlane/core'
import { SqsFifoDeduplication, SqsFifoMessageGroup, sqsQueue, sqsTransport } from '@runlane/transport-sqs'

const emailsQueue = queue({ name: 'emails' })

const transport = sqsTransport({
  client,
  queues: [
    sqsQueue(emailsQueue, {
      queueUrl: process.env.RUNLANE_EMAILS_FIFO_QUEUE_URL,
      fifo: {
        messageGroup: SqsFifoMessageGroup.Run,
        deduplication: SqsFifoDeduplication.Outbox,
      },
    }),
  ],
})
```

Default `messageGroup: SqsFifoMessageGroup.Run` prevents one stuck run from blocking the whole logical queue. Use `messageGroup: SqsFifoMessageGroup.Queue` only when you intentionally want one ordered provider group for the whole logical queue.

Default `deduplication: SqsFifoDeduplication.Outbox` sends a provider-safe `MessageDeduplicationId` derived from the durable outbox message id. Use `deduplication: SqsFifoDeduplication.ContentBased` only when the FIFO queue has content-based deduplication enabled; the adapter will omit `MessageDeduplicationId`.

FIFO does not mean exactly-once task execution. Storage leases, durable run state, idempotency keys, singleton keys, and idempotent user code remain the correctness boundary.

If a configured queue URL or name ends in `.fifo`, include `fifo` options. If `fifo` options are present, the queue URL or name must point at a FIFO queue.

## Lambda Consumer [#lambda-consumer]

Use the Lambda helper when AWS invokes a function with SQS records:

```ts
import { createRunlane } from '@runlane/core'
import { createSqsLambdaHandler } from '@runlane/transport-sqs'

const runlane = createRunlane({
  lane,
  queues,
  tasks: { syncQuickBooksInvoices },
})

export const handler = createSqsLambdaHandler({
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
})
```

The helper does not import core. It parses each SQS record body, calls the supplied `executeDelivery(message, options)` callback, and returns Lambda's partial batch failure shape:

```ts
{
  batchItemFailures: [{ itemIdentifier: record.messageId }]
}
```

`createSqsLambdaHandler()` options:

| Option              | Required | Default | What it controls                                                                                                                                               |
| ------------------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `executeDelivery`   | Yes      | None    | Callback that should call `runlane.executeDelivery(message, options)` for the runtime handling this queue.                                                     |
| `deliveryOptions`   | No       | `{}`    | `leaseDuration`, `heartbeatInterval`, `workerId`, and `signal` forwarded into every delivery attempt.                                                          |
| `fifo`              | No       | `false` | Stops processing after the first failed record and returns that record plus later unprocessed records as batch failures. Required for FIFO ordering semantics. |
| `onDeliveryFailure` | No       | None    | Observer called before the helper reports a failed record. Rejections from the observer are treated as handler failures.                                       |

Do not run `runlane.tick()` at the end of this handler. The Lambda helper is the SQS record acknowledgment boundary; adding a global maintenance pass there couples provider batch latency to schedules, outbox recovery, lease recovery, and cancellation cleanup. Run maintenance from a separate scheduled function, cron process, container sidecar, or explicit CLI command.

Enable `ReportBatchItemFailures` on the Lambda event source mapping. Without it, Lambda does not process the returned `batchItemFailures` list: a handler that returns normally can acknowledge the whole provider batch, while a thrown handler failure retries the whole batch.

```sh
aws lambda create-event-source-mapping \
  --function-name runlane-worker \
  --event-source-arn arn:aws:sqs:us-east-1:123456789012:runlane-default-production \
  --batch-size 10 \
  --function-response-types ReportBatchItemFailures
```

For standard queues, the helper continues after a failed record and reports only failed records. For FIFO queues, pass `fifo: true`; the helper stops after the first failure and returns that failed record plus every unprocessed later record so provider ordering is preserved.

```ts
export const handler = createSqsLambdaHandler({
  fifo: true,
  deliveryOptions: {
    leaseDuration: '2m',
  },
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
})
```

`deliveryOptions.leaseDuration` is not an SQS visibility timeout. It is the Runlane lease written to storage when `runlane.executeDelivery(message, options)` claims the run.

In a Postgres lane, the lease means "this run is held by worker invocation X until time T." If the Lambda crashes or times out before persisting a task outcome, the lease expires and a later `tick()` can request delivery again for another consumer.

Use a lease at least as long as the Lambda timeout. A 1-minute Lambda commonly uses `leaseDuration: '2m'` to leave a recovery buffer.

Persisted user task outcomes are not SQS failures. If core records `run.succeeded`, `run.failed`, `run.retry_scheduled`, `run.released`, or `run.cancelled`, the SQS record is acknowledged because delivery handling completed. Framework, storage, projection, registration, or persistence failures reject so Lambda can retry the SQS record.

`onDeliveryFailure` is optional. Use it to log parse or execution failures before the helper reports partial batch failure. It receives:

```ts
type SqsDeliveryFailure = {
  phase: SqsDeliveryFailurePhase
  providerMessageId?: string
  deliveryMessage?: DeliveryMessage
  queueUrl?: string
  cause: unknown
}
```

Lambda failures use `SqsDeliveryFailurePhase.Message`, `SqsDeliveryFailurePhase.Parse`, or `SqsDeliveryFailurePhase.Execute`. If `onDeliveryFailure` rejects, the helper rejects too; observer failures are treated as application failures, not silently swallowed.

## Long-Running Consumer [#long-running-consumer]

Use the long-running consumer on EC2, ECS, Fargate, or another Node process when SQS is the work acquisition path. Its `start()` and `stop()` methods are only the lifecycle for this optional SQS receive loop; the core integration remains the supplied `executeDelivery(message, options)` callback.

```ts
import { SQSClient } from '@aws-sdk/client-sqs'
import { createRunlane } from '@runlane/core'
import { createSqsDeliveryConsumer } from '@runlane/transport-sqs'

const runlane = createRunlane({
  lane,
  queues,
  tasks: { syncQuickBooksInvoices },
})

const consumer = createSqsDeliveryConsumer({
  client: new SQSClient({ region: 'us-east-1' }),
  queueUrl: process.env.RUNLANE_QUICKBOOKS_QUEUE_URL,
  deliveryOptions: {
    leaseDuration: '2m',
  },
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
  maxNumberOfMessages: 5,
  waitTimeSeconds: 10,
  visibilityTimeoutSeconds: 120,
})

await consumer.start()

process.once('SIGTERM', () => {
  void consumer.stop()
})
```

Treat a consumer instance as single-start. `start()` is idempotent while the loop is already running; after `stop()` or a fatal `closed` rejection, create a new consumer instance to resume receiving.

The consumer loop:

1. Calls SQS `ReceiveMessage`.
2. Parses each wakeup body.
3. Calls `executeDelivery(message, options)`.
4. Deletes only messages whose delivery resolves.

`deliveryOptions` supports the same execution controls as core's `executeDelivery()` options: `leaseDuration`, `heartbeatInterval`, `maxAttemptDuration`, `signal`, and diagnostic `workerId`. Long-running consumers link forwarded signals to `stop()` and to the consumer-level `signal`, so cooperative task code can stop through `context.signal`.

Messages whose provider shape, wakeup body, or execution path rejects are not deleted. SQS visibility timeout and redrive policy handle retry and DLQ movement.

`onDeliveryFailure` receives the same `SqsDeliveryFailure` shape as the Lambda helper. Consumer failures may also include `SqsDeliveryFailurePhase.Message`, `SqsDeliveryFailurePhase.Delete`, and `queueUrl`.

`createSqsDeliveryConsumer()` options:

| Option                     | Required | Default       | What it controls                                                                           |
| -------------------------- | -------- | ------------- | ------------------------------------------------------------------------------------------ |
| `client`                   | Yes      | None          | AWS SDK v3 SQS client or compatible `send()` surface.                                      |
| `queueUrl`                 | Yes      | None          | SQS queue URL this consumer receives from. Start one consumer per provider queue URL.      |
| `executeDelivery`          | Yes      | None          | Callback that should call `runlane.executeDelivery(message, options)`.                     |
| `deliveryOptions`          | No       | `{}`          | Delivery lease, heartbeat, signal, and worker id controls forwarded into each run attempt. |
| `fifo`                     | No       | `false`       | Must be `true` when reading a FIFO queue URL ending in `.fifo`.                            |
| `maxNumberOfMessages`      | No       | `10`          | SQS receive batch size, bounded by the provider maximum.                                   |
| `waitTimeSeconds`          | No       | `20`          | SQS long-poll wait time per receive call.                                                  |
| `visibilityTimeoutSeconds` | No       | Queue default | Per-receive visibility override. Keep it longer than the delivery handling window.         |
| `signal`                   | No       | None          | Stops the receive loop cooperatively and links into forwarded delivery signals.            |
| `onDeliveryFailure`        | No       | None          | Observer for parse, execute, message-shape, or delete failures.                            |

If `DeleteMessageBatch` returns per-entry failures, the consumer reports each failed delete through `onDeliveryFailure` with `SqsDeliveryFailurePhase.Delete` when that observer is configured, and keeps the receive loop alive.

SQS will make undeleted messages visible again according to the queue visibility timeout and redrive policy. SDK-level receive and delete call failures reject `consumer.closed` because the loop cannot prove provider receive or acknowledgement state for that batch.

One consumer loop reads one SQS queue URL. Start one consumer per SQS queue that the process should handle:

```ts
const emailsConsumer = createSqsDeliveryConsumer({
  client,
  queueUrl: process.env.RUNLANE_EMAILS_QUEUE_URL,
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
})

const mediaConsumer = createSqsDeliveryConsumer({
  client,
  queueUrl: process.env.RUNLANE_MEDIA_QUEUE_URL,
  executeDelivery: (message, options) => runlane.executeDelivery(message, options),
})

await emailsConsumer.start()
await mediaConsumer.start()
```

This is different from `runlane.worker({ mode: WorkerMode.Poll })`. A polling worker scans storage and ignores SQS messages. An SQS consumer waits on SQS receive and handles delivered wakeups.

## IAM [#iam]

Producer permissions when every queue uses `queueUrl`:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["sqs:SendMessage"],
      "Resource": [
        "arn:aws:sqs:us-east-1:123456789012:runlane-default-production",
        "arn:aws:sqs:us-east-1:123456789012:runlane-emails-production"
      ]
    }
  ]
}
```

Add `sqs:GetQueueUrl` when using `queueName` resolution:

```json
{
  "Effect": "Allow",
  "Action": ["sqs:GetQueueUrl"],
  "Resource": "*"
}
```

Consumers need receive/delete permissions for the queues they drain. Lambda event source mappings commonly also require queue attribute reads; the long-running helper itself sends `ReceiveMessage` and `DeleteMessageBatch`.

```json
{
  "Effect": "Allow",
  "Action": ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes"],
  "Resource": "arn:aws:sqs:us-east-1:123456789012:runlane-default-production"
}
```

Add the required KMS permissions if your queues use a customer-managed KMS key.

## AWS Batch And External Compute [#aws-batch-and-external-compute]

Do not put AWS Batch support in the SQS transport. SQS should wake the lightweight Runlane delivery process; AWS Batch can own heavyweight compute such as headless Chrome.

Model external compute as ordinary task code that checkpoints the provider submission with `context.step.run()`, then releases the Runlane run while the external job is still running:

```ts
const batchSubmissionSchema = z.object({
  jobId: z.string().min(1),
})

const renderPdf = task({
  id: 'pdf.render',
  schema: renderPdfSchema,
  async run(payload, context) {
    const submission = await context.step.run('submit-batch-job', { output: batchSubmissionSchema }, ({ token }) =>
      submitBatchJob(payload, { clientToken: token, signal: context.signal }),
    )
    const job = await describeBatchJob(submission.jobId, { signal: context.signal })

    if (job.status !== 'SUCCEEDED') {
      return context.release('30s', { reason: 'batch_not_finished' })
    }

    await saveRenderedPdf(job)
  },
})
```

The step callback stores the small provider pointer needed to poll later attempts. If the run releases, retries, or crashes after the step completes, the next attempt reads the stored `jobId` and does not submit another Batch job. Pass the step token to AWS Batch or to your own submission ledger so the crash window between provider acceptance and Runlane step completion is idempotent.

That keeps transport, durable run lifecycle, and compute orchestration in the right layers. SQS wakes the Runlane delivery process; Runlane storage owns the run and step checkpoint; Batch owns the heavyweight compute.

## Provider Constraints [#provider-constraints]

The adapter encodes AWS hard ceilings as provider constraints and exposes caller-tunable options inside those bounds. Defaults favor efficient provider usage, but production consumers can reduce batch and long-poll sizes for latency, shutdown behavior, cost, or deployment-specific fairness.

| Option                              | Default                   | Valid range                                          | Applies to                                                                                                             |
| ----------------------------------- | ------------------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `batchSize`                         | `10`                      | `1..10`                                              | `sqsTransport()` publish batching.                                                                                     |
| `maxNumberOfMessages`               | `10`                      | `1..10`                                              | `createSqsDeliveryConsumer()` receive size.                                                                            |
| `waitTimeSeconds`                   | `20`                      | `0..20`                                              | `createSqsDeliveryConsumer()` long polling.                                                                            |
| `visibilityTimeoutSeconds`          | Provider queue default    | `0..43200`                                           | `createSqsDeliveryConsumer()` per-receive visibility override.                                                         |
| `deliveryOptions.leaseDuration`     | Core lease default (`5m`) | Positive duration string                             | Durable Runlane ownership window written by `executeDelivery()`. Set it greater than or equal to the consumer timeout. |
| `deliveryOptions.heartbeatInterval` | Half of lease duration    | Positive duration string shorter than lease duration | How often `executeDelivery()` extends the Runlane lease while task code runs.                                          |
| `deliveryOptions.workerId`          | Runtime default worker id | Runlane worker id                                    | Diagnostic identity recorded on leases; queues route work, worker ids do not.                                          |
| `deliveryOptions.signal`            | None                      | `AbortSignal`                                        | Cooperative abort signal forwarded into `TaskContext.signal`; long-running consumers also link it to consumer stop.    |

* `SendMessageBatch` sends at most 10 entries per request.
* `ReceiveMessage` receives at most 10 messages per request.
* SQS long polling waits at most 20 seconds per receive request.
* Message visibility timeout is bounded by SQS to 12 hours.
* `MessageGroupId` is required for FIFO queues.
* `MessageDeduplicationId` is required for FIFO queues unless content-based deduplication is enabled and configured in Runlane as `deduplication: SqsFifoDeduplication.ContentBased`.
* Lambda partial batch failure responses use `batchItemFailures[].itemIdentifier`.

AWS references: [SendMessageBatch](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html), [ReceiveMessage](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html), [Lambda with SQS](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html), [Lambda SQS event source configuration](https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-configure.html), and [SQS partial batch failures](https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-errorhandling.html).

## Local SQS Gate [#local-sqs-gate]

The package includes an opt-in Ministack integration suite for local SQS-compatible behavior. Start Ministack on the default edge port:

```sh
docker run --rm --name runlane-ministack-test -p 4566:4566 ministackorg/ministack
```

Then run the transport package's live SQS gate:

```sh
pnpm --filter @runlane/transport-sqs test:sqs:ministack
```

Stop the local container when the gate finishes:

```sh
docker stop runlane-ministack-test
```

The suite creates ephemeral standard and FIFO queues, publishes through `sqsTransport()`, receives through the real AWS SDK SQS client, and exercises the long-running consumer delete-after-resolve path. Set `RUNLANE_SQS_MINISTACK_ENDPOINT` or `RUNLANE_SQS_MINISTACK_REGION` when Ministack is not running at `http://localhost:4566` in `us-east-1`.

The package-local wrapper lives at `packages/transport-sqs/scripts/test-sqs-ministack-local.ts`; keep local SQS test setup changes there so the entry point remains discoverable from the package.


# Testing Package (/contracts/testing)



`@runlane/testing` contains reusable test support and executable adapter conformance suites for Runlane packages. It is a development dependency for package and adapter tests, not a second runtime, storage adapter, transport, or lane implementation.

Install it in packages that own Runlane tests:

```sh
pnpm add -D @runlane/testing
```

Install `vitest` only when the package imports the Vitest binding from `@runlane/testing/vitest`:

```sh
pnpm add -D vitest
```

The root `@runlane/testing` entry point is runner-agnostic and does not import Vitest.

Use it for deterministic fixtures and small observation helpers:

```ts
import { asId, RunStatus } from '@runlane/contracts'
import { appendQueuedRun, createControlledClock, waitForRunStatus } from '@runlane/testing'

const clock = createControlledClock()
const queuedRunFixture = await appendQueuedRun(lane, clock, asId<'run'>('run_123'))

await waitForRunStatus(runtime, queuedRunFixture.queuedRun.id, RunStatus.Succeeded)
```

`appendQueuedRun()` only needs a lane-shaped value with `storage`. `waitForRunStatus()` only needs `{ environment, lane: { storage } }`, which lets tests observe durable state without depending on a full runtime type. `waitForRunStatus()` and `waitForRunEvent()` accept optional polling controls such as `{ timeoutMs, intervalMs }` for live adapter tests.

## Entry Points [#entry-points]

| Entry point               | Use                                                                                                                                            |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `@runlane/testing`        | Runner-agnostic conformance definitions plus clocks, fixtures, observation helpers, fake loggers, storage stubs, and acknowledging transports. |
| `@runlane/testing/vitest` | Vitest registration helpers that bind the same conformance suites to Vitest's `describe()` and `test()`.                                       |

## Fixtures And Helpers [#fixtures-and-helpers]

| Helper family                   | Exports                                                                                                                                                                                                       |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Clocks                          | `createControlledClock()`, `createStepClock()`                                                                                                                                                                |
| Polling and observation         | `waitForCondition()`, `waitForRunStatus()`, `waitForRunEvent()`, `listRunEventTypes()`                                                                                                                        |
| Assertions                      | `getAt()`                                                                                                                                                                                                     |
| Run fixtures                    | `createQueuedRunData()`, `createScheduledRunData()`, `createSucceededRunData()`, `appendQueuedRun()`, `appendAndStartRun()`, `appendScheduledRun()`, `appendSucceededRun()`, `claimQueuedRun()`, `getLease()` |
| Shared identities               | `testEnvironment`, `testQueue`, `testSystemActor`, `testTaskId`, `testWorkerId`                                                                                                                               |
| Delivery and transport fixtures | `createDeliveryMessage()`, `createAcknowledgingTransport()`                                                                                                                                                   |
| Logger and storage stubs        | `createFakeLogger()`, `createStorageAdapterFixture()`                                                                                                                                                         |
| Runtime harness                 | `createRuntimeHarness()`                                                                                                                                                                                      |

`createStorageAdapterFixture()` is for focused unit tests. Its methods reject by default, and the test supplies only the storage methods it expects to call. Do not use it as evidence that a real adapter conforms to the storage contract; use storage conformance for that.

## Conformance Suites [#conformance-suites]

Conformance is the executable form of the primitive contracts. Adapter and lane packages provide fresh resources to the suite; each suite verifies one boundary instead of pretending a composed lane owns runtime behavior.

The stack is:

* storage conformance proves storage adapter shape, event atomicity, defensive copies, stale sequence conflicts, idempotency ownership, durable step completion, observation stream scans and checkpoints, lease and bounded-queue races, outbox transitions, run scans, operator reads, schedule occurrence claims, and pruning behavior.
* transport conformance proves transport adapter shape, publish command validation, rejected promises for malformed publish commands, and one valid publish outcome per wakeup attempt.
* lane composition conformance proves `Lane` shape, capability exposure, and the composed storage-to-transport outbox publish flow for transport delivery lanes.

Core runtime behavior is tested in `@runlane/core` against conforming adapters. Lane packages can add product-level integration tests over `createRunlane()` when they promise a ready-to-use experience, as `@runlane/lane-local` does. Worker loops, schedules, operator APIs, retry, release, and cancellation do not belong in lane composition conformance or lane implementation code.

Vitest users can use the first-party runner adapter:

```ts
import { createLocalStorage } from '@runlane/local-adapters'
import { describeStorageConformance } from '@runlane/testing/vitest'

describeStorageConformance({
  createStorage: createLocalStorage,
  name: 'local memory',
})
```

Jest or another runner can bind the same suite without pulling in Vitest:

```ts
import { createLocalStorage } from '@runlane/local-adapters'
import { defineStorageConformanceSuite, runConformanceSuite } from '@runlane/testing'

runConformanceSuite(
  {
    describe,
    test,
  },
  defineStorageConformanceSuite({
    createStorage: createLocalStorage,
    name: 'local memory',
  }),
)
```

Use conformance for the shared contract. Keep adapter-specific tests beside the adapter for migrations, provider errors, SQLSTATE or SDK mapping, connection lifecycle, backend-specific performance or locking behavior, and local implementation regression history.

## How Registration Works [#how-registration-works]

The Vitest helpers only register tests. They do not discover local, Postgres, SQS, or custom implementations. The adapter package chooses the implementation by passing a factory:

```ts
import { createLocalTransport } from '@runlane/local-adapters'
import { describeTransportConformance } from '@runlane/testing/vitest'

describeTransportConformance({
  createTransport: createLocalTransport,
  name: 'local memory',
})
```

The same pattern exists for lane composition:

```ts
import { createLocalLane } from '@runlane/lane-local'
import { describeLaneCompositionConformance } from '@runlane/testing/vitest'

describeLaneCompositionConformance({
  createLane: createLocalLane,
  name: 'local',
})
```

Each conformance test calls the supplied factory for its own fresh resource. If the returned adapter or lane has `start()` or `close()` hooks, conformance runs them around that single test. If the factory needs external cleanup, return an envelope with the resource key and optional cleanup:

* storage: `{ storage, cleanup? }`
* transport: `{ transport, cleanup? }`
* lane: `{ lane, cleanup? }`

`cleanup()` runs after `close()`. The Postgres and Postgres/SQS package tests use this shape to create one migrated schema per conformance resource and drop it after the test.

Internally, helpers such as `describeStorageConformance(...)` and `describeLaneCompositionConformance(...)` call their matching `define*ConformanceSuite(options)` functions.

The suite definition is a runner-agnostic tree of suite names and test bodies. `runConformanceSuite(runner, suite)` walks that tree and calls the runner's `describe(...)` and `test(...)`.

The first-party `@runlane/testing/vitest` export supplies Vitest as that runner; Jest users can pass Jest's globals directly to `runConformanceSuite()`.

The important separation is:

* `define*ConformanceSuite(...)`: runner-agnostic suite definition.
* `runConformanceSuite(...)`: bridge from a suite tree to any compatible test runner.
* `@runlane/testing/vitest`: first-party Vitest registration helpers.
* `*.conformance.test.ts`: adapter package-owned test files that choose which implementation factory is under test.

## What Conformance Does Not Prove [#what-conformance-does-not-prove]

Conformance is necessary for adapter authors, but it is not an operational test plan. It does not prove:

* migrations or schema upgrade paths
* provider credentials, IAM, queue provisioning, redrive policies, FIFO grouping, or regional behavior
* driver-specific error mapping beyond the contract errors the suite exercises
* production performance, locking behavior, or query plans
* runtime behavior such as task registration, workers, schedules, retry, release, cancellation, pruning orchestration, or operator APIs

Those tests stay in the package that owns the adapter, lane, or runtime behavior. For example, `@runlane/postgres-storage` adds migration and query-plan tests beside storage conformance, and `@runlane/lane-local` adds `createRunlane()` integration tests beside lane composition conformance.

## Boundary Rules [#boundary-rules]

The testing package depends on contracts, utilities, and test-only libraries. It does not depend on `@runlane/core`, `@runlane/lane-local`, or adapter packages. The root `@runlane/testing` export is runner-agnostic; `@runlane/testing/vitest` is the first-party Vitest binding. Package-specific tests pass their own adapters, transports, or lanes into conformance factories.

The runtime harness is intentionally thin. It supplies shared clock and environment defaults to a caller-provided factory; concrete runtime packages still own their construction rules.

Inside this repository, workspace tests resolve Runlane packages to source through the shared Vitest config. That keeps test execution on the code under edit instead of stale package `dist` output.


# Examples (/examples)



This section points to Runlane examples backed by shipped source or checked API snippets.

## Available Examples [#available-examples]

| Example                                         | Covers                                                                                                                                                                                                                    | Source Path                    |
| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| [Local Lane](./local-lane.mdx)                  | One-process development/test lane with task registration, trigger, drain worker, schedule tick, release, CLI dev bridge, and operator reads.                                                                              | `docs/examples/local-lane.mdx` |
| [Postgres + SQS On AWS](./postgres-sqs-aws.mdx) | Deployable AWS reference app using RDS Postgres, standard and FIFO SQS queues, SQS-Lambda consumers, EventBridge tick Lambda, HTTP trigger Lambda, a Fargate worker, local `pnpm dev`, and shared runtime factory wiring. | `examples/aws/`                |

Full reference apps name the shipped directory to run. Focused pages keep the code short, but still use exported packages and CLI commands.


# Local Lane (/examples/local-lane)



This example runs Runlane in one Node process with `@runlane/lane-local`. It uses the same task, run, event, lease, schedule, release, and operator APIs as production lanes, but all state lives in memory, reports `processLocalState: true`, and disappears when the process exits.

Use this path for first setup, demos, and application tests. Do not use it as a production durability boundary.

## Install [#install]

```sh
pnpm add @runlane/core @runlane/lane-local @runlane/cli zod
```

`@runlane/cli` is only needed for the CLI section. The runtime example itself uses `@runlane/core`, `@runlane/lane-local`, and your schema package.

## Define Work [#define-work]

```ts
import { queue, task } from '@runlane/core'
import * as z from 'zod'

const sentEmails: string[] = []
const emailQueue = queue({ name: 'emails', default: true })

const sendWelcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema: z.object({ userId: z.string().min(1) }),
  idempotencyKey: (payload) => `emails.welcome.${payload.userId}`,
  async run(payload, context) {
    sentEmails.push(payload.userId)
    context.signal.throwIfAborted()
  },
})
```

The idempotency key means duplicate producer calls for the same user return the same active or retained terminal run instead of creating a second welcome-email run.

## Create The Runtime [#create-the-runtime]

```ts
import { createRunlane } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'

export function createAppRunlane() {
  return createRunlane({
    lane: createLocalLane(),
    queues: [emailQueue],
    tasks: { sendWelcomeEmail },
  })
}

const runlane = createAppRunlane()
```

The named task catalog is authoritative. Trigger with `runlane.tasks.sendWelcomeEmail` so the task handle comes from the same registry workers execute.

## Trigger And Drain [#trigger-and-drain]

```ts
import { WorkerMode } from '@runlane/core'

const { run: queuedRun } = await runlane.trigger(runlane.tasks.sendWelcomeEmail, {
  userId: 'user_123',
})

const worker = runlane.worker({ mode: WorkerMode.Drain })
await worker.closed

const completedRun = await runlane.runs.get(queuedRun.id)
```

Drain mode processes currently due work and exits. It is the easiest worker mode for tests because it does not leave a polling loop running.

## Run From The CLI [#run-from-the-cli]

`runlane dev` gives local lanes a two-terminal workflow without turning process memory into shared storage. The long-running dev process owns the in-memory lane, starts a polling worker, runs maintenance ticks, and opens a loopback control bridge. Stateful commands in another terminal proxy into that process when they use the same resolved config path.

Create `runlane.config.ts` next to `package.json`:

```ts
import { type RunlaneCliConfig } from '@runlane/cli'

import { createAppRunlane } from './src/runlane.ts'

export default {
  runtime: createAppRunlane,
} satisfies RunlaneCliConfig
```

```json
{
  "scripts": {
    "dev": "runlane dev"
  }
}
```

```sh
# Terminal 1
pnpm dev

# Terminal 2
pnpm exec runlane trigger emails.welcome '{"userId":"user_123"}'
pnpm exec runlane runs list
pnpm exec runlane runs get <runId> --events
```

Restart `runlane dev` after changing task definitions or runtime wiring. Proxied commands execute against the task catalog already loaded in the dev process.

## Add Business Waiting [#add-business-waiting]

```ts
const reportsSeen = new Set<string>()

const pollReport = task({
  id: 'reports.poll',
  queue: emailQueue,
  schema: z.object({ reportId: z.string().min(1) }),
  async run(payload, context) {
    if (!reportsSeen.has(payload.reportId)) {
      reportsSeen.add(payload.reportId)

      return context.release('5s', { reason: 'provider_not_ready' })
    }

    await saveReport(payload.reportId, { signal: context.signal })
  },
})
```

`context.release()` records expected waiting, not failure. The run becomes `released`; after the delay, `tick()` requests delivery again and a worker executes the next attempt.

## Add A Schedule [#add-a-schedule]

```ts
const dailyDigest = task({
  id: 'digests.daily',
  queue: emailQueue,
  schema: z.object({ tenantId: z.string().min(1) }),
  schedule: {
    id: 'digests.daily.default',
    cron: '0 9 * * *',
    timeZone: 'America/New_York',
    payload: { tenantId: 'tenant_123' },
  },
  async run(payload) {
    await sendDailyDigest(payload.tenantId)
  },
})
```

Register scheduled tasks in the runtime's task catalog and run maintenance:

```ts
const scheduledRunlane = createRunlane({
  lane: createLocalLane(),
  queues: [emailQueue],
  tasks: { dailyDigest },
})

await scheduledRunlane.tick()
await scheduledRunlane.worker({ mode: WorkerMode.Drain }).closed
```

`tick()` materializes due schedule occurrences and writes ordinary runs. It does not execute handlers; workers still execute the materialized runs.

## Inspect History [#inspect-history]

```ts
import { RunEventSortField, SortDirection } from '@runlane/core'

const events = await runlane.runs.events(queuedRun.id, {
  sortBy: RunEventSortField.Sequence,
  sortDirection: SortDirection.Asc,
})
```

The local lane keeps append-only event history in memory, so tests can assert public behavior through runs and events instead of mocking internal functions.

## Local Limits [#local-limits]

| Behavior                       | Local lane result                                                                 |
| ------------------------------ | --------------------------------------------------------------------------------- |
| Process restart                | All runs, events, schedules, leases, and ownership state are lost.                |
| Idempotency and singleton keys | Enforced only inside the current process.                                         |
| Queue concurrency              | Enforced only inside the current process.                                         |
| Process boundaries             | A second process cannot load the same state directly; CLI dev proxying is narrow. |
| Wakeup transport               | No external transport; workers poll the in-memory storage adapter.                |
| Production crash recovery      | Not proven. Use a production storage adapter and conformance tests.               |
| Operator APIs                  | Available for development and tests through `runlane.runs.*`.                     |


# Postgres + SQS On AWS (/examples/postgres-sqs-aws)



`@runlane/example-aws` is a deployable reference app showing the production lane composed from `@runlane/postgres-storage` and `@runlane/transport-sqs`.

It deploys RDS Postgres, three SQS wakeup queues, two dead-letter queues, two SQS-Lambda consumers, an EventBridge cron Lambda for `runlane.tick()`, an HTTP trigger Lambda, and a Fargate service running `createSqsDeliveryConsumer`. Prisma owns the application data layer.

The full source lives in `examples/aws/`. This page explains the architecture and the design choices behind it.

## Topology [#topology]

| Component                            | Role                                                                                                                                                                                |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| RDS Postgres `db.t4g.micro`          | Durable Runlane storage + app data, publicly reachable for tear-down demo convenience                                                                                               |
| SQS queue `Emails` (standard)        | Wakeups consumed by the SQS-Lambda handler                                                                                                                                          |
| SQS queue `Heavy` (standard)         | Wakeups consumed by the Fargate long-running consumer                                                                                                                               |
| SQS queue `Ordered` (FIFO)           | Wakeups consumed by the FIFO SQS-Lambda handler with `fifo: true` semantics                                                                                                         |
| SQS queue `Dlq` (standard)           | Shared dead-letter target for `Emails` and `Heavy` (`retry: 5`)                                                                                                                     |
| SQS queue `DlqFifo` (FIFO)           | Dead-letter target for `Ordered`; SQS requires FIFO sources to use FIFO DLQs                                                                                                        |
| Lambda `delivery-emails`             | `createSqsLambdaHandler` invoking `runlane.executeDelivery()` for `Emails`                                                                                                          |
| Lambda `delivery-ordered`            | `createSqsLambdaHandler({ fifo: true })` for `Ordered`                                                                                                                              |
| Lambda `Tick` (EventBridge cron, 1m) | `runlane.tick()` for schedules, retries, lease recovery, outbox flush                                                                                                               |
| Lambda `Trigger` (Function URL)      | HTTP `runlane.trigger()` for ad-hoc smoke tests                                                                                                                                     |
| Fargate Service `Worker`             | `createSqsDeliveryConsumer` for `Heavy`, ECS Exec enabled                                                                                                                           |
| VPC without NAT or bastion           | Hosts the Fargate cluster; Lambdas run outside the VPC and all compute reaches the public RDS endpoint                                                                              |
| `sst.x.DevCommand("AppEnv")`         | Exposes deploy outputs (DATABASE\_URL, queue URLs, Trigger URL) for `sst shell --target AppEnv -- <cmd>` so Prisma migrate, seed, and smoke scripts find them through `process.env` |

Three acquisition paths run side by side:

| Queue     | Consumer                      | Why                                                                                                                        |
| --------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `Emails`  | SQS-Lambda                    | AWS invokes Lambda with provider records, and the handler resolves runs through `executeDelivery`.                         |
| `Heavy`   | Fargate long-running consumer | The task calls `ReceiveMessage` and forwards each delivery to `executeDelivery`.                                           |
| `Ordered` | FIFO SQS-Lambda               | `fifo: true` stops a batch on first failure; `messageGroup: 'queue'` uses one provider group for cross-run queue ordering. |

All three write to the same Postgres durable store.

`tick()` runs in its own Lambda on a one-minute EventBridge schedule. It is **not** automatic with the Lambda SQS path; Lambda handles the provider records AWS delivered to that invocation, not global maintenance.

Schedules, retries, releases, lease recovery, and outbox publishing all need `tick()` running somewhere. Without it, schedules never fire, retried runs never wake, and crashed leases never recover.

## Why both Lambda and Fargate [#why-both-lambda-and-fargate]

The example demonstrates that Runlane treats them as interchangeable consumers of the same durable lane:

* **Lambda** suits fast, stateless tasks within the 15-minute ceiling. Pay-per-invocation. No process management.
* **Fargate** suits long-running tasks, tasks that need warm connections to slow downstream services, or workloads that exceed Lambda's runtime/memory ceilings. Always-on cost; per-second billing.

Each Runlane logical queue is consumed by exactly one acquisition path. The example routes `emails.welcome` and `verify.heartbeat` to `Emails` (Lambda), `media.transcode` to `Heavy` (Fargate), and `account.apply-event` to `Ordered` (FIFO Lambda). Workloads that span both don't share a queue — split them by the resource constraint they actually have.

## Local Development [#local-development]

The same example runs locally without AWS. Local Postgres still owns app data through Prisma, but Runlane state uses the in-memory local lane:

```sh
pnpm install
cd examples/aws
pnpm db:generate
pnpm db:emit-runlane-sql

docker run -d --name runlane-pg -p 5432:5432 \
  -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=runlane-test postgres:17

cp .env.example .env.local
pnpm db:migrate
pnpm seed:local

# Terminal 1: local worker plus maintenance loop.
pnpm dev

# Terminal 2: trigger and inspect work through the dev bridge.
pnpm exec runlane trigger emails.welcome '{"userId":"usr_demo_alice"}'
pnpm exec runlane runs list
pnpm exec runlane runs get <runId> --events
```

Set `APP_ENV=local` in `.env.local`. `createAppRuntime()` then composes `createLocalLane()`, and `runlane dev` owns the in-memory runtime, starts the polling worker and maintenance loop, and opens the CLI dev bridge. Stateful CLI commands proxy into that running process, so triggers and operator reads see the same in-memory lane.

`APP_ENV=dev` is the AWS-shaped path used by `sst dev`, `sst shell`, and deployed resources. It requires `RUNLANE_QUEUE_EMAILS_URL`, `RUNLANE_QUEUE_HEAVY_URL`, and `RUNLANE_QUEUE_ORDERED_URL`, then composes the Postgres+SQS lane.

`pnpm dev` intentionally builds `@runlane/cli` before starting `runlane dev`. In this monorepo, the workspace `runlane` binary points at `packages/cli/dist`, so the preflight build prevents local examples from accidentally using stale CLI/core output after source edits.

## Runtime Factory [#runtime-factory]

Every role imports the same runtime factory. That keeps lane composition, queue policy, provider bindings, environment name, and task registration in one place:

```ts
import { createRunlane, queue, type Lane } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'
import { postgresSqsLane } from '@runlane/lane-postgres-sqs'
import { SqsFifoDeduplication, SqsFifoMessageGroup, sqsQueue } from '@runlane/transport-sqs'

import { env } from '../env.js'
import { buildApplyAccountEventTask } from '../tasks/apply-account-event.js'
import { buildHeartbeatTask } from '../tasks/heartbeat.js'
import { buildTranscodeMediaTask } from '../tasks/transcode-media.js'
import { buildWelcomeEmailTask } from '../tasks/welcome-email.js'
import { getSqsClient } from './sqs.js'

export const emailQueue = queue({
  default: true,
  name: 'emails',
})

export const heavyQueue = queue({
  dispatchTimeout: '5m',
  concurrencyLimit: 5,
  name: 'heavy',
})

export const orderedQueue = queue({
  dispatchTimeout: '2m',
  concurrencyLimit: 1,
  name: 'ordered',
})

export function createAppRuntime() {
  let lane: Lane

  if (env.APP_ENV === 'local') {
    lane = createLocalLane()
  } else {
    if (
      env.RUNLANE_QUEUE_EMAILS_URL === undefined ||
      env.RUNLANE_QUEUE_HEAVY_URL === undefined ||
      env.RUNLANE_QUEUE_ORDERED_URL === undefined
    ) {
      throw new Error(
        'APP_ENV=dev requires RUNLANE_QUEUE_EMAILS_URL, RUNLANE_QUEUE_HEAVY_URL, and RUNLANE_QUEUE_ORDERED_URL. Set APP_ENV=local in .env.local for in-memory dev without SQS.',
      )
    }

    lane = postgresSqsLane({
      postgres: {
        connectionString: env.DATABASE_URL,
        schema: 'public',
      },
      sqs: {
        client: getSqsClient(env.AWS_REGION),
        queues: [
          sqsQueue(emailQueue, { queueUrl: env.RUNLANE_QUEUE_EMAILS_URL }),
          sqsQueue(heavyQueue, { queueUrl: env.RUNLANE_QUEUE_HEAVY_URL }),
          sqsQueue(orderedQueue, {
            queueUrl: env.RUNLANE_QUEUE_ORDERED_URL,
            fifo: {
              deduplication: SqsFifoDeduplication.ContentBased,
              messageGroup: SqsFifoMessageGroup.Queue,
            },
          }),
        ],
      },
    })
  }

  return createRunlane({
    environment: { name: env.RUNLANE_ENVIRONMENT },
    lane,
    queues: [emailQueue, heavyQueue, orderedQueue],
    tasks: {
      applyAccountEvent: buildApplyAccountEventTask(env),
      heartbeat: buildHeartbeatTask(),
      transcodeMedia: buildTranscodeMediaTask(env),
      welcomeEmail: buildWelcomeEmailTask(env),
    },
  })
}
```

`heavy` and `ordered` are bounded queues because they set `concurrencyLimit`. That is durable queue capacity enforced in Postgres, not local worker concurrency. `heavy` allows five active dispatch slots; `ordered` allows one. For bounded queues, a trigger creates the run first, then `tick()` reserves capacity and writes `run.delivery_requested` when a slot is available.

The CLI config stays a thin adapter over that factory:

```ts
import { type RunlaneCliConfig } from '@runlane/cli'

import { createAppRuntime } from './src/runtime/runlane.js'

export default {
  runtime: () => createAppRuntime(),
} satisfies RunlaneCliConfig
```

## Role Entrypoints [#role-entrypoints]

The standard SQS Lambda handler starts the shared runtime once per execution environment and delegates every provider record to `runlane.executeDelivery()`:

```ts
import { createSqsLambdaHandler } from '@runlane/transport-sqs'

import { formatLogCause } from '../runtime/logging.js'
import { createAppRuntime } from '../runtime/runlane.js'

const runlane = createAppRuntime()
const startupPromise = runlane.start()

export const handler = createSqsLambdaHandler({
  deliveryOptions: {
    leaseDuration: '2m',
  },
  async executeDelivery(message, options) {
    await startupPromise
    return runlane.executeDelivery(message, options)
  },
  onDeliveryFailure(failure) {
    process.stderr.write(
      `[delivery-emails] phase=${failure.phase} runId=${failure.deliveryMessage?.runId ?? 'unknown'} cause=${formatLogCause(failure.cause)}\n`,
    )
  },
})
```

The FIFO SQS Lambda handler adds `fifo: true`, so the helper stops the batch after the first failure and reports that record plus later unprocessed records as batch failures:

```ts
import { createSqsLambdaHandler } from '@runlane/transport-sqs'

import { formatLogCause } from '../runtime/logging.js'
import { createAppRuntime } from '../runtime/runlane.js'

const runlane = createAppRuntime()
const startupPromise = runlane.start()

export const handler = createSqsLambdaHandler({
  fifo: true,
  deliveryOptions: {
    leaseDuration: '2m',
  },
  async executeDelivery(message, options) {
    await startupPromise
    return runlane.executeDelivery(message, options)
  },
  onDeliveryFailure(failure) {
    process.stderr.write(
      `[delivery-ordered] phase=${failure.phase} runId=${failure.deliveryMessage?.runId ?? 'unknown'} cause=${formatLogCause(failure.cause)}\n`,
    )
  },
})
```

The Fargate worker uses the long-running SQS consumer helper for `Heavy`. It still uses SQS as the acquisition path; it does not run a storage-polling worker:

```ts
import { createSqsDeliveryConsumer } from '@runlane/transport-sqs'

import { env } from '../env.js'
import { formatLogCause } from '../runtime/logging.js'
import { createAppRuntime } from '../runtime/runlane.js'
import { getSqsClient } from '../runtime/sqs.js'

async function main(): Promise<void> {
  const heavyQueueUrl = env.RUNLANE_QUEUE_HEAVY_URL

  if (heavyQueueUrl === undefined) {
    throw new Error('Worker requires RUNLANE_QUEUE_HEAVY_URL. This entrypoint runs the deployed Fargate consumer.')
  }

  const runlane = createAppRuntime()

  await runlane.start()

  const consumer = createSqsDeliveryConsumer({
    client: getSqsClient(env.AWS_REGION),
    queueUrl: heavyQueueUrl,
    deliveryOptions: {
      leaseDuration: '5m',
    },
    executeDelivery: (message, options) => runlane.executeDelivery(message, options),
    maxNumberOfMessages: 5,
    waitTimeSeconds: 20,
    visibilityTimeoutSeconds: 360,
    onDeliveryFailure(failure) {
      process.stderr.write(
        `[worker] phase=${failure.phase} runId=${failure.deliveryMessage?.runId ?? 'unknown'} cause=${formatLogCause(failure.cause)}\n`,
      )
    },
  })

  process.once('SIGTERM', () => {
    process.stdout.write('[worker] SIGTERM received, draining...\n')
    void consumer.stop()
  })
  process.once('SIGINT', () => {
    process.stdout.write('[worker] SIGINT received, draining...\n')
    void consumer.stop()
  })

  await consumer.start()
  process.stdout.write(`[worker] listening on ${heavyQueueUrl}\n`)

  await consumer.closed
  await runlane.close()
  process.stdout.write('[worker] shutdown complete\n')
}
```

Maintenance is separate from SQS delivery. The tick Lambda runs the global maintenance pass on an EventBridge schedule:

```ts
import { type ScheduledHandler } from 'aws-lambda'

import { createAppRuntime } from '../runtime/runlane.js'

const runlane = createAppRuntime()

export const handler: ScheduledHandler = async () => {
  await runlane.start()
  await runlane.tick()
}
```

## Tasks [#tasks]

### `emails.welcome` (`emailQueue`, Lambda) [#emailswelcome-emailqueue-lambda]

```ts
const welcomeEmailDeliverySchema = z.object({
  providerMessageId: z.string().min(1),
})

const welcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema: z.object({ userId: z.string().min(1) }),
  idempotencyKey: (payload) => `emails.welcome.${payload.userId}`,
  retry: { maxAttempts: 3, backoff: { type: RetryBackoffType.Exponential, delay: '5s', maxDelay: '1m' } },
  async run(payload, context) {
    const user = await prisma.user.findUnique({ where: { id: payload.userId } })
    if (!user) {
      throw new Error(`User ${payload.userId} not found`)
    }

    if (user.welcomeEmailSentAt) return

    await context.step.run('send-welcome-email', { output: welcomeEmailDeliverySchema }, ({ token }) =>
      sendEmailViaProvider({
        name: user.name,
        signal: context.signal,
        to: user.email,
        token,
      }),
    )
    await prisma.user.update({
      where: { id: user.id },
      data: { welcomeEmailSentAt: new Date() },
    })
  },
})
```

Idempotency key collapses duplicate triggers for the same user to one run. The durable step checkpoints the provider send, so a crash after the email provider accepts the message but before the Prisma update can retry the run without sending another message after the step has completed. The provider should receive the step token as its idempotency token when it supports one.

### `media.transcode` (`heavyQueue`, Fargate) [#mediatranscode-heavyqueue-fargate]

```ts
const transcodeSubmissionSchema = z.object({
  jobId: z.string().min(1),
})

const transcodeMedia = task({
  id: 'media.transcode',
  queue: heavyQueue,
  schema: z.object({ mediaId: z.string().min(1) }),
  singletonKey: (payload) => `media.transcode.${payload.mediaId}`,
  retry: { maxAttempts: 5, backoff: { type: RetryBackoffType.Exponential, delay: '15s', maxDelay: '5m' } },
  async run(payload, context) {
    const media = await prisma.media.findUnique({ where: { id: payload.mediaId } })
    if (!media) {
      throw new Error(`Media ${payload.mediaId} not found`)
    }

    if (media.status === 'ready') return

    if (media.status !== 'transcoding') {
      await prisma.media.update({ where: { id: media.id }, data: { status: 'transcoding' } })
    }

    const submission = await context.step.run(
      'submit-transcode-job',
      { output: transcodeSubmissionSchema },
      ({ token }) =>
        submitFakeBatchJob({
          mediaId: media.id,
          signal: context.signal,
          token,
        }),
    )
    const status = await pollFakeBatchJob({
      jobId: submission.jobId,
      mediaId: media.id,
      attempt: context.attempt,
      signal: context.signal,
    })
    if (status === 'pending') {
      return context.release('30s', { reason: 'transcode_in_progress' })
    }

    await prisma.media.update({
      where: { id: media.id },
      data: { status: 'ready', transcodedAt: new Date() },
    })
  },
})
```

Singleton key prevents two concurrent transcode runs for the same media. `context.step.run('submit-transcode-job', ...)` checkpoints the provider job id so a later release, retry, timeout, or process crash does not submit the external job again after the step is complete. `context.release('30s', ...)` is the canonical "external batch job not done yet" continuation pattern from the [SQS transport docs](../contracts/sqs-transport.mdx#aws-batch-and-external-compute) and [Durable Steps](../concepts/durable-steps.mdx): the run becomes `released`, `tick()` re-publishes a wakeup at the future `runAt`, and the Fargate consumer picks it up again.

### `account.apply-event` (`orderedQueue`, FIFO Lambda) [#accountapply-event-orderedqueue-fifo-lambda]

```ts
const applyAccountEvent = task({
  id: 'account.apply-event',
  queue: orderedQueue,
  schema: z.object({
    userId: z.string().min(1),
    eventId: z.string().min(1),
    kind: z.enum(['plan_upgraded', 'plan_downgraded', 'address_changed']),
  }),
  idempotencyKey: (payload) => `account.apply-event.${payload.userId}.${payload.eventId}`,
  retry: { maxAttempts: 3, backoff: { type: RetryBackoffType.Exponential, delay: '5s', maxDelay: '1m' } },
  async run(payload) {
    const user = await prisma.user.findUnique({ where: { id: payload.userId } })
    if (!user) {
      throw new Error(`User ${payload.userId} not found`)
    }

    process.stdout.write(`[account.apply-event] user=${user.id} event=${payload.eventId} kind=${payload.kind}\n`)
  },
})
```

The queue binding uses SQS FIFO with `messageGroup: SqsFifoMessageGroup.Queue`, so all `Ordered` wakeups share one provider group. The task still owns duplicate event collapse through an idempotency key; FIFO ordering does not replace application-level idempotency.

## Trigger Lambda [#trigger-lambda]

The HTTP trigger Lambda is only a smoke-test gateway. It still uses the same task objects registered into the runtime:

```ts
const runlane = createAppRuntime()

await runlane.start()

const triggerResult = await dispatchTrigger(request.taskId, request.payload, {
  traceCarrier: { 'aws.lambda.request_id': event.requestContext.requestId },
})
if (!triggerResult) return reply(404, { code: 'unknown_task', message: 'Unknown task id' })
return reply(202, { runId: triggerResult.run.id, status: triggerResult.run.status })

async function dispatchTrigger(taskId: string, payload: unknown, options: TriggerRunOptions) {
  switch (taskId) {
    case runlane.tasks.welcomeEmail.id:
      return runlane.trigger(runlane.tasks.welcomeEmail, welcomeEmailPayloadSchema.parse(payload), options)
    case runlane.tasks.transcodeMedia.id:
      return runlane.trigger(runlane.tasks.transcodeMedia, transcodeMediaPayloadSchema.parse(payload), options)
    case runlane.tasks.applyAccountEvent.id:
      return runlane.trigger(runlane.tasks.applyAccountEvent, applyAccountEventPayloadSchema.parse(payload), options)
    default:
      return null
  }
}
```

Do not rebuild task definitions inside request handlers. A runtime with an authoritative `tasks` catalog rejects different task objects with the same id because that is configuration drift. Also keep request ids in trace metadata; do not pass them as `idempotencyKey`, or they override task-owned idempotency and singleton keys.

## Migration approach [#migration-approach]

The example takes [option 1 from the Postgres storage migration tooling guide](../contracts/postgres-storage.mdx#option-1-prisma-migrate-runs-both-recommended): Prisma runs both Runlane and app migrations.

```sh
pnpm db:emit-runlane-sql                    # snapshots @runlane/postgres-storage SQL
pnpm db:migrate                             # prisma migrate deploy applies all in order
```

`schema.prisma` describes only app tables (`User`, `Media`). `getPostgresStorageMigrationSql({ schema: 'public' })` owns the Runlane table definition; the snapshot script writes it into `prisma/migrations/00000000000000_runlane_init/migration.sql`. The all-zero timestamp prefix sorts first lexicographically so Runlane tables exist before any app migration that depends on the schema.

Application code never queries Runlane tables through Prisma. Use `runlane.runs.list`, `runlane.runs.get`, and the rest of the operator API.

## Deploy and smoke test [#deploy-and-smoke-test]

Run from the repo root first, then the example directory:

```sh
pnpm install
cd examples/aws
pnpm db:generate
pnpm db:emit-runlane-sql
pnpm aws:deploy
pnpm seed
pnpm smoke
```

`pnpm aws:deploy` runs `prisma generate`, `sst deploy`, then `pnpm db:migrate:remote`. The remote migrate runs under `sst shell --target AppEnv`, so `DATABASE_URL` comes from the deployed stack outputs instead of a copied password. `pnpm smoke` exercises Lambda delivery, Fargate delivery, FIFO ordering, release plus `tick()` recovery, cancellation, schedule materialization, and operator reads.

Day-to-day remote commands run through SST's `AppEnv` target so they receive the deployed `DATABASE_URL`, queue URLs, and trigger URL:

```sh
pnpm trigger emails.welcome '{"userId":"usr_demo_alice"}'
pnpm trigger media.transcode '{"mediaId":"med_demo_clip_1"}'
pnpm trigger account.apply-event '{"userId":"usr_demo_alice","eventId":"evt_1","kind":"plan_upgraded"}'

pnpm logs:trigger
pnpm logs:tick
pnpm logs:emails
pnpm logs:ordered
pnpm logs:worker

pnpm sst shell --target AppEnv -- pnpm exec runlane runs list --limit 10
pnpm sst shell --target AppEnv -- pnpm exec runlane runs get <runId> --events
```

## SST configuration shape [#sst-configuration-shape]

```ts
// sst.config.ts (excerpt)
const isProd = $app.stage === 'production'
const vpc = new sst.aws.Vpc('Vpc', {})
const dbPassword = new random.RandomPassword('DbPassword', { length: 32, special: false })

const dbSecurityGroup = new aws.ec2.SecurityGroup('DbSg', {
  vpcId: vpc.id,
  ingress: [{ protocol: 'tcp', fromPort: 5432, toPort: 5432, cidrBlocks: ['0.0.0.0/0'] }],
  egress: [{ protocol: '-1', fromPort: 0, toPort: 0, cidrBlocks: ['0.0.0.0/0'] }],
})

const dbSubnetGroup = new aws.rds.SubnetGroup('DbSubnetGroup', {
  name: `runlane-example-db-${$app.stage}`,
  subnetIds: vpc.publicSubnets,
})

const db = new aws.rds.Instance('Db', {
  identifier: `runlane-example-${$app.stage}`,
  engine: 'postgres',
  engineVersion: '17',
  instanceClass: 'db.t4g.micro',
  allocatedStorage: 20,
  username: 'runlane',
  password: dbPassword.result,
  dbName: 'runlane',
  vpcSecurityGroupIds: [dbSecurityGroup.id],
  dbSubnetGroupName: dbSubnetGroup.name,
  publiclyAccessible: true,
  skipFinalSnapshot: true,
  deletionProtection: false,
  applyImmediately: true,
})

const databaseUrl = $interpolate`postgresql://${db.username}:${dbPassword.result}@${db.address}:${db.port}/${db.dbName}?sslmode=no-verify`

const dlq = new sst.aws.Queue('Dlq')
const dlqFifo = new sst.aws.Queue('DlqFifo', {
  fifo: true,
})

const emailsQueue = new sst.aws.Queue('Emails', {
  visibilityTimeout: '6 minutes',
  dlq: { queue: dlq.arn, retry: 5 },
})
const heavyQueue = new sst.aws.Queue('Heavy', {
  visibilityTimeout: '6 minutes',
  dlq: { queue: dlq.arn, retry: 5 },
})
const orderedQueue = new sst.aws.Queue('Ordered', {
  visibilityTimeout: '6 minutes',
  fifo: { contentBasedDeduplication: true },
  dlq: { queue: dlqFifo.arn, retry: 5 },
})

const sharedEnvironment = {
  DATABASE_URL: databaseUrl,
  RUNLANE_QUEUE_EMAILS_URL: emailsQueue.url,
  RUNLANE_QUEUE_HEAVY_URL: heavyQueue.url,
  RUNLANE_QUEUE_ORDERED_URL: orderedQueue.url,
  RUNLANE_ENVIRONMENT: $app.stage,
}
const sharedLink = [emailsQueue, heavyQueue, orderedQueue]

emailsQueue.subscribe(
  {
    handler: 'src/lambda/delivery-emails.handler',
    link: sharedLink,
    environment: sharedEnvironment,
    timeout: '1 minute',
    memory: '512 MB',
  },
  { batch: { partialResponses: true, size: 10 } },
)

orderedQueue.subscribe(
  {
    handler: 'src/lambda/delivery-ordered.handler',
    link: sharedLink,
    environment: sharedEnvironment,
    timeout: '1 minute',
    memory: '512 MB',
  },
  { batch: { partialResponses: true, size: 10 } },
)

new sst.aws.CronV2('Tick', {
  schedule: 'rate(1 minute)',
  job: {
    handler: 'src/lambda/tick.handler',
    link: sharedLink,
    environment: sharedEnvironment,
    timeout: '2 minutes',
    memory: '512 MB',
  },
})

const trigger = new sst.aws.Function('Trigger', {
  handler: 'src/lambda/trigger.handler',
  url: true,
  link: sharedLink,
  environment: sharedEnvironment,
  timeout: '15 seconds',
  memory: '512 MB',
  ...(isProd ? { concurrency: { provisioned: 1 } } : {}),
})

new sst.x.DevCommand('AppEnv', {
  environment: {
    ...sharedEnvironment,
    RUNLANE_TRIGGER_URL: trigger.url,
  },
  dev: {
    autostart: false,
    command: 'true',
  },
})

const cluster = new sst.aws.Cluster('Cluster', { vpc })
new sst.aws.Service('Worker', {
  cluster,
  link: sharedLink,
  environment: sharedEnvironment,
  image: { context: '../..', dockerfile: 'examples/aws/Dockerfile' },
  cpu: '0.25 vCPU',
  memory: '0.5 GB',
  scaling: { min: 1, max: 1 },
  transform: {
    service: { enableExecuteCommand: true },
  },
})
```

Each compute resource is linked to the SQS queues it may publish to or consume from so SST grants IAM.

The database URL and queue URLs are passed through `environment` so application code reads them from `process.env`. `AppEnv` exposes the same values to post-deploy scripts through `sst shell --target AppEnv`.

The database URL includes `sslmode=no-verify` because the demo encrypts the public RDS connection without bundling the Amazon RDS CA into every runtime. Production should bundle the CA and use full certificate verification.

The Fargate image uses the monorepo root as its build context because the Dockerfile copies workspace packages as well as `examples/aws`.

## Operational notes [#operational-notes]

* **Public RDS.** The database accepts TCP 5432 from anywhere, protected by the generated password printed in deploy outputs. Production should use private subnets, RDS Proxy, and a private access path.
* **No NAT.** Lambdas run outside the VPC. The Fargate service runs in the VPC public subnets and reaches SQS/RDS through public endpoints, so the example does not pay for a NAT gateway or NAT instance.
* **DLQ shape.** Standard source queues can share a standard DLQ. The FIFO `Ordered` queue uses a separate FIFO DLQ because SQS redrive policies require the dead-letter queue type to match the source queue type.
* **Cold starts.** The example sets provisioned concurrency on the Trigger Lambda only for the `production` stage. Non-production stages accept first-invocation latency.
* **Tear down.** `removal: 'remove'` for non-production stages means `sst remove` deletes RDS too. Production stages retain resources by default.

## Choosing between this example and your own deploy [#choosing-between-this-example-and-your-own-deploy]

This example deliberately bundles both the SQS-Lambda demo and the long-running consumer demo to show the trade-off in one place. A real deployment usually picks one acquisition path per logical queue and either:

* Lambda only — if every task fits the runtime and resource ceiling, no Fargate
* Fargate only — if you don't want a Lambda dependency, no SQS-Lambda
* Both — if some tasks need long-running compute and others benefit from Lambda's burst scaling

`tick()` always needs a home. The reference patterns are an EventBridge cron Lambda (used here) or a `setInterval` inside a long-running container.


# Implement With AI (/introduction/implement-with-ai)



Use this page when you want a coding assistant to add Runlane to an existing TypeScript project. The goal is simple: give the agent enough product context, Markdown docs, and implementation constraints that it can move quickly without inventing APIs or taking shortcuts.

Runlane docs expose Markdown for agents:

| Need                  | URL or request                                                                       |
| --------------------- | ------------------------------------------------------------------------------------ |
| Docs map              | [`/llms.txt`](/llms.txt)                                                             |
| Full processed docs   | [`/llms-full.txt`](/llms-full.txt)                                                   |
| One page as Markdown  | Append `.md`, such as [`/introduction/quick-start.md`](/introduction/quick-start.md) |
| Header-based Markdown | Request the normal page with `Accept: text/markdown`                                 |

Example:

```sh
curl -H "Accept: text/markdown" https://www.runlane.sh/introduction/quick-start
```

## Copy-Paste Prompt [#copy-paste-prompt]

Paste this into your coding assistant after giving it access to your project:

```text
Implement Runlane in this project.

Before changing code:
1. Read Runlane docs as Markdown, not rendered HTML.
2. Fetch https://www.runlane.sh/llms.txt for the docs map.
3. Fetch the required docs below with ".md" appended, or request the normal URL with "Accept: text/markdown".
4. Do not assume Runlane APIs from memory. Treat the docs as the source of truth.
5. Inspect this project before proposing changes:
   - package manager and dependency conventions
   - existing async, background job, queue, cron, or worker code
   - database and runtime configuration
   - deployment and worker model
   - environment variable conventions
   - validation and schema style
   - logging and error handling style
   - test structure and verification commands

Required Runlane docs:
- https://www.runlane.sh/introduction.md
- https://www.runlane.sh/introduction/quick-start.md
- https://www.runlane.sh/concepts.md
- https://www.runlane.sh/concepts/job-shapes.md
- https://www.runlane.sh/concepts/tasks-and-runs.md
- https://www.runlane.sh/concepts/queues.md
- https://www.runlane.sh/concepts/workers.md
- https://www.runlane.sh/concepts/retry-vs-release.md
- https://www.runlane.sh/concepts/lanes-storage-and-transport.md
- https://www.runlane.sh/concepts/local-development.md
- https://www.runlane.sh/configuration.md

Pick the integration path from the docs:
- Use @runlane/lane-local for local development, tests, demos, and first integration.
- Use a production lane only when this project already has the required infrastructure or the task explicitly includes adding it.
- If production uses Postgres and SQS, read:
  - https://www.runlane.sh/contracts/postgres-sqs-lane.md
  - https://www.runlane.sh/contracts/postgres-storage.md
  - https://www.runlane.sh/contracts/sqs-transport.md
  - https://www.runlane.sh/configuration/cli.md
- If the app needs schedules, read https://www.runlane.sh/concepts/schedules.md.
- If the app needs operator tooling, read https://www.runlane.sh/concepts/operator-apis.md.
- If the app needs custom storage or transport, read:
  - https://www.runlane.sh/contracts.md
  - https://www.runlane.sh/contracts/adapter-authoring.md
  - https://www.runlane.sh/contracts/testing.md

Implementation rules:
- Keep Runlane runtime and lane configuration centralized.
- Define tasks with payload schemas at the boundary where work enters Runlane.
- Keep task handlers close to the domain code they execute.
- Wire the producer path that calls trigger(), runNow(), or schedules based on the docs and the project need.
- Wire the execution path: local executeNext() or worker() for development, executeDelivery(message) for delivered wakeups, or the documented production worker shape.
- Wire tick() only where the docs require maintenance for schedules, releases, retries, expired leases, cancellation cleanup, or outbox recovery.
- Do not hand-roll storage, transport, retry, release, schedule, cancellation, or lease behavior.
- Do not duplicate adapter-owned option schemas or hide contract failures behind wrappers.
- Do not add fake integrations, no-op workers, placeholder task handlers, or untested production config.

Verification:
- Add focused tests for the project behavior being moved into Runlane.
- Prove at least one end-to-end path: trigger a run, execute it through the selected execution path, and assert the observable result.
- If schedules, retries, releases, cancellation, or operator APIs are used, test those behaviors explicitly.
- Run the project's normal typecheck, lint, and test commands.
- Report the Runlane docs you used, files changed, verification commands, and any production infrastructure still required.
```

## Required Reading Path [#required-reading-path]

If the agent has limited context, start with these pages in order:

| Step | Page                                                                   | Why it matters                                                                       |
| ---- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| 1    | [Runlane](/introduction)                                               | Product model and the smallest durable primitives.                                   |
| 2    | [Quick Start](/introduction/quick-start)                               | First runnable local integration.                                                    |
| 3    | [Primitives](/concepts)                                                | The nav-order mental model for tasks, runs, workers, and lanes.                      |
| 4    | [Job Shapes](/concepts/job-shapes)                                     | Whether the project needs background jobs, schedules, polling, or inline execution.  |
| 5    | [Tasks And Runs](/concepts/tasks-and-runs)                             | Task schemas, ids, idempotency, singleton keys, concurrency keys, and run lifecycle. |
| 6    | [Queues](/concepts/queues)                                             | Provider-neutral routing and capacity policy.                                        |
| 7    | [Workers](/concepts/workers)                                           | Polling workers, drain workers, delivered wakeups, leases, and heartbeats.           |
| 8    | [Retry Vs Release](/concepts/retry-vs-release)                         | The difference between failure pressure and business waiting.                        |
| 9    | [Lanes, Storage, And Transport](/concepts/lanes-storage-and-transport) | The boundary that prevents agents from reimplementing adapter behavior.              |
| 10   | [Configuration](/configuration)                                        | How runtime, lane, adapter, CLI, and deployment config fit together.                 |

## Conditional Pages [#conditional-pages]

Add these pages when the project needs the corresponding capability:

| Need                        | Read                                                                                                                                                                               |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Local-only dev or tests     | [Local Development](/concepts/local-development), [Local Lane Example](/examples/local-lane)                                                                                       |
| Scheduled jobs              | [Schedules](/concepts/schedules)                                                                                                                                                   |
| Inline durable execution    | [Current-Process Execution](/concepts/current-process-execution)                                                                                                                   |
| Operator tooling            | [Operator APIs](/concepts/operator-apis), [Cancellation](/concepts/cancellation), [Rerun And Manual Retry](/concepts/rerun-and-manual-retry), [Pruning](/concepts/pruning)         |
| Production Postgres and SQS | [Postgres + SQS Lane](/contracts/postgres-sqs-lane), [Postgres Storage](/contracts/postgres-storage), [SQS Transport](/contracts/sqs-transport), [Runlane CLI](/configuration/cli) |
| Custom adapters             | [Contracts](/contracts), [Adapter Authoring](/contracts/adapter-authoring), [Testing](/contracts/testing), [Vocabulary](/contracts/vocabulary)                                     |

## What The Agent Should Avoid [#what-the-agent-should-avoid]

The common failure mode is not moving too slowly. It is moving before understanding the boundary.

* Do not treat SQS, a cron job, or an in-memory queue as the durable source of truth.
* Do not skip payload validation because the caller "already has types."
* Do not invent wrapper APIs around Runlane until the project has a real repeated pattern.
* Do not put `tick()` inside a delivery handler just to make a demo pass.
* Do not use the local lane as a production durability boundary.
* Do not call a transport wakeup a job. Storage owns runs; transport wakes workers.

The first good implementation is usually small: one local lane, one task, one trigger path, one execution path, and tests that prove the behavior. Expand to production lanes, schedules, and operator APIs only when the project needs those capabilities.


# Runlane (/introduction)



Runlane helps a TypeScript app run async work durably without hiding the work in a separate platform. You define tasks in code, trigger runs from your app, and let storage preserve the state that workers need after deploys, crashes, retries, and operator actions.

> **Durable tasks, schedules, queues, and wakeups that stay on your infrastructure.**
>
> **Contracts define the durable shapes, core owns behavior, and storage and transport adapters stay replaceable.**

Runlane is for work that must outlive the request that created it: welcome emails, account syncs, provider polling, media processing, scheduled maintenance, and operator-controlled recovery.

The model is intentionally small. A task is user code plus a payload schema. A run is the durable record of one execution. Storage owns truth. Transport only wakes workers.

## What You Build With It [#what-you-build-with-it]

| Need                                             | Runlane primitive                                                        | First page                                                                                |
| ------------------------------------------------ | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- |
| Run work outside a request path                  | `task()` + `trigger()`                                                   | [Tasks And Runs](/concepts/tasks-and-runs)                                                |
| Start one durable attempt in the current process | `runNow()`                                                               | [Current-Process Execution](/concepts/current-process-execution)                          |
| Keep one resource from overlapping               | `singletonKey`                                                           | [Identifiers](/contracts/vocabulary/identifiers)                                          |
| Collapse producer retries                        | `idempotencyKey`                                                         | [Tasks And Runs](/concepts/tasks-and-runs#task-ids-idempotency-singleton-and-concurrency) |
| Wait for external state without failure noise    | `context.release()`                                                      | [Retry Vs Release](/concepts/retry-vs-release)                                            |
| Run work on a cadence                            | task-colocated `schedule`                                                | [Schedules](/concepts/schedules)                                                          |
| Execute locally                                  | `createLocalLane()` + `executeNext()`                                    | [Quick Start](/introduction/quick-start)                                                  |
| Execute from SQS                                 | `executeDelivery(message)`                                               | [SQS Transport](/contracts/sqs-transport)                                                 |
| Inspect and recover production work              | `runlane.runs.*` and `runlane runs`, `retry`, `rerun`, `cancel`, `prune` | [Operator APIs](/concepts/operator-apis)                                                  |

## 10-Minute Tutorial [#10-minute-tutorial]

Start with the local lane. It keeps Runlane state in memory for the current process, which makes it useful for development, tests, and examples.

```sh
pnpm add @runlane/core @runlane/lane-local zod
```

Define one queue and one task:

```ts
import { queue, task } from '@runlane/core'
import * as z from 'zod'

const emailQueue = queue({ name: 'emails', default: true })

const sendWelcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema: z.object({ userId: z.string().min(1) }),
  idempotencyKey: (payload) => `emails.welcome.${payload.userId}`,
  async run(payload, context) {
    // Replace this with your email provider call.
    await emailProvider.sendWelcome(payload.userId, { signal: context.signal })
  },
})
```

Create a local runtime:

```ts
import { createRunlane } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'

const runlane = createRunlane({
  lane: createLocalLane(),
  queues: [emailQueue],
  tasks: { sendWelcomeEmail },
})
```

Trigger durable work. `trigger()` validates the payload, creates the run, records a delivery request, and by default tries to publish that wakeup immediately:

```ts
const { run } = await runlane.trigger(runlane.tasks.sendWelcomeEmail, {
  userId: 'user_123',
})
```

Execute one due run locally:

```ts
const completed = await runlane.executeNext()
```

Use a drain worker when a script or test should process currently due work and exit:

```ts
import { WorkerMode } from '@runlane/core'

const worker = runlane.worker({ mode: WorkerMode.Drain })
await worker.closed
```

Use `runNow()` when the caller should create a durable run and execute its first attempt inline, without waiting for a worker or transport wakeup:

```ts
const completedInline = await runlane.runNow(runlane.tasks.sendWelcomeEmail, {
  userId: 'user_123',
})
```

Add business waiting without recording a failure. A released run becomes due again later; maintenance records a fresh delivery request when it is time to continue:

```ts
const pollReport = task({
  id: 'reports.poll',
  schema: z.object({ reportId: z.string().min(1) }),
  async run(payload, context) {
    const report = await reports.get(payload.reportId)

    if (report.status === 'processing') {
      return context.release('30s', { reason: 'provider_not_ready' })
    }

    await saveReport(report)
  },
})
```

Run maintenance from a cron, scheduled function, container sidecar, or local development loop:

```ts
await runlane.tick()
```

One maintenance pass can:

* materialize due schedules
* wake released or retried runs
* recover expired leases
* finalize abandoned cancellations
* flush outbox rows

For production SQS delivery, use the Postgres/SQS lane and an SQS consumer. The SQS message carries only a wakeup; the handler calls `executeDelivery(message)` so core can re-read storage, claim a lease, and ignore stale or duplicate wakeups safely.

```ts
import { createRunlane } from '@runlane/core'
import { postgresSqsLane } from '@runlane/lane-postgres-sqs'

const runlane = createRunlane({
  lane: postgresSqsLane({ postgres, sqs }),
  queues: [emailQueue],
  tasks: { sendWelcomeEmail },
})

await runlane.executeDelivery(message)
```

Keep `tick()` separate from SQS record handling. Delivery consumers should handle the wakeups they received; maintenance handles schedules, retries, releases, lease recovery, cancellation cleanup, and outbox recovery.

## Where to start [#where-to-start]

* **[Concepts](/concepts)** — learn the durable primitives.
* **[Quick Start](/introduction/quick-start)** — run one local task end to end.
* **[Implement With AI](/introduction/implement-with-ai)** — hand a coding assistant the right Markdown docs and implementation prompt.
* **[Reference](/contracts)** — read the TypeScript contracts.
* **[Configuration](/configuration)** — understand lane, storage, and transport capabilities.


# Quick Start (/introduction/quick-start)



This page builds the smallest durable Runlane program:

1. Install core, the local lane, and a payload schema library.
2. Define one queue and one task.
3. Create a local runtime.
4. Trigger one run.
5. Execute that run and inspect the stored result.

## Install [#install]

```sh
pnpm add @runlane/core @runlane/lane-local zod
```

## Smallest Program [#smallest-program]

Put this in a TypeScript file that runs inside your app or test process:

```ts
import { createRunlane, queue, task } from '@runlane/core'
import { createLocalLane } from '@runlane/lane-local'
import * as z from 'zod'

const sentWelcomeEmails: string[] = []
const emailQueue = queue({ name: 'emails', default: true })

const sendWelcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema: z.object({
    userId: z.string(),
  }),
  run(payload) {
    sentWelcomeEmails.push(payload.userId)
  },
})

const runlane = createRunlane({
  lane: createLocalLane(),
  queues: [emailQueue],
  tasks: { sendWelcomeEmail },
})

const { run: queuedRun } = await runlane.trigger(runlane.tasks.sendWelcomeEmail, {
  userId: 'user_123',
})

const completedRun = await runlane.executeNext()

console.log(queuedRun.status)
console.log(completedRun?.status)
console.log(JSON.stringify(sentWelcomeEmails))
```

Expected output:

```text
queued
succeeded
["user_123"]
```

That is the first successful path: `trigger()` creates durable work, and `executeNext()` claims one due run and calls the task handler.

## What Happened [#what-happened]

`queue()` creates a provider-neutral queue definition. The queue is registered on the runtime through `queues: [emailQueue]`. Marking it `default: true` lets tasks use it when no other queue is selected.

`task()` defines user code plus the payload validator for that code. Runlane accepts Standard Schema-compatible validators, so Zod works directly. The payload is validated before the run is created and again before the handler runs.

`createRunlane()` creates a runtime over a lane, a queue registry, and a task catalog. Passing `tasks: { sendWelcomeEmail }` makes that catalog authoritative and exposes the same handles on `runlane.tasks`.

`createLocalLane()` is the ready-to-use in-memory lane from `@runlane/lane-local`. It stores runs, events, leases, and schedules in the current process. Use it for development, examples, and tests. It is not a production durability boundary, and process exit loses local state.

## Trigger And Execute [#trigger-and-execute]

`trigger()` validates the payload, writes a run, writes the first delivery request, and returns `{ run, outcome }`. With transport-delivery lanes and the default dispatch policy, it also tries to publish the new wakeup immediately.

The returned `run` is the queued run record. `outcome` is `TriggerOutcomeType.Created` for new work and `TriggerOutcomeType.ReturnedExisting` when idempotency returns an existing owner for the same key.

`executeNext()` is the deterministic single-run execution primitive. It scans for one due run, claims a lease, records `run.started`, runs the handler, and persists the attempt result. It returns the executed run, or `undefined` when no due run is available.

Use a worker when a process should keep polling or drain a backlog:

```ts
import { WorkerMode } from '@runlane/core'

const worker = runlane.worker({ mode: WorkerMode.Drain })
await worker.closed
```

Drain mode processes currently due work and exits. Omitting `mode` starts a polling worker that keeps scanning until stopped.

## Inspect The Result [#inspect-the-result]

Use the runtime operator API to inspect the stored run:

```ts
const storedRun = await runlane.runs.get(queuedRun.id)

console.log(storedRun?.status)
```

For the program above, the stored run is `succeeded` after `executeNext()` finishes.

The important invariant is that execution state is not just an in-memory callback result. Even with the local lane, Runlane writes an append-only event history: created, delivery requested, lease claimed, started, then succeeded or another terminal or waiting outcome.

## Next Steps [#next-steps]

* Use [Workers](/concepts/workers) when a process should keep executing queued work.
* Use [Local Development](/concepts/local-development) for local lane behavior, limits, and testing patterns.
* Use [Tasks And Runs](/concepts/tasks-and-runs) for task options, idempotency, singleton keys, and trigger options.
* Use [Schedules](/concepts/schedules) when `tick()` should materialize scheduled runs.
* Use [Implement With AI](/introduction/implement-with-ai) when you want a coding assistant to add Runlane to an existing project from Markdown docs.

You do not need `tick()` for the immediate trigger above. Call `tick()` from maintenance infrastructure when you use schedules, released or retried runs, expired leases, cancellation finalization, or deferred outbox publishing.

Production lane packages follow the same runtime shape. They compose storage and transport adapters into a `Lane`, then pass that lane into `createRunlane()`.


# Delivery And Workers (/contracts/vocabulary/delivery-and-workers)



Delivery vocabulary spans lane delivery modes, contract-owned outbox state, transport publish outcomes, and public `@runlane/core` worker/runtime results. Use these values when building lanes, transports, worker orchestration, or operator views.

Worker, `executeNext()`, and `executeDelivery()` execution options all accept `leaseDuration`, `heartbeatInterval`, `maxAttemptDuration`, `signal`, and `workerId`. `maxAttemptDuration` is an attempt deadline: storage persists `attemptDeadlineAt` at lease claim, heartbeats do not extend it, and timeout finalization only applies after the attempt records `run.started`. Timeout failures use `ErrorCode.TaskTimedOut`.

## LaneDeliveryMode [#lanedeliverymode]

| Enum                              | Value             | Meaning                                                                                                 |
| --------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------- |
| `LaneDeliveryMode.StoragePolling` | `storage_polling` | The lane has no external transport; workers discover due work by polling and leasing runs from storage. |
| `LaneDeliveryMode.Transport`      | `transport`       | The lane persists storage outbox rows and uses a transport adapter to publish external wakeups.         |

Storage remains durable truth in both modes. Transport delivery adds a wakeup publisher; it does not make the transport own run state.

## OutboxMessageStatus [#outboxmessagestatus]

| Enum                               | Value           | Meaning                                                             |
| ---------------------------------- | --------------- | ------------------------------------------------------------------- |
| `OutboxMessageStatus.Claimed`      | `claimed`       | A publisher owns the outbox row for a publish attempt.              |
| `OutboxMessageStatus.DeadLettered` | `dead_lettered` | Publishing is terminally abandoned for this outbox row.             |
| `OutboxMessageStatus.Failed`       | `failed`        | Last publish attempt failed and the row may become available again. |
| `OutboxMessageStatus.Pending`      | `pending`       | Row is waiting to be claimed for publish.                           |
| `OutboxMessageStatus.Published`    | `published`     | Transport accepted the wakeup.                                      |

Outbox failure records use `ErrorCode` for `code`; provider-specific detail belongs in `meta`.

| Path                                           | Outbox behavior                                                                                                                                                                             |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Default `trigger()`                            | Persists the run and, when creation produced outbox rows, eagerly attempts to publish those new rows. Bounded queues produce outbox rows only after maintenance reserves dispatch capacity. |
| `tick()`                                       | Claims due pending or failed rows for maintenance flushes.                                                                                                                                  |
| Retryable transport publish failure            | Leaves the run queued and marks the outbox row failed for later recovery.                                                                                                                   |
| Storage failure or non-transport publish error | Rejects the trigger or maintenance call.                                                                                                                                                    |
| `TriggerDispatchMode.Deferred`                 | Persists any trigger-created outbox rows and leaves publishing to `tick()`.                                                                                                                 |

## dispatch.onTrigger [#dispatchontrigger]

The default is `contractDefaults.dispatch.onTrigger`.

| Enum                           | Value      | Meaning                                                                                                                                    | Default |
| ------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `TriggerDispatchMode.Eager`    | `eager`    | `trigger()` persists the run, then immediately attempts to publish any outbox rows created by that trigger.                                | Yes     |
| `TriggerDispatchMode.Deferred` | `deferred` | `trigger()` persists the run and any outbox rows created by that trigger, but publishing waits for `tick()` or another maintenance runner. | No      |

## WakeupPublishOutcomeType [#wakeuppublishoutcometype]

| Enum                                 | Value       | Meaning                                                                                                                   |
| ------------------------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------- |
| `WakeupPublishOutcomeType.Failed`    | `failed`    | Provider rejected the indexed wakeup publish attempt; the outcome carries an `OutboxFailureRecord`.                       |
| `WakeupPublishOutcomeType.Published` | `published` | Provider accepted the indexed wakeup publish attempt; the outcome carries `publishedAt` and optional provider message id. |

`publishWakeups()` returns outcomes by index, not by echoed ids: `outcomes[index]` is the result for `command.attempts[index]`. A returned result must include exactly one outcome for every attempted wakeup. Missing, extra, or malformed outcomes are adapter contract violations; operation-level transport outages should reject with `TransportUnavailable` or `TransportPublishFailed` instead.

## WorkerMode [#workermode]

| Enum               | Value   | Meaning                                                                   |
| ------------------ | ------- | ------------------------------------------------------------------------- |
| `WorkerMode.Drain` | `drain` | Execute currently due work until storage is idle or `maxRuns` is reached. |
| `WorkerMode.Poll`  | `poll`  | Keep polling storage and sleeping on empty polls until stopped.           |

## ExecuteDeliveryStatus [#executedeliverystatus]

| Enum                             | Value      | Meaning                                                                                     |
| -------------------------------- | ---------- | ------------------------------------------------------------------------------------------- |
| `ExecuteDeliveryStatus.Executed` | `executed` | The delivered wakeup led to a persisted run outcome. Transport adapters can acknowledge it. |
| `ExecuteDeliveryStatus.Ignored`  | `ignored`  | The delivered wakeup was stale or not executable, but safe to acknowledge.                  |

## ExecuteDeliveryIgnoredReason [#executedeliveryignoredreason]

| Enum                                         | Value            | Meaning                                                                                                                                                       |
| -------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ExecuteDeliveryIgnoredReason.Abandoned`     | `abandoned`      | This worker started an attempt but lost lease ownership before it could persist the outcome.                                                                  |
| `ExecuteDeliveryIgnoredReason.AlreadyLeased` | `already_leased` | Another worker currently owns the run lease.                                                                                                                  |
| `ExecuteDeliveryIgnoredReason.ClaimLost`     | `claim_lost`     | Storage had due work, but the atomic lease claim did not succeed because another caller changed ownership, sequence, dispatch reservation, or capacity first. |
| `ExecuteDeliveryIgnoredReason.NotDue`        | `not_due`        | The run exists but is scheduled for the future.                                                                                                               |
| `ExecuteDeliveryIgnoredReason.RunNotFound`   | `run_not_found`  | The wakeup points at a run that no longer exists or was never persisted.                                                                                      |
| `ExecuteDeliveryIgnoredReason.Terminal`      | `terminal`       | The run is already terminal.                                                                                                                                  |
| `ExecuteDeliveryIgnoredReason.WrongQueue`    | `wrong_queue`    | The delivered queue no longer matches the run's current queue.                                                                                                |

Ignored delivery results are ack-safe. Framework, storage, projection, cancellation-before-claim, and persistence failures reject instead so provider integrations can retry or report batch item failure. Missing task or queue registration is persisted as a non-retryable run failure and returns `ExecuteDeliveryStatus.Executed`.


# Error Codes (/contracts/vocabulary/error-codes)



`ErrorCode` is the stable failure vocabulary across `RunlaneError`, `RunFailure`, and `OutboxFailureRecord`. Public code should branch on `code`, not parse `message`.

Raw driver, provider, or business-domain failures can be attached as `cause` for server-side logs, or summarized in `meta` for operator views. They are not the public code namespace.

`retryable` is part of each error or persisted failure record; it is not derived from `ErrorCode` alone. `RunlaneError` defaults to non-retryable, the storage-conflict helper defaults to retryable, and adapters or runtime code set retryability from the specific boundary failure.

## ErrorCode [#errorcode]

| Code                       | Value                        | Typical trigger                                                                   | First response                                                                          |
| -------------------------- | ---------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| `AdapterContractViolation` | `adapter_contract_violation` | Adapter or transport contract mismatch                                            | Fix the adapter/caller boundary.                                                        |
| `CapabilityUnsupported`    | `capability_unsupported`     | Lane lacks a requested guarantee                                                  | Use a capable lane or disable the feature.                                              |
| `ConfigurationInvalid`     | `configuration_invalid`      | Malformed runtime, task, CLI, adapter, or lane options                            | Fix input/config; retrying unchanged will not help.                                     |
| `InternalError`            | `internal_error`             | Unexpected uncategorized framework failure                                        | Inspect `cause`; narrow the code in implementation when understood.                     |
| `InvariantViolation`       | `invariant_violation`        | Durable state or projection is impossible                                         | Treat as bug/corruption; inspect run history and storage.                               |
| `OperationCancelled`       | `operation_cancelled`        | Cooperative abort before completion                                               | Let provider/runtime retry only when the boundary requires it.                          |
| `ObservationExportFailed`  | `observation_export_failed`  | Durable observation sink or telemetry export boundary failed                      | Retry from the previous checkpoint when retryable; inspect sink metadata.               |
| `RunNotFound`              | `run_not_found`              | Command references a missing run                                                  | Check environment/run id or use explicit new work APIs.                                 |
| `ScheduleNotFound`         | `schedule_not_found`         | Command references a missing schedule-owned record                                | Check environment/schedule identity.                                                    |
| `StorageConflict`          | `storage_conflict`           | Stale sequence, owner, lease, occurrence, or outbox claim                         | Use `storageConflictKind` to retry, ignore, or return the winner.                       |
| `StorageUnavailable`       | `storage_unavailable`        | Storage backend outage or unknown commit state                                    | Retry only when safe; log backend `cause`.                                              |
| `TaskFailed`               | `task_failed`                | User task threw an unstructured error                                             | Persist generic failure; put safe domain detail in `meta`.                              |
| `TaskNotFound`             | `task_not_found`             | Runtime lacks the run's registered task                                           | Fix task catalog/deployment routing.                                                    |
| `TaskOutputInvalid`        | `task_output_invalid`        | Handler returned non-JSON output                                                  | Return JSON-compatible output or `undefined` for void success.                          |
| `TaskTimedOut`             | `task_timed_out`             | Attempt exceeded its fixed max-attempt deadline                                   | Pass `context.signal` through task I/O, shorten work, or raise the configured deadline. |
| `TransportPublishFailed`   | `transport_publish_failed`   | Provider rejected a wakeup publish                                                | Inspect provider metadata; retry only if retryable.                                     |
| `TransportUnavailable`     | `transport_unavailable`      | Transport backend cannot complete operation                                       | Let outbox/provider retry after infrastructure recovers.                                |
| `ValidationFailed`         | `validation_failed`          | Payload, persisted data, cursor, filter, or domain precondition failed validation | Fix the invalid value or schema boundary.                                               |

### `ErrorCode.AdapterContractViolation` (`adapter_contract_violation`) [#errorcodeadaptercontractviolation-adapter_contract_violation]

Triggered when a Runlane package receives an internally inconsistent command or an adapter returns data that violates its contract.

Common causes:

* storage returns a run, event, outbox, or schedule record whose shape does not match the contract
* transport returns publish outcomes that are missing, extra, malformed, or no longer aligned by index
* an SQS helper receives a provider record shape or wakeup body that cannot be treated as a Runlane delivery message
* core detects that an adapter accepted a command it should have rejected

Treat this as an implementation bug or integration bug. Fix the adapter, transport bridge, test fixture, or caller boundary. Do not mask it with default values.

### `ErrorCode.CapabilityUnsupported` (`capability_unsupported`) [#errorcodecapabilityunsupported-capability_unsupported]

Triggered when public runtime code asks the lane for a guarantee the lane does not report.

Common causes:

* using singleton keys with storage that does not report `enforcesSingleton`
* using bounded queue capacity with storage that does not report `enforcesQueueConcurrency`
* calling `context.step.run()` with storage that does not report `durableSteps`
* running a durable observation exporter against storage that does not report `exportsObservations`
* calling operator reads or pruning against a lane that does not expose those capabilities
* materializing schedules against storage that cannot claim schedule occurrences

Select a lane with the required capability, change the feature configuration, or fail the application action before calling Runlane.

### `ErrorCode.ConfigurationInvalid` (`configuration_invalid`) [#errorcodeconfigurationinvalid-configuration_invalid]

Triggered when application-supplied configuration or public options are malformed.

Common causes:

* invalid task ids, queue names, worker ids, schedules, retry policy, release values, duration strings, or runtime options
* queue registry drift, missing default queues, duplicate queues, or queue definitions that do not match the runtime registry
* invalid CLI config modules, command flags, dates, cursors, or runtime loading options
* invalid Postgres connection/schema options, SQS queue bindings, FIFO options, or lane package options
* malformed delivery messages at the runtime boundary before a run is claimed

Fix the input or deployment configuration. Retrying the same command without changing input should produce the same error.

### `ErrorCode.InternalError` (`internal_error`) [#errorcodeinternalerror-internal_error]

Triggered when Runlane cannot classify an unexpected framework, adapter, or CLI failure with a narrower code.

Common causes:

* an adapter maps an unknown backend error after it fails to match known storage or transport categories
* persisted backend rows fail an adapter-owned parse in a way that indicates a bug or incompatible migration
* CLI command execution fails with an untyped thrown value

Treat this as a bug or corrupted environment until proven otherwise. Preserve the `cause` in server logs and reduce it to a narrower code when the failure mode becomes understood.

### `ErrorCode.InvariantViolation` (`invariant_violation`) [#errorcodeinvariantviolation-invariant_violation]

Triggered when durable state or reducer state violates a framework invariant.

Common causes:

* impossible run event sequences or lifecycle transitions
* materialized run projections that disagree with replayed events
* lease, cancellation, dispatch, schedule, or outbox state that cannot be reconciled
* adapter-owned state that violates its own contract after being read back

This is not normal user input failure. Stop treating the affected run as healthy, inspect storage history, and fix the adapter/runtime bug or data corruption.

### `ErrorCode.OperationCancelled` (`operation_cancelled`) [#errorcodeoperationcancelled-operation_cancelled]

Triggered when work is intentionally aborted before Runlane can finish the operation.

Common causes:

* `executeDelivery(message, { signal })` receives an already-aborted signal before the run is claimed
* an observation export sink is aborted during shutdown
* task code observes cancellation and throws a structured Runlane cancellation error
* a worker or process shutdown path aborts cooperative task execution

This is not a provider outage. For delivered transport messages, the operation rejects before acknowledgement so the provider can retry or redrive according to its policy. For task attempts, core maps the persisted task-facing message to `Task was cancelled.`

### `ErrorCode.ObservationExportFailed` (`observation_export_failed`) [#errorcodeobservationexportfailed-observation_export_failed]

Triggered when an out-of-band durable observation export sink rejects a batch or the telemetry boundary cannot accept it.

Common causes:

* an OTLP/HTTP Collector endpoint returns a retryable status such as `429`, `502`, `503`, or `504`
* the exporter cannot reach the configured Collector endpoint before its sink timeout
* a custom sink rejects the batch after partial or unknown external delivery

The exporter checkpoints only after the sink accepts the batch, so retrying resumes from the previous checkpoint and may redeliver records. Sinks that need external exactly-once behavior must dedupe by observation record id.

### `ErrorCode.RunNotFound` (`run_not_found`) [#errorcoderunnotfound-run_not_found]

Triggered when a command requires a specific run but storage cannot find it.

Common causes:

* an operator read, CLI command, cancellation, rerun, retry, or lease-owned operation references a run id outside the runtime environment
* the run was pruned or never existed
* a test or adapter command used the wrong environment/run id pair

Check the runtime environment and run id. Do not create a replacement run silently; use `rerun()` or `trigger()` explicitly when new work is desired.

### `ErrorCode.ScheduleNotFound` (`schedule_not_found`) [#errorcodeschedulenotfound-schedule_not_found]

Defined for commands that require a schedule occurrence or schedule-owned record that does not exist.

Common causes:

* a command references a missing schedule occurrence
* schedule storage state was pruned, migrated incorrectly, or queried under the wrong environment

Current first-party runtime paths usually report schedule materialization races through `StorageConflict` or bad schedule definitions through `ConfigurationInvalid`. If this code appears, treat it like a missing durable schedule record and inspect environment/schedule identity.

### `ErrorCode.StorageConflict` (`storage_conflict`) [#errorcodestorageconflict-storage_conflict]

Triggered when storage rejects a stale owner, stale sequence, or competing claim.

Common causes:

* another caller appended to the run before this command's `expectedSequence`
* another trigger owns the idempotency key or singleton key
* another worker owns or won the run lease
* another scheduler owns the schedule occurrence
* another publisher owns the outbox claim
* another observation exporter advanced the same consumer checkpoint first

This code is often expected under concurrency. Inspect `meta.storageConflictKind` with `getStorageConflictKind(error)` to decide whether to retry, read the winning owner, ignore a lost race, or surface a real conflict to the caller.

### `ErrorCode.StorageUnavailable` (`storage_unavailable`) [#errorcodestorageunavailable-storage_unavailable]

Triggered when the storage backend is reachable through the adapter contract but cannot complete the operation.

Common causes:

* Postgres connection failure, startup refusal, timeout, or transient backend outage
* database failover or pool exhaustion
* storage SDK or driver failure before Runlane can know whether a write committed

Treat this as retryable only when the error reports `retryable: true` or the surrounding operation is idempotent. Keep the raw driver failure in server logs through `cause`.

### `ErrorCode.TaskFailed` (`task_failed`) [#errorcodetaskfailed-task_failed]

Triggered when user task code fails without throwing a structured `RunlaneError`.

Common causes:

* the task handler throws an ordinary `Error`
* the handler rejects with an unstructured value
* application code catches a provider failure and rethrows a generic error

Core persists a generic task-facing message, `Task failed.`, so raw provider or secret-bearing text does not leak into run history. Put safe business/provider detail in failure `meta` when the task needs operator-visible context.

### `ErrorCode.TaskNotFound` (`task_not_found`) [#errorcodetasknotfound-task_not_found]

Triggered when a runtime tries to execute, rerun, retry, or inspect work for a task that is not registered in that runtime.

Common causes:

* a worker deployment is missing a task from `createRunlane({ tasks })`
* a queue routes runs to a runtime that does not own that task catalog
* an operator rerun/manual retry references a source run whose task is no longer registered

Fix task registration or deployment routing. For delivered wakeups, core records a non-retryable failed run outcome so a poison provider message does not redeliver forever.

### `ErrorCode.TaskOutputInvalid` (`task_output_invalid`) [#errorcodetaskoutputinvalid-task_output_invalid]

Triggered when a task handler or durable step callback completes successfully from JavaScript's point of view but returns a value that cannot be persisted as Runlane JSON output.

Common causes:

* returning a `Date`, class instance, function, `Map`, `Set`, symbol, bigint, cyclic object, or other non-JSON value
* returning a provider SDK response object directly instead of selecting a small public JSON shape
* returning durable step output that does not match the step output schema
* expecting a plain object with a `delay` field to release the run instead of returning `context.release(...)`

Core records this as a task attempt failure so retry policy can apply when configured. Fix the handler output shape; do not rely on transport, storage, or operator views to serialize arbitrary JavaScript objects.

### `ErrorCode.TaskTimedOut` (`task_timed_out`) [#errorcodetasktimedout-task_timed_out]

Triggered when a running attempt exceeds its fixed `attemptDeadlineAt`.

Common causes:

* task-level or execution-path `maxAttemptDuration` is shorter than the provider operation
* task code does not pass `context.signal` to cancellable SDK calls
* user code keeps running after the deadline and only returns later
* the owning process died before recording an attempt outcome, and maintenance finalized the timeout

Core records this as a task attempt failure so retry policy can apply when configured. The deadline bounds one attempt, not the whole run, and heartbeats do not extend it.

### `ErrorCode.TransportPublishFailed` (`transport_publish_failed`) [#errorcodetransportpublishfailed-transport_publish_failed]

Triggered when a transport provider rejects a wakeup publish attempt but the transport backend is still reachable enough to report the failure.

Common causes:

* AWS SQS returns failed `SendMessageBatch` entries
* provider validation rejects message size, FIFO dedupe/group options, queue permissions, or queue state
* a transport response is operation-level failed but not a connectivity outage

Provider request ids, response codes, sender-fault flags, and queue detail belong in `meta`. Retry only when the error or outbox failure is retryable; fix configuration/IAM/FIFO/provider policy for permanent provider rejections.

### `ErrorCode.TransportUnavailable` (`transport_unavailable`) [#errorcodetransportunavailable-transport_unavailable]

Triggered when the transport backend cannot complete the operation or cannot produce trustworthy per-message publish outcomes.

Common causes:

* SQS SDK/network failure before publish outcomes exist
* long-running consumer receive/delete calls fail at the provider boundary
* local/test transport deliberately simulates transport outage

Treat this as infrastructure failure. Triggered runs remain durable; outbox recovery through `tick()` can retry publish after the configured retry delay.

### `ErrorCode.ValidationFailed` (`validation_failed`) [#errorcodevalidationfailed-validation_failed]

Triggered when public data reaches the right boundary but fails validation.

Common causes:

* trigger payload validation fails before a run is created
* persisted payload revalidation fails before handler execution after schema drift
* storage query filters, pagination cursors, prune cursors, or date bounds are malformed at the adapter boundary
* operator commands fail domain preconditions, such as rerunning a non-terminal source run or manually retrying a non-failed source run
* adapter command or record data fails a public query/result schema that represents caller input rather than adapter internals

Fix the payload, cursor, filter, or schema boundary. Do not turn this into a generic task failure unless the validation failure happened inside user-owned task code.

## StorageConflictKind [#storageconflictkind]

Storage adapters must create stale-owner conflicts with `createStorageConflictError({ kind: StorageConflictKind.* })`. That helper stores the stable kind in `meta.storageConflictKind` and defaults the error to retryable. Do not construct bare `RunlaneError` values with `ErrorCode.StorageConflict` for these cases.

| Enum                                     | Value                 | Use                                                                             |
| ---------------------------------------- | --------------------- | ------------------------------------------------------------------------------- |
| `StorageConflictKind.EventSequence`      | `event_sequence`      | The caller's expected run event sequence no longer owns the projection.         |
| `StorageConflictKind.IdempotencyKey`     | `idempotency_key`     | A task-scoped idempotency key is still owned by another active or retained run. |
| `StorageConflictKind.LeaseOwnership`     | `lease_ownership`     | A lease heartbeat, release, or outcome no longer owns the run lease.            |
| `StorageConflictKind.OutboxClaim`        | `outbox_claim`        | A publisher update no longer owns the claimed outbox row.                       |
| `StorageConflictKind.ScheduleOccurrence` | `schedule_occurrence` | A schedule occurrence claim or completion no longer owns the occurrence.        |
| `StorageConflictKind.SingletonKey`       | `singleton_key`       | An environment-scoped singleton key is still owned by another active run.       |

## Persisted Failures [#persisted-failures]

`RunFailure.code` is always an `ErrorCode`. Unknown handler exceptions become `ErrorCode.TaskFailed`; structured `RunlaneError` failures preserve their code while core maps the message to task-facing text.

Use `RunFailure.meta` for user or provider detail that Runlane does not own as lifecycle vocabulary.

Keep metadata small, JSON-safe, and non-secret. Prefer flat diagnostic values such as provider codes, request ids, or resource ids. Do not persist raw provider responses, credentials, tokens, large payloads, or personally sensitive data in run history.

For example, a payment task can persist `{ providerCode: 'payment_declined' }` while keeping `code: ErrorCode.TaskFailed`; dashboards can group task failures by Runlane code and still show provider-specific detail.

`OutboxFailureRecord.code` is also an `ErrorCode`. Transport adapters should put provider response codes, request ids, and diagnostic detail in `meta` instead of creating a second code namespace.


# Identifiers (/contracts/vocabulary/identifiers)



Runlane's public identity values are opaque strings. Application code writes them as plain strings at authoring boundaries; Runlane validates and brands them before durable records, leases, schedules, and delivery messages are written.

Shared rules:

* Values must be non-empty strings.
* Values must not contain `:`.
* Treat values as whole identifiers, not parseable prefix paths.
* Use queues to route work; do not use worker IDs as routing targets.

`:` is reserved so storage adapters can compose their own backend-internal keys without colliding with public Runlane ids. Use dots, underscores, or a hashed segment for application structure:

| Need                           | Good                                  | Rejected                                         |
| ------------------------------ | ------------------------------------- | ------------------------------------------------ |
| Task id                        | `emails.welcome`                      | `emails:welcome`                                 |
| Idempotency key                | `emails.welcome.user_123`             | `emails:welcome:user_123`                        |
| Singleton key from external id | `quickbooks_invoices_user_123`        | `quickbooks:invoices:user_123`                   |
| Dynamic unsafe external id     | `provider_account_${hash(accountId)}` | Raw value that may contain `:` or unbounded text |

## Public Identity Values [#public-identity-values]

`Ideal owner` names who should supply or manage the value in normal application code. Durable records and operator APIs may still expose values owned by Runlane.

| Value                   | Ideal owner                                                     | What it names                                                  | What it is for                                                                                        |
| ----------------------- | --------------------------------------------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Task ID                 | User/app author                                                 | A stable task definition, such as `emails.welcome`             | Selects the registered handler and schema. Do not put tenant, user, or resource IDs here.             |
| Run ID                  | Runlane by default; user/operator only for external correlation | One durable execution record                                   | Names a specific run for operator reads or external correlation.                                      |
| Run step key            | User/app author                                                 | One named checkpoint inside one run                            | Lets `context.step.run()` replay a completed intermediate operation without rerunning its callback.   |
| Run step token          | Runlane                                                         | Deterministic token for `environment + runId + stepKey`        | Provider idempotency token or application-ledger key for the step callback's side effect.             |
| Observation record ID   | Runlane                                                         | One durable telemetry export record                            | Lets sinks dedupe at-least-once observation export deliveries.                                        |
| Observation consumer ID | User/operator config                                            | One exporter checkpoint stream, such as `production_otel`      | Lets independent sinks checkpoint their own progress through durable observation records.             |
| Schedule ID             | User/app author                                                 | A registered schedule definition                               | Deduplicates and audits schedule materialization. Use one stable ID per logical schedule.             |
| Queue name              | User/app deployment config                                      | A logical routing lane such as `default`, `emails`, or `media` | Serializes a provider-neutral queue definition into durable records and transport envelopes.          |
| Idempotency key         | User/app producer or task definition                            | One logical trigger request for one task/environment           | Returns the original active or retained terminal run for duplicate producer calls.                    |
| Singleton key           | User/app producer or task definition                            | One active resource that must not overlap                      | Prevents concurrent active runs for the same resource.                                                |
| Concurrency key         | User/app producer or task definition                            | One bounded-queue capacity partition                           | Limits execution concurrency per tenant, account, or resource without deduplicating trigger requests. |
| Worker ID               | Runlane by default; user config only for diagnostics            | The process or invocation that claimed work                    | Records diagnostic ownership for leases, schedule claims, and outbox claims. It does not route work.  |

Common authoring locations include:

* Task and schedule identity: `task({ id })` and `task({ schedule: { id } })`.
* Durable step identity: `context.step.run(stepKey, ...)`, with the callback receiving a deterministic `token`.
* Observation exporter identity: `createRunlaneObservationExporter({ consumer })`.
* Queue routing: `queue({ name })`, `task({ queue })`, trigger/runNow queue options, and worker queue filters.
* Run creation keys: static `task({ idempotencyKey, singletonKey, concurrencyKey })` values, task key resolvers, and explicit `trigger()` or `runNow()` options.
* Run and worker identity: `trigger(..., { runId })`, `runNow(..., { runId, workerId })`, `createRunlane({ workerId })`, `worker({ workerId })`, `executeNext({ workerId })`, and `executeDelivery(..., { workerId })`.

`environment.name` is also public, but it is a namespace rather than a Runlane ID. It scopes durable records so development, staging, production, tenants, or test suites do not see each other's runs.

## Choosing The Right Value [#choosing-the-right-value]

When deciding where a value belongs:

* If it names the kind of work, use the task ID.
* If it chooses which worker pool should process the work, use a queue definition.
* If it deduplicates a producer retry or returns a retained terminal owner, use the idempotency key. Repeat triggers with the same retained idempotency key return the original run.
* If it prevents overlap for one resource, use the singleton key.
* If every request should become a run but execution must be limited by partition, use the concurrency key on a bounded queue.
* If it identifies one execution record, use the run ID.
* If it identifies one checkpointed operation inside a run, use the run step key.
* If it identifies recurring materialization, use the schedule ID.
* If it identifies who claimed work, use the worker ID and keep it out of producer routing logic.

Runlane idempotency keys with a finite `idempotencyKeyTTL` are retained after successful or cancelled terminal completion, released immediately after failure, and released at any terminal state when the TTL is `'active'`. If a public API must reject payload mismatches or replay response bodies, keep that request ledger in the application boundary and trigger Runlane from the accepted request record.

## Authoring And Validation [#authoring-and-validation]

Public authoring APIs accept strings in place:

```ts
const emailQueue = queue({ name: 'emails', default: true })

const sendWelcomeEmail = task({
  id: 'emails.welcome',
  queue: emailQueue,
  schema,
  idempotencyKey: (payload) => `emails.welcome.${payload.userId}`,
  async run(payload) {
    await emailProvider.send(payload.userId)
  },
})

const runlane = createRunlane({
  lane,
  queues: [emailQueue],
  tasks: { sendWelcomeEmail },
})

await runlane.trigger(runlane.tasks.sendWelcomeEmail, { userId: 'user_123' })
await runlane.worker({ queues: [emailQueue] }).closed
```

Literal task IDs, static task keys, and task-colocated schedule IDs get TypeScript checks where possible. Queue names, explicit run IDs, worker IDs, and dynamic key resolver results still go through runtime validation because TypeScript cannot prove values from payloads, environment variables, HTTP requests, or databases are safe.


# Vocabulary (/contracts/vocabulary)



Runlane vocabulary is the set of stable values callers, adapters, transports, and operator tooling can branch on. These values are part of the public contract when they appear in persisted records, public errors, or runtime API results.

Use this section when you need to store, query, display, or switch on Runlane state. Do not invent parallel string values for concepts that already have a Runlane enum.

## What Belongs Here [#what-belongs-here]

| Category                                                           | Use                                                                                                                                                  |
| ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Error Codes](/contracts/vocabulary/error-codes)                   | Structured failures and storage conflict kinds from framework, adapters, task failure summaries, and outbox failure summaries.                       |
| [Identifiers](/contracts/vocabulary/identifiers)                   | Public task IDs, run IDs, run step keys, schedule IDs, queue names, idempotency keys, singleton keys, and worker IDs.                                |
| [Observability](/contracts/vocabulary/observability)               | Observation types, observation source kinds, exporter tick statuses, consumer ids, and cursor rules.                                                 |
| [Run Lifecycle](/contracts/vocabulary/run-lifecycle)               | Run statuses, status groups, event types, run sources, trigger outcomes, attempt summaries, dispatch reservations, and audit actors.                 |
| [Tasks And Schedules](/contracts/vocabulary/tasks-and-schedules)   | Retry backoff types, schedule shapes, schedule occurrence status, and duration units.                                                                |
| [Delivery And Workers](/contracts/vocabulary/delivery-and-workers) | Lane delivery modes, outbox message status, trigger dispatch modes, wakeup publish outcomes, worker modes, and transport-delivery execution results. |
| [Operator Reads](/contracts/vocabulary/operator-reads)             | Sort fields, sort directions, and cursor kinds for cursor-backed operator reads.                                                                     |

## What Stays Open [#what-stays-open]

These values are open vocabulary unless the contract gives you a specific enum:

* IDs and queue names
* environment names
* cursors
* cron expressions and time zones
* release and cancellation reasons
* provider message IDs
* metadata keys

Public identity values are covered in [Identifiers](/contracts/vocabulary/identifiers). Runlane-branded IDs and key values are opaque non-empty strings and must not contain `:`; treat them as whole values, not parseable prefix paths.

Provider-specific and business-domain error codes also stay out of Runlane enums. Store them in `meta` on the relevant failure record so operator tooling can branch on `ErrorCode` and still show domain detail.


# Observability (/contracts/vocabulary/observability)



Observability vocabulary appears in live observer payloads, durable observation export records, storage stream rows, and `@runlane/observability` exporter tick results. These values are stable public contracts for telemetry adapters, storage adapters, and operator tooling.

## RunlaneObservationType [#runlaneobservationtype]

| Enum                              | Value       | Meaning                                                                                |
| --------------------------------- | ----------- | -------------------------------------------------------------------------------------- |
| `RunlaneObservationType.RunEvent` | `run_event` | The observation describes a durable `RunEventRecord` that storage committed.           |
| `RunlaneObservationType.RunStep`  | `run_step`  | The observation describes a first-time durable step completion that storage committed. |

Both live observers and durable observation export use the same observation payload union. Payloads are sanitized by contract and do not include task payload, successful output, durable step output, failure detail, release metadata, or operator-expanded run rows by default.

## RunlaneObservationSourceKind [#runlaneobservationsourcekind]

| Enum                                    | Value       | Meaning                                 |
| --------------------------------------- | ----------- | --------------------------------------- |
| `RunlaneObservationSourceKind.RunEvent` | `run_event` | The durable source is a run event id.   |
| `RunlaneObservationSourceKind.RunStep`  | `run_step`  | The durable source is a run step token. |

Observation source kind and source id link each export record back to the committed fact that produced it. The same source must produce the same observation record id across retries and exporter restarts.

## RunlaneObservationExporterTickStatus [#runlaneobservationexportertickstatus]

| Enum                                            | Value      | Meaning                                                                                                  |
| ----------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------- |
| `RunlaneObservationExporterTickStatus.Aborted`  | `aborted`  | The exporter observed an aborted signal before or during the tick.                                       |
| `RunlaneObservationExporterTickStatus.Exported` | `exported` | The exporter scanned a non-empty batch, the sink accepted it, and the checkpoint advanced.               |
| `RunlaneObservationExporterTickStatus.Failed`   | `failed`   | The sink or checkpoint path failed before the checkpoint advanced; the tick includes the original error. |
| `RunlaneObservationExporterTickStatus.Idle`     | `idle`     | The scan returned no records after the current checkpoint.                                               |

`Failed` ticks are replay-safe because the checkpoint is not advanced. The long-running exporter loop retries retryable failures and untyped sink failures, but rejects non-retryable `RunlaneError` failures such as invalid configuration or unsupported storage capabilities. `Exported` ticks may still be delivered more than once externally when a sink accepts a batch but crashes before its downstream provider commits; sinks that require stronger semantics should dedupe by observation record id.

## Consumer IDs And Cursors [#consumer-ids-and-cursors]

Observation exporter consumers are ordinary Runlane id values: non-empty strings that must not contain `:`. Use one consumer id per independent sink, such as `production_otel` or `production_console`.

Observation cursors are opaque checkpoint values owned by storage. Pass the whole returned cursor back to the same storage adapter; do not parse stream positions or assume one backend's cursor encoding applies to another.


# Operator Reads (/contracts/vocabulary/operator-reads)



Operator read vocabulary appears in `runlane.runs` reads, storage `listRuns()` / `listRunEvents()` queries, and cursor-backed continuation tokens. Runtime APIs validate public options and inject the current environment; storage adapters own indexes, cursor encoding, and stable ordering.

Cursor strings are opaque to callers. A storage adapter may encode filter and sort state inside the cursor, but callers should only pass the whole `nextCursor` token back to the same API with the same logical request.

## Runtime APIs [#runtime-apis]

| API                                   | Storage contract                                                                               | Return shape                   |
| ------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------ |
| `runlane.runs.get(runId)`             | `getRun({ environment, runId })`                                                               | `RunRecord` or `undefined`     |
| `runlane.runs.list(options)`          | `listRuns({ filter, pagination })`                                                             | `Page<RunSummary>`             |
| `runlane.runs.events(runId, options)` | `listRunEvents({ filter, pagination })`                                                        | `Page<RunEventRecord>`         |
| `runlane.runs.attempts(runId)`        | Paged `listRunEvents()` with `RunEventSortField.Sequence` and `SortDirection.Asc`              | `readonly RunAttemptSummary[]` |
| `runlane.runs.findActive(options)`    | `listRuns()` over active statuses and task/key filters, then `getRun()` for the selected match | `RunRecord` or `undefined`     |
| `runlane.runs.findCurrent(options)`   | `getRunByIdempotencyKey()`                                                                     | `RunRecord` or `undefined`     |

`Page<T>` always returns `data: readonly T[]` and may return `nextCursor`. `PaginationParams.limit` is optional and should be clamped by storage using `contractDefaults.pagination`; `PaginationParams.cursor` is the opaque continuation token from the previous page.

## RunFilter [#runfilter]

`RunFilter` is the storage-facing filter for `listRuns()`. Runtime `ListRunsOptions` uses the same public knobs except that `environment` is injected and public `queues` are registered queue definitions that core resolves to queue names before storage sees them.

| Field            | Meaning                                                                                                                                    |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `environment`    | Required storage scope. Runtime injects the current environment.                                                                           |
| `statuses`       | Include only matching `RunStatus` values.                                                                                                  |
| `taskIds`        | Include only matching task ids.                                                                                                            |
| `queues`         | Include only matching queue names.                                                                                                         |
| `createdAt`      | Inclusive `{ from?, to? }` bounds over run creation time.                                                                                  |
| `updatedAt`      | Inclusive `{ from?, to? }` bounds over materialized run update time.                                                                       |
| `runAt`          | Inclusive `{ from?, to? }` bounds over the persisted `run.runAt` value. Runs without `runAt` match only when no `runAt` range is supplied. |
| `idempotencyKey` | Include runs that captured this idempotency key.                                                                                           |
| `singletonKey`   | Include runs that captured this singleton key.                                                                                             |
| `sourceRunId`    | Include linked children created from the source run, including operator reruns, manual retries, and task-context child triggers.           |
| `sortBy`         | Stable run sort field. Defaults to `contractDefaults.sort.runs.field`.                                                                     |
| `sortDirection`  | Stable run sort direction. Defaults to `contractDefaults.sort.runs.direction`.                                                             |

## RunEventFilter [#runeventfilter]

`RunEventFilter` is the storage-facing filter for `listRunEvents()`. Runtime `runlane.runs.events(runId, options)` always supplies `runId`; direct storage callers may omit it only when using globally meaningful event ordering.

| Field           | Meaning                                                                                |
| --------------- | -------------------------------------------------------------------------------------- |
| `environment`   | Required storage scope. Runtime injects the current environment.                       |
| `runId`         | Restricts results to one run's append-only history. Required when sorting by sequence. |
| `types`         | Include only matching `RunEventType` values.                                           |
| `occurredAt`    | Inclusive `{ from?, to? }` bounds over event occurrence time.                          |
| `sortBy`        | Stable event sort field. Defaults to `contractDefaults.sort.runEvents.field`.          |
| `sortDirection` | Stable event sort direction. Defaults to `contractDefaults.sort.runEvents.direction`.  |

## SortDirection [#sortdirection]

| Enum                 | Value  | Meaning                                     |
| -------------------- | ------ | ------------------------------------------- |
| `SortDirection.Asc`  | `asc`  | Return oldest or lowest sort values first.  |
| `SortDirection.Desc` | `desc` | Return newest or highest sort values first. |

## RunSortField [#runsortfield]

| Enum                     | Value        | Meaning                                                                                                                                                           |
| ------------------------ | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RunSortField.CreatedAt` | `created_at` | Sort runs by creation time. This is the default run-list field.                                                                                                   |
| `RunSortField.RunAt`     | `run_at`     | Sort runs by next runnable availability, using the same policy as `getRunRunnableAvailableAt({ run })`. Runs with no runnable time sort after runs that have one. |
| `RunSortField.UpdatedAt` | `updated_at` | Sort runs by last materialized update time.                                                                                                                       |

`RunSortField.RunAt` is not the same as the `runAt` filter. The filter checks the persisted `run.runAt` field; the sort key uses runnable availability so queued runs without `runAt` sort by `updatedAt`, waiting runs sort by `runAt`, and running runs sort by lease expiry.

## RunEventSortField [#runeventsortfield]

| Enum                           | Value         | Meaning                                                                                                                       |
| ------------------------------ | ------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `RunEventSortField.OccurredAt` | `occurred_at` | Sort events by occurrence time. This is the default event-list field.                                                         |
| `RunEventSortField.Sequence`   | `sequence`    | Sort events by per-run sequence. Only use with a `runId` filter because sequences are monotonic inside one run, not globally. |

## Result Shapes [#result-shapes]

`RunSummary` is the compact shape returned by run lists. It includes `id`, `environment`, `taskId`, `queue`, `status`, `counters`, `createdAt`, and `updatedAt`, plus optional `runAt`, `startedAt`, `finishedAt`, `failure`, `source`, and `wait`. It intentionally omits payloads, outputs, leases, trace carriers, metadata, idempotency keys, singleton keys, concurrency keys, attempt deadlines, and dispatch reservation fields.

`RunEventRecord` is the append-only history record returned by event lists. It includes the storage-assigned event `id`, the per-run `sequence`, and the typed `event` payload. Event sequences are stable inside one run; cross-run event lists should use occurrence-time ordering.

`RunAttemptSummary` is derived by `runlane.runs.attempts(runId)` from replayed run events. It is not persisted separately.

| Field         | Meaning                                                                                                     |
| ------------- | ----------------------------------------------------------------------------------------------------------- |
| `attempt`     | Attempt number from execution events.                                                                       |
| `startedAt`   | Start time derived from `run.started.occurredAt`.                                                           |
| `status`      | Derived `RunAttemptStatus`. See [Run lifecycle](/contracts/vocabulary/run-lifecycle#runattemptstatus).      |
| `completedAt` | Outcome event time for succeeded, failed, retrying, released, or cancelled attempts.                        |
| `output`      | JSON output from `run.succeeded`, omitted for void success and non-success outcomes.                        |
| `failure`     | Failure summary from `run.failed` or `run.retry_scheduled`.                                                 |
| `retry`       | Retry delay and `retryAt` from `run.retry_scheduled`.                                                       |
| `release`     | Release delay or external wait condition, optional reason, and time-release `resumeAt` from `run.released`. |

Only `run.started` creates an attempt summary. `run.succeeded`, `run.failed`, `run.retry_scheduled`, and `run.released` complete the matching attempt. `run.cancelled` marks the latest running attempt cancelled; cancelling queued, released, retrying, or scheduled work without a started attempt does not create an attempt row.

## PaginationCursorKind [#paginationcursorkind]

First-party storage adapters use these values inside opaque cursor payloads. Callers must still treat cursor strings as whole tokens and never parse them.

| Enum                                | Value            | Meaning                                   |
| ----------------------------------- | ---------------- | ----------------------------------------- |
| `PaginationCursorKind.RunList`      | `run_list`       | Cursor for operator run summary pages.    |
| `PaginationCursorKind.RunEventList` | `run_event_list` | Cursor for run event history pages.       |
| `PaginationCursorKind.RunPrune`     | `run_prune`      | Cursor for terminal run pruning progress. |


# Run Lifecycle (/contracts/vocabulary/run-lifecycle)



Run lifecycle vocabulary appears in durable run records and append-only run events. Storage, operator tools, and dashboards can branch on these values.

## RunStatus [#runstatus]

| Enum                              | Value                    | Meaning                                                                                                                                                 |
| --------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RunStatus.CancellationRequested` | `cancellation_requested` | Cancellation was requested for a leased running run; the owner may stop cooperatively, and maintenance can finalize after lease expiry.                 |
| `RunStatus.Cancelled`             | `cancelled`              | Terminal cancellation outcome.                                                                                                                          |
| `RunStatus.Failed`                | `failed`                 | Terminal failure outcome after retry budget is exhausted, retry is not allowed, or a cancellation-requested attempt fails.                              |
| `RunStatus.Queued`                | `queued`                 | Waiting for worker acquisition or dispatch; runnable when `runAt ?? updatedAt` is due and queue capacity or an existing dispatch reservation allows it. |
| `RunStatus.Released`              | `released`               | Business waiting; runnable again when a time wait is due, a signal is sent, or a source run reaches terminal state.                                     |
| `RunStatus.Retrying`              | `retrying`               | Failure-driven retry; runnable again when `runAt` is due.                                                                                               |
| `RunStatus.Running`               | `running`                | A worker owns an active lease; the attempt is about to start or is executing.                                                                           |
| `RunStatus.Scheduled`             | `scheduled`              | Delivery has been requested for a future `runAt`.                                                                                                       |
| `RunStatus.Succeeded`             | `succeeded`              | Terminal success outcome.                                                                                                                               |

## RunStatusClassification [#runstatusclassification]

| Enum                               | Value      | Meaning                                                                       |
| ---------------------------------- | ---------- | ----------------------------------------------------------------------------- |
| `RunStatusClassification.Active`   | `active`   | The run can still move through lifecycle transitions.                         |
| `RunStatusClassification.Terminal` | `terminal` | The run does not reactivate; rerun and manual retry create linked child runs. |

Use `runStatusValues.isActive()` and `runStatusValues.isTerminal()` instead of duplicating classification lists.

Status classification is broader than worker eligibility. `cancellation_requested` is active because it can still move to `cancelled` or `failed`, but it is not a runnable candidate. Terminal runs never accept more lifecycle events; rerun and manual retry create linked child runs instead.

## Scan candidate status groups [#scan-candidate-status-groups]

Storage scans use narrower status groups than active/terminal classification. These groups are status-only gates; time and lease eligibility still come from `getRunRunnableAvailableAt()`, `getRunDeliveryRecoveryAvailableAt()`, and `getRunCancellationFinalizationAvailableAt()`.

| Group                                                | Values                                                   | Meaning                                                                                                         |
| ---------------------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `runStatusValues.runnableCandidates`                 | `queued`, `released`, `retrying`, `running`, `scheduled` | Statuses that `listRunnableRuns()` may return when the run is due or its lease has expired.                     |
| `runStatusValues.deliveryRecoveryCandidates`         | `released`, `retrying`, `running`, `scheduled`           | Statuses that `listRunsNeedingDelivery()` may return when maintenance needs to append a fresh delivery request. |
| `runStatusValues.cancellationFinalizationCandidates` | `cancellation_requested`                                 | Statuses that `listRunsNeedingCancellationFinalization()` may return after the current lease expires.           |

`queued` is excluded from delivery recovery candidates because it already has durable delivery intent and outbox state.

Use the matching predicates, such as `runStatusValues.isDeliveryRecoveryCandidate()`, instead of duplicating status switches in adapters.

Attempt-timeout finalization is narrower than a status group: it applies only to `running` runs whose current lease has recorded `run.started` and whose fixed `attemptDeadlineAt` has passed. Use `getRunAttemptTimeoutFinalizationAvailableAt()` for that predicate instead of treating all expired leases as timeouts.

## Dispatch Reservation Predicate [#dispatch-reservation-predicate]

Bounded queues reserve capacity before publishing a delivery wakeup. Use this helper anywhere storage code needs to reason about that reservation window.

| Helper                                                                                      | Condition | Meaning                                                               |
| ------------------------------------------------------------------------------------------- | --------- | --------------------------------------------------------------------- |
| `isRunDispatchReservation({ run, at, condition: RunDispatchReservationCondition.Active })`  | `active`  | The queued run still owns a dispatch slot at `at`.                    |
| `isRunDispatchReservation({ run, at, condition: RunDispatchReservationCondition.Expired })` | `expired` | The queued run had a dispatch slot, but it expired at or before `at`. |

## RunCounters [#runcounters]

`RunRecord.counters` is materialized from replayed events. It is durable run state, not a derived dashboard-only summary.

| Field      | Incremented by                      | Meaning                                                                       |
| ---------- | ----------------------------------- | ----------------------------------------------------------------------------- |
| `attempts` | `run.started`                       | Count of started execution attempts.                                          |
| `failures` | `run.failed`, `run.retry_scheduled` | Failure pressure recorded for terminal failures and retry-scheduled failures. |
| `releases` | `run.released`                      | Business waits that did not count as failure pressure.                        |
| `retries`  | `run.retry_scheduled`               | Failed attempts that scheduled another try.                                   |

`run.succeeded` and `run.cancelled` do not increment counters by themselves. They close an existing attempt or, for waiting-run cancellation, finish a run that may have no attempts.

## RunAttemptStatus [#runattemptstatus]

`RunAttemptStatus` appears in `runlane.runs.attempts(runId)` summaries. It is derived from replayed run events; it is not stored as a separate durable run field.

| Enum                         | Value       | Meaning                                                        |
| ---------------------------- | ----------- | -------------------------------------------------------------- |
| `RunAttemptStatus.Cancelled` | `cancelled` | The attempt ended after cooperative cancellation.              |
| `RunAttemptStatus.Failed`    | `failed`    | The attempt failed without scheduling another try.             |
| `RunAttemptStatus.Released`  | `released`  | The attempt released the run to resume later.                  |
| `RunAttemptStatus.Retrying`  | `retrying`  | The attempt failed and scheduled another try.                  |
| `RunAttemptStatus.Running`   | `running`   | The attempt has started and has no terminal attempt event yet. |
| `RunAttemptStatus.Succeeded` | `succeeded` | The attempt completed successfully.                            |

Only `run.started` creates an attempt summary. A later `run.cancelled` marks the latest still-running attempt as cancelled; cancelling queued, released, retrying, or scheduled work without a started attempt does not create an attempt row.

## RunEventType [#runeventtype]

| Enum                                 | Value                        | Meaning                                                                                                                   |
| ------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `RunEventType.CancellationRequested` | `run.cancellation_requested` | Operator or system requested cooperative cancellation of a leased running run.                                            |
| `RunEventType.Cancelled`             | `run.cancelled`              | Waiting work, a cancellation-requested run, or the latest running attempt completed with terminal cancellation.           |
| `RunEventType.Created`               | `run.created`                | First durable event for a run.                                                                                            |
| `RunEventType.DeliveryRequested`     | `run.delivery_requested`     | Runtime recorded durable wakeup intent; transport-delivery lanes may also request a matching outbox row.                  |
| `RunEventType.Failed`                | `run.failed`                 | Current started attempt ended in terminal failure.                                                                        |
| `RunEventType.LeaseClaimed`          | `run.lease_claimed`          | Worker or inline execution claimed ownership for execution.                                                               |
| `RunEventType.LeaseHeartbeat`        | `run.lease_heartbeat`        | Worker extended the current lease with the same worker id and token.                                                      |
| `RunEventType.Released`              | `run.released`               | Attempt chose business waiting without counting a failure. It may carry a time, signal, or run-completion wait condition. |
| `RunEventType.RetryScheduled`        | `run.retry_scheduled`        | Current started attempt failed and scheduled another try.                                                                 |
| `RunEventType.Started`               | `run.started`                | Execution attempt started.                                                                                                |
| `RunEventType.Succeeded`             | `run.succeeded`              | Attempt completed successfully, optionally with JSON output.                                                              |

## RunSourceType [#runsourcetype]

| Enum                        | Value          | Meaning                                                       |
| --------------------------- | -------------- | ------------------------------------------------------------- |
| `RunSourceType.ManualRetry` | `manual_retry` | Operator-created child retry linked to a failed source run.   |
| `RunSourceType.Rerun`       | `rerun`        | Operator-created child rerun linked to a terminal source run. |
| `RunSourceType.Schedule`    | `schedule`     | Schedule materialization created the run.                     |
| `RunSourceType.Trigger`     | `trigger`      | Application trigger created the run.                          |

Task-context child triggers also use `RunSourceType.Trigger` with `sourceRunId` set to the parent run. Top-level public triggers usually have no source.

## RunWaitConditionType [#runwaitconditiontype]

| Enum                                 | Value            | Meaning                                                                                                                                                               |
| ------------------------------------ | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RunWaitConditionType.RunCompletion` | `run_completion` | Released run resumes when the referenced source run reaches terminal state, immediately if that source is already terminal, or when its optional timeout becomes due. |
| `RunWaitConditionType.Signal`        | `signal`         | Released run resumes when application code sends the matching signal key, or when its optional timeout becomes due.                                                   |
| `RunWaitConditionType.Time`          | `time`           | Released run resumes when `resumeAt` becomes due through maintenance delivery.                                                                                        |

## TriggerOutcomeType [#triggeroutcometype]

`TriggerOutcomeType` appears in `TriggerRunResult` from `runlane.trigger()`. It describes the result of the trigger call, not the run lifecycle status.

| Enum                                  | Value               | Meaning                                                                          |
| ------------------------------------- | ------------------- | -------------------------------------------------------------------------------- |
| `TriggerOutcomeType.Created`          | `created`           | The trigger call persisted a new run.                                            |
| `TriggerOutcomeType.ReturnedExisting` | `returned_existing` | Idempotency returned the active or retained run already owning the selected key. |

## ActorType [#actortype]

| Enum                 | Value      | Meaning                       |
| -------------------- | ---------- | ----------------------------- |
| `ActorType.Operator` | `operator` | Human or operator action.     |
| `ActorType.Schedule` | `schedule` | Schedule materializer action. |
| `ActorType.System`   | `system`   | Framework/system action.      |
| `ActorType.Worker`   | `worker`   | Worker execution action.      |

`contractDefaults.actor.system` and `contractDefaults.actor.operator` provide the canonical actor records for framework and operator actions that do not carry a more specific actor id.

## Output And Failure Fields [#output-and-failure-fields]

`RunRecord.output` is the current visible successful output. It is present only when `run.succeeded` carried JSON output; `undefined` handler returns are void success and leave the field absent. `null` is explicit JSON output and remains visible as `output: null`.

`RunRecord.failure` is the current visible failure summary for failed and retrying runs. Succeeded runs may have `output`; failed and retrying runs may have `failure`; released and cancelled runs have neither current success output nor current failure. Use `runlane.runs.attempts(runId)` or raw events when you need previous attempt outcomes.

`RunRecord.wait` is the current visible business-wait condition for released runs. `RunSummary.wait` carries the same compact condition for list rows. `RunRecord.attemptDeadlineAt` is visible while a running attempt has a fixed max-attempt deadline and clears when that attempt leaves `running`.


# Tasks And Schedules (/contracts/vocabulary/tasks-and-schedules)



Task and schedule vocabulary appears in public task configuration, runtime trigger boundaries, release results, and durable schedule occurrence state. Use these values when authoring task definitions, validating runtime options, or implementing storage schedule claims.

Task ids, queue names, idempotency keys, singleton keys, and concurrency keys are opaque public identifiers; see [Identifiers](/contracts/vocabulary/identifiers). Trigger outcomes and run sources are run-lifecycle vocabulary; see [Run Lifecycle](/contracts/vocabulary/run-lifecycle). Queue policy fields are provider-neutral configuration rather than enum values; see [Queues](/concepts/queues).

## Task Boundary Vocabulary [#task-boundary-vocabulary]

\| Value | Stable shape | Meaning |
\| --- | --- | --- | --- |
\| `retry.maxAttempts` | Positive safe integer | Total attempt budget, including the first attempt. |
\| `retry.backoff` | Duration string or `RetryBackoff` object | Delay policy used only after retryable failures. A string is fixed backoff. |
\| `maxAttemptDuration` | Duration string | Maximum wall-clock duration for one attempt. The fixed deadline is written when the attempt lease is claimed. |
\| `context.release(delay, options)` | `TaskReleaseType.Release` result | Business waiting result that the handler must return to release the run. It is not a failure or retry. |
\| `context.waitForSignal(signalKey, options)` | `TaskReleaseType.Release` result | Business wait until `runlane.signals.send(signalKey)` requests delivery, with optional timeout fallback. |
\| `context.waitForRun(runId, options)` | `TaskReleaseType.Release` result | Business wait until the referenced run reaches terminal state, with optional timeout fallback. |
\| `runlane.signals.send(signalKey, options)` | `Promise<readonly RunRecord[]>` | Runtime API that appends durable resume delivery requests for released runs waiting on the signal key. It does not execute handlers inline. |
\| `context.trigger(task, payload, options)` | `Promise<TriggerRunResult>` | Child trigger that validates and creates work through the same runtime path as public `trigger()`, linked to the parent run. |
\| `context.step.run(stepKey, { output }, callback)` | `Promise<SchemaOutput>` | Durable per-run checkpoint. Completed steps return stored output and skip the callback on later attempts. |
\| handler JSON return | `JsonValue | undefined` | Successful output. `undefined` is void success; `null` is explicit JSON output. |
\| `idempotencyKeyTTL` | Duration string or `'active'` | Retention policy for the selected idempotency owner. It requires an idempotency key. |

When an idempotency key is selected and no TTL is supplied, run creation captures `30d`. With a finite TTL, successful and cancelled terminal owners remain readable until that TTL expires; failed terminal owners release automatically. The literal `'active'` releases ownership when the run reaches any terminal state.

Release `reason` strings, cancellation reasons, cron expressions, time zones, and metadata keys are open vocabulary. Keep provider or business-specific values in `meta` instead of inventing Runlane enum values.

## RetryBackoffType [#retrybackofftype]

| Enum                           | Value         | Meaning                                                                       |
| ------------------------------ | ------------- | ----------------------------------------------------------------------------- |
| `RetryBackoffType.Exponential` | `exponential` | Retry delay doubles from the base delay and respects `maxDelay` when present. |
| `RetryBackoffType.Fixed`       | `fixed`       | Every retry uses the same delay.                                              |

A string `retry.backoff` is accepted as shorthand for `RetryBackoffType.Fixed`. Object backoff policies should use `RetryBackoffType`. When a retry policy omits `backoff`, core uses `contractDefaults.retry.backoff`, currently `30s`.

When `retry` itself is omitted, task failures do not schedule automatic retries.

`RetryBackoffType.Exponential` uses the failed attempt number: the first retry waits the base delay, then later retries double from that base. `maxDelay` must be greater than or equal to `delay`.

## TaskRelease [#taskrelease]

`TaskRelease` is the typed handler result returned by `context.release(...)`, `context.waitForSignal(...)`, and `context.waitForRun(...)`. `type: 'task_release'` is reserved as the release discriminant, so successful output objects must not use that exact `type` value.

| Field    | Required              | Meaning                                                                               |
| -------- | --------------------- | ------------------------------------------------------------------------------------- |
| `type`   | Yes                   | `TaskReleaseType.Release`, serialized as `task_release`.                              |
| `delay`  | Time waits only       | Runlane duration string until the released run becomes due again.                     |
| `wait`   | Signal/run waits only | Durable wait condition for `context.waitForSignal(...)` or `context.waitForRun(...)`. |
| `reason` | No                    | Short public reason for operator views. Runlane treats values as open vocabulary.     |
| `meta`   | No                    | JSON object with structured domain detail.                                            |

| Enum                      | Value          | Meaning                                                                 |
| ------------------------- | -------------- | ----------------------------------------------------------------------- |
| `TaskReleaseType.Release` | `task_release` | Handler result that releases the current attempt into business waiting. |

Signal and run-completion releases store `RunWaitConditionType.Signal` or `RunWaitConditionType.RunCompletion`, plus an optional timeout. They do not duplicate that timeout as `resumeAt` on the event. Time releases store `delay` and the reducer-owned `resumeAt`; run projection materializes `RunWaitConditionType.Time` from that due time. Core validates release arguments before returning the result. A plain object with fields such as `delay`, `reason`, or `meta` but no `TaskReleaseType.Release` discriminant is ordinary handler output and must be JSON-compatible.

## RunlaneSignalsApi [#runlanesignalsapi]

`runlane.signals.send(signalKey, options)` resumes released runs waiting on one external signal key. It validates `signalKey` with the same public identifier rules as other Runlane id values: non-empty string, no `:`, and no hidden parsing contract.

| Option          | Required | Meaning                                        |
| --------------- | -------- | ---------------------------------------------- |
| `signalKey`     | Yes      | External fact the released run is waiting for. |
| `options.limit` | No       | Positive integer cap for one resume scan.      |

The return value contains the `RunRecord` values whose wait condition was cleared by a newly appended `run.delivery_requested` event. It is not an execution result and does not guarantee the next attempt has run. Duplicate sends after a wait is already cleared return no run for that cleared wait.

Signals do not carry payloads. Store webhook bodies, approval detail, provider result data, and other domain facts in application storage; use the signal key only to wake runs that should reread that state.

## TaskStepContext [#taskstepcontext]

`context.step.run(stepKey, { output }, callback)` persists one named step completion for the current run. The callback returns the `output` schema input shape, and the resolved value is the parsed JSON-safe output shape. A completed step is replayed for the same run and step key without calling the callback again.

Step callbacks receive `{ token }`, a deterministic `RunStepToken` derived from environment, run id, and step key. Pass it to providers that support idempotency tokens, or use it as the key for an application submission ledger. The token is opaque and should not be parsed.

Step keys follow public identifier rules: non-empty string, no `:`, stable across deployments. Store only small continuation pointers such as provider job ids in step output; domain state belongs in application storage and final task results belong in handler output.

## Schedule Authoring Vocabulary [#schedule-authoring-vocabulary]

Task-colocated schedules infer their kind from exactly one selector field. The public authoring shape must not include `type` or `taskId`; core attaches those lower-level fields during registration.

| Selector | Inferred `ScheduleType` | Meaning                                            |
| -------- | ----------------------- | -------------------------------------------------- |
| `runAt`  | `ScheduleType.Once`     | Materialize one run at a specific `Date`.          |
| `every`  | `ScheduleType.Interval` | Materialize recurring runs on a duration cadence.  |
| `cron`   | `ScheduleType.Cron`     | Materialize recurring runs from a cron expression. |

Shared schedule authoring fields are `id`, optional `queue`, optional `enabled`, and optional or required static `payload` depending on the task schema. Schedule queues must match the runtime queue registry. Static payloads validate synchronously at task registration and again before materialization.

## ScheduleType [#scheduletype]

| Enum                    | Value      | Meaning                                       |
| ----------------------- | ---------- | --------------------------------------------- |
| `ScheduleType.Cron`     | `cron`     | Materialize runs from a cron expression.      |
| `ScheduleType.Interval` | `interval` | Materialize runs on a fixed duration cadence. |
| `ScheduleType.Once`     | `once`     | Materialize one run at a specific time.       |

Lower-level `ScheduleDefinition` records use this discriminant after core normalizes task-colocated schedules.

Interval schedules anchor to Unix epoch unless `startsAt` is supplied. `endsAt` caps the latest eligible interval boundary. Cron schedules use the supplied `timeZone` when present. Each maintenance pass materializes at most one latest due occurrence per schedule; missed intermediate boundaries are not backfilled in the same tick.

## ScheduleOccurrenceStatus [#scheduleoccurrencestatus]

| Enum                                 | Value       | Meaning                                                                    |
| ------------------------------------ | ----------- | -------------------------------------------------------------------------- |
| `ScheduleOccurrenceStatus.Claimed`   | `claimed`   | A scheduler owns the occurrence and may be creating or recovering its run. |
| `ScheduleOccurrenceStatus.Completed` | `completed` | The occurrence has recorded the run ids it produced.                       |

Occurrence ids are deterministic for `environment + scheduleId + fireAt`. If a scheduler creates the run and crashes before completion, another materializer can reclaim the occurrence after claim expiry, recover the already-created scheduled run, and complete the occurrence with that run id.

## DurationUnit [#durationunit]

| Enum                       | Value | Meaning                                |
| -------------------------- | ----- | -------------------------------------- |
| `DurationUnit.Day`         | `d`   | Day unit for duration strings.         |
| `DurationUnit.Hour`        | `h`   | Hour unit for duration strings.        |
| `DurationUnit.Millisecond` | `ms`  | Millisecond unit for duration strings. |
| `DurationUnit.Minute`      | `m`   | Minute unit for duration strings.      |
| `DurationUnit.Second`      | `s`   | Second unit for duration strings.      |

Duration strings look like `500ms`, `0.5s`, `30s`, `5m`, `2h`, or `7d`. Durations must include a unit, be no longer than 100 characters, and resolve to positive whole safe-integer milliseconds.

Use `durationSchema` or `createDurationSchema(message)` at boundaries; read `.duration` for the public string and `.milliseconds` for executable time math. Duration strings appear in retry backoff, release delay, signal/run-completion wait timeouts, max attempt duration, idempotency TTLs, interval schedules, worker lease options, dispatch timeouts, maintenance claim durations, and pruning filters.