Postgres storage driver

@runlane/postgres-storage implements durable StorageDriver records and the optional OperatorReadDriver projection. Most applications should use it through a ready-made Postgres lane.

Apply migrations

Run migrations as a deployment step, before application processes start:

import { applyPostgresStorageMigrations } from '@runlane/postgres-storage'
import * as z from 'zod'

const { DATABASE_URL } = z.object({ DATABASE_URL: z.string().min(1) }).parse(process.env)

await applyPostgresStorageMigrations({ connectionString: DATABASE_URL, schema: 'runlane' })

The package's schema declarations are the source of truth. Release tooling generates immutable SQL and an ordered journal; explicit migration commands apply that history and validate the installed version, ordering, and ledger hashes. The driver neither applies nor inspects migrations at application startup, so deployment must complete migration validation before starting application processes.

The current schema version is 1, derived from the package migration journal and exported as postgresStorageSchemaVersion. The single 0000_initial.sql migration creates the complete storage schema, including separate run payload storage, indexed operator columns on canonical run and event records, event-ID-keyed canonical records with native run sequencing, initial-event payload references, and the native record-write assertion function. Runlane records its application in <schema>.runlane_migrations.

Run payloads live in record_payload, separate from changing lifecycle metadata; reads reconstruct the same logical document. Unchanged payloads retain their PostgreSQL TOAST storage during status updates. Canonical run and event rows own their indexed operator metadata, without duplicate run or event documents. A canonical event's UUID is its record_id; typed run_id and sequence columns provide the native RunEventsBySequence path without duplicating its encoded partition and sort columns.

When one atomic creation writes an exact canonical run and its initial Created event with the same payload, PostgreSQL stores that payload only on the run. The event carries a durable reference marker, and raw and operator reads reconstruct the complete event. Before either record changes in a way that would break that relationship, the same transaction copies the payload into the event and clears the marker. Opaque, copied, or independently written events retain their inline payloads.

The migration account must be able to create runlane_assert_record_write, and runtime accounts must be able to execute it. The function aborts an entire write statement when a conditional mutation or append check fails. See transaction internals for the implicit-commit path this enables.

This baseline intentionally replaces the schema shipped with Runlane 0.3. A 0.3 database cannot be upgraded in place; deploy 0.4 to a fresh schema. The migration runner rejects mismatched ledgers. Starting with the 0.4 baseline, released migrations are append-only. There are no down migrations; use a database snapshot or a new forward migration for rollback.

Create the driver

import { createPostgresStorageDriver } from '@runlane/postgres-storage'

export async function openPostgresDriver(connectionString: string) {
  const driver = createPostgresStorageDriver({
    connectionString,
    schema: 'runlane',
  })

  await driver.start?.()
  return driver
}

The caller owns driver.close() after it is finished.

The provider uses READ COMMITTED, optimistic validation, advisory locks, and atomic transactions. maxTransactionAttempts defaults to 8. Exhaustion returns a retryable storage conflict.

It supports optional ordered append and exclusive candidates. Core's runnable selection uses FOR UPDATE SKIP LOCKED to avoid workers selecting the same busy row. Observation positions are allocated in PostgreSQL at the end of each transaction, shortening the time the shared environment head is held. The head remains a durable row shared by all processes. These optimizations require no additional tables or application API changes.

Use alongside Prisma

Keep Runlane tables out of schema.prisma. Run runlane adapter postgres migrate for Runlane, then prisma migrate deploy for application tables. The two histories may target the same PostgreSQL schema because they own distinct tables and ledgers; separate schemas remain easier for local prisma migrate dev drift checks.

Know what is stored

Postgres uses normalized tables for the closed logical collection catalog. Important groups are:

Documents remain opaque jsonb. Core's codecs own their shape. Provider columns project only contract-owned keys and named indexes.

One non-cycling bigint sequence assigns record versions across every table. Rollbacks can leave gaps. Versions are opaque fences, not counters.

Understand operator reads

Canonical run and event rows update their indexed operator metadata in the same transaction as their documents. getRun() and listRuns() read runlane_runs directly. listRunEvents() pages indexed event metadata first, checks each candidate's visible parent by the run primary key, and fetches an event document by its primary key only when that parent exists. Hidden orphan metadata still advances the keyset cursor, so filtering cannot stall or skip later visible events and orphan documents are not returned.

A malformed or non-canonical raw document remains point-readable as an opaque record but does not populate operator columns. Tombstoning hides a run and its events immediately, even while bounded cleanup removes child rows. Run and event rows deliberately have no foreign key: deleting a raw run retains its event rows until explicit child cleanup. The retained canonical event's (environment_key, record_id) primary key continues to reserve its event UUID during that interval.

Understand transaction time

The provider reports transaction_authoritative clock semantics:

1. The attempt collects its point-read dependencies.
2. It locks and validates them in canonical key order. An exclusive candidate query may already own the complete read set.
3. The first tx.now() freezes a clock_timestamp() sampled after the complete read set was locked, either by the exclusive candidate query or by read-set sealing.
4. Data-modifying statements apply point mutations, derived operator columns, and any ordered append on the same connection.
5. PostgreSQL commits once.

Busy blocking admission has a two-second server-side lock timeout. A changed dependency, busy lock, serialization failure, or deadlock consumes a bounded replay attempt.

Do not open another storage transaction inside the callback. Do not perform external effects there. The callback may run again.

Effective default limits

These values describe the provider's effective profile. Application payloads should stay much smaller than the hard record boundary.

Verify a provider change

After changing migrations, mappings, SQL, locks, clocks, or error handling, run:

pnpm --filter @runlane/postgres-storage test:postgres:local

The live suite covers fresh setup, tamper detection, conformance, operator-column rollback, contention, clock-after-lock behavior, and operator-query parity.

See Postgres storage internals for the exact physical mappings and replay algorithm. The generic rules are in build a storage provider, and stable names are in storage contract values.