Concurrency

Use queue concurrency to protect model capacity and prevent one user from occupying every worker. Storage enforces the limit across all workers in the same Runlane environment.

Limit concurrent requests per user

The Runlane queue and concurrency key below are real. The marked generateAiResponse() call is pseudocode for your model provider:

import { queue, task } from '@runlane/core'
import * as z from 'zod'

export const aiRequestsQueue = queue({
  name: 'ai-requests',
  concurrencyLimit: 2,
})

const responseSchema = z.object({ response: z.string(), tier: z.enum(['free', 'pro']) })

export const processAiRequest = task({
  id: 'ai.process-request',
  queue: aiRequestsQueue,
  schema: z.object({
    prompt: z.string().min(1),
    tier: z.enum(['free', 'pro']),
    userId: z.string(),
  }),
  output: responseSchema,
  concurrencyKey: ({ userId }) => userId,
  async run({ prompt, tier }, context) {
    //////////////////////////////////
    // MAKE YOUR API CALLS HERE
    //////////////////////////////////
    const response = await generateAiResponse({ prompt, signal: context.signal, tier })

    return responseSchema.parse({ response, tier })
  },
})

The queue allows two active attempts for each userId. Different users receive separate capacity partitions. The tier can select application behavior or a model, but it does not change the Runlane queue limit at runtime.

If plans need different limits, register separate queues and tasks with fixed limits. Runlane does not create or override queue definitions dynamically for each trigger.

Do not confuse two controls

A worker may have eight local slots while storage permits only two active attempts in one user partition.

Before production

• Register the same queue definition in producers, workers, and maintenance.
• Choose a stable key with bounded cardinality.
• Test more worker slots than the durable limit.
• Test two users to confirm they progress independently.
• Monitor leases or reservations that keep a partition full.

Read limit concurrency for worker setup and verification.