Skip to content
Console
Menu

Getting Started

Authentication

KV Store

Queues

What a queue is, how a message moves through it, and where to go next.

A Queue holds messages until a worker has finished with them. Producers send messages; workers lease them, do the work, and ack each one. A message that is not acked comes back, and a message that keeps failing moves to a dead-letter set you can replay once the bug is fixed. Queues are part of Sylphx Events: there is nothing to deploy, one key in your project is enough, and your workers run wherever they already run.

#How a message moves

  1. Send. A message is a CloudEvent: id, source, type and a data payload. Up to 100 go in one call, and delay hides them for a while before the first lease.
  2. Lease. A worker leases up to 100 messages at a time. A leased message is hidden from every other worker for the queue's visibility_timeout (30 seconds by default, up to 12 hours). With wait, a lease on an empty queue waits up to 20 seconds for a message instead of answering empty at once.
  3. Ack or nack. An ack removes the message for good. A nack returns it at once, or after a delay you set. A lease that runs out without either also returns the message, and counts as a delivery.
  4. Dead-letter. After max_deliveries deliveries (5 by default, up to 100) the message leaves the ready set: into the queue's own dead-letter set, or into another queue named by dead_letter_queue.
  5. Replay. replay returns dead-lettered messages to ready, all of them or those dead-lettered in a time range, with a fresh delivery count.

#Leases are fenced

Every lease carries a lease_generation. When a lease runs out and another worker takes the message, the first worker's lease is stale: its ack or nack is not applied and comes back in stale_leases. A slow worker can therefore never ack away a message another worker is still processing. Delivery is at least once, so make the work idempotent on the message id.

#Routing events into a queue

A Subscription sends the environment's events whose type and source match into a queue, so a worker can consume events without the producer naming the queue.

#Names and ids

Queue
orgs/{org}/projects/{project}/envs/{env}/queues/{queue}
Queue id
que_<cell><ulid>
Message id
Returned by send; the same on every delivery

#The scopes

  • events:publish — sending messages.
  • events:write — creating, changing and deleting a queue, and leasing, acking, nacking and replaying its messages.
  • events:read — reading queues and their counts.

Keys and scopes covers creating a key with these scopes.