# Configuration reference (https://sockudo.io/docs/reference/configuration)
> Reference major Sockudo configuration sections and the deployment decisions they control.
Sockudo's TOML configuration is organized by runtime responsibility: one table per subsystem, plus
a handful of top-level keys for the listener and process. This page describes each section, what
it controls, and when you would change it.
Configuration resolves in layers, and each later layer wins:
1. compiled defaults, which apply to every key you omit
2. the configuration file (`config/config.toml`, `config/config.json`, or the path given with
`--config`), with `${VAR}` and `${VAR:-default}` references expanded from the environment
3. supported environment-variable overrides
4. validation, which rejects inconsistent settings at startup
Only explicitly mapped settings have an environment variable; nested policy such as app
definitions and channel namespaces lives in the file or the app manager. For loading details, JSON
equivalence, and guidance on what belongs in files, environment overrides, or secret stores, see
[Static configuration](/docs/deployment/static-configuration). The complete
[environment variable reference](/docs/reference/environment-variables) lists every runtime
variable parsed by the server and push subsystem.
Defaults on this page are the compiled defaults used when a key is omitted. The repository's
`config/config.toml` is a development profile that overrides several of them, for example
`mode = "development"`, `shutdown_grace_period = 2`, and `connection_recovery.enabled = true`.
Start production files from the defaults you need, not from the development profile.
## Top-level
Process-wide keys that sit outside any table. Change these to set the bind address, connection
limits, timeouts, and graceful-shutdown behavior.
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `host` | string | `"0.0.0.0"` | Bind host for WebSocket and HTTP API. Use a specific interface to avoid listening on every network. |
| `port` | integer | `6001` | Bind port for WebSocket and HTTP API. |
| `mode` | string | `"production"` | Runtime mode. `"production"` rejects node-local memory push drivers unless explicitly allowed and warns about risky paths such as a Unix socket under `/tmp`. Set `"development"` for local work. |
| `debug` | boolean | `false` | Enables verbose diagnostics for local development, including source locations in logs. |
| `max_connections` | integer | `0` | Per-node WebSocket connection limit. When exceeded the server returns close code 4100 so clients reconnect with backoff. `0` disables the limit. Set it to protect each node's memory and to drive `/ready` capacity gating. |
| `activity_timeout` | integer (seconds) | `120` | Pusher protocol activity timeout announced to clients in `connection_established`. |
| `user_authentication_timeout` | integer (seconds) | `3600` | How long a connection may stay open without completing user authentication when the app requires it. |
| `websocket_max_payload_kb` | integer (KiB) | `64` | Legacy maximum inbound WebSocket payload size. |
| `health_check_timeout_ms` | integer (ms) | `2000` | Timeout for each subsystem check in `/up` (adapter, cache, queue, app manager). |
| `shutdown_grace_period` | integer (seconds) | `10` | Seconds the listener stays open after SIGTERM/SIGINT while `/ready` reports 503. Raise it when load balancers need longer to drain a node. |
| `shutdown_close_delay_ms` | integer (ms) | `0` | Milliseconds to wait after the shutdown signal before closing existing WebSockets with 4200, so load balancers see the node draining first. |
| `shutdown_close_spread_ms` | integer (ms) | `0` | Milliseconds over which existing WebSockets are closed with 4200, evenly in small batches, so clients do not all reconnect at once. `0` closes all at once. Delay plus spread must not exceed `shutdown_grace_period`. |
| `server_role` | string | `"default"` | Deployment topology role. `"default"` runs the full server. `"api"` runs HTTP publish, history, mutable messages, and push — without WebSocket listeners, cluster heartbeats, or presence sync. |
| `path_prefix` | string | `"/"` | Parsed for compatibility but not currently applied to any route. Leave it at `"/"` and put a reverse proxy in front for path rewriting. |
## Section index
Every other setting lives in a table. Use this index to find the section that owns a behavior.
| Group | Sections | Controls |
| --- | --- | --- |
| Listener and transport | `[ssl]`, `[unix_socket]`, `[cors]`, `[http_api]`, `[http_api.readiness]`, `[http_api.accept_traffic]`, `[websocket]`, `[instance]` | TLS, Unix socket listener, browser CORS, HTTP body limit and operator endpoints, probes and admission, per-connection buffers and heartbeats, and node identity. |
| Apps and limits | `[app_manager]`, `[channel_limits]`, `[event_limits]`, `[presence]` | Where apps are stored, plus global channel, event, and presence limits that apps inherit. |
| Runtime backends | `[adapter]`, `[cache]`, `[queue]`, `[rate_limiter]`, `[database]`, `[database_pooling]`, `[cleanup]` | Cross-node fanout, shared cache, background work, rate limits, database connections, and disconnect cleanup. |
| Protocol features | `[connection_recovery]`, `[history]`, `[presence_history]`, `[versioned_messages]`, `[annotations]`, `[delta_compression]`, `[tag_filtering]`, `[idempotency]`, `[ephemeral]`, `[echo_control]`, `[event_name_filtering]` | Protocol V2 continuity, durable state, and delivery controls. |
| Integrations | `[webhooks]`, `[push]`, `[[push_rules]]`, `[ai_transport]`, `[mcp]`, `[ably_compat]` | Outbound webhooks, push notifications, AI Transport, the MCP server, and the Ably facade. |
| Observability | `[metrics]`, `[opentelemetry]`, `[logging]` | Prometheus, OTLP export, and log formatting. |
## Server role
`server_role` controls which subsystems the process starts. Use `"api"` to scale HTTP publishing
and history independently of WebSocket capacity.
```toml
server_role = "api"
[adapter]
driver = "redis-cluster"
fallback_to_local = false
```
API pods publish to the horizontal broadcast stream and serve HTTP endpoints
(publish, history, mutable messages, annotations, push). They skip WebSocket
listeners, adapter subscriptions, heartbeats, presence sync, and request/reply —
so they never appear in peer accounting and WS pods never await replies from them.
The server rejects startup when `server_role = "api"` is combined with
`adapter.driver = "local"` or `adapter.fallback_to_local = true`.
For endpoint availability details see
[HTTP endpoints — availability by server role](/docs/reference/http-endpoints#availability-by-server-role).
For capacity and operational metrics see
[Capacity planning — API pod topology](/docs/deployment/capacity-planning#api-pod-topology).
## Capacity readiness
Configure this when a load balancer should stop sending new connections to a full node.
`GET /ready` runs the same dependency checks as `/up` and also reports `503 CAPACITY`
when live native WebSocket connections reach the high watermark of `max_connections`.
It stays unavailable until occupancy reaches the low watermark, avoiding repeated
load-balancer changes near the limit. Disconnect cleanup backlog is excluded. Existing
connections stay open; this endpoint does not rebalance them.
```toml
max_connections = 10000
[http_api.readiness]
high_watermark = 0.95
low_watermark = 0.85
```
Watermarks must satisfy `0 <= low_watermark < high_watermark <= 1`. A zero
`max_connections` disables capacity gating. Use `/ready` for load-balancer admission,
`/up` for dependency health and rollout checks, and `/live` for process liveness.
Capacity does not change `/up`, so a full pod can still pass deployment health checks.
Standalone Redis adapter health probes reuse a dedicated reconnecting connection,
separate from command, event, and blocking queue connections. Probes still execute
`PING` and fail on dependency errors or timeouts; they do not open a new connection
on every probe. There is no adapter connection-pool size to tune.
## Memory-pressure connection admission
Memory-pressure admission control is opt-in under `[http_api.accept_traffic]`. Enable it when
nodes run under a container memory limit and you would rather refuse new connections than be
OOM-killed:
```toml
[http_api.accept_traffic]
enabled = true
memory_threshold = 0.90
# memory_limit_bytes = 2147483648 # optional; otherwise discover the cgroup limit
sample_interval_ms = 500
```
On Linux, Sockudo samples its process RSS on `sample_interval_ms`, independently of Prometheus
scrapes. It compares RSS with `memory_limit_bytes` when configured, otherwise with the finite cgroup
v2 `memory.max` or cgroup v1 `memory.limit_in_bytes` value. At or above `memory_threshold`, native
and Ably WebSocket upgrades return `503`, and the post-upgrade admission gate returns protocol close
code `4100` if pressure changes during the upgrade. Existing connections remain open.
The sampler fails open when RSS or a finite limit is unavailable, including on non-Linux systems.
`GET /accept-traffic` exposes the admission decision for a load balancer; `/up` deliberately remains
a dependency/readiness check and does not fail because of memory pressure. Use
`sockudo_memory_pressure_shedding` and `sockudo_memory_pressure_rejections_total` to alert on the
current state and rejected connection count.
## App manager
The app manager stores app credentials, limits, and policy. The default `memory` driver reads
apps from `[[app_manager.array.apps]]`; switch to a database driver when apps are created at
runtime or shared by several nodes.
| Section | Purpose |
| --- | --- |
| `[app_manager]` | Select app storage driver: `memory`, `mysql`, `pgsql` (aliases `postgres`, `postgresql`), `dynamodb`, `surrealdb` (alias `surreal`), or `scylladb` (alias `scylla`). `availability_check_interval_seconds` (default `15`, `0` disables) sets how often each node closes live connections of apps that were disabled (close code 4003) or deleted (4001). The `[app_manager.cache]` TTL adds to that delay. |
| `[app_manager.array]` | Static in-config app definitions. |
| `[[app_manager.array.apps]]` | App ID, key, secret, enabled flag, limits, and policy. |
## Channel namespaces
Namespaces apply per-prefix channel rules within an app: naming patterns, client permissions,
annotations, and history overrides. Configure namespace rules per app under `policy.channels.channel_namespaces`
(`[[app_manager.array.apps.policy.channels.channel_namespaces]]` in TOML).
For example, `private-chat:room-1` selects the namespace named `chat`: Sockudo
strips the channel type prefix and uses the text before the first `:`. The first
entry with that exact name wins. The regex validates that namespace's channel
names; it does not select a namespace. A channel containing `:` with no matching
namespace is rejected. Channels without `:` do not match a namespace.
The three `allow_*_for_client` permission flags apply to **Protocol V2** client
operations. **Unset or `null` means allowed**, preserving the existing default.
Set `allow_publish_for_client = false` explicitly for server-published namespaces.
These flags do not grant access by themselves: app-level client-message settings,
private/presence channel authentication, subscription requirements, and connection
capabilities still apply. They do not restrict authenticated server HTTP publishes
and do not replace Protocol V1's Pusher-compatible authorization rules.
| Field | Meaning and unset behavior |
| --- | --- |
| `name` | Required, non-empty namespace name, without `:`. Exact match after stripping a channel type prefix. |
| `channel_name_pattern` | Optional Rust regular expression applied to the logical channel name, including `name:`, without `private-`, `private-encrypted-`, or `presence-` and without a `#user1,user2` suffix. Unset adds no regex restriction; use `^` and `$` for a whole-name match. |
| `max_channel_name_length` | Positive limit in bytes for the complete wire channel name, including prefix and user suffix. Unset inherits `policy.limits.max_channel_name_length`, then `200`. |
| `allow_user_limited_channels` | Allows V2 `#user1,user2` channel syntax. Unset means **denied**. The authenticated user's identity must also satisfy the channel access check. |
| `allow_subscribe_for_client` | V2 client subscription permission. Unset means **allowed**; `false` rejects client subscriptions. |
| `allow_publish_for_client` | V2 client publish permission. Unset means **allowed**; `false` rejects client publishes. |
| `allow_presence_for_client` | V2 presence subscription permission. Unset means **allowed**; `false` rejects subscriptions to `presence-` channels in this namespace. `allow_subscribe_for_client = false` also denies presence subscriptions. |
| `annotations_enabled` | Inherits `policy.channels.annotations_enabled`, then `false`. Enabling annotations requires the global annotation feature and implies durable message history for the namespace. |
| `history` | Optional overrides: `rewind_enabled`, `retention_window_seconds`, `max_messages_per_channel`, `max_bytes_per_channel`. Each omitted field inherits app `policy.history`, then global `[history]`. There is no namespace `history.enabled` field. |
| `presence_history` | Optional overrides: `enabled`, `retention_window_seconds`, `max_events_per_channel`, `max_bytes_per_channel`. Each omitted field inherits app `policy.presence_history`, then global `[presence_history]`. |
An explicit read-only client policy for a server-published namespace:
```toml
[[app_manager.array.apps.policy.channels.channel_namespaces]]
name = "broadcast"
channel_name_pattern = "^broadcast:[A-Za-z0-9._-]+$"
max_channel_name_length = 200
allow_user_limited_channels = false
allow_subscribe_for_client = true
allow_publish_for_client = false
allow_presence_for_client = false
[app_manager.array.apps.policy.channels.channel_namespaces.history]
rewind_enabled = true
retention_window_seconds = 3600
max_messages_per_channel = 1000
max_bytes_per_channel = 10485760
```
The equivalent app JSON nests `channel_namespaces` inside `policy.channels`.
JSON does not support comments; the shipped JSON example spells out the permissive
client flags. Set its publish flag to `false` wherever clients must only receive
server-authored events. The dashboard exposes the same settings in the app's channel
namespace editor.
## Ably compatibility keys
Only relevant for binaries built with the `ably-compat` Cargo feature. `App.key` and `App.secret` are always the implicit primary credential. When the
server is built with `ably-compat`, `[ably_compat]` can opt an app into additional
Ably key names without creating sibling apps:
```toml
[ably_compat]
enabled = true
realtime_admission = "accept"
attach_timeout_ms = 10000
max_token_ttl_ms = 86400000
token_request_timestamp_skew_ms = 900000
nonce_ttl_seconds = 900
stats_fixture_ingest_enabled = false
stats_queue_capacity = 4096
stats_flush_interval_ms = 10
stats_retention_seconds = 34560000
stats_max_scan_entries = 100000
stats_cas_retries = 8
[[ably_compat.keys]]
app_id = "app-id"
key_name = "app-key-readonly"
secret = "replace-with-a-secret"
capability = '{"chat:*":["subscribe","history"]}'
revocable_tokens = true
enabled = true
rotation_id = "2026-07"
```
Each entry supports optional `created_at_ms`, `expires_at_ms`, and
`revoked_at_ms` rotation boundaries. Disabled, expired, or revoked keys reject
new key authentication. Tokens from a key with `revocable_tokens = true` are
also rejected after that key is disabled, revoked, expired, or replaced with a
different `rotation_id`. Secrets and bearer tokens are never written to logs.
TokenRequest `timestamp` and `ttl` values accept JSON integers or decimal
integer strings, matching values commonly forwarded from auth URL query
parameters. Timestamps are accepted only within the configured skew. Nonces are
claimed atomically in the configured cache for `nonce_ttl_seconds`, and issued
token records use the same cache, allowing issuance on one node and use on
another. Use a shared cache driver such as Redis in multi-node deployments.
The server's `ably-compat` Cargo feature explicitly enables native AI Transport,
recovery, delta, and push support. Runtime `[ably_compat].enabled` and the key
registry remain the opt-in boundary; Protocol V1 routes and frames are unchanged.
`realtime_admission = "placement_constraint"` is an operator/drain mode for a
listener: authenticated Ably WebSocket upgrades receive `DISCONNECTED` error
`50320`, prompting the SDK to try its configured fallback hosts over WebSocket.
It does not enable Comet, polling, streaming, SSE, or any other realtime
transport. The default is `accept`.
`attach_timeout_ms` bounds the complete native presence/recovery/history attach
path. On expiry Sockudo removes the partial subscription and returns
`DETACHED`/`50003`; the default is 10 seconds.
Ably realtime data queues use `[websocket].max_messages` and
`[websocket].max_bytes`. When a slow session exceeds either configured bound,
Sockudo marks its continuity lost and requires recovery instead of allowing the
session to buffer beyond the operator-configured limit. The shipped
configuration uses a 4 MiB byte bound so a healthy short burst of maximum-size
application events is not mistaken for a stalled reader.
Ably `delta=vcdiff` is negotiated per realtime channel and uses a fixed bounded
compatibility cache; native `[delta_compression]` settings continue to configure
only Sockudo's Pusher/V2 delta contract and do not select the Ably VCDIFF mode.
Compatibility statistics use canonical UTC minute buckets in the configured
cache and derive hour, day, and month results when queried. Sum counters and
peak gauges retain their distinct rollup semantics. In multi-node or
restart-safe deployments, use a shared persistent cache such as Redis; the
memory cache is node-local and does not survive a restart.
Redis-backed entries retain their configured TTL across an individual Sockudo
node's graceful shutdown; use the explicit cache administration APIs for
destructive prefix cleanup.
The recording worker is bounded by `stats_queue_capacity`, combines at most 256
observations per write batch, and waits up to `stats_flush_interval_ms` to form a
batch. REST and realtime publishes that receive an application ACK wait for the
canonical bucket merge first. Outbound delivery accounting uses the nonblocking
bounded path, so an abrupt process crash can lose observations that were still
queued; queue saturation is reported as a drop rather than allowing unbounded
memory growth. Retention, bounded query scans, and compare-and-swap retries are
controlled by `stats_retention_seconds`, `stats_max_scan_entries`, and
`stats_cas_retries`.
`stats_fixture_ingest_enabled` exposes authenticated `POST /stats` only for
conformance fixture provisioning and defaults to `false`. Fixture intervals are
validated and written through the same minute-store and rollup contract as live
observations; the HTTP handler does not return canned fixture values.
Compatibility device registrations and channel subscriptions use
`[push].storage_driver`, so clustered or restart-safe deployments should select
the same durable push store used by native push APIs.
## Runtime backends
These sections choose the infrastructure Sockudo coordinates through. A single node works with the
defaults; any multi-node deployment needs a shared adapter and cache.
| Section | Default driver | Purpose |
| --- | --- | --- |
| `[adapter]` | `local` | Cross-node fanout. Use `redis`, `redis-cluster`, `nats`, or another broker for more than one node. |
| `[cache]` | `memory` | Shared cache and coordination. Use `redis` or `redis-cluster` with any non-local adapter. |
| `[queue]` | `redis` | Webhook and push background work. The compiled default expects Redis; set `memory` explicitly for a single development node. |
| `[rate_limiter]` | `memory` | Request, connection, event, and push limits. Enabled by default; use a shared driver so limits apply across nodes. |
`adapter.driver = "local"` and `cache.driver = "memory"` keep all state inside one process. Running
several nodes with these defaults silently splits channels and presence between nodes. Choose a
shared adapter and cache before scaling out.
## Rate limiter client IP
Set this when Sockudo runs behind a proxy or load balancer, otherwise every client shares the
proxy's rate-limit bucket. `rate_limiter.api_rate_limit.trust_hops` and
`rate_limiter.websocket_rate_limit.trust_hops` select the client's **one-based position from the
right** of `X-Forwarded-For`, not a number of proxy machines. The Helm equivalents are
`config.rateLimiter.apiTrustHops` and `wsTrustHops`.
| Path directly in front of Sockudo | Header after the proxy appends its entries | Value |
| --- | --- | --- |
| NGINX using `$proxy_add_x_forwarded_for` | `, ` | `1` |
| Google external Application Load Balancer | `, , ` | `2` |
| No trusted proxy | Headers ignored; socket peer used | `0` (default) |
Only enable a positive value when network access to Sockudo is restricted to trusted proxies,
and each proxy sanitizes or appends the appropriate forwarding headers. Extra intermediaries
can change the required value. Choosing too large an offset can select client-supplied data;
hop counting does not authenticate the immediate peer.
With a positive value, Sockudo uses `X-Real-IP` only when `X-Forwarded-For` is absent. The trusted
proxy must overwrite `X-Real-IP`. A malformed, duplicated, or too-short `X-Forwarded-For` header
falls back directly to the socket peer, even if `X-Real-IP` is present. Invalid or duplicated
`X-Real-IP` also falls back. A warning is emitted once per extractor and contains only the
configured offset, entry count, and failure reason. All accepted IPs are normalized before
use as rate-limit keys.
**Migration:** `0` now ignores **both** forwarding headers. Earlier versions honored
`X-Real-IP` even at `0`, allowing directly connected clients to choose their rate-limit bucket.
Deployments relying on a trusted proxy's `X-Real-IP` must explicitly configure a positive value
for each enabled limiter and restrict direct access to the server. Without this configuration,
proxied clients share the socket peer's bucket.
## OMQ adapter
Choose OMQ when you want horizontal fanout without operating a broker. `adapter.driver = "omq"` enables brokerless horizontal fanout over OMQ PUB/SUB.
Each node binds one subscriber endpoint and connects its publisher to every
other node's subscriber endpoint.
```toml
[adapter]
driver = "omq"
fallback_to_local = false
[adapter.omq]
bind_endpoint = "tcp://0.0.0.0:5556"
connect_endpoints = [
"tcp://sockudo-1.internal:5556",
"tcp://sockudo-2.internal:5556",
]
prefix = "sockudo_adapter"
request_timeout_ms = 5000
nodes_number = 3
io_threads = 1
send_hwm = 100000
recv_hwm = 100000
```
OMQ uses transport topics derived from `prefix`: broadcast, request, response,
and per-node request topics. The adapter is not persistent; if a process or
network path is unavailable, messages in flight are not retained by a broker.
## Apache Iggy adapter and queue
`adapter.driver = "iggy"` and `queue.driver = "iggy"` use Apache Iggy (build with
the `iggy` Cargo feature). Sockudo ships the Iggy Rust SDK 0.11, which speaks
the Iggy 0.9 server wire protocol: run Iggy server 0.9 or newer. Older 0.8.x
servers are not wire-compatible, and Iggy 0.9 starts from a fresh data
directory. An Iggy 0.9 server that binds a wildcard address (`0.0.0.0`) also
needs `node.advertised_address` (`IGGY_NODE_ADVERTISED_ADDRESS`) set to the
address clients dial.
The same settings work for a single Iggy server and for a Viewstamped
Replication (VSR) cluster. Against a cluster, the SDK reads the cluster roster
after sign-in, follows the current leader, fails over to the surviving replicas
when its node disappears, routes consumer polls and offset commits to the
partition primary, and rejoins consumer groups after membership loss. No
Sockudo setting is needed for any of that.
```toml
[adapter]
driver = "iggy"
[adapter.iggy]
connection_string = "iggy://iggy:iggy@iggy-1.internal:8090"
# Extra replicas tried in order when the connection_string node is down at startup.
cluster_seeds = ["iggy-2.internal:8090", "iggy-3.internal:8090"]
connect_timeout_ms = 2000
failover_timeout_ms = 15000
consumer_name = ""
stream = "sockudo"
topic_prefix = "sockudo-adapter"
request_timeout_ms = 5000
poll_interval_ms = 5
poll_batch_size = 100
partitions_count = 1
partition_id = 0
auto_create = true
start_from_latest = true
durability = "replicated"
consumer_offset_durability = "replicated"
[queue]
driver = "iggy"
[queue.iggy]
connection_string = "iggy://iggy:iggy@iggy-1.internal:8090"
cluster_seeds = ["iggy-2.internal:8090", "iggy-3.internal:8090"]
connect_timeout_ms = 2000
failover_timeout_ms = 15000
stream = "sockudo"
queue_topic_prefix = "sockudo-queue"
consumer_group_prefix = "sockudo-workers"
partitions_count = 3
durability = "persisted"
consumer_offset_durability = "replicated"
```
| Key | Default | Purpose |
| --- | --- | --- |
| `connection_string` | `iggy://iggy:iggy@127.0.0.1:8090` | First node to dial. Its credentials and query options apply to every seed. |
| `cluster_seeds` | `[]` | Extra `host:port` replicas, tried in order when an earlier one cannot be reached. They only bootstrap the connection; afterwards the SDK fails over using the roster the cluster advertises. |
| `connect_timeout_ms` | `2000` | Bound on the TCP dial to each node while `cluster_seeds` is non-empty. A seed that refuses, cannot be resolved, or does not accept within this bound is skipped. Without seeds the SDK's own reconnection policy applies, as before. |
| `failover_timeout_ms` | `15000` | How long a broadcast or queue publish, and the sign-in to a seed that accepted the dial, wait for the cluster to elect a leader or fail over before giving up. Keep it above Iggy's `cluster.heartbeat_timeout` (5 s by default) plus the election. Cross-node requests and health checks keep `request_timeout_ms`. |
| `durability` | `replicated` | Message durability for topics Sockudo creates. `replicated` acknowledges after a VSR quorum commit; `persisted` also waits for recoverable stable-storage copies at that quorum. |
| `consumer_offset_durability` | `replicated` | Same choice for consumer offsets stored on those topics. |
Durability applies only to topics Sockudo creates (`auto_create = true`); it
does not change existing topics. Single-node servers accept both values.
The connection string's query options tune the SDK's failover:
`reconnection_retries` (passes over the known nodes, default `unlimited`),
`reconnection_interval` (default `1s`), `reestablish_after` (default `5s`), and
`heartbeat_interval` (default `5s`), plus `tls`, `tls_domain`, `tls_ca_file`,
and `nodelay`. For example,
`iggy://iggy:secret@iggy-1:8090?reconnection_interval=500ms&heartbeat_interval=2s`.
`nodes_number` is the number of **Sockudo** nodes the adapter waits on for
request/response fan-in, not the number of Iggy replicas.
A publish that runs out of time stops waiting, but its Iggy call keeps running
to completion in the background. The SDK reconnects and signs in inside
whichever call first meets a dead connection, and cancelling that call
mid-sign-in would leave the shared client permanently disconnected. As a result,
a publish reported as failed can still be delivered once the cluster recovers,
and a retry after a reconnect can deliver a message twice: Iggy deduplicates
retries only within one connection session. Protocol V2 clients drop duplicates
by `message_id`. At most 10,000 abandoned calls run at once per component;
beyond that, new calls fail immediately until the backlog drains.
The adapter and queue health checks ping the connected Iggy node. The SDK keeps
every client on the leader, so when the leader crashes, every Sockudo node's
`/up` can return 503 until the election finishes (6 to 9 seconds with Iggy's
defaults in testing). Size readiness probes to ride that out: Kubernetes'
defaults (`periodSeconds: 10`, `failureThreshold: 3`) already do. Each
component logs `cluster topology discovered` at startup with the cluster name,
roster size, and current leader.
`docker-compose.iggy-cluster.yml` runs a three-replica Iggy cluster with three
Sockudo nodes for local failover testing.
## Redis connections, Sentinel, and TLS
Shared by every Redis-backed component. Change it to point at your Redis deployment, enable TLS,
or use Sentinel for automatic failover. `[database.redis]` configures the Redis connection used by the Redis adapter, cache, queue, rate limiter, and delta coordinator. Without Sentinel, `[database.redis.master_tls]` secures the direct Redis data connection for every consumer. When `sentinels` is non-empty, Sockudo connects through Redis Sentinel using a native Sentinel client (rather than a direct URL) and can secure both connection hops independently. Redis Cluster also reuses `master_tls` certificate material; the legacy `cluster.use_tls` switch continues to enable TLS with the default root store.
`[database.redis.cluster]` is the canonical Redis Cluster topology for every cluster component — adapter, cache, rate limiter, delta coordinator, and queue. When `queue.driver = "redis-cluster"` and `queue.redis_cluster.nodes` is not explicitly set, the queue inherits the `database.redis.cluster` seeds (normalized with the shared cluster username/password and TLS settings). Set `queue.redis_cluster.nodes` only to override the queue's seeds independently of the rest of the deployment.
| Key | Purpose |
| --- | --- |
| `sentinels` | List of `{ host, port }` Sentinel nodes. A non-empty list enables Sentinel mode. |
| `name` | Monitored master (Sentinel service) name. |
| `username` / `password` | Auth for the master/replica data connection. |
| `sentinel_username` / `sentinel_password` | Auth for the Sentinel control connection. |
| `[database.redis.sentinel_tls]` | TLS for the client→Sentinel control connection. |
| `[database.redis.master_tls]` | TLS for direct Redis, Redis Cluster, or the Sentinel-resolved master/replica data connection. |
Each TLS block accepts `enabled`, `accept_invalid_certs` (skips verification; never enable it in production), `ca_path` (PEM CA for private CAs), and `client_cert_path` + `client_key_path` (PEM pair for mutual TLS / client-certificate auth).
For a direct Redis deployment signed by a private CA, no Sentinel configuration is required:
```toml
[database.redis]
host = "redis.internal"
port = 6380
[database.redis.master_tls]
enabled = true
ca_path = "/etc/sockudo/tls/redis-ca.pem"
```
```toml
[database.redis]
name = "mymaster"
db = 0
username = "appuser"
password = "app-secret"
sentinel_username = "sentineluser"
sentinel_password = "sentinel-secret"
[[database.redis.sentinels]]
host = "sentinel-1.internal"
port = 26379
[[database.redis.sentinels]]
host = "sentinel-2.internal"
port = 26379
[database.redis.sentinel_tls]
enabled = true
ca_path = "/etc/sockudo/tls/ca.pem"
client_cert_path = "/etc/sockudo/tls/client.pem"
client_key_path = "/etc/sockudo/tls/client.key"
[database.redis.master_tls]
enabled = true
ca_path = "/etc/sockudo/tls/ca.pem"
client_cert_path = "/etc/sockudo/tls/client.pem"
client_key_path = "/etc/sockudo/tls/client.key"
```
All Redis consumers inherit the native Sentinel topology when they do not define a URL override. An adapter, cache, queue, or rate-limiter URL override selects its standalone path; `master_tls` still applies to that direct data connection.
Each Sentinel-backed consumer polls Sentinel once per second with `SENTINEL GET-MASTER-ADDR-BY-NAME` and follows a failover without a restart: when Sentinel promotes a new primary, cached connections are dropped and rebuilt against it, and the adapter's Pub/Sub listener resubscribes. Recovery normally completes about a second after the promotion, so health probes such as `/up` recover on their own. If you restrict the Sentinel user with ACLs, allow that command alongside the ones used for primary resolution (`SENTINEL MASTERS` and `ROLE`). Primary changes are logged at `info` as `redis sentinel primary changed, reconnecting`.
## Protocol features
These sections enable and bound the Protocol V2 feature set. Most are off by default or bounded
conservatively; enable a feature here, then opt apps or channel namespaces into it through policy
where required. Protocol V1 delivery is unaffected by every section in this table.
| Section | Purpose |
| --- | --- |
| `[connection_recovery]` | V2 replay buffers and resume behavior: `enabled` (default `false`), `buffer_ttl_seconds` (default `120`), and `max_buffer_size` (default `100`), which bounds both the per-channel hot buffer and one durable fallback replay. |
| `[history]` | Durable channel history. |
| `[versioned_messages]` | V2 mutable-message storage, version paging, and retention. Requires the `versioned-messages` Cargo feature (also enabled by `ai-transport`); without it the server logs a warning and disables this section. |
| `[presence_history]` | Historical presence transitions and snapshots. |
| `[annotations]` | V2 annotation publish/delete/summary surfaces. Requires the `versioned-messages` Cargo feature (also enabled by `ai-transport`); without it the server logs a warning and disables this section. |
| `[delta_compression]` | Delta algorithms, per-socket state limits, full-message interval, and conflation behavior. Enabled by default with the `fossil` algorithm. |
| `[tag_filtering]` | V2 event/tag/JMESPath subscription predicates and tag projection controls. Predicate source, AST, tree, and projected-message memory are hard-bounded. |
| `[idempotency]` | HTTP publish deduplication: `enabled` (default `true`), `ttl_seconds` (default `120`), `max_key_length` (default `128`). |
| `[ephemeral]` | Allows `extras.ephemeral` messages that skip the recovery buffer and webhooks. Enabled by default. |
| `[echo_control]` | Lets V2 connections opt out of receiving their own publishes: `enabled` and `default_echo_messages` (both default `true`). |
| `[event_name_filtering]` | V2 subscription event-name filters: `enabled` (default `true`), `max_events_per_filter` (default `50`), `max_event_name_length` (default `200`). |
| `[webhooks]` | Webhook delivery, batching, and retry. |
| `[ai_transport]` | AI Transport validation and session-channel matching. |
| `[ai_transport.rollup]` | Append rollup egress coalescing and orphan tracking. |
| `[[ai_transport.channels]]` | Channel prefixes where AI Transport validation applies. |
| `[mcp]` | Embedded Model Context Protocol server: path or dedicated port, host/origin allow-lists, rate limits, tool hiding. Requires the `mcp` Cargo feature. |
| `[[mcp.tokens]]` | Bearer tokens with `read` / `write` / `admin` scopes and app allow-lists. See [MCP server](/docs/server/mcp). |
Annotations, including the optional Ably projection, require both global
`[annotations].enabled = true` and app/channel policy opt-in. Set
`annotations_enabled = true` under `[app_manager.array.apps.policy.channels]`
or on the matching channel namespace. Enabling the Ably facade does not bypass
either gate, and Protocol V1 never receives annotation frames.
`[versioned_messages].driver` also selects the annotation authority. `postgres`,
`mysql`, `dynamodb`, `scylladb`, and `surrealdb` each use their native atomic or
conditional commit primitives for annotation serials, stable create IDs,
canonical events, projections, replay, and retention. The corresponding Cargo
feature and database configuration must be present. `memory` is a single-node
development implementation; clustered startup rejects it.
For any non-local adapter, use `cache.driver = "redis"` or
`cache.driver = "redis-cluster"` for compatibility tokens, nonce replay,
revocations, session-owner leases, idempotency receipts, and retained stats.
Sockudo fails startup or the affected security operation when this coordination
authority is unavailable; it does not fall back to a private per-process cache.
## Push
Push requires the `push` Cargo feature plus a provider feature for each platform (`push-fcm`,
`push-apns`, `push-webpush`, `push-hms`, `push-wns`). Configure storage, queue, workers, and
per-provider switches in `[push]`. Provider credentials are stored through the
[push credential API](/docs/reference/http-endpoints#credentials-and-templates) or supplied to
workers through [provider environment variables](/docs/reference/environment-variables#push-providers);
there is no `[push.providers.*]` table.
| Section | Purpose |
| --- | --- |
| `[push]` | Storage and queue drivers, `fcm_enabled`, `apns_enabled`, `webpush_enabled`, `hms_enabled`, `wns_enabled`, optional `*_credential_ref` values, worker counts, fanout thresholds, backpressure, status retention, scheduler, repair, and cleanup. |
| `[push.apns]` | APNs topic, endpoint, Live Activities, broadcast channels, connection pooling, and extra CA roots. |
| `[push.retry]` | Provider delivery retry attempts, backoff, jitter, and `Retry-After` handling. |
| `[push.circuit_breaker]` | Provider circuit-breaker failure threshold, cooldown, and half-open concurrency. |
| `[push.default_quotas]` | Default per-app acceptance rate, daily delivery quota, fanout, and in-flight limits. |
| `[push.payload_redaction]` | Redaction of payloads, template data, and provider overrides in logs. |
| `[[push_rules]]` | Optional channel-publish to push-notification rules. |
## Capability tokens
Protocol V2 capability tokens do not currently have a TOML `[auth.capability_tokens]` switch. They
are accepted through the WebSocket `token` query parameter and refreshed with `sockudo:auth`.
Limits are compiled constants in `sockudo-core/src/capability_token.rs`: HS256 only, max token size
8 KiB, max `client_id` 128 bytes, max `jti` 128 bytes, max lifetime 24 hours, and 30 seconds clock
skew. Revocation uses signed HTTP `POST /apps/{appId}/revocations` and shared cache keys.
## AI Transport and Mutable Defaults
Runtime config uses `[versioned_messages]`; `VersionStore` is the Rust trait/storage abstraction.
There is no separate `[version_store]` TOML section in the current code.
The following block is machine checked by `sockudo-core` tests. Update it whenever code defaults
change.
```toml doc-defaults
[versioned_messages]
enabled = false
driver = "memory"
max_page_size = 100
retention_window_seconds = 0
purge_interval_seconds = 300
purge_batch_size = 1000
max_purge_per_tick = 100000
[history]
enabled = false
rewind_enabled = true
backend = "postgres"
retention_window_seconds = 86400
max_page_size = 100
writer_shards = 16
writer_queue_capacity = 4096
purge_interval_seconds = 300
purge_batch_size = 1000
max_purge_per_tick = 100000
[history.postgres]
table_prefix = "sockudo_history"
write_timeout_ms = 5000
[presence_history]
enabled = false
retention_window_seconds = 86400
max_page_size = 100
[annotations]
enabled = false
[ai_transport]
enabled = false
max_accumulated_message_bytes = 1048576
max_appends_per_message = 4096
max_transport_value_bytes = 256
max_steer_codec_message_ids_bytes = 2048
max_open_streaming_messages_per_channel = 1024
[ai_transport.rollup]
enabled = true
default_window_ms = 40
min_window_ms = 0
max_window_ms = 500
orphan_ttl_ms = 60000
wheel_tick_ms = 5
shards = 64
[push]
storage_driver = "memory"
queue_driver = "memory"
allow_memory_drivers = false
fcm_enabled = false
apns_enabled = false
webpush_enabled = false
hms_enabled = false
wns_enabled = false
accept_worker_count = 1
planner_worker_count = 1
shard_worker_count = 1
dispatch_worker_count = 1
dispatch_max_outbound_requests = 32
feedback_worker_count = 1
retry_worker_count = 1
queue_partition_count = 1
channel_shard_count = 1
fanout_fast_threshold = 10000
fanout_shard_size = 100000
fanout_sync_threshold = 0
backpressure_lag_threshold_secs = 60
publish_status_ttl_days = 30
stale_device_max_age_days = 90
dry_run = false
analytics_enabled = false
analytics_retention_days = 30
scheduler_interval_secs = 5
repair_interval_secs = 30
repair_min_age_secs = 30
repair_batch_size = 100
cleanup_interval_secs = 300
cleanup_batch_size = 1000
cleanup_max_deleted_per_tick = 100000
[push.apns]
topic = "com.example.app"
endpoint = "https://api.push.apple.com"
live_activities_enabled = false
live_activity_topic = "" # derives .push-type.liveactivity
bundle_id = "" # derives topic
broadcast_enabled = false
broadcast_endpoint = "https://api-broadcast.push.apple.com"
management_endpoint = "https://api-manage-broadcast.push.apple.com:2196"
default_broadcast_expiration_secs = 3600 # maximum 28800
connect_timeout_ms = 5000
request_timeout_ms = 10000
pool_idle_timeout_secs = 90
max_idle_connections_per_host = 128
tcp_keepalive_secs = 60
http2_keepalive_interval_secs = 30
http2_keepalive_timeout_secs = 10
ca_certificate_path = "" # optional extra PEM roots for private egress proxies or staging mocks
[push.retry]
max_attempts = 5
initial_backoff_ms = 1000
max_backoff_ms = 60000
max_elapsed_secs = 86400
jitter = true
jitter_ratio_percent = 20
respect_retry_after = true
[push.circuit_breaker]
failure_threshold = 5
cooldown_secs = 60
half_open_max_inflight = 10
[push.default_quotas]
acceptance_rps = 100
delivery_quota_daily = 0
fanout_max = 0
inflight_max = 1000
[push.payload_redaction]
redact_payload = true
redact_template_data = true
redact_provider_overrides = true
allow_debug_payload_logging = false
[[push_rules]]
enabled = true
channel_pattern = ""
event_filter = []
rate_limit_per_second = 100
[push_rules.payload_mapping]
title_field = "title"
body_field = "body"
template_data_field = "data"
include_remaining_fields = true
[mcp]
enabled = false
path = "/mcp"
# port = 6100
allowed_hosts = []
allowed_origins = []
allow_anonymous = false
anonymous_scopes = ["read"]
request_timeout_ms = 30000
max_body_bytes = 1048576
session_ttl_seconds = 1800
rate_limit_per_minute = 600
disabled_tools = []
# [[mcp.tokens]]
# name = "ops-agent"
# token = "${SOCKUDO_MCP_OPS_TOKEN}"
# scopes = ["read", "write"]
# apps = ["*"]
```
`max_messages_per_channel`, `max_bytes_per_channel`, `max_events_per_channel`, credential refs, and
external key refs default to unset and are omitted from the checked block.
`storage_driver = "memory"` and `queue_driver = "memory"` are node-local development drivers. In
`mode = "production"`, Sockudo rejects them unless `allow_memory_drivers = true` is set explicitly.
With `push.queue_driver = "memory"`, push uses a separate node-local queue and does not require
`queue.driver` to match. Broker-backed push queue work runs on the shared `[queue]` backend, so
`push.queue_driver` must name the same driver as `queue.driver`; Sockudo refuses to start when they
differ. Push needs per-message delays for scheduled sends and retries, so supported broker drivers
are `redis` (including Sentinel), `redis-cluster`, standard `sqs` and `pulsar`.
Sockudo refuses to start push on `nats`, `rabbitmq`, `kafka`, `iggy`,
`google-pubsub` and SQS FIFO queues (`queue.sqs.fifo = true`), which have no per-message delay.
SQS caps delays at 15 minutes; longer push delays are re-deferred when the job arrives.
Push publish admission also requires a healthy queue, a safe store, local pipeline workers, and
local provider-worker capability. Raw provider-specific recipients require the matching provider
worker before the request is accepted; shard-path publishes also require a local shard worker.
`backpressure_lag_threshold_secs` limits the oldest actionable ready or inflight push queue item
age; set it to `0` to rely only on queue-depth thresholds.
Push repair runs in monolith workers when `repair_interval_secs > 0`; it scans stale queued
durable publish logs and recreates missing `push.publish.v1` queue work after the publish has been
queued for at least `repair_min_age_secs`.
Push cleanup runs in monolith workers when `cleanup_interval_secs > 0`. It purges terminal publish
statuses older than `publish_status_ttl_days`, delivery events and operator invalidations older
than `analytics_retention_days`, expired idempotency records, expired scheduler locks, and any
persisted dead-letter inspection rows written by a selected backend. Queue-native dead-letter
metadata is inspected through `/apps/{appId}/push/deadLetters`. Cleanup is bounded by
`cleanup_batch_size` per category and `cleanup_max_deleted_per_tick` overall.
## Observability
Metrics are on by default; tracing export is opt-in. Change these sections to rename metrics, move
the scrape listener, or ship telemetry to an OpenTelemetry collector.
| Section | Purpose |
| --- | --- |
| `[metrics]` | Enables metrics and configures the Prometheus scrape endpoint. Defaults: `enabled = true`, `host = "0.0.0.0"`, `port = 9601`. |
| `[metrics.prometheus]` | Prometheus metric naming options. |
| `[metrics.tcp_exporter]` | Optional metrics-rs TCP event exporter for live clients and sidecars. |
| `[opentelemetry]` | Optional OTLP traces, metrics, logs, resource identity, batching, and propagation. |
| `[logging]` | Controls ANSI colors and tracing-target inclusion for text/JSON output. |
| [logging environment](/docs/reference/logging) | Runtime filters, structured output, lifecycle fields, and data-safety policy. |
OpenTelemetry is disabled by default and is additive to Prometheus and local logs:
```toml
[opentelemetry]
enabled = true
traces_enabled = true
metrics_enabled = true
logs_enabled = true
service_name = "sockudo"
service_namespace = "realtime"
deployment_environment = "production"
resource_attributes = { "cloud.region" = "eu-central-1" }
endpoint = "http://otel-collector:4317"
export_timeout_ms = 10000
batch_scheduled_delay_ms = 5000
batch_max_queue_size = 2048
batch_max_export_batch_size = 512
metric_export_interval_ms = 60000
propagation_trace_context = true
propagation_baggage = true
```
| Key | Default | Purpose |
| --- | --- | --- |
| `enabled` | `false` | Initializes OpenTelemetry and enables the selected signals. |
| `traces_enabled` | `true` | Exports spans when OpenTelemetry is enabled. |
| `metrics_enabled` | `true` | Exports Sockudo metrics through OTLP independently of the Prometheus listener. |
| `logs_enabled` | `true` | Exports structured log records while retaining local output. |
| `service_name` | `sockudo` | OpenTelemetry `service.name`. Must not be empty when enabled. |
| `service_namespace` | unset | Optional `service.namespace`. |
| `deployment_environment` | unset | Optional `deployment.environment.name`. |
| `resource_attributes` | `{}` | Additional string-valued resource attributes. Never store secrets here. |
| `endpoint` | unset | Optional common OTLP endpoint. When unset, standard SDK/exporter configuration applies. |
| `export_timeout_ms` | `10000` | Timeout for one exporter request. |
| `batch_scheduled_delay_ms` | `5000` | Maximum span/log batching delay. |
| `batch_max_queue_size` | `2048` | Bounded batch processor queue capacity. |
| `batch_max_export_batch_size` | `512` | Maximum export batch size; cannot exceed the queue capacity. |
| `metric_export_interval_ms` | `60000` | Periodic metric export interval. |
| `propagation_trace_context` | `true` | Extracts and injects W3C `traceparent` and `tracestate`. |
| `propagation_baggage` | `true` | Extracts and injects W3C baggage. Do not place secrets in baggage. |
Use standard `OTEL_*` environment variables for the OTLP protocol (`grpc`, `http/protobuf`, or
`http/json`), signal-specific endpoints, headers, compression, sampling, and SDK limits.
The complete mapping is in the
[environment variable reference](/docs/reference/environment-variables#opentelemetry).
HTTPS exporters use platform/web PKI roots; custom CA bundles and mTLS exporter configuration are
not supported by the current Rust SDK integration.
The shipped CORS configuration allows `traceparent`, `tracestate`, and `baggage`; retain those
headers in a custom allowlist when browser clients participate in distributed traces.
Collector failures are fail-open: export uses bounded queues and does not change request handling,
`/live`, or `/up`. Invalid static settings are still rejected during startup. Stable traces,
metrics, and logs are supported; OpenTelemetry profiles are not.
With the defaults, `/metrics` is served on every interface at port `9601` without authentication.
Bind `[metrics].host` to an internal interface or restrict the port with network policy.
## Memory cache expiration
The `memory` cache driver honors the TTL supplied by each write. `cache.memory.ttl`
(default `300` seconds) is both the default for writes with a zero TTL and the
maximum for writes with a longer TTL. Set it to `0` to remove that maximum; a
write with a zero TTL then creates a nonexpiring entry. Capacity eviction and
process restarts can still remove entries.
Overwriting an entry resets its expiry. Incrementing an existing counter with a
zero TTL preserves its current expiry. Reading an entry does not extend its TTL.
The same rules apply to the memory fallback used during a Redis outage. Recovery
skips entries that expired while synchronization was in progress and copies live
entries with their remaining TTL, rounded up to whole seconds for Redis.
## Example production skeleton
A minimal multi-node starting point: shared Redis for fanout, cache, and queue, with metrics on
the default port and durable push storage.
```toml title="config.toml"
host = "0.0.0.0"
port = 6001
debug = false
[adapter]
driver = "redis"
[cache]
driver = "redis"
[queue]
driver = "redis"
[metrics]
enabled = true
port = 9601
[metrics.tcp_exporter]
enabled = false
host = "127.0.0.1"
port = 5000
buffer_size = 1024
[push]
storage_driver = "postgres"
queue_driver = "redis"
publish_status_ttl_days = 30
analytics_retention_days = 30
cleanup_interval_secs = 300
```
## Queue v2 reliability
Tune `[queue.reliability]` to trade redelivery window against throughput. Queue v2 provides at-least-once delivery. Successful processors acknowledge a
job; errors retry with bounded exponential backoff and jitter; exhausted jobs
move to a dead-letter queue. Exactly-once execution is not promised, so handlers
must remain idempotent. Redis, Redis Sentinel, and Redis Cluster additionally
provide stable job IDs, deduplication, delayed jobs, renewable leases, stalled
worker recovery, lifecycle events, and queue depth statistics through atomic Lua
transitions.
Redis workers preserve the configured callback concurrency while pipelining
Redis I/O. `worker_prefetch` bounds the number of leased jobs buffered ahead of
each callback worker; the default `16` amortizes claim, acknowledgement, and
lease-renewal round trips without creating unbounded memory or failover
exposure. Lower it for a smaller redelivery window, or raise it when Redis
latency leaves callback workers idle. `queue.redis.response_timeout_ms`
defaults to `5000`, allowing Sentinel clients to discard a stale primary
connection promptly; `0` disables that deadline.
For the legacy-equivalent high-throughput workload—batch enqueue, generated
job IDs, immediate delivery, the configured default attempt count, no
application deduplication key, and `event_retention = 0`—the Redis drivers use
compact length-delimited frames and one raw atomic Redis transaction. Workers
materialize each frame into the same leased Queue v2 state machine when they
claim it, so retries, acknowledgements, stalled recovery, and dead lettering
are unchanged while admission avoids per-job Redis writes. Supplying a stable
`job_id`, delay, deduplication key, attempt override, or lifecycle-event
retention selects the stricter per-job atomic path. Use that path when a caller
must safely retry an enqueue whose network result was ambiguous.
```toml
[queue]
driver = "redis" # memory, redis, redis-cluster, nats, rabbitmq, kafka, iggy, pulsar, google-pubsub, sqs, sns
[queue.reliability]
max_attempts = 5
retry_base_delay_ms = 1000
retry_max_delay_ms = 60000
retry_jitter = 0.2
lease_duration_ms = 30000
lease_renew_interval_ms = 10000
stalled_batch_size = 100
worker_poll_interval_ms = 500
worker_prefetch = 16
shutdown_timeout_ms = 30000
completed_retention = 1000
failed_retention = 10000
event_retention = 10000
deduplication_ttl_ms = 300000
memory_capacity = 100000
max_batch_size = 1000
```
`shutdown_timeout_ms` bounds the lifecycle/webhook drain after the listener grace period and
also the subsequent queue worker shutdown. These are separate budgets; reserve time for both
in your process supervisor. See [shutdown delivery](/docs/server/webhooks#shutdown-delivery).
Broker-backed queues use native bulk/pipelined publishing rather than the
compatibility loop: NATS and RabbitMQ pipeline acknowledgements, Kafka feeds
librdkafka concurrently, Iggy and Pulsar use their producer batch APIs, Google
Pub/Sub reuses its batching publisher, and SQS/SNS use provider batches of at
most 10 entries. `max_batch_size` is an upper bound; a provider's smaller hard
limit still applies. Topic, stream, queue, and publisher handles are cached
after successful provisioning instead of issuing control-plane requests for
every job.
`worker_prefetch` also bounds concurrent callbacks for NATS, RabbitMQ, Pulsar,
Google Pub/Sub, and standard SQS queues. SQS FIFO batches remain serial to
preserve message-group order. `max_attempts` configures NATS delivery limits,
Kafka and Iggy retry/DLQ transitions, RabbitMQ republish/DLQ transitions, and
Pulsar's native dead-letter policy. SQS dead-lettering uses an operator-managed
redrive policy; Sockudo never creates an SQS DLQ. Before consuming the
`webhooks` queue, Sockudo reads its `RedrivePolicy`. If one exists,
`maxReceiveCount` bounds delivery and the queue reports `dead_letter`
capability. If none exists, Sockudo logs a warning at startup and deletes a
failing job after `queue.reliability.max_attempts` receives. Set
`queue.sqs.require_redrive_policy = true`
(`QUEUE_SQS_REQUIRE_REDRIVE_POLICY=true`) to fail startup instead. Push stage
queues are not validated.
Terraform example (DLQ retention must exceed the source queue's: for standard
queues, SQS keeps the original enqueue timestamp when it moves a message, so a
shorter DLQ retention can expire it immediately):
```hcl
resource "aws_sqs_queue" "webhooks_dlq" {
name = "sockudo-webhooks-dlq"
message_retention_seconds = 1209600 # 14 days
}
resource "aws_sqs_queue" "webhooks" {
name = "sockudo-webhooks"
message_retention_seconds = 345600 # 4 days
redrive_policy = jsonencode({
deadLetterTargetArn = aws_sqs_queue.webhooks_dlq.arn
maxReceiveCount = 3
})
}
```
FIFO note: with FIFO, the DLQ must also be FIFO, and one failing job blocks its
message group (the whole queue with the default single `message_group_id`) until
redrive or the fallback delete. Prefer a standard queue for webhooks.
Queue v2 options are fail-closed. A broker that cannot honor per-job delay,
deduplication, or attempt overrides returns an error instead of silently
discarding the option. SQS standard queues support delays up to 15 minutes;
SQS FIFO queues support stable IDs/deduplication but not per-message delay.
Pulsar supports per-job delay through native delayed delivery (`deliverAt`) on the
Shared subscription Sockudo's workers use; the broker's `delayedDeliveryEnabled`
must stay on (the default). NATS, RabbitMQ, Kafka, Iggy and Google Pub/Sub reject
per-job delay.
The Redis drivers use a new `:v2:{queue-id}:*` key namespace. All keys for one
logical queue share a Redis Cluster hash tag, so enqueue, claim, ack, retry,
lease renewal, delayed promotion, and dead-letter transitions are single-slot
and atomic. Existing RPUSH/BLPOP list keys are not deleted during startup or
shutdown; drain them with the previous release before switching producers.
When `database.redis.sentinels` is configured and `queue.driver = "redis"`, the
queue resolves the current primary through native Sentinel support. Sentinel and
primary authentication/TLS settings remain independent. A `queue.redis.url_override`
selects standalone Redis/rediss instead and intentionally disables inherited
Sentinel discovery for that queue.
When `queue.driver = "redis-cluster"`, the queue resolves its cluster seeds in
this order: explicit `queue.redis_cluster.nodes` (normalized with the shared
cluster auth/TLS options), then `database.redis.cluster` when the queue nodes are
unset, then a localhost dev fallback. Leaving `queue.redis_cluster.nodes` empty
is the recommended way to share one Redis Cluster topology across the adapter,
cache, rate limiter, delta coordinator, and queue.
Primary promotion preserves at-least-once, not exactly-once, semantics. A hard
primary loss can redeliver callbacks that were active or prefetched when the node failed; a
planned Sentinel switchover can temporarily expose a larger duplicate window
while the old primary is still reachable. Use the stable job ID as an
idempotency key, and quiesce producers/workers before a planned switchover when
duplicate callbacks must be minimized.
SNS is producer-only. Configuring a consumer on SNS now fails instead of
reporting a successful no-op. A requested backend that was not compiled into the
binary also fails startup instead of silently falling back to memory.
## Next steps
- [Environment variables](/docs/reference/environment-variables): Override these settings per environment and inject secrets.
- [Static configuration](/docs/deployment/static-configuration): Loading order, TOML versus JSON, and secret handling.
- [Scaling](/docs/server/scaling): Choose adapters, caches, and queues for horizontal fanout.
- [Security](/docs/server/security): Harden auth, origins, rate limits, and operator endpoints.