# Sockudo Documentation (/docs)



Sockudo is a self-hosted realtime platform for teams that want Pusher-compatible APIs with deeper control over protocol evolution, horizontal scaling, history, recovery, observability, and SDK behavior.

<section className="docs-control-center not-prose" aria-labelledby="docs-hub-heading">
  <div className="docs-hub-panel">
    <div className="docs-hub-top">
      <div className="docs-hub-copy">
        <span className="docs-hub-badge">
          Production realtime docs
        </span>

        <h2 id="docs-hub-heading">
          Ship Pusher-compatible realtime now. Add durability when you need it.
        </h2>

        <p>
          {"Sockudo keeps the Pusher-shaped WebSocket and HTTP API surface familiar while giving teams a native Protocol V2 path for recovery, history, mutable messages, annotations, push workflows, and horizontal fanout."}
        </p>

        <a className="docs-hub-cta" href="/docs/getting-started/installation">
          {"Start the production path "}

          <span aria-hidden="true">
            ->
          </span>
        </a>
      </div>

      <div className="docs-hub-narrative" aria-label="Sockudo documentation scope">
        <p>
          <strong>Keep existing clients moving.</strong>

          {" Protocol V1 preserves pusher-js, Laravel Echo, Pusher auth, and familiar server publish flows."}
        </p>

        <p>
          <strong>Add native capabilities deliberately.</strong>

          {" Protocol V2 adds serials, message IDs, rewind, deltas, tags, annotations, mutable messages, and recovery metadata without leaking into V1 delivery."}
        </p>

        <p>
          <strong>Operate it like infrastructure.</strong>

          {" Server docs cover adapters, app managers, queues, caches, metrics, rate limits, webhooks, push, and cluster failure paths."}
        </p>
      </div>
    </div>

    <dl className="docs-hub-stats" aria-label="Sockudo documentation highlights">
      <div>
        <dt>
          Drop-in surface
        </dt>

        <dd>
          Protocol V1
        </dd>
      </div>

      <div>
        <dt>
          Native layer
        </dt>

        <dd>
          Protocol V2
        </dd>
      </div>

      <div>
        <dt>
          Runtime
        </dt>

        <dd>
          Rust + Tokio
        </dd>
      </div>

      <div>
        <dt>
          Operations
        </dt>

        <dd>
          Cluster ready
        </dd>
      </div>
    </dl>
  </div>
</section>

<Cards>
  <Card title="Install and verify" href="/docs/getting-started/installation" description="Run Sockudo locally, choose features, inspect health probes, and understand the default development app." />

  <Card title="Connect the first loop" href="/docs/getting-started/first-connection" description="Subscribe from a client, publish from a trusted backend, add idempotency, and confirm metrics move." />

  <Card title="Migrate from Pusher" href="/docs/getting-started/migration" description="Map pusher-js, Laravel Echo, Channels auth, event names, and publish calls onto Sockudo." />

  <Card title="Design Protocol V2 streams" href="/docs/clients/protocol-v2" description="Use message IDs, serial continuity, rewind, deltas, tag filters, annotations, and recovery." />

  <Card title="Deploy to production" href="/docs/deployment" description="Choose a topology, configure files and secrets, tune hosts, and deploy on VMs, containers, Kubernetes, or cloud platforms." />

  <Card title="Reference the API" href="/docs/reference/http-endpoints" description="Find every server endpoint for publishing, channel state, history, mutations, annotations, push, and operations." />
</Cards>

## Production topology [#production-topology]

Sockudo sits between realtime clients and trusted backends. Clients hold WebSocket connections and subscribe to public, private, presence, or encrypted channels. Backends use the HTTP API or server SDKs to publish events, sign channel auth, mutate V2 messages, manage push devices, and inspect operational state.

<div className="docs-topology-showcase not-prose">
  <div className="docs-topology-copy">
    <span>
      Runtime shape
    </span>

    <h3>
      One protocol edge, multiple durability choices.
    </h3>

    <p>
      {"Start with the in-memory local profile, then move app records, caches, queues, adapters, history, and push status into production backends as the deployment grows."}
    </p>

    <ul>
      <li>
        <strong>WebSocket ingress:</strong>

        {" client connect, subscribe, auth, presence, and realtime delivery."}
      </li>

      <li>
        <strong>HTTP control plane:</strong>

        {" trusted publishes, channel state, history reads, message mutations, annotations, and push workflows."}
      </li>

      <li>
        <strong>Horizontal fanout:</strong>

        {" Redis, NATS, Kafka, RabbitMQ, Pulsar, Google Pub/Sub, Iggy, or another configured adapter."}
      </li>
    </ul>
  </div>

  <figure>
    <img src="/diagrams/architecture.svg" alt="Sockudo architecture with clients, the Rust server, and backend publish surfaces." />

    <figcaption>
      {"Protocol V1 compatibility and Protocol V2 durability run through the same server edge."}
    </figcaption>
  </figure>
</div>

## Recommended path [#recommended-path]

1. Run a local Sockudo server from [Installation](/docs/getting-started/installation).
2. Publish and receive your first event from [First connection](/docs/getting-started/first-connection).
3. Add private and presence channel auth from [Authentication](/docs/getting-started/authentication).
4. Choose Protocol V1 or V2 from [Compatibility](/docs/reference/compatibility).
5. Wire your client from [Realtime Clients](/docs/clients) and your backend from [Server SDKs](/docs/server-sdks).
6. Follow the [Deployment guide](/docs/deployment), then harden the runtime with [Scaling](/docs/server/scaling), [Security](/docs/server/security), and [Observability](/docs/server/observability).

## Developer quick loops [#developer-quick-loops]

The examples use the default development app. Keep these values local only:

```bash
export SOCKUDO_APP_ID=app-id
export SOCKUDO_APP_KEY=app-key
export SOCKUDO_APP_SECRET=app-secret
export SOCKUDO_HOST=127.0.0.1
export SOCKUDO_PORT=6001
```

### Run Sockudo and check health [#run-sockudo-and-check-health]

```bash
docker compose up sockudo redis
curl -f http://127.0.0.1:6001/up
curl -f http://127.0.0.1:9601/metrics | head
```

Use Docker Compose when you want Redis and repeatable local dependencies. Use `cargo run --release` when you want to validate a source build or a specific feature set.

### Subscribe from a client [#subscribe-from-a-client]

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  enabledTransports: ["ws"],
  protocolVersion: 2,
});

client.subscribe("orders").bind("order.created", (payload) => {
  console.log("received", payload);
});
```

Set `protocolVersion: 2` when the client needs Sockudo-native recovery, rewind, message IDs, tags, deltas, annotations, or mutable messages. Leave existing Pusher-compatible clients on Protocol V1 until you are ready to adopt V2 behavior.

### Publish from a trusted backend [#publish-from-a-trusted-backend]

```ts
import { Sockudo } from "sockudo";

const sockudo = new Sockudo({
  appId: "app-id",
  key: "app-key",
  secret: "app-secret",
  host: "127.0.0.1",
  port: 6001,
  useTLS: false,
});

await sockudo.trigger(
  "orders",
  "order.created",
  { id: "ord_123", total: 4200 },
  { idempotency_key: "order-created-ord_123" },
);
```

Server SDKs sign requests, format payloads, and hide HMAC details. Use raw HTTP only when you are implementing or debugging an SDK.

### Turn local config into a production shape [#turn-local-config-into-a-production-shape]

```toml
port = 6001
host = "0.0.0.0"
debug = false

[app_manager]
driver = "postgres"

[adapter]
driver = "redis"

[cache]
driver = "redis"

[queue]
driver = "redis"

[metrics]
enabled = true
port = 9601
```

The important move is not a single backend choice. It is separating connection fanout, app records, cache state, queue work, durable history, push status, and metrics so each subsystem can fail, scale, and be observed independently.

## Local baseline [#local-baseline]

The examples use the default development app:

```toml
port = 6001
host = "0.0.0.0"

[app_manager]
driver = "memory"

[[app_manager.array.apps]]
id = "app-id"
key = "app-key"
secret = "app-secret"
enabled = true
```

For production, replace every example credential, enable TLS at the edge, use managed app storage, and keep app secrets only on trusted servers.

## Production readiness checklist [#production-readiness-checklist]

* **Compatibility:** decide which clients stay on Protocol V1 and which channels may use Protocol V2 features.
* **Authentication:** keep app secrets server-side, sign private and presence auth responses, and reject stale HTTP signatures.
* **Fanout:** choose an adapter, configure sticky sessions when required, and test cross-node broadcast delivery before launch.
* **Durability:** enable history, recovery, version storage, annotations, and push status only for channels that need those contracts.
* **Idempotency:** attach stable keys to retried publishes, message mutations, push requests, and backend workflows.
* **Observability:** scrape `/metrics`, alert on connection churn, publish failures, adapter errors, webhook retries, push provider outcomes, and recovery failures.
* **Operations:** document feature flags, runtime config, secret rotation, rollout order, and rollback behavior for every backend dependency.


# .NET realtime (/docs/clients/dotnet)



`Sockudo.Client` is the official .NET realtime SDK. It defaults to Protocol V2 and supports V1 compatibility, public, private, presence, encrypted channels, recovery, filters, MessagePack, Protobuf, and delta reconstruction.

The current monorepo package targets .NET 10. Choose
[`SockudoServer`](/docs/server-sdks/dotnet) instead when the process needs app
credentials to publish or authorize clients over HTTP.

## Install [#install]

Install the published NuGet package:

```bash
dotnet add package Sockudo.Client --version 2.2.0
```

Or add it directly to your project file:

```xml
<PackageReference Include="Sockudo.Client" Version="2.2.0" />
```

## Connect [#connect]

```csharp
using Sockudo.Client;

var client = new SockudoClient(
    "app-key",
    new SockudoOptions
    {
        Cluster = "local",
        ForceTls = false,
        WsHost = "127.0.0.1",
        WsPort = 6001,
        ProtocolVersion = 2,
        ConnectionRecovery = true,
    }
);

var channel = client.Subscribe("public-updates");
channel.Bind("price-updated", (data, meta) => Console.WriteLine(data));

await client.ConnectAsync();
```

For production, use the public ingress or load-balancer hostname with
`ForceTls = true`. Reuse one client for the application lifetime rather than
opening a WebSocket per operation.

## Connection lifecycle [#connection-lifecycle]

```csharp
var stateToken = client.Bind("state_change", (value, _) =>
{
    var change = (StateChange)value!;
    Console.WriteLine($"connection: {change.Previous} -> {change.Current}");
});
client.Bind("connected", (_, _) =>
    Console.WriteLine($"socket id: {client.SocketId}"));
client.Bind("connecting", (_, _) =>
    Console.WriteLine("connecting"));
client.Bind("reconnecting", (_, _) =>
    Console.WriteLine("reconnecting"));
client.Bind("error", (error, _) =>
    Console.Error.WriteLine(error));

await client.ConnectAsync();
```

A connected socket can still have a pending or rejected channel. Wait for the
subscription-success event before treating channel data as live. Clean up
bindings and subscriptions with `Unbind`, `UnsubscribeAsync`, and
`DisconnectAsync`:

```csharp
channel.Unbind("price-updated");
client.Unbind("state_change", stateToken);
await client.UnsubscribeAsync("public-updates");
await client.DisconnectAsync();
```

Unexpected disconnects use quadratic backoff (0s, 1s, 4s, 9s) capped at 120
seconds and emit `ConnectionState.Reconnecting`. Set `MaxReconnectAttempts`
(default `6`, `null` for unlimited) and `MaxReconnectGapInSeconds` in
`SockudoOptions`. Protocol retry and TLS-upgrade close codes reconnect
immediately.

## Auth [#auth]

```csharp
var client = new SockudoClient(
    "app-key",
    new SockudoOptions
    {
        Cluster = "local",
        WsHost = "127.0.0.1",
        WsPort = 6001,
        ChannelAuthorization = new ChannelAuthorizationOptions(
            Endpoint: "https://api.example.com/sockudo/auth"
        ),
    }
);
```

The endpoint authenticates the caller, authorizes the exact requested channel,
and signs with the app secret. Derive presence identity from the backend
session; never trust a user ID supplied by the client.

Protocol V2 can use scoped capability tokens with refresh:

```csharp
var client = new SockudoClient(
    "app-key",
    new SockudoOptions(
        Cluster: "local",
        WsHost: "realtime.example.com",
        ForceTls: true,
        TokenAuthentication: new TokenAuthenticationOptions(
            TokenProvider: cancellationToken =>
                FetchSockudoTokenAsync(cancellationToken)
        )
    )
);
```

JWT expiry metadata enables proactive refresh. Token expiration and revocation
are also exposed as typed errors.
Capability-token options require Protocol V2; V1 configuration fails during
client construction rather than opening an unauthenticated socket.

## Presence [#presence]

```csharp
var channel = client.Subscribe("presence-lobby");

channel.Bind("sockudo:subscription_succeeded", (data, meta) =>
    Console.WriteLine($"members: {data}"));
channel.Bind("sockudo:member_added", (data, meta) =>
    Console.WriteLine($"joined: {data}"));
channel.Bind("sockudo:member_removed", (data, meta) =>
    Console.WriteLine($"left: {data}"));
```

Protocol V2 presence data can change without a leave/rejoin:

```csharp
var presence = (PresenceChannel)channel;
presence.Bind("sockudo:presence_update", (member, _) =>
    Console.WriteLine(member));

await presence.UpdateAsync(new Dictionary<string, object?>
{
    ["status"] = "editing",
});
```

## Filters [#filters]

```csharp
var channel = client.Subscribe(
    "price:btc",
    new SubscriptionOptions(
        Filter: Filter.And(
            Filter.Eq("market", "spot"),
            Filter.Gt("spread", "0")
        ),
        Events: new[] { "price.updated" },
        Expression: "data.price >= `100`"
    )
);
```

Combine filters with delta compression and bounded rewind for high-volume
state feeds:

```csharp
var orderBook = client.Subscribe(
    "orderbook:btc-usd",
    new SubscriptionOptions(
        Delta: new ChannelDeltaSettings(
            Enabled: true,
            Algorithm: DeltaAlgorithm.Xdelta3
        ),
        Rewind: new SubscriptionRewind.Seconds(30)
    )
);

client.Bind("sockudo:resume_failed", (data, _) =>
    Console.WriteLine($"reload authoritative state: {data}"));
```

On a failed resume or missing delta base, discard derived state and load a
fresh snapshot.

## Presence history proxy [#presence-history-proxy]

```csharp
var client = new SockudoClient(
    "app-key",
    new SockudoOptions(
        Cluster: "local",
        WsHost: "127.0.0.1",
        WsPort: 6001,
        PresenceHistory: new PresenceHistoryOptions(
            Endpoint: "https://api.example.com/sockudo/presence-history"
        )
    )
);

var channel = (PresenceChannel)client.Subscribe("presence-lobby");
var page = await channel.HistoryAsync(
    new PresenceHistoryParams(Limit: 50, Direction: "newest_first")
);
```

Proxy endpoints keep the app secret server-side. They must authenticate the
caller, check channel access, bound page sizes, and forward opaque cursors.
Channel history with `UntilAttach: true` provides a gap-free late join.

## Versioned and encrypted messages [#versioned-and-encrypted-messages]

Configure `VersionedMessages.Endpoint` to create and mutate messages through a
trusted backend:

```csharp
var chat = client.Subscribe("chat:room-1");
var created = await chat.CreateVersionedMessageAsync(
    "chat.message",
    "Hello",
    new VersionedMessageCreateOptions(MessageId: "message-1")
);
await chat.AppendVersionedMessageAsync(created.MessageSerial, " world");
await chat.UpdateVersionedMessageAsync(
    created.MessageSerial,
    new VersionedMessageMutationOptions(Data: "Hello world!")
);
```

Apply V2 update, delete, and append actions in serial order. If an append
arrives without a known base, fetch the latest visible version first.

`private-encrypted-*` payloads decrypt automatically when channel auth returns
the derived `SharedSecret`. Keep the encryption master key on the backend and
continue to use TLS.

## User sign-in [#user-sign-in]

Configure `UserAuthenticationOptions` and sign in after connecting when the
application uses user-targeted events or watchlists:

```csharp
await client.ConnectAsync();
await client.User.SignInAsync();
```

## Push proxy helper [#push-proxy-helper]

```csharp
var push = new SockudoPushRegistration(
    new PushRegistrationOptions(
        Endpoint: "https://api.example.com/sockudo/push",
        Headers: new Dictionary<string, string>
        {
            ["Authorization"] = "Bearer session-token",
        }
    )
);

var publish = await push.PublishAsync(
    new Dictionary<string, object?>
    {
        ["recipients"] = new[]
        {
            new Dictionary<string, object?> { ["type"] = "channel", ["channel"] = "orders" },
        },
        ["payload"] = new Dictionary<string, object?>
        {
            ["title"] = "Order updated",
            ["body"] = "Ready for pickup",
        },
    }
);
```

The helper points at your backend proxy. Keep Sockudo app secrets on the backend.

## Production checklist [#production-checklist]

* Register the client as a singleton or hosted-service dependency and dispose
  it during shutdown.
* Pass cancellation tokens through long-running application operations.
* Keep callbacks short and move blocking work away from the receive path.
* Handle auth failure, token expiry, rate limiting, and transport failure as
  different cases.
* Make event processing idempotent and reload state after failed recovery.
* Protect history, mutation, and push proxy endpoints with the application's
  normal authorization.
* Test reconnects, service restarts, token rotation, and graceful host shutdown.


# Filters and delta compression (/docs/clients/filters-delta)



Filters and deltas solve different bandwidth problems. Filters prevent irrelevant messages from being delivered. Deltas shrink relevant messages by sending changes against a known base.

## Tag filtering [#tag-filtering]

The publisher attaches tags:

```ts
await sockudo.trigger("market:btc", "tick", { price: 67210 }, {
  tags: {
    market: "spot",
    asset: "BTC",
    region: "us",
  },
});
```

The client subscribes with a filter expression:

```ts
import { Filter } from "@sockudo/client/filter";

client.subscribe("market:btc", {
  filter: Filter.and(
    Filter.eq("market", "spot"),
    Filter.eq("asset", "BTC"),
  ),
});
```

Protocol V2 subscriptions can combine event names, tag filters, and a bounded
JMESPath expression. Every supplied component must match:

```ts
client.subscribe("orders.*", {
  events: ["order.updated"],
  filter: Filter.eq("region", "eu"),
  expression: 'data.total >= `100` && headers.priority == `"high"`',
});
```

The native .NET, Flutter/Dart, Kotlin, Python, and Swift clients expose the same
compound predicate through their language-specific `SubscriptionOptions` type.

## Supported comparisons [#supported-comparisons]

| Comparison               | Meaning                                                           |
| ------------------------ | ----------------------------------------------------------------- |
| `eq`, `neq`              | equality and inequality                                           |
| `in`, `nin`              | membership                                                        |
| `ex`, `nex`              | exists and not exists                                             |
| `sw`, `ew`, `ct`         | starts with, ends with, contains                                  |
| `gt`, `gte`, `lt`, `lte` | lexical or numeric-style comparisons, depending on tag convention |

## Delta compression [#delta-compression]

Enable delta compression per subscription:

```ts
client.subscribe("doc:123", {
  delta: {
    enabled: true,
    algorithm: "xdelta3",
  },
});
```

Sockudo supports Fossil and Xdelta3/VCDIFF paths in native clients. Use deltas when messages are frequent and similar, such as document snapshots, sports state, price books, or device telemetry.

## Conflation keys [#conflation-keys]

Conflation groups messages by logical entity so the right base is used.

```json
{
  "event": "price.update",
  "channel": "market",
  "data": { "symbol": "BTC", "price": 67210 },
  "extras": {
    "delta": {
      "conflation_key": "BTC"
    }
  }
}
```

## Operational guidance [#operational-guidance]

* Start with tag filtering when clients receive too many irrelevant events.
* Add deltas when relevant events are large or highly repetitive.
* Track raw bytes, compressed bytes, reconstruction failures, and resync requests.
* Do not use deltas for tiny messages unless measurements show a real win.
* Include stable IDs in push payloads instead of attempting delta-style push notifications.


# Flutter and Dart (/docs/clients/flutter)



`sockudo_flutter` is the official Flutter and Dart realtime SDK. It supports public, private, presence, encrypted channels, auth, V2 recovery, rewind, filters, deltas, mutable messages, presence history proxies, and push registration workflows.

The current package requires Dart 3.11.3+ and Flutter 3.35+ when used in a
Flutter application. Protocol V1 compatibility is the default; opt into
Protocol V2 for Sockudo-native features.

## Install [#install]

Install the published package from `pub.dev`:

```bash
flutter pub add sockudo_flutter
# or, for pure Dart apps:
dart pub add sockudo_flutter
```

```dart
import 'package:sockudo_flutter/sockudo_flutter.dart';
```

## Connect [#connect]

```dart
final client = SockudoClient(
  'app-key',
  const SockudoOptions(
    cluster: 'local',
    forceTls: false,
    enabledTransports: <SockudoTransport>[SockudoTransport.ws],
    wsHost: '127.0.0.1',
    wsPort: 6001,
    wssPort: 6001,
    protocolVersion: 2,
    connectionRecovery: true,
  ),
);

final channel = client.subscribe('public-updates');
channel.bind('price-updated', (data, _) {
  print(data);
});

client.connect();
```

Use a public load-balancer or ingress hostname with `forceTls: true` in
production. Create the client in an application-scoped service rather than in a
widget's `build` method.

## Lifecycle and cleanup [#lifecycle-and-cleanup]

Keep binding tokens when a widget or controller needs to remove a specific
handler:

```dart
final stateToken = client.bind('state_change', (change, _) {
  print('connection: $change');
});
final orderToken = channel.bind('order.created', (data, _) {
  print(data);
});

channel.unbind(eventName: 'order.created', token: orderToken);
client.unbind(eventName: 'state_change', token: stateToken);
client.unsubscribe('public-updates');
client.disconnect();
```

Wait for the channel subscription-success event before treating its data as
live. Unsubscribe in the owning service or controller's disposal path, and
disconnect on explicit sign-out or process shutdown. Let the SDK handle
temporary transport failures and resubscription.

Unexpected disconnects emit `ConnectionState.reconnecting` and use quadratic
backoff (0s, 1s, 4s, 9s) capped at 120 seconds. Configure
`maxReconnectAttempts` (default `6`, `null` for unlimited) and
`maxReconnectGapInSeconds`; protocol retry and TLS-upgrade close codes remain
immediate.

```dart
const options = SockudoOptions(
  cluster: 'local',
  maxReconnectAttempts: 10,
  maxReconnectGapInSeconds: 60,
);
```

## Auth [#auth]

```dart
final client = SockudoClient(
  'app-key',
  SockudoOptions(
    cluster: 'local',
    forceTls: false,
    wsHost: '127.0.0.1',
    wsPort: 6001,
    channelAuthorization: ChannelAuthorizationOptions(
      endpoint: 'https://api.example.com/sockudo/auth',
    ),
  ),
);
```

The endpoint must authenticate the user and authorize the exact requested
channel. Presence identity comes from the backend session, not from an
untrusted request field.

Protocol V2 can use a scoped capability token and async refresh callback:

```dart
final client = SockudoClient(
  'app-key',
  SockudoOptions(
    cluster: 'local',
    protocolVersion: 2,
    wsHost: 'realtime.example.com',
    forceTls: true,
    token: initialToken,
    authCallback: () async => fetchFreshSockudoToken(),
  ),
);
```

JWTs with expiry metadata refresh proactively. After the optional initial
token, the callback is invoked before reconnects. Expiry code `40142` can
refresh in place; revocation code `40160` cannot. Token auth is rejected
outside Protocol V2.

## Presence [#presence]

```dart
final presence = client.subscribe('presence-lobby') as PresenceChannel;

presence.bind('sockudo:member_added', (member, _) => print('joined: $member'));
presence.bind('sockudo:member_removed', (member, _) => print('left: $member'));
presence.bind('sockudo:presence_update', (member, _) => print('updated: $member'));

presence.update(<String, Object?>{'status': 'editing'});
```

V2 presence updates modify member data without a leave and rejoin cycle.

## Filters and deltas [#filters-and-deltas]

```dart
final channel = client.subscribe(
  'price:btc',
  options: const SubscriptionOptions(
    filter: FilterNode(key: 'market', cmp: 'eq', val: 'spot'),
    events: <String>['price.updated'],
    expression: SubscriptionExpression('data.price >= `100`'),
    delta: ChannelDeltaSettings(
      enabled: true,
      algorithm: DeltaAlgorithm.xdelta3,
    ),
  ),
);
```

## Recovery and rewind [#recovery-and-rewind]

```dart
final channel = client.subscribe(
  'market:BTC',
  options: const SubscriptionOptions(
    rewind: SubscriptionRewind.seconds(30),
  ),
);

channel.bind('message', (_, __) {
  print(client.getRecoveryPosition('market:BTC'));
});

client.bind('sockudo:resume_success', (data, _) {
  print(data);
});
```

## Mutable messages [#mutable-messages]

```dart
MutableMessageState? state;

final channel = client.subscribe('chat:room-1');
channel.bindGlobal((eventName, data) {
  if (data is! SockudoEvent || !isMutableMessageEvent(data)) return;
  state = reduceMutableMessageEvent(state, data);
});
```

Configure `versionedMessages` with a trusted backend endpoint to create,
append, update, or delete messages:

```dart
final created = await channel.createMessage(
  const VersionedMessageCreateRequest(data: 'hello'),
);
await channel.appendMessage(created.messageSerial, ' world');
await channel.updateMessage(
  created.messageSerial,
  const VersionedMessageMutation(data: <String, Object?>{'text': 'edited'}),
);
await channel.deleteMessage(created.messageSerial);
```

Apply mutation events in serial order. If an append arrives before its base,
fetch the latest visible message through the proxy first.

## Presence history proxy [#presence-history-proxy]

```dart
final client = SockudoClient(
  'app-key',
  SockudoOptions(
    cluster: 'local',
    forceTls: false,
    wsHost: '127.0.0.1',
    wsPort: 6001,
    presenceHistory: const PresenceHistoryOptions(
      endpoint: 'https://api.example.com/sockudo/presence-history',
    ),
  ),
);

final channel = client.subscribe('presence-lobby') as PresenceChannel;
final page = await channel.history(
  const PresenceHistoryParams(limit: 50, direction: 'newest_first'),
);
```

The history proxy owns the app secret, authenticates the caller, checks channel
access, and forwards opaque pagination cursors. Use channel history with
`untilAttach: true` for a gap-free late join.

## Encrypted channels [#encrypted-channels]

`private-encrypted-*` channels decrypt automatically when protected-channel
authorization returns the derived `sharedSecret`:

```dart
final encrypted = client.subscribe('private-encrypted-documents');
encrypted.bind('doc-updated', (payload, _) => print(payload));
```

Keep the encryption master key on the backend. End-to-end encryption does not
replace TLS or channel authorization.

## Push registration [#push-registration]

Use a platform push plugin to get the provider token, then send the token to your backend.

```dart
final token = await FirebaseMessaging.instance.getToken();

await http.post(
  Uri.parse('https://api.example.com/sockudo/push/devices'),
  headers: {'Content-Type': 'application/json'},
  body: jsonEncode({
    'device_id': deviceId,
    'platform': 'fcm',
    'provider_token': token,
  }),
);
```

The backend registers the device with Sockudo and enforces user ownership.

## Production checklist [#production-checklist]

* Scope the client above individual widgets and avoid reconnecting on rebuild.
* Remove bindings and subscriptions when their lifecycle owner is disposed.
* Keep event callbacks fast and move blocking work to an isolate or async
  service.
* On `sockudo:resume_failed`, discard derived state and load an authoritative
  snapshot.
* Protect and rate-limit auth, history, mutable-message, and push proxy
  endpoints.
* Refresh provider tokens through the backend and associate them with the
  authenticated user.
* Test background/foreground transitions, offline recovery, token expiry, and
  application process recreation.


# Realtime clients (/docs/clients)



Realtime client SDKs connect to Sockudo over WebSocket, subscribe to channels, receive events, authorize protected channels through your backend, and expose Protocol V2 features.

Install client SDKs from the package names below. The old per-SDK repositories should be treated as
archived/legacy mirrors.

## Official clients [#official-clients]

| SDK                     | Package                     | Runtime                                                   | Default                   |
| ----------------------- | --------------------------- | --------------------------------------------------------- | ------------------------- |
| JavaScript / TypeScript | `@sockudo/client`           | Web, Node, worker, React, Vue, React Native, NativeScript | Protocol V1 compatibility |
| Swift                   | `SockudoSwift` via SwiftPM  | iOS, macOS, tvOS, watchOS, visionOS                       | Protocol V1 compatibility |
| Kotlin                  | `io.sockudo:sockudo-kotlin` | Android and JVM                                           | Protocol V2 by default    |
| Flutter / Dart          | `sockudo_flutter`           | Flutter and Dart                                          | Protocol V1 compatibility |
| .NET realtime           | `Sockudo.Client`            | .NET apps                                                 | Protocol V2 by default    |
| Python realtime         | `sockudo-python`            | Python 3.10+ and asyncio                                  | Protocol V2 by default    |

Use Protocol V1 when migrating existing Pusher clients. Use Protocol V2 for recovery, rewind, delta compression, tag filters, mutable messages, annotations, and native Sockudo metadata.

The realtime and HTTP SDKs have different trust boundaries. Realtime clients
hold only the public app key and maintain WebSocket subscriptions. Server SDKs
hold the app secret, publish over HTTP, authorize protected subscriptions, and
validate webhooks. For Python and .NET in particular, make sure you choose the
package whose role matches your process.

| Language                | Realtime package  | Trusted HTTP package     |
| ----------------------- | ----------------- | ------------------------ |
| JavaScript / TypeScript | `@sockudo/client` | `sockudo`                |
| Python                  | `sockudo-python`  | `sockudo-http-python`    |
| .NET                    | `Sockudo.Client`  | `SockudoServer`          |
| Swift                   | `SockudoSwift`    | `Sockudo` server package |

## Client responsibilities [#client-responsibilities]

Client SDKs should:

* connect with public app key only
* call your backend for private, presence, encrypted, push, and history proxy operations
* keep app secrets out of the client bundle
* bind events and update UI state
* store recovery positions when V2 recovery is enabled
* register devices for push through trusted backend endpoints

They should never contain the app secret, provider credentials, an encryption
master key, or an unrestricted capability token.

## Server responsibilities [#server-responsibilities]

Server SDKs should:

* publish events over the HTTP API
* sign channel and user authentication responses
* proxy presence history and versioned message reads for clients
* validate webhooks
* manage push registration, credentials, channel push subscriptions, and publish workflows

## Common V2 setup [#common-v2-setup]

```ts
const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  protocolVersion: 2,
  connectionRecovery: true,
});
```

Use this sequence in every client:

1. Construct one long-lived client with your public key and public WebSocket
   endpoint.
2. Bind connection diagnostics and event handlers.
3. Subscribe to channels, using a backend auth endpoint for protected names.
4. Connect and let the SDK resubscribe after transient failures.
5. If recovery fails, replace derived state from an authoritative snapshot.
6. Unbind or unsubscribe when a screen or worker no longer needs a channel.
7. Disconnect during application shutdown.

## Choosing features [#choosing-features]

| Requirement                   | Client setting or API             | Backend requirement                     |
| ----------------------------- | --------------------------------- | --------------------------------------- |
| Pusher migration              | Protocol V1                       | Pusher-compatible app config            |
| Resume after a short outage   | Protocol V2 recovery              | Durable or hot recovery configured      |
| Initial backlog               | Subscription rewind               | History enabled with suitable retention |
| Gap-free late join            | History with `until_attach`       | Authorized history proxy                |
| Lower high-volume bandwidth   | Tag filters and delta compression | V2 filtering/delta enabled              |
| Private data                  | Private channel auth              | Session-aware auth endpoint             |
| End-to-end content encryption | `private-encrypted-*`             | Shared-secret auth and key management   |
| User-targeted events          | User sign-in                      | User auth endpoint                      |
| Mobile notifications          | Provider token collection         | Push registration and publish backend   |

## Connection and state model [#connection-and-state-model]

A successful WebSocket connection does not mean every channel is subscribed.
Wait for each channel's subscription-success event before treating its state as
live. During reconnect, keep the last rendered state but mark it stale. After a
successful resume, continue from the recovered serial; after a failed resume,
discard derived state and fetch a fresh snapshot.

Event handlers should be idempotent. Network reconnects, application retries,
and upstream publishing workflows can all repeat logical work even when the
client deduplicates messages by `message_id`.

## Protected-channel auth flow [#protected-channel-auth-flow]

```text
client -> your backend: socket_id + requested channel
your backend: authenticate session and authorize that exact channel
your backend -> client: short-lived signed auth response
client -> Sockudo: subscribe with auth response
```

Do not implement an auth endpoint that signs every requested channel. Validate
tenant ownership, resource access, and presence identity before signing. For
encrypted channels, return only the derived channel shared secret—not the
encryption master key.

## Production checklist [#production-checklist]

* Terminate TLS at Sockudo or a load balancer and use `wss://`.
* Keep a single client per app or process unless isolation is intentional.
* Observe connected, unavailable, failed, and reconnecting states.
* Set auth and proxy request timeouts; handle `401`, `403`, and `429`
  separately from transport failures.
* Bound rewind and history page sizes.
* Test a rolling restart, network interruption, auth expiry, and resume failure.
* Unsubscribe and unbind handlers when views or jobs end.

## Push from clients [#push-from-clients]

Mobile and browser clients can collect provider tokens, but they should not call Sockudo push admin APIs directly. Send provider tokens to your backend, bind them to the authenticated user, and let the backend call Sockudo.

```ts
await fetch("/api/push/devices", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    device_id: "browser-device-1",
    platform: "webpush",
    provider_token: subscription,
  }),
});
```

## Next steps [#next-steps]

<Cards>
  <Card title="JavaScript" href="/docs/clients/javascript" description="Browser, Node, React, Vue, worker, React Native, and NativeScript examples." />

  <Card title="Swift" href="/docs/clients/swift" description="Apple platforms with auth, recovery, filters, deltas, and push proxy notes." />

  <Card title="Kotlin" href="/docs/clients/kotlin" description="Android and JVM WebSocket client examples." />

  <Card title="Flutter" href="/docs/clients/flutter" description="Flutter and pure Dart client examples." />

  <Card title=".NET realtime" href="/docs/clients/dotnet" description="Protocol V2 by default for .NET applications." />

  <Card title="Python realtime" href="/docs/clients/python" description="Asyncio client for services, agents, workers, and CLIs." />
</Cards>


# JavaScript and TypeScript (/docs/clients/javascript)



`@sockudo/client` is the official JavaScript and TypeScript realtime SDK. It preserves the Pusher subscribe/bind model and adds runtime-specific entrypoints.

The default is Protocol V1 compatibility. Set `protocolVersion: 2` for
Sockudo-native recovery, rewind, filters, deltas, binary wire formats, and
versioned-message metadata.

## Install [#install]

Install the published package from npm:

```bash
npm install @sockudo/client
# or: bun add @sockudo/client
# or: pnpm add @sockudo/client
# or: yarn add @sockudo/client
```

## Runtime imports [#runtime-imports]

```ts
import Sockudo from "@sockudo/client";
import { Filter } from "@sockudo/client/filter";
import SockudoEncrypted from "@sockudo/client/with-encryption";
import WorkerSockudo from "@sockudo/client/worker";
import { SockudoProvider, useChannel } from "@sockudo/client/react";
import { createSockudoPlugin, useChannel as useSockudoChannel } from "@sockudo/client/vue";
import ReactNativeSockudo from "@sockudo/client/react-native";
import NativeScriptSockudo from "@sockudo/client/nativescript";
```

## Connect [#connect]

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  enabledTransports: ["ws"],
  protocolVersion: 2,
  connectionRecovery: true,
});

const channel = client.subscribe("public-updates");
channel.bind("price-updated", (payload) => {
  console.log(payload);
});
```

For self-hosted production, `wsHost` should be the public ingress or load
balancer name, `wssPort` is normally `443`, and `forceTLS` should be `true`.
Specify `enabledTransports: ["ws"]` only when debugging a direct local
connection; allowing both secure and fallback transports is safer across
browser and mobile runtimes.

## Capability-token authentication [#capability-token-authentication]

Protocol V2 can authorize the WebSocket with a scoped capability token:

```ts
const client = new Sockudo("app-key", {
  cluster: "local",
  protocolVersion: 2,
  token: initialToken,
  authCallback: async ({ socketId, reason }) =>
    fetchCapabilityToken({ socketId, reason }),
});
```

The optional static token is used on the first connection. The callback is
called before reconnects and at 80% of a JWT's lifetime, then the client sends
`sockudo:auth` without dropping the socket. `authUrl` is also supported for a
JSON POST token endpoint. Expiry code `40142` refreshes only through a
provider; revocation code `40160` surfaces `TokenRevokedError` without an
in-place retry. Token auth configured outside Protocol V2 is rejected.

## Connection lifecycle [#connection-lifecycle]

Observe the connection separately from channel subscription state:

```ts
client.connection.bind("state_change", ({ previous, current }) => {
  console.log(`connection: ${previous} -> ${current}`);
});
client.connection.bind("error", (error) => {
  console.error("Sockudo connection error", error);
});

const channel = client.subscribe("orders");
channel.bind("sockudo:subscription_succeeded", () => {
  console.log("orders is live");
});
channel.bind("sockudo:subscription_error", (error) => {
  console.error("orders subscription failed", error);
});
```

A connected socket can still have an unauthorized or pending channel. Gate
channel-dependent UI on `subscription_succeeded`, not only the global
`connected` event.

Keep the binding callback when you need to remove one handler, or remove the
whole subscription when its owning component or job ends:

```ts
const onOrder = (order: unknown) => console.log(order);
channel.bind("order.created", onOrder);

channel.unbind("order.created", onOrder);
client.unsubscribe("orders");
client.disconnect();
```

Framework hooks clean up their own bindings when components unmount. When using
the core client directly, lifecycle cleanup is your responsibility.

Unexpected disconnects emit `reconnecting` and use quadratic backoff (0s, 1s,
4s, 9s) capped at 120 seconds. Configure `maxReconnectAttempts` (default `6`,
`null` for unlimited) and `maxReconnectGapInSeconds`; protocol retry and
TLS-upgrade close codes remain immediate.

```ts
const client = new Sockudo("app-key", {
  cluster: "local",
  maxReconnectAttempts: 10,
  maxReconnectGapInSeconds: 60,
});

client.connection.bind("reconnecting", () => console.log("reconnecting"));
```

## Binary wire formats [#binary-wire-formats]

Protocol V2 can negotiate JSON, MessagePack, or Protobuf with `wireFormat`.
MessagePack and Protobuf preserve `Uint8Array` message data as native binary
values without a JSON or base64 intermediate.

```ts
const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  protocolVersion: 2,
  wireFormat: "messagepack",
});

const channel = client.subscribe("private-binary");
channel.trigger("client-binary", new Uint8Array([0xde, 0xad, 0xbe, 0xef]));
```

Native MessagePack uses the additive tagged data variant
`["binary", <bin>]`. Existing string, structured, and JSON variants retain
their previous representation.

## Private and presence auth [#private-and-presence-auth]

```ts
const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  channelAuthorization: {
    endpoint: "/sockudo/auth",
  },
  userAuthentication: {
    endpoint: "/sockudo/user-auth",
  },
});

const presence = client.subscribe("presence-lobby");
presence.bind("pusher:member_added", console.log);
```

The auth endpoint must validate the current application session and authorize
the exact `channel_name` sent by the SDK before signing. For presence channels,
derive `user_id` on the server; do not trust a client-supplied identity. The
client bundle contains only the public app key.

```ts
presence.bind("pusher:subscription_succeeded", (members) => {
  console.log("initial members", members);
});
presence.bind("pusher:member_removed", (member) => {
  console.log("left", member);
});
```

Configure `userAuthentication.endpoint` and call `client.signin()` when using
user-targeted events or watchlists. The backend signs the current `socket_id`
and authenticated user data.

## React [#react]

```tsx
import Sockudo from "@sockudo/client";
import { SockudoProvider, useChannel } from "@sockudo/client/react";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
});

client.connect();

function Orders() {
  const { subscribed, events } = useChannel("orders", ["order.created"]);
  return <pre>{JSON.stringify({ subscribed, events }, null, 2)}</pre>;
}

export function App() {
  return (
    <SockudoProvider client={client}>
      <Orders />
    </SockudoProvider>
  );
}
```

## Vue [#vue]

```ts
import Sockudo from "@sockudo/client";
import { createSockudoPlugin, useChannel } from "@sockudo/client/vue";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
});

app.use(createSockudoPlugin(client));

const { subscribed, bind } = useChannel("orders");
bind("order.created", (payload) => console.log(payload));
```

## Filters and deltas [#filters-and-deltas]

```ts
const channel = client.subscribe("market:btc", {
  filter: Filter.and(
    Filter.eq("market", "spot"),
    Filter.gte("price", "100"),
  ),
  delta: {
    enabled: true,
    algorithm: "xdelta3",
  },
});
```

## Recovery and rewind [#recovery-and-rewind]

```ts
const channel = client.subscribe("market:BTC", {
  rewind: { seconds: 30 },
});

channel.bind("message", () => {
  console.log(client.getRecoveryPosition("market:BTC"));
});

client.bind("sockudo:resume_success", (payload) => {
  console.log(payload.recovered, payload.failed);
});
```

## Versioned messages [#versioned-messages]

Configure a backend proxy for REST reads:

```ts
const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  versionedMessages: {
    endpoint: "/sockudo/versioned",
  },
});

const channel = client.subscribe("chat:room-1");
const latest = await channel.getMessage("42");
const versions = await channel.getMessageVersions("42", {
  limit: 20,
  direction: "oldest_first",
});
```

The proxy endpoint must authenticate the caller, authorize access to the
channel, and sign the upstream HTTP request with the app secret. Use
`until_attach` channel history for a gap-free late join: first attach, then load
history ending at the server-provided attach serial, then apply buffered live
events.

## Encrypted channels [#encrypted-channels]

Use the encryption entrypoint for `private-encrypted-*` channels:

```ts
import SockudoEncrypted from "@sockudo/client/with-encryption";

const encryptedClient = new SockudoEncrypted("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  channelAuthorization: { endpoint: "/sockudo/auth" },
});

encryptedClient
  .subscribe("private-encrypted-documents")
  .bind("doc.updated", (payload) => console.log(payload));
```

Your auth response supplies the derived channel `shared_secret`. Keep the
encryption master key on the trusted backend. End-to-end encryption complements
TLS and authorization; it does not replace either.

## Push proxy helpers [#push-proxy-helpers]

Browser push registration should flow through your backend. Keep app secrets and provider credentials server-side.

```ts
await fetch("/sockudo/push/devices", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    device_id: "browser-device-1",
    platform: "webpush",
    provider_token: pushSubscription,
  }),
});
```

Use a server SDK to publish push after validating user and tenant policy.

## Error handling and production guidance [#error-handling-and-production-guidance]

* Reuse one client per browser tab, worker, or native application runtime.
* Create the client outside React render functions and Vue component setup.
* Use the runtime-specific entrypoint so bundlers do not pull in incompatible
  browser or Node transports.
* Treat auth `401`/`403` as an application-permission problem, not a reconnect
  loop.
* On `sockudo:resume_failed`, clear derived state and fetch an authoritative
  snapshot before applying new events.
* Make handlers idempotent and validate unknown payloads at the application
  boundary.
* Unbind handlers, unsubscribe unused channels, and disconnect during
  application shutdown.
* Never expose the app secret, push credentials, or encryption master key in a
  browser or mobile bundle.


# Kotlin (/docs/clients/kotlin)



`sockudo-kotlin` is the official Android and JVM realtime SDK. It uses OkHttp for WebSockets and exposes the same channel model as other Sockudo clients.

The current package targets JVM 23 and uses Protocol V2 by default. Set
`protocolVersion = 1` only when strict Pusher Protocol V1 compatibility is
required.

## Install [#install]

Install the published package from Maven Central:

```kotlin
dependencies {
    implementation("io.sockudo:sockudo-kotlin:2.2.0")
}
```

For Maven projects:

```xml
<dependency>
  <groupId>io.sockudo</groupId>
  <artifactId>sockudo-kotlin</artifactId>
  <version>2.2.0</version>
</dependency>
```

## Connect [#connect]

```kotlin
import io.sockudo.client.SockudoClient
import io.sockudo.client.SockudoOptions
import io.sockudo.client.SockudoTransport

val client =
    SockudoClient(
        "app-key",
        SockudoOptions(
            cluster = "local",
            forceTls = false,
            enabledTransports = listOf(SockudoTransport.ws),
            wsHost = "127.0.0.1",
            wsPort = 6001,
            wssPort = 6001,
            protocolVersion = 2,
            connectionRecovery = true,
        ),
    )

val channel = client.subscribe("public-updates")
channel.bind("price-updated") { data, _ ->
    println(data)
}

client.connect()
```

For production, point `wsHost` at the public load balancer or ingress, set
`forceTls = true`, and use the secure WebSocket port. Keep one long-lived
client per application process or signed-in mobile session.

## Lifecycle and cleanup [#lifecycle-and-cleanup]

Connection and channel events use the same binding model as application events:

```kotlin
val stateToken =
    client.bind("state_change") { change, _ ->
        println("connection: $change")
    }

val eventToken =
    channel.bind("price-updated") { data, _ ->
        println(data)
    }

channel.unbind("price-updated", eventToken)
client.unbind("state_change", stateToken)
client.unsubscribe("public-updates")
client.disconnect()
```

Wait for the channel subscription-success event before treating channel state
as live. Unsubscribe and unbind when an Android lifecycle owner no longer needs
updates; disconnect on explicit sign-out. A temporary network loss should be
left to the SDK's reconnect path.

Unexpected disconnects emit `ConnectionState.RECONNECTING` and use quadratic
backoff (0s, 1s, 4s, 9s) capped at 120 seconds. Configure
`maxReconnectAttempts` (default `6`, `null` for unlimited) and
`maxReconnectGapInSeconds`; protocol retry and TLS-upgrade close codes remain
immediate.

## Auth [#auth]

```kotlin
import io.sockudo.client.*

val client =
    SockudoClient(
        "app-key",
        SockudoOptions(
            cluster = "local",
            forceTls = false,
            wsHost = "127.0.0.1",
            wsPort = 6001,
            channelAuthorization =
                ChannelAuthorizationOptions(
                    endpoint = "https://api.example.com/sockudo/auth",
                ),
        ),
    )
```

Your auth endpoint must validate the application session and authorize the
exact requested channel. For presence, derive `user_id` from the server-side
identity. Never ship the app secret or an encryption master key in the APK.

Protocol V2 capability tokens can be supplied with a refresh provider:

```kotlin
val client =
    SockudoClient(
        "app-key",
        SockudoOptions(
            cluster = "local",
            protocolVersion = 2,
            authTokenProvider =
                ClientAuthTokenProvider { request ->
                    fetchCapabilityToken(
                        reason = request.reason,
                        socketId = request.socketId,
                    )
                },
        ),
    )
```

JWT expiry metadata enables proactive refresh, and the provider is called
before reconnects. Only expiry code `40142` refreshes in place; revocation code
`40160` and static tokens are not resent. Token auth is rejected outside
Protocol V2.

## Presence [#presence]

```kotlin
val presence = client.subscribe("presence-lobby") as PresenceChannel

presence.bind("sockudo:member_added") { member, _ -> println("joined: $member") }
presence.bind("sockudo:member_removed") { member, _ -> println("left: $member") }
presence.bind("sockudo:presence_update") { member, _ -> println("updated: $member") }

presence.update(mapOf("status" to "typing"))
```

The update method changes V2 presence member data without a leave/rejoin cycle.

## Filters and deltas [#filters-and-deltas]

```kotlin
val channel =
    client.subscribe(
        "price:btc",
        SubscriptionOptions(
            filter = Filter.eq("market", "spot"),
            events = listOf("price.updated"),
            expression = SubscriptionExpression.Source("data.price >= `100`"),
            delta = ChannelDeltaSettings(
                enabled = true,
                algorithm = DeltaAlgorithm.xdelta3,
            ),
        ),
    )
```

## Recovery and rewind [#recovery-and-rewind]

```kotlin
val channel =
    client.subscribe(
        "market:BTC",
        SubscriptionOptions(rewind = SubscriptionRewind.Seconds(30)),
    )

channel.bind("message") { _, _ ->
    println(client.getRecoveryPosition("market:BTC"))
}

client.bind("sockudo:resume_success") { data, _ ->
    println(data)
}
```

## Presence history proxy [#presence-history-proxy]

```kotlin
val client =
    SockudoClient(
        "app-key",
        SockudoOptions(
            cluster = "local",
            forceTls = false,
            wsHost = "127.0.0.1",
            wsPort = 6001,
            presenceHistory =
                PresenceHistoryOptions(
                    endpoint = "https://api.example.com/sockudo/presence-history",
                ),
        ),
    )

val channel = client.subscribe("presence-lobby") as PresenceChannel
val page = channel.history(PresenceHistoryParams(limit = 50, direction = "newest_first"))
val snapshot = channel.snapshot(PresenceSnapshotParams(atSerial = 4))
```

History, snapshots, and versioned-message helpers call your trusted proxy. The
proxy must authenticate the caller, authorize the channel, bound page sizes,
and sign the upstream Sockudo request. Use channel history with
`untilAttach = true` when building a gap-free late join.

## Mutable messages [#mutable-messages]

Apply V2 message actions in serial order. An update replaces local data, a
delete becomes the latest visible version, and an append concatenates to a
known string base. Configure `VersionedMessagesOptions.endpoint` for
proxy-backed writes:

```kotlin
val chat = client.subscribe("chat:room-1")
val ack = chat.createMessage("chat.message", mapOf("text" to "hello"))
chat.appendMessage(ack.messageSerial, " world")
chat.updateMessage(ack.messageSerial, mapOf("text" to "edited"))
chat.deleteMessage(ack.messageSerial)
```

If an append arrives before its base, fetch the latest message through the
proxy before applying subsequent actions.

## Encrypted channels [#encrypted-channels]

`private-encrypted-*` subscriptions decrypt automatically when the auth
response includes the derived channel shared secret:

```kotlin
val encrypted = client.subscribe("private-encrypted-documents")
encrypted.bind("doc-updated") { payload, _ -> println(payload) }
```

Encrypted channels still require TLS and normal channel authorization.

## Android push registration [#android-push-registration]

Use Firebase Messaging or the platform provider to obtain a token, then register through your backend.

```kotlin
class PushTokenService : FirebaseMessagingService() {
    override fun onNewToken(token: String) {
        registerDeviceWithBackend(
            deviceId = Settings.Secure.getString(contentResolver, Settings.Secure.ANDROID_ID),
            platform = "fcm",
            providerToken = token,
        )
    }
}
```

The backend should call Sockudo push registration APIs with app credentials. The Android app should not hold Sockudo secrets.

## Production checklist [#production-checklist]

* Tie subscriptions to an Android lifecycle owner and avoid duplicate bindings
  after configuration changes.
* Keep callbacks short; move blocking work off the WebSocket callback path.
* Treat auth failures as permission failures instead of endlessly reconnecting.
* On `sockudo:resume_failed`, reload authoritative application state.
* Bound rewind and proxy history requests.
* Refresh FCM tokens through the backend when `onNewToken` fires.
* Test airplane-mode transitions, process recreation, capability-token expiry,
  and rolling Sockudo restarts.


# Presence history (/docs/clients/presence-history)



Presence answers two separate questions:

* Who is subscribed now?
* What membership transitions happened over time?

Current presence is realtime channel state. Presence history is a durable event log.

## Current members [#current-members]

```ts
const channel = client.subscribe("presence-lobby");

channel.bind("pusher:subscription_succeeded", (members) => {
  console.log("members", members);
});

channel.bind("pusher:member_added", console.log);
channel.bind("pusher:member_removed", console.log);
```

## History proxy [#history-proxy]

Client SDKs call your backend proxy. The proxy signs the Sockudo HTTP API request.

```ts
const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  presenceHistory: {
    endpoint: "/sockudo/presence-history",
  },
});

const channel = client.subscribe("presence-lobby");
const page = await channel.history({
  limit: 50,
  direction: "newest_first",
});
```

## Snapshot [#snapshot]

Snapshots reconstruct effective membership at a point in the presence stream.

```ts
const snapshot = await channel.snapshot({
  atSerial: 42,
});

console.log(snapshot.members);
```

## Push relationship [#push-relationship]

Presence is about connected clients. Push registrations are about reachable devices. A user can be absent from a presence channel and still have a device eligible for push.

Use presence for live collaboration indicators. Use push channel subscriptions for offline notification eligibility.


# Protocol V2 (/docs/clients/protocol-v2)



Protocol V2 is Sockudo's native protocol. It keeps the channel model familiar while adding metadata and events that are intentionally not part of Pusher compatibility mode.

## Enable V2 [#enable-v2]

```ts
const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  protocolVersion: 2,
  connectionRecovery: true,
});
```

## V2 metadata [#v2-metadata]

V2 broadcasts can include:

| Field        | Purpose                                                                        |
| ------------ | ------------------------------------------------------------------------------ |
| `message_id` | Stable broadcast identity for deduplication.                                   |
| `serial`     | Stream position used for continuity.                                           |
| `stream_id`  | Opaque stream identity.                                                        |
| `extras`     | Headers, tags, echo controls, idempotency metadata, and feature-specific data. |

When recovery is enabled, V2 `sockudo_internal:subscription_succeeded` acknowledgements also include a `stream_id` and `serial`. Store that initial position even if no application events have been received yet; it lets the client resume an attached but idle channel and receive messages published during a short disconnect.

Successful V2 subscriptions may include `attach_serial` in `sockudo_internal:subscription_succeeded` when durable history is enabled for the channel. That value is the channel history head captured at attach time.

## Binary wire data [#binary-wire-data]

MessagePack and Protobuf preserve application message data as native bytes:
`byte[]` on .NET, `Uint8List` on Dart, `ByteArray` on Kotlin, `bytes` on Python,
`Data` on Swift, and `Uint8Array` on JavaScript. MessagePack uses the additive
`["binary", <bin>]` tagged data variant; the existing string and JSON variants
remain unchanged.

## Native events [#native-events]

Protocol V2 uses `sockudo:` and `sockudo_internal:` prefixes for system events. Protocol V1 keeps `pusher:` and `pusher_internal:` prefixes.

## Recovery contract [#recovery-contract]

Recovery is authoritative only when Sockudo returns a successful resume. If the stream reset, buffer expired, or continuity cannot be proven, the client must resubscribe and rebuild state.

```ts
client.bind("sockudo:resume_success", (payload) => {
  console.log("recovered", payload.recovered);
});

client.bind("sockudo:resume_failed", (payload) => {
  console.warn("rebuild state", payload);
});
```

Each recovered entry may include an authoritative terminal `position`. Official
clients apply it even when a subscription predicate suppressed the recovered
tail, so the next resume starts from the canonical channel position.

When `payload.code === "position_expired"`, resubscribe and use channel history with `until_attach: true` to backfill up to the new attach serial.

## Client history [#client-history]

V2 clients request channel history over the WebSocket with the existing `channel_history` action:

```json
{
  "event": "sockudo:channel_history",
  "data": {
    "channel": "chat",
    "limit": 100,
    "direction": "backwards",
    "cursor": null,
    "until_attach": true
  }
}
```

The response event is `sockudo:channel_history`. It returns `items`, `has_more`, `next_cursor`, `bounds`, `continuity`, and `stream_state`. `limit` defaults to `100` and is capped at `1000`. `until_attach: true` returns only history at or below the subscription `attach_serial`, so combining those items with live messages produces a gap-free view without duplicates.

## Heartbeats [#heartbeats]

Protocol V2 prefers native WebSocket ping/pong frames. Browser-like runtimes may still use lightweight `sockudo:ping` and `sockudo:pong` fallback messages for liveness checks. Fallback heartbeat messages are not broadcast continuity events and do not carry `message_id`, `serial`, or `stream_id`.

## Use cases [#use-cases]

Use V2 for:

* collaboration documents that need reconnect continuity
* dashboards that benefit from deltas
* market data streams with tag filters
* chat systems with mutable messages and annotations
* mobile apps combining realtime channels and push notifications
* operators who need history and recovery visibility


# Python (/docs/clients/python)



`sockudo-python` is the official asynchronous realtime client for Python. It is
intended for long-running services, agents, workers, CLIs, and other
`asyncio` applications that need a WebSocket connection to Sockudo. It is
different from [`sockudo-http-python`](/docs/server-sdks/python), which is the
trusted server-side HTTP SDK used to publish and sign authorization responses.

The client requires Python 3.10 or newer and uses Protocol V2 by default.

## Install [#install]

```bash
python -m pip install sockudo-python
```

For development from this monorepo:

```bash
python -m pip install -e client-sdks/sockudo-python
```

## Connect and subscribe [#connect-and-subscribe]

Create one client per application process, bind handlers before connecting, and
close it during application shutdown:

```python
import asyncio

from sockudo_python import SockudoClient, SockudoOptions


async def main() -> None:
    client = SockudoClient(
        "app-key",
        SockudoOptions(
            cluster="local",
            ws_host="127.0.0.1",
            ws_port=6001,
            force_tls=False,
            protocol_version=2,
            connection_recovery=True,
        ),
    )

    channel = client.subscribe("public-updates")
    channel.bind("price-updated", lambda payload, meta: print(payload))

    try:
        await client.connect()
        await asyncio.Event().wait()
    finally:
        await client.disconnect()


asyncio.run(main())
```

`connect()` returns after starting the connection; event callbacks run on the
same asyncio event loop. Do not perform blocking file, database, or network work
inside a callback. Schedule or await that work in an async task instead.

### Self-hosted connection options [#self-hosted-connection-options]

| Option                         | Purpose                                                       | Production guidance                                      |
| ------------------------------ | ------------------------------------------------------------- | -------------------------------------------------------- |
| `ws_host`                      | WebSocket host without a scheme                               | Use the public load balancer or ingress hostname         |
| `ws_port` / `wss_port`         | Plaintext and TLS WebSocket ports                             | Usually `80` and `443` behind a proxy                    |
| `force_tls`                    | Select `wss://` when true                                     | Enable outside local development                         |
| `protocol_version`             | `2` for Sockudo-native features; `1` for Pusher compatibility | Prefer `2` for new applications                          |
| `connection_recovery`          | Resume from the last `stream_id` and `serial`                 | Enable when missed events matter                         |
| `max_reconnect_attempts`       | Retry limit; `None` means unlimited                           | Defaults to `6`                                          |
| `max_reconnect_gap_in_seconds` | Maximum quadratic backoff delay                               | Defaults to `120.0` seconds                              |
| `wire_format`                  | JSON, MessagePack, or Protobuf                                | Use a binary format only when every endpoint supports it |

`cluster` remains required by the options type, but a custom `ws_host` is the
important routing setting for self-hosted deployments.

## Lifecycle and cleanup [#lifecycle-and-cleanup]

Bind connection events to make health and reconnect behavior visible:

```python
def on_state_change(change, _) -> None:
    print(f"connection: {change['previous']} -> {change['current']}")


client.bind("state_change", on_state_change)
client.bind(
    "connected",
    lambda data, _: print("socket id:", data.get("socket_id")),
)
client.bind("connecting", lambda *_: print("connecting"))
client.bind("reconnecting", lambda *_: print("reconnecting"))
client.bind("error", lambda error, _: print("error:", error))
```

Remove a subscription when its consumer goes away. This also clears the
channel's recovery and delta state:

```python
await client.unsubscribe("public-updates")
```

`await client.close()` is an alias for `disconnect()`. A manual disconnect
stops reconnect timers; an unexpected transport failure uses the client's
reconnect policy. Retries emit `reconnecting` and use quadratic backoff (0s,
1s, 4s, 9s) capped by `max_reconnect_gap_in_seconds`; protocol retry and
TLS-upgrade close codes remain immediate.

## Private and presence authorization [#private-and-presence-authorization]

The client sends its `socket_id` and channel name to your backend. Your backend
must authenticate the user, enforce access to the requested channel, and use a
[server SDK](/docs/server-sdks) to sign the response. Never put the app secret
in Python client code.

```python
from sockudo_python import ChannelAuthorizationOptions

client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        ws_host="realtime.example.com",
        force_tls=True,
        channel_authorization=ChannelAuthorizationOptions(
            endpoint="https://api.example.com/sockudo/auth",
            headers={"X-Client": "worker"},
        ),
    ),
)

private_orders = client.subscribe("private-orders")
private_orders.bind("order-placed", lambda data, _: print(data))
```

Presence subscriptions receive the initial member set and subsequent joins and
leaves:

```python
presence = client.subscribe("presence-lobby")
presence.bind(
    "sockudo:subscription_succeeded",
    lambda members, _: print("members:", members),
)
presence.bind("sockudo:member_added", lambda member, _: print("joined:", member))
presence.bind("sockudo:member_removed", lambda member, _: print("left:", member))

await client.connect()
await presence.update({"status": "editing"})
```

The `update()` call is a Protocol V2 presence update. It changes member data
without a leave and rejoin cycle.

## Capability-token authentication [#capability-token-authentication]

Protocol V2 can authorize the WebSocket itself with a scoped capability token.
Use an async callback so the client can obtain fresh credentials:

```python
from sockudo_python import TokenAuthData


async def fetch_token() -> TokenAuthData:
    response = await call_your_backend()
    return TokenAuthData(
        token=response["token"],
        expires_in=response["expires_in"],
    )


client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        ws_host="realtime.example.com",
        force_tls=True,
        auth_callback=fetch_token,
    ),
)
```

Expiry metadata lets the SDK schedule a refresh before expiration. Opaque
callback tokens without expiry metadata are refreshed after code `40142` and
before reconnects. Static and revoked tokens are never resent. Token auth is
rejected outside Protocol V2.

## Filters, event selection, and delta compression [#filters-event-selection-and-delta-compression]

Protocol V2 subscriptions can combine event names, tag filters, and a bounded
expression. All supplied conditions must match:

```python
from sockudo_python import (
    ChannelDeltaSettings,
    DeltaAlgorithm,
    Filter,
    SubscriptionOptions,
)

market = client.subscribe(
    "price:btc",
    options=SubscriptionOptions(
        events=["price.updated"],
        filter=Filter.and_(
            Filter.eq("market", "spot"),
            Filter.gt("spread", "0"),
        ),
        expression="data.price >= `100`",
        delta=ChannelDeltaSettings(
            enabled=True,
            algorithm=DeltaAlgorithm.XDELTA3,
        ),
    ),
)
```

Use delta compression for frequently changing, structurally similar payloads
such as order books. The SDK reconstructs full payloads before invoking your
handler. Monitor `client.get_delta_stats()` and fall back to full events if
base continuity is lost. See [Filters and delta compression](/docs/clients/filters-delta).

## Recovery, rewind, and late joins [#recovery-rewind-and-late-joins]

Recovery resumes a previously attached channel after a connection interruption.
Rewind asks the server for a bounded amount of earlier history at subscription
time:

```python
from sockudo_python import SubscriptionOptions, SubscriptionRewind

market = client.subscribe(
    "market:btc",
    options=SubscriptionOptions(
        rewind=SubscriptionRewind.seconds_back(30),
    ),
)

client.bind("sockudo:resume_success", lambda data, _: print(data))
client.bind("sockudo:resume_failed", lambda data, _: print(data))
market.bind("sockudo:rewind_complete", lambda data, _: print(data))
```

Treat `resume_failed` as a continuity break: clear derived local state and fetch
an authoritative snapshot before applying new events.

For gap-free late joins, proxy channel history through your backend and request
history only up to the subscription's attach point:

```python
from sockudo_python import ChannelHistoryOptions, ChannelHistoryParams

client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        channel_history=ChannelHistoryOptions(
            endpoint="https://api.example.com/sockudo/channel-history",
        ),
    ),
)

channel = client.subscribe("orders")
page = await channel.history(
    ChannelHistoryParams(limit=50, until_attach=True),
)
```

The proxy owns Sockudo credentials and validates which channels the requesting
user may read.

## Presence history and snapshots [#presence-history-and-snapshots]

Presence history is also proxy-backed. The endpoint receives a channel, action,
and parameters, then calls the signed server REST API:

```python
from sockudo_python.client import (
    PresenceHistoryOptions,
    PresenceHistoryParams,
    PresenceSnapshotParams,
)

client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        presence_history=PresenceHistoryOptions(
            endpoint="https://api.example.com/sockudo/presence-history",
        ),
    ),
)

presence = client.subscribe("presence-lobby")
page = await presence.history(
    PresenceHistoryParams(limit=50, direction="newest_first"),
)
if page.has_next():
    next_page = await page.next()

snapshot = await presence.snapshot(PresenceSnapshotParams(at_serial=4))
```

Use current membership for live UI, history for audit or timelines, and a
snapshot to reconstruct membership at a specific serial. See
[Presence history](/docs/clients/presence-history).

## Versioned messages [#versioned-messages]

Create and mutate versioned messages through a trusted proxy. These helpers do
not expose app credentials or send mutation frames over the WebSocket:

```python
from sockudo_python import VersionedMessageOptions

client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        versioned_messages=VersionedMessageOptions(
            endpoint="https://api.example.com/sockudo/versioned-messages",
        ),
    ),
)

ack = await client.versioned_messages.create(
    "chat:room-1",
    "message.created",
    {"text": "hello"},
)
if ack.message_id is None:
    raise RuntimeError("Sockudo did not return a message_id")

await client.versioned_messages.append(
    "chat:room-1",
    ack.message_id,
    {"text": " world"},
)
await client.versioned_messages.update(
    "chat:room-1",
    ack.message_id,
    {"text": "hello world"},
)
```

Apply mutations in serial order. If an append arrives before its base, fetch
the latest visible message from the proxy before applying later appends.

## Encrypted channels [#encrypted-channels]

`private-encrypted-*` subscriptions are decrypted automatically. The
authorization response must include a `shared_secret` derived by your trusted
backend:

```python
encrypted = client.subscribe("private-encrypted-documents")
encrypted.bind("doc-updated", lambda data, _: print(data))
```

Encryption protects event content end to end. It does not replace TLS,
authorization, channel naming policy, or secret rotation.

## User sign-in [#user-sign-in]

Configure a user-auth endpoint when using user-targeted events or watchlists:

```python
from sockudo_python import UserAuthenticationOptions

client = SockudoClient(
    "app-key",
    SockudoOptions(
        cluster="local",
        user_authentication=UserAuthenticationOptions(
            endpoint="https://api.example.com/sockudo/user-auth",
        ),
    ),
)

await client.connect()
await client.user.sign_in()
```

The backend must bind the authenticated application user to the current
`socket_id`; never accept a user ID supplied by the client without checking the
session.

## Error handling and production checklist [#error-handling-and-production-checklist]

* Catch `SockudoException` around explicit connect, auth, proxy, and mutation
  operations; connection errors are also emitted through the connection.
* Put timeouts and authentication on every proxy endpoint.
* Use TLS and a public load-balancer hostname outside local development.
* Keep callbacks fast and move blocking work off the asyncio event loop.
* Make handlers idempotent because reconnect and application retries can repeat
  work even when transport deduplication is enabled.
* On a failed resume or unrecoverable delta, reload authoritative state.
* Unsubscribe unused channels and disconnect cleanly during process shutdown.
* Use the HTTP [Python server SDK](/docs/server-sdks/python) for publishing,
  signing auth, webhooks, history administration, and push.


# Swift (/docs/clients/swift)



`SockudoSwift` is the official realtime client for Apple platforms. It supports public, private, presence, encrypted channels, auth endpoints, V2 recovery, rewind, filters, deltas, mutable messages, and proxy-backed history helpers.

Supported deployment targets are iOS 13+, macOS 10.15+, tvOS 13+, watchOS
6+, and visionOS 1+. The package uses Swift 6.2 concurrency and isolates the
client, channels, and callbacks on `@SockudoActor`.

## Install [#install]

Install through Swift Package Manager from the SockudoSwift mirror. SwiftPM resolves this from
the `v3.0.0` Git tag and the mirror repository's root `Package.swift` manifest:

```swift
.package(url: "https://github.com/sockudo/sockudo-swift", from: "3.0.0")
```

```swift
.target(
    name: "YourApp",
    dependencies: [
        .product(name: "SockudoSwift", package: "sockudo-swift"),
    ]
)
```

## Connect [#connect]

```swift
import SockudoSwift

let client = try SockudoClient(
    "app-key",
    options: .init(
        cluster: "local",
        forceTLS: false,
        enabledTransports: [.ws],
        wsHost: "127.0.0.1",
        wsPort: 6001,
        wssPort: 6001,
        protocolVersion: 2,
        connectionRecovery: true
    )
)

let channel = client.subscribe("public-updates")
channel.bind("price-updated") { data, _ in
    print(data ?? "")
}

client.connect()
```

Protocol V1 compatibility is the default. Set `protocolVersion: 2` explicitly
for new Sockudo applications that need continuity metadata and V2 features.
Use the public ingress hostname with `forceTLS: true` in production.

## Concurrency and lifecycle [#concurrency-and-lifecycle]

From code outside `@SockudoActor`, access client methods with `await`:

```swift
func startRealtime() async {
    await client.connect()
}

func stopRealtime() async {
    await client.unsubscribe("public-updates")
    await client.disconnect()
}
```

Event callbacks execute on the dedicated Sockudo actor, not the main thread.
Hop to `MainActor` for UIKit or SwiftUI state:

```swift
channel.bind("price-updated") { data, _ in
    Task { @MainActor in
        viewModel.latestPrice = String(describing: data)
    }
}
```

Keep the `EventBindingToken` returned by `bind` when you want to remove one
handler with `unbind(eventName:token:)`; call `unbindAll()` only when the
channel owner is being torn down.

Unexpected disconnects reconnect with bounded quadratic backoff. Tune
`maxReconnectAttempts` and `maxReconnectGapInSeconds` for the app's foreground
and background policy. The attempt counter resets after a successful connection
and on explicit `connect()` or `disconnect()` calls. Client events emitted while
disconnected are buffered up to 50 per channel and replayed after subscription;
unsubscribing clears that buffer.

## Auth [#auth]

```swift
let client = try SockudoClient(
    "app-key",
    options: .init(
        cluster: "local",
        forceTLS: false,
        wsHost: "127.0.0.1",
        wsPort: 6001,
        channelAuthorization: .init(
            endpoint: "https://api.example.com/sockudo/auth"
        )
    )
)
```

The backend must authenticate the current user and authorize the exact
`channel_name` before signing. Presence identity must come from the server-side
session, not a value supplied by the app.

Protocol V2 also supports scoped capability tokens:

```swift
let client = try SockudoClient(
    "app-key",
    options: .init(
        cluster: "local",
        protocolVersion: 2,
        forceTLS: true,
        wsHost: "realtime.example.com",
        capabilityToken: .init(asyncProvider: {
            try await tokenService.fetchSockudoToken()
        })
    )
)
```

JWTs with expiry metadata refresh proactively, and providers are called before
reconnects. Only expiry code `40142` refreshes in place; revoked and static
tokens are not resent. Token auth is rejected outside Protocol V2.

## Presence [#presence]

Subscribe as `PresenceChannel` to access members and V2 in-place presence
updates:

```swift
let presence = client.subscribe("presence-agent:session-123") as! PresenceChannel

presence.bind("sockudo:member_added") { member, _ in
    print(member as Any)
}
presence.bind("sockudo:presence_update") { member, _ in
    print(member as Any)
}

try presence.update(data: ["status": "thinking"])
```

Wait for the subscription-success event before treating the initial membership
set as authoritative.

## Filters and deltas [#filters-and-deltas]

```swift
let channel = client.subscribe(
    "price:btc",
    options: .init(
        filter: .eq("market", "spot"),
        delta: .init(enabled: true, algorithm: .xdelta3),
        events: ["price.updated"],
        expression: .source("data.price >= `100`")
    )
)
```

## Recovery and rewind [#recovery-and-rewind]

```swift
let channel = client.subscribe(
    "market:BTC",
    options: .init(rewind: .seconds(30))
)

channel.bind("message") { _, _ in
    print(client.recoveryPosition(for: "market:BTC") as Any)
}

client.bind("sockudo:resume_success") { data, _ in
    print(data as Any)
}
```

## Mutable messages [#mutable-messages]

```swift
var state: MutableMessageState? = nil

let channel = client.subscribe("chat:room-1")
channel.bindGlobal { _, data in
    guard
        let event = data as? SockudoEvent,
        isMutableMessageEvent(event)
    else { return }

    state = try? reduceMutableMessageEvent(current: state, event: event)
}
```

## Presence history proxy [#presence-history-proxy]

```swift
let client = try SockudoClient(
    "app-key",
    options: .init(
        cluster: "local",
        forceTLS: false,
        wsHost: "127.0.0.1",
        wsPort: 6001,
        presenceHistory: .init(
            endpoint: "https://api.example.com/sockudo/presence-history"
        )
    )
)

let channel = client.subscribe("presence-lobby") as! PresenceChannel
channel.history(.init(limit: 50, direction: "newest_first")) { result in
    print(result)
}
```

History endpoints are backend proxies because a mobile client must not sign
Sockudo REST requests. Authorize the caller and channel on every request, bound
page sizes, and forward opaque cursors unchanged.

## Encrypted channels [#encrypted-channels]

`private-encrypted-*` channels decrypt automatically when the protected-channel
auth response contains a derived `shared_secret`:

```swift
let encrypted = client.subscribe("private-encrypted-documents")
encrypted.bind("doc-updated") { payload, _ in
    print(payload as Any)
}
```

Keep the encryption master key on the backend. Continue to use TLS and normal
channel authorization.

## Push notifications on Apple platforms [#push-notifications-on-apple-platforms]

Use APNs to obtain a device token, then send it to your backend. Your backend registers the device with Sockudo and keeps APNs credentials server-side.

```swift
func application(
    _ application: UIApplication,
    didRegisterForRemoteNotificationsWithDeviceToken deviceToken: Data
) {
    let token = deviceToken.map { String(format: "%02x", $0) }.joined()
    Task {
        await registerDeviceWithBackend(
            deviceId: UIDevice.current.identifierForVendor?.uuidString ?? token,
            platform: "apns",
            providerToken: token
        )
    }
}
```

Do not embed APNs private keys or Sockudo app secrets in the app.

## Production checklist [#production-checklist]

* Use one long-lived client for the app session and disconnect it on explicit
  sign-out.
* Select Protocol V2 deliberately; do not assume the default changed during a
  Pusher migration.
* Move UI mutations from `@SockudoActor` to `MainActor`.
* Apply an app-specific reconnect limit for backgrounded mobile processes.
* Treat failed recovery as a signal to reload authoritative application state.
* Keep auth, history, versioned-message, and push proxy credentials on the
  backend.
* Unbind view-owned callbacks to prevent retained views and duplicate updates.
* Test offline/online transitions, token expiry, app suspension, and a rolling
  Sockudo restart.


# Capacity planning and benchmarks (/docs/deployment/capacity-planning)



There is no single “connections per node” number. A quiet public-channel socket, a presence member,
a filtered Protocol V2 subscriber, and a slow client receiving large payloads have different memory,
CPU, and network costs. Capacity is the lowest limit across Sockudo, the process, the node, the load
balancer, shared backends, and the client generator.

## Name the workload [#name-the-workload]

Always report at least these independent dimensions:

| Dimension                       |              Example | Why it changes the result                                      |
| ------------------------------- | -------------------: | -------------------------------------------------------------- |
| Concurrent connected sockets    |              100,000 | Drives file descriptors, memory, LB state, and heartbeat work. |
| New connections per second      |              2,000/s | Drives TCP/TLS handshake, auth, and reconnect CPU.             |
| Subscriptions per socket        |                    5 | Drives local channel state and adapter subscription behavior.  |
| Subscribe/unsubscribe cycles    |                500/s | Drives churn and presence/count coordination.                  |
| Backend publishes per second    |              1,000/s | Measures ingest and adapter work.                              |
| Average subscribers per publish |                  100 | Converts publishes into 100,000 client deliveries/s.           |
| Payload size                    |                1 KiB | Drives serialization, buffers, broker traffic, and egress.     |
| Private/presence percentage     |            30% / 10% | Adds auth and membership work.                                 |
| Protocol features               | V2 recovery + deltas | Adds continuity, cache, compression, or storage work.          |
| Client RTT and TLS              |      180 ms, TLS 1.3 | Changes ramp, handshake, ack, and reconnect timing.            |

Never publish “messages per second” without saying whether it means accepted publishes, adapter
messages, or delivered client frames.

## Capacity is the minimum budget [#capacity-is-the-minimum-budget]

For one Sockudo node:

```text
safe connections = min(
  Sockudo admission limit,
  process file-descriptor budget,
  measured memory budget,
  measured CPU/latency budget,
  node and vNIC limits,
  load-balancer target limits,
  shared-backend budget
) - operational headroom
```

Use 20–30% headroom after a stable test, more when traffic is bursty or failure of one node must be
absorbed by the others.

For an `N`-node cluster that must survive one node loss:

```text
normal per-node target <= cluster peak connections / (N - 1)
```

That is an availability constraint, not just a scaling calculation. The remaining nodes also need
CPU, network, broker, and memory headroom for the reconnect burst.

## File descriptor budget [#file-descriptor-budget]

Estimate:

```text
nofile soft limit
- backend sockets
- listener, metrics, logging, and runtime descriptors
- 10–20% descriptor reserve
= maximum socket descriptor budget
```

Then set `SOCKUDO_MAX_CONNECTIONS` below that result. Verify the actual process:

```bash
pid="$(pidof sockudo)"
cat "/proc/$pid/limits" | grep -i 'open files'
find "/proc/$pid/fd" -maxdepth 1 -type l | wc -l
```

In a container or pod, run the check inside that container. Host login limits do not prove the
container process limit.

## Measure memory per workload [#measure-memory-per-workload]

Do not borrow a per-socket number from another server or another Sockudo feature set.

1. Start the exact release build, allocator, config, and container limit.
2. Hold the server idle and record steady resident memory.
3. Ramp to a known socket and subscription count.
4. Wait for allocations, buffers, and metrics to settle.
5. Exercise the intended message, presence, filter, delta, and recovery workload.
6. Record both steady and peak resident memory.
7. Repeat at several counts; do not extrapolate from 100 sockets to 100,000.

Approximate the marginal result:

```text
marginal bytes per socket =
  (steady RSS at high count - baseline RSS) / active sockets
```

Use peak RSS, not only marginal steady memory, when setting a cgroup or pod memory limit. Include
allocator fragmentation, slow-client buffers, adapter queues, recovery buffers, and rolling-deploy
overlap.

An OOM kill is a correlated disconnect event. Leave enough margin to avoid turning a transient
fanout burst into a reconnect storm.

## CPU and event throughput [#cpu-and-event-throughput]

Separate tests for:

* connection and TLS handshake rate
* public subscribe/unsubscribe churn
* private and presence authentication
* presence first-join and last-leave transitions
* backend publish admission
* fanout deliveries
* delta compression and tag filtering
* durable history and version writes
* replay and reconnect recovery
* webhook and push queue work

A workload at 1,000 publishes/s with one subscriber is not comparable to 1,000 publishes/s with
10,000 subscribers.

Approximate client delivery rate:

```text
deliveries/s = publishes/s × average matching subscribers per publish
```

Adapter and serialization cost may scale with publishes while egress and per-client write cost scale
with deliveries.

## Network budget [#network-budget]

Approximate payload egress before protocol, TLS, and TCP overhead:

```text
payload egress bytes/s =
  publishes/s × matching subscribers × average delivered payload bytes
```

Also account for:

* WebSocket frame and Sockudo/Pusher envelope size
* TLS and TCP/IP overhead
* heartbeat traffic for every quiet socket
* retransmissions at the measured client RTT and loss
* cross-zone or cross-node adapter traffic
* metrics, webhook, history, and push traffic

Cloud VM “up to” bandwidth and packets-per-second limits can produce a sharp knee. Inspect vNIC and
load-balancer metrics while increasing load.

## The load generator is often the first limit [#the-load-generator-is-often-the-first-limit]

A remote benchmark can fail without Sockudo being saturated.

Check every generator for:

* CPU and scheduler saturation
* memory and garbage collection
* its own `nofile` limit
* local ephemeral port exhaustion
* `TIME_WAIT` accumulation between repeated runs
* source NAT or gateway connection tracking
* NIC bandwidth and packets per second
* DNS and TLS handshake rate
* dropped iterations or failure to maintain the requested arrival rate

One source IP connecting to one target IP and port has a finite TCP 4-tuple space, commonly much
less than a high-density Sockudo node can accept. To test beyond it, shard across generators, source
IPs, or target addresses. Do not “solve” a generator's ephemeral ports by changing
`ip_local_port_range` on the Sockudo server.

Inspect a Linux generator:

```bash
ulimit -n
sysctl net.ipv4.ip_local_port_range
ss -s
nstat -az | grep -E 'TCPSynRetrans|TCPAbort|ListenDrop'
sar -n DEV,TCP,ETCP 1
```

If achieved request rate falls while Sockudo CPU, memory, network, queues, and latency remain flat,
the server is probably not the limiting system.

## Geography changes the benchmark [#geography-changes-the-benchmark]

Record both generator and server region. “Clients in Taiwan” is not a reproducible topology:
document the cloud, city/region, ISP or cloud network, public or private path, RTT distribution, and
packet loss.

Long RTT changes:

* how quickly sockets can be established during a fixed ramp
* TLS handshake duration
* subscribe/auth acknowledgement latency
* retransmission recovery
* reconnect convergence after a failure

For geographically distributed users:

1. run generators in the actual client regions
2. synchronize their start time
3. report each region separately as well as the aggregate
4. keep server-side publish load independent from client-region socket generation
5. compare public internet and private cloud paths only when clearly labeled

A short test may end before a high-RTT generator reaches the intended connection count. Report
achieved active sockets over time, not only configured virtual users.

## Use distinct benchmark phases [#use-distinct-benchmark-phases]

### 1. Connect ramp [#1-connect-ramp]

Measure connection success, TLS and WebSocket upgrade latency, admission rejections, and achieved
connections per second. Add jitter so the default test is not an unrealistic simultaneous SYN
flood, then run a separate reconnect-storm test intentionally.

### 2. Quiet-socket soak [#2-quiet-socket-soak]

Hold target connections long enough to observe memory stability, heartbeats, load-balancer idle
behavior, NAT state, and unexpected platform timeouts.

### 3. Subscription workload [#3-subscription-workload]

Apply the real channel cardinality, subscriptions per socket, private/presence mix, and room churn.

### 4. Publish and fanout [#4-publish-and-fanout]

Ramp accepted publishes and matching subscribers independently. Track delivery correctness and
end-to-end p50, p95, and p99 latency.

### 5. Failure and recovery [#5-failure-and-recovery]

Remove a Sockudo node, interrupt the adapter, and slow a durable store. Measure reconnect time,
recovery success, duplicates, gaps, readiness changes, and backlog drain.

### 6. Cooldown [#6-cooldown]

Allow connections and `TIME_WAIT` state to clear, or start from a fresh generator fleet. Back-to-back
runs without cooldown are not independent.

## Repository benchmarks [#repository-benchmarks]

### Subscription churn [#subscription-churn]

`benches/subscription-churn.js` models room switching. Churn rate is approximately:

```text
cycles/s = VUS / (ROOM_SWITCH_INTERVAL_MS / 1000)
```

For 10,000 sockets and about 500 unsubscribe/subscribe cycles per second:

```bash
k6 run \
  -e WS_HOSTS=wss://ws.example.com/app/app-key \
  -e VUS=10000 \
  -e CHANNEL_COUNT=4000 \
  -e ROOM_SWITCH_INTERVAL_MS=20000 \
  -e SOCKET_LIFETIME_MS=600000 \
  -e DURATION=10m \
  benches/subscription-churn.js
```

Shard `WS_HOSTS` or run several generators when one process or source IP approaches its limit.

### Horizontal adapter fanout [#horizontal-adapter-fanout]

`benches/adapter-horizontal.js` records successful publishes, client deliveries, delivery ratio,
and latency:

```bash
SUBSCRIBERS=900 \
CHANNELS=100 \
MESSAGES_PER_SECOND=1000 \
WARMUP_SECONDS=30 \
DURATION_SECONDS=120 \
DRAIN_SECONDS=20 \
RESULT_PATH=/tmp/sockudo-adapter.json \
k6 run --quiet benches/adapter-horizontal.js
```

Use the same image, nodes, subscriber distribution, payload, warmup, and generator placement when
comparing adapters.

## Diagnose the first knee [#diagnose-the-first-knee]

| Observation                                   | Likely area                                    | Next evidence                                                        |
| --------------------------------------------- | ---------------------------------------------- | -------------------------------------------------------------------- |
| Connect failures, Sockudo mostly idle         | Generator, LB, firewall, NAT, SYN path         | Generator ports/CPU, LB target errors, SYN retransmits, listen drops |
| `ListenOverflows` rises                       | Host listen/accept path                        | CPU, `somaxconn`, SYN backlog, connection ramp shape                 |
| High Sockudo CPU, low broker latency          | Protocol, auth, compression, filtering, fanout | Flamegraph/profile and feature-isolated tests                        |
| Low Sockudo CPU, broker latency/backlog rises | Adapter, queue, cache, or database             | Backend CPU, connections, command latency, network                   |
| Memory climbs after clients stabilize         | Buffers, retained state, leak, slow clients    | Heap/RSS profile, send queues, feature comparison                    |
| P99 rises, throughput still correct           | Queueing before saturation                     | Per-stage latency and utilization; reduce offered load               |
| Delivery ratio falls but publishes succeed    | Fanout, slow clients, generator receive path   | Per-node delivery counters, socket errors, client CPU                |
| Reconnect test collapses only after node loss | Insufficient failure headroom                  | Remaining-node admission, LB distribution, backend burst             |

Stop increasing the offered load at the first sustained latency or correctness failure. The
maximum technically accepted rate beyond that point is not the production capacity.

## Benchmark report template [#benchmark-report-template]

Publish enough information for another operator to reproduce the result:

```text
Sockudo version and commit:
Build features and release profile:
Allocator and container image:
Server region, instance type, vCPU, RAM, NIC:
Replica count and per-node max_connections:
Load balancer and timeout:
Adapter, cache, queue, app manager, history:
Backend topology and region:
Protocol version and enabled features:
Generator version, count, regions, source IP count:
RTT p50/p95 and packet loss:
TLS termination path:
Connection ramp and achieved sockets:
Subscriptions, channels, presence/private mix, churn:
Publish rate, matching subscribers, delivery rate:
Payload and wire size:
Duration, warmup, drain, and cooldown:
Success/error/duplicate/gap counts:
Latency p50/p95/p99:
Sockudo, generator, LB, node, and backend utilization:
```

Without that context, benchmark numbers are anecdotes.

## Set the production limit [#set-the-production-limit]

After the stable knee is known:

1. choose a point below the first latency, error, or resource cliff
2. subtract failure and rollout headroom
3. set `SOCKUDO_MAX_CONNECTIONS` per node
4. configure load-balancer and autoscaling thresholds to add capacity before admission rejects
5. alert on connection utilization, rejection rate, memory, CPU throttling, network, adapter
   latency, and recovery failures
6. rerun after changing the binary, features, instance type, kernel, runtime, CNI, load balancer, or
   shared backend

Capacity belongs to the complete deployment, not the Sockudo process in isolation.

## API pod topology [#api-pod-topology]

When HTTP publish traffic is large relative to WebSocket connection count, running
separate API pods (`server_role = "api"`) isolates publish latency and cuts memory
on the API tier. See [server role configuration](/docs/reference/configuration#server-role)
for setup and requirements.

Expected operational behavior:

| Metric                                         | API pod           | WS pod    |
| ---------------------------------------------- | ----------------- | --------- |
| `sockudo_horizontal_broadcast_published_total` | > 0               | varies    |
| `sockudo_horizontal_broadcast_received_total`  | 0                 | > 0       |
| `sockudo_horizontal_request_received_total`    | 0                 | > 0       |
| WebSocket connections                          | 0                 | > 0       |
| Memory (100K+ connections)                     | \~50 MiB baseline | \~2–4 GiB |

API pods report `"server_role": "api"` in `/stats` so dashboards can distinguish
them from WS pods reporting zero connections.


# Cloud platforms (/docs/deployment/cloud-platforms)



Sockudo uses the same runtime model on every cloud: long-lived client connections at the edge,
shared fanout and coordination between instances, and optional durable stores. Provider services
change the failure modes and tuning controls, not those requirements.

## Platform map [#platform-map]

| Need                          | AWS                              | Google Cloud                          | Azure                                                   |
| ----------------------------- | -------------------------------- | ------------------------------------- | ------------------------------------------------------- |
| Direct VM control             | EC2 Auto Scaling group           | Compute Engine managed instance group | Virtual Machine Scale Sets                              |
| Managed Kubernetes            | EKS                              | GKE Standard or Autopilot             | AKS                                                     |
| Managed containers            | ECS, including Fargate           | Cloud Run                             | Azure Container Apps                                    |
| WebSocket ingress             | ALB or NLB                       | External Application Load Balancer    | Application Gateway, Load Balancer, or platform ingress |
| Redis-compatible shared state | Managed Redis-compatible service | Memorystore for Redis                 | Managed Redis-compatible service                        |
| Relational app/history store  | RDS or Aurora                    | Cloud SQL or AlloyDB                  | Azure Database for PostgreSQL/MySQL                     |
| Native fanout option          | Redis, MSK, or self-managed NATS | Google Pub/Sub adapter                | Redis, Kafka-compatible service, or self-managed NATS   |

The table is a starting map, not a requirement to use every managed service. Keep latency-sensitive
dependencies in the same region as Sockudo and test cross-zone or cross-region traffic costs.

## AWS [#aws]

### Recommended shapes [#recommended-shapes]

| Shape                  | Use when                                                                 | Notes                                                                                |
| ---------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ |
| EC2 + ALB/NLB          | You need full kernel, `rlimit`, NIC, and process control.                | Best match for very high connection density and stable workloads.                    |
| EKS managed node group | You already operate Kubernetes and need pod-level rollout and placement. | Use a dedicated node pool when tuning or noisy neighbors matter.                     |
| ECS on EC2             | You want task scheduling with host control.                              | Set task `ulimits`; tune the EC2 host separately.                                    |
| ECS Fargate            | Moderate workloads and low node-operations overhead.                     | Validate supported `ulimits`, system controls, ENI, and per-task connection density. |

For a regional cluster, put Sockudo targets in at least two Availability Zones. Use one shared
adapter and cache across all replicas. Keep a minimum target count available during deployments.

### Application Load Balancer [#application-load-balancer]

Application Load Balancers support WebSockets natively. Their idle timeout defaults to 60 seconds,
which is too close to many heartbeat configurations. Set it deliberately and test through every
proxy layer:

```bash
aws elbv2 modify-load-balancer-attributes \
  --load-balancer-arn "$SOCKUDO_ALB_ARN" \
  --attributes Key=idle_timeout.timeout_seconds,Value=180
```

AWS documents the default, valid range, and keepalive interaction in
[Application Load Balancer attributes](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/edit-load-balancer-attributes.html).
ALB access logs identify `ws` and `wss` requests and are useful for separating target disconnects
from client-side failures.

Configure:

* target health on `/up/<app-id>` and process monitoring on `/live`
* deregistration delay and service termination grace as one drain budget
* security groups that expose the Sockudo port only to the load balancer
* metrics on target connection errors, rejected connections, and 5xx responses

### EKS on Amazon Linux 2023 [#eks-on-amazon-linux-2023]

Amazon EKS managed node groups that use launch templates require MIME multipart user data for
Amazon Linux AMIs. Amazon Linux 2023 uses `nodeadm` and the `application/node.eks.aws` content type.
AWS documents the merge behavior and required self-managed fields in
[Customize managed nodes with launch templates](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html).

This example applies a conservative node profile and allows a pod to request the otherwise unsafe
`net.core.somaxconn` sysctl:

```text
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="SOCKUDO"

--SOCKUDO
Content-Type: text/x-shellscript; charset="us-ascii"

#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail

cat > /etc/sysctl.d/99-sockudo.conf <<'EOF'
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 60
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_slow_start_after_idle = 0
EOF

sysctl --system

--SOCKUDO
Content-Type: application/node.eks.aws

apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  kubelet:
    config:
      allowedUnsafeSysctls:
        - "net.core.somaxconn"

--SOCKUDO--
```

For a managed node group, EKS merges its bootstrap data with the launch-template user data. For a
self-managed AL2023 group or a custom AMI workflow, include the required `spec.cluster.name`,
`apiServerEndpoint`, base64 certificate authority, and service CIDR described by AWS.

The two parts do different jobs:

* the shell script changes the node-wide baseline
* `allowedUnsafeSysctls` only permits a pod to request that exact sysctl in its own security context

If the pod does not request an unsafe sysctl, the allowlist alone changes nothing. Verify effective
values inside the Sockudo pod.

Do not add `/etc/security/limits.d` to EKS user data expecting it to raise a container's `nofile`.
PAM limits do not control the existing container process. Check `/proc/1/limits` inside the pod and
customize the node runtime only if the effective value is too low.

Apply this profile to a dedicated, canary node group first. A launch-template or node configuration
change replaces nodes and causes pod movement.

### AWS service choices [#aws-service-choices]

* a managed Redis-compatible service is the simplest shared adapter, cache, queue, and rate-limit
  authority for most clusters
* RDS or Aurora PostgreSQL/MySQL can hold dynamic apps and supported durable state
* DynamoDB is available for supported app/history/version paths when the matching feature is built
* use private subnets and security groups; broker and database ports should not be public
* use workload identity or instance/task roles instead of long-lived AWS access keys

## Google Cloud [#google-cloud]

### Recommended shapes [#recommended-shapes-1]

| Shape                                 | Use when                                                   | Notes                                                                       |
| ------------------------------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------- |
| Compute Engine managed instance group | Maximum connection density and host control.               | Pair with an external load balancer and health checks.                      |
| GKE Standard                          | Kubernetes with node-pool and sysctl control.              | Best GKE fit for dedicated realtime nodes.                                  |
| GKE Autopilot                         | Lower cluster operations overhead.                         | Node configuration is more constrained; verify all required controls.       |
| Cloud Run                             | Moderate connection count with forced reconnect tolerance. | WebSockets are requests with a finite timeout and per-instance concurrency. |

Google Pub/Sub is a supported horizontal adapter, but durable feature coordination still needs the
documented shared cache and stores. Choose it when managed GCP fanout fits the latency and request
pattern; do not assume it replaces every Redis responsibility.

### GKE node system configuration [#gke-node-system-configuration]

GKE Standard supports a node system configuration file for kubelet and a defined set of Linux
sysctls. Google warns that changing it recreates nodes and recommends testing with a PodDisruption
Budget. See [Customizing node system configuration](https://cloud.google.com/kubernetes-engine/docs/how-to/node-system-config).

```yaml
kubeletConfig:
  allowedUnsafeSysctls:
    - "net.core.somaxconn"

linuxConfig:
  sysctl:
    net.core.somaxconn: "65535"
    net.core.netdev_max_backlog: "16384"
```

Create a dedicated node pool with the file:

```bash
gcloud container node-pools create sockudo \
  --cluster="$GKE_CLUSTER" \
  --location="$GKE_LOCATION" \
  --system-config-from-file=gke-sockudo-system.yaml
```

GKE accepts only its documented sysctl set and ranges. Check the current provider table before
adding a value. Use an exact unsafe allowlist rather than `net.*`, then set the matching pod sysctl
only when the node baseline is insufficient.

### Cloud Run [#cloud-run]

Cloud Run supports WebSockets, but treats them as long-running HTTP requests. Google currently
documents a request timeout of up to 60 minutes, best-effort session affinity, and up to 1000
concurrent connections per container. See
[Using WebSockets on Cloud Run](https://cloud.google.com/run/docs/triggering/websockets).

For Sockudo that means:

* configure the maximum request timeout and expect forced reconnects
* use a horizontal adapter and shared stores because a reconnect may reach another instance
* enable and test Protocol V2 recovery when continuity matters
* set minimum instances for latency and capacity rather than relying on a cold scale-out
* set maximum concurrency only after measuring memory per socket and burst CPU
* do not enable end-to-end HTTP/2 for the WebSocket service
* account for active WebSocket instances in the billing model

Cloud Run is useful when those constraints are acceptable. GKE Standard or Compute Engine is a
better fit when connections should remain open indefinitely or node-level tuning is required.

### Google Cloud service choices [#google-cloud-service-choices]

* Memorystore for Redis can provide shared Redis state close to GKE or Compute Engine
* Cloud SQL or AlloyDB can provide supported relational app and durable stores
* workload identity avoids shipping service-account JSON in images
* place Sockudo, Redis, and database services in compatible regions and private networks
* report load-balancer and Cloud NAT limits separately from pod or VM limits

## Azure [#azure]

### Recommended shapes [#recommended-shapes-2]

| Shape                      | Use when                                                               | Notes                                                                               |
| -------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| Virtual Machine Scale Sets | Full host control and dense, stable sockets.                           | Configure load balancing, health probes, and rolling upgrades explicitly.           |
| AKS                        | Kubernetes rollout, node pools, identities, and managed control plane. | Use a dedicated Linux node pool for custom OS or kubelet settings.                  |
| Azure Container Apps       | Managed container ingress for moderate workloads.                      | WebSocket ingress is supported; validate request/idle timeout and scaling behavior. |

Azure Application Gateway and Azure Load Balancer have different layers, health semantics, and
timeout controls. Document the selected path and test a quiet WebSocket through the public endpoint,
not only a busy local client.

### AKS custom node configuration [#aks-custom-node-configuration]

AKS accepts separate kubelet and Linux OS configuration files when creating a cluster or node pool.
Microsoft documents the supported ranges and notes that unsafe sysctls can affect node security and
stability in [Customize AKS node configuration](https://learn.microsoft.com/azure/aks/custom-node-configuration).

`aks-kubelet.json`:

```json
{
  "allowedUnsafeSysctls": [
    "net.core.somaxconn"
  ]
}
```

`aks-linux-os.json`:

```json
{
  "sysctls": {
    "netCoreSomaxconn": 65535,
    "netCoreNetdevMaxBacklog": 16384,
    "netIpv4TcpMaxSynBacklog": 65535,
    "netIpv4TcpKeepaliveTime": 600,
    "netIpv4TcpKeepaliveIntvl": 60,
    "netIpv4TcpKeepaliveProbes": 5
  }
}
```

Create a dedicated pool:

```bash
az aks nodepool add \
  --resource-group "$AKS_RESOURCE_GROUP" \
  --cluster-name "$AKS_CLUSTER" \
  --name sockudo \
  --kubelet-config ./aks-kubelet.json \
  --linux-os-config ./aks-linux-os.json
```

AKS uses camelCase names in the JSON API even though the documentation tables show kernel names.
Provider defaults can already be high; for example, system-wide file handle ceilings on current
AKS Linux images may not be the limiting `nofile`. Always inspect the process limit inside the pod.

### Azure Container Apps [#azure-container-apps]

Azure Container Apps ingress supports WebSockets. Environment ingress settings can include an idle
request timeout, and the platform owns the underlying nodes. See
[Container Apps ingress](https://learn.microsoft.com/azure/container-apps/ingress-overview) and
[environment ingress configuration](https://learn.microsoft.com/azure/container-apps/ingress-environment-configuration).

Use the same managed-container precautions as Cloud Run:

* shared fanout and coordination from the first multi-replica deployment
* minimum replicas sized for quiet socket capacity
* tested reconnect and recovery behavior
* no dependency on host sysctl or custom container-runtime limits
* explicit ingress idle-timeout validation in the selected environment tier

Use AKS or Virtual Machine Scale Sets when connection density requires host control.

### Azure service choices [#azure-service-choices]

* use a managed Redis-compatible service for the common shared adapter/cache path
* Azure Database for PostgreSQL or MySQL can provide supported relational storage
* use managed identities and Key Vault-backed secret delivery instead of static cloud credentials
* use private endpoints or VNet integration for Redis and databases
* watch SNAT/connection tracking for outbound dependencies and load generators

## Other providers [#other-providers]

The same decision tree works on DigitalOcean, Hetzner, Oracle Cloud, Fly.io, Render, Railway, or an
on-premises Kubernetes platform:

1. Confirm native WebSocket support and the maximum idle or request duration.
2. Confirm per-instance concurrent connection, file descriptor, bandwidth, and packet limits.
3. Confirm whether you control node sysctls and process `rlimit`.
4. Confirm scale-down and connection-drain semantics.
5. Use a shared adapter before adding a second instance.
6. Keep stateful dependencies close enough for the measured latency budget.
7. Run a quiet-socket soak and reconnect storm, not only a short message-throughput test.

If the provider does not publish a socket or timeout limit, treat it as an unknown to test and
monitor, not as unlimited.


# Docker and Compose (/docs/deployment/docker)



The published Sockudo image runs as a non-root user and starts the binary with
`--config /app/config/config.json`. The bundled file is a development example so a bare
`docker run` can start safely with node-local drivers. For production, pin a release tag, mount or
generate the exact configuration, inject secrets separately, and set limits on the container
itself.

## Production-like docker run [#production-like-docker-run]

```bash
docker run -d \
  --name sockudo \
  --restart unless-stopped \
  --init \
  --stop-timeout 45 \
  --ulimit nofile=262144:262144 \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --mount type=bind,src="$PWD/config.toml",dst=/app/config/production.toml,readonly \
  --env-file "$PWD/sockudo.env" \
  -p 6001:6001 \
  -p 127.0.0.1:9601:9601 \
  ghcr.io/sockudo/sockudo:5.0.1 \
  sockudo --config /app/config/production.toml
```

The arguments after the image replace its default command, so include both `sockudo` and
`--config`. Mount metrics only on a private or loopback interface unless the network policy already
restricts it.

The `CONFIG_FILE` environment value embedded in the image is descriptive; the server selects a
non-default file through `--config`.

## Compose profile [#compose-profile]

This example keeps the Sockudo process immutable and connects it to an external or separately
managed Redis:

```yaml
name: sockudo

services:
  sockudo:
    image: ghcr.io/sockudo/sockudo:5.0.1
    command: ["sockudo", "--config", "/app/config/production.toml"]
    restart: unless-stopped
    init: true
    stop_grace_period: 45s
    read_only: true
    tmpfs:
      - /tmp:size=64m,mode=1777
    ulimits:
      nofile:
        soft: 262144
        hard: 262144
    ports:
      - "6001:6001"
      - "127.0.0.1:9601:9601"
    volumes:
      - ./config/production.toml:/app/config/production.toml:ro
    env_file:
      - ./secrets/sockudo.env
    environment:
      HOST: "0.0.0.0"
      PORT: "6001"
      METRICS_HOST: "0.0.0.0"
      METRICS_PORT: "9601"
      INSTANCE_PROCESS_ID: "compose-1"
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://127.0.0.1:6001/up/production"]
      interval: 10s
      timeout: 3s
      start_period: 20s
      retries: 3
    logging:
      driver: local
      options:
        max-size: "20m"
        max-file: "5"
```

For local development, adding Redis to the same Compose project is convenient. In production, a
single-host Redis container has the same host failure domain as Sockudo. Use a managed or
independently operated Redis topology when the shared adapter, cache, queue, or rate limiter must
survive that host.

## Configuration and secrets [#configuration-and-secrets]

Prefer this separation:

* mount the complete non-secret TOML or JSON read-only
* supply secret-backed environment variables through the orchestrator
* mount TLS keys, provider credentials, and service-account JSON as read-only secret files
* reference secret file paths from Sockudo configuration or environment

An `env_file` is still a plaintext secret file on the Docker host. Limit its owner and mode, keep it
out of image build context and source control, and rotate it through the host's secret-management
workflow.

Do not use Docker build arguments for credentials. Build arguments and layers can remain visible in
image metadata or caches.

## Limits: host versus container [#limits-host-versus-container]

Three different layers can cap a container:

| Layer                           | Check                                   | Configure                                    |
| ------------------------------- | --------------------------------------- | -------------------------------------------- |
| Sockudo admission               | `SOCKUDO_MAX_CONNECTIONS`               | Static config or environment                 |
| Process `nofile`                | `docker exec sockudo sh -c 'ulimit -n'` | `--ulimit` or Compose `ulimits`              |
| Host kernel and service manager | `sysctl`, Docker daemon unit limits     | Host image, sysctl, and daemon configuration |

Changing `/etc/security/limits.d` inside the image does not raise the running container's file
limit. Set it through the runtime and verify from inside the container.

Host-level connection queues and NIC settings remain host concerns. Namespaced sysctls may be set
per container, but only after confirming the runtime supports them and the value is safe for other
workloads.

## CPU and memory [#cpu-and-memory]

Start with a memory limit that leaves room for peak sockets, payload buffers, adapter queues, and
allocator fragmentation. An out-of-memory kill disconnects every client on that container at once.

CPU quotas can create latency cliffs during reconnect or fanout bursts. Set CPU reservations, watch
throttling metrics, and add a hard CPU limit only when multi-tenant isolation requires one and the
load test includes the same quota.

Measure:

```bash
docker stats sockudo
docker inspect sockudo --format '{{json .HostConfig.Ulimits}}'
docker exec sockudo sh -c 'ulimit -n; cat /proc/1/limits'
docker exec sockudo sh -c 'curl -fsS http://127.0.0.1:6001/live'
```

## Health and startup ordering [#health-and-startup-ordering]

Use `/live` when the runtime needs to decide whether the process is alive. Use `/up` or
`/up/<app-id>` when the load balancer needs dependency-aware readiness.

Compose `depends_on` can order startup, but it does not make a dependency permanently available.
Sockudo must still expose failed readiness and operators must alert on adapter, cache, queue, and
app-manager failures.

Do not restart a healthy process merely because Redis or a database is briefly slow. Repeated
container restarts amplify reconnect load.

## Multi-host containers [#multi-host-containers]

Moving Compose services onto two VMs does not create Sockudo clustering automatically. Every
Sockudo replica must have:

* the same app identity and policy, or access to one shared app manager
* a unique `INSTANCE_PROCESS_ID`
* a shared horizontal adapter
* a shared cache for cross-node coordination and distributed limits
* durable queue and state backends where the enabled features require them
* a WebSocket-aware load balancer with readiness and connection drain

Do not place a local memory adapter behind a multi-host load balancer. Clients connected to
different hosts would occupy isolated realtime islands.

For larger container deployments, use the [Kubernetes and Helm](/docs/deployment/kubernetes)
guide. For one dedicated container host, also apply the measured host guidance from
[Linux VM and bare metal](/docs/deployment/linux).


# Deployment guide (/docs/deployment)



A good Sockudo deployment starts with the failure model, not the platform logo. Decide whether one
process is sufficient, which state must survive a restart, and whether a client may reconnect to a
different node. Those decisions determine the adapter, cache, queue, app manager, and history
backends.

<Cards>
  <Card title="Static configuration" href="/docs/deployment/static-configuration" description="Use TOML or JSON for structure, environment variables for overrides, and secret stores for credentials." />

  <Card title="Linux host" href="/docs/deployment/linux" description="Run a binary under systemd, raise file limits correctly, tune the node, and configure a reverse proxy." />

  <Card title="Docker and Compose" href="/docs/deployment/docker" description="Mount configuration, set container limits, handle secrets, and move from one host to several." />

  <Card title="Kubernetes and Helm" href="/docs/deployment/kubernetes" description="Configure probes, disruption budgets, resources, ingress, autoscaling, and pod or node sysctls." />

  <Card title="Cloud platforms" href="/docs/deployment/cloud-platforms" description="Apply the same architecture on AWS, Google Cloud, and Azure, including managed-node tuning examples." />

  <Card title="Capacity planning" href="/docs/deployment/capacity-planning" description="Budget connections, find the real bottleneck, and run geographically honest benchmarks." />
</Cards>

## Choose a topology [#choose-a-topology]

| Deployment                            | Good fit                                                         | Runtime backends                                                                               | Main limitation                                                                      |
| ------------------------------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| One process, memory only              | Local development, demos, CI                                     | Local adapter, memory cache, queue, app manager, history                                       | Restart loses process-local state; no failover.                                      |
| One production VM                     | Small stable workload with a simple operational model            | Local adapter; static apps or a durable app manager; optional durable history                  | The host is one failure domain.                                                      |
| Multiple nodes with Redis             | Most regional production workloads                               | Redis adapter, cache, queue, and rate limiter; shared app manager when apps change dynamically | Redis availability and capacity become part of the realtime path.                    |
| Multiple nodes with NATS              | High fanout or high subscription churn                           | NATS adapter, Redis cache, durable queue and app manager                                       | NATS carries fanout, but coordination and durable features still need shared stores. |
| Durable Protocol V2 cluster           | Recovery, history, mutable messages, annotations, AI Transport   | Shared adapter and cache plus durable history and version stores                               | More write amplification and more dependencies to operate.                           |
| Managed/serverless container platform | Moderate socket counts with platform-managed rollout and scaling | Shared adapter and stores from the first replica                                               | Platform connection duration, concurrency, and node-tuning limits apply.             |

Memory drivers are not shared simply because several processes use identical configuration. Every
multi-node deployment needs a horizontal adapter. Any feature whose correctness crosses nodes also
needs its documented shared authority.

## Recommended production baseline [#recommended-production-baseline]

A conventional regional cluster has:

* at least two Sockudo instances spread across failure domains
* a load balancer that supports WebSocket upgrades and has a deliberate idle timeout
* Redis or another supported horizontal adapter
* a shared Redis or Redis Cluster cache for coordination, idempotency, and distributed rate limits
* durable app storage if app records change at runtime
* a durable queue when webhook or push work must survive a node failure
* durable history and version storage when Protocol V2 recovery or mutable state must cross nodes
* metrics, structured logs, readiness checks, and a tested drain path

Start with [Scaling](/docs/server/scaling) for cross-node semantics and
[Security](/docs/server/security) for network and secret boundaries.

## Configuration strategy [#configuration-strategy]

Use three layers with distinct responsibilities:

1. Put stable structure in a version-controlled TOML or JSON file: enabled features, driver choices,
   limits, queue reliability, history retention, and per-app policy.
2. Put environment-specific addresses and non-secret switches in deployment variables or a
   generated config file.
3. Inject app secrets, database passwords, broker credentials, and provider keys from the
   platform's secret store.

Sockudo does not treat environment variables as a complete alternative syntax for every nested
configuration field. Complex policy and multi-app definitions belong in a file or persistent app
manager. See [Static configuration](/docs/deployment/static-configuration) for exact precedence and
equivalent TOML and JSON examples.

## Workload questions to answer first [#workload-questions-to-answer-first]

Document these inputs before choosing instance sizes:

| Input                                       | Why it matters                                                                      |
| ------------------------------------------- | ----------------------------------------------------------------------------------- |
| Peak concurrent sockets and reconnect surge | Sets file descriptor, memory, load-balancer, and admission budgets.                 |
| Messages per second and average payload     | Sets CPU, network, adapter, and serialization load.                                 |
| Subscribers per publish                     | Distinguishes ingest throughput from fanout throughput.                             |
| Public, private, and presence mix           | Adds authentication, membership, and transition work.                               |
| Subscription churn                          | Can dominate otherwise quiet chat or room workloads.                                |
| Protocol V1 or V2 features                  | Recovery, deltas, filtering, history, and mutations change state and compute costs. |
| Retention and recovery objectives           | Selects memory buffers versus durable stores.                                       |
| Client geography                            | Changes handshake time, RTT, reconnect behavior, and apparent benchmark throughput. |

## Rollout sequence [#rollout-sequence]

1. Build or pin one exact Sockudo image or binary version and required Cargo features.
2. Validate the resolved config in staging, including startup logs and `/up/<app-id>`.
3. Load test from realistic client regions and confirm the generators are not saturated.
4. Set a per-node `SOCKUDO_MAX_CONNECTIONS` below the measured safe ceiling.
5. Deploy at minimum capacity before enabling autoscaling.
6. Remove a node while traffic is active and verify reconnect and recovery behavior.
7. Fail or pause each shared dependency and verify readiness, alerts, and backpressure.
8. Roll in small batches while watching connection churn, adapter errors, and recovery failures.

## Production gate [#production-gate]

Do not call a deployment ready until all of these are true:

* `/live` is used for liveness and startup checks
* `/up` or `/up/<app-id>` is used for readiness
* the load balancer idle timeout and Sockudo heartbeat behavior have been tested together
* file descriptor limits have been checked in the actual process or container
* credentials are absent from images, ConfigMaps, command lines, and logs
* every replica has a stable, unique `INSTANCE_PROCESS_ID`
* multi-node drivers and feature flags pass Sockudo startup validation
* termination grace exceeds Sockudo's shutdown grace period
* load testing includes connect, steady-state, fanout, churn, and reconnect phases
* dashboards distinguish a Sockudo bottleneck from the load generator, load balancer, NAT, or broker

Continue with [Static configuration](/docs/deployment/static-configuration), then select the
platform-specific guide.


# Kubernetes and Helm (/docs/deployment/kubernetes)



Kubernetes is a good Sockudo platform when you need repeatable rollouts, failure-domain placement,
secret injection, metrics discovery, and several replicas. It does not remove the state model:
pods still need shared adapters and stores, and scale-down still disconnects their clients.

## Install the chart [#install-the-chart]

```bash
helm upgrade --install sockudo oci://ghcr.io/sockudo/charts/sockudo \
  --version 4.7.0 \
  --namespace sockudo \
  --create-namespace \
  --values values-production.yaml
```

Pin the chart and image version in GitOps or release automation. Do not deploy `latest`.

To work on the chart itself, install from a checkout instead: `helm upgrade --install sockudo
./charts/sockudo`.

## ArgoCD [#argocd]

The chart is published as an OCI artifact, so ArgoCD consumes it directly without cloning this
repository:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sockudo
  namespace: argocd
spec:
  project: default
  source:
    repoURL: ghcr.io/sockudo/charts
    chart: sockudo
    targetRevision: 4.7.0
    helm:
      valuesObject:
        replicaCount: 3
  destination:
    server: https://kubernetes.default.svc
    namespace: sockudo
  syncPolicy:
    syncOptions:
      - CreateNamespace=true
```

Note `repoURL` carries no `oci://` prefix: for a Helm chart source, ArgoCD takes the bare
registry path and infers the OCI protocol from `chart` being set.

## Configuration from a Secret [#configuration-from-a-secret]

By default the chart renders the `config.*` values into a ConfigMap and mounts it at
`/app/config/config.json`. Under GitOps that ConfigMap is generated from values held in a
repository, so it can only carry configuration that is safe to commit.

Per-app webhooks are the case where it is not. Webhooks are expressible only in the config file,
never through environment variables, so an app that has them must carry its secret in the same
file. `defaultApp.existingSecret` does not help: it configures the environment-defined default
app, which has no webhook support.

Point `config.existingSecret` at a Secret to supply the whole file instead:

```yaml
config:
  existingSecret: sockudo-config
  existingSecretKey: config.json
```

The ConfigMap is then not rendered, and the `config` volume is backed by the Secret. Combining
`config.existingSecret` with `configJson` fails the render rather than silently ignoring one of
them.

The content must be JSON: it is mounted as `config.json` and Sockudo selects its parser from the
file extension, even though the annotated reference configuration is TOML.

Helm never sees the Secret's content, so the `checksum/config` pod annotation is not emitted in
this mode. Rotating the Secret updates the mounted file but does not restart the pods, and Sockudo
reads its configuration only at startup - use a reloader controller or restart the Deployment.

This is the shape that pairs with a secret operator. With External Secrets Operator, an
`ExternalSecret` pulls the config file out of the backing store and the chart mounts the Secret it
produces:

```yaml
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: sockudo-config
  namespace: sockudo
spec:
  refreshInterval: 5m
  secretStoreRef:
    kind: ClusterSecretStore
    name: my-secret-store
  target:
    name: sockudo-config
    creationPolicy: Owner
    template:
      engineVersion: v2
      data:
        config.json: "{{ .sockudoConfig }}"
  data:
    - secretKey: sockudoConfig
      remoteRef:
        key: SOCKUDO_CONFIG_FILE
```

## Production values example [#production-values-example]

The following is a regional three-pod baseline with one immutable app sourced from a Kubernetes
Secret and Redis used for fanout, cache, queue, and rate limits:

```yaml
replicaCount: 3

image:
  repository: ghcr.io/sockudo/sockudo
  tag: "5.0.1"
  pullPolicy: IfNotPresent

config:
  mode: production
  debug: false
  logOutputFormat: json
  rustLog: "info,sockudo=info"
  shutdownGracePeriod: 30
  adapterDriver: redis
  appManagerDriver: memory
  cacheDriver: redis
  queueDriver: redis
  rateLimiterDriver: redis
  rateLimiter:
    enabled: true
    apiMaxRequests: 1000
    apiWindowSeconds: 60
    apiTrustHops: 1
    wsMaxRequests: 100
    wsWindowSeconds: 60
    wsTrustHops: 1
  metrics:
    enabled: true

defaultApp:
  enabled: true
  existingSecret: sockudo-app
  maxConnections: 50000
  enableClientMessages: false
  enableUserAuthentication: true

redis:
  host: redis.internal
  port: 6379
  existingSecret: sockudo-redis

extraEnv:
  - name: SOCKUDO_MAX_CONNECTIONS
    value: "50000"
  - name: ADAPTER_FALLBACK_TO_LOCAL
    value: "false"
  - name: SOCKUDO_DEFAULT_APP_ALLOWED_ORIGINS
    value: "https://app.example.com"

resources:
  requests:
    cpu: "1"
    memory: 1Gi
  limits:
    memory: 2Gi

readinessProbe:
  httpGet:
    path: /up/production
    port: http
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

livenessProbe:
  httpGet:
    path: /live
    port: http
  periodSeconds: 20
  timeoutSeconds: 3
  failureThreshold: 3

startupProbe:
  httpGet:
    path: /live
    port: http
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 30

terminationGracePeriodSeconds: 45

pdb:
  enabled: true
  minAvailable: 2

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: sockudo

serviceMonitor:
  enabled: true
  interval: 30s
  scrapeTimeout: 10s

ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
  hosts:
    - host: ws.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: sockudo-tls
      hosts:
        - ws.example.com
```

Adapt the topology label selector when the release name or chart labels differ. If application
records must change dynamically, select a durable app manager instead of `memory` and configure its
database credentials from a Secret.

Create the one-app Secret without placing values in the shell history or a checked-in manifest.
The required keys are:

```text
default-app-id
default-app-key
default-app-secret
```

For advanced nested configuration, use the chart's `configJson` value or mount your own static
file. Keep secret values out of `configJson`; combine it with `extraEnvFrom` or file-backed Secrets.

## Probes [#probes]

Use different endpoints for different decisions:

| Probe                    | Endpoint                | Meaning                                                                    |
| ------------------------ | ----------------------- | -------------------------------------------------------------------------- |
| Startup                  | `/live`                 | The process has started.                                                   |
| Liveness                 | `/live`                 | The process runtime is alive.                                              |
| Readiness                | `/up` or `/up/<app-id>` | Required apps and shared dependencies are available.                       |
| New-connection admission | `/accept-traffic`       | The pod is not draining or shedding new connections under memory pressure. |

Do not use dependency-aware `/up` as liveness. Restarting every pod because Redis is slow creates a
reconnect storm and does not repair Redis.

Set probe timeout below its period and above normal dependency-check latency. Sockudo's
`health_check_timeout_ms` bounds each dependency check separately.

## Resources and connection admission [#resources-and-connection-admission]

Persistent connections do not map cleanly onto CPU-only autoscaling:

* memory rises with sockets, subscriptions, pending writes, and per-connection feature state
* CPU rises with handshake, auth, messages, fanout, compression, filtering, and reconnect bursts
* a pod can have low CPU while holding too many quiet sockets to remove safely

Set `SOCKUDO_MAX_CONNECTIONS` on every pod so a single replica cannot absorb more connections than
its measured memory and file-descriptor budget. Keep minimum replicas high enough for the peak
quiet-socket count even when CPU is low.

For a second guard based on actual memory use, enable the chart's memory-pressure admission control:

```yaml
config:
  httpApi:
    acceptTraffic:
      enabled: true
      memoryThreshold: 0.9
      sampleIntervalMs: 500
```

With `memoryLimitBytes: null` (the default), Sockudo discovers the pod's finite cgroup memory limit.
When process RSS reaches the configured fraction, the pod rejects only new native and Ably
WebSocket connections with `503`/retry guidance while established sessions continue. It resumes
admission after a later sample falls below the threshold. The sampler is independent of metrics
scrapes and fails open if Linux RSS or cgroup data is unavailable.

Keep `/up` as Kubernetes readiness so dependency failures remain visible. If the ingress or external
load balancer supports a separate backend-admission or network-watcher probe, point that probe at
`/accept-traffic`. Direct admission enforcement remains active even without such a probe.

CPU limits can throttle a pod during the exact reconnect or fanout burst it needs to process.
Requests plus a memory limit are a reasonable starting point on a dedicated node pool. Add a CPU
limit only when isolation requires it and benchmark with the same quota.

## Autoscaling [#autoscaling]

The chart can configure an HPA:

```yaml
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 12
  targetCPUUtilizationPercentage: 65
  targetMemoryUtilizationPercentage: 75
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
    scaleDown:
      stabilizationWindowSeconds: 600
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120
```

CPU and memory are fallback signals, not a complete realtime scaling policy. A stronger setup also
uses active connections per pod, connection admission rejections, event-loop or fanout latency,
adapter backlog, and reconnect rate through an external/custom metrics adapter.

`autoscaling.metrics` is passed through to the HPA verbatim and accepts any `autoscaling/v2` metric
type, so those signals can be expressed directly. `sockudo_connected` is the connection gauge:

```yaml
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 12
  metrics:
    - type: External
      external:
        metric:
          name: sockudo_connected
        target:
          type: AverageValue
          averageValue: "40000"
```

Setting it **replaces** the CPU and memory metrics generated from
`targetCPUUtilizationPercentage` and `targetMemoryUtilizationPercentage` rather than adding to
them. That is deliberate: an HPA acts on the highest recommendation from any metric, so leaving
CPU in place would keep overriding a connection-based policy. Put CPU back in the list explicitly
if you want both.

Scale up early. Scale down slowly and one pod at a time. Every removed pod makes its clients
reconnect, so an aggressive HPA can oscillate and manufacture load.

## Rollouts and disruptions [#rollouts-and-disruptions]

Set:

* a PodDisruptionBudget that leaves enough connection capacity online
* zone or host topology spread
* a termination grace period longer than `SHUTDOWN_GRACE_PERIOD`
* a rollout `maxUnavailable` compatible with the PDB and remaining socket capacity
* load-balancer deregistration or endpoint propagation time inside the drain budget

`strategy` is passed through to the Deployment. Left unset, Kubernetes applies its default of
`maxUnavailable: 25%` / `maxSurge: 25%`, which takes a quarter of the connection capacity offline
during a rollout - exactly while the clients from the replaced pods are reconnecting. Surging
instead keeps capacity flat:

```yaml
strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 0
    maxSurge: 25%
```

Note that a PodDisruptionBudget does not cover this. A PDB governs the eviction API - node drains
and upgrades - not Deployment rollouts.

During a rollout, watch connection count by pod. Kubernetes considers replica count, not how many
sockets each replica owns. One terminating pod may hold a disproportionate share after a long
uptime.

If strict connection drain is required, remove readiness first, wait for endpoint and load-balancer
propagation, then terminate. Test this behavior with the actual ingress controller; it is not
identical across controllers.

## Ingress and load balancers [#ingress-and-load-balancers]

Confirm all layers:

* accept HTTP/1.1 WebSocket upgrades
* preserve `Upgrade` and `Connection` semantics
* have idle timeouts above the heartbeat interval with margin
* do not buffer WebSocket traffic
* expose a dependency-aware readiness path
* drain targets during pod termination

The ingress annotation example above is specific to ingress-nginx. Use the equivalent settings for
AWS Load Balancer Controller, Google Cloud Load Balancing, Azure Application Gateway, Traefik,
Envoy, HAProxy, or a service mesh.

Sticky sessions are not required for basic pub/sub with a shared adapter. They can reduce
reconnect-to-new-node churn, but must not substitute for shared state.

When preserving client IPs with `externalTrafficPolicy: Local` or proxy-protocol features, retest
load distribution and health checks. Source-IP preservation can reduce the set of eligible nodes.

## Pod and node sysctls [#pod-and-node-sysctls]

First deploy with provider defaults and inspect counters. Managed Kubernetes node images already
ship with non-trivial tuning, and an unnecessary override can reduce stability.

Kubernetes classifies sysctls as safe or unsafe. Safe, namespaced values can be applied in the pod
security context when the cluster version and admission policy allow them:

```yaml
podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000
  sysctls:
    - name: net.ipv4.tcp_keepalive_time
      value: "600"
    - name: net.ipv4.tcp_keepalive_intvl
      value: "60"
    - name: net.ipv4.tcp_keepalive_probes
      value: "5"
```

`net.core.somaxconn` is commonly treated as unsafe. A pod-level value requires both:

1. an exact kubelet `allowedUnsafeSysctls` entry on every eligible node
2. the matching `podSecurityContext.sysctls` value and an admission policy that permits it

See Kubernetes' [sysctl documentation](https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/)
and the provider-specific node-pool guide before enabling it. Do not grant a privileged container
or broad `net.*` permission just to set one value.

Node-level sysctl configuration and pod network namespaces are distinct. Verify the effective value
inside the scheduled pod:

```bash
kubectl exec -n sockudo deploy/sockudo -- cat /proc/sys/net/core/somaxconn
kubectl exec -n sockudo deploy/sockudo -- sh -c 'ulimit -n; cat /proc/1/limits'
```

Kubernetes has no portable pod field for POSIX `rlimit` values. The effective `nofile` comes from
the container runtime and node image. If it is too low, change the runtime/node configuration or
use a provider-supported node image customization, then verify again inside the pod.

Provider examples are in [Cloud platforms](/docs/deployment/cloud-platforms).

## Network capacity [#network-capacity]

High socket counts can hit infrastructure before the pod:

* load-balancer connection and target limits
* node vNIC packets-per-second or bandwidth limits
* CNI IP allocation and conntrack tables
* NAT gateway state for outbound dependencies or load generators
* cross-zone traffic and broker latency

Inbound WebSockets do not consume a Sockudo pod's local ephemeral port range. Avoid “fixing”
`ip_local_port_range` unless evidence points to an outbound connection or NAT bottleneck.

## Verify the deployed state [#verify-the-deployed-state]

```bash
helm get values sockudo -n sockudo
kubectl get pods -n sockudo -o wide
kubectl get pdb -n sockudo
kubectl exec -n sockudo deploy/sockudo -- sh -c 'ulimit -n'
kubectl port-forward -n sockudo svc/sockudo 6001:6001
curl -fsS http://127.0.0.1:6001/live
curl -fsS http://127.0.0.1:6001/up/production
```

Then run a node drain and a rolling restart under load. A manifest that renders successfully is not
yet a tested realtime deployment.


# Linux VM and bare metal (/docs/deployment/linux)



A Linux VM is often the simplest production deployment for a known, stable workload. It gives you
direct control over file limits, kernel settings, CPU scheduling, network queues, and the process
lifecycle. Start with one host only if a host restart is an acceptable outage; use at least two
hosts and a shared adapter for availability.

## Host layout [#host-layout]

Use separate paths for the binary, configuration, secrets, and service account:

```text
/usr/local/bin/sockudo
/etc/sockudo/config.toml
/etc/sockudo/sockudo.env
/etc/systemd/system/sockudo.service
```

The service account needs read access to its configuration and secret-backed files. Sockudo does
not need root privileges when it binds to ports `6001` and `9601`.

```bash
sudo useradd --system --home /var/lib/sockudo --create-home --shell /usr/sbin/nologin sockudo
sudo install -o root -g root -m 0755 target/release/sockudo /usr/local/bin/sockudo
sudo install -d -o root -g sockudo -m 0750 /etc/sockudo
sudo install -o root -g sockudo -m 0640 config/production.toml /etc/sockudo/config.toml
```

Keep `/etc/sockudo/sockudo.env` at mode `0640` or stricter. Do not put secrets directly in the
systemd unit because unit contents are commonly collected in diagnostics and configuration
management.

## systemd unit [#systemd-unit]

```ini
[Unit]
Description=Sockudo realtime server
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=sockudo
Group=sockudo
WorkingDirectory=/var/lib/sockudo
EnvironmentFile=-/etc/sockudo/sockudo.env
ExecStart=/usr/local/bin/sockudo --config /etc/sockudo/config.toml
Restart=on-failure
RestartSec=2s
LimitNOFILE=1048576
TimeoutStopSec=45s
KillSignal=SIGTERM
NoNewPrivileges=true
PrivateTmp=true
PrivateDevices=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictSUIDSGID=true
LockPersonality=true

[Install]
WantedBy=multi-user.target
```

Set `TimeoutStopSec` longer than `shutdown_grace_period`. Add `ReadWritePaths=` only when a selected
feature genuinely writes to local disk. A stricter security directive may need adjustment for a
provider SDK or a TLS key path; keep any exception narrow.

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now sockudo
sudo systemctl status sockudo
```

## File descriptor limits [#file-descriptor-limits]

Every accepted socket consumes a file descriptor, as do logs, shared-backend connections, metrics,
and internal listeners. Budget:

```text
required nofile >= planned sockets + backend connections + listeners + operational reserve
```

Use at least 10–20% reserve and never set `SOCKUDO_MAX_CONNECTIONS` equal to the hard `nofile`
limit.

`/etc/security/limits.d/*.conf` applies to PAM login sessions. It does not reliably change a
systemd service. `LimitNOFILE=` in the unit is the relevant setting.

Verify the running process rather than the shell:

```bash
systemctl show sockudo -p LimitNOFILE
cat /proc/"$(pidof sockudo)"/limits | grep -i 'open files'
ls /proc/"$(pidof sockudo)"/fd | wc -l
```

Also check system-wide pressure:

```bash
sysctl fs.file-max fs.nr_open
cat /proc/sys/fs/file-nr
```

## Baseline kernel profile [#baseline-kernel-profile]

Treat this as a starting profile for a dedicated Sockudo node, not a universal requirement:

```ini
# /etc/sysctl.d/99-sockudo.conf

# Listen and SYN queues for connection bursts.
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

# Input backlog; raise only when packet drops show the default is too small.
net.core.netdev_max_backlog = 16384

# Detect dead peers eventually at the TCP layer. Sockudo protocol heartbeats
# remain the primary application-level liveness mechanism.
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 60
net.ipv4.tcp_keepalive_probes = 5

# Avoid restarting congestion control from the initial window after idle.
net.ipv4.tcp_slow_start_after_idle = 0
```

Apply and verify:

```bash
sudo sysctl --system
sysctl net.core.somaxconn net.ipv4.tcp_max_syn_backlog
sysctl net.core.netdev_max_backlog
sysctl net.ipv4.tcp_keepalive_time net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes
```

Why these are not all “set once and forget” values:

* a larger `somaxconn` helps only if the application's listen backlog and accept loop can use it
* a larger SYN queue helps connection bursts, not steady-state message fanout
* a larger `netdev_max_backlog` can trade drops for latency and memory when CPU is already saturated
* TCP keepalive does not replace Sockudo's protocol ping/pong and should not fire more aggressively
  than necessary
* modern Linux already autotunes TCP buffers; raising `rmem_max` and `wmem_max` without high
  bandwidth-delay-product evidence can increase per-socket memory risk

Optional settings such as `tcp_fin_timeout`, socket buffer ceilings, IRQ affinity, CPU isolation,
busy polling, and NIC ring sizes should follow a measured bottleneck. Record the before/after
kernel counters and repeat the same workload.

Do not widen `ip_local_port_range` to increase inbound WebSocket capacity. Inbound sockets use the
server's listening port. The ephemeral range matters for high-volume outbound connections from
Sockudo, a NAT gateway, or the load generator.

## Inspect the network path [#inspect-the-network-path]

Useful counters during a connect or reconnect surge:

```bash
ss -s
ss -lnt
nstat -az | grep -E 'ListenOverflows|ListenDrops|TCPBacklogDrop|TCPSynRetrans'
ip -s link
ethtool -S eth0
```

Interpret them together:

* `ListenOverflows` or `ListenDrops` suggests the accept/listen path cannot keep up
* interface drops suggest host networking, vNIC, or CPU pressure
* SYN retransmits can be the client network, load balancer, firewall, or server
* no server-side drops with low achieved load usually points upstream or at the generator

## Reverse proxy example [#reverse-proxy-example]

Terminate TLS at a load balancer or a carefully configured reverse proxy. NGINX needs HTTP/1.1
upgrade forwarding and long enough timeouts:

```nginx
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream sockudo {
    least_conn;
    server 10.0.1.10:6001 max_fails=3 fail_timeout=10s;
    server 10.0.2.10:6001 max_fails=3 fail_timeout=10s;
}

server {
    listen 443 ssl;
    server_name ws.example.com;

    location / {
        proxy_pass http://sockudo;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 180s;
        proxy_send_timeout 180s;
        proxy_buffering off;
    }
}
```

The default Sockudo `activity_timeout` is 120 seconds. Set every proxy or load-balancer idle timeout
above the longest expected interval between application or heartbeat traffic, with margin. A
180-second starting value works with the default, but validate it with the actual client SDKs and
network path.

If the proxy adds entries to `X-Forwarded-For`, configure `RATE_LIMITER_API_TRUST_HOPS` and
`RATE_LIMITER_WS_TRUST_HOPS` to the exact trusted proxy count. Never trust a client-supplied chain
from the public internet.

## One host versus several [#one-host-versus-several]

| Concern               | One host                           | Two or more hosts                                                 |
| --------------------- | ---------------------------------- | ----------------------------------------------------------------- |
| Adapter               | `local` is valid                   | Use Redis, NATS, or another horizontal adapter                    |
| App definitions       | Static memory app can work         | All nodes need identical static apps, or use a shared app manager |
| Rate limits           | Memory is per host                 | Use Redis or Redis Cluster for cluster-wide limits                |
| Queue                 | Memory work is lost with the host  | Use a durable shared queue for webhooks and push                  |
| History and mutations | Memory can be acceptable for tests | Use a supported durable backend                                   |
| Load balancer         | Optional reverse proxy             | Required, with health checks and drain                            |

For multiple nodes, set `fallback_to_local = false`. A production cluster should fail readiness or
startup when its horizontal adapter is unavailable instead of silently splitting into isolated
local islands.

## Drain and upgrade [#drain-and-upgrade]

1. Remove the host from load-balancer readiness.
2. Wait for the balancer's deregistration delay or connection drain policy.
3. Stop Sockudo with `systemctl stop sockudo`.
4. Allow `TimeoutStopSec` to cover Sockudo's configured shutdown grace.
5. Replace the binary and configuration atomically.
6. Start the service, wait for `/up/<app-id>`, then restore load-balancer membership.

Watch reconnect attempts, recovery success, adapter errors, and connection distribution throughout
the rollout. Continue with [Capacity planning](/docs/deployment/capacity-planning) before setting a
production connection limit.


# Static configuration (/docs/deployment/static-configuration)



Sockudo supports a static TOML file, a legacy-compatible JSON file, and runtime environment
overrides. Use a file for the complete deployment shape. Use environment variables for the subset
of settings that vary per environment and for secret references supplied by the platform.

## Loading order [#loading-order]

Configuration is resolved in this order; each later source wins:

1. compiled defaults
2. `config/config.toml` from the process working directory, when it exists and parses
3. otherwise `config/config.json` from the process working directory
4. the file passed with `--config`, when it loads successfully
5. supported environment-variable overrides
6. final configuration validation

An explicit `.toml` path is parsed as TOML. Other filename extensions are parsed as JSON. Prefer a
`.toml` or `.json` suffix so the format is obvious.

```bash
sockudo --config /etc/sockudo/config.toml
sockudo --config /etc/sockudo/config.json
```

Use `--config` to select a non-default path. The `CONFIG_FILE` environment variable present in some
container environments does not select the file by itself; the published image's default command
passes it an explicit path.

The current server logs an explicit file read or parse failure and retains the configuration
resolved before that file. In production, alert on `explicit configuration load failed` and require
the `explicit configuration applied` log before sending traffic. Validate the file in CI rather
than relying only on process startup.

Configuration is read at startup. Change it through a rollout or service restart; there is no
general-purpose hot reload.

## TOML or JSON? [#toml-or-json]

| Format           | Prefer it when                                                | Notes                                                                            |
| ---------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| TOML             | Humans maintain the file and comments are valuable.           | Preferred for bare-metal, VM, and source deployments.                            |
| JSON             | A chart, control plane, or program generates the file.        | Useful for Helm `configJson` and legacy installations; comments are not allowed. |
| Environment only | A small, single-app deployment uses only supported overrides. | Not every nested policy has an environment equivalent.                           |

TOML and JSON deserialize into the same `ServerOptions` structure. Missing fields receive code
defaults; a static file is not merged field-by-field with the auto-discovered file. The selected
file becomes the complete file-based configuration, then environment overrides are applied.

## Equivalent single-node files [#equivalent-single-node-files]

This TOML profile keeps all state in one process and is suitable for a first production-like test,
not for failover:

```toml
mode = "production"
host = "0.0.0.0"
port = 6001
debug = false
max_connections = 20000
activity_timeout = 120
shutdown_grace_period = 30

[adapter]
driver = "local"

[cache]
driver = "memory"

[queue]
driver = "memory"

[rate_limiter]
enabled = true
driver = "memory"

[metrics]
enabled = true
driver = "prometheus"
host = "0.0.0.0"
port = 9601

[app_manager]
driver = "memory"

[app_manager.array]
[[app_manager.array.apps]]
id = "example"
key = "replace-me"
secret = "replace-me"
enabled = true

[app_manager.array.apps.policy.limits]
max_connections = 20000
max_client_events_per_second = 100

[app_manager.array.apps.policy.features]
enable_client_messages = false
enable_user_authentication = true

[app_manager.array.apps.policy.channels]
allowed_origins = ["https://app.example.com"]
```

The equivalent JSON is:

```json
{
  "mode": "production",
  "host": "0.0.0.0",
  "port": 6001,
  "debug": false,
  "max_connections": 20000,
  "activity_timeout": 120,
  "shutdown_grace_period": 30,
  "adapter": {
    "driver": "local"
  },
  "cache": {
    "driver": "memory"
  },
  "queue": {
    "driver": "memory"
  },
  "rate_limiter": {
    "enabled": true,
    "driver": "memory"
  },
  "metrics": {
    "enabled": true,
    "driver": "prometheus",
    "host": "0.0.0.0",
    "port": 9601
  },
  "app_manager": {
    "driver": "memory",
    "array": {
      "apps": [
        {
          "id": "example",
          "key": "replace-me",
          "secret": "replace-me",
          "enabled": true,
          "policy": {
            "limits": {
              "max_connections": 20000,
              "max_client_events_per_second": 100
            },
            "features": {
              "enable_client_messages": false,
              "enable_user_authentication": true
            },
            "channels": {
              "allowed_origins": ["https://app.example.com"]
            }
          }
        }
      ]
    }
  }
}
```

Replace the example credentials before use. For a committed production file, remove the entire
inline app definition and bootstrap the app from a secret-backed environment, or use a persistent
app manager.

## Clustered Redis profile [#clustered-redis-profile]

The stable structure can remain in TOML:

```toml
mode = "production"
host = "0.0.0.0"
port = 6001
max_connections = 50000
shutdown_grace_period = 30

[adapter]
driver = "redis"
fallback_to_local = false

[cache]
driver = "redis"

[queue]
driver = "redis"

[rate_limiter]
enabled = true
driver = "redis"

[app_manager]
driver = "memory"

[metrics]
enabled = true
host = "0.0.0.0"
port = 9601
```

Inject the shared address, node identity, and one immutable app at runtime:

```bash
REDIS_URL=rediss://sockudo@redis.internal:6379/0
INSTANCE_PROCESS_ID=sockudo-a-01
SOCKUDO_DEFAULT_APP_ENABLED=true
SOCKUDO_DEFAULT_APP_ID=production
SOCKUDO_DEFAULT_APP_KEY=secret-store-value
SOCKUDO_DEFAULT_APP_SECRET=secret-store-value
SOCKUDO_DEFAULT_APP_ALLOWED_ORIGINS=https://app.example.com
```

`REDIS_URL` updates the Redis adapter, cache, queue, and rate limiter connection. Driver selection
still must be `redis`, either in the file or through `ADAPTER_DRIVER`, `CACHE_DRIVER`,
`QUEUE_DRIVER`, and `RATE_LIMITER_DRIVER`.

Use a persistent app manager instead of environment bootstrap when operators create, rotate, or
disable multiple applications without rebuilding the deployment.

## What belongs where [#what-belongs-where]

| Setting                                  | Static file                       | Environment                       | Secret store                         |
| ---------------------------------------- | --------------------------------- | --------------------------------- | ------------------------------------ |
| Driver selection and feature gates       | Yes                               | Optional rollout override         | No                                   |
| Nested app/channel policy                | Yes, or persistent app manager    | Only the documented subset        | Credentials only                     |
| Retention, buffer, and queue reliability | Yes                               | Use supported overrides sparingly | No                                   |
| Per-environment hostnames and ports      | Template or generated file        | Yes                               | Only if address contains credentials |
| App keys and secrets                     | Avoid in committed files          | Inject from secret reference      | Yes                                  |
| Redis, SQL, broker passwords             | Avoid                             | Inject from secret reference      | Yes                                  |
| TLS private keys and push provider keys  | File path or credential reference | Path/reference only               | Yes                                  |

Do not place secrets in image layers, Helm values committed to Git, Kubernetes ConfigMaps, process
arguments, or Terraform plan output.

## Environment override behavior [#environment-override-behavior]

Environment variables are explicit mappings in the Sockudo configuration loader, not a generic
`SECTION_FIELD=value` translation. The [environment variable reference](/docs/reference/environment-variables)
is the authoritative list.

Several app-bootstrap variables have deliberate replacement behavior:

* a complete `SOCKUDO_DEFAULT_APP_ID`, `SOCKUDO_DEFAULT_APP_KEY`, and
  `SOCKUDO_DEFAULT_APP_SECRET` set registers the environment-backed default app
* `SOCKUDO_DEFAULT_APP_ENABLED=false` disables that registration
* `SOCKUDO_SKIP_INLINE_APPS=true` skips apps declared inside the file
* `APP_MANAGER_REGISTER_INLINE_APPS=false` also prevents inline registration

Avoid mixing an inline app and environment bootstrap accidentally. Choose one source for each app
and confirm the startup messages.

## Container and Kubernetes mounts [#container-and-kubernetes-mounts]

Mount a read-only file into the published container and replace the default command:

```bash
docker run --rm \
  --mount type=bind,src="$PWD/config.toml",dst=/app/config/production.toml,readonly \
  ghcr.io/sockudo/sockudo:5.0.1 \
  sockudo --config /app/config/production.toml
```

In Kubernetes, mount non-secret structure from a ConfigMap and inject secrets separately:

```yaml
containers:
  - name: sockudo
    args: ["--config", "/etc/sockudo/config.toml"]
    envFrom:
      - secretRef:
          name: sockudo-runtime
    volumeMounts:
      - name: config
        mountPath: /etc/sockudo
        readOnly: true
volumes:
  - name: config
    configMap:
      name: sockudo-config
```

The Helm chart also accepts `configJson` for advanced generated configuration and
`extraEnvFrom` for Secret or ConfigMap references.

## Validate before rollout [#validate-before-rollout]

At minimum:

```bash
taplo check config.toml
jq empty config.json
sockudo --config /etc/sockudo/config.toml
```

For the startup check, use the same binary, features, environment, working directory, and mounted
secrets as production. Confirm:

* `explicit configuration applied`
* `Applied environment variable overrides`
* no `configuration validation failed`
* `/live` returns success
* `/up/<app-id>` returns success after shared dependencies are available

Then continue with the [Linux](/docs/deployment/linux),
[Docker](/docs/deployment/docker), or [Kubernetes](/docs/deployment/kubernetes) guide.


# Authentication (/docs/getting-started/authentication)



Sockudo never trusts client-provided access to protected channels. Clients request a subscription, your application server decides whether the user is allowed, then the server returns a signed response.

## Channel authorization flow [#channel-authorization-flow]

1. Client subscribes to `private-orders` or `presence-room`.
2. The client SDK sends `socket_id` and `channel_name` to your auth endpoint.
3. Your backend validates the current user and channel policy.
4. A server SDK signs the response using the app secret.
5. Sockudo validates the signature and completes the subscription.

## Node.js auth endpoint [#nodejs-auth-endpoint]

```ts
import express from "express";
import { Sockudo } from "sockudo";

const app = express();
app.use(express.urlencoded({ extended: false }));

const sockudo = new Sockudo({
  appId: process.env.SOCKUDO_APP_ID!,
  key: process.env.SOCKUDO_APP_KEY!,
  secret: process.env.SOCKUDO_APP_SECRET!,
  host: "127.0.0.1",
  port: 6001,
  useTLS: false,
});

app.post("/sockudo/auth", (req, res) => {
  const { socket_id, channel_name } = req.body;
  const user = requireUser(req);

  if (!canAccessChannel(user, channel_name)) {
    return res.status(403).json({ error: "Forbidden" });
  }

  if (channel_name.startsWith("presence-")) {
    return res.json(
      sockudo.authorizeChannel(socket_id, channel_name, {
        user_id: user.id,
        user_info: { name: user.name, role: user.role },
      }),
    );
  }

  return res.json(sockudo.authorizeChannel(socket_id, channel_name));
});
```

## Client configuration [#client-configuration]

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  channelAuthorization: {
    endpoint: "/sockudo/auth",
  },
});

client.subscribe("private-orders");
```

## User authentication [#user-authentication]

User authentication signs a connection identity. It powers watchlists and user-targeted server events.

```ts
app.post("/sockudo/user-auth", (req, res) => {
  const user = requireUser(req);

  res.json(
    sockudo.authenticateUser(req.body.socket_id, {
      id: user.id,
      user_info: { name: user.name },
    }),
  );
});
```

## Protocol V2 capability tokens [#protocol-v2-capability-tokens]

Protocol V2 clients may authenticate the WebSocket connection with a short-lived capability token:

```ts
const client = new Sockudo("app-key", {
  protocolVersion: 2,
  auth: {
    endpoint: "/sockudo/token",
  },
});
```

Your backend issues the JWT after authenticating the user. Never ship the app secret to browsers, mobile apps, or untrusted clients.

Node.js example:

```ts
import jwt from "jsonwebtoken";
import crypto from "node:crypto";

app.post("/sockudo/token", (req, res) => {
  const user = requireUser(req);
  const now = Math.floor(Date.now() / 1000);

  const capability = {
    "private-orders:*": ["subscribe", "history"],
    "presence-orders:*": ["subscribe", "presence"],
    "private-orders:input": ["publish"],
  };

  const token = jwt.sign(
    {
      "x-sockudo-capability": JSON.stringify(capability),
      "x-sockudo-client-id": user.id,
      iat: now,
      exp: now + 3600,
      jti: crypto.randomUUID(),
    },
    process.env.SOCKUDO_APP_SECRET!,
    {
      algorithm: "HS256",
      header: { kid: process.env.SOCKUDO_APP_KEY! },
    },
  );

  res.json({ token });
});
```

Rust example:

```rust
use jsonwebtoken::{encode, Algorithm, EncodingKey, Header};
use serde::Serialize;
use std::collections::BTreeMap;

#[derive(Serialize)]
struct Claims {
    #[serde(rename = "x-sockudo-capability")]
    capability: String,
    #[serde(rename = "x-sockudo-client-id")]
    client_id: String,
    iat: i64,
    exp: i64,
    jti: String,
}

fn issue_token(app_key: &str, app_secret: &str, client_id: &str) -> anyhow::Result<String> {
    let now = chrono::Utc::now().timestamp();
    let mut capability = BTreeMap::new();
    capability.insert("private-orders:*", vec!["subscribe", "history"]);

    let mut header = Header::new(Algorithm::HS256);
    header.kid = Some(app_key.to_owned());

    Ok(encode(
        &header,
        &Claims {
            capability: serde_json::to_string(&capability)?,
            client_id: client_id.to_owned(),
            iat: now,
            exp: now + 3600,
            jti: uuid::Uuid::new_v4().to_string(),
        },
        &EncodingKey::from_secret(app_secret.as_bytes()),
    )?)
}
```

Token rules:

* Header `kid` must equal the Sockudo app key and `alg` must be `HS256`.
* `x-sockudo-capability` is a JSON map from channel pattern to operations; its stringified form is
  accepted for compatibility.
* Operations include `publish`, `subscribe`, `history`, and `presence`; AI Transport also accepts
  the explicit annotation, message-mutation, and push operations documented in
  [AI Transport authentication](/docs/server/ai-transport-authentication).
* Patterns are exact names, namespace prefixes such as `orders:*`, or `*`; matching is case-sensitive.
* `x-sockudo-client-id` is the verified identity used for presence and user-limited channels.
* `iat`, `exp`, and `jti` are required. Tokens may live at most 24 hours; 1 hour or less is recommended.
* Clients refresh in place with `sockudo:auth` carrying `{ "token": "..." }`. Expired tokens emit `sockudo:token_expired` with code `40142` and close after a 30 second grace window.

## Encrypted channels [#encrypted-channels]

Encrypted channels start with `private-encrypted-`. The server signs the subscription and returns a per-channel shared secret derived from your encryption master key.

```ts
const sockudo = new Sockudo({
  appId: "app-id",
  key: "app-key",
  secret: "app-secret",
  host: "127.0.0.1",
  port: 6001,
  encryptionMasterKeyBase64: process.env.SOCKUDO_ENCRYPTION_MASTER_KEY!,
});
```

Keep the master key outside the browser. Client SDKs only receive the derived shared secret for authorized encrypted channels.

## Push authorization boundary [#push-authorization-boundary]

Push registration and push publish helpers are also authentication boundaries. Mobile and browser clients should call your backend proxy, not Sockudo directly with app secrets.

```ts
app.post("/sockudo/push", async (req, res) => {
  const user = requireUser(req);
  await assertDeviceBelongsToUser(user, req.body.device_id);

  const response = await sockudo.publishPush({
    recipients: [{ type: "client", client_id: user.id }],
    payload: req.body.payload,
    sync: false,
  });

  res.status(202).json(response);
});
```

## Security checklist [#security-checklist]

* Read `socket_id` and `channel_name` from the request body; do not accept them from query parameters unless your framework requires it.
* Authenticate the user before signing anything.
* Enforce channel ownership and tenant boundaries before calling the SDK signing helper.
* Return presence user data that is safe for all channel members to see.
* Keep app secrets, webhook secrets, push provider credentials, and encryption master keys server-side only.
* Issue capability tokens from a backend endpoint only; do not mint them in client code.
* Use short request timeouts and structured audit logs for denied auth attempts.


# First connection (/docs/getting-started/first-connection)



This guide proves the full loop: Sockudo accepts a WebSocket connection, the client subscribes, and a server-side SDK publishes an event over the HTTP API.

## Start Sockudo [#start-sockudo]

```bash
docker compose up sockudo redis
```

Use the default development app credentials:

```bash
export SOCKUDO_APP_ID=app-id
export SOCKUDO_APP_KEY=app-key
export SOCKUDO_APP_SECRET=app-secret
export SOCKUDO_HOST=127.0.0.1
export SOCKUDO_PORT=6001
```

## Subscribe from JavaScript [#subscribe-from-javascript]

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  enabledTransports: ["ws"],
  protocolVersion: 2,
});

const channel = client.subscribe("orders");

channel.bind("order.created", (payload) => {
  console.log("received", payload);
});
```

Protocol V1 clients use the same server with Pusher-compatible options:

```ts
import Pusher from "pusher-js";

const pusher = new Pusher("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  cluster: "local",
  enabledTransports: ["ws"],
});

pusher.subscribe("orders").bind("order.created", console.log);
```

## Publish from Node.js [#publish-from-nodejs]

```ts
import { Sockudo } from "sockudo";

const sockudo = new Sockudo({
  appId: "app-id",
  key: "app-key",
  secret: "app-secret",
  host: "127.0.0.1",
  port: 6001,
  useTLS: false,
});

await sockudo.trigger("orders", "order.created", {
  id: "ord_123",
  total: 42_00,
});
```

## Publish over raw HTTP [#publish-over-raw-http]

Server SDKs are preferred because they handle signing, but the underlying API is simple:

```bash
curl -X POST "http://127.0.0.1:6001/apps/app-id/events" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "order.created",
    "channel": "orders",
    "data": {"id": "ord_123", "total": 4200}
  }'
```

Production requests must include Sockudo/Pusher HMAC authentication query parameters. Use a server SDK unless you are implementing a new SDK.

## Use idempotency for retries [#use-idempotency-for-retries]

When a backend may retry a publish after a timeout, attach an idempotency key. Sockudo deduplicates matching keys inside the configured window.

```ts
await sockudo.trigger(
  "orders",
  "order.created",
  { id: "ord_124" },
  { idempotency_key: "order-created-ord_124" },
);
```

## Add push notification fanout [#add-push-notification-fanout]

Realtime events and push notifications are complementary. Use WebSockets for connected clients, and push for devices that are offline, backgrounded, or outside the active channel session.

```ts
await sockudo.publishPush({
  recipients: [{ type: "channel", channel: "orders" }],
  payload: {
    title: "Order created",
    body: "Order ord_124 is ready for review",
    data: { order_id: "ord_124" },
  },
  sync: false,
});
```

Push publishes should be async in production. Expect `202 Accepted` and track the returned `publish_id`.

## Expected result [#expected-result]

1. The client receives `order.created` in the console.
2. The HTTP publish returns success.
3. If push is configured, the push publish returns a `publish_id`.
4. Metrics under `/metrics` show connection and publish counters moving.

## Debug checkpoints [#debug-checkpoints]

Use this table when the first loop does not behave as expected.

| Symptom                                  | Check                                                                                                | Fix                                                                                     |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| WebSocket connection fails               | Browser devtools shows a failed request to `/app/app-key`.                                           | Confirm `wsHost`, `wsPort`, `forceTLS`, and `enabledTransports` match the local server. |
| Subscribe succeeds but no event arrives  | The backend publish may target a different app, channel, or event name.                              | Reuse the same `app-key`, `app-id`, channel, and event name from the snippets above.    |
| Private or presence channels reject auth | Auth signatures are generated on a trusted backend and include the exact socket ID and channel name. | Use a server SDK for channel auth before writing custom signing code.                   |
| Retried publishes create duplicates      | The request does not include a stable idempotency key.                                               | Derive keys from business identifiers such as `order-created-ord_123`.                  |
| Metrics do not move                      | You may be checking the wrong process, port, or app credentials.                                     | Query `/up`, then scrape `:9601/metrics` while publishing a fresh event.                |

When debugging compatibility, first prove the same event works with a public channel. Then add private auth, presence user data, Protocol V2 options, history, push, or mutations one feature at a time.


# Installation (/docs/getting-started/installation)



The fastest way to evaluate Sockudo is Docker Compose. On Linux, the release installer downloads a
verified prebuilt binary. For container platforms, pull the published image from GitHub Container
Registry or Docker Hub. Other operating systems can build from crates.io or source.

## Docker Compose [#docker-compose]

```bash
git clone https://github.com/sockudo/sockudo.git
cd sockudo
docker compose up sockudo redis
```

The default server listens on:

| Service                | URL                             |
| ---------------------- | ------------------------------- |
| HTTP and WebSocket API | `http://127.0.0.1:6001`         |
| Prometheus metrics     | `http://127.0.0.1:9601/metrics` |

Use Compose when you want Redis, local configuration, and repeatable development infrastructure without compiling Rust first.

## Install a Linux binary [#install-a-linux-binary]

The installer supports x86\_64 and ARM64 Linux, detects GNU libc or musl, and verifies the release
archive against its published SHA-256 checksum. It installs to `~/.local/bin` by default.

```bash
curl --proto '=https' --tlsv1.2 -sSfL \
  https://sockudo.io/install.sh | sh
```

`sockudo.io/install.sh` is the stable public URL. It redirects to the installer attached to the
latest GitHub release; the installer then downloads and verifies the selected version from GitHub.

Pin the binary version in deployments:

```bash
curl --proto '=https' --tlsv1.2 -sSfL \
  https://sockudo.io/install.sh \
  | sh -s -- --version 5.0.1
```

Choose another writable installation directory with `--bin-dir`:

```bash
curl --proto '=https' --tlsv1.2 -sSfL \
  https://sockudo.io/install.sh \
  | sh -s -- --bin-dir "$HOME/bin"
```

Run the installed binary with a local configuration file:

```bash
sockudo --config ./config/config.toml
```

Only Linux binaries are published. `cargo binstall sockudo` remains available for the same Linux
release assets; use a source build on macOS, Windows, and other platforms.

## Install from crates.io [#install-from-cratesio]

Use `cargo install` when you prefer building the published crate on the target host or need a
platform without a prebuilt binary:

```bash
cargo install sockudo --locked
```

Pin a release or enable the same production-oriented features you would use from source:

```bash
cargo install sockudo --version 5.0.1 --locked
cargo install sockudo --locked --features "redis,postgres,push"
```

This path compiles on the machine where you run it, so it needs a Rust toolchain and any native
libraries required by the selected storage or adapter features.

## Pull a container image [#pull-a-container-image]

Sockudo publishes multi-architecture images to GitHub Container Registry and Docker Hub. GHCR is the
primary registry; Docker Hub mirrors the same release tags.

```bash
docker pull ghcr.io/sockudo/sockudo:latest
docker pull sockudo/sockudo:latest
```

Use versioned tags for production rollouts:

```bash
docker pull ghcr.io/sockudo/sockudo:5.0.1
docker pull sockudo/sockudo:5.0.1
```

Start a local container with an in-memory app and Prometheus metrics exposed:

```bash
docker run --rm --name sockudo \
  -p 6001:6001 \
  -p 9601:9601 \
  -e HOST=0.0.0.0 \
  -e PORT=6001 \
  -e METRICS_PORT=9601 \
  -e METRICS_ENABLED=true \
  -e SOCKUDO_DEFAULT_APP_ID=demo-app \
  -e SOCKUDO_DEFAULT_APP_KEY=demo-key \
  -e SOCKUDO_DEFAULT_APP_SECRET=demo-secret \
  ghcr.io/sockudo/sockudo:latest
```

Mount a checked-in configuration file when you need the same settings locally, in CI, and in
production:

```bash
docker run --rm --name sockudo \
  -p 6001:6001 \
  -p 9601:9601 \
  -v "$PWD/config/config.toml:/app/config/config.toml:ro" \
  ghcr.io/sockudo/sockudo:latest \
  sockudo --config /app/config/config.toml
```

## Run from source [#run-from-source]

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
git clone https://github.com/sockudo/sockudo.git
cd sockudo
cargo run --release
```

The default feature set is intentionally local-friendly. Add only the backends your deployment needs:

```bash
cargo build --release --features "redis,postgres"
cargo build --release --features "redis-cluster,mysql"
cargo build --release --features "nats,postgres"
cargo build --release --features "kafka,postgres"
cargo build --release --features "iggy,postgres"
cargo build --release --features full
```

## Feature flags [#feature-flags]

| Feature                                                        | Enables                                                                     |
| -------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `local`                                                        | In-memory app, cache, queue, adapter, and rate limit implementations.       |
| `v2`                                                           | Sockudo-native protocol features. Enabled by default.                       |
| `recovery`                                                     | Serial continuity, `message_id`, replay buffers, resume events, and rewind. |
| `delta`                                                        | Fossil and Xdelta3/VCDIFF delta compression for V2 clients.                 |
| `tag-filtering`                                                | Server-side tag filter expressions for V2 subscriptions.                    |
| `redis`                                                        | Redis adapter, cache, queue, and rate limiter support.                      |
| `redis-cluster`                                                | Cluster-aware Redis transport and adapter support.                          |
| `nats`, `kafka`, `rabbitmq`, `pulsar`, `google-pubsub`, `iggy` | Horizontal transport adapters.                                              |
| `mysql`, `postgres`, `dynamodb`, `scylladb`, `surrealdb`       | Persistent app manager backends.                                            |
| `full`                                                         | All production backends and optional integrations.                          |

## Minimal Pusher-compatible build [#minimal-pusher-compatible-build]

If you only need a small Pusher-compatible server, build without default features:

```bash
cargo build --release --no-default-features
```

Add V2 features explicitly when needed:

```bash
cargo build --release --no-default-features --features "recovery,delta,tag-filtering"
```

## Configuration file [#configuration-file]

Sockudo prefers TOML configuration. The server looks for `config/config.toml` first, with JSON kept as a fallback.

```toml
port = 6001
host = "0.0.0.0"
debug = false

[app_manager]
driver = "memory"

[app_manager.array]
[[app_manager.array.apps]]
id = "app-id"
key = "app-key"
secret = "app-secret"
enabled = true

[app_manager.array.apps.policy.limits]
max_connections = 10000

[app_manager.array.apps.policy.features]
enable_client_messages = false
```

For exact loading precedence, equivalent JSON, secret injection, and clustered examples, see
[Static configuration](/docs/deployment/static-configuration).

## Kubernetes with Helm [#kubernetes-with-helm]

```bash
helm install sockudo oci://ghcr.io/sockudo/charts/sockudo --version 4.7.0 \
  --set config.adapterDriver=redis \
  --set redis.host=redis-master \
  --set autoscaling.enabled=true \
  --set pdb.enabled=true \
  --set ingress.enabled=true \
  --set serviceMonitor.enabled=true
```

Use Kubernetes when you need autoscaling, service monitors, disruption budgets, secret-backed app credentials, and cluster-level rollout controls. Continue with the production
[Kubernetes and Helm guide](/docs/deployment/kubernetes).

## Verify [#verify]

```bash
curl -f http://127.0.0.1:6001/up
curl -f http://127.0.0.1:9601/metrics | head
```

When health is green, continue with [First connection](/docs/getting-started/first-connection). Before
shipping, follow the [Deployment guide](/docs/deployment).


# Migration (/docs/getting-started/migration)



Sockudo can be adopted in phases. Start with Protocol V1 compatibility, then opt selected clients into Protocol V2 when you want native Sockudo features.

## Compatibility-first migration [#compatibility-first-migration]

For existing Pusher clients, point the host and ports at Sockudo:

```ts
import Pusher from "pusher-js";

const pusher = new Pusher(process.env.NEXT_PUBLIC_PUSHER_KEY!, {
  wsHost: "realtime.example.com",
  wsPort: 80,
  wssPort: 443,
  forceTLS: true,
  enabledTransports: ["ws", "wss"],
  cluster: "local",
});
```

Backend SDKs continue to publish through Pusher-compatible REST endpoints. Change host, port, TLS, and credentials, then run your existing private and presence auth tests.

## Native migration [#native-migration]

After compatibility is stable, migrate clients to Sockudo SDKs and enable Protocol V2:

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  protocolVersion: 2,
  connectionRecovery: true,
});
```

Protocol V2 is the right choice for recovery, rewind, deltas, filters, versioned messages, annotations, and push helper workflows.

## Server configuration differences [#server-configuration-differences]

Sockudo prefers TOML:

```toml
[app_manager]
driver = "postgres"

[adapter]
driver = "redis"

[cache]
driver = "redis"

[queue]
driver = "redis"
```

Keep one source of truth for app credentials. If you previously hardcoded credentials in application config, move them into Secret Manager, Kubernetes Secrets, or the persistent app manager.

## Push notification migration [#push-notification-migration]

If your old realtime stack only handled WebSockets, plan push as part of the migration instead of after it. Sockudo's push platform covers:

* device registration and activation
* provider credentials for FCM, APNs, Web Push, HMS, and WNS
* channel push subscriptions
* async publish admission
* scheduled push
* provider delivery status callbacks
* push metrics and capacity planning

Keep push provider credentials in Sockudo or your trusted backend. Clients should only receive short-lived activation or identity tokens.

## Cutover checklist [#cutover-checklist]

1. Run Sockudo next to the existing realtime provider.
2. Mirror app credentials into Sockudo.
3. Point staging clients at Sockudo in Protocol V1 mode.
4. Verify public, private, presence, encrypted, webhook, and push flows.
5. Enable metrics and alerting before production traffic.
6. Move a small tenant, region, or traffic shard first.
7. Migrate selected applications to Protocol V2 only after V1 parity is verified.

## Rollback [#rollback]

Rollback is simplest when client configuration is environment-driven:

```bash
REALTIME_HOST=realtime.example.com
REALTIME_PROTOCOL_VERSION=1
REALTIME_FORCE_TLS=true
```

Keep idempotency keys on backend publishes during cutover. If a retry crosses providers, use an application-level operation ID so downstream consumers can suppress duplicates.


# Overview (/docs/getting-started/overview)



Sockudo is a Rust realtime server that exposes the Pusher Channels protocol while adding a native Sockudo protocol for features that do not fit inside the Pusher compatibility contract.

## Mental model [#mental-model]

Sockudo has four major surfaces:

| Surface          | Purpose                                                                                                 | Where to start                                             |
| ---------------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| WebSocket server | Accepts client connections, subscriptions, auth results, presence transitions, and realtime events.     | [First connection](/docs/getting-started/first-connection) |
| HTTP API         | Lets trusted backends publish events, inspect channels, read history, mutate messages, and manage push. | [HTTP API](/docs/server/http-api)                          |
| Client SDKs      | Browser, mobile, desktop, .NET, and Python realtime clients that subscribe and react to events.         | [Realtime Clients](/docs/clients)                          |
| Server SDKs      | Backend libraries that sign auth responses, publish events, validate webhooks, and call REST endpoints. | [Server SDKs](/docs/server-sdks)                           |

## Pick your path [#pick-your-path]

| You are building                       | Start with                                                                                                                                             | You should finish with                                                                       |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------- |
| A Pusher replacement                   | [Migration](/docs/getting-started/migration) and [Protocol reference](/docs/reference/protocol)                                                        | Existing clients connecting through Protocol V1 with unchanged auth and publish semantics.   |
| A new realtime product                 | [First connection](/docs/getting-started/first-connection) and [Authentication](/docs/getting-started/authentication)                                  | Public, private, and presence channels wired through client and server SDKs.                 |
| A clustered production deployment      | [Deployment](/docs/deployment), [Scaling](/docs/server/scaling), and [Observability](/docs/server/observability)                                       | Load-balanced nodes, shared adapter, health probes, metrics, and dependency-specific alerts. |
| A durable collaboration or AI workflow | [Protocol V2](/docs/clients/protocol-v2), [History and recovery](/docs/server/history-recovery), and [Mutable messages](/docs/server/mutable-messages) | Serial continuity, message IDs, rewind, annotations, and safe retry behavior.                |

## Protocol modes [#protocol-modes]

Protocol V1 is the compatibility layer. Use it when you want existing `pusher-js`, Laravel Echo, or Pusher-shaped server SDK integrations to keep working with minimal changes.

Protocol V2 is the native layer. Use it when your application needs one or more of:

* connection recovery with `stream_id` and `serial`
* `message_id` on broadcasts
* subscribe-time rewind
* delta compression and conflation
* server-side tag filtering
* durable channel history
* versioned mutable messages
* annotations for reactions, receipts, moderation, and summaries
* push notification helper workflows

```mermaid
flowchart LR
  A[Existing Pusher clients] --> B[Protocol V1 compatibility]
  C[New Sockudo clients] --> D[Protocol V2 native features]
  B --> E{Need durable semantics?}
  E -->|No| F[Keep V1 contracts stable]
  E -->|Yes| G[Move selected channels to V2]
  D --> G
  G --> H[Recovery, rewind, tags, deltas, annotations]
```

Adopt V2 by channel or client cohort. Keeping V1 clients stable while enabling V2 for new experiences is the safest migration shape.

## Channel types [#channel-types]

Sockudo follows Pusher channel naming rules for public, private, presence, and encrypted channels.

| Channel   | Prefix               | Auth required          | Typical use                                       |
| --------- | -------------------- | ---------------------- | ------------------------------------------------- |
| Public    | none                 | No                     | dashboards, public feeds, unauthenticated status  |
| Private   | `private-`           | Yes                    | account data, order updates, team-only streams    |
| Presence  | `presence-`          | Yes with user data     | rooms, collaboration, online member lists         |
| Encrypted | `private-encrypted-` | Yes with shared secret | payloads that should stay opaque to the transport |

## Production shape [#production-shape]

A production deployment usually has:

1. One or more Sockudo nodes behind a load balancer.
2. A shared adapter such as Redis, Redis Cluster, NATS, Kafka, RabbitMQ, Pulsar, Google Pub/Sub, or Iggy.
3. A persistent app manager such as PostgreSQL, MySQL, DynamoDB, ScyllaDB, SurrealDB, or Redis when apps are not configured statically.
4. A cache and queue backend for rate limits, idempotency, webhooks, and optional push workflows.
5. Prometheus scraping and alerting for connection, fanout, push, history, and adapter health.

## What to read next [#what-to-read-next]

<Cards>
  <Card title="Install Sockudo" href="/docs/getting-started/installation" description="Docker, Cargo, feature flags, and local configuration." />

  <Card title="Connect a client" href="/docs/getting-started/first-connection" description="Subscribe to a channel and publish from a backend." />

  <Card title="Add authentication" href="/docs/getting-started/authentication" description="Private, presence, encrypted, and user authentication." />

  <Card title="Migrate from Pusher" href="/docs/getting-started/migration" description="Compatibility notes and cutover strategy." />

  <Card title="Deploy to production" href="/docs/deployment" description="Static configuration, Linux, Docker, Kubernetes, cloud platforms, and capacity planning." />
</Cards>


# Troubleshooting (/docs/getting-started/troubleshooting)



Start with the path that failed: connection, subscription, publish, fanout, history, or push. Sockudo is easier to debug when each boundary is checked separately.

## Connection fails [#connection-fails]

Verify the server is reachable:

```bash
curl -f http://127.0.0.1:6001/up
```

Check the client host options:

```ts
new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  enabledTransports: ["ws"],
});
```

Common causes:

* using `forceTLS: true` against an HTTP-only local server
* forgetting `enabledTransports: ["ws"]` in local browser tests
* proxying WebSockets without `Upgrade` and `Connection` headers
* sending clients to one host while HTTP publish uses another app environment

## Private or presence subscription fails [#private-or-presence-subscription-fails]

Inspect the auth endpoint response. Private auth must return an `auth` value. Presence auth must return both `auth` and `channel_data`.

```json
{
  "auth": "app-key:signature",
  "channel_data": "{\"user_id\":\"42\",\"user_info\":{\"name\":\"Ada\"}}"
}
```

If signatures fail, verify:

* `socket_id` is exact
* `channel_name` is exact
* app key and secret match the Sockudo app
* presence `channel_data` is serialized before signing
* framework body parsers are not dropping form fields

## Events publish but clients do not receive them [#events-publish-but-clients-do-not-receive-them]

Check the channel name and protocol mode first. Protocol V1 clients receive Pusher-shaped event prefixes. Protocol V2 clients receive Sockudo-native prefixes and V2 metadata.

```bash
curl -s http://127.0.0.1:9601/metrics | rg "publish|connection|subscription"
```

In a cluster, verify all nodes share the same adapter and cache configuration. Local memory adapters do not fan out between nodes.

## Recovery or rewind is incomplete [#recovery-or-rewind-is-incomplete]

Recovery depends on continuity. Sockudo should fail closed when the stream cannot be proven.

Check:

* `protocolVersion: 2`
* `connectionRecovery: true`
* replay buffer TTL and size
* channel-specific rewind limits
* adapter delivery latency and duplicate suppression metrics

## Push notification publish is accepted but nothing arrives [#push-notification-publish-is-accepted-but-nothing-arrives]

Push is asynchronous by design. A successful admission response means Sockudo accepted the publish, not that every provider delivered it.

Check in order:

1. The device registration exists and is active.
2. The device has a provider token for the target platform.
3. The channel push subscription exists when targeting a channel.
4. Provider credentials are configured for the platform.
5. The publish returned a `publish_id`.
6. The publish status endpoint shows accepted, dispatched, failed, or scheduled.
7. Provider delivery callbacks are reaching Sockudo when configured.

```bash
curl -s "http://127.0.0.1:6001/apps/app-id/push/publish/pub_123/status"
```

Common push causes:

* APNs topic or environment mismatch
* Web Push VAPID key mismatch
* FCM project credentials for the wrong app
* expired device provider token
* client registration stored under a different `client_id`
* publish targeting `sync: true` under production fanout pressure

## Webhooks are rejected [#webhooks-are-rejected]

Webhook validation uses the raw body, not a parsed JSON object.

```ts
const webhook = sockudo.webhook({
  rawBody,
  headers: req.headers,
});

if (!webhook.isValid()) {
  return res.status(401).send("invalid signature");
}
```

Keep the raw request bytes from your framework before JSON parsing mutates whitespace or key order.

## Escalation bundle [#escalation-bundle]

When reporting an issue, include:

* Sockudo version and feature flags
* protocol version used by affected clients
* app manager, adapter, cache, and queue drivers
* relevant configuration with secrets redacted
* server logs around the failure
* metrics for connection, publish, fanout, history, and push
* a minimal client and server SDK reproduction


# Compatibility (/docs/reference/compatibility)



Sockudo's compatibility rule is simple: Protocol V1 preserves Pusher behavior; Protocol V2 adds
Sockudo-native behavior.

> Sockudo's third-party compatibility layers are community-built and
> community-maintained. They are not products of Pusher or Ably, and those
> companies do not provide support for Sockudo. Use Sockudo's project and
> community support channels for help.

## Pusher-compatible [#pusher-compatible]

| Feature            | Compatibility                                |
| ------------------ | -------------------------------------------- |
| Public channels    | V1 and V2                                    |
| Private channels   | V1 and V2                                    |
| Presence channels  | V1 and V2                                    |
| Encrypted channels | V1 and V2 with compatible auth               |
| HTTP publish       | Pusher-compatible                            |
| Batch publish      | Pusher-compatible                            |
| Channel state      | Pusher-compatible                            |
| Webhook validation | Pusher-compatible shape                      |
| Laravel Echo       | Use Protocol V1 / Pusher-compatible settings |
| pusher-js          | Use Protocol V1 / Pusher-compatible settings |

## Sockudo-native [#sockudo-native]

| Feature                  | Requirement                                                                     |
| ------------------------ | ------------------------------------------------------------------------------- |
| `message_id`             | Protocol V2                                                                     |
| `serial` and `stream_id` | Protocol V2                                                                     |
| Connection recovery      | Protocol V2                                                                     |
| Subscribe-time rewind    | Protocol V2                                                                     |
| Delta compression        | Protocol V2 and Sockudo client SDK                                              |
| Tag filtering            | Protocol V2 and Sockudo client SDK                                              |
| Mutable messages         | Protocol V2                                                                     |
| Annotations              | Protocol V2 APIs and SDK helpers                                                |
| Push management          | Sockudo HTTP API and server SDK push helpers                                    |
| AI Transport             | Protocol V2, `ai-transport` Cargo feature, and runtime `[ai_transport]` enabled |

## Migration strategy [#migration-strategy]

1. Run Pusher-compatible clients against Sockudo first.
2. Verify public, private, presence, encrypted, webhook, and backend publish flows.
3. Adopt Sockudo server SDKs where you need idempotency, history, annotations, or push.
4. Move selected clients to Sockudo-native SDKs and enable V2.

## Push compatibility [#push-compatibility]

Push notifications are not part of the Pusher WebSocket protocol. They are Sockudo-native HTTP APIs.
Existing Pusher-compatible realtime clients can still coexist with Sockudo push because push is
managed by backend services and device registration flows.

## AI Transport compatibility [#ai-transport-compatibility]

AI Transport is additive and default-off. The server release order is:

1. Ship Sockudo with the `ai-transport` feature available but runtime-disabled by default.
2. Release Sockudo client SDK support after server, SDK, and conformance evidence is green.
3. Enable AI Transport only for V2 clients and scoped channel prefixes.

| Surface                        | Compatibility                                                                                                                                                                                                        |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Protocol V1 / pusher-js canary | Must remain byte-identical with AI Transport disabled                                                                                                                                                                |
| Protocol V2 non-AI clients     | Existing recovery, rewind, history, mutable messages, annotations, and push remain additive                                                                                                                          |
| `@sockudo/ai-transport`        | Requires server AI Transport feature, runtime profile, and `@sockudo/client`                                                                                                                                         |
| `@ably/ai-transport`           | Optional `ably-compat` feature exposing the tested Ably REST and WebSocket surface, excluding Live Objects; see [Ably REST, WebSocket, and AI Transport compatibility](/docs/server/ably-ai-transport-compatibility) |
| Existing client SDKs           | Full product parity depends on Protocol V2 feature enablement in each SDK                                                                                                                                            |
| Server HTTP SDKs               | Existing Pusher-compatible publish APIs continue to work; AI helpers are additive                                                                                                                                    |

Sockudo does not claim full Ably platform compatibility. The evidence-backed wording is **Ably REST
and WebSocket compatibility, excluding Live Objects**. The opt-in `ably-compat` surface is validated
in two layers. Pull requests fetch the current `main` heads of `ably-js`, `ably-go`, and
`ably-ai-transport-js`, record their resolved commit IDs, and run the Node REST/WebSocket, official
Go unit and JSON/MsgPack integration, and complete AI Transport suites against the pull request.
Realtime is forced to WebSocket; Live Objects and multiple/non-WebSocket transport coverage are
excluded. Three named ably-js SDK/harness-only assertions are also excluded: the Comet inventory
and two default TLS/port checks overridden by local routing. The Go and AI Transport suites have no
test exclusions.

Pinned Node, browser, strict-upstream-pending, Go, and AI Transport manifests remain the source of
reproducible release evidence. Upstream-default pending results are audited and executed separately,
without changing their bodies, assertions, or expected values. A latest-upstream or default-lane
pass is not reported as a strict-completeness, browser, or immutable release-evidence pass.

The current pinned source-build evidence is green: Node defaults are 575/575, strict completeness
is 250/250, Chromium defaults are 574/574 with no browser-boundary errors, Chromium strict is
250/250, and AIT is 50/50. Release verification is separate: an explicit published tag must pass
the checksum-verifying released-binary workflow before that tag is promoted as verified.

The implementation is isolated in `crates/sockudo-ably-compat` and constructed
per server instance. Its realtime surface is WebSocket-only; Ably fallback
transports are intentionally unsupported.

## Compatibility performance evidence [#compatibility-performance-evidence]

Compatibility releases use two complementary guards:

* `make ably-compat-bench` measures the production JSON/MessagePack codec, commit-envelope and
  history projection, error encoding, maximum 4,096-message recovery projection, and the actual
  subscriber registry plus bounded socket queues at 1, 100, 1,000, and 10,000 subscribers. Fanout
  assertions require one shared encoded buffer per active wire format.
* `tests/load/ably-compat/capacity-runner.mjs` starts real one-node and Redis/Postgres-backed
  two-node topologies. The release profile covers steady and burst publish, 64 KiB and encrypted
  values, 1,000-subscriber fanout, 10% stalled peers, recovery, presence, append, stats, and push
  enqueue workloads.

Only executable result documents with a binary hash, exact config hash, evidence-harness hashes,
hardware/tool metadata, latencies, throughput, resource samples, runtime counters, and
zero-loss/duplicate/reordering/unexpected-delivery audits count as evidence. The driver retains
exact bounded sequence bitmaps and a deterministic capped latency reservoir, never full delivered
payloads. Plan output and `.not-run` manifests are rejected. The release guard also requires RSS to
plateau under stalled peers and after disconnect, and can compare three or more independent
current/baseline runs with a one-sided statistical regression test.

## Release gates [#release-gates]

Protocol-visible server PRs must pass the Protocol Change checklist and relevant fixtures for the
changed surface. When protocol-owned paths change, CI also requires the `sockudo-js` compatibility
lane because that SDK is the reference client for cross-SDK compatibility.

SDK release workflows must run their conformance lanes against both the latest released Sockudo
server and server `main` before publishing. API-diff jobs block non-additive public API changes
unless the release is intentionally major.

Rollback starts by disabling the new server feature flag. Client packages can then be pinned back
independently because valid V1 and existing V2 traffic remain compatible.

GA evidence is tracked in
[`docs/specs/ai-transport-ga-readiness.md`](/specs/ai-transport-ga-readiness).


# Configuration reference (/docs/reference/configuration)



Sockudo's TOML configuration is organized by runtime responsibility.

For configuration loading order, JSON equivalence, and guidance on what belongs in files,
environment overrides, or secret stores, see [Static configuration](/docs/deployment/static-configuration).
The complete [environment variable reference](/docs/reference/environment-variables) lists every
runtime variable parsed by the server and push subsystem.

## Top-level [#top-level]

| Key               | Purpose                                                                                                                                                                                                               |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `host`            | Bind host for WebSocket and HTTP API.                                                                                                                                                                                 |
| `port`            | Bind port for WebSocket and HTTP API.                                                                                                                                                                                 |
| `max_connections` | Per-node WebSocket connection limit. When exceeded the server returns close code 4100 so clients reconnect with backoff. `0` disables the limit.                                                                      |
| `debug`           | Enables verbose diagnostics for local development.                                                                                                                                                                    |
| `server_role`     | Deployment topology role. `"default"` runs the full server. `"api"` runs HTTP publish, history, mutable messages, and push — without WebSocket listeners, cluster heartbeats, or presence sync. Default: `"default"`. |

## Server role [#server-role]

`server_role` controls which subsystems the process starts.

```toml
server_role = "api"

[adapter]
driver = "redis-cluster"
fallback_to_local = false
```

API pods publish to the horizontal broadcast stream and serve HTTP endpoints
(publish, history, mutable messages, annotations, push). They skip WebSocket
listeners, adapter subscriptions, heartbeats, presence sync, and request/reply —
so they never appear in peer accounting and WS pods never await replies from them.

The server rejects startup when `server_role = "api"` is combined with
`adapter.driver = "local"` or `adapter.fallback_to_local = true`.

For endpoint availability details see
[HTTP endpoints — availability by server role](/docs/reference/http-endpoints#availability-by-server-role).
For capacity and operational metrics see
[Capacity planning — API pod topology](/docs/deployment/capacity-planning#api-pod-topology).

## Memory-pressure connection admission [#memory-pressure-connection-admission]

Memory-pressure admission control is opt-in under `[http_api.accept_traffic]`:

```toml
[http_api.accept_traffic]
enabled = true
memory_threshold = 0.90
# memory_limit_bytes = 2147483648 # optional; otherwise discover the cgroup limit
sample_interval_ms = 500
```

On Linux, Sockudo samples its process RSS on `sample_interval_ms`, independently of Prometheus
scrapes. It compares RSS with `memory_limit_bytes` when configured, otherwise with the finite cgroup
v2 `memory.max` or cgroup v1 `memory.limit_in_bytes` value. At or above `memory_threshold`, native
and Ably WebSocket upgrades return `503`, and the post-upgrade admission gate returns protocol close
code `4100` if pressure changes during the upgrade. Existing connections remain open.

The sampler fails open when RSS or a finite limit is unavailable, including on non-Linux systems.
`GET /accept-traffic` exposes the admission decision for a load balancer; `/up` deliberately remains
a dependency/readiness check and does not fail because of memory pressure. Use
`sockudo_memory_pressure_shedding` and `sockudo_memory_pressure_rejections_total` to alert on the
current state and rejected connection count.

## App manager [#app-manager]

| Section                      | Purpose                                                |
| ---------------------------- | ------------------------------------------------------ |
| `[app_manager]`              | Select app storage driver.                             |
| `[app_manager.array]`        | Static in-config app definitions.                      |
| `[[app_manager.array.apps]]` | App ID, key, secret, enabled flag, limits, and policy. |

## Ably compatibility keys [#ably-compatibility-keys]

`App.key` and `App.secret` are always the implicit primary credential. When the
server is built with `ably-compat`, `[ably_compat]` can opt an app into additional
Ably key names without creating sibling apps:

```toml
[ably_compat]
enabled = true
realtime_admission = "accept"
attach_timeout_ms = 10000
max_token_ttl_ms = 86400000
token_request_timestamp_skew_ms = 900000
nonce_ttl_seconds = 900
stats_fixture_ingest_enabled = false
stats_queue_capacity = 4096
stats_flush_interval_ms = 10
stats_retention_seconds = 34560000
stats_max_scan_entries = 100000
stats_cas_retries = 8

[[ably_compat.keys]]
app_id = "app-id"
key_name = "app-key-readonly"
secret = "replace-with-a-secret"
capability = '{"chat:*":["subscribe","history"]}'
revocable_tokens = true
enabled = true
rotation_id = "2026-07"
```

Each entry supports optional `created_at_ms`, `expires_at_ms`, and
`revoked_at_ms` rotation boundaries. Disabled, expired, or revoked keys reject
new key authentication. Tokens from a key with `revocable_tokens = true` are
also rejected after that key is disabled, revoked, expired, or replaced with a
different `rotation_id`. Secrets and bearer tokens are never written to logs.

TokenRequest `timestamp` and `ttl` values accept JSON integers or decimal
integer strings, matching values commonly forwarded from auth URL query
parameters. Timestamps are accepted only within the configured skew. Nonces are
claimed atomically in the configured cache for `nonce_ttl_seconds`, and issued
token records use the same cache, allowing issuance on one node and use on
another. Use a shared cache driver such as Redis in multi-node deployments.

The server's `ably-compat` Cargo feature explicitly enables native AI Transport,
recovery, delta, and push support. Runtime `[ably_compat].enabled` and the key
registry remain the opt-in boundary; Protocol V1 routes and frames are unchanged.
`realtime_admission = "placement_constraint"` is an operator/drain mode for a
listener: authenticated Ably WebSocket upgrades receive `DISCONNECTED` error
`50320`, prompting the SDK to try its configured fallback hosts over WebSocket.
It does not enable Comet, polling, streaming, SSE, or any other realtime
transport. The default is `accept`.
`attach_timeout_ms` bounds the complete native presence/recovery/history attach
path. On expiry Sockudo removes the partial subscription and returns
`DETACHED`/`50003`; the default is 10 seconds.
Ably realtime data queues use `[websocket].max_messages` and
`[websocket].max_bytes`. When a slow session exceeds either configured bound,
Sockudo marks its continuity lost and requires recovery instead of allowing the
session to buffer beyond the operator-configured limit. The shipped
configuration uses a 4 MiB byte bound so a healthy short burst of maximum-size
application events is not mistaken for a stalled reader.
Ably `delta=vcdiff` is negotiated per realtime channel and uses a fixed bounded
compatibility cache; native `[delta_compression]` settings continue to configure
only Sockudo's Pusher/V2 delta contract and do not select the Ably VCDIFF mode.
Compatibility statistics use canonical UTC minute buckets in the configured
cache and derive hour, day, and month results when queried. Sum counters and
peak gauges retain their distinct rollup semantics. In multi-node or
restart-safe deployments, use a shared persistent cache such as Redis; the
memory cache is node-local and does not survive a restart.
Redis-backed entries retain their configured TTL across an individual Sockudo
node's graceful shutdown; use the explicit cache administration APIs for
destructive prefix cleanup.

The recording worker is bounded by `stats_queue_capacity`, combines at most 256
observations per write batch, and waits up to `stats_flush_interval_ms` to form a
batch. REST and realtime publishes that receive an application ACK wait for the
canonical bucket merge first. Outbound delivery accounting uses the nonblocking
bounded path, so an abrupt process crash can lose observations that were still
queued; queue saturation is reported as a drop rather than allowing unbounded
memory growth. Retention, bounded query scans, and compare-and-swap retries are
controlled by `stats_retention_seconds`, `stats_max_scan_entries`, and
`stats_cas_retries`.

`stats_fixture_ingest_enabled` exposes authenticated `POST /stats` only for
conformance fixture provisioning and defaults to `false`. Fixture intervals are
validated and written through the same minute-store and rollup contract as live
observations; the HTTP handler does not return canned fixture values.

Compatibility device registrations and channel subscriptions use
`[push].storage_driver`, so clustered or restart-safe deployments should select
the same durable push store used by native push APIs.

## Runtime backends [#runtime-backends]

| Section          | Purpose                                      |
| ---------------- | -------------------------------------------- |
| `[adapter]`      | Cross-node fanout.                           |
| `[cache]`        | Shared cache and coordination.               |
| `[queue]`        | Webhook and push background work.            |
| `[rate_limiter]` | Request, connection, event, and push limits. |

## OMQ adapter [#omq-adapter]

`adapter.driver = "omq"` enables brokerless horizontal fanout over OMQ PUB/SUB.
Each node binds one subscriber endpoint and connects its publisher to every
other node's subscriber endpoint.

```toml
[adapter]
driver = "omq"
fallback_to_local = false

[adapter.omq]
bind_endpoint = "tcp://0.0.0.0:5556"
connect_endpoints = [
  "tcp://sockudo-1.internal:5556",
  "tcp://sockudo-2.internal:5556",
]
prefix = "sockudo_adapter"
request_timeout_ms = 5000
nodes_number = 3
io_threads = 1
send_hwm = 100000
recv_hwm = 100000
```

OMQ uses transport topics derived from `prefix`: broadcast, request, response,
and per-node request topics. The adapter is not persistent; if a process or
network path is unavailable, messages in flight are not retained by a broker.

## Redis connections, Sentinel, and TLS [#redis-connections-sentinel-and-tls]

`[database.redis]` configures the Redis connection used by the Redis adapter, cache, queue, rate limiter, and delta coordinator. Without Sentinel, `[database.redis.master_tls]` secures the direct Redis data connection for every consumer. When `sentinels` is non-empty, Sockudo connects through Redis Sentinel using a native Sentinel client (rather than a direct URL) and can secure both connection hops independently. Redis Cluster also reuses `master_tls` certificate material; the legacy `cluster.use_tls` switch continues to enable TLS with the default root store.

| Key                                       | Purpose                                                                                       |
| ----------------------------------------- | --------------------------------------------------------------------------------------------- |
| `sentinels`                               | List of `{ host, port }` Sentinel nodes. A non-empty list enables Sentinel mode.              |
| `name`                                    | Monitored master (Sentinel service) name.                                                     |
| `username` / `password`                   | Auth for the master/replica data connection.                                                  |
| `sentinel_username` / `sentinel_password` | Auth for the Sentinel control connection.                                                     |
| `[database.redis.sentinel_tls]`           | TLS for the client→Sentinel control connection.                                               |
| `[database.redis.master_tls]`             | TLS for direct Redis, Redis Cluster, or the Sentinel-resolved master/replica data connection. |

Each TLS block accepts `enabled`, `accept_invalid_certs` (skips verification; dangerous), `ca_path` (PEM CA for private CAs), and `client_cert_path` + `client_key_path` (PEM pair for mutual TLS / client-certificate auth).

For a direct Redis deployment signed by a private CA, no Sentinel configuration is required:

```toml
[database.redis]
host = "redis.internal"
port = 6380

[database.redis.master_tls]
enabled = true
ca_path = "/etc/sockudo/tls/redis-ca.pem"
```

```toml
[database.redis]
name = "mymaster"
db = 0
username = "appuser"
password = "app-secret"
sentinel_username = "sentineluser"
sentinel_password = "sentinel-secret"

[[database.redis.sentinels]]
host = "sentinel-1.internal"
port = 26379

[[database.redis.sentinels]]
host = "sentinel-2.internal"
port = 26379

[database.redis.sentinel_tls]
enabled = true
ca_path = "/etc/sockudo/tls/ca.pem"
client_cert_path = "/etc/sockudo/tls/client.pem"
client_key_path = "/etc/sockudo/tls/client.key"

[database.redis.master_tls]
enabled = true
ca_path = "/etc/sockudo/tls/ca.pem"
client_cert_path = "/etc/sockudo/tls/client.pem"
client_key_path = "/etc/sockudo/tls/client.key"
```

> **Note:** All Redis consumers inherit the native Sentinel topology when they do not define a URL override. An adapter, cache, queue, or rate-limiter URL override selects its standalone path; `master_tls` still applies to that direct data connection.

## Protocol features [#protocol-features]

| Section                     | Purpose                                                                                                                                                |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `[recovery]`                | V2 replay buffers and resume behavior.                                                                                                                 |
| `[history]`                 | Durable channel history.                                                                                                                               |
| `[versioned_messages]`      | V2 mutable-message storage, version paging, and retention.                                                                                             |
| `[presence_history]`        | Historical presence transitions and snapshots.                                                                                                         |
| `[annotations]`             | V2 annotation publish/delete/summary surfaces.                                                                                                         |
| `[delta]`                   | Delta algorithms, cache size, and conflation behavior.                                                                                                 |
| `[tag_filtering]`           | V2 event/tag/JMESPath subscription predicates and tag projection controls. Predicate source, AST, tree, and projected-message memory are hard-bounded. |
| `[webhooks]`                | Webhook delivery, batching, and retry.                                                                                                                 |
| `[ai_transport]`            | AI Transport validation and session-channel matching.                                                                                                  |
| `[ai_transport.rollup]`     | Append rollup egress coalescing and orphan tracking.                                                                                                   |
| `[[ai_transport.channels]]` | Channel prefixes where AI Transport validation applies.                                                                                                |
| `[mcp]`                     | Embedded Model Context Protocol server: path or dedicated port, host/origin allow-lists, rate limits, tool hiding. Requires the `mcp` Cargo feature.   |
| `[[mcp.tokens]]`            | Bearer tokens with `read` / `write` / `admin` scopes and app allow-lists. See [MCP server](/docs/server/mcp).                                          |

Annotations, including the optional Ably projection, require both global
`[annotations].enabled = true` and app/channel policy opt-in. Set
`annotations_enabled = true` under `[app_manager.array.apps.policy.channels]`
or on the matching channel namespace. Enabling the Ably facade does not bypass
either gate, and Protocol V1 never receives annotation frames.

`[versioned_messages].driver` also selects the annotation authority. `postgres`,
`mysql`, `dynamodb`, `scylladb`, and `surrealdb` each use their native atomic or
conditional commit primitives for annotation serials, stable create IDs,
canonical events, projections, replay, and retention. The corresponding Cargo
feature and database configuration must be present. `memory` is a single-node
development implementation; clustered startup rejects it.

For any non-local adapter, use `cache.driver = "redis"` or
`cache.driver = "redis-cluster"` for compatibility tokens, nonce replay,
revocations, session-owner leases, idempotency receipts, and retained stats.
Sockudo fails startup or the affected security operation when this coordination
authority is unavailable; it does not fall back to a private per-process cache.

## Push [#push]

| Section                    | Purpose                                                 |
| -------------------------- | ------------------------------------------------------- |
| `[push]`                   | Enables push, async behavior, limits, status retention. |
| `[push.providers.fcm]`     | Firebase Cloud Messaging credentials.                   |
| `[push.providers.apns]`    | Apple Push Notification service credentials.            |
| `[push.providers.webpush]` | Web Push VAPID configuration.                           |
| `[push.providers.hms]`     | Huawei Mobile Services credentials.                     |
| `[push.providers.wns]`     | Windows Notification Service credentials.               |
| `[[push_rules]]`           | Optional channel-publish to push-notification rules.    |

## Capability tokens [#capability-tokens]

Protocol V2 capability tokens do not currently have a TOML `[auth.capability_tokens]` switch. They
are accepted through the WebSocket `token` query parameter and refreshed with `sockudo:auth`.
Limits are compiled constants in `sockudo-core/src/capability_token.rs`: HS256 only, max token size
8 KiB, max `client_id` 128 bytes, max `jti` 128 bytes, max lifetime 24 hours, and 30 seconds clock
skew. Revocation uses signed HTTP `POST /apps/{appId}/revocations` and shared cache keys.

## AI Transport and Mutable Defaults [#ai-transport-and-mutable-defaults]

Runtime config uses `[versioned_messages]`; `VersionStore` is the Rust trait/storage abstraction.
There is no separate `[version_store]` TOML section in the current code.

The following block is machine checked by `sockudo-core` tests. Update it whenever code defaults
change.

```toml doc-defaults
[versioned_messages]
enabled = false
driver = "memory"
max_page_size = 100
retention_window_seconds = 0
purge_interval_seconds = 300
purge_batch_size = 1000
max_purge_per_tick = 100000

[history]
enabled = false
rewind_enabled = true
backend = "postgres"
retention_window_seconds = 86400
max_page_size = 100
writer_shards = 16
writer_queue_capacity = 4096
purge_interval_seconds = 300
purge_batch_size = 1000
max_purge_per_tick = 100000

[history.postgres]
table_prefix = "sockudo_history"
write_timeout_ms = 5000

[presence_history]
enabled = false
retention_window_seconds = 86400
max_page_size = 100

[annotations]
enabled = false

[ai_transport]
enabled = false
max_accumulated_message_bytes = 1048576
max_appends_per_message = 4096
max_transport_value_bytes = 256
max_steer_codec_message_ids_bytes = 2048
max_open_streaming_messages_per_channel = 1024

[ai_transport.rollup]
enabled = true
default_window_ms = 40
min_window_ms = 0
max_window_ms = 500
orphan_ttl_ms = 60000
wheel_tick_ms = 5
shards = 64

[push]
storage_driver = "memory"
queue_driver = "memory"
allow_memory_drivers = false
fcm_enabled = false
apns_enabled = false
webpush_enabled = false
hms_enabled = false
wns_enabled = false
accept_worker_count = 1
planner_worker_count = 1
shard_worker_count = 1
dispatch_worker_count = 1
dispatch_max_outbound_requests = 32
feedback_worker_count = 1
retry_worker_count = 1
queue_partition_count = 1
channel_shard_count = 1
fanout_fast_threshold = 10000
fanout_shard_size = 100000
fanout_sync_threshold = 0
backpressure_lag_threshold_secs = 60
publish_status_ttl_days = 30
stale_device_max_age_days = 90
dry_run = false
analytics_enabled = false
analytics_retention_days = 30
scheduler_interval_secs = 5
repair_interval_secs = 30
repair_min_age_secs = 30
repair_batch_size = 100
cleanup_interval_secs = 300
cleanup_batch_size = 1000
cleanup_max_deleted_per_tick = 100000

[push.retry]
max_attempts = 5
initial_backoff_ms = 1000
max_backoff_ms = 60000
max_elapsed_secs = 86400
jitter = true
jitter_ratio_percent = 20
respect_retry_after = true

[push.circuit_breaker]
failure_threshold = 5
cooldown_secs = 60
half_open_max_inflight = 10

[push.default_quotas]
acceptance_rps = 100
delivery_quota_daily = 0
fanout_max = 0
inflight_max = 1000

[push.payload_redaction]
redact_payload = true
redact_template_data = true
redact_provider_overrides = true
allow_debug_payload_logging = false

[[push_rules]]
enabled = true
channel_pattern = ""
event_filter = []
rate_limit_per_second = 100

[push_rules.payload_mapping]
title_field = "title"
body_field = "body"
template_data_field = "data"
include_remaining_fields = true

[mcp]
enabled = false
path = "/mcp"
# port = 6100
allowed_hosts = []
allowed_origins = []
allow_anonymous = false
anonymous_scopes = ["read"]
request_timeout_ms = 30000
max_body_bytes = 1048576
session_ttl_seconds = 1800
rate_limit_per_minute = 600
disabled_tools = []

# [[mcp.tokens]]
# name = "ops-agent"
# token = "${SOCKUDO_MCP_OPS_TOKEN}"
# scopes = ["read", "write"]
# apps = ["*"]
```

`max_messages_per_channel`, `max_bytes_per_channel`, `max_events_per_channel`, credential refs, and
external key refs default to unset and are omitted from the checked block.

`storage_driver = "memory"` and `queue_driver = "memory"` are node-local development drivers. In
`mode = "production"`, Sockudo rejects them unless `allow_memory_drivers = true` is set explicitly.
Push publish admission also requires a healthy queue, a safe store, local pipeline workers, and
local provider-worker capability. Raw provider-specific recipients require the matching provider
worker before the request is accepted; shard-path publishes also require a local shard worker.
`backpressure_lag_threshold_secs` limits the oldest actionable ready or inflight push queue item
age; set it to `0` to rely only on queue-depth thresholds.
Push repair runs in monolith workers when `repair_interval_secs > 0`; it scans stale queued
durable publish logs and recreates missing `push.publish.v1` queue work after the publish has been
queued for at least `repair_min_age_secs`.

Push cleanup runs in monolith workers when `cleanup_interval_secs > 0`. It purges terminal publish
statuses older than `publish_status_ttl_days`, delivery events and operator invalidations older
than `analytics_retention_days`, expired idempotency records, expired scheduler locks, and any
persisted dead-letter inspection rows written by a selected backend. Queue-native dead-letter
metadata is inspected through `/apps/{appId}/push/deadLetters`. Cleanup is bounded by
`cleanup_batch_size` per category and `cleanup_max_deleted_per_tick` overall.

## Observability [#observability]

| Section                                        | Purpose                                                                            |
| ---------------------------------------------- | ---------------------------------------------------------------------------------- |
| `[metrics]`                                    | Enables metrics and configures the Prometheus scrape endpoint.                     |
| `[metrics.prometheus]`                         | Prometheus metric naming options.                                                  |
| `[metrics.tcp_exporter]`                       | Optional metrics-rs TCP event exporter for live clients and sidecars.              |
| `[opentelemetry]`                              | Optional OTLP traces, metrics, logs, resource identity, batching, and propagation. |
| `[logging]`                                    | Controls ANSI colors and tracing-target inclusion for text/JSON output.            |
| [logging environment](/docs/reference/logging) | Runtime filters, structured output, lifecycle fields, and data-safety policy.      |

OpenTelemetry is disabled by default and is additive to Prometheus and local logs:

```toml
[opentelemetry]
enabled = true
traces_enabled = true
metrics_enabled = true
logs_enabled = true
service_name = "sockudo"
service_namespace = "realtime"
deployment_environment = "production"
resource_attributes = { "cloud.region" = "eu-central-1" }
endpoint = "http://otel-collector:4317"
export_timeout_ms = 10000
batch_scheduled_delay_ms = 5000
batch_max_queue_size = 2048
batch_max_export_batch_size = 512
metric_export_interval_ms = 60000
propagation_trace_context = true
propagation_baggage = true
```

| Key                           | Default   | Purpose                                                                                 |
| ----------------------------- | --------- | --------------------------------------------------------------------------------------- |
| `enabled`                     | `false`   | Initializes OpenTelemetry and enables the selected signals.                             |
| `traces_enabled`              | `true`    | Exports spans when OpenTelemetry is enabled.                                            |
| `metrics_enabled`             | `true`    | Exports Sockudo metrics through OTLP independently of the Prometheus listener.          |
| `logs_enabled`                | `true`    | Exports structured log records while retaining local output.                            |
| `service_name`                | `sockudo` | OpenTelemetry `service.name`. Must not be empty when enabled.                           |
| `service_namespace`           | unset     | Optional `service.namespace`.                                                           |
| `deployment_environment`      | unset     | Optional `deployment.environment.name`.                                                 |
| `resource_attributes`         | `{}`      | Additional string-valued resource attributes. Never store secrets here.                 |
| `endpoint`                    | unset     | Optional common OTLP endpoint. When unset, standard SDK/exporter configuration applies. |
| `export_timeout_ms`           | `10000`   | Timeout for one exporter request.                                                       |
| `batch_scheduled_delay_ms`    | `5000`    | Maximum span/log batching delay.                                                        |
| `batch_max_queue_size`        | `2048`    | Bounded batch processor queue capacity.                                                 |
| `batch_max_export_batch_size` | `512`     | Maximum export batch size; cannot exceed the queue capacity.                            |
| `metric_export_interval_ms`   | `60000`   | Periodic metric export interval.                                                        |
| `propagation_trace_context`   | `true`    | Extracts and injects W3C `traceparent` and `tracestate`.                                |
| `propagation_baggage`         | `true`    | Extracts and injects W3C baggage. Do not place secrets in baggage.                      |

Use standard `OTEL_*` environment variables for the OTLP protocol (`grpc`, `http/protobuf`, or
`http/json`), signal-specific endpoints, headers, compression, sampling, and SDK limits.
The complete mapping is in the
[environment variable reference](/docs/reference/environment-variables#opentelemetry).
HTTPS exporters use platform/web PKI roots; custom CA bundles and mTLS exporter configuration are
not supported by the current Rust SDK integration.
The shipped CORS configuration allows `traceparent`, `tracestate`, and `baggage`; retain those
headers in a custom allowlist when browser clients participate in distributed traces.

Collector failures are fail-open: export uses bounded queues and does not change request handling,
`/live`, or `/up`. Invalid static settings are still rejected during startup. Stable traces,
metrics, and logs are supported; OpenTelemetry profiles are not.

## Example production skeleton [#example-production-skeleton]

```toml
host = "0.0.0.0"
port = 6001
debug = false

[adapter]
driver = "redis"

[cache]
driver = "redis"

[queue]
driver = "redis"

[metrics]
enabled = true
port = 9601

[metrics.tcp_exporter]
enabled = false
host = "127.0.0.1"
port = 5000
buffer_size = 1024

[push]
storage_driver = "postgres"
queue_driver = "redis"
publish_status_ttl_days = 30
analytics_retention_days = 30
cleanup_interval_secs = 300
```

## Queue v2 reliability [#queue-v2-reliability]

Queue v2 provides at-least-once delivery. Successful processors acknowledge a
job; errors retry with bounded exponential backoff and jitter; exhausted jobs
move to a dead-letter queue. Exactly-once execution is not promised, so handlers
must remain idempotent. Redis, Redis Sentinel, and Redis Cluster additionally
provide stable job IDs, deduplication, delayed jobs, renewable leases, stalled
worker recovery, lifecycle events, and queue depth statistics through atomic Lua
transitions.

Redis workers preserve the configured callback concurrency while pipelining
Redis I/O. `worker_prefetch` bounds the number of leased jobs buffered ahead of
each callback worker; the default `16` amortizes claim, acknowledgement, and
lease-renewal round trips without creating unbounded memory or failover
exposure. Lower it for a smaller redelivery window, or raise it when Redis
latency leaves callback workers idle. `queue.redis.response_timeout_ms`
defaults to `5000`, allowing Sentinel clients to discard a stale primary
connection promptly; `0` disables that deadline.

For the legacy-equivalent high-throughput workload—batch enqueue, generated
job IDs, immediate delivery, the configured default attempt count, no
application deduplication key, and `event_retention = 0`—the Redis drivers use
compact length-delimited frames and one raw atomic Redis transaction. Workers
materialize each frame into the same leased Queue v2 state machine when they
claim it, so retries, acknowledgements, stalled recovery, and dead lettering
are unchanged while admission avoids per-job Redis writes. Supplying a stable
`job_id`, delay, deduplication key, attempt override, or lifecycle-event
retention selects the stricter per-job atomic path. Use that path when a caller
must safely retry an enqueue whose network result was ambiguous.

```toml
[queue]
driver = "redis" # memory, redis, redis-cluster, nats, rabbitmq, kafka, iggy, pulsar, google-pubsub, sqs, sns

[queue.reliability]
max_attempts = 5
retry_base_delay_ms = 1000
retry_max_delay_ms = 60000
retry_jitter = 0.2
lease_duration_ms = 30000
lease_renew_interval_ms = 10000
stalled_batch_size = 100
worker_poll_interval_ms = 500
worker_prefetch = 16
shutdown_timeout_ms = 30000
completed_retention = 1000
failed_retention = 10000
event_retention = 10000
deduplication_ttl_ms = 300000
memory_capacity = 100000
max_batch_size = 1000
```

Broker-backed queues use native bulk/pipelined publishing rather than the
compatibility loop: NATS and RabbitMQ pipeline acknowledgements, Kafka feeds
librdkafka concurrently, Iggy and Pulsar use their producer batch APIs, Google
Pub/Sub reuses its batching publisher, and SQS/SNS use provider batches of at
most 10 entries. `max_batch_size` is an upper bound; a provider's smaller hard
limit still applies. Topic, stream, queue, and publisher handles are cached
after successful provisioning instead of issuing control-plane requests for
every job.

`worker_prefetch` also bounds concurrent callbacks for NATS, RabbitMQ, Pulsar,
Google Pub/Sub, and standard SQS queues. SQS FIFO batches remain serial to
preserve message-group order. `max_attempts` configures NATS delivery limits,
Kafka and Iggy retry/DLQ transitions, RabbitMQ republish/DLQ transitions, and
Pulsar's native dead-letter policy. SQS retry/dead-letter limits remain an AWS
queue redrive-policy concern; Sockudo does not advertise a DLQ unless it owns
that policy.

Queue v2 options are fail-closed. A broker that cannot honor per-job delay,
deduplication, or attempt overrides returns an error instead of silently
discarding the option. SQS standard queues support delays up to 15 minutes;
SQS FIFO queues support stable IDs/deduplication but not per-message delay.

The Redis drivers use a new `:v2:{queue-id}:*` key namespace. All keys for one
logical queue share a Redis Cluster hash tag, so enqueue, claim, ack, retry,
lease renewal, delayed promotion, and dead-letter transitions are single-slot
and atomic. Existing RPUSH/BLPOP list keys are not deleted during startup or
shutdown; drain them with the previous release before switching producers.

When `database.redis.sentinels` is configured and `queue.driver = "redis"`, the
queue resolves the current primary through native Sentinel support. Sentinel and
primary authentication/TLS settings remain independent. A `queue.redis.url_override`
selects standalone Redis/rediss instead and intentionally disables inherited
Sentinel discovery for that queue.

Primary promotion preserves at-least-once, not exactly-once, semantics. A hard
primary loss can redeliver callbacks that were active or prefetched when the node failed; a
planned Sentinel switchover can temporarily expose a larger duplicate window
while the old primary is still reachable. Use the stable job ID as an
idempotency key, and quiesce producers/workers before a planned switchover when
duplicate callbacks must be minimized.

SNS is producer-only. Configuring a consumer on SNS now fails instead of
reporting a successful no-op. A requested backend that was not compiled into the
binary also fails startup instead of silently falling back to memory.


# Environment variables (/docs/reference/environment-variables)



Sockudo loads configuration from files first, then applies supported environment variable
overrides. Environment variables are explicit mappings, not a generic alternative syntax for every
nested TOML or JSON field. Use them for deployment-specific values and secret injection; keep
complex app policy, retention, queue reliability, and feature structure in a static file or
persistent manager. See [Static configuration](/docs/deployment/static-configuration) for the full
precedence and equivalent file examples.

Values are parsed as strings unless the target option is numeric or boolean. Boolean values follow
the server parser and accept common true/false forms.

This page lists every runtime environment variable referenced by the server, core configuration loader, push subsystem, app managers, adapters, queues, and provider workers.

## Config File Interpolation [#config-file-interpolation]

Sockudo supports `${VAR}` environment variable interpolation in TOML and JSON
configuration files. Variables are resolved from the process environment at
startup, before the config file is parsed.

### Syntax [#syntax]

| Pattern           | Behavior                                                                    |
| ----------------- | --------------------------------------------------------------------------- |
| `${VAR}`          | Replaced with the value of `VAR`. &#x2A;*Startup error if unset or empty.** |
| `${VAR:-default}` | Replaced with the value of `VAR`, or `default` if unset or empty.           |

### Examples [#examples]

Inject a secret from the environment (e.g. from AWS Secrets Manager via a pod environment variable):

```toml
[[app_manager.array.apps]]
id = "my-app"
key = "app-key"
secret = "${SOCKUDO_APP_SECRET}"
enabled = true
```

Use defaults for optional values:

```toml
port = 6001

[adapter]
driver = "${ADAPTER_DRIVER:-local}"

[adapter.redis.redis_pub_options]
url = "${REDIS_URL:-redis://localhost:6379}"
```

### Behavior [#behavior]

* **Missing variables**: If a `${VAR}` reference has no default and the
  variable is unset or empty, sockudo prints an error listing **all**
  unresolved variables and exits. This is fail-fast by design — you see
  every missing variable at once, not one at a time.

* **Empty values**: An environment variable set to an empty string (`VAR=""`)
  is treated as unset. The `:-default` fallback will activate.

* **Single pass**: Interpolation runs once on the raw file text. If a
  resolved value itself contains `${…}`, it is **not** re-expanded. This
  prevents accidental recursion and injection.

* **No bare `$VAR`**: Only the braced `${VAR}` syntax is recognized. A bare
  `$` character (e.g., in regex patterns or currency symbols) is never
  matched.

### Limitations [#limitations]

* **Comments**: Interpolation operates on raw text before TOML/JSON parsing.
  A `${VAR}` reference inside a TOML comment (`# secret = "${MISSING}"`)
  will still be resolved — and will cause a startup error if the variable
  is missing. Remove or replace `${…}` patterns in comments if they
  reference variables that are not set.

* **Typed fields**: Interpolation produces string replacements. For string
  config values (secrets, URLs, connection strings), this works naturally.
  For numeric or boolean fields, prefer the existing environment variable
  overrides instead.

* **Defaults cannot contain `}`**: The default value in `${VAR:-default}`
  ends at the first `}` character. This is sufficient for most values
  (strings, URLs, ports) but not for JSON fragments.

## Process and listener [#process-and-listener]

| Variable                                     | Purpose                                                                                                                                      |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENVIRONMENT`                                | Runtime mode label.                                                                                                                          |
| `DEBUG`                                      | Forces debug mode on when truthy.                                                                                                            |
| `DEBUG_MODE`                                 | Overrides the configured debug flag.                                                                                                         |
| `HOST`                                       | Main HTTP and WebSocket bind host.                                                                                                           |
| `PORT`                                       | Main HTTP and WebSocket bind port.                                                                                                           |
| `ACTIVITY_TIMEOUT`                           | Pusher protocol activity timeout.                                                                                                            |
| `SHUTDOWN_GRACE_PERIOD`                      | Grace period for server shutdown.                                                                                                            |
| `USER_AUTHENTICATION_TIMEOUT`                | Timeout used by user authentication flows.                                                                                                   |
| `SOCKUDO_MAX_CONNECTIONS`                    | Per-node WebSocket connection limit. `0` disables.                                                                                           |
| `WEBSOCKET_MAX_PAYLOAD_KB`                   | Legacy maximum WebSocket payload size in KiB.                                                                                                |
| `SOCKUDO_SERVER_ROLE`                        | Deployment topology role. `"default"` or `"api"`. Overrides `server_role` in the config file.                                                |
| `INSTANCE_PROCESS_ID`                        | Stable node/process identity used by clustering and Iggy consumers.                                                                          |
| `HOSTNAME`                                   | Fallback node identity for Iggy when `INSTANCE_PROCESS_ID` is not set.                                                                       |
| `HEALTH_CHECK_TIMEOUT_MS`                    | Per-dependency timeout for `/up` health checks in milliseconds. Defaults to `2000`.                                                          |
| `HTTP_API_USAGE_ENABLED`                     | Enables `/usage` and `/stats` operational endpoints.                                                                                         |
| `HTTP_API_ACCEPT_TRAFFIC_ENABLED`            | Enables memory-pressure admission control for new WebSocket connections. Defaults to `false`.                                                |
| `HTTP_API_ACCEPT_TRAFFIC_MEMORY_THRESHOLD`   | Fraction of the effective memory limit that closes new-connection admission. Defaults to `0.9`; valid range is greater than `0` through `1`. |
| `HTTP_API_ACCEPT_TRAFFIC_MEMORY_LIMIT_BYTES` | Optional explicit memory limit in bytes. When unset, Linux cgroup v2/v1 limits are discovered.                                               |
| `HTTP_API_ACCEPT_TRAFFIC_SAMPLE_INTERVAL_MS` | Independent memory-pressure sampling interval in milliseconds. Defaults to `500`.                                                            |

## Logging [#logging]

| Variable             | Purpose                                           |
| -------------------- | ------------------------------------------------- |
| `RUST_LOG`           | Highest-precedence tracing filter.                |
| `SOCKUDO_LOG_DEBUG`  | Default log filter when debug mode is active.     |
| `SOCKUDO_LOG_PROD`   | Default log filter when debug mode is not active. |
| `LOG_OUTPUT_FORMAT`  | Set to `json` for JSON logs.                      |
| `LOG_COLORS_ENABLED` | Enables or disables colored text logs.            |
| `LOG_INCLUDE_TARGET` | Includes tracing targets in text or JSON logs.    |

See [Logging](/docs/reference/logging) for level policy, stable lifecycle fields, and data-safety requirements.

## OpenTelemetry [#opentelemetry]

OpenTelemetry is disabled unless `[opentelemetry].enabled` or `SOCKUDO_OTEL_ENABLED` is true.
When enabled, Sockudo can export the stable OpenTelemetry traces, metrics, and logs signals over
OTLP. Export is additive to local logs and the Prometheus `/metrics` endpoint. OpenTelemetry
profiles are not supported.

### Sockudo controls [#sockudo-controls]

These variables map directly to the `[opentelemetry]` configuration block:

| Variable                                   | Default     | Purpose                                                                                      |
| ------------------------------------------ | ----------- | -------------------------------------------------------------------------------------------- |
| `SOCKUDO_OTEL_ENABLED`                     | `false`     | Enables OpenTelemetry initialization and export.                                             |
| `SOCKUDO_OTEL_TRACES_ENABLED`              | `true`      | Enables trace export while OpenTelemetry is enabled.                                         |
| `SOCKUDO_OTEL_METRICS_ENABLED`             | `true`      | Enables OTLP metric export while preserving Prometheus independently.                        |
| `SOCKUDO_OTEL_LOGS_ENABLED`                | `true`      | Enables OTLP log export while preserving local log output.                                   |
| `SOCKUDO_OTEL_SERVICE_NAME`                | `sockudo`   | Sets the `service.name` resource attribute.                                                  |
| `SOCKUDO_OTEL_SERVICE_NAMESPACE`           | unset       | Sets `service.namespace`. An empty value clears it.                                          |
| `SOCKUDO_OTEL_DEPLOYMENT_ENVIRONMENT`      | unset       | Sets `deployment.environment.name`. An empty value clears it.                                |
| `SOCKUDO_OTEL_RESOURCE_ATTRIBUTES`         | unset       | Comma-separated `key=value` resource attributes. Invalid entries are ignored with a warning. |
| `SOCKUDO_OTEL_ENDPOINT`                    | SDK default | Common OTLP collector endpoint used by enabled signals.                                      |
| `SOCKUDO_OTEL_EXPORT_TIMEOUT_MS`           | `10000`     | Request timeout for an OTLP export attempt.                                                  |
| `SOCKUDO_OTEL_BATCH_SCHEDULED_DELAY_MS`    | `5000`      | Maximum trace/log batching delay.                                                            |
| `SOCKUDO_OTEL_BATCH_MAX_QUEUE_SIZE`        | `2048`      | Bounded trace/log export queue capacity.                                                     |
| `SOCKUDO_OTEL_BATCH_MAX_EXPORT_BATCH_SIZE` | `512`       | Maximum trace/log records per export batch; cannot exceed queue capacity.                    |
| `SOCKUDO_OTEL_METRIC_EXPORT_INTERVAL_MS`   | `60000`     | Periodic metric export interval.                                                             |
| `SOCKUDO_OTEL_PROPAGATION_TRACE_CONTEXT`   | `true`      | Enables W3C `traceparent` and `tracestate` extraction and injection.                         |
| `SOCKUDO_OTEL_PROPAGATION_BAGGAGE`         | `true`      | Enables W3C baggage extraction and injection.                                                |

### Standard OpenTelemetry SDK and OTLP variables [#standard-opentelemetry-sdk-and-otlp-variables]

Sockudo also honors the standard `OTEL_*` SDK and exporter variables implemented by the bundled
Rust SDK. Use these for protocol, sampling, per-signal endpoints, compression, and authenticated
collector headers. A
signal-specific exporter value is more specific than its common OTLP equivalent.

| Variable or family                                                                             | Purpose                                                                                                         |
| ---------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `OTEL_SDK_DISABLED`                                                                            | Hard-disables the OpenTelemetry SDK when true.                                                                  |
| `OTEL_SERVICE_NAME`                                                                            | Standard `service.name` resource setting.                                                                       |
| `OTEL_RESOURCE_ATTRIBUTES`                                                                     | Standard comma-separated resource attributes.                                                                   |
| `OTEL_PROPAGATORS`                                                                             | Comma-separated `tracecontext` and `baggage` propagators. Other propagator names are ignored.                   |
| `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER`, `OTEL_LOGS_EXPORTER`                          | Selects `otlp` or disables an individual signal with `none`.                                                    |
| `OTEL_EXPORTER_OTLP_ENDPOINT`                                                                  | Common OTLP endpoint. For HTTP it is a base URL and the SDK appends `/v1/traces`, `/v1/metrics`, or `/v1/logs`. |
| `OTEL_EXPORTER_OTLP_PROTOCOL`                                                                  | `grpc`, `http/protobuf`, or `http/json`.                                                                        |
| `OTEL_EXPORTER_OTLP_HEADERS`                                                                   | Comma-separated exporter headers. Treat the complete value as a secret.                                         |
| `OTEL_EXPORTER_OTLP_COMPRESSION`                                                               | Export compression: `gzip` or `zstd`.                                                                           |
| `OTEL_EXPORTER_OTLP_TIMEOUT`                                                                   | Common exporter timeout in milliseconds.                                                                        |
| `OTEL_EXPORTER_OTLP_TRACES_*`                                                                  | Trace-specific endpoint, protocol, headers, compression, and timeout.                                           |
| `OTEL_EXPORTER_OTLP_METRICS_*`                                                                 | Metric-specific endpoint, protocol, headers, compression, timeout, and temporality preference.                  |
| `OTEL_EXPORTER_OTLP_LOGS_*`                                                                    | Log-specific endpoint, protocol, headers, compression, and timeout.                                             |
| `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG`                                               | Head sampler and its argument; for example `parentbased_traceidratio` and `0.10`.                               |
| `OTEL_BSP_SCHEDULE_DELAY`, `OTEL_BSP_MAX_QUEUE_SIZE`, `OTEL_BSP_MAX_EXPORT_BATCH_SIZE`         | Batch span processor settings. The delay is in milliseconds.                                                    |
| `OTEL_BLRP_SCHEDULE_DELAY`, `OTEL_BLRP_MAX_QUEUE_SIZE`, `OTEL_BLRP_MAX_EXPORT_BATCH_SIZE`      | Batch log-record processor settings. The delay is in milliseconds.                                              |
| `OTEL_METRIC_EXPORT_INTERVAL`                                                                  | Periodic metric reader interval in milliseconds.                                                                |
| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE`                                            | Metric temporality preference.                                                                                  |
| `OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT`, `OTEL_SPAN_EVENT_COUNT_LIMIT`, `OTEL_SPAN_LINK_COUNT_LIMIT` | Trace span limits implemented by the bundled SDK.                                                               |

The standard signal-specific endpoint is used exactly as supplied. With HTTP, include its final
signal path when setting a signal-specific endpoint. Keep `OTEL_EXPORTER_OTLP_HEADERS` and private
API tokens in a secret store rather than a configuration file or command line. HTTPS exporters use
the bundled platform/web PKI roots. Custom CA bundles, client certificates, and mTLS exporter
configuration are not supported by the current Rust SDK integration.

Exporter connection failures, timeouts, and bounded-queue drops do not fail requests and do not
change `/live` or `/up`. Invalid Sockudo configuration still fails validation at startup. See
[Logging](/docs/reference/logging#opentelemetry-export) for log correlation and data-safety details.

## Driver selection [#driver-selection]

| Variable                            | Purpose                                                                                                   |
| ----------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `ADAPTER_DRIVER`                    | Horizontal fanout driver.                                                                                 |
| `CACHE_DRIVER`                      | Cache driver.                                                                                             |
| `QUEUE_DRIVER`                      | Queue driver.                                                                                             |
| `APP_MANAGER_DRIVER`                | App manager driver.                                                                                       |
| `RATE_LIMITER_DRIVER`               | Rate limiter backend driver.                                                                              |
| `ADAPTER_BUFFER_MULTIPLIER_PER_CPU` | Adapter fanout buffer sizing multiplier.                                                                  |
| `ADAPTER_ENABLE_SOCKET_COUNTING`    | Enables adapter socket counting.                                                                          |
| `ADAPTER_AGGREGATE_COUNTS`          | Enables gossiped aggregate channel counts for local count reads.                                          |
| `ADAPTER_FAST_PRESENCE_TRANSITIONS` | Uses the replicated presence registry for first-join/last-leave checks; faster but eventually consistent. |
| `ADAPTER_FALLBACK_TO_LOCAL`         | Falls back to the local adapter when a configured adapter cannot start.                                   |
| `APP_MANAGER_REGISTER_INLINE_APPS`  | Allows inline app definitions from config to be registered.                                               |
| `SOCKUDO_SKIP_INLINE_APPS`          | Skips inline apps even if present in the config file.                                                     |

## Default app bootstrap [#default-app-bootstrap]

| Variable                                               | Purpose                                                   |
| ------------------------------------------------------ | --------------------------------------------------------- |
| `SOCKUDO_DEFAULT_APP_ID`                               | Default app ID created from environment.                  |
| `SOCKUDO_DEFAULT_APP_KEY`                              | Default app key created from environment.                 |
| `SOCKUDO_DEFAULT_APP_SECRET`                           | Default app secret created from environment.              |
| `SOCKUDO_DEFAULT_APP_ENABLED`                          | Enables or disables default app registration.             |
| `SOCKUDO_DEFAULT_APP_MAX_CONNECTIONS`                  | Default app connection limit.                             |
| `SOCKUDO_DEFAULT_APP_MAX_BACKEND_EVENTS_PER_SECOND`    | Default app backend publish rate limit.                   |
| `SOCKUDO_DEFAULT_APP_MAX_CLIENT_EVENTS_PER_SECOND`     | Default app client event rate limit.                      |
| `SOCKUDO_DEFAULT_APP_MAX_READ_REQUESTS_PER_SECOND`     | Default app read API rate limit.                          |
| `SOCKUDO_DEFAULT_APP_MAX_PRESENCE_MEMBERS_PER_CHANNEL` | Default app presence member limit per channel.            |
| `SOCKUDO_DEFAULT_APP_MAX_PRESENCE_MEMBER_SIZE_IN_KB`   | Default app presence member payload size limit.           |
| `SOCKUDO_DEFAULT_APP_MAX_CHANNEL_NAME_LENGTH`          | Default app channel name length limit.                    |
| `SOCKUDO_DEFAULT_APP_MAX_EVENT_CHANNELS_AT_ONCE`       | Default app maximum fanout channels per event.            |
| `SOCKUDO_DEFAULT_APP_MAX_EVENT_NAME_LENGTH`            | Default app event name length limit.                      |
| `SOCKUDO_DEFAULT_APP_MAX_EVENT_PAYLOAD_IN_KB`          | Default app event payload size limit.                     |
| `SOCKUDO_DEFAULT_APP_MAX_EVENT_BATCH_SIZE`             | Default app batch publish item limit.                     |
| `SOCKUDO_DEFAULT_APP_ENABLE_CLIENT_MESSAGES`           | Enables client events for the default app.                |
| `SOCKUDO_ENABLE_CLIENT_MESSAGES`                       | Legacy fallback for default app client events.            |
| `SOCKUDO_DEFAULT_APP_ENABLE_USER_AUTHENTICATION`       | Enables user authentication features for the default app. |
| `SOCKUDO_DEFAULT_APP_ENABLE_WATCHLIST_EVENTS`          | Enables watchlist events for the default app.             |
| `SOCKUDO_DEFAULT_APP_ALLOWED_ORIGINS`                  | Comma-separated origin allowlist for the default app.     |
| `SOCKUDO_DEFAULT_APP_ANNOTATIONS_ENABLED`              | Enables annotations for the default app channel policy.   |

## Ably compatibility credentials [#ably-compatibility-credentials]

| Variable                                              | Purpose                                                                                                                                                                                                                |
| ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SOCKUDO_ABLY_COMPAT_ENABLED`                         | Enables the optional additional-key registry. The primary `App.key`/`App.secret` remains available independently.                                                                                                      |
| `SOCKUDO_ABLY_COMPAT_REALTIME_ADMISSION`              | Ably WebSocket admission mode: `accept` (default) or `placement_constraint`, which returns `DISCONNECTED` error `50320` so clients try configured WebSocket fallback hosts. No fallback realtime transport is enabled. |
| `SOCKUDO_ABLY_COMPAT_ATTACH_TIMEOUT_MS`               | Maximum native ATTACH processing time before Sockudo removes the partial subscription and returns `DETACHED`/`50003`. Defaults to `10000`.                                                                             |
| `SOCKUDO_ABLY_COMPAT_KEYS_JSON`                       | JSON array of additional keys with `app_id`, `key_name`, `secret`, optional `capability`, `revocable_tokens`, `enabled`, and rotation metadata. Treat this value as a secret.                                          |
| `SOCKUDO_ABLY_COMPAT_MAX_TOKEN_TTL_MS`                | Maximum accepted TokenRequest TTL in milliseconds. Defaults to 24 hours.                                                                                                                                               |
| `SOCKUDO_ABLY_COMPAT_TOKEN_REQUEST_TIMESTAMP_SKEW_MS` | Maximum absolute TokenRequest timestamp skew. Defaults to 15 minutes.                                                                                                                                                  |
| `SOCKUDO_ABLY_COMPAT_NONCE_TTL_SECONDS`               | Shared replay-protection TTL for TokenRequest nonces. Defaults to 15 minutes.                                                                                                                                          |
| `SOCKUDO_ABLY_COMPAT_STATS_FIXTURE_INGEST_ENABLED`    | Enables authenticated conformance fixture writes through `POST /stats`. Defaults to `false`; do not enable on public deployments.                                                                                      |
| `SOCKUDO_ABLY_COMPAT_STATS_QUEUE_CAPACITY`            | Maximum bounded aggregation backlog. Defaults to `4096`.                                                                                                                                                               |
| `SOCKUDO_ABLY_COMPAT_STATS_FLUSH_INTERVAL_MS`         | Maximum batching delay before a stats-store merge. Defaults to `10`.                                                                                                                                                   |
| `SOCKUDO_ABLY_COMPAT_STATS_RETENTION_SECONDS`         | Cache retention for canonical minute buckets. Defaults to 400 days.                                                                                                                                                    |
| `SOCKUDO_ABLY_COMPAT_STATS_MAX_SCAN_ENTRIES`          | Maximum minute buckets read by one stats query. Defaults to `100000`.                                                                                                                                                  |
| `SOCKUDO_ABLY_COMPAT_STATS_CAS_RETRIES`               | Maximum compare-and-swap retries when merging a shared bucket. Defaults to `8`.                                                                                                                                        |

## Redis [#redis]

| Variable                                           | Purpose                                                                                                                  |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `REDIS_URL`                                        | Shared Redis URL override for adapter, cache, queue, and rate limiter Redis clients.                                     |
| `DATABASE_REDIS_HOST`                              | Redis host.                                                                                                              |
| `DATABASE_REDIS_PORT`                              | Redis port.                                                                                                              |
| `DATABASE_REDIS_USERNAME`                          | Redis username.                                                                                                          |
| `DATABASE_REDIS_PASSWORD`                          | Redis password.                                                                                                          |
| `DATABASE_REDIS_DB`                                | Redis database index.                                                                                                    |
| `DATABASE_REDIS_KEY_PREFIX`                        | Redis key prefix.                                                                                                        |
| `DATABASE_REDIS_SENTINEL_USERNAME`                 | ACL username for authenticating to the Sentinel nodes.                                                                   |
| `DATABASE_REDIS_SENTINEL_PASSWORD`                 | Password for authenticating to the Sentinel nodes.                                                                       |
| `DATABASE_REDIS_SENTINEL_TLS_ENABLED`              | Enables TLS for the connection to the Sentinel nodes.                                                                    |
| `DATABASE_REDIS_SENTINEL_TLS_ACCEPT_INVALID_CERTS` | Skips certificate/hostname verification for the Sentinel connection (dangerous).                                         |
| `DATABASE_REDIS_SENTINEL_TLS_CA_PATH`              | PEM CA certificate path for the Sentinel connection (private CAs).                                                       |
| `DATABASE_REDIS_SENTINEL_TLS_CLIENT_CERT_PATH`     | PEM client certificate path for mutual TLS to the Sentinel nodes.                                                        |
| `DATABASE_REDIS_SENTINEL_TLS_CLIENT_KEY_PATH`      | PEM client private key path for mutual TLS to the Sentinel nodes.                                                        |
| `DATABASE_REDIS_MASTER_TLS_ENABLED`                | Enables TLS for direct Redis, Redis Cluster, or the Sentinel-resolved master/replica data connection.                    |
| `DATABASE_REDIS_MASTER_TLS_ACCEPT_INVALID_CERTS`   | Skips certificate/hostname verification for Redis data connections (dangerous).                                          |
| `DATABASE_REDIS_MASTER_TLS_CA_PATH`                | PEM CA certificate path for Redis data connections (private CAs).                                                        |
| `DATABASE_REDIS_MASTER_TLS_CLIENT_CERT_PATH`       | PEM client certificate path for mutual TLS to Redis data connections.                                                    |
| `DATABASE_REDIS_MASTER_TLS_CLIENT_KEY_PATH`        | PEM client private key path for mutual TLS to Redis data connections.                                                    |
| `DATABASE_REDIS_CLUSTER_NODES`                     | Comma-separated Redis Cluster seed nodes.                                                                                |
| `REDIS_CLUSTER_NODES`                              | Comma-separated Redis Cluster seed nodes, also applied to adapter and queue cluster config.                              |
| `DATABASE_REDIS_CLUSTER_USERNAME`                  | Redis Cluster username.                                                                                                  |
| `DATABASE_REDIS_CLUSTER_PASSWORD`                  | Redis Cluster password.                                                                                                  |
| `DATABASE_REDIS_CLUSTER_USE_TLS`                   | Enables TLS for Redis Cluster with the default root store; use the master-TLS variables for private CA or mTLS material. |
| `REDIS_CLUSTER_QUEUE_CONCURRENCY`                  | Redis Cluster queue worker concurrency.                                                                                  |
| `REDIS_CLUSTER_QUEUE_PREFIX`                       | Redis Cluster queue key prefix.                                                                                          |

## SQL and app storage [#sql-and-app-storage]

| Variable                        | Purpose                                                          |
| ------------------------------- | ---------------------------------------------------------------- |
| `DATABASE_MYSQL_HOST`           | MySQL host.                                                      |
| `DATABASE_MYSQL_PORT`           | MySQL port.                                                      |
| `DATABASE_MYSQL_USERNAME`       | MySQL username.                                                  |
| `DATABASE_MYSQL_PASSWORD`       | MySQL password.                                                  |
| `DATABASE_MYSQL_DATABASE`       | MySQL database name.                                             |
| `DATABASE_MYSQL_TABLE_NAME`     | MySQL app table name.                                            |
| `DATABASE_MYSQL_POOL_MIN`       | MySQL minimum pool size.                                         |
| `DATABASE_MYSQL_POOL_MAX`       | MySQL maximum pool size.                                         |
| `DATABASE_POSTGRES_HOST`        | PostgreSQL host.                                                 |
| `DATABASE_POSTGRES_PORT`        | PostgreSQL port.                                                 |
| `DATABASE_POSTGRES_USERNAME`    | PostgreSQL username used by the core config loader.              |
| `DATABASE_POSTGRES_USER`        | PostgreSQL username fallback used by the PostgreSQL app manager. |
| `DATABASE_POSTGRES_PASSWORD`    | PostgreSQL password.                                             |
| `DATABASE_POSTGRES_DATABASE`    | PostgreSQL database name.                                        |
| `DATABASE_POSTGRES_POOL_MIN`    | PostgreSQL minimum pool size.                                    |
| `DATABASE_POSTGRES_POOL_MAX`    | PostgreSQL maximum pool size.                                    |
| `DATABASE_POOLING_ENABLED`      | Enables database pooling.                                        |
| `DATABASE_POOL_MIN`             | Global minimum pool size.                                        |
| `DATABASE_POOL_MAX`             | Global maximum pool size.                                        |
| `DATABASE_CONNECTION_POOL_SIZE` | Global connection pool size applied to MySQL and PostgreSQL.     |

## DynamoDB, SurrealDB, and ScyllaDB [#dynamodb-surrealdb-and-scylladb]

| Variable                         | Purpose                                                |
| -------------------------------- | ------------------------------------------------------ |
| `DATABASE_DYNAMODB_REGION`       | DynamoDB region.                                       |
| `DATABASE_DYNAMODB_TABLE_NAME`   | DynamoDB table name.                                   |
| `DATABASE_DYNAMODB_ENDPOINT_URL` | DynamoDB custom endpoint URL.                          |
| `AWS_ACCESS_KEY_ID`              | AWS access key used by DynamoDB and AWS-backed queues. |
| `AWS_SECRET_ACCESS_KEY`          | AWS secret key used by DynamoDB and AWS-backed queues. |
| `DATABASE_SURREALDB_URL`         | SurrealDB connection URL.                              |
| `DATABASE_SURREALDB_NAMESPACE`   | SurrealDB namespace.                                   |
| `DATABASE_SURREALDB_DATABASE`    | SurrealDB database.                                    |
| `DATABASE_SURREALDB_USERNAME`    | SurrealDB username.                                    |
| `DATABASE_SURREALDB_PASSWORD`    | SurrealDB password.                                    |
| `DATABASE_SURREALDB_TABLE_NAME`  | SurrealDB app table name.                              |
| `SCYLLADB_NODES`                 | Comma-separated ScyllaDB nodes.                        |
| `SCYLLADB_KEYSPACE`              | ScyllaDB keyspace.                                     |
| `SCYLLADB_REPLICATION_CLASS`     | ScyllaDB replication class.                            |
| `SCYLLADB_REPLICATION_FACTOR`    | ScyllaDB replication factor.                           |

## Cache [#cache]

| Variable                 | Purpose                                                                 |
| ------------------------ | ----------------------------------------------------------------------- |
| `CACHE_TTL_SECONDS`      | Cache TTL applied to app, channel, database, and memory cache settings. |
| `CACHE_CLEANUP_INTERVAL` | Cache cleanup interval for in-memory and database cache paths.          |
| `CACHE_MAX_CAPACITY`     | Cache maximum capacity for supported cache stores.                      |

## TLS, Unix sockets, and CORS [#tls-unix-sockets-and-cors]

| Variable                      | Purpose                                                                                                                                                               |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SSL_ENABLED`                 | Enables TLS listener mode.                                                                                                                                            |
| `SSL_CERT_PATH`               | TLS certificate path.                                                                                                                                                 |
| `SSL_KEY_PATH`                | TLS private key path.                                                                                                                                                 |
| `SSL_REDIRECT_HTTP`           | Enables HTTP-to-HTTPS redirect support.                                                                                                                               |
| `SSL_HTTP_PORT`               | HTTP redirect listener port.                                                                                                                                          |
| `UNIX_SOCKET_ENABLED`         | Enables Unix socket listener mode.                                                                                                                                    |
| `UNIX_SOCKET_PATH`            | Unix socket filesystem path.                                                                                                                                          |
| `UNIX_SOCKET_PERMISSION_MODE` | Unix socket permission mode in octal.                                                                                                                                 |
| `CORS_ORIGINS`                | Comma-separated CORS origin patterns.                                                                                                                                 |
| `CORS_METHODS`                | Comma-separated CORS methods. Defaults include `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, and `OPTIONS` so browser clients can use every supported REST mutation route. |
| `CORS_HEADERS`                | Comma-separated allowed headers.                                                                                                                                      |
| `CORS_CREDENTIALS`            | Enables CORS credentials.                                                                                                                                             |

## Metrics and rate limiting [#metrics-and-rate-limiting]

| Variable                           | Purpose                                                |
| ---------------------------------- | ------------------------------------------------------ |
| `METRICS_DRIVER`                   | Metrics backend driver.                                |
| `METRICS_ENABLED`                  | Enables the metrics server.                            |
| `METRICS_HOST`                     | Metrics bind host.                                     |
| `METRICS_PORT`                     | Metrics bind port.                                     |
| `METRICS_PROMETHEUS_PREFIX`        | Prefix for Prometheus metric names.                    |
| `METRICS_TCP_EXPORTER_ENABLED`     | Enables the metrics-rs TCP event exporter.             |
| `METRICS_TCP_EXPORTER_HOST`        | TCP exporter bind host.                                |
| `METRICS_TCP_EXPORTER_PORT`        | TCP exporter bind port.                                |
| `METRICS_TCP_EXPORTER_BUFFER_SIZE` | TCP exporter internal and per-client buffer size.      |
| `RATE_LIMITER_ENABLED`             | Enables HTTP and WebSocket rate limiting.              |
| `RATE_LIMITER_API_MAX_REQUESTS`    | API requests allowed per rate limit window.            |
| `RATE_LIMITER_API_WINDOW_SECONDS`  | API rate limit window length.                          |
| `RATE_LIMITER_API_TRUST_HOPS`      | Trusted proxy hops for API client IP extraction.       |
| `RATE_LIMITER_WS_MAX_REQUESTS`     | WebSocket upgrade attempts allowed per window.         |
| `RATE_LIMITER_WS_WINDOW_SECONDS`   | WebSocket rate limit window length.                    |
| `RATE_LIMITER_WS_TRUST_HOPS`       | Trusted proxy hops for WebSocket client IP extraction. |
| `RATE_LIMITER_REDIS_PREFIX`        | Redis key prefix for rate limiter state.               |

## Queue backends [#queue-backends]

| Variable                                 | Purpose                                                                                                                                                                                       |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `QUEUE_REDIS_CONCURRENCY`                | Redis queue worker concurrency.                                                                                                                                                               |
| `QUEUE_REDIS_PREFIX`                     | Redis queue key prefix.                                                                                                                                                                       |
| `QUEUE_REDIS_RESPONSE_TIMEOUT_MS`        | Redis queue command response timeout in milliseconds. Defaults to `5000` so Sentinel failover invalidates stale primary connections promptly; set to `0` to disable the client-side deadline. |
| `QUEUE_MAX_ATTEMPTS`                     | Maximum Queue v2 deliveries before dead-lettering. Defaults to `5`.                                                                                                                           |
| `QUEUE_RETRY_BASE_DELAY_MS`              | Initial retry delay. Defaults to `1000`.                                                                                                                                                      |
| `QUEUE_RETRY_MAX_DELAY_MS`               | Maximum exponential retry delay. Defaults to `60000`.                                                                                                                                         |
| `QUEUE_RETRY_JITTER`                     | Symmetric retry jitter fraction from `0.0` to `1.0`. Defaults to `0.2`.                                                                                                                       |
| `QUEUE_LEASE_DURATION_MS`                | Redis Queue v2 in-flight lease duration. Defaults to `30000`.                                                                                                                                 |
| `QUEUE_LEASE_RENEW_INTERVAL_MS`          | Redis Queue v2 lease renewal interval; must be below the lease duration. Defaults to `10000`.                                                                                                 |
| `QUEUE_STALLED_BATCH_SIZE`               | Maximum expired/due jobs recovered by one Redis atomic claim. Defaults to `100`.                                                                                                              |
| `QUEUE_WORKER_POLL_INTERVAL_MS`          | Worker recovery poll and delayed-delivery promotion bound. Defaults to `500`.                                                                                                                 |
| `QUEUE_WORKER_PREFETCH`                  | Maximum jobs processed concurrently or prefetched per worker for bounded broker I/O. SQS FIFO stays serial. Defaults to `16`.                                                                 |
| `QUEUE_SHUTDOWN_TIMEOUT_MS`              | Graceful worker drain timeout. Defaults to `30000`.                                                                                                                                           |
| `QUEUE_COMPLETED_RETENTION`              | Completed Redis job metadata retained per queue. `0` removes immediately. Defaults to `1000`.                                                                                                 |
| `QUEUE_FAILED_RETENTION`                 | Dead-letter jobs retained per queue. Defaults to `10000`.                                                                                                                                     |
| `QUEUE_EVENT_RETENTION`                  | Approximate Redis lifecycle event-stream retention. Defaults to `10000`.                                                                                                                      |
| `QUEUE_DEDUPLICATION_TTL_MS`             | Application deduplication-key lifetime. Defaults to `300000`.                                                                                                                                 |
| `QUEUE_MEMORY_CAPACITY`                  | Maximum outstanding jobs per in-memory logical queue. Capacity errors fail closed. Defaults to `100000`.                                                                                      |
| `QUEUE_MAX_BATCH_SIZE`                   | Maximum jobs encoded into one atomic/backend-native enqueue batch. Defaults to `1000`.                                                                                                        |
| `REDIS_CLUSTER_QUEUE_REQUEST_TIMEOUT_MS` | Redis Cluster queue command request timeout in milliseconds. Set to `0` to disable the cluster client deadline.                                                                               |
| `QUEUE_SQS_REGION`                       | SQS region.                                                                                                                                                                                   |
| `QUEUE_SQS_VISIBILITY_TIMEOUT`           | SQS visibility timeout.                                                                                                                                                                       |
| `QUEUE_SQS_MAX_MESSAGES`                 | SQS receive batch size.                                                                                                                                                                       |
| `QUEUE_SQS_WAIT_TIME_SECONDS`            | SQS long-poll wait time.                                                                                                                                                                      |
| `QUEUE_SQS_CONCURRENCY`                  | SQS worker concurrency.                                                                                                                                                                       |
| `QUEUE_SQS_FIFO`                         | Enables FIFO queue behavior.                                                                                                                                                                  |
| `QUEUE_SQS_ENDPOINT_URL`                 | SQS custom endpoint URL.                                                                                                                                                                      |
| `QUEUE_SNS_REGION`                       | SNS region.                                                                                                                                                                                   |
| `QUEUE_SNS_TOPIC_ARN`                    | SNS topic ARN.                                                                                                                                                                                |
| `QUEUE_SNS_ENDPOINT_URL`                 | SNS custom endpoint URL.                                                                                                                                                                      |

## NATS, Pulsar, RabbitMQ, Pub/Sub, Kafka, Iggy, and OMQ [#nats-pulsar-rabbitmq-pubsub-kafka-iggy-and-omq]

| Variable                           | Purpose                                                                                                   |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `NATS_SERVERS`                     | Comma-separated NATS server URLs.                                                                         |
| `NATS_USERNAME`                    | NATS username.                                                                                            |
| `NATS_PASSWORD`                    | NATS password.                                                                                            |
| `NATS_TOKEN`                       | NATS token.                                                                                               |
| `NATS_PREFIX`                      | NATS subject prefix.                                                                                      |
| `NATS_CONNECTION_TIMEOUT_MS`       | NATS connection timeout.                                                                                  |
| `NATS_REQUEST_TIMEOUT_MS`          | NATS request timeout.                                                                                     |
| `NATS_DISCOVERY_MAX_WAIT_MS`       | NATS node discovery maximum wait.                                                                         |
| `NATS_DISCOVERY_IDLE_WAIT_MS`      | NATS node discovery idle wait.                                                                            |
| `NATS_NODES_NUMBER`                | Expected Sockudo node count for horizontal request aggregation when cluster-health discovery is not used. |
| `NATS_SUBSCRIPTION_CAPACITY`       | NATS subscription channel capacity.                                                                       |
| `NATS_CLIENT_CAPACITY`             | NATS client channel capacity.                                                                             |
| `NATS_MAX_RECONNECTS`              | NATS reconnect limit.                                                                                     |
| `NATS_PRESENCE_SYNC_CHUNK_SIZE`    | Presence sync chunk size for NATS.                                                                        |
| `NATS_NO_ECHO`                     | Suppress delivery of a NATS connection's own horizontal messages.                                         |
| `PULSAR_URL`                       | Pulsar service URL.                                                                                       |
| `PULSAR_PREFIX`                    | Pulsar topic prefix.                                                                                      |
| `PULSAR_TOKEN`                     | Pulsar auth token.                                                                                        |
| `PULSAR_REQUEST_TIMEOUT_MS`        | Pulsar request timeout.                                                                                   |
| `PULSAR_NODES_NUMBER`              | Expected Pulsar node count.                                                                               |
| `RABBITMQ_URL`                     | RabbitMQ connection URL.                                                                                  |
| `RABBITMQ_PREFIX`                  | RabbitMQ exchange or routing prefix.                                                                      |
| `RABBITMQ_CONNECTION_TIMEOUT_MS`   | RabbitMQ connection timeout.                                                                              |
| `RABBITMQ_REQUEST_TIMEOUT_MS`      | RabbitMQ request timeout.                                                                                 |
| `RABBITMQ_NODES_NUMBER`            | Expected RabbitMQ node count.                                                                             |
| `GOOGLE_PUBSUB_PROJECT_ID`         | Google Pub/Sub project ID.                                                                                |
| `GOOGLE_PUBSUB_PREFIX`             | Google Pub/Sub topic prefix.                                                                              |
| `PUBSUB_EMULATOR_HOST`             | Pub/Sub emulator host.                                                                                    |
| `GOOGLE_PUBSUB_REQUEST_TIMEOUT_MS` | Google Pub/Sub request timeout.                                                                           |
| `GOOGLE_PUBSUB_NODES_NUMBER`       | Expected Pub/Sub node count.                                                                              |
| `KAFKA_BROKERS`                    | Comma-separated Kafka brokers.                                                                            |
| `KAFKA_PREFIX`                     | Kafka topic prefix.                                                                                       |
| `KAFKA_SECURITY_PROTOCOL`          | Kafka security protocol.                                                                                  |
| `KAFKA_SASL_MECHANISM`             | Kafka SASL mechanism.                                                                                     |
| `KAFKA_SASL_USERNAME`              | Kafka SASL username.                                                                                      |
| `KAFKA_SASL_PASSWORD`              | Kafka SASL password.                                                                                      |
| `KAFKA_REQUEST_TIMEOUT_MS`         | Kafka request timeout.                                                                                    |
| `KAFKA_NODES_NUMBER`               | Expected Kafka node count.                                                                                |
| `IGGY_CONNECTION_STRING`           | Apache Iggy connection string.                                                                            |
| `IGGY_USERNAME`                    | Apache Iggy username.                                                                                     |
| `IGGY_PASSWORD`                    | Apache Iggy password.                                                                                     |
| `IGGY_CONSUMER_NAME`               | Apache Iggy consumer name.                                                                                |
| `IGGY_STREAM`                      | Apache Iggy stream name.                                                                                  |
| `IGGY_TOPIC_PREFIX`                | Apache Iggy adapter topic prefix.                                                                         |
| `IGGY_QUEUE_TOPIC_PREFIX`          | Apache Iggy queue topic prefix.                                                                           |
| `IGGY_CONSUMER_GROUP_PREFIX`       | Apache Iggy consumer group prefix.                                                                        |
| `IGGY_REQUEST_TIMEOUT_MS`          | Apache Iggy request timeout.                                                                              |
| `IGGY_POLL_INTERVAL_MS`            | Apache Iggy poll interval in milliseconds (default: `5`).                                                 |
| `IGGY_POLL_BATCH_SIZE`             | Apache Iggy poll batch size.                                                                              |
| `IGGY_PARTITIONS_COUNT`            | Shared Apache Iggy partition count.                                                                       |
| `ADAPTER_IGGY_PARTITIONS_COUNT`    | Adapter-specific Apache Iggy partition count.                                                             |
| `OMQ_BIND_ENDPOINT`                | OMQ subscriber bind endpoint for this node.                                                               |
| `OMQ_CONNECT_ENDPOINTS`            | Comma-separated OMQ peer subscriber endpoints.                                                            |
| `OMQ_PREFIX`                       | OMQ transport topic prefix.                                                                               |
| `OMQ_REQUEST_TIMEOUT_MS`           | OMQ request timeout.                                                                                      |
| `OMQ_NODES_NUMBER`                 | Expected OMQ node count. Defaults to peer endpoint count plus this node.                                  |
| `OMQ_IO_THREADS`                   | OMQ data-plane IO threads.                                                                                |
| `OMQ_SEND_HWM`                     | OMQ publisher high-water mark in messages.                                                                |
| `OMQ_RECV_HWM`                     | OMQ subscriber high-water mark in messages.                                                               |
| `QUEUE_IGGY_PARTITIONS_COUNT`      | Queue-specific Apache Iggy partition count.                                                               |
| `IGGY_PARTITION_ID`                | Shared Apache Iggy partition ID.                                                                          |
| `ADAPTER_IGGY_PARTITION_ID`        | Adapter-specific Apache Iggy partition ID.                                                                |
| `QUEUE_IGGY_PARTITION_ID`          | Queue-specific Apache Iggy partition ID.                                                                  |
| `IGGY_AUTO_CREATE`                 | Auto-create Apache Iggy streams and topics.                                                               |
| `IGGY_START_FROM_LATEST`           | Start adapter consumption from latest messages.                                                           |
| `IGGY_NODES_NUMBER`                | Expected Apache Iggy node count.                                                                          |

## Cleanup and cluster health [#cleanup-and-cluster-health]

| Variable                            | Purpose                                                     |
| ----------------------------------- | ----------------------------------------------------------- |
| `CLEANUP_ASYNC_ENABLED`             | Enables asynchronous cleanup.                               |
| `CLEANUP_FALLBACK_TO_SYNC`          | Falls back to synchronous cleanup when async cleanup fails. |
| `CLEANUP_QUEUE_BUFFER_SIZE`         | Cleanup queue buffer size.                                  |
| `CLEANUP_BATCH_SIZE`                | Cleanup batch size.                                         |
| `CLEANUP_BATCH_TIMEOUT_MS`          | Cleanup batch timeout.                                      |
| `CLEANUP_WORKER_THREADS`            | Cleanup worker count or `auto`.                             |
| `CLEANUP_MAX_RETRY_ATTEMPTS`        | Cleanup retry limit.                                        |
| `CLUSTER_HEALTH_ENABLED`            | Enables cluster health tracking.                            |
| `CLUSTER_HEALTH_HEARTBEAT_INTERVAL` | Cluster heartbeat interval.                                 |
| `CLUSTER_HEALTH_NODE_TIMEOUT`       | Node timeout before a peer is considered stale.             |
| `CLUSTER_HEALTH_CLEANUP_INTERVAL`   | Cleanup interval for cluster health state.                  |

## WebSocket and Protocol V2 features [#websocket-and-protocol-v2-features]

| Variable                                     | Purpose                                                                                                                                                                                                |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `TAG_FILTERING_ENABLED`                      | Enables Protocol V2 tag filtering.                                                                                                                                                                     |
| `WEBSOCKET_MAX_MESSAGES`                     | Maximum buffered outbound WebSocket messages, or `none`/`0` for unlimited.                                                                                                                             |
| `WEBSOCKET_MAX_BYTES`                        | Maximum buffered outbound WebSocket bytes, or `none`/`0` for unlimited.                                                                                                                                |
| `WEBSOCKET_DISCONNECT_ON_BUFFER_FULL`        | Disconnects slow clients instead of dropping messages when buffers fill.                                                                                                                               |
| `WEBSOCKET_MAX_MESSAGE_SIZE`                 | WebSocket message size limit.                                                                                                                                                                          |
| `WEBSOCKET_MAX_FRAME_SIZE`                   | WebSocket frame size limit.                                                                                                                                                                            |
| `WEBSOCKET_WRITE_BUFFER_SIZE`                | WebSocket write buffer size.                                                                                                                                                                           |
| `WEBSOCKET_MAX_BACKPRESSURE`                 | WebSocket backpressure limit.                                                                                                                                                                          |
| `WEBSOCKET_AUTO_PING`                        | Enables native WebSocket Ping/Pong keepalive for Protocol V2. Protocol V1 retains its Pusher-compatible application heartbeat.                                                                         |
| `WEBSOCKET_PING_INTERVAL`                    | Inbound inactivity in seconds before a Protocol V2 native Ping. `0` disables proactive native Ping frames; nonzero values are bounded to at least half of `ACTIVITY_TIMEOUT` (and at least 5 seconds). |
| `WEBSOCKET_IDLE_TIMEOUT`                     | Hard inbound-idle timeout in seconds for Protocol V2 native connections. `0` disables the hard idle deadline.                                                                                          |
| `WEBSOCKET_COMPRESSION`                      | WebSocket compression mode.                                                                                                                                                                            |
| `CONNECTION_RECOVERY_ENABLED`                | Enables Protocol V2 connection recovery.                                                                                                                                                               |
| `CONNECTION_RECOVERY_BUFFER_TTL`             | Recovery buffer TTL.                                                                                                                                                                                   |
| `CONNECTION_RECOVERY_MAX_BUFFER_SIZE`        | Recovery buffer maximum size.                                                                                                                                                                          |
| `EPHEMERAL_ENABLED`                          | Enables ephemeral event handling.                                                                                                                                                                      |
| `ECHO_CONTROL_ENABLED`                       | Enables per-connection echo control.                                                                                                                                                                   |
| `ECHO_CONTROL_DEFAULT_ECHO_MESSAGES`         | Default echo behavior when echo control is enabled.                                                                                                                                                    |
| `EVENT_NAME_FILTERING_ENABLED`               | Enables event-name filtering for subscriptions.                                                                                                                                                        |
| `EVENT_NAME_FILTERING_MAX_EVENTS_PER_FILTER` | Maximum event names in one subscription filter.                                                                                                                                                        |
| `EVENT_NAME_FILTERING_MAX_EVENT_NAME_LENGTH` | Maximum event name length inside a filter.                                                                                                                                                             |

## Presence [#presence]

| Variable                                           | Purpose                                                                                      |
| -------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `PRESENCE_UPDATE_RATE_LIMIT_PER_MEMBER_PER_SECOND` | Maximum presence updates accepted per member per second.                                     |
| `PRESENCE_UNGRACEFUL_TIMEOUT_SECONDS`              | Protocol V1/Pusher presence lease after an abrupt disconnect. Defaults to immediate removal. |
| `PRESENCE_V2_UNGRACEFUL_TIMEOUT_SECONDS`           | Protocol V2 presence lease after an abrupt disconnect. Defaults to 15 seconds.               |

## History, idempotency, versioning, and annotations [#history-idempotency-versioning-and-annotations]

| Variable                                      | Purpose                                             |
| --------------------------------------------- | --------------------------------------------------- |
| `HISTORY_ENABLED`                             | Enables durable channel history.                    |
| `HISTORY_REWIND_ENABLED`                      | Enables rewind reads from history.                  |
| `HISTORY_RETENTION_WINDOW_SECONDS`            | Durable history retention window.                   |
| `HISTORY_MAX_PAGE_SIZE`                       | Maximum history page size.                          |
| `HISTORY_WRITER_SHARDS`                       | Durable history writer shard count.                 |
| `HISTORY_WRITER_QUEUE_CAPACITY`               | Durable history writer queue capacity.              |
| `HISTORY_BACKEND`                             | Durable history backend.                            |
| `HISTORY_MAX_MESSAGES_PER_CHANNEL`            | Optional retained message count limit per channel.  |
| `HISTORY_MAX_BYTES_PER_CHANNEL`               | Optional retained byte limit per channel.           |
| `HISTORY_POSTGRES_TABLE_PREFIX`               | PostgreSQL history table prefix.                    |
| `HISTORY_POSTGRES_WRITE_TIMEOUT_MS`           | PostgreSQL history write timeout.                   |
| `HISTORY_PURGE_INTERVAL_SECONDS`              | History purge interval.                             |
| `HISTORY_PURGE_BATCH_SIZE`                    | History purge batch size.                           |
| `HISTORY_MAX_PURGE_PER_TICK`                  | Maximum purged history rows per purge tick.         |
| `PRESENCE_HISTORY_ENABLED`                    | Enables retained presence history.                  |
| `PRESENCE_HISTORY_RETENTION_WINDOW_SECONDS`   | Presence history retention window.                  |
| `PRESENCE_HISTORY_MAX_PAGE_SIZE`              | Maximum presence history page size.                 |
| `PRESENCE_HISTORY_MAX_EVENTS_PER_CHANNEL`     | Optional retained presence event limit per channel. |
| `PRESENCE_HISTORY_MAX_BYTES_PER_CHANNEL`      | Optional retained presence byte limit per channel.  |
| `IDEMPOTENCY_ENABLED`                         | Enables HTTP publish idempotency.                   |
| `IDEMPOTENCY_TTL_SECONDS`                     | Idempotency cache TTL.                              |
| `IDEMPOTENCY_MAX_KEY_LENGTH`                  | Maximum idempotency key length.                     |
| `VERSIONED_MESSAGES_ENABLED`                  | Enables Protocol V2 mutable messages.               |
| `VERSIONED_MESSAGES_DRIVER`                   | Versioned message backend driver.                   |
| `VERSIONED_MESSAGES_MAX_PAGE_SIZE`            | Maximum versioned-message page size.                |
| `VERSIONED_MESSAGES_RETENTION_WINDOW_SECONDS` | Versioned-message retention window.                 |
| `VERSIONED_MESSAGES_PURGE_INTERVAL_SECONDS`   | Versioned-message purge interval.                   |
| `VERSIONED_MESSAGES_PURGE_BATCH_SIZE`         | Versioned-message purge batch size.                 |
| `VERSIONED_MESSAGES_MAX_PURGE_PER_TICK`       | Maximum versioned-message rows purged per tick.     |
| `ANNOTATIONS_ENABLED`                         | Enables message annotations globally.               |

## MCP [#mcp]

Embedded Model Context Protocol server (`[mcp]`, requires the `mcp` Cargo feature). See
[MCP server](/docs/server/mcp).

| Variable                    | Maps to                     | Notes                                                         |
| --------------------------- | --------------------------- | ------------------------------------------------------------- |
| `MCP_ENABLED`               | `mcp.enabled`               | Boolean.                                                      |
| `MCP_PATH`                  | `mcp.path`                  | URL path, default `/mcp`.                                     |
| `MCP_PORT`                  | `mcp.port`                  | Dedicated listener port; unset shares the main HTTP listener. |
| `MCP_HOST`                  | `mcp.host`                  | Bind host for the dedicated listener.                         |
| `MCP_ALLOWED_HOSTS`         | `mcp.allowed_hosts`         | Comma-separated `Host` authorities.                           |
| `MCP_ALLOWED_ORIGINS`       | `mcp.allowed_origins`       | Comma-separated browser origins.                              |
| `MCP_ALLOW_ANONYMOUS`       | `mcp.allow_anonymous`       | Boolean; development only.                                    |
| `MCP_ANONYMOUS_SCOPES`      | `mcp.anonymous_scopes`      | Comma-separated scopes.                                       |
| `MCP_REQUEST_TIMEOUT_MS`    | `mcp.request_timeout_ms`    | Per-tool upstream timeout.                                    |
| `MCP_MAX_BODY_BYTES`        | `mcp.max_body_bytes`        | Maximum MCP POST body.                                        |
| `MCP_SESSION_TTL_SECONDS`   | `mcp.session_ttl_seconds`   | Idle session lifetime.                                        |
| `MCP_RATE_LIMIT_PER_MINUTE` | `mcp.rate_limit_per_minute` | Per-token budget; `0` disables.                               |
| `MCP_DISABLED_TOOLS`        | `mcp.disabled_tools`        | Comma-separated tool names to hide.                           |
| `MCP_INSTRUCTIONS`          | `mcp.instructions`          | Extra MCP instructions text.                                  |
| `MCP_TOKEN`                 | `[[mcp.tokens]]` entry      | Adds or replaces one token principal from the environment.    |
| `MCP_TOKEN_NAME`            | `mcp.tokens[].name`         | Name for `MCP_TOKEN`, default `env`.                          |
| `MCP_TOKEN_SCOPES`          | `mcp.tokens[].scopes`       | Comma-separated, default `read,write`.                        |
| `MCP_TOKEN_APPS`            | `mcp.tokens[].apps`         | Comma-separated app ids, default `*`.                         |

The standalone `sockudo-mcp` binary reads `SOCKUDO_URL`, `SOCKUDO_MCP_APPS`, `SOCKUDO_APP_ID`,
`SOCKUDO_APP_KEY`, `SOCKUDO_APP_SECRET`, `SOCKUDO_MCP_TRANSPORT`, `SOCKUDO_MCP_LISTEN`,
`SOCKUDO_MCP_PATH`, `SOCKUDO_MCP_TOKENS`, `SOCKUDO_MCP_SCOPES`, `SOCKUDO_MCP_ALLOW_ANONYMOUS`,
`SOCKUDO_METRICS_URL`, `SOCKUDO_MCP_TIMEOUT_MS`, `SOCKUDO_MCP_ALLOWED_HOSTS`,
`SOCKUDO_MCP_ALLOWED_ORIGINS`, `SOCKUDO_MCP_DISABLED_TOOLS`, and `SOCKUDO_MCP_INSTRUCTIONS`.

## Webhooks [#webhooks]

| Variable                    | Purpose                    |
| --------------------------- | -------------------------- |
| `WEBHOOK_BATCHING_ENABLED`  | Enables webhook batching.  |
| `WEBHOOK_BATCHING_DURATION` | Webhook batching duration. |
| `WEBHOOK_BATCHING_SIZE`     | Webhook batching size.     |

## Push core [#push-core]

| Variable                                | Purpose                                                                                                                      |
| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `PUSH_STORAGE_DRIVER`                   | Push storage backend driver.                                                                                                 |
| `PUSH_QUEUE_DRIVER`                     | Push queue backend driver.                                                                                                   |
| `PUSH_ALLOW_MEMORY_DRIVERS`             | Explicitly allows node-local memory push store/queue in production; use only for local/dev acknowledgment.                   |
| `PUSH_CREDENTIAL_ENCRYPTION_KEY`        | Master key for encrypted push credential material.                                                                           |
| `PUSH_ACCEPTANCE_RATE_LIMIT`            | Admission rate limit for push publish requests.                                                                              |
| `PUSH_ANALYTICS_ENABLED`                | Enables push analytics collection.                                                                                           |
| `PUSH_FANOUT_FAST_THRESHOLD`            | Recipient threshold for fast-path push fanout.                                                                               |
| `PUSH_FANOUT_SHARD_SIZE`                | Recipient shard size for large fanout.                                                                                       |
| `PUSH_FANOUT_SYNC_THRESHOLD`            | Threshold below which fanout can remain synchronous.                                                                         |
| `PUSH_BACKPRESSURE_LAG_THRESHOLD_SECS`  | Oldest actionable push queue age, in seconds, tolerated before push backpressure; `0` disables age-based queue backpressure. |
| `PUSH_BACKPRESSURE`                     | Forces push backpressure responses in the HTTP layer.                                                                        |
| `PUSH_BACKPRESSURE_RETRY_AFTER_SECONDS` | Retry-After value emitted for push backpressure.                                                                             |
| `PUSH_PUBLISH_LOG_MAX_LAG`              | Maximum publish-log queue depth tolerated before push backpressure.                                                          |
| `PUSH_CRITICAL_QUEUE_MAX_LAG`           | Maximum queue depth tolerated for shard, delivery result, retry, dead-letter, and active provider delivery stages.           |
| `PUSH_PUBLISH_STATUS_TTL_DAYS`          | Retention for push publish status records.                                                                                   |
| `PUSH_DISPATCH_MAX_OUTBOUND_REQUESTS`   | Maximum outbound provider requests each dispatch worker hands to a provider dispatcher at once.                              |
| `PUSH_RETRY_WORKER_COUNT`               | Number of monolith retry scheduler workers consuming `push.retry.v1`.                                                        |
| `PUSH_RETRY_MAX_ATTEMPTS`               | Maximum provider delivery attempts before retry work is dead-lettered.                                                       |
| `PUSH_RETRY_INITIAL_BACKOFF_MS`         | Initial retry backoff when the provider does not send `Retry-After`.                                                         |
| `PUSH_RETRY_MAX_BACKOFF_MS`             | Maximum exponential retry backoff.                                                                                           |
| `PUSH_RETRY_MAX_ELAPSED_SECS`           | Maximum retry age from first provider attempt.                                                                               |
| `PUSH_RETRY_JITTER`                     | Enables deterministic retry jitter.                                                                                          |
| `PUSH_RETRY_JITTER_RATIO_PERCENT`       | Retry jitter spread as a percentage of the selected backoff.                                                                 |
| `PUSH_RETRY_RESPECT_RETRY_AFTER`        | Uses provider `Retry-After` deadlines when present.                                                                          |
| `PUSH_REPAIR_INTERVAL_SECS`             | Interval for the monolith push repair worker; `0` disables durable publish-log repair.                                       |
| `PUSH_REPAIR_MIN_AGE_SECS`              | Minimum queued publish-log age before repair recreates missing `push.publish.v1` work.                                       |
| `PUSH_REPAIR_BATCH_SIZE`                | Maximum durable publish-log rows scanned per app and repair tick.                                                            |
| `PUSH_FAILURE_THRESHOLD`                | Push circuit-breaker failure threshold.                                                                                      |
| `PUSH_SCHEDULER_INTERVAL_SECS`          | Scheduled push scan interval.                                                                                                |
| `PUSH_CLEANUP_INTERVAL_SECS`            | Push cleanup worker interval. Set `0` to disable the worker.                                                                 |
| `PUSH_CLEANUP_BATCH_SIZE`               | Maximum rows considered per cleanup category per tick.                                                                       |
| `PUSH_CLEANUP_MAX_DELETED_PER_TICK`     | Overall cap on rows deleted by one push cleanup tick.                                                                        |
| `PUSH_STALE_DEVICE_MAX_AGE_DAYS`        | Stale device age threshold.                                                                                                  |
| `PUSH_DEFAULT_ACCEPTANCE_RPS`           | Default per-app push acceptance RPS quota.                                                                                   |
| `PUSH_DEFAULT_DELIVERY_QUOTA_DAILY`     | Default per-app daily delivery quota.                                                                                        |
| `PUSH_DEFAULT_FANOUT_MAX`               | Default per-app maximum fanout.                                                                                              |
| `PUSH_DEFAULT_INFLIGHT_MAX`             | Default per-app in-flight publish limit.                                                                                     |
| `PUSH_FANOUT_MAX`                       | HTTP-layer fanout cap override.                                                                                              |
| `PUSH_DELIVERY_QUOTA_DAILY`             | HTTP-layer daily delivery quota override.                                                                                    |

## Push providers [#push-providers]

| Variable                              | Purpose                                                            |
| ------------------------------------- | ------------------------------------------------------------------ |
| `PUSH_FCM_ENABLED`                    | Enables FCM dispatch.                                              |
| `FCM_PROJECT_ID`                      | FCM project ID for monolith dispatch workers.                      |
| `PUSH_FCM_PROJECT_ID`                 | FCM project ID alias.                                              |
| `FCM_SERVICE_ACCOUNT_JSON_PATH`       | Path to FCM service account JSON for auto-refreshing OAuth tokens. |
| `PUSH_FCM_SERVICE_ACCOUNT_JSON_PATH`  | FCM service account JSON path alias.                               |
| `FCM_SERVICE_ACCOUNT_JSON`            | Inline FCM service account JSON for auto-refreshing OAuth tokens.  |
| `PUSH_FCM_SERVICE_ACCOUNT_JSON`       | Inline FCM service account JSON alias.                             |
| `FCM_APP_ID`                          | App ID used to load stored FCM credentials.                        |
| `PUSH_FCM_APP_ID`                     | Stored FCM credential app ID alias.                                |
| `FCM_CREDENTIAL_ID`                   | Stored FCM credential ID.                                          |
| `PUSH_FCM_CREDENTIAL_ID`              | Stored FCM credential ID alias.                                    |
| `FCM_PROVIDER_TOKEN`                  | Legacy static FCM OAuth token for short-lived dispatch.            |
| `PUSH_FCM_PROVIDER_TOKEN`             | Legacy static FCM token alias.                                     |
| `FCM_ENDPOINT`                        | FCM endpoint override.                                             |
| `PUSH_FCM_ENDPOINT`                   | FCM endpoint alias.                                                |
| `PUSH_APNS_ENABLED`                   | Enables APNs dispatch.                                             |
| `APNS_TOPIC`                          | APNs topic or bundle ID.                                           |
| `PUSH_APNS_TOPIC`                     | APNs topic alias.                                                  |
| `APNS_ENDPOINT`                       | APNs endpoint override.                                            |
| `PUSH_APNS_ENDPOINT`                  | APNs endpoint alias.                                               |
| `APNS_APP_ID`                         | App ID used to load stored APNs credentials.                       |
| `PUSH_APNS_APP_ID`                    | Stored APNs app ID alias.                                          |
| `APNS_CREDENTIAL_ID`                  | Stored APNs credential ID.                                         |
| `PUSH_APNS_CREDENTIAL_ID`             | Stored APNs credential ID alias.                                   |
| `APNS_PROVIDER_TOKEN`                 | Static APNs provider token.                                        |
| `PUSH_APNS_PROVIDER_TOKEN`            | Static APNs provider token alias.                                  |
| `APNS_TEAM_ID`                        | APNs token-auth team ID.                                           |
| `PUSH_APNS_TEAM_ID`                   | APNs team ID alias.                                                |
| `APNS_KEY_ID`                         | APNs token-auth key ID.                                            |
| `PUSH_APNS_KEY_ID`                    | APNs key ID alias.                                                 |
| `APNS_PRIVATE_KEY`                    | APNs token-auth private key content.                               |
| `PUSH_APNS_PRIVATE_KEY`               | APNs private key content alias.                                    |
| `APNS_PRIVATE_KEY_PATH`               | Filesystem path to APNs token-auth private key.                    |
| `PUSH_APNS_PRIVATE_KEY_PATH`          | APNs private key path alias.                                       |
| `PUSH_WEBPUSH_ENABLED`                | Enables Web Push dispatch.                                         |
| `VAPID_PRIVATE_KEY`                   | Web Push VAPID private key.                                        |
| `PUSH_WEBPUSH_VAPID_PRIVATE_KEY`      | Web Push VAPID private key alias.                                  |
| `VAPID_CONTACT`                       | Web Push VAPID contact subject.                                    |
| `PUSH_WEBPUSH_VAPID_CONTACT`          | Web Push VAPID contact alias.                                      |
| `PUSH_ALLOW_LOCAL_WEB_PUSH_ENDPOINTS` | Allows local Web Push endpoints for development and tests.         |
| `PUSH_WEBPUSH_ALLOWED_HOSTS`          | Comma-separated Web Push endpoint allowlist.                       |
| `PUSH_WEBPUSH_DENIED_HOSTS`           | Comma-separated Web Push endpoint denylist.                        |
| `PUSH_HMS_ENABLED`                    | Enables HMS dispatch.                                              |
| `HMS_APP_ID`                          | HMS app ID for dispatch workers.                                   |
| `PUSH_HMS_APP_ID`                     | HMS app ID alias.                                                  |
| `HMS_PROVIDER_TOKEN`                  | HMS provider token.                                                |
| `PUSH_HMS_PROVIDER_TOKEN`             | HMS provider token alias.                                          |
| `HMS_ENDPOINT`                        | HMS endpoint override.                                             |
| `PUSH_HMS_ENDPOINT`                   | HMS endpoint alias.                                                |
| `PUSH_WNS_ENABLED`                    | Enables WNS dispatch.                                              |
| `WNS_PROVIDER_TOKEN`                  | WNS provider token.                                                |
| `PUSH_WNS_PROVIDER_TOKEN`             | WNS provider token alias.                                          |


# Error codes (/docs/reference/errors)



Sockudo uses HTTP status codes for REST failures and protocol events for WebSocket failures.

## HTTP status [#http-status]

| Status | Meaning                                                    | Action                                                                             |
| ------ | ---------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| `202`  | Async work accepted.                                       | Store returned IDs and inspect status later.                                       |
| `400`  | Invalid request.                                           | Validate request body and feature flags.                                           |
| `401`  | Authentication failed.                                     | Check app key, secret, timestamp, and signature.                                   |
| `403`  | Forbidden or disabled.                                     | Check app policy and feature enablement.                                           |
| `404`  | Not found.                                                 | Check app ID, channel, message serial, device ID, or publish ID.                   |
| `409`  | Conflict.                                                  | Check idempotency, version, or resource lifecycle.                                 |
| `413`  | Payload too large.                                         | Reduce event or push payload size.                                                 |
| `429`  | Rate limited.                                              | Back off and inspect quota policy.                                                 |
| `503`  | Backpressure or required coordination backend unavailable. | Honor `Retry-After` when present and retry idempotent publishes with the same key. |
| `5xx`  | Server or dependency failure.                              | Inspect logs, metrics, adapter, cache, queue, or provider status.                  |

HTTP `5xx` bodies expose a stable error code and a generic public message. Backend
driver diagnostics, connection strings, credentials, and internal failure details
are never returned to the caller; use credential-safe server metrics and logs for
operator diagnosis.

## Auth failures [#auth-failures]

Most auth failures are caused by:

* wrong app key or secret
* stale timestamp
* mismatched `socket_id`
* signing a different `channel_name` than the client requested
* mutating presence `channel_data` after signing
* missing raw body for webhook validation
* expired or revoked Protocol V2 capability tokens
* capability-token `kid`, `alg`, `jti`, `exp`, or channel-operation map violations

Protocol V2 capability-token expiry emits `sockudo:token_expired` with code `40142`; clients should call their backend for a fresh token and send `sockudo:auth` during the 30 second grace window. Revocation emits the same event with code `40160`; clients should obtain a new authenticated session before reconnecting.

## Recovery failures [#recovery-failures]

Recovery can fail when:

* the client did not use Protocol V2
* the replay buffer expired
* the stream reset
* the reconnect landed after retention was purged
* adapter or cache state was unavailable

Clients should rebuild state after recovery failure.

For `resume_failed` with `code: "position_expired"`, V2 clients should resubscribe and request WebSocket `channel_history` with `until_attach: true`. That backfills retained durable history up to the new subscription attach serial while live delivery continues above it.

## Push errors [#push-errors]

| Error class            | Typical cause                                        |
| ---------------------- | ---------------------------------------------------- |
| Device not found       | Registration missing or deleted.                     |
| Provider token invalid | APNs, FCM, Web Push, HMS, or WNS rejected the token. |
| Credential missing     | Provider credentials are not configured.             |
| Publish not found      | Status retention expired or ID is wrong.             |
| Scheduled push invalid | Missing `notBeforeMs` or scheduled time is invalid.  |
| Provider throttled     | Provider rate limits or campaign burst.              |
| Payload too large      | Platform-specific push payload limit exceeded.       |

Push publish admission can succeed while provider delivery fails later. Always inspect publish status for notification incidents.


# HTTP endpoints (/docs/reference/http-endpoints)



Sockudo exposes three endpoint groups:

* the main server port, usually `6001`, for WebSocket upgrades and the signed app HTTP API
* optional operational endpoints on the main server port
* the metrics server port, usually `9601`, for Prometheus scraping

Every route under `/apps/{appId}` requires Pusher-compatible signed query authentication. Push routes also evaluate the push capability headers described below.

## Availability by server role [#availability-by-server-role]

With `server_role = "api"` (see [Configuration](/docs/reference/configuration#server-role)),
endpoints that depend on live socket state return 404: channels, channel users,
terminate/force-reconnect, and the WebSocket upgrade. All publish, history, mutable-message,
annotation, revocation, and push endpoints remain available.

Additional restrictions in API mode:

* `info` values `user_count` and `subscription_count` are rejected with HTTP 400.
* `socket_id` on mutation and annotation requests is rejected with HTTP 400.
* When built with `ably-compat`, Ably live-state routes (presence, channel detail)
  return 404; the REST publish surface is available.

## WebSocket [#websocket]

| Method | Path            | Auth                                                  | Purpose                                                    |
| ------ | --------------- | ----------------------------------------------------- | ---------------------------------------------------------- |
| `GET`  | `/app/{appKey}` | App key and channel-level protocol auth after connect | WebSocket upgrade for Protocol V1 and Protocol V2 clients. |

Supported query parameters:

| Parameter       | Scope       | Meaning                                                                                                          |
| --------------- | ----------- | ---------------------------------------------------------------------------------------------------------------- |
| `protocol`      | WebSocket   | `1` or omitted uses Pusher-compatible Protocol V1. `2` enables Sockudo Protocol V2.                              |
| `client`        | WebSocket   | Client library identifier. Accepted for compatibility and observability.                                         |
| `version`       | WebSocket   | Client library version. Accepted for compatibility and observability.                                            |
| `format`        | Protocol V2 | Wire format selector. Unknown V2 formats are rejected with `400 Bad Request`.                                    |
| `echo_messages` | Protocol V2 | Overrides per-connection echo behavior when echo control is enabled.                                             |
| `token`         | Protocol V2 | Capability-token JWT alternative to unauthenticated app-key connect. The JWT header `kid` must match `{appKey}`. |

## MCP [#mcp]

| Method                    | Path                                               | Auth                                | Purpose                                                                                                                                                                                                                           |
| ------------------------- | -------------------------------------------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST` / `GET` / `DELETE` | `/mcp` (configurable; optionally a dedicated port) | `Authorization: Bearer <mcp token>` | Model Context Protocol Streamable HTTP endpoint. Requires the `mcp` Cargo feature and `[mcp].enabled`. Tools are scope-gated (`read`, `write`, `admin`) and call the routes below in-process. See [MCP server](/docs/server/mcp). |

## Publish [#publish]

| Method | Path                                                 | Auth           | Purpose                                                                                                                                |
| ------ | ---------------------------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `POST` | `/apps/{appId}/events`                               | Signed app API | Publish one event to one or more channels.                                                                                             |
| `POST` | `/apps/{appId}/batch_events`                         | Signed app API | Publish multiple events in one request.                                                                                                |
| `POST` | `/apps/{appId}/revocations`                          | Signed app API | Revoke Protocol V2 capability tokens by `jti`, `client_id`, or both. Matching local sockets receive `sockudo:token_expired` and close. |
| `POST` | `/apps/{appId}/users/{userId}/terminate_connections` | Signed app API | Disconnect every active socket for the supplied user ID.                                                                               |
| `POST` | `/apps/{appId}/users/{userId}/force_reconnect`       | Signed app API | Close every active socket for the supplied user ID with code `4200`, prompting clients to reconnect.                                   |

Useful request controls:

| Field or header     | Endpoints         | Meaning                                                                                                                                          |
| ------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `socket_id`         | publish endpoints | Suppresses echo to the originating socket.                                                                                                       |
| `info`              | publish endpoints | Optional response info such as `subscription_count`.                                                                                             |
| `x-idempotency-key` | publish endpoints | Deduplicates accepted publishes when idempotency is enabled. Coordination failure returns retryable `503` instead of publishing without a claim. |
| `idempotency_key`   | publish bodies    | Body-level idempotency key accepted by Sockudo publish paths; retries must reuse the same key.                                                   |
| `message_id`        | publish bodies    | SDK-facing idempotent create key for AI/versioned-message publishes. Duplicate values return the original serial acknowledgement.                |

Revocation request body:

```json
{
  "jti": "token-id",
  "client_id": "user-42",
  "expires_at": 1710003600,
  "ttl_seconds": 3600,
  "reason": "session revoked"
}
```

At least one of `jti` or `client_id` is required. `ttl_seconds` overrides `expires_at`; otherwise revocations default to 24 hours. Use the token's remaining lifetime for the TTL when it is known.

## Channel state [#channel-state]

| Method | Path                                         | Auth           | Purpose                                     |
| ------ | -------------------------------------------- | -------------- | ------------------------------------------- |
| `GET`  | `/apps/{appId}/channels`                     | Signed app API | List occupied channels.                     |
| `GET`  | `/apps/{appId}/channels/{channelName}`       | Signed app API | Read state for one channel.                 |
| `GET`  | `/apps/{appId}/channels/{channelName}/users` | Signed app API | List current members of a presence channel. |

Supported `info` tokens:

| Token                | Endpoints                                                 | Meaning                                  |
| -------------------- | --------------------------------------------------------- | ---------------------------------------- |
| `subscription_count` | `/channels`, `/channels/{channelName}`, publish responses | Include active subscription counts.      |
| `user_count`         | presence channel state                                    | Include presence user count.             |
| `cache`              | cache channel state                                       | Include cache-channel occupancy details. |

For channels matched by `[ai_transport]`, `/channels/{channelName}` includes an
`ai` object with `active_streams`, `last_history_serial`, and `message_count`.

## Durable history [#durable-history]

| Method | Path                                                 | Auth           | Purpose                                                       |
| ------ | ---------------------------------------------------- | -------------- | ------------------------------------------------------------- |
| `GET`  | `/apps/{appId}/channels/{channelName}/history`       | Signed app API | Read durable event history for one channel.                   |
| `GET`  | `/apps/{appId}/channels/{channelName}/history/state` | Signed app API | Inspect stream continuity and degradation state.              |
| `POST` | `/apps/{appId}/channels/{channelName}/history/reset` | Signed app API | Rotate and reset a damaged or intentionally abandoned stream. |
| `POST` | `/apps/{appId}/channels/{channelName}/history/purge` | Signed app API | Purge retained history without rotating continuity state.     |

History query parameters:

| Parameter                      | Meaning                                     |
| ------------------------------ | ------------------------------------------- |
| `limit`                        | Page size, bounded by server configuration. |
| `cursor`                       | Opaque cursor returned by a previous page.  |
| `direction`                    | `newest_first` or `oldest_first`.           |
| `start_serial`, `end_serial`   | Serial range filter.                        |
| `start_time_ms`, `end_time_ms` | Millisecond timestamp range filter.         |

## Presence history [#presence-history]

| Method | Path                                                             | Auth           | Purpose                                                           |
| ------ | ---------------------------------------------------------------- | -------------- | ----------------------------------------------------------------- |
| `GET`  | `/apps/{appId}/channels/{channelName}/presence/history`          | Signed app API | Read retained presence join/leave history.                        |
| `GET`  | `/apps/{appId}/channels/{channelName}/presence/history/state`    | Signed app API | Inspect presence-history continuity and degradation state.        |
| `POST` | `/apps/{appId}/channels/{channelName}/presence/history/reset`    | Signed app API | Rotate and reset a presence-history stream.                       |
| `GET`  | `/apps/{appId}/channels/{channelName}/presence/history/snapshot` | Signed app API | Reconstruct a presence membership snapshot from retained history. |

Presence history uses the same pagination and range controls as durable channel history. The `snapshot` endpoint is for operational reconstruction and should not be confused with `/users`, which returns current live members.

## Versioned messages [#versioned-messages]

| Method | Path                                                                     | Auth           | Purpose                                                  |
| ------ | ------------------------------------------------------------------------ | -------------- | -------------------------------------------------------- |
| `GET`  | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}`          | Signed app API | Read the latest visible version of a mutable V2 message. |
| `GET`  | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/versions` | Signed app API | List preserved versions of a mutable V2 message.         |
| `POST` | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/update`   | Signed app API | Apply a shallow mixin update and store a new version.    |
| `POST` | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/delete`   | Signed app API | Apply a soft delete version.                             |
| `POST` | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/append`   | Signed app API | Append string content and store the rolled-up result.    |

Versioned-message endpoints require `versioned_messages.enabled = true` and a channel policy that permits mutable messages. They are Sockudo Protocol V2 surfaces. `update`, `delete`, and `append` accept optional `op_id` for mutation replay dedupe. Mutation responses include `channel`, `message_serial`, `action`, `accepted`, `version_serial`, `history_serial`, `delivery_serial`, and `status`.

## Annotations [#annotations]

| Method   | Path                                                                                           | Auth           | Purpose                                   |
| -------- | ---------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------- |
| `GET`    | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/annotations`                    | Signed app API | List annotation create/delete events.     |
| `POST`   | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/annotations`                    | Signed app API | Publish an annotation event.              |
| `DELETE` | `/apps/{appId}/channels/{channelName}/messages/{messageSerial}/annotations/{annotationSerial}` | Signed app API | Publish a delete event for an annotation. |

Annotation request fields:

| Field                    | Endpoint            | Meaning                                                       |
| ------------------------ | ------------------- | ------------------------------------------------------------- |
| `type`                   | create, list filter | Annotation type such as `reactions:distinct.v1`.              |
| `name`                   | create              | Optional logical annotation name.                             |
| `clientId` / `client_id` | create              | User or client identity associated with the annotation.       |
| `socketId` / `socket_id` | create/delete/list  | Optional socket used for capability checks and audit linkage. |
| `count`                  | create              | Numeric annotation count.                                     |
| `data`                   | create              | Arbitrary JSON payload.                                       |
| `encoding`               | create              | Payload encoding marker.                                      |
| `from_serial`            | list                | Return annotations after a known annotation serial.           |
| `limit`                  | list                | Page size.                                                    |

Annotations require versioned messages, global `annotations.enabled`, and channel-level annotation permission.

When built with `ably-compat`, the root compatibility router additionally
exposes `GET` and `POST`
`/channels/{channelName}/messages/{messageSerial}/annotations`. POST accepts an
Ably annotation array for create or delete. GET negotiates JSON or MsgPack and
returns relative `first`/`next` Link relations with an opaque cursor scoped to
the app, channel, and message. Compatibility capabilities
`annotation-publish`, `annotation-subscribe`, `annotation-delete-own`, and
`annotation-delete-any` remain independent from message mutation capabilities.

## Push notifications [#push-notifications]

Push notifications are a core Sockudo API surface. Push endpoints are available when the binary is built with the `push` feature. They use signed app authentication plus capability headers.

With `ably-compat`, the root router also exposes `/push/publish`,
`/push/deviceRegistrations`, `/push/channelSubscriptions`, and `/push/channels`.
These are DTO/auth projections over the same durable store, queue, planner,
provider, feedback, retry, status, scheduler, and cleanup services listed below.
The Ably realtime transport is a real `MessageService` provider delivery, not a
synthetic provider success. Run the native monolith workers in an in-process
deployment, or provide the corresponding external workers for the configured
queue stages.

| Method   | Path                                                   | Auth                                       | Purpose                                                                                                                    |
| -------- | ------------------------------------------------------ | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `POST`   | `/apps/{appId}/push/credentials/fcm`                   | Signed app API, push admin                 | Create or replace Firebase Cloud Messaging credential material.                                                            |
| `POST`   | `/apps/{appId}/push/credentials/apns`                  | Signed app API, push admin                 | Create or replace APNs credential material.                                                                                |
| `POST`   | `/apps/{appId}/push/credentials/webpush`               | Signed app API, push admin                 | Create or replace Web Push VAPID credential material.                                                                      |
| `POST`   | `/apps/{appId}/push/credentials/hms`                   | Signed app API, push admin                 | Create or replace Huawei Mobile Services credential material.                                                              |
| `POST`   | `/apps/{appId}/push/credentials/wns`                   | Signed app API, push admin                 | Create or replace Windows Notification Service credential material.                                                        |
| `GET`    | `/apps/{appId}/push/credentials`                       | Signed app API, push admin                 | List provider credentials with secrets redacted.                                                                           |
| `POST`   | `/apps/{appId}/push/templates`                         | Signed app API, push admin                 | Create or update a push payload template.                                                                                  |
| `GET`    | `/apps/{appId}/push/templates`                         | Signed app API, push admin                 | List push templates.                                                                                                       |
| `GET`    | `/apps/{appId}/push/templates/{id}`                    | Signed app API, push admin                 | Read one push template.                                                                                                    |
| `DELETE` | `/apps/{appId}/push/templates/{id}`                    | Signed app API, push admin                 | Delete one push template.                                                                                                  |
| `POST`   | `/apps/{appId}/push/deviceRegistrations`               | Signed app API, push admin or subscribe    | Register or update a device.                                                                                               |
| `GET`    | `/apps/{appId}/push/deviceRegistrations`               | Signed app API, push admin                 | List device registrations.                                                                                                 |
| `DELETE` | `/apps/{appId}/push/deviceRegistrations`               | Signed app API, push admin                 | Delete devices by filter. Empty filters are rejected.                                                                      |
| `GET`    | `/apps/{appId}/push/deviceRegistrations/{id}`          | Signed app API, push admin or device token | Read one device registration.                                                                                              |
| `DELETE` | `/apps/{appId}/push/deviceRegistrations/{id}`          | Signed app API, push admin or device token | Delete one device registration.                                                                                            |
| `POST`   | `/apps/{appId}/push/channelSubscriptions`              | Signed app API, push admin or subscribe    | Upsert a channel push subscription.                                                                                        |
| `GET`    | `/apps/{appId}/push/channelSubscriptions`              | Signed app API, push admin                 | List channel push subscriptions.                                                                                           |
| `DELETE` | `/apps/{appId}/push/channelSubscriptions`              | Signed app API, push admin or subscribe    | Delete subscriptions by channel or device.                                                                                 |
| `GET`    | `/apps/{appId}/push/channelSubscriptions/channels`     | Signed app API, push admin                 | List channels with push subscriptions.                                                                                     |
| `POST`   | `/apps/{appId}/push/publish`                           | Signed app API, push admin                 | Admit one push publish request.                                                                                            |
| `POST`   | `/apps/{appId}/push/batch/publish`                     | Signed app API, push admin                 | Admit multiple push publish requests.                                                                                      |
| `GET`    | `/apps/{appId}/push/publish/{publishId}/status`        | Signed app API, push admin                 | Read aggregate push publish status.                                                                                        |
| `GET`    | `/apps/{appId}/push/deadLetters`                       | Signed app API, push admin                 | List safe queue-native dead-letter metadata with optional `provider`, `sinceMs`, `untilMs`, `limit`, and `cursor` filters. |
| `POST`   | `/apps/{appId}/push/deadLetters/{deadLetterId}/replay` | Signed app API, push admin                 | Re-enqueue a replayable dead-letter's original queue item.                                                                 |
| `DELETE` | `/apps/{appId}/push/scheduled/{jobId}`                 | Signed app API, push admin                 | Cancel a scheduled publish before dispatch when possible.                                                                  |
| `POST`   | `/apps/{appId}/push/deliveryStatus`                    | Signed app API, push admin                 | Ingest worker or provider delivery feedback.                                                                               |

Push publish bodies accept `notBeforeMs` to delay provider dispatch and `expiresAtMs` to stop late
delivery and retry. The public API does not expose `POST /push/scheduled`; delayed delivery is
created through `/push/publish`, while `DELETE /push/scheduled/{jobId}` only cancels jobs already
persisted by scheduler/store integrations.

Publish responses include `renderedPayloads`. These previews are rendered from the same effective
payload queued for delivery: templates are resolved at admission, request `providerOverrides` win
over template overrides, and retries reuse the accepted rendered payload semantics. Missing
templates, missing template variables, malformed overrides, and oversized provider payloads fail
before provider dispatch.

Publish status inspection is available through `GET /apps/{appId}/push/publish/{publishId}/status`.
Dead-letter messages are retained in the configured queue backend and are not exposed through a
public HTTP listing endpoint yet.

Push headers:

| Header                                   | Values                         | Meaning                                                                                                                 |
| ---------------------------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------- |
| `x-sockudo-push-capability`              | `push-admin`, `push-subscribe` | Omitted defaults to admin capability for trusted server calls. Client-assisted subscription flows use `push-subscribe`. |
| `x-sockudo-device-identity-token`        | Bearer-style device token      | Required when a non-admin request reads, updates, or deletes an existing device.                                        |
| `x-sockudo-rotate-device-identity-token` | boolean                        | Admin-only device registration control that rotates the device identity token.                                          |
| `x-sockudo-push-quota-override`          | implementation-defined         | Admin-only quota override hook for controlled operations.                                                               |

Push list endpoints use cursor pagination with `limit` and opaque `cursor`.

## Operational endpoints [#operational-endpoints]

| Method | Path                          | Auth                          | Purpose                                                                                                                                |
| ------ | ----------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `GET`  | `/live`                       | None                          | Liveness probe. Does not perform dependency checks.                                                                                    |
| `GET`  | `/accept-traffic`             | None                          | New-connection admission probe. Returns `503` while draining or shedding for memory pressure, without closing established connections. |
| `GET`  | `/up`                         | None                          | Global health probe.                                                                                                                   |
| `GET`  | `/up/{appId}`                 | None                          | App-specific health probe.                                                                                                             |
| `GET`  | `/usage`                      | None, when enabled            | Memory usage snapshot. Enabled by `HTTP_API_USAGE_ENABLED`.                                                                            |
| `GET`  | `/stats`                      | Dispatches by auth            | Native operator summary without Ably credentials; Ably interval stats with an Ably key or bearer token.                                |
| `GET`  | `/operator/stats`             | None, when enabled            | Unambiguous alias for the native occupancy, memory, and history-health response.                                                       |
| `GET`  | `/operator/stats/aggregation` | None, with Ably compatibility | Stats-worker backlog, capacity, dropped observations, flush failures, batch count, observation count, and last flush lag.              |
| `POST` | `/stats`                      | Ably key or bearer token      | Conformance fixture ingestion through the real stats store; available only when `stats_fixture_ingest_enabled = true`.                 |

Ably `GET /stats` accepts interval IDs or epoch milliseconds for `start` and
`end`, `unit`/`by` values `minute`, `hour`, `day`, or `month`, forward/backward
direction, a bounded `limit`, and opaque cursors carried in `Link` headers.
Pagination is anchored to interval IDs and a fixed query horizon.

Disable `/usage`, `/operator/stats`, and operator aggregation stats in exposed
production environments unless they are behind trusted ingress controls.

`/accept-traffic` returns `200 ACCEPTING` when new connections may be admitted, including when the
memory sampler is disabled or unavailable (fail-open). It returns `503 MEMORY_PRESSURE` while the
configured RSS threshold is exceeded and `503 DRAINING` during shutdown. The
`X-Memory-Pressure-Monitor` response header distinguishes `DISABLED`, `UNAVAILABLE`, `AVAILABLE`,
and `SHEDDING`; `X-Memory-Limit-Source` reports `configured`, `cgroup_v2`, `cgroup_v1`, or
`unavailable`. Keep `/up` as the dependency/readiness probe unless a load balancer can poll a
separate admission endpoint.

## Metrics endpoint [#metrics-endpoint]

| Method | Path       | Port                         | Auth            | Purpose                               |
| ------ | ---------- | ---------------------------- | --------------- | ------------------------------------- |
| `GET`  | `/metrics` | Metrics port, default `9601` | None by default | Prometheus plaintext scrape endpoint. |

The metrics server is independent from the main WebSocket/API listener and is enabled by `METRICS_ENABLED`.

## Signed app API parameters [#signed-app-api-parameters]

All `/apps/*` requests use Pusher-style query signing:

| Parameter        | Meaning                                                 |
| ---------------- | ------------------------------------------------------- |
| `auth_key`       | App key.                                                |
| `auth_timestamp` | Unix timestamp used to reject stale requests.           |
| `auth_version`   | Signing protocol version, usually `1.0`.                |
| `body_md5`       | MD5 of the raw JSON body for body-bearing requests.     |
| `auth_signature` | HMAC-SHA256 over method, path, and sorted query string. |

Use server SDKs for signing when possible. Custom signers must sign the canonical path exactly as routed, including `/apps/{appId}`.


# Logging (/docs/reference/logging)



## Configuration [#configuration]

Sockudo installs its logging subscriber once at startup, so early boot diagnostics are visible.
After configuration files and environment overrides are resolved, the filter and formatting
settings are reloaded once. Output format is selected at process startup and cannot be changed by
configuration.

| Variable             | Default                               | Description                                              |
| -------------------- | ------------------------------------- | -------------------------------------------------------- |
| `LOG_OUTPUT_FORMAT`  | `text`                                | `text` for human-readable, `json` for structured JSON    |
| `RUST_LOG`           | —                                     | Standard filter directive; overrides everything when set |
| `SOCKUDO_LOG_DEBUG`  | `info,sockudo=debug,tower_http=debug` | Filter when `debug = true`                               |
| `SOCKUDO_LOG_PROD`   | `info`                                | Filter when `debug = false`                              |
| `LOG_INCLUDE_TARGET` | `true`                                | Include the module path in output                        |
| `LOG_COLORS_ENABLED` | `true`                                | ANSI colors in text output                               |

Filter precedence is `RUST_LOG`, then the mode-specific override, then the built-in default.
Source location (file and line) is included when `debug = true`. The `[logging]` config section
controls `include_target` and `colors_enabled` after configuration loads.

## JSON output [#json-output]

With `LOG_OUTPUT_FORMAT=json`, each line is a JSON object. Application event attributes live under
`fields` so they cannot collide with formatter metadata such as `timestamp`, `level`, and
`target`. The current span is emitted as `span`; the redundant full span list is omitted.

```json
{
  "timestamp": "2026-07-18T12:00:00.000Z",
  "level": "INFO",
  "target": "sockudo_adapter::handler",
  "span": { "name": "socket", "app_id": "my-app", "socket_id": "123.456" },
  "fields": { "message": "socket connected", "channel_count": 3 }
}
```

## OpenTelemetry export [#opentelemetry-export]

When `[opentelemetry].enabled` and `logs_enabled` are true, the same structured tracing events are
also emitted as OpenTelemetry log records. OTLP export does not replace stdout/stderr logging and
does not change `LOG_OUTPUT_FORMAT`. Log records created inside a sampled trace carry its trace and
span context, allowing an OpenTelemetry backend to correlate logs with request, WebSocket,
horizontal-fanout, webhook, and queue spans.

Explicit `sockudo_telemetry` instrumentation spans remain enabled while trace export is active,
independently of the local `RUST_LOG` verbosity. This preserves distributed trace continuity
without enabling debug or trace log events on stdout/stderr.

Sockudo supports the stable OpenTelemetry traces, metrics, and logs signals over OTLP gRPC,
HTTP/protobuf, and HTTP/JSON. W3C `traceparent`/`tracestate` propagation and baggage are enabled by
default when telemetry is active. OpenTelemetry profiles are not supported.

Export is asynchronous, bounded, and fail-open. Collector connection failures, exporter timeouts,
or a full telemetry queue never make `/live` or `/up` fail and do not block normal request handling.
Sockudo attempts a bounded flush during graceful shutdown. Configure collectors, protocols,
sampling, and credentials with the
[OpenTelemetry environment variables](/docs/reference/environment-variables#opentelemetry).

## Severity levels [#severity-levels]

| Level   | Meaning                                                              | Monitoring guidance                  |
| ------- | -------------------------------------------------------------------- | ------------------------------------ |
| `error` | Terminal operation or delivery failure                               | Alert — will not self-heal           |
| `warn`  | Degraded behavior, rejected input, scheduled retry                   | Watch for sustained patterns         |
| `info`  | Startup, shutdown, connection lifecycle, final webhook/push outcomes | Dashboards and audit                 |
| `debug` | Queue, cache, persistence, coordination mechanics                    | Enable for troubleshooting           |
| `trace` | Per-message and detailed timing events                               | High volume; targeted debugging only |

## Correlation fields [#correlation-fields]

Filter and correlate events with these stable fields:

| Field       | Description                                                   |
| ----------- | ------------------------------------------------------------- |
| `app_id`    | Application identifier                                        |
| `socket_id` | WebSocket connection identifier                               |
| `user_id`   | Authenticated user identifier                                 |
| `channel`   | Channel name                                                  |
| `protocol`  | Realtime protocol surface (`pusher`, `sockudo_v2`, or `ably`) |
| `adapter`   | Horizontal transport backend (`nats`, `redis`, `kafka`, ...)  |
| `error`     | Error description for failed operations                       |
| `retryable` | Whether the operation will be retried                         |
| `worker_id` | Background worker identifier                                  |

## Production lifecycle events [#production-lifecycle-events]

The default `info` filter records socket connection, sign-in, subscription, explicit
unsubscription, terminal disconnection (with cause), configuration loading, and final webhook and
push outcomes. Message traffic, presence-member transitions, queue traffic, and retry internals
remain at `debug` or `trace`. A scheduled webhook retry is a `warn`; permanent delivery failure is
an `error`.

The optional Ably compatibility surface uses the same lifecycle policy. Its connection, attachment,
detachment, recovery-source, and disconnection events carry `protocol = "ably"` plus the applicable
`app_id`, `connection_id`, and `channel` fields.

## Data safety [#data-safety]

Logs may contain operational IDs, channel names, user IDs, and event names. Logs never contain app
keys or secrets, signatures, tokens, authorization data, request or response bodies, message
payloads, raw queries, webhook URLs, provider credentials, device tokens, or private keys. This
applies at every level: enabling `debug` or `trace` never permits payload, credential, or token
logging. The same rule applies to OpenTelemetry logs, span attributes, events, resource attributes,
baggage, and exporter diagnostics. Store OTLP headers and client private keys outside the config
file, and do not put secrets or user payloads in `OTEL_RESOURCE_ATTRIBUTES` or baggage.


# Protocol reference (/docs/reference/protocol)



Sockudo exposes two protocol layers on the same server.

## Protocol V1 [#protocol-v1]

Protocol V1 is Pusher-compatible. It preserves familiar event prefixes, channel names, subscription flows, auth response shapes, and HTTP API semantics.

Use V1 for:

* existing `pusher-js` clients
* Laravel Echo migrations
* backend SDK compatibility
* minimal drop-in deployments

Typical V1 connection and event frames keep the Pusher names and payload conventions:

```json
{
  "event": "pusher:connection_established",
  "data": "{\"socket_id\":\"123.456\",\"activity_timeout\":120}"
}
```

```json
{
  "event": "order.created",
  "channel": "orders",
  "data": "{\"id\":\"ord_123\",\"total\":4200}"
}
```

Keep V1 payloads compatible with existing clients. Do not require V2-only fields such as `message_id`, `serial`, `stream_id`, tags, deltas, or annotation metadata when the recipient negotiated V1.

## Protocol V2 [#protocol-v2]

Protocol V2 is Sockudo-native and uses `sockudo:` system event prefixes.

Use V2 for:

* `message_id`
* `serial`
* `stream_id`
* connection recovery
* subscribe-time rewind
* delta compression
* tag filtering
* durable history
* mutable messages
* annotations
* push-helper client workflows through backend proxies

V2 keeps the event/channel shape familiar, then adds metadata required for continuity and durable workflows:

```json
{
  "event": "order.created",
  "channel": "orders",
  "data": { "id": "ord_123", "total": 4200 },
  "message_id": "msg_01HX6J2P5N0Z9E",
  "stream_id": "orders",
  "serial": 42,
  "extras": {
    "headers": { "tenant": "acme" },
    "tags": { "status": "paid", "region": "eu" }
  }
}
```

Use V2 when the client needs to reconnect without guessing what it missed, rewind a subscription from known history, filter by event, tags, or message content, receive deltas, or render mutable message state.

### Subscription predicates [#subscription-predicates]

V2 subscriptions accept one compound `filter`. Its non-empty `events`, `tags`, and `expression` components use AND semantics; separate exact and wildcard subscriptions use OR semantics. The expression form is either a JMESPath source string or `{ "language": "jmespath", "source": "..." }`.

```json
{
  "channel": "orders.*",
  "filter": {
    "events": ["order.updated"],
    "tags": { "cmp": "eq", "key": "region", "val": "eu" },
    "expression": "data.total >= `100` && headers.priority == `\"high\"`"
  }
}
```

Predicates compile once at subscription time and apply consistently to live fanout, rewind, and recovery. Expression projection exposes `event`, `channel`, `data`, `name`, `userId`, `tags`, public `headers`, `messageId`, `streamId`, `serial`, and `action`. Internal `sockudo_` headers, extras, and idempotency keys are never exposed. Source, AST, filter-tree, and projected-document sizes are bounded; invalid input is rejected and evaluation errors fail closed. This remains gated by Protocol V2 and `[tag_filtering].enabled`; Protocol V1 channel validation and delivery are unchanged.

```mermaid
sequenceDiagram
  participant Client
  participant Sockudo
  participant History
  Client->>Sockudo: subscribe orders with protocol_version=2
  Sockudo-->>Client: sockudo:subscription_succeeded
  Sockudo->>History: persist event with stream_id and serial
  Sockudo-->>Client: order.created with message_id and serial
  Client->>Sockudo: resume from last serial
  Sockudo->>History: read gap
  Sockudo-->>Client: replay missed events
```

## Prefixes [#prefixes]

| Event family         | V1                 | V2                        |
| -------------------- | ------------------ | ------------------------- |
| Public system events | `pusher:`          | `sockudo:`                |
| Internal events      | `pusher_internal:` | `sockudo_internal:`       |
| Mutable messages     | not available      | `sockudo:message.*`       |
| Recovery             | not available      | `sockudo:resume_*`        |
| Rewind               | not available      | `sockudo:rewind_complete` |

## Channel names [#channel-names]

| Channel   | Prefix               |
| --------- | -------------------- |
| Public    | none                 |
| Private   | `private-`           |
| Presence  | `presence-`          |
| Encrypted | `private-encrypted-` |

## Broadcast metadata [#broadcast-metadata]

```json
{
  "event": "order.updated",
  "channel": "orders",
  "data": { "id": "ord_123" },
  "message_id": "msg_01HX",
  "stream_id": "orders",
  "serial": 42,
  "extras": {
    "headers": { "tenant": "acme" },
    "tags": { "status": "packed" }
  }
}
```

V1 clients should not depend on V2-only fields.

## Compatibility boundary [#compatibility-boundary]

The compatibility rule is simple: negotiate the richest protocol a client can safely understand, then deliver only fields that belong to that protocol.

| Concern          | V1 behavior                                | V2 behavior                                                                          |
| ---------------- | ------------------------------------------ | ------------------------------------------------------------------------------------ |
| System prefixes  | `pusher:` and `pusher_internal:`           | `sockudo:` and `sockudo_internal:`                                                   |
| Event metadata   | Pusher-compatible event, channel, and data | Adds serials, message IDs, stream IDs, extras, tags, and headers where enabled.      |
| Recovery         | Client reconnects and resubscribes         | Client can resume from continuity metadata when recovery is configured.              |
| Mutable messages | Not exposed                                | Message updates, deletes, appends, versions, and annotations are explicit V2 events. |
| Server publish   | Pusher-shaped HTTP API                     | Same trusted API plus V2 acknowledgement fields when enabled.                        |

For mixed deployments, treat Protocol V1 as a stable contract and Protocol V2 as an opt-in capability layer. A backend may publish once, but Sockudo must shape the delivery for each subscriber according to the subscriber protocol.

## Push is outside the WebSocket protocol [#push-is-outside-the-websocket-protocol]

Push notifications are HTTP-driven. They target device registrations, channel push subscriptions, clients, or explicit recipients. A push payload may reference a realtime channel or message serial, but provider delivery is not part of WebSocket ordering.

The optional Ably compatibility facade can resolve an Ably realtime-channel push
recipient into a native `MessageService` publish. This is an internal push target,
not another network provider or fallback transport; it follows the channel's
ordinary fanout, persistence, history, version, and idempotency semantics.


# Ably REST, WebSocket, and AI Transport compatibility (/docs/server/ably-ai-transport-compatibility)



Sockudo can expose an Ably-compatible REST and WebSocket surface when the server
is built with Cargo feature `ably-compat`. The evidence-backed scope is **Ably
REST and WebSocket compatibility, excluding Live Objects**. It also supports
Ably AI Transport clients through Sockudo's native mutable-message, history,
recovery, presence, annotation, stats, and push services.

> **Community support notice:** Sockudo and this compatibility layer are
> community-built and community-maintained. They are not Ably products, and
> Ably does not provide support for them. Use the Sockudo community and project
> support channels for help.

The facade lives in the optional `sockudo-ably-compat` workspace crate. The
server constructs one `AblyCompatRuntime` per server instance and merges its
routes. Socket writers and attachment state remain local to that process;
authorization, session ownership, recovery, and durable channel state use the
shared authorities described below. Ably realtime is WebSocket-only: Comet,
XHR polling or streaming, SSE, long polling, and other fallback transports are
intentionally unsupported.

Use **Ably REST and WebSocket compatibility, excluding Live Objects** in public
material. Do not describe this as full Ably compatibility.

The pinned manifest classifies 41 target files. Exactly the two Live Objects
files and the multiple/non-WebSocket transport file are excluded, and there are
no per-test exclusions. Node defaults, browser execution, strict execution of
upstream-pending bodies, and AI Transport have separate reports and pass/fail
states. A result from one lane is never used to claim another lane passed.

Pull requests also run a moving latest-upstream gate. It resolves and records
the current `main` commits of `ably-js`, `ably-go`, and
`ably-ai-transport-js`. It runs ably-js over WebSocket, the official ably-go
unit and JSON/MsgPack integration suites, and the complete AI Transport unit
and integration suites. The structurally defined exceptions are the two Live
Objects files, the multi-transport file, the Comet inventory assertion, and two
local-routing TLS/port assertions. The Go and AI Transport suites have no test
exclusions.

Fresh pinned source-build evidence is green: Node defaults are 575/575,
strict completeness is 250/250, Chromium defaults are 574/574 with no browser
boundary errors, Chromium strict is 250/250, and AI Transport is 50/50. A new
published Sockudo tag must still pass the released-binary workflow before that
tag is promoted as verified.

## Compatibility Matrix [#compatibility-matrix]

| Surface                                                | Status                                       | Scope                                                                                                                                                                                                                                                                                                                       |
| ------------------------------------------------------ | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ably Realtime WebSocket JSON/MsgPack protocol          | Node, Chromium, and Go source evidence green | Root WebSocket endpoint accepts Ably `ProtocolMessage` connect, attach, detach, message, presence, auth, heartbeat, ACK, and NACK flows covered by pinned `ably@2.21.0` and ably-go JSON/MsgPack lanes. Non-WebSocket transports are intentionally unsupported.                                                             |
| Ably REST time/token/history/message reads and publish | Node, Chromium, and Go source evidence green | `/time`, `/keys/{keyName}/requestToken`, `/keys/{keyName}/revokeTokens`, `/messages` batch publish, `/channels/{channel}/messages`, the Go SDK's `/channels/{channel}/history` alias, message lookup, version lookup, and presence reads/history exist for the selected SDK path. Full Ably REST API parity is not claimed. |
| Run lifecycle vocabulary                               | Supported                                    | Native AI Transport docs and fixtures use `ai-run-start`, `ai-run-suspend`, `ai-run-resume`, `ai-run-end`, and `ai-cancel`.                                                                                                                                                                                                 |
| Legacy turn vocabulary                                 | Legacy alias                                 | `ai-turn-start`, `ai-turn-end`, `turn-id`, and `turn-reason` remain accepted for migration, but new docs and demos use run vocabulary.                                                                                                                                                                                      |
| Mutable messages and output streaming                  | Pinned Node and Chromium evidence green      | Ably `publish`, `appendMessage`, `updateMessage`, `deleteMessage`, `getMessage`, `getMessageVersions`, and history map to Sockudo versioned messages.                                                                                                                                                                       |
| Message annotations                                    | Supported for the pinned annotation suite    | Realtime action `21`, REST create/delete/list, summaries, pagination, JSON, and MsgPack translate onto Sockudo's native annotation store and projection pipeline. Protocol V1 remains unchanged.                                                                                                                            |

Compatibility projections read a shared commit-time message envelope. This
keeps IDs, publisher identity, timestamps, encoding chains, supported extras,
history/recovery positions, and version operation metadata consistent across
live delivery and REST reads while leaving Protocol V1's projection unchanged.

## Distributed state authority [#distributed-state-authority]

Clustered compatibility requires Redis or Redis Cluster for coordination and a
non-memory durable history/version driver. Sockudo does not use a process-local
`DashMap` as cross-node authority and does not fall back from an unavailable
coordination backend to private memory.

| State                                                                             | Class                                            | Authority and retention                                                                                                                                                                                                          | Backend failure behavior                                                                                                                                                                                                                               |
| --------------------------------------------------------------------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| App keys and configured rotated keys                                              | Durable configuration                            | `AppManager` plus the deployment's versioned key configuration; every node must load the same generation.                                                                                                                        | Unknown or stale generations reject authentication. Secrets are never logged.                                                                                                                                                                          |
| Opaque tokens                                                                     | Replicated ephemeral                             | Shared cache record for the token TTL. A process-local decoded copy is only an optimization after the shared record is validated.                                                                                                | Token resolution fails closed when shared state cannot be read.                                                                                                                                                                                        |
| Nonce replay and token revocations                                                | Replicated ephemeral security state              | Atomic shared-cache claims. Revocations live for at least the maximum accepted token TTL.                                                                                                                                        | Replay/revocation checks fail closed. Revocation polling disconnects sockets on every node; renewal changes the shared authorization generation so stale timers cannot restore access.                                                                 |
| WebSocket writer, auth timer, channel gate, negotiated attachment                 | Local connection state                           | The node owning the socket. Gates and queues have count and byte bounds and end with the transport.                                                                                                                              | The socket closes when safe local delivery cannot continue; another node never fabricates ownership.                                                                                                                                                   |
| Session owner and recovery key                                                    | Replicated ephemeral                             | Compare-and-swap owner lease in shared cache, bounded by the connection-state TTL. Recovery keys are single-use and key rotation invalidates the previous key.                                                                   | Resume fails closed when the owner lease or continuity record cannot be verified. A successful claim on another node supersedes the old owner.                                                                                                         |
| Publish idempotency claim and acknowledgement                                     | Replicated ephemeral plus durable repair         | Atomic cache claim/receipt for the idempotency TTL; canonical history/version records repair an ambiguous post-commit receipt.                                                                                                   | A backend error does not permit a second winner. A retry is acknowledged only after its canonical commit is proven.                                                                                                                                    |
| Current presence                                                                  | Replicated ephemeral                             | Per-connection typed membership, replicated by the horizontal adapter and tied to the node-health lease. Abrupt transport loss starts a bounded 15-second removal lease, independently of the two-minute connection-state lease. | A recovered live owner cancels pending removal. Duplicate leave/dead-node notifications are idempotent. Node death removes every orphan connection; failure to coordinate cleanup is degraded and observable rather than reported as a clean snapshot. |
| Presence transitions                                                              | Durable                                          | Configured durable history backend and its retention policy.                                                                                                                                                                     | Current membership can remain available, but continuity-sensitive history fails closed while degraded.                                                                                                                                                 |
| Attachment high-water and reconnect continuity                                    | Local attachment plus durable canonical position | The live attachment owner captures the high-water; durable history/version streams prove reconnect ranges.                                                                                                                       | `untilAttach` requires routing affinity to the attachment owner. Resume/recover returns reset-required when stream identity or a contiguous range cannot be proven.                                                                                    |
| Remote delivery suppression                                                       | Local bounded projection state                   | Each subscriber node keeps a contiguous per-stream delivery high-water plus a bounded set of out-of-order serials; Redis fanout carries the complete canonical envelope.                                                         | Serials at or below proven continuity remain suppressed without retaining an unbounded ledger. Missing canonical identity is not guessed; reorder-window overflow is counted and fails reset-required instead of evicting duplicate evidence.          |
| Mutable versions and delivery serials                                             | Durable                                          | Configured `VersionStore`; create, predecessor compare-and-apply, operation receipt, and serial allocation are one backend commit.                                                                                               | Mutation and recovery fail closed on conflicts, gaps, or backend errors.                                                                                                                                                                               |
| Annotation serials, create receipts, event stream, and summaries                  | Durable                                          | The configured non-memory version-storage driver owns atomic per-channel serial allocation, stable create IDs, canonical events, materialized summaries, and bounded retention.                                                  | Annotation writes and rebuilds fail closed; no node-local projection is used in a cluster.                                                                                                                                                             |
| Stats buckets                                                                     | Replicated retained aggregate                    | Bounded local ingestion workers merge minute buckets into shared cache with TTL.                                                                                                                                                 | Queue drops and backend failures are counted. Queries cannot treat an unflushed or unreadable bucket as a complete cluster barrier.                                                                                                                    |
| Push registrations, subscriptions, statuses, publish log, feedback, and schedules | Durable                                          | Configured push storage and queue; conditional status/idempotency/worker claims use backend operations.                                                                                                                          | Production rejects accidental memory drivers. Failed provider/queue/storage work remains observable through durable status and backlog metrics.                                                                                                        |

Memory cache, history, version, annotation, push-storage, and push-queue
implementations are single-node development choices only. Clustered startup
rejects combinations that would split an authority between processes.

Mutable REST and realtime operations call the native typed mutation service.
That service atomically validates the predecessor, owns the delivery position,
commits the aggregate and individual version, enforces stream limits, and stores
the operation receipt in the configured `VersionStore`. The compatibility layer
only decodes/encodes Ably JSON or MsgPack shapes. It has no parallel mutable
state, counter, or handler-to-handler call.

### Publish identity and retries [#publish-identity-and-retries]

Ably messages may omit `name`, `data`, or both; the facade does not manufacture
an event name. The commit envelope retains the original encoding chain,
`headers`, `ref`, `ai`, and `push` extras, message ID, authenticated publisher
client ID, originating connection ID, and server publish timestamp. Live
delivery, durable history, rewind, recovery, message reads, and mutation
projections all read those same commit-time facts.

`X-Ably-ClientId` is decoded as strict standard-base64 UTF-8, matching the Ably
SDK request format. A query/header identity mismatch, fixed-token mismatch,
message identity mismatch, or mutation-operation identity mismatch is rejected
before persistence. REST `message.connectionKey` is accepted only when it
identifies a currently live connection in the same app; the target identity is
then used for authorization and delivery stamping. `connectionId` remains a
server-assigned output field.

`message.id` uses Sockudo's shared publish-idempotency coordinator. The first
node atomically claims the app/channel/id tuple, commits through the normal
history/version/fanout pipeline, and stores the exact serial receipt. An
identical REST or realtime retry returns that receipt without publishing again;
the same ID with a different canonical payload or publisher identity fails
deterministically. Redis and Redis Cluster use atomic compare-and-swap scripts,
so a handler-local acknowledgement cache is never the cross-node authority.
The durable envelope stores only the hashed idempotency identity and payload
fingerprint. If a node exits after history/version persistence but before the
receipt compare-and-swap, another node reads the authoritative durable record,
atomically repairs the pending receipt, and returns the original acknowledgement
without repeating persistence or fanout. Ordinary publishes retain their
message-ID acknowledgement shape; versioned publishes retain the exact original
message, history, delivery, and version serial tuple.

Realtime echo combines the connection `echoMessages` setting, the channel
`echo` parameter, and the optional message-level `extras.echo` override. Only
the actual originating connection is suppressed; another connection with the
same client ID still receives the message. A publish frame is validated in full
before the first commit, then receives exactly one ACK or NACK covering its
inbound `msgSerial`/`count` range, including a frame containing multiple
messages.

### Presence across reconnects [#presence-across-reconnects]

Ably `DISCONNECT` and `CLOSE` have different presence lifecycles. `DISCONNECT`
ends the current WebSocket transport and starts the Ably presence grace period.
The default is 15 seconds; the `remainPresentFor` transport parameter may reduce
it to a minimum of one second. Recovery during that window keeps the
authoritative member set and cancels the pending removal. After the grace period,
Sockudo removes the members and fans out their leave transitions while retaining
the connection recovery state for its independent two-minute lease. `CLOSE` is
terminal: it removes the connection's members immediately and invalidates its
recovery state. Both paths still unsubscribe the terminated transport session
from local channel delivery.

On automatic presence re-entry, Sockudo preserves a supplied member ID only
when it is a valid `connectionId:msgSerial:index` ID owned by the current
connection. This keeps same-connection re-entry idempotent while a missing,
malformed, or foreign-connection ID is replaced with a server-assigned ID.
Presence snapshots remain keyed by the `(clientId, connectionId)` pair, so the
same client ID on distinct connections remains distinct.

### Batch REST routing [#batch-rest-routing]

The Ably compatibility router accepts both `POST /messages` request families.
A raw `Rest.request()` batch spec returns the legacy ordered channel array with
`201` when every publish commits. If any channel fails, already committed
channels remain successful and the route returns `400`/`40020` with
`{ error, batchResponse }`; every entry retains its original position and its
own `messageId` or `error`. The `batchPublish()` array body returns an ordered
array of `{ successCount, failureCount, results }` envelopes with `200`, including
per-channel partial failures.

Every message is parsed and validated before its channel work starts. Successful
items call the native `MessageService` and are not acknowledged until its
idempotency, durable history/version, fanout, metrics, webhook, and optional push
path returns a commit receipt. `GET /presence` reads each channel independently
through the native `PresenceService`; `/keys/{keyName}/revokeTokens` writes each
valid target through the shared revocation store. A failure in one presence read
or revocation write is therefore an item failure, not a fabricated whole-batch
success or a rollback of unrelated items.

Batch work is bounded and ordered. At most eight channel/target operations run
concurrently. A request accepts at most 100 specs, 1,000 channel results, 10,000
channel-message publish operations, and 10 MiB of encoded JSON or MsgPack. The
per-spec channel and message limits are additionally capped by
`event_limits.max_channels_at_once` and `event_limits.max_batch_size`. Known
resources reject unsupported methods immediately with `405` and `Allow`; unknown
resources return `404`. JSON/MsgPack content types, Ably error headers, and
pagination links use the shared REST encoders.

### Channel names [#channel-names]

The Ably facade parses channel names independently from Sockudo's native
Pusher-compatible validator. After REST path decoding, Ably channels may contain
spaces, namespace colons, braces, quotes, and Unicode. Empty names, control
characters, reserved leading `:` names, malformed qualifiers, and names larger
than 16 KiB of UTF-8 are rejected with Ably error code `40010`. These rules apply
consistently to REST channel operations and realtime attach, publish, and
presence frames; Protocol V1 validation is unchanged.

Qualified names such as `[?rewind=1]room` and `[filter=...]room` retain their
full name on Ably wire messages, capability checks, attach state, and errors.
Their base name (`room`) is the sole identity used by Sockudo fanout, history,
recovery, presence, and mutable-message services. Attaching both a base and a
qualified name therefore creates two subscription projections over one channel
authority, not duplicate durable state.

Derived `[filter=<base64>]` subscriptions compile the decoded JMESPath expression
once at attach. The expression is size-bounded and filters both live messages and
recovery backlog before wire encoding; malformed base64, UTF-8, or JMESPath fails
only that channel with code `40010` and leaves the connection usable.
Compiled predicates are shared by expression through a bounded 1,024-entry,
1 MiB cache. Cache hit, miss, eviction, current-entry, and current-byte counters
are included in the compatibility runtime metrics snapshot. Compilation happens
outside the cache lock, and delivery evaluates the cached predicate over the
real Ably message projection, including extras and headers.

Realtime attach negotiates channel modes as typed flags. `params.modes` takes
precedence over the protocol mode flags, unknown parameters are not reflected,
and an attach without explicit modes starts from the supported Ably default set.
The requested/default modes are intersected with the authenticated token's
channel capabilities. Unsupported modes are omitted from `ATTACHED` rather than
rejecting an otherwise authorized attachment; an attachment fails with `40160`
only when the intersection is empty. This mode intersection never widens
authority: annotation operations and object publishing still require their
explicit capabilities, object subscription follows Ably's `subscribe` or
`object-subscribe` rule, and an operation outside the granted modes is rejected.
String, boolean, and finite numeric attach parameters are normalized at the
JSON/MsgPack protocol boundary; binary, null, array, object, and non-finite
values are rejected instead of widening runtime parameter handling.
The negotiated parameters and flags are echoed on every `ATTACHED` outcome.
Each connection retains a typed attachment contract containing the parsed
qualified/base name, accepted parameters, explicit/default modes, compiled
filter, attach position, and presence state. Reattach requests that omit these
fields reuse the retained contract instead of reverting to unrelated defaults.
Publish, presence entry, message subscription, and presence subscription are
enforced independently; a disallowed publish or presence operation is NACKed
with `40160` without failing the connection or another channel.

REST history uses Sockudo's opaque durable-history cursor. When more results are
available, the facade emits Ably-compatible relative `first` and `next` Link
relations and preserves the original direction, limit, and bounds without
reflecting credentials into the link. JSON and MsgPack pagination use the same
cursor and base-channel state.
For realtime rewind, the selected durable/versioned message is collected before
`ATTACHED`; `HAS_BACKLOG` is set exactly when a replay item follows that frame.
Rewind accepts positive message counts and positive `s`, `m`, or `h` time forms,
is capped by the resolved channel history policy, and is delivered oldest first.
An `ATTACH_RESUME` request retains the channel position but does not replay the
rewind selection a second time.

### Ably VCDIFF delta delivery [#ably-vcdiff-delta-delivery]

An Ably channel may request `params.delta = "vcdiff"` on `ATTACH`. The accepted
value is echoed on `ATTACHED`; other delta modes are not accepted or reflected.
This is an Ably-only egress projection and is separate from Sockudo's native
Pusher/V2 delta events and `[delta_compression]` algorithm settings.

Each attached subscriber retains at most one base for each requested channel,
keyed by the last delivered Ably message ID. A base is limited to 64 KiB and
expires after 120 seconds; the periodic compatibility sweep removes idle expired
bases. Detach, connection close, reattach, recovery discontinuity, and a native
stream-generation change reset the base. VCDIFF is emitted only when encoding
succeeds and the delta is smaller than the complete encoded payload. Otherwise
the subscriber receives a full message and that full payload becomes the next
eligible base.

JSON deltas use the portable `json/utf-8/vcdiff/base64` encoding chain and
`extras.delta = { from, format: "vcdiff" }`. Unsupported, binary, or cipher
encoding chains pass through unchanged and break the chain rather than being
reinterpreted as JSON. JSON and MsgPack subscribers use the same payload and ID
semantics. Missing bases and decode errors are handled by ably-js as `40018`
reattach recovery; a client that requested VCDIFF without installing its plugin
fails the channel with `40019`. The reattach starts with a full canonical
message before delta delivery resumes.

Only the per-subscriber wire projection is compressed. Durable history, hot
replay, recovery inputs, webhooks, annotations, push, and cross-node fanout all
retain and distribute the original complete message. Run
`cargo bench -p sockudo-ably-compat --bench ably_vcdiff --features delta` to
measure CPU time, allocation count/bytes, and encoded/full ratios for similar
and dissimilar 1 KiB and 64 KiB payloads at 1, 100, and 1,000 subscribers.

Subscriber registration happens before the canonical attach high-water mark is
captured. A bounded count-and-byte gate holds concurrent fanout until `ATTACHED`,
drops gated rows already covered by rewind/recovery, and then drains only later
positions in order. Gate overflow or a stream-generation change fails that
channel closed instead of silently losing a publish.

### Presence [#presence]

Ably virtual connections register with Sockudo's typed `PresenceService` and a
bounded current-member registry keyed by app, base channel, connection, and
client. Enter, update, leave, detach, explicit close, recovery expiry,
failed-resume re-entry, ATTACHED `HAS_PRESENCE`, and SYNC all project from that
authority. Duplicate disconnects are idempotent; two connections with one
client ID remain separate members while native first-join/last-leave facts are
preserved. Presence extras, data, timestamps, encoding chains, and
`connectionId:serial:index` member IDs are retained. Large SYNC sets move
members into ordered 100-item continuation frames without per-chunk member
clones. Inbound action-16 frames request the current authoritative SYNC instead
of failing the connection.

Virtual members are also written to Sockudo's native horizontal presence
registry with stable virtual socket identities. Native new-node presence-state
sync rebuilds current snapshots, while a dedicated protocol-neutral broadcast
envelope carries live transitions to Ably subscribers on other nodes. Receiving
nodes consume that envelope before Protocol V1 fanout, so compatibility
replication cannot appear as a Pusher event.

`GET /channels/{channel}/presence` reads the current native snapshot with limit,
client-id, connection-id, and opaque pagination cursor support. Presence
transitions are recorded through Sockudo's configured `PresenceHistoryStore`;
the Ably history route supports direction, time bounds, the same filters,
policy-capped limits, and cursor-preserving `first`/`next` Link relations. Sparse
filters use bounded native-store scans and return a continuation rather than
performing unbounded work. The route projects native records back to Ably
actions and retains data, encoding, extras, timestamps, and IDs. A degraded or
reset-required durable stream returns `503`/`50003` instead of presenting an
unproven history as complete. Qualified attaches and REST reads use the same
base-channel registry and durable history, so qualifiers never create parallel
presence state.

The current-member hot path uses a lock-free authenticated-app index,
per-connection channel handles, inline storage for the common one-connection
client, and bounded sharded capacity reservations. Same-client transitions
remain serialized to preserve exact first-join/last-leave facts, while
disjoint clients avoid shared channel-lookup and global-capacity cache lines.
Empty app and connection channel caches are bounded and pruned only when no
current member or connection handle depends on them.

Run `scripts/presence-bench-guard.sh` (or `make presence-bench-guard`) after a
presence hot-path change. The guard compares authoritative and legacy cycles in
the same Criterion process and requires the authoritative confidence interval
to remain faster for both single-worker and eight-worker disjoint-client loads.

### Message annotations [#message-annotations]

Realtime annotation action `21` and
`GET|POST /channels/{channel}/messages/{messageSerial}/annotations` are edge
projections over Sockudo's native `AnnotationStore`; the facade does not keep a
second annotation store. Create and delete events retain their annotation
action, message serial, type, name, client identity, data, encoding, server
timestamp, and monotonically ordered annotation serial. A caller-supplied
annotation ID makes an identical retry idempotent.

Channel modes and capabilities are separate concerns. `annotation_publish`
permits create, `annotation_subscribe` permits raw action-21 delivery, and
`annotation-delete-own` / `annotation-delete-any` authorize delete independently
from mutable-message permissions. Ordinary subscribers receive only
`message.summary`; raw annotations are emitted only when the negotiated
annotation-subscribe mode is present. Multiple/counting summaries rename the
native `clientCounts` member to Ably's `clientIds` only at this edge.

REST annotation pages negotiate JSON or MsgPack and return relative `first` and
`next` Link relations. The cursor is opaque and scoped to the authenticated app,
base channel, and original message serial; credentials are never reflected into
links. Live fanout and recovery use the same durable delivery positions as the
native summary/raw events. An unprovable history gap therefore fails through
the existing recovery path instead of fabricating annotation state.

The surface requires global `[annotations].enabled`, versioned messages, and an
app/channel policy with annotations enabled. Protocol V1 receives neither the
raw annotation event nor the V2/Ably summary projection.

### Realtime delivery bounds [#realtime-delivery-bounds]

Compatibility delivery is injected at the native local delivery boundary. It
is interest-gated, synchronous, and non-blocking when no Ably session is
attached to a channel. Active sessions use separate bounded control and data
queues, each limited by both frame count and bytes. The data queue honors the
server's `[websocket].max_messages` and `[websocket].max_bytes` bounds; unset
limits retain bounded compatibility defaults. ACK, ERROR, heartbeat, and
close-control frames are prioritized over data frames. A data overflow marks
continuity lost, stops further data for that session, and sends a recoverable
`90003` error when possible; the next attach/recovery reads canonical Sockudo
hot replay or durable history.

Messages are encoded once for each active JSON/MessagePack group and shared as
immutable bytes across that group's subscribers. Session, token, channel, and
queue state have hard bounds and periodic expiry. No compatibility lock is held
while encoding or fanning out, and the compatibility projection does not alter
Protocol V1 delivery. `/operator/stats/ably-runtime` exposes `data_encoded` for
these shared projections separately from `encoded`, which also includes
connection, channel, ACK, and heartbeat control frames.

For process-local deployments, compatibility coordination state is kept in the
runtime rather than the general bounded memory cache. Redis and Redis Cluster
remain the shared coordination authority for multi-node deployments; the
configured cache still persists compatibility statistics in both deployment
modes.

In Redis-backed deployments, connection recovery keys are stored for the
advertised connection-state TTL in the shared cache as well as the local
runtime. A reconnect routed to another node can therefore validate the key and
recover each reattached channel from canonical hot replay or durable history; a
missing or expired `recover` key returns `80018`. The active key lease is
refreshed at half the advertised TTL, then receives a full TTL from an
ungraceful disconnect; an explicit `CLOSE`
invalidates the active key after sending `CLOSED`. Cached state never includes
app secrets or provider credentials.

Realtime remains WebSocket-only. Operator listeners configured with
`[ably_compat].realtime_admission = "placement_constraint"` return a genuine
`DISCONNECTED`/`50320` placement response, allowing the SDK to reconnect to a
configured fallback host over WebSocket. This admission mode does not add
Comet, XHR, SSE, polling, or streaming transports. Realtime messages without an
explicit ID derive the stable Ably ID `connectionId:msgSerial:index`; retrying
an unacknowledged frame therefore reuses the native idempotency path instead of
duplicating persistence or fanout.

ATTACH processing is bounded by `ably_compat.attach_timeout_ms` across native
presence snapshot, hot recovery, and durable history dependencies. A timeout
cancels that work, removes the partially registered subscriber, and returns a
channel-scoped `DETACHED`/`50003`, allowing the SDK's normal channel retry timer
to run without leaving a delivery route active before `ATTACHED`.

Token expiry and revocation checks are generation-stamped. An `AUTH` update
atomically replaces credential identity, client identity, capability, expiry,
and revocation metadata, so an older timer or delayed revocation cannot close a
renewed session. Revocations are stored in the configured shared cache and are
visible to compatibility sessions on every node. The targets `clientId`,
`revocationKey`, and `channel`, `issuedBefore`, and the delayed
`allowReauthMargin` flow are supported. Tokens, JWTs, key secrets, and signing
material are not written to compatibility logs.

### Statistics and push [#statistics-and-push]

Authenticated `GET /stats` supports minute, hour, day, and month units;
inclusive interval-ID or epoch-millisecond `start` and `end` bounds;
forward/backward direction; policy-capped limits; opaque, query-scoped cursors;
and credential-free `first`/`next` Link relations. Canonical UTC minute buckets
are merged through the configured cache using compare-and-swap, and larger units
are deterministic read-time projections. A shared persistent cache such as
Redis provides cross-node and restart-safe history; the memory backend remains
node-local.

Live counters cover inbound and delivered outbound message counts and encoded
bytes, API and token-request outcomes, connections, channels, presence, and push
admission. A bounded batching worker persists publish, attach, and connection
observations before their application acknowledgements; nonblocking outbound
delivery observations report drops on saturation. The worker's backlog,
capacity, drops, flush failures, completed batches, observation count, and lag
are available from `GET /operator/stats/aggregation`. Authenticated conformance
interval ingestion is separately disabled by default and, when enabled, writes
through the same typed minute store rather than returning canned fixture data.

The Ably push projection is compiled only with `ably-compat`, which also enables
Sockudo's native `push` feature. Device registrations and channel subscriptions
use the configured native `DynPushStore`. `/push/publish` and ordinary publishes
with `extras.push` create native `PublishIntent` records through `PushPipeline`;
they do not publish directly from the REST handler. Planning, queue admission,
provider batching, retry, feedback, aggregate status, repair, and retention use
the same workers and durable records as Sockudo's native push API.

The Ably-channel test transport is the native realtime provider. Its dispatch
worker publishes `__ably_push__` through `MessageService` and reports an accepted
provider outcome only after real realtime fanout succeeds. Retries reuse a
bounded delivery id so duplicate provider attempts remain idempotent. Client-ID
channel subscriptions are represented by typed client-scoped native
subscriptions and resolve the client's current durable device set at planning
time. `push-admin` and `push-subscribe` capabilities remain channel scoped;
device-scoped operations verify the hashed device identity token, and updates do
not rotate that token. Recipient scans are paged, and provider credentials,
device tokens, private keys, and raw payloads are not logged.

For an in-process deployment, build with `monolith` so planner, retry, feedback,
scheduler, repair, and cleanup workers run beside the compatibility realtime
provider worker. External FCM, APNs, Web Push, HMS, and WNS outcomes are never
synthesized: delivery requires the corresponding native provider feature,
worker, and real credentials.

The standard Sockudo Docker image includes `monolith`. Custom images that
override the `SOCKUDO_FEATURES` build argument must include it when Ably push
compatibility uses a broker-backed `push.queue_driver`.

## Additional compatibility surfaces [#additional-compatibility-surfaces]

| Surface | Status | Scope |
| ------- | ------ | ----- |

\| History and recovery | Supported for tested Pub/Sub subset | The harness covers forward/backward REST pagination, `untilAttach`, history projection, and recovery of missed append fragments. Broader Ably history API parity is not claimed. |
\| Presence enter/update/leave | Supported for tested Pub/Sub subset | Realtime presence is ACKed, server-stamped, synchronized, and fanned out; REST current presence and durable presence history share the same base-channel authority. Full Ably presence API parity is not claimed. |
\| Message annotations | Supported for the pinned annotation suite | Realtime create/delete and raw subscription, summary projection, REST pagination, and JSON/MsgPack all use the native annotation subsystem. |
\| Keys and capabilities | Supported for tested Pub/Sub subset | `App.key`/`App.secret` is the primary credential; opt-in `[ably_compat]` keys share its app state. Key and token capabilities are enforced for REST publish/history/status/message reads and realtime attach/publish/presence operations. |
\| Token lifecycle | Supported for tested WebSocket subset | Canonical `access_token` and `client_id` connection parameters are accepted with documented camel-case aliases. Signed HS256 Ably JWTs and issued opaque tokens carry identity, capability, and expiry into the connection. In-band `AUTH` renews authorization without changing the connection ID; expiry and revocation emit `DISCONNECTED` 40142 and 40141. Capability downgrades fail affected channels only. |
\| LiveObjects/object modes | Intentionally out of scope | The current compatibility target does not implement Ably LiveObjects. |
\| Ably Push | Supported for the pinned REST subset | Device registration, channel subscription, listing/deletion, direct publish, and `extras.push` use native durable storage, queue, planner, provider dispatch, feedback, retry, status, scheduler, and cleanup paths. Full provider-platform parity is not claimed. |
\| Binary MsgPack Ably protocol | Supported for the tested surface | Realtime Pub/Sub and REST publish/history/time are selected from pinned `ably@2.21.0` with `useBinaryProtocol: true`. Full Ably platform MsgPack parity is not claimed. |
\| Full Ably platform API parity | Intentionally out of scope | This surface is for AI Transport compatibility, not a replacement for all Ably services. |

## Test Commands [#test-commands]

Start Sockudo locally with the Ably compatibility feature, AI Transport, durable
history, and versioned messages enabled. The default repository config uses
`app-key:app-secret` and an AI channel prefix of `private-ai-`.

```bash
cargo run -p sockudo --features "v2,ai-transport,ably-compat,redis,postgres,push,monolith" -- \
  --config config/config.toml
```

Then run the compatibility lanes from the pinned harness:

```bash
cd sockudo-compatibility
make conformance
make strict-completeness
make browser-install
make browser-conformance
make browser-strict
make ait-conformance
```

`make browser-matrix` runs Chromium first, then the pinned Firefox and WebKit
revisions. Browser reports fail on console or page errors and leaked contexts in
addition to assertion, runner, pending, and accounting failures. Browser support
services are loopback-only and real; the runner blocks requests to Ably-hosted
infrastructure.

The targets read these environment variables:

| Variable                    | Default                                |
| --------------------------- | -------------------------------------- |
| `ABLY_KEY`                  | `app-key:app-secret`                   |
| `ABLY_ENDPOINT`             | `127.0.0.1`                            |
| `ABLY_PORT`                 | `6001`                                 |
| `ABLY_TLS`                  | `false`                                |
| `ABLY_CLIENT_ID`            | script-specific                        |
| `ABLY_AGENT_CLIENT_ID`      | `sockudo-ably-ait-agent`               |
| `ABLY_CHANNEL`              | generated per run                      |
| `ABLY_DEMO_PROMPT`          | `Say hello from Sockudo AI Transport.` |
| `ABLY_BROWSER_ORIGIN_PORTS` | `4173,3001,5174`                       |

## Demos [#demos]

`make ably-ai-demo` runs two headless demos:

1. A stock `@ably/ai-transport@0.4.0` chat demo using
   `@ably/ai-transport/vercel`, pinned `ably@2.21.0`, and Sockudo as the
   configured endpoint.
2. A recovery/history demo that disconnects during an AI output stream, appends
   additional fragments while offline, reconnects with an Ably recovery key, and
   verifies that only missed fragments replay and the final aggregate is intact.

These AI Transport demos are deliberately Node-based so CI can run them without
a browser.

## Evidence Files [#evidence-files]

The working compatibility evidence lives in repository docs:

* `docs/ably-compat/ABLY_COMPAT_SCORECARD.md`
* `docs/ably-compat/ABLY_COMPAT_GAPS.md`
* `docs/ably-compat/TEST_PLAN.md`

Update those files whenever a supported/deferred status changes.


# Agent Presence (/docs/server/agent-presence)



Agent presence uses normal presence channels plus the V2-only `sockudo:presence_update` frame. It lets an authenticated member update its presence data without leaving and re-entering the channel.

Recommended status convention:

```json
{ "status": "idle" }
{ "status": "thinking" }
{ "status": "streaming" }
```

Send updates from a subscribed presence member:

```json
{
  "event": "sockudo:presence_update",
  "data": {
    "channel": "presence-agent:session-123",
    "data": { "status": "thinking" }
  }
}
```

Limits and access:

* Protocol V2 only.
* The connection must already be a member of the presence channel.
* Capability-token connections need `presence` permission for the channel.
* `data` is capped at 1 KiB.
* Updates are rate limited per app/channel/member by `[presence].update_rate_limit_per_member_per_second`, default `10`.
* V1/Pusher clients do not receive update events.

Late joiners see the latest member data in the presence snapshot returned with `subscription_succeeded`.

`[presence].ungraceful_timeout_seconds` controls Protocol V1 abnormal disconnect
grace and defaults to `0`, preserving legacy immediate `member_removed` timing.
Protocol V2 uses `[presence].v2_ungraceful_timeout_seconds`, which defaults to
`15` seconds so transient transport loss does not produce a leave/enter flap.
Clean unsubscribe and clean WebSocket close still remove immediately.

Presence update ordering is independent from channel message ordering. Treat presence status as latest-known member state, not as a causal marker for `ai-output` or mutable-message delivery.


# Authentication for AI Transport (/docs/server/ai-transport-authentication)



AI Transport uses three layers of authority:

| Layer                         | Used for                                                                | Authority                               |
| ----------------------------- | ----------------------------------------------------------------------- | --------------------------------------- |
| App-key HTTP auth             | Server publishes, mutations, history reads, push management, revocation | Trusted backend with app secret         |
| Private/presence channel auth | Subscription admission                                                  | Customer backend signs channel auth     |
| Protocol V2 capability token  | Client connection identity and channel-scoped capabilities              | Customer backend issues short-lived JWT |

Never trust `client_id` from message bodies or AI headers. Verified identity comes from signed-in
socket state, capability-token claims, or trusted server app-key context.

## Capability token claims [#capability-token-claims]

Capability tokens are HS256 JWTs signed with the app secret. Header `kid` must equal the app key.
Claims are:

```json
{
  "x-sockudo-client-id": "user-42",
  "x-sockudo-capability": {
    "private-ai:user-42:*": [
      "subscribe",
      "publish",
      "history",
      "message_append_own"
    ]
  },
  "iat": 1764835200,
  "nbf": 1764835200,
  "exp": 1764838800,
  "jti": "tok_01J..."
}
```

`x-sockudo-capability` accepts either the JSON object shown above or its stringified form. Each
object key is a channel pattern and its value is the list of operations granted on matching
channels.

Limits from code: token at most 8 KiB, `client_id` at most 128 bytes, `jti` at most 128 bytes,
lifetime at most 24 hours, and 30 seconds clock skew.

## Node example [#node-example]

```js
import crypto from "node:crypto";

function base64url(value) {
  return Buffer.from(JSON.stringify(value)).toString("base64url");
}

export function issueSockudoToken({ appKey, appSecret, clientId, capabilities, now = Math.floor(Date.now() / 1000) }) {
  const header = { alg: "HS256", typ: "JWT", kid: appKey };
  const payload = {
    "x-sockudo-client-id": clientId,
    "x-sockudo-capability": JSON.stringify(capabilities),
    iat: now,
    nbf: now,
    exp: now + 3600,
    jti: crypto.randomUUID()
  };
  const signingInput = `${base64url(header)}.${base64url(payload)}`;
  const signature = crypto.createHmac("sha256", appSecret).update(signingInput).digest("base64url");
  return `${signingInput}.${signature}`;
}
```

## Rust example [#rust-example]

```rust
use jsonwebtoken::{Algorithm, EncodingKey, Header, encode};
use serde::Serialize;
use std::collections::BTreeMap;

#[derive(Serialize)]
struct Claims {
    #[serde(rename = "x-sockudo-client-id")]
    client_id: String,
    #[serde(rename = "x-sockudo-capability")]
    capability: BTreeMap<String, Vec<String>>,
    iat: i64,
    nbf: i64,
    exp: i64,
    jti: String,
}

fn issue_sockudo_token(
    app_key: &str,
    app_secret: &str,
    client_id: &str,
    capabilities: BTreeMap<String, Vec<String>>,
    now: i64,
    jti: String,
) -> jsonwebtoken::errors::Result<String> {
    let mut header = Header::new(Algorithm::HS256);
    header.kid = Some(app_key.to_owned());
    encode(
        &header,
        &Claims {
            client_id: client_id.to_owned(),
            capability: capabilities,
            iat: now,
            nbf: now,
            exp: now + 3600,
            jti,
        },
        &EncodingKey::from_secret(app_secret.as_bytes()),
    )
}
```

## Refresh and revocation [#refresh-and-revocation]

Refresh a V2 connection with `sockudo:auth`:

```json
{
  "event": "sockudo:auth",
  "data": { "token": "<new-token>" }
}
```

Refresh cannot change `client_id`. To revoke tokens, call signed HTTP:

```http
POST /apps/{appId}/revocations
```

with `jti`, `client_id`, or both. Matching local sockets receive token expiry/revocation handling
and close with the configured grace path.

## Capability patterns [#capability-patterns]

Patterns accept exact channel names, `*`, and wildcard matching as implemented by
`ConnectionCapabilities`. Keep user tokens narrow:

```json
{
  "private-ai:user-42:*": [
    "subscribe",
    "publish",
    "history",
    "message_append_own",
    "message_update_own",
    "annotation-subscribe"
  ],
  "presence-ai:user-42:*": ["subscribe", "presence"]
}
```

Annotation publish, subscribe, and delete operations remain independent from ordinary
`publish`/`subscribe`. Message mutation operations accept their documented underscore spelling
and equivalent hyphenated spelling. Push capability names accept both spellings as well.

Agent/server workers should use trusted app-key HTTP or a deliberately scoped server-issued token.


# AI Transport conventions (/docs/server/ai-transport-conventions)



AI Transport support is additive and default-off. Enable it with the `ai-transport` Cargo feature and runtime `[ai_transport]` config, then scope validation to channel prefixes:

```toml
[ai_transport]
enabled = true

[[ai_transport.channels]]
prefix = "private-ai-"
```

AI lifecycle events are plain channel events: `ai-input`, `ai-output`, `ai-run-start`, `ai-run-suspend`, `ai-run-resume`, `ai-run-end`, and `ai-cancel`. Server-side validation applies only on enabled AI channels. WebSocket clients may publish `ai-input` and `ai-cancel` when their token allows `publish`; agent events require trusted server-side HTTP publish until an explicit agent token marker is specified.

AI metadata is carried in the existing V2 `extras` envelope:

```json
{
  "extras": {
    "ai": {
      "transport": {
        "run-id": "run-1",
        "status": "streaming",
        "role": "assistant"
      },
      "codec": {
        "content-type": "application/json"
      }
    }
  }
}
```

Each `transport` and `codec` tier is limited to 32 string keys. Keys must be at most 64 bytes. Transport keys match `[a-z0-9-]+`; codec keys match `[A-Za-z0-9_-]+` for Ably/Vercel codec fields. Transport values and ordinary codec values must be at most 256 bytes; `extras.ai.codec.providerMetadata` may be up to 8 KiB because Vercel provider metadata is JSON-serialized there. Transport keys are validated against the wire protocol registry. Codec keys are opaque but still bounded. Ably-native regenerate publishes use `extras.ai.transport.msg-regenerate` as the codec-message-id of the assistant message being regenerated. Any `*-client-id` transport value on an untrusted WebSocket publish must match the verified signed-in client identity. Trusted app-key publishes may preserve empty `run-client-id` and `step-client-id` values as unknown-owner sentinels; derived identities treat those empty values as absent.

Existing `idempotency_key` behavior is unchanged. AI SDKs may additionally send `message_id` on HTTP publish for idempotent creates, and mutation requests may send `op_id` for no-op replay of append/update/delete operations. V2 publish and mutation acknowledgements expose `message_serial`, `history_serial`, `delivery_serial`, and `version_serial` when those serials are assigned.

## Deployment matrix [#deployment-matrix]

AI Transport requires durable history and versioned messages. Startup rejects `ai_transport.enabled = true` unless both `[history].enabled` and `[versioned_messages].enabled` are true.

| Adapter                                                                                  | Cache                  | History backend                                     | Version store                                       | Support                                   |
| ---------------------------------------------------------------------------------------- | ---------------------- | --------------------------------------------------- | --------------------------------------------------- | ----------------------------------------- |
| `local`                                                                                  | `memory`               | `memory`                                            | `memory`                                            | Single-node development only              |
| `local`                                                                                  | Redis or memory        | PostgreSQL, MySQL, DynamoDB, ScyllaDB, or SurrealDB | PostgreSQL, MySQL, DynamoDB, ScyllaDB, or SurrealDB | Single-node or externally isolated worker |
| `redis`, `redis-cluster`, `nats`, `pulsar`, `rabbitmq`, `google-pubsub`, `kafka`, `iggy` | Redis or Redis Cluster | PostgreSQL, MySQL, DynamoDB, ScyllaDB, or SurrealDB | PostgreSQL, MySQL, DynamoDB, ScyllaDB, or SurrealDB | Supported horizontal deployment           |
| any horizontal adapter                                                                   | `memory` or `none`     | any                                                 | any                                                 | Rejected at startup                       |
| any horizontal adapter                                                                   | any                    | `memory`                                            | any                                                 | Rejected at startup                       |
| any horizontal adapter                                                                   | any                    | any                                                 | `memory`                                            | Rejected at startup                       |

Under AI Transport, versioned-message delivery serials are leased in 128-position blocks per node and channel. The backing store still owns monotonic reservation authority; the lease cache reduces append-stream reservation contention without changing stored serials or recovery semantics.

Append rollup is egress-only and defaults to a 40 ms window. Streaming messages are tracked in the shared cache while their `extras.ai.transport.status` is `streaming`. Any node can claim a stale stream after `ai_transport.rollup.orphan_ttl_ms`, which defaults to 60000 ms, then append a normal versioned `message.update` with `status=cancelled` and `error-code=orphan_timeout`. The claim key prevents duplicate cancellation across nodes, and the latest version is re-read before mutation so active streams that advanced meanwhile are refreshed instead of cancelled.

Apps can subscribe to the webhook event type `ai_stream_orphaned` to observe synthetic orphan cancellation. The event payload includes `channel`, `message_serial`, and `reason`.

## Scale-out verification [#scale-out-verification]

The repository includes a three-node Redis/PostgreSQL AI Transport stack and verification scripts:

```bash
docker compose -f docker-compose.ai-transport.yml up --build -d
node scripts/ai-transport-3node-bench.mjs
scripts/ai-transport-jepsen-lite.sh
```

`scripts/ai-transport-3node-bench.mjs` runs a same-stack baseline plus a cross-node workload, verifies every node reads the same final aggregate, and enforces the cross-node added append p99 budget. `scripts/ai-transport-jepsen-lite.sh` runs the same correctness check before, during, and after a docker `pause`/`unpause` partition of one node; steady state uses `MAX_APPEND_ADDED_P99_MS` and partition/heal phases use `PARTITION_MAX_APPEND_ADDED_P99_MS`.


# Limits and quotas (/docs/server/ai-transport-limits-quotas)



This is the canonical server-side limit table for AI Transport-adjacent features.

| Area                       | Key or constant                                        | Default                       | Notes                                             |
| -------------------------- | ------------------------------------------------------ | ----------------------------- | ------------------------------------------------- |
| HTTP API                   | `http_api.request_limit_in_mb`                         | `100`                         | Whole request cap.                                |
| Event payload              | `event_limits.max_payload_in_kb`                       | `100`                         | App policy can lower effective limit.             |
| Event batch                | `event_limits.max_batch_size`                          | `10`                          | Applies to batch publish.                         |
| Version history page       | `versioned_messages.max_page_size`                     | `100`                         | Server caps version pages.                        |
| Version retention          | `versioned_messages.retention_window_seconds`          | `0`                           | `0` means no expiry.                              |
| Version purge batch        | `versioned_messages.purge_batch_size`                  | `1000`                        | SQL/memory purge worker bound.                    |
| History page               | `history.max_page_size`                                | `100`                         | Client history may be further capped at `1000`.   |
| History retention          | `history.retention_window_seconds`                     | `86400`                       | Per-app/namespace policy can override.            |
| History writer queue       | `history.writer_queue_capacity`                        | `4096`                        | Bounded writer queue.                             |
| Presence history page      | `presence_history.max_page_size`                       | `100`                         | Presence transition history, not current members. |
| Presence history retention | `presence_history.retention_window_seconds`            | `86400`                       | Durable or memory retention.                      |
| AI accumulated content     | `ai_transport.max_accumulated_message_bytes`           | `1048576`                     | Latest aggregated mutable message content cap.    |
| AI appends per message     | `ai_transport.max_appends_per_message`                 | `4096`                        | Per logical stream.                               |
| AI open streams/channel    | `ai_transport.max_open_streaming_messages_per_channel` | `1024`                        | Active streaming messages on one channel.         |
| Rollup windows             | `append_rollup_window`                                 | `0`, `20`, `40`, `100`, `500` | V2 query values accepted by validation.           |
| Rollup default             | `ai_transport.rollup.default_window_ms`                | `40`                          | Node-local egress coalescing window.              |
| Rollup orphan TTL          | `ai_transport.rollup.orphan_ttl_ms`                    | `60000`                       | Stale stream claim/cancel threshold.              |
| Capability token size      | constant                                               | `8192` bytes                  | HS256 JWT only.                                   |
| Capability token lifetime  | constant                                               | `24h`                         | `iat` to `exp`.                                   |
| Capability `client_id`     | constant                                               | `128` bytes                   | Required.                                         |
| Capability `jti`           | constant                                               | `128` bytes                   | Required.                                         |
| Push fanout fast path      | `push.fanout_fast_threshold`                           | `10000`                       | Planning threshold.                               |
| Push shard size            | `push.fanout_shard_size`                               | `100000`                      | Channel fanout shard size.                        |
| Push acceptance quota      | `push.default_quotas.acceptance_rps`                   | `100`                         | Per-app default quota.                            |
| Push inflight quota        | `push.default_quotas.inflight_max`                     | `1000`                        | Per-app default.                                  |
| Push status retention      | `push.publish_status_ttl_days`                         | `30`                          | Support/audit retention.                          |
| Push rule event filter     | `push_rules.event_filter`                              | max `32` names                | Each event name at most 200 bytes.                |

The machine-checked default block is in [Configuration reference](/docs/reference/configuration).


# AI Transport overview (/docs/server/ai-transport-overview)



AI Transport is a Protocol V2 convention layer. It does not replace Sockudo's realtime protocol or
create AI-only storage. A session is a channel, every run is normal channel traffic, and the
server supplies durable, ordered, recoverable primitives that SDKs reduce into UI state.

Sockudo AI Transport is heavily inspired by [Ably AI Transport](https://ably.com/docs/ai-transport).
Ably's public AI Transport docs and SDK were the main product inspiration; Sockudo adapts that
model to Protocol V2, versioned messages, durable history, presence, push, and the local
`@sockudo/client` SDK.

Sockudo AI Transport is community-built and community-maintained. It is not an Ably product and is
not supported by Ably.

Enable it with the `ai-transport` Cargo feature and runtime config:

```toml
[ai_transport]
enabled = true

[[ai_transport.channels]]
prefix = "private-ai-"
```

## End-to-end model [#end-to-end-model]

<div className="ai-model-diagram not-prose" role="img" aria-label="AI Transport end-to-end model from clients and providers through a Sockudo session channel into durable stores and fanout surfaces.">
  <div className="ai-model-grid">
    <div className="ai-model-column">
      <span className="ai-diagram-label">
        Run sources
      </span>

      <div className="ai-model-node">
        <strong>
          User device
        </strong>

        <small>
          {"Creates ai-input and receives live output."}
        </small>
      </div>

      <div className="ai-model-node">
        <strong>
          Agent worker
        </strong>

        <small>
          {"Owns ai-output create, append, update, and finish."}
        </small>
      </div>

      <div className="ai-model-node">
        <strong>
          Model provider
        </strong>

        <small>
          {"Streams tokens, tool calls, and reasoning to the worker."}
        </small>
      </div>
    </div>

    <div className="ai-diagram-rail" aria-hidden="true">
      <span>
        ai-input
      </span>

      <span>
        ai-output
      </span>

      <span>
        provider chunks
      </span>
    </div>

    <div className="ai-model-channel">
      <span>
        Source of truth
      </span>

      <strong>
        AI session channel
      </strong>

      <p>
        {
                  "A normal Protocol V2 channel that carries every run event, message mutation, presence update, and recovery cursor."
                }
      </p>
    </div>

    <div className="ai-diagram-rail" aria-hidden="true">
      <span>
        persist
      </span>

      <span>
        version
      </span>

      <span>
        fanout
      </span>
    </div>

    <div className="ai-model-column">
      <span className="ai-diagram-label">
        Durable outcomes
      </span>

      <div className="ai-model-node ai-model-node-store">
        <strong>
          History store
        </strong>

        <small>
          {"Replay, rewind, and recovery after refresh."}
        </small>
      </div>

      <div className="ai-model-node ai-model-node-store">
        <strong>
          Version store
        </strong>

        <small>
          {"Append/update/delete state and branch history."}
        </small>
      </div>

      <div className="ai-model-node">
        <strong>
          Other devices
        </strong>

        <small>
          {"Presence-aware live fanout and transcript sync."}
        </small>
      </div>

      <div className="ai-model-node">
        <strong>
          Push notifications
        </strong>

        <small>
          {"Completion and background delivery workflows."}
        </small>
      </div>
    </div>
  </div>
</div>

The durable channel is the source of truth. HTTP streaming can start a run, but the answer is no
longer tied to that HTTP response. A browser refresh, mobile handoff, tab close, or temporary
network loss can recover from the same channel history.

## The nine primitives [#the-nine-primitives]

| Primitive     | Sockudo subsystem                                                                                                                   |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| Session       | One channel, usually private or presence, for a user/agent conversation.                                                            |
| Input         | `ai-input` channel event from a verified user-capable connection or trusted server.                                                 |
| Output        | `ai-output` mutable message created and appended by the agent/server side.                                                          |
| Run lifecycle | `ai-run-start`, `ai-run-suspend`, `ai-run-resume`, `ai-run-end`, and `ai-cancel` events with bounded `extras.ai.transport` headers. |
| Mutable state | Existing versioned messages: create, update, delete, append, summary.                                                               |
| Replay        | Durable history, rewind, `until_attach`, and hot-to-durable recovery.                                                               |
| Identity      | V2 capability tokens, signed channel auth, and server app-key HTTP auth.                                                            |
| Agent state   | Presence channels plus V2 `sockudo:presence_update`.                                                                                |
| Notifications | Existing push device/channel-subscription pipeline and optional `[[push_rules]]`.                                                   |

## Run lifecycle [#run-lifecycle]

<div className="turn-lifecycle-diagram not-prose" role="img" aria-label="AI Transport run lifecycle from client input through Sockudo persistence, agent output, append rollup, and final history state.">
  <div className="turn-lifecycle-actors" aria-hidden="true">
    <span>
      Client
    </span>

    <span>
      Sockudo
    </span>

    <span>
      Agent worker
    </span>

    <span>
      History + versions
    </span>
  </div>

  <ol className="turn-lifecycle-steps">
    <li>
      <span>
        01
      </span>

      <div>
        <strong>
          Input creates the run
        </strong>

        <p>
          {
                      "The client publishes ai-input create; Sockudo persists the input before agent work starts."
                    }
        </p>
      </div>

      <code>
        Client -> Sockudo -> Store
      </code>
    </li>

    <li>
      <span>
        02
      </span>

      <div>
        <strong>
          Agent opens output state
        </strong>

        <p>
          {
                      "The worker emits ai-run-start and creates the ai-output mutable message for this run."
                    }
        </p>
      </div>

      <code>
        Agent -> Sockudo
      </code>
    </li>

    <li className="turn-lifecycle-stream">
      <span>
        03
      </span>

      <div>
        <strong>
          Provider stream becomes append operations
        </strong>

        <p>
          {
                      "Every token, tool delta, reasoning chunk, or metadata update is encoded as message.append."
                    }
        </p>
      </div>

      <code>
        append loop
      </code>
    </li>

    <li>
      <span>
        04
      </span>

      <div>
        <strong>
          Live clients receive rolled-up fanout
        </strong>

        <p>
          {
                      "Sockudo can compact append bursts for WebSocket egress while preserving each original mutation internally."
                    }
        </p>
      </div>

      <code>
        Sockudo -> Client
      </code>
    </li>

    <li>
      <span>
        05
      </span>

      <div>
        <strong>
          Run closes into durable history
        </strong>

        <p>
          {
                      "The agent emits ai-run-end complete; the final aggregate is visible to history, rewind, and recovery."
                    }
        </p>
      </div>

      <code>
        Agent -> Sockudo -> Store
      </code>
    </li>
  </ol>
</div>

`ai-output` is a normal mutable message. Streaming text, reasoning, tool input, tool output, and
source metadata are encoded as append/update operations by the SDK codec. Late joiners read the
aggregated visible state, while conformance, history, webhooks, and push can still observe the
original operation log.

## What already exists [#what-already-exists]

These features are not new AI-only subsystems:

* `MessageAction` and `VersionStore` for mutable messages.
* `HistoryStore`, rewind, and two-tier recovery.
* Presence and presence-history.
* Annotation publish/delete/summary projection.
* Push providers, device registrations, channel subscriptions, publish status, queues, retries,
  feedback, and scheduler.
* Protocol V2 capability tokens and revocation.
* V1 isolation: V1 clients keep Pusher-compatible output.

## What `ai-transport` adds [#what-ai-transport-adds]

The feature adds validation and operational conventions:

* well-known AI event names
* bounded `extras.ai.transport` and `extras.ai.codec`
* anti-spoof checks for `*-client-id` headers
* stream lifecycle tracking and active-stream limits
* append rollup as WebSocket egress optimization
* orphan stream cancellation
* conformance, benchmark, load, and chaos suites

The formal wire reference is [AI Transport wire protocol](/specs/ai-transport-wire-protocol).
The provider integration guide is [AI Transport providers](/docs/server/ai-transport-providers).

## AI SDK agents and durability [#ai-sdk-agents-and-durability]

Sockudo durability and AI SDK agent durability solve different recovery problems. Use both when you
run long-lived agents.

| Layer                                 | Owns                                                                                              | Recovers from                                                                                  |
| ------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Sockudo AI Transport                  | Published user inputs, assistant output chunks, approval responses, run lifecycle, and cursors.   | Browser refresh, mobile handoff, reconnect, late join, transcript replay, and audit workflows. |
| AI SDK `WorkflowAgent` or agent store | Pending agent steps, sleeps, retries, tool execution state, model-local context, and checkpoints. | Worker crash, deployment restart, long-running wait states, and agent workflow resume.         |
| Model provider runtime                | Provider-native realtime sessions, reasoning controls, uploaded files, and provider tool context. | Provider-specific retries, session resume, or media reconnect semantics.                       |

Sockudo history is intentionally not a workflow checkpoint store. Persist the AI SDK workflow run id
or agent checkpoint id in run metadata, then correlate it with `channelName`, `runId`, and
`invocationId`. When a workflow resumes, it can reload its own checkpoint and continue appending to
the same Sockudo session channel; clients recover the visible transcript from Sockudo.

## Minimal server route [#minimal-server-route]

```ts
import { streamText, toUIMessageStream } from "ai";
import { createAgentSession } from "@sockudo/ai-transport/vercel";

export async function runAgent({ body, sockudo }) {
  const session = createAgentSession({
    client: sockudo,
    channelName: body.channelName,
  });

  const run = session.createRun({
    runId: body.runId,
    invocationId: body.invocationId,
    inputEventId: body.inputEventId,
    clientId: body.clientId,
  });

  await run.start();
  const result = streamText({
    model: "openai/gpt-5-mini",
    instructions: "You are a concise incident-room assistant.",
    prompt: body.prompt,
    abortSignal: run.abortSignal,
  });
  await run.streamResponse(toUIMessageStream({ stream: result.stream }));
  await run.end("complete");
  session.close();
}
```

## Minimal client [#minimal-client]

```ts
import Sockudo from "@sockudo/client";
import { useChat } from "@ai-sdk/vue";
import { provideChatTransport } from "@sockudo/ai-transport/vercel/vue";

const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  protocolVersion: 2,
  channelAuthorization: { endpoint: "/api/sockudo/channel-auth" },
});

const channelName = "private-ai:user-42:sess-01J";
const provider = provideChatTransport({
  api: "/api/chat",
  channelName,
  client,
  clientId: "user-42",
});

const { sendMessage } = useChat({
  id: channelName,
  transport: provider.chatTransport.value,
});

await sendMessage({ text: "Investigate the last failed deploy." });
```

## Session channel model [#session-channel-model]

Use channel names that encode tenant/user/session ownership in your backend, for example:

```text
private-ai:tenant-7:user-42:sess-01J...
presence-ai:tenant-7:sess-01J...
```

Clients should never assert `client_id` or tenant identity in message bodies. Derive channel access
from signed auth or capability-token claims, then scope `subscribe`, `publish`, `history`,
`message_append_own`, and related capabilities to the exact channel or a narrow wildcard.

## Compatibility [#compatibility]

AI Transport is V2-only. V1 connections can still coexist on the same server, but they do not see
V2 extras, mutable-message fields, recovery frames, annotations, or AI validation semantics.


# Production checklist (/docs/server/ai-transport-production-checklist)



Use this checklist before enabling AI Transport for production traffic.

## Build and config [#build-and-config]

* Build with `v2`, `ai-transport`, the selected shared adapter/cache/store features, and `push`
  when push recipes are used.
* Enable `[history]`, `[versioned_messages]`, and `[ai_transport]`.
* Use shared history, version store, and cache for horizontal deployments.
* Scope `[[ai_transport.channels]]` to narrow prefixes.
* Set history and version retention to cover reconnect, rewind, and support windows.

## Auth [#auth]

* Issue short-lived V2 capability tokens from a backend only.
* Keep capability patterns exact or tenant/user scoped.
* Use app-key HTTP only from trusted server workers.
* Revoke by `jti` or `client_id` for incident response.
* Do not accept client-asserted identity fields.

## Operations [#operations]

* Scrape Prometheus metrics from every node.
* Alert on history degraded/reset-required channels, recovery failures, rollup flush latency, active
  stream leaks, horizontal transport drops, and push queue/provider failures.
* Run the AIT-S conformance suite before release.
* Run `scripts/ai-transport-bench-guard.sh` and review budgets.
* Run `scripts/ai-transport-ga-gate.sh ci-evidence` on every release-candidate branch.
* Run `scripts/ai-transport-ga-gate.sh release-evidence` before any GA tag; it fails until the
  external S14 scale/chaos, rolling-upgrade, and full SDK compatibility manifests are committed.
* Generate release evidence with `scripts/ai-transport-s14-release-evidence.sh`,
  `scripts/ai-transport-rolling-upgrade-redis.mjs --execute`, and
  `scripts/sdk-compat-full-matrix.mjs --execute`.
* Execute `test/load` and `tools/chaos` profiles on production-like hardware before headline claims.
* Build Docker images both with and without `ai-transport` enabled.

## Client behavior [#client-behavior]

* Treat Sockudo history as authoritative after reconnect or notification open.
* Use `until_attach` for late-join history to avoid gaps.
* Reduce mutable-message events by `message_serial` and `version.serial`.
* Send terminal stream status for completed/cancelled/error runs.
* Refresh tokens before expiry and back off on auth errors.

## GA readiness [#ga-readiness]

The canonical readiness record is
[`docs/specs/ai-transport-ga-readiness.md`](/specs/ai-transport-ga-readiness). Server releases ship
AI Transport defaults off before SDK releases that depend on the feature. Full product parity still
depends on the SDK plans in `plans/ai-transport/02-sdk-prompts.md` and the E4/E5 enablement work in
`plans/ai-transport/03-existing-sdks-prompts.md`.


# AI Transport production operations (/docs/server/ai-transport-production-ops)



This runbook covers the S14 scale, soak, and chaos validation path for AI Transport deployments.
Use it with the profiles in `test/load/profiles`, the runner in `test/load/ai-scale-runner.mjs`,
and the chaos harness in `tools/chaos`.

## Capacity Model [#capacity-model]

Start with these formulas, then replace the inputs with measured numbers from your hardware:

| Dimension                  | Formula                                                              |
| -------------------------- | -------------------------------------------------------------------- |
| Connections per node       | `target_connections / node_count`, with 20 percent headroom reserved |
| Active streams per node    | `active_streams / node_count`, with shard imbalance budgeted at p95  |
| Append ingress rate        | `active_streams * tokens_per_second`                                 |
| Fanout egress rate         | `append_ingress_rate * mean_subscribers_per_session`                 |
| Rollup memory ceiling      | `active_streams_per_node * 4 KiB`, excluding payload buffers         |
| History bytes per response | `mean_response_tokens * mean_token_bytes * retention_multiplier`     |
| History store write rate   | `append_ingress_rate + create/update/delete/summary_rate`            |

The headline target is 1M concurrent connections, 50k active streams, 100 tok/s, and one to five
subscriber devices per session for a 30-minute run. The 24-hour soak target is 20 percent of that
load.

## Required Runs [#required-runs]

Run the five-node local smoke first:

```bash
make ai-scale-smoke
```

Generate a production fleet plan:

```bash
node test/load/ai-scale-runner.mjs --profile test/load/profiles/headline-1m.json --plan
```

Execute only on prepared hardware:

```bash
node test/load/ai-scale-runner.mjs \
  --profile test/load/profiles/headline-1m.json \
  --execute \
  --urls "$SOCKUDO_NODE_URLS" \
  --metricsUrls "$SOCKUDO_METRICS_URLS" \
  --output docs/specs/ai-transport-results/headline-1m.json
```

The required pass criteria are zero sampled transcript mismatches, clean serial monotonicity,
p99 append-to-delivery under 25 ms intra-region, flat memory after warm-up, and no file descriptor
or Tokio task leaks.

## Chaos Playbooks [#chaos-playbooks]

Run all local chaos scenarios:

```bash
tools/chaos/ai-chaos-runner.sh all
```

Node kill mid-stream:
confirm post-heal transcript audits pass, `sockudo_ai_active_streams` returns to the expected
level, and janitor cleanup leaves no unbounded active rollup stream growth.

Redis restart or failover:
expect a bounded stall, no versioned-message corruption, and no durable history reset-required
channels. Watch `sockudo_horizontal_transport_reconnections_total`,
`sockudo_horizontal_transport_messages_dropped_total`, `sockudo_history_degraded_channels`, and
`sockudo_history_reset_required_channels`.

Inter-node partition:
verify S8 duplicate/delay semantics, then confirm healed nodes converge through sampled latest
reads and transcript audits.

Slow subscribers:
validate that the existing slow-consumer policy protects hot channels and that
`sockudo_broadcast_latency_ms` recovers after the pressure phase.

Clock skew:
the local script verifies signed client timestamp skew does not affect serial monotonicity. True
node clock mutation must run on privileged production-like infrastructure; serial reservation is
u64 store-backed and must not depend on wall-clock ordering.

## Alert Thresholds [#alert-thresholds]

Use these as starting alerts and tune with measured baselines:

| Signal                                                                | Alert                                                                                       |
| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| `sockudo_connected`                                                   | above planned per-node budget for 5 minutes                                                 |
| `sockudo_rate_limit_triggered_total`                                  | reconnect storm lane does not trigger shaping, or triggers remain elevated after 10 minutes |
| `sockudo_broadcast_latency_ms`                                        | p99 above 25 ms during headline run or above 10 ms during single-node fanout benchmark      |
| `sockudo_ai_active_streams`                                           | does not return to baseline after stream TTL plus janitor window                            |
| `sockudo_appends_received_total` vs `sockudo_appends_delivered_total` | delivered ratio drops outside configured rollup expectations                                |
| `sockudo_flush_latency`                                               | 99.9 percent exceeds rollup window plus 5 ms                                                |
| `sockudo_history_recovery_failures_total`                             | any sustained non-auth failure during reconnect tests                                       |
| `sockudo_history_degraded_channels`                                   | nonzero for more than one scrape interval                                                   |
| `sockudo_history_reset_required_channels`                             | any nonzero value pages                                                                     |
| `sockudo_horizontal_transport_messages_dropped_total`                 | any increase outside explicit partition tests                                               |
| `sockudo_horizontal_transport_queue_depth`                            | sustained growth across three scrape intervals                                              |
| `sockudo_tokio_active_tasks`                                          | positive slope after warm-up in soak                                                        |

## Troubleshooting [#troubleshooting]

| Symptom                             | Likely cause                                                         | Operator action                                                                                    |
| ----------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Mutations rejected as not permitted | App/channel capability lacks the create/update/delete/append action  | Inspect token capability and server app policy; client-supplied identity is not authoritative      |
| History retention too short         | Rewind or recovery request points before retained durable history    | Increase retention count/time or reduce expected rewind window; watch retained message/byte gauges |
| Run never ends                      | Missing terminal append/update/summary or rollup orphan TTL too long | Check `sockudo_ai_active_streams`, terminal event rates, and client run lifecycle                  |
| Suspended-state publishes           | Producer continued after suspend/continuation boundary               | Reject or route through the continuation flow; audit op\_id/message\_id idempotency                |
| Reconnect loop on bad tokens        | Clients retrying auth failures without refreshing credentials        | Rate-limit, inspect auth error codes, and fix token refresh logic                                  |
| Rollup mis-tuning                   | Window too high for latency budget or too low for fanout reduction   | Compare `sockudo_flush_latency` and `sockudo_rollup_ratio`; tune the window per channel class      |


# AI Transport providers (/docs/server/ai-transport-providers)



Sockudo AI Transport is provider-neutral. Sockudo carries ordered, durable run state; your agent
chooses how to call the model and converts provider output into AI SDK UI message chunks.

The JavaScript SDK has two integration families:

| Integration              | Import                            | Best for                                                                                                    |
| ------------------------ | --------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| Vercel AI SDK transport  | `@sockudo/ai-transport/vercel`    | Apps already using `ai`, `streamText`, `useChat`, Vue, React, or Svelte AI SDK UI primitives.               |
| Direct provider adapters | `@sockudo/ai-transport/providers` | Workers that call OpenAI, Anthropic, OpenAI-compatible HTTP/SSE endpoints, or local model servers directly. |
| Core codec transport     | `@sockudo/ai-transport`           | Custom model protocols, custom UI projections, or non-Vercel message shapes.                                |

## Credits [#credits]

Sockudo AI Transport is heavily inspired by [Ably AI Transport](https://ably.com/docs/ai-transport),
and Ably's public AI Transport docs and SDK were the main product inspiration for this layer. The
Sockudo implementation maps those ideas onto Sockudo Protocol V2, durable history, versioned
messages, presence, push, and the `@sockudo/client` SDK in this monorepo.

Sockudo AI Transport is community-built and community-maintained. It is not an Ably product and is
not supported by Ably.

## Provider flow [#provider-flow]

<div className="provider-flow-diagram not-prose" role="img" aria-label="Provider flow showing how the user, Sockudo channel, agent worker, model provider, and durable stores cooperate during an AI run.">
  <div className="provider-flow-lanes" aria-hidden="true">
    <span>
      User client
    </span>

    <span>
      Sockudo channel
    </span>

    <span>
      Agent worker
    </span>

    <span>
      Model provider
    </span>

    <span>
      Stores
    </span>
  </div>

  <ol className="provider-flow-steps">
    <li>
      <span>
        01
      </span>

      <div>
        <strong>
          User starts a run
        </strong>

        <p>
          {
                      "The client publishes ai-input create into the private AI session channel."
                    }
        </p>
      </div>

      <code>
        User -> Sockudo
      </code>
    </li>

    <li>
      <span>
        02
      </span>

      <div>
        <strong>
          Sockudo persists intent
        </strong>

        <p>
          {
                      "The channel records the user message and run start before provider work begins."
                    }
        </p>
      </div>

      <code>
        Sockudo -> stores
      </code>
    </li>

    <li>
      <span>
        03
      </span>

      <div>
        <strong>
          Agent streams provider output
        </strong>

        <p>
          {
                      "The worker calls the selected model provider and receives deltas, tool calls, and finish signals."
                    }
        </p>
      </div>

      <code>
        Agent <-> provider
      </code>
    </li>

    <li>
      <span>
        04
      </span>

      <div>
        <strong>
          Every mutation becomes realtime state
        </strong>

        <p>
          {
                      "The worker creates ai-output and appends or updates the message through Sockudo."
                    }
        </p>
      </div>

      <code>
        Agent -> Sockudo
      </code>
    </li>

    <li>
      <span>
        05
      </span>

      <div>
        <strong>
          Clients recover from the channel
        </strong>

        <p>
          {
                      "Live clients receive rollups; refreshed clients rewind history and reconstruct the transcript."
                    }
        </p>
      </div>

      <code>
        Sockudo -> clients
      </code>
    </li>
  </ol>
</div>

The provider never needs to know about connection recovery, rewind, history cursors, branch trees,
presence, or multi-device fanout. It streams chunks. Sockudo makes those chunks durable and
observable.

## Vercel AI SDK [#vercel-ai-sdk]

Use Vercel AI SDK when you want provider packages, tool calling, and UI message streams to stay in
the `ai` ecosystem.

```ts
// server/api/chat.post.ts
import { streamText, toUIMessageStream } from "ai";
import { createAgentSession } from "@sockudo/ai-transport/vercel";
import { realtimeClient } from "../utils/sockudo";

export default defineEventHandler(async (event) => {
  const body = await readBody(event);

  const session = createAgentSession({
    client: realtimeClient(),
    channelName: body.channelName,
  });

  const run = session.createRun({
    runId: body.runId,
    invocationId: body.invocationId,
    inputEventId: body.inputEventId,
    clientId: body.clientId,
    onCancel(request) {
      return (
        request.filter.all === true ||
        request.matchedRunIds.includes(body.runId)
      );
    },
  });

  event.waitUntil?.(
    (async () => {
      await run.start();
      const result = streamText({
        model: "openai/gpt-5-mini",
        instructions: "You are a concise incident-room assistant.",
        prompt:
          body.messages
            .at(-1)
            ?.parts?.map((part) => part.text ?? "")
            .join("") ?? "",
        abortSignal: run.abortSignal,
      });
      await run.streamResponse(toUIMessageStream({ stream: result.stream }));
      await run.end("complete");
      session.close();
    })(),
  );

  setResponseStatus(event, 202);
});
```

The frontend can use the Vercel-compatible transport in React, Vue, or Svelte.

```ts
import { useChat } from "@ai-sdk/vue";
import { provideChatTransport } from "@sockudo/ai-transport/vercel/vue";

const provider = provideChatTransport({
  api: "/api/chat",
  channelName: "private-ai:user-42:sess-01J",
  client: sockudoClient,
  clientId: "user-42",
});

const { sendMessage } = useChat({
  id: "private-ai:user-42:sess-01J",
  transport: provider.chatTransport.value,
});

await sendMessage({ text: "Summarize the last deploy failure." });
```

## AI SDK 7 feature coverage [#ai-sdk-7-feature-coverage]

AI SDK 7 does not require a separate Sockudo server mode. The compatibility surface is the AI SDK UI
message stream: Sockudo persists ordered UI chunks, run lifecycle events, approval responses, and
metadata, while the app runtime keeps provider calls, tool execution, and agent-local state.

| AI SDK capability                         | Sockudo behavior                                                                                                                                       |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Reasoning controls                        | Pass model options through `streamText`; Sockudo carries emitted reasoning text and reasoning-file chunks.                                             |
| Tool context and runtime context          | Keep context on the trusted worker. Publish only safe derived metadata, never provider secrets or raw tool credentials.                                |
| Provider file and skill uploads           | Let the provider own uploads and references. Sockudo can carry file, source, reasoning-file, or custom chunks that point at externally stored assets.  |
| MCP Apps and custom UI parts              | Preserve `custom` chunks and provider metadata. Rendering app iframes or custom widgets remains a client responsibility.                               |
| Tool approvals                            | Persist approval requests and responses as normal run state. The app still owns approval policy, signature checks, and tool execution authorization.   |
| `WorkflowAgent` and `HarnessAgent` output | Stream their UI-message output through `toUIMessageStream`. Use the workflow store for agent checkpoints and Sockudo for transcript/reconnect history. |
| Realtime voice                            | Keep low-latency audio on the provider's realtime channel. Mirror transcripts, tool calls, approvals, and final state through Sockudo.                 |
| Video generation                          | Store generated media outside Sockudo and publish progress, asset URLs, thumbnails, or final file parts through the AI session channel.                |
| Telemetry and lifecycle callbacks         | Correlate AI SDK spans with `channelName`, `runId`, and `invocationId`. Sockudo metrics continue to describe transport, history, and fanout health.    |

## Direct provider support [#direct-provider-support]

The `@sockudo/ai-transport/providers` entry point supports these built-in adapters:

| Provider path                       | Function                                                                  | Notes                                                           |
| ----------------------------------- | ------------------------------------------------------------------------- | --------------------------------------------------------------- |
| OpenAI-compatible HTTP/SSE          | `streamOpenAICompatibleText`                                              | Uses Chat Completions-compatible `/chat/completions` streaming. |
| OpenAI-compatible reusable provider | `createOpenAICompatibleProvider`                                          | Good for named provider registries.                             |
| OpenAI SDK Chat Completions         | `streamOpenAIChatCompletion` / `createOpenAISdkProvider`                  | Uses a structural subset of the official OpenAI SDK.            |
| OpenAI SDK Responses                | `streamOpenAIResponse` / `createOpenAISdkProvider({ mode: "responses" })` | Maps output text and tool argument deltas to UI chunks.         |
| Anthropic SDK Messages              | `streamAnthropicMessage` / `createAnthropicSdkProvider`                   | Maps text, thinking, tool use, and finish reasons.              |

OpenAI-compatible presets are:

| Name         | Default base URL                        |
| ------------ | --------------------------------------- |
| `openai`     | `https://api.openai.com/v1`             |
| `openrouter` | `https://openrouter.ai/api/v1`          |
| `groq`       | `https://api.groq.com/openai/v1`        |
| `togetherai` | `https://api.together.xyz/v1`           |
| `fireworks`  | `https://api.fireworks.ai/inference/v1` |
| `deepseek`   | `https://api.deepseek.com`              |
| `perplexity` | `https://api.perplexity.ai`             |
| `mistral`    | `https://api.mistral.ai/v1`             |
| `xai`        | `https://api.x.ai/v1`                   |
| `ollama`     | `http://127.0.0.1:11434/v1`             |
| `lmstudio`   | `http://127.0.0.1:1234/v1`              |

Local providers such as Ollama and LM Studio may omit `apiKey` when the local server does not
require one.

## OpenAI-compatible HTTP example [#openai-compatible-http-example]

```ts
import {
  createOpenAICompatibleProvider,
  runDirectLlm,
} from "@sockudo/ai-transport/providers";

const provider = createOpenAICompatibleProvider({
  provider: "groq",
  apiKey: process.env.GROQ_API_KEY,
  model: "llama-3.3-70b-versatile",
});

await runDirectLlm(run, provider, {
  prompt: "Write a remediation plan for a Redis fanout incident.",
  maxOutputTokens: 800,
});
```

## OpenAI SDK examples [#openai-sdk-examples]

```ts
import OpenAI from "openai";
import {
  createOpenAISdkProvider,
  runDirectLlm,
} from "@sockudo/ai-transport/providers";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const chatProvider = createOpenAISdkProvider({
  client: openai,
  mode: "chat",
  model: "gpt-4.1-mini",
});

await runDirectLlm(run, chatProvider, {
  messages: [
    { role: "system", content: "Answer as a production engineer." },
    { role: "user", content: "Why did reconnect recovery fail?" },
  ],
});
```

```ts
const responsesProvider = createOpenAISdkProvider({
  client: openai,
  mode: "responses",
  model: "gpt-5-mini",
});

await runDirectLlm(run, responsesProvider, {
  prompt: "Explain the latest channel history page in plain English.",
  body: {
    reasoning: { effort: "low" },
  },
});
```

## Anthropic SDK example [#anthropic-sdk-example]

```ts
import Anthropic from "@anthropic-ai/sdk";
import {
  createAnthropicSdkProvider,
  runDirectLlm,
} from "@sockudo/ai-transport/providers";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const provider = createAnthropicSdkProvider({
  client: anthropic,
  model: "claude-sonnet-4-5",
  system: "Be concise and include risk levels.",
});

await runDirectLlm(run, provider, {
  prompt: "Audit this failed push notification delivery chain.",
});
```

## Provider registry [#provider-registry]

Use a registry when the UI lets a user or tenant select a model provider.

```ts
import {
  createAnthropicSdkProvider,
  createDirectLlmProviderRegistry,
  createOpenAICompatibleProvider,
} from "@sockudo/ai-transport/providers";

const providers = createDirectLlmProviderRegistry({
  groq: createOpenAICompatibleProvider({
    provider: "groq",
    apiKey: process.env.GROQ_API_KEY,
    model: "llama-3.3-70b-versatile",
  }),
  local: createOpenAICompatibleProvider({
    provider: "ollama",
    model: "llama3.2",
  }),
  anthropic: createAnthropicSdkProvider({
    client: anthropic,
    model: "claude-sonnet-4-5",
  }),
});

const stream = await providers.streamText("local", {
  prompt: "Return a JSON incident summary.",
});
```

## Custom provider [#custom-provider]

Any provider that returns `ReadableStream<VercelOutput>` can participate.

```ts
import type { DirectLlmProvider } from "@sockudo/ai-transport/providers";

const provider: DirectLlmProvider = {
  async streamText(request) {
    const words = (request.prompt ?? "").split(/\s+/);

    return new ReadableStream({
      start(controller) {
        controller.enqueue({ type: "start" });
        controller.enqueue({ type: "text-start", id: "answer" });
        for (const word of words) {
          controller.enqueue({
            type: "text-delta",
            id: "answer",
            delta: `${word} `,
          });
        }
        controller.enqueue({ type: "text-end", id: "answer" });
        controller.enqueue({ type: "finish", finishReason: "stop" });
        controller.close();
      },
    });
  },
};
```

## Tool calling and human approval [#tool-calling-and-human-approval]

Tool calls are just streamed UI message chunks. The transport persists every tool-input delta and
the final tool state, so a user can approve from another tab or after a reconnect.

```mermaid
sequenceDiagram
  participant A as Agent
  participant S as Sockudo
  participant W as Web client
  participant T as Tool executor

  A->>S: tool-input-start + deltas
  S-->>W: pending approval appears
  W->>S: approval response
  T->>S: tool output
  A->>S: continue ai-output append
```

For AI SDK 7 approval flows, treat the request and response as part of the transcript:

1. The agent streams a `tool-approval-request` chunk with a stable approval id and tool call id.
2. Sockudo stores that request on the `ai-output` mutable message and fans it out to every attached
   client.
3. The client records an approval response on the assistant tool part. The Vercel chat transport
   diffs the optimistic message overlay and publishes a `tool-approval-response` input.
4. The worker verifies the response, runs or denies the tool, then appends `tool-result`,
   `tool-result-error`, or `output-denied` state.

Use channel auth and V2 capabilities to decide who can approve. If the provider supplies approval
signatures or automatic-approval markers, keep them with the approval chunk so the worker can verify
them before executing the tool. Do not treat a client-rendered button click as sufficient authority
by itself.

## Realtime voice and generated media [#realtime-voice-and-generated-media]

AI SDK realtime voice sessions are optimized for provider-native WebSocket or WebRTC media paths.
Sockudo should not proxy high-rate PCM frames, microphone buffers, or generated video bytes through
mutable-message appends. Use Sockudo as the durable control and transcript plane:

* publish session start, model, voice, and participant metadata
* append transcript deltas, final transcript text, tool calls, and approval state
* fan out presence for the user, agent, and handoff devices
* publish push notifications or completion events when the media job finishes
* store recordings, uploaded files, thumbnails, and generated videos in object storage, then publish
  references as file, source, or custom chunks

This keeps reconnect, rewind, audit, and multi-device UI recovery in Sockudo without putting
latency-sensitive media on the same path as durable chat history.

## Choosing an integration [#choosing-an-integration]

| You have                                 | Use                                                                    |
| ---------------------------------------- | ---------------------------------------------------------------------- |
| Vercel AI SDK `streamText` already wired | `@sockudo/ai-transport/vercel`                                         |
| A provider with OpenAI-compatible SSE    | `createOpenAICompatibleProvider`                                       |
| Official OpenAI SDK                      | `createOpenAISdkProvider`                                              |
| Official Anthropic SDK                   | `createAnthropicSdkProvider`                                           |
| A custom internal model gateway          | `DirectLlmProvider`                                                    |
| A non-Vercel UI model                    | Core codec API from `@sockudo/ai-transport`                            |
| AI SDK agent or workflow output          | Convert to UI message chunks, then stream through the Vercel transport |
| Realtime voice or generated video        | Provider media path plus Sockudo transcript/control events             |

Keep provider API keys server-side. Browser clients should only receive Sockudo public app keys,
private/presence auth responses, and short-lived Protocol V2 capability tokens.


# Push notifications for AI Transport (/docs/server/ai-transport-push-notifications)



AI Transport uses normal Sockudo channel events for lifecycle and output. Push rules let those same channel publishes wake devices through the existing push notification pipeline.

## Channel rule recipe [#channel-rule-recipe]

Configure a rule that watches each user's notification channel:

```toml
[[push_rules]]
enabled = true
channel_pattern = "notifications:*"
event_filter = ["agent-complete"]
rate_limit_per_second = 100

[push_rules.payload_mapping]
title_field = "title"
body_field = "body"
template_data_field = "data"
include_remaining_fields = true
```

Register each device and subscribe it to the same channel the app is allowed to read:

```http
POST /apps/{appId}/push/channelSubscriptions
```

The long-running agent publishes a normal event when the answer is ready:

```json
{
  "name": "agent-complete",
  "channel": "notifications:user-123",
  "data": {
    "title": "Agent finished",
    "body": "Your answer is ready",
    "sessionId": "sess_123"
  }
}
```

Sockudo maps `title` and `body` into the push notification and copies the remaining fields into provider data under `data`. The push target is `Channel { channel: "notifications:user-123" }`, so existing channel subscriptions determine which devices receive FCM, APNs, Web Push, HMS, or WNS delivery.

When the user taps the notification, the client should load `sessionId`, attach to the realtime session channel, and read history/recovery state for the authoritative transcript.

## Webhook recipe [#webhook-recipe]

For teams that keep push orchestration in their backend, enable the `ai_run_ended` webhook. On `reason = "complete"`, the backend can call the existing push publish API:

```json
{
  "recipients": [
    { "type": "channel", "channel": "notifications:user-123" }
  ],
  "payload": {
    "title": "Agent finished",
    "body": "Your answer is ready",
    "templateData": {
      "data": { "sessionId": "sess_123" }
    }
  },
  "sync": false
}
```

This path is useful when the backend enriches notifications, applies product-specific quiet hours, or joins run metadata with an application database before sending push.

## Authorization [#authorization]

Use capability patterns so clients can subscribe only to their own notification channels, for example `notifications:user-123`, while trusted backends or agent workers publish `agent-complete`. Do not accept `client_id` from the message body when choosing a notification channel; derive the channel from verified auth state or server-side session ownership.

## Verification surface [#verification-surface]

The recipe relies on existing push device registration, channel subscriptions, channel publish targets, scheduled/status handling, delivery feedback, retries, and dead-letter behavior. Push rules add only the channel-to-push trigger; accepted work still goes through the same durable status, fanout, queue, retry, and feedback transitions as explicit push publishes.


# AI Transport troubleshooting (/docs/server/ai-transport-troubleshooting)



| Symptom                              | Likely cause                                                                                | Operator action                                                                                          |
| ------------------------------------ | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| Mutations rejected as not permitted  | Capability lacks `message_*_own` or `message_*_any`, or actor identity does not match owner | Inspect the verified connection identity and token capability map; do not rely on body `client_id`.      |
| History retention too short          | Rewind or recovery points before retained durable history                                   | Increase `[history]` retention or lower expected rewind/recovery window; watch retained gauges.          |
| Run never ends                       | Missing terminal append/update/summary or orphan TTL too high                               | Check `sockudo_ai_active_streams`, terminal event rates, and `ai_transport.rollup.orphan_ttl_ms`.        |
| Suspended-state publishes            | Producer continued after suspend/continuation boundary                                      | Reject or route through the continuation flow; audit `message_id` and `op_id`.                           |
| Reconnect loop on bad tokens         | Client retries expired/revoked/bad JWT without refresh                                      | Fix token refresh, rate-limit retry storms, revoke compromised `jti` or `client_id`.                     |
| Rollup latency too high              | Window too high or slow flush under load                                                    | Compare `sockudo_flush_latency` to the configured window and lower `default_window_ms`.                  |
| Rollup savings too low               | Window too low or terminal flushes dominate                                                 | Compare `sockudo_appends_received_total`, `sockudo_appends_delivered_total`, and `sockudo_rollup_ratio`. |
| Push channel misses user             | Device not registered or not subscribed to the notification channel                         | Inspect push channel subscriptions and provider token state.                                             |
| Presence flaps during transient loss | Ungraceful timeout is zero or too low                                                       | Raise `[presence].v2_ungraceful_timeout_seconds` for Protocol V2 agent presence channels.                |
| Recovery fails after node loss       | Durable state degraded/reset-required or shared cache/store unavailable                     | Inspect `sockudo_history_recovery_failures_total`, degraded/reset-required gauges, and store health.     |

For scale and chaos-specific playbooks, see [AI Transport production operations](/docs/server/ai-transport-production-ops).


# Configuration (/docs/server/configuration)



Sockudo configuration should describe the runtime shape explicitly: app storage, fanout adapter, cache, queue, protocol features, security controls, metrics, webhooks, and push notification providers.
For TOML versus JSON, file discovery, `--config`, environment precedence, and secret placement, start
with [Static configuration](/docs/deployment/static-configuration).

## Minimal local config [#minimal-local-config]

```toml
port = 6001
host = "0.0.0.0"
debug = true

[app_manager]
driver = "memory"

[app_manager.array]
[[app_manager.array.apps]]
id = "app-id"
key = "app-key"
secret = "app-secret"
enabled = true

[app_manager.array.apps.policy.limits]
max_connections = 1000

[app_manager.array.apps.policy.features]
enable_client_messages = false
```

## Production shape [#production-shape]

```toml
port = 6001
host = "0.0.0.0"
debug = false

[app_manager]
driver = "postgres"

[adapter]
driver = "redis"

[cache]
driver = "redis"

[queue]
driver = "redis"

[metrics]
enabled = true
host = "0.0.0.0"
port = 9601

[metrics.tcp_exporter]
enabled = false
host = "127.0.0.1"
port = 5000
buffer_size = 1024
```

## App manager [#app-manager]

The app manager stores app credentials and app-level policy. Memory is useful for local development; persistent managers are preferred for production.

| Driver                | Use when                                                          |
| --------------------- | ----------------------------------------------------------------- |
| `memory`              | Credentials are static and local to one process.                  |
| `postgres` or `mysql` | You need relational app records and standard operational tooling. |
| `redis`               | You need lightweight shared app state.                            |
| `dynamodb`            | You run on AWS and want managed key-value storage.                |
| `scylladb`            | You need wide-column scale.                                       |
| `surrealdb`           | You use SurrealDB 3 for app metadata.                             |

## Adapter [#adapter]

The adapter controls cross-node fanout.

```toml
[adapter]
driver = "redis"
enable_socket_counting = true
aggregate_counts = false
fast_presence_transitions = false

[adapter.redis]
host = "redis"
port = 6379
prefix = "sockudo"
```

Use a shared adapter for every multi-node deployment. Local memory adapters are intentionally process-local.

`enable_socket_counting` keeps the adapter's request/reply socket-count path enabled by default.
`aggregate_counts` is off by default; enable it when you want each node to maintain gossiped
cluster-wide channel counts for local count reads.
`fast_presence_transitions` is off by default. Enabling it uses the replicated presence registry
for first-join and last-leave checks, which avoids request/reply fanout but makes presence
webhook/history transition decisions eventually consistent.

For high-churn benchmark runs that should avoid distributed count fanout, use:

```bash
ADAPTER_ENABLE_SOCKET_COUNTING=false
ADAPTER_AGGREGATE_COUNTS=true
# Optional: enables eventual-consistency presence transition checks.
ADAPTER_FAST_PRESENCE_TRANSITIONS=true
```

## Recovery and history [#recovery-and-history]

```toml
[recovery]
enabled = true
buffer_size = 1000
ttl_seconds = 120

[history]
enabled = true
retention_seconds = 86400
max_items_per_channel = 10000
```

Recovery buffers are for reconnect continuity. Durable history is for API reads, rewind, versioned messages, and operational inspection. Keep those concerns separate.

## Push notifications [#push-notifications]

Push is a core Sockudo subsystem. Configure it with a queue, provider credentials, retention, admission limits, and metrics before sending production traffic.

```toml
[push]
storage_driver = "postgres"
queue_driver = "redis"
publish_status_ttl_days = 30
analytics_retention_days = 30
scheduler_interval_secs = 5
cleanup_interval_secs = 300
cleanup_batch_size = 1000
cleanup_max_deleted_per_tick = 100000

# Runtime env for FCM monolith workers:
# PUSH_FCM_ENABLED=true
# PUSH_FCM_SERVICE_ACCOUNT_JSON_PATH=/var/run/secrets/fcm-service-account.json
# PUSH_FCM_PROJECT_ID is optional when the service account JSON has project_id.

# Runtime env for APNs monolith workers:
# PUSH_APNS_ENABLED=true
# PUSH_APNS_TOPIC=com.example.app
# PUSH_APNS_PRIVATE_KEY_PATH=/var/run/secrets/apns-auth-key.p8
```

Use the queue backend for push fanout. Direct synchronous provider delivery is only suitable for tests and can hide production latency.

Channel push rules are top-level entries. They trigger the existing push pipeline after matching realtime publishes:

```toml
[[push_rules]]
enabled = true
channel_pattern = "notifications:*"
event_filter = ["agent-complete"]
rate_limit_per_second = 100

[push_rules.payload_mapping]
title_field = "title"
body_field = "body"
template_data_field = "data"
include_remaining_fields = true
```

## Presence [#presence]

```toml
[presence]
max_members_per_channel = 100
max_member_size_in_kb = 2
update_rate_limit_per_member_per_second = 10
ungraceful_timeout_seconds = 0
v2_ungraceful_timeout_seconds = 15
```

`ungraceful_timeout_seconds = 0` preserves Protocol V1/Pusher's legacy immediate
`member_removed` behavior. Protocol V2 defaults
`v2_ungraceful_timeout_seconds` to `15`, retaining presence through short abnormal
disconnects without leave/enter flaps. Clean unsubscribe and clean WebSocket close
remove presence immediately for both protocol versions.

## Webhooks [#webhooks]

```toml
[webhooks]
enabled = true
batching_enabled = true
max_batch_size = 50
flush_interval_ms = 500
timeout_ms = 5000
```

Webhook consumers must validate signatures with the raw request body. Configure retry behavior and dead-letter visibility before relying on webhooks for workflows.

## Rate limits [#rate-limits]

```toml
[rate_limiter]
enabled = true
driver = "redis"

[rate_limiter.limits]
connection_per_ip = 50
events_per_second = 100
```

Keep limits close to product intent. A collaboration app, trading dashboard, and push-heavy mobile app need different ceilings.

## Environment variables [#environment-variables]

Use environment variables for secrets and deployment-specific settings:

```bash
SOCKUDO_DEFAULT_APP_ID=app-id
SOCKUDO_DEFAULT_APP_KEY=app-key
SOCKUDO_DEFAULT_APP_SECRET=app-secret
REDIS_URL=redis://redis:6379/0
PUSH_FCM_PROJECT_ID=project-id
PUSH_FCM_PROVIDER_TOKEN=oauth2-access-token
```

Avoid putting app secrets, encryption master keys, webhook secrets, and push provider credentials in committed config files.

Every runtime environment override is listed in the [environment variable reference](/docs/reference/environment-variables).


# History and recovery (/docs/server/history-recovery)



Sockudo separates connection recovery from durable history. Recovery keeps a connection continuous after a short interruption. History is an application feature for reading and reconstructing past state.

## Recovery [#recovery]

Protocol V2 broadcasts carry stream metadata:

```json
{
  "event": "order.updated",
  "channel": "orders",
  "data": { "id": "ord_123" },
  "message_id": "msg_01HX",
  "stream_id": "orders:main",
  "serial": 42
}
```

Clients store recovery positions and provide them during reconnect. Sockudo replays missed messages if continuity can be proven.

For V2 clients, the subscription acknowledgement carries the first recovery position when recovery is enabled. Keep that position even before the channel receives any application event. A reconnect can then resume from the subscribed-but-idle checkpoint and replay messages published while the client was disconnected.

```ts
const client = new Sockudo("app-key", {
  wsHost: "127.0.0.1",
  wsPort: 6001,
  forceTLS: false,
  protocolVersion: 2,
  connectionRecovery: true,
});

client.bind("sockudo:resume_success", (payload) => {
  console.log(payload.recovered, payload.failed);
});
```

If a channel emits `sockudo:resume_failed` with `code: "position_expired"`, the hot replay buffer and durable recovery window could not prove continuity. Resubscribe, then backfill with V2 channel history using `until_attach: true` so the history page is bounded at the subscription's attach serial.

`code: "continuity_unverifiable"` means Sockudo rejected an ahead, non-contiguous, duplicate, or non-progressing recovery position. Treat it the same way: resubscribe and backfill instead of advancing the client cursor.

Recovery is two-tier:

1. Hot replay uses the bounded per-channel replay buffer.
2. Durable recovery uses `HistoryStore` when `stream_id`, serial bounds, and retention prove
   continuity.

Both tiers enforce `connection_recovery.max_buffer_size` for each recovered
channel. A larger gap fails with `position_expired` and
`reason: "recovery_window_exceeded"` before Sockudo builds or partially sends an
unbounded replay. When the SDK has already reattached a subscription, a bounded
continuity gate holds concurrent live messages until replay has been enqueued;
the aggregate `resume_success` is enqueued before those live messages are
released. Gate overflow closes that connection instead of delivering frames out
of order.

Both tiers replay the Protocol V2 wire message, not Sockudo's durable storage
wrapper. Durable rows are decoded through the shared history projection first,
so recovery and ordinary history reads preserve the same IDs, publisher fields,
version projection, stream ID, and serial.

Versioned mutations keep the delivery position reserved by the native version
service. The hot replay buffer adopts that position instead of allocating a
second serial. Recovery validates every serial in the returned range; a missing
position or mixed stream generation falls through to durable recovery and then
fails closed if the durable tier cannot prove the same continuity.

The native V2 attach and resume gates register the subscriber before capturing
the durable high-water mark. Concurrent publishes are held in a count-and-byte-
bounded gate, shared across subscribers with immutable message references, then
drained in ordered batches without a lock held across delivery. Overflow closes
the affected connection because the live-plus-replay sequence can no longer be
proven.

If the durable stream is degraded or reset-required, Sockudo fails closed instead of pretending the
stream is continuous.

## Rewind [#rewind]

Subscribe-time rewind asks Sockudo to send recent history when a client joins:

```ts
const channel = client.subscribe("orders", {
  rewind: { seconds: 30 },
});

channel.bind("sockudo:rewind_complete", (payload) => {
  console.log(payload.historical_count, payload.complete);
});
```

When a V2 subscription has a compound predicate, rewind scans durable history in bounded pages until it fills the requested matching count or reaches the scan/retention boundary. Suppressed rows still advance the channel's canonical continuity position. Hot and durable recovery apply the same predicate after the client has reattached that subscription; a legacy resume sent before reattachment retains the unfiltered recovery contract. Each recovered-channel entry in `sockudo:resume_success` includes the terminal `position` so clients advance continuity even when the predicate suppresses the tail.

## Durable channel history [#durable-channel-history]

Server SDKs expose app-key channel history helpers. Use signed app-key HTTP history for backend jobs, administrative reads, and server SDKs.

```python
page = sockudo.get_channel_history(
    "orders",
    HistoryParams(limit=50, direction="newest_first"),
)
```

Opaque cursors make storage implementation details private. Store the cursor as a token and pass it back unchanged.

Protocol V2 clients use WebSocket `channel_history` frames instead of direct HTTP history. Token-authenticated clients need the `history` capability for the requested channel. The existing app-key HTTP endpoint remains server-only; there is no client-token HTTP history surface in v1.

```json
{
  "event": "sockudo:channel_history",
  "data": {
    "channel": "orders",
    "limit": 100,
    "direction": "backwards",
    "cursor": null,
    "start_serial": 1,
    "end_serial": 100,
    "until_attach": true
  }
}
```

The response uses the same history payload shape as app-key HTTP history: `items`, `direction`, `limit`, `has_more`, `next_cursor`, `bounds`, `continuity`, and `stream_state`. Limits are capped at `1000` and may be lowered by channel policy. `direction` defaults to backwards/newest-first. `until_attach: true` bounds results to `history_serial <= attach_serial`, while live delivery after subscribe continues above that attach serial. Mutable messages are returned as the latest aggregated V2 message, not raw operation log rows.

`until_attach` is the late-join gaplessness rule: the history page stops at the serial captured when
the subscription succeeded, and live fanout continues above that serial. Clients should reduce
history first, then live messages. In a cluster, route this request to the node that owns the live
attachment. If Sockudo cannot find that authoritative attachment position, it rejects the read
instead of silently removing the upper bound and returning a page that could overlap live delivery.

## Presence history [#presence-history]

Presence history records joins, leaves, causes, and continuity metadata. It is different from the current presence member list.

Protocol projections must write transitions through the native presence
service. Current membership remains available if the durable tier fails, but
the tracking store marks that channel degraded. Continuity-sensitive history
reads then fail closed until the durable stream is healthy or explicitly reset;
they must not return a partial page as complete.

```ts
const channel = client.subscribe("presence-lobby");
const page = await channel.history({
  limit: 50,
  direction: "newest_first",
});

const snapshot = await channel.snapshot({ atSerial: 4 });
```

Client helpers call your backend proxy. They do not sign Sockudo REST requests directly.

## Mutable messages [#mutable-messages]

Protocol V2 mutable messages use action events:

* `sockudo:message.update`
* `sockudo:message.delete`
* `sockudo:message.append`

Clients reduce those events into local state. Server SDKs can fetch the latest visible state or list preserved versions.
When a V2 subscription uses rewind, historical rows that point at mutable messages are delivered as the latest visible version, so late joiners see accumulated appended content rather than only the original create payload.

```ts
const latest = await channel.getMessage("42");
const versions = await channel.getMessageVersions("42", {
  limit: 20,
  direction: "oldest_first",
});
```

## Push and history [#push-and-history]

Push notifications should include stable identifiers that let the app fetch authoritative state after the user opens the notification.

```json
{
  "title": "Order updated",
  "body": "Order ord_123 is now packed",
  "data": {
    "channel": "orders",
    "message_serial": "42",
    "order_id": "ord_123"
  }
}
```

Do not encode the entire durable state into the push payload. Use push to wake the app, then read the latest message or application API state.


# HTTP API (/docs/server/http-api)



Sockudo's HTTP API is a trusted server interface. Every production request must be signed with app credentials. Use a server SDK unless you are writing a custom integration.

## Signing model [#signing-model]

Signed requests include:

| Parameter        | Purpose                                                     |
| ---------------- | ----------------------------------------------------------- |
| `auth_key`       | App key.                                                    |
| `auth_timestamp` | Unix timestamp used to reject stale requests.               |
| `auth_version`   | Signing protocol version.                                   |
| `body_md5`       | MD5 of the JSON body for body-bearing requests.             |
| `auth_signature` | HMAC SHA-256 signature over method, path, and sorted query. |

## Publish an event [#publish-an-event]

```http
POST /apps/{app_id}/events
Content-Type: application/json
```

```json
{
  "name": "order.created",
  "channel": "orders",
  "data": {
    "id": "ord_123"
  },
  "idempotency_key": "order-created-ord_123"
}
```

Use `socket_id` to suppress echo back to the originating connection.

When `ai_transport` and versioned messages are enabled for the target channel,
AI events and publishes with `message_id` return serial acknowledgements under
the channel entry. Retries with the same `message_id` or `X-Idempotency-Key`
return the original serials without publishing a duplicate event.
If the publishing node exits after durable persistence but before recording the
idempotency acknowledgement, a retry reconstructs and atomically repairs the
receipt from native history/version storage. Raw idempotency keys are not stored
in message envelopes; recovery uses an app/channel-scoped hash plus the canonical
payload fingerprint.

Idempotent publishes fail closed when their shared coordination backend is
unavailable: Sockudo returns `503 service_unavailable` instead of publishing
without a proven claim. Concurrent retries whose first request is still active also
return `503 backpressure` with `Retry-After: 1`. Retry either response with the same
idempotency key; a load balancer may route the retry to a healthy node.

```json
{
  "channels": {
    "ai:session-1": {
      "message_serial": "00000000000000000001:test:00000000000000000001",
      "history_serial": 1,
      "delivery_serial": 1,
      "version_serial": "00000000000000000001:test:00000000000000000001"
    }
  }
}
```

## Batch publish [#batch-publish]

```http
POST /apps/{app_id}/batch_events
Content-Type: application/json
```

```json
{
  "batch": [
    {
      "channel": "orders",
      "name": "order.created",
      "data": { "id": "ord_123" }
    },
    {
      "channel": "orders",
      "name": "order.paid",
      "data": { "id": "ord_123" }
    }
  ]
}
```

Batching reduces HTTP overhead, but do not use it to hide unbounded payloads. Keep per-event data compact.
`X-Idempotency-Key` identifies the whole batch request, while an
`idempotency_key` inside an item identifies only that logical event. The two key
domains do not collide. Every per-event key is validated with the same non-empty
and maximum-length rules as a single-event publish before any item is accepted,
then atomically claimed so concurrent batches cannot publish that event twice.

When Cargo feature and runtime option `ably-compat` are enabled, the root REST
router also exposes Ably-compatible batch operations:

```http
POST /messages
GET /presence?channels=channel-a,channel-b
POST /keys/{keyName}/revokeTokens
```

`POST /messages` accepts the raw object used by `Rest.request()` and the array of
batch specs used by `batchPublish()`. Results preserve spec and channel order.
The raw form returns `201` plus a flat channel result array on full success, or
`400`/`40020` plus `{ error, batchResponse }` when one or more channels fail.
The array form, batch presence, and token revocation return BatchResult envelopes
with exact `successCount`, `failureCount`, and ordered item results. Publish
success is reported only after the normal native publish pipeline commits; item
failures do not roll back unrelated successful items.

The compatibility batch edge runs no more than eight channel/target operations
at once and rejects requests exceeding 100 specs, 1,000 results, 10,000 publish
operations, or 10 MiB. Native event channel/message limits can lower those
ceilings. Bodies and responses negotiate JSON or MsgPack through the same REST
codec, and error responses include `X-Ably-ErrorCode` and
`X-Ably-ErrorMessage`.

## Channel state [#channel-state]

```http
GET /apps/{app_id}/channels
GET /apps/{app_id}/channels/{channel_name}
GET /apps/{app_id}/channels/{channel_name}/users
```

Use state APIs for dashboards and admin tooling. Do not poll them as a substitute for realtime subscription events.

For channels matched by `[ai_transport]`, `GET /apps/{app_id}/channels/{channel_name}`
also includes:

```json
{
  "ai": {
    "active_streams": 1,
    "last_history_serial": 42,
    "message_count": 7
  }
}
```

## History [#history]

```http
GET /apps/{app_id}/channels/{channel_name}/history?limit=50&direction=newest_first
```

History responses use opaque cursors. Store and replay cursors as strings; do not parse them.
Use `direction=oldest_first`/`forwards` or `direction=newest_first`/`backwards`
with `start`, `end`, and the policy-capped `limit` to bound a stable page. A
cursor is opaque and must be replayed with the same query shape that produced
it. The Ably-compatible `/channels/{channel}/messages` projection additionally
returns credential-free `first` and optional `next` Link relations that preserve
that query shape.

## Versioned messages [#versioned-messages]

Protocol V2 mutable messages expose latest visible state and preserved versions:

```http
GET /apps/{app_id}/channels/{channel_name}/messages/{message_serial}
GET /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/versions
POST /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/update
POST /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/delete
POST /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/append
```

The optional Ably compatibility route is
`PATCH /channels/{channel_name}/messages/{message_serial}`. Its encoded message
body selects `message.update`, `message.delete`, or `message.append` through the
`action` field and carries operation metadata in `version`. It is a projection
over the same native mutation service and `VersionStore`; it does not maintain
a compatibility-only message chain. Request and response formats are negotiated
independently as JSON or MsgPack.

```json
{
  "data": {
    "body": "edited message"
  },
  "description": "user edit",
  "op_id": "edit-msg-1-v2"
}
```

Mutation responses preserve the existing fields and include serials for SDK
ordering and retries:

```json
{
  "channel": "ai:session-1",
  "message_serial": "msg_123",
  "action": "append",
  "accepted": true,
  "version_serial": "ver_456",
  "history_serial": 12,
  "delivery_serial": 34,
  "status": "applied"
}
```

## Annotations [#annotations]

Annotations attach secondary state to a message without rewriting the message itself.

```http
POST /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/annotations
GET /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/annotations
DELETE /apps/{app_id}/channels/{channel_name}/messages/{message_serial}/annotations/{annotation_serial}
```

```json
{
  "type": "reactions:distinct.v1",
  "name": "like",
  "client_id": "user-1",
  "count": 1
}
```

Use annotations for reactions, read receipts, moderation signals, and summary projections.

With Cargo feature `ably-compat`, the compatibility router also exposes
`GET|POST /channels/{channel}/messages/{messageSerial}/annotations`. The POST
body is an Ably annotation array and supports both create and delete actions;
both operations call the same native annotation service shown above. Responses
negotiate JSON or MsgPack. List pages return credential-free relative Link
headers with an opaque app/channel/message-scoped cursor. Realtime action `21`
is delivered only to an attachment that negotiated `annotation_subscribe`,
while `message.summary` uses the original message serial for ordinary
subscribers. `annotation-publish`, `annotation-subscribe`,
`annotation-delete-own`, and `annotation-delete-any` are evaluated independently
from message mutation permissions.

## Push notifications [#push-notifications]

Push APIs are first-class HTTP APIs, not separate infrastructure. They cover device registration, channel subscriptions, credential management, publish admission, status inspection, scheduling, cancellation, and provider callbacks.

When the server is built with `ably-compat`, the root Ably REST projection also
provides `/stats`, `/push/publish`, `/push/deviceRegistrations`,
`/push/channelSubscriptions`, and `/push/channels`. These routes authenticate
with Ably-compatible credentials but call a bounded typed compatibility stats
store plus the native push domain services; they do not call Sockudo HTTP
handlers internally. Stats pages use opaque cursors and stable Link query
propagation. Direct push and `extras.push` enter the native durable publish log,
queue, planner, provider-dispatch, feedback, retry, status, scheduler, and
retention pipeline. The realtime provider delivers `__ably_push__` through the
ordinary `MessageService` fanout path and records the actual outcome. External
providers continue through their configured native workers and fail truthfully
when the required feature or credentials are unavailable. Restricted Ably keys
must carry `push-admin` or channel-scoped `push-subscribe`; device-owned requests
also verify the stored hashed device identity token.

The same projection exposes `GET /channels/{channel}/presence` and
`GET /channels/{channel}/presence/history`. Current presence supports
`clientId`, `connectionId`, `limit`, and opaque cursor pagination. Presence
history additionally supports forward/backward direction and millisecond
`start`/`end` bounds. Both routes use the native presence service and return
credential-free `first`/`next` Link relations. History reads fail with Ably code
`50003` when durable continuity is degraded or reset-required.

### Register a device [#register-a-device]

```http
POST /apps/{app_id}/push/deviceRegistrations
Content-Type: application/json
```

```json
{
  "device_id": "ios-device-1",
  "client_id": "user-42",
  "platform": "apns",
  "provider_token": "provider-token",
  "metadata": {
    "app_version": "4.5.0"
  }
}
```

### Subscribe a device to a channel [#subscribe-a-device-to-a-channel]

```http
POST /apps/{app_id}/push/channelSubscriptions
Content-Type: application/json
```

```json
{
  "device_id": "ios-device-1",
  "channel": "orders",
  "client_id": "user-42"
}
```

### Publish push [#publish-push]

```http
POST /apps/{app_id}/push/publish
Content-Type: application/json
```

```json
{
  "recipients": [
    { "type": "channel", "channel": "orders" }
  ],
  "payload": {
    "title": "Order updated",
    "body": "Order ord_123 moved to packing",
    "data": { "order_id": "ord_123" }
  },
  "sync": false,
  "idempotency_key": "push-order-ord_123-packing"
}
```

Async push returns `202 Accepted` with a `publish_id`.

### Inspect publish status [#inspect-publish-status]

```http
GET /apps/{app_id}/push/publish/{publish_id}/status
```

Use this endpoint for operator tools, not user-facing busy loops.

## Complete endpoint reference [#complete-endpoint-reference]

This page explains the common server workflows. The complete route matrix, including health probes, stats, metrics, all push credential/template/device/subscription endpoints, presence history state, durable history repair, and annotation filters, is in the [HTTP endpoint reference](/docs/reference/http-endpoints).

## Error handling [#error-handling]

Treat HTTP status codes as authoritative:

| Status | Meaning                                      |
| ------ | -------------------------------------------- |
| `202`  | Async work accepted, usually push delivery.  |
| `400`  | Invalid request shape or unsupported option. |
| `401`  | Signature or key failure.                    |
| `403`  | App or feature disabled.                     |
| `404`  | Resource does not exist.                     |
| `413`  | Payload too large.                           |
| `429`  | Rate limited.                                |
| `5xx`  | Server or backend dependency failure.        |


# Indestructibility simulator (/docs/server/indestructibility-simulator)



The Sockudo simulator is a seed-replayable disaster lab for durable Protocol V2 state. It runs
inside one process, drives real `sockudo-core` memory stores, calls the real `sockudo-push` memory
store and accept pipeline for durable push boundaries, injects node, network, IO, queue, and
fake-provider faults, and checks a shadow model
continuously.

Use it to prove that Sockudo-side durability contracts survive dropped live fanout, duplicated
delivery, node crashes, pauses, partitions, stream resets, retention purges, reconnect recovery,
storage-level dropped writes, torn multi-record writes, stale or corrupted reads, delayed commit
visibility, push queue loss, lost write responses, retryable provider failures, invalid tokens, and
repair.

The simulator deliberately does **not** claim that APNs, FCM, WebPush, HMS, WNS, browsers, mobile
OSes, radios, or devices will deliver every notification. External providers are modeled as
fallible systems. "Indestructible" means Sockudo does not lose, corrupt, double-apply logically, or
make unrecoverable an accepted Sockudo-side operation.

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- --seed 42 --ticks 10000
```

or through the Makefile wrapper:

```bash
make simulator SIM_SEED=42 SIM_TICKS=10000
```

For a disaster-heavy JSON profile:

```bash
make simulator-disaster SIM_SEED=12648430 SIM_TICKS=50000
```

For VOPR-style randomized distributions and liveness checks:

```bash
make simulator-swarm SIM_SEED=3735928559 SIM_TICKS=50000
make simulator-liveness SIM_SEED=3735928559 SIM_TICKS=50000
```

Every failure prints a replay command with the seed. Keep the seed, tick count, and fault arguments
unchanged when reducing a failure.

## What It Exercises [#what-it-exercises]

The simulator covers these high-value durability surfaces:

| Surface                | Oracle                                                                                                                                                                                                                                                                                                                         |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Protocol delivery      | V1 renderings are serialized through `sockudo-protocol` and must not contain V2-only fields; V2 renderings must preserve continuity fields and strip internal idempotency keys.                                                                                                                                                |
| Durable history        | Reserved serials, retained rows, page cursors, stream inspection, retention purges, and reset behavior match the shadow model.                                                                                                                                                                                                 |
| Connection recovery    | Simulated V2 clients recover dropped live fanout from durable history unless retention has legitimately truncated the gap; stale and corrupt storage reads must fail closed or return a valid prefix, and recovery cursors must remain contiguous.                                                                             |
| Versioned messages     | Delivery serial replay is contiguous, version chains page in both directions, cursors round-trip through JSON, latest reads match the shadow, version/history/delivery serials stay monotonic, and `latest_by_history` preserves original history ordering.                                                                    |
| Presence history       | First-join/last-leave edge decisions, retained events, cursor paging, stream inspection, and reconstructed snapshots match the shadow model under deterministic reconnect churn.                                                                                                                                               |
| Push workflow          | Device registration, channel subscription, scheduled-job storage, publish acceptance, initial status, publish logs, and publish-id idempotency go through real `sockudo-push` memory store/pipeline APIs. The simulator then faults modeled worker queues, status transitions, and provider outcomes and checks they converge. |
| Crash/restart recovery | Restarted nodes immediately re-read durable history, versions, presence history, and push status/log/idempotency state through the same safety oracles.                                                                                                                                                                        |
| Rolling upgrades       | Opt-in upgrade runs restart nodes one by one with mixed legacy/target feature gates, schema activation, V1/V2 wire checks, before/during/after durable data counters, and push status oracles.                                                                                                                                 |

It does not replace live multi-node integration or Jepsen-style external testing. It is the fast,
deterministic inner loop for the durable primitives those tests rely on.

## Outside-In Binary Chaos Harness [#outside-in-binary-chaos-harness]

Sockudo also has a separate **outside-in chaos** runner for manual local experiments against actual
`sockudo` server binaries and real client traffic. It is inspired by TigerBeetle Vortex in spirit,
but it is not deterministic, not a CI gate, and not part of the simulator's seed-replayable model.

```bash
make binary-chaos CHAOS_SEED=42 CHAOS_DURATION_MS=12000
```

The Make target builds `target/debug/sockudo`, starts it as a child process with a generated local
memory-backed config, connects clients over WebSocket, publishes signed HTTP API events, and injects
bounded outside-in faults:

| Fault surface                  | Current behavior                                                                                                                                                                                                        |
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Process restart/kill           | Sends `SIGKILL`, restarts the binary, waits for `/up/app-id`, and reconnects clients.                                                                                                                                   |
| Network delay/drop/duplication | By default runs clients and HTTP publishes through a local TCP proxy that applies seeded connection/chunk delay and stream drops. Duplicate publish probes remain application-level and reuse the same idempotency key. |
| Config changes/reloads         | Writes a second config and applies it by process restart because Sockudo loads config at startup.                                                                                                                       |
| Push-provider fake outcomes    | Optionally starts seeded `scripts/push-mock-provider.mjs`, points FCM worker env vars at it when the binary is built with push/monolith features, and can require the mock provider to produce a seeded fake outcome.   |
| Recovery/reconnect             | Enables Protocol V2 connection recovery, drops client connections, reconnects, and records resume success/failure counters.                                                                                             |

Every run writes an artifact directory under `target/outside-in-chaos/<timestamp>-seed-<seed>/`
containing:

| File                         | Purpose                                                                                                                                                             |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `artifact.json`              | Seed, command, replay command with effective flags, generated config paths, fault timeline, counters, publish results, provider metrics, and recovery observations. |
| `sockudo-chaos.toml`         | Initial generated server config.                                                                                                                                    |
| `sockudo-chaos-restart.toml` | Restart config used for the config-change fault.                                                                                                                    |
| `sockudo.log`                | Captured server stdout/stderr.                                                                                                                                      |
| `push-provider.log`          | Captured mock provider stdout/stderr when enabled.                                                                                                                  |

Push-provider exercise is opt-in because it requires a compatible feature build:

```bash
make binary-chaos-push CHAOS_SEED=42 CHAOS_DURATION_MS=12000
```

For network experiments, the default `--network-fault-mode proxy` is unprivileged and local-only.
Use `--network-fault-mode publisher` to limit faults to HTTP publisher delay/drop behavior, or
`--network-fault-mode off` to run only process/client/config/push chaos. The replay command in
`artifact.json` reuses the same seed, ports, timings, probabilities, and push-provider settings;
outside-in wall-clock scheduling can still vary between runs.

Sockudo's provider dispatch layer rejects private/local provider URLs by design. If that guard
prevents the server process from calling the local mock provider, the artifact records the guarded
dispatch attempt and the harness performs a direct mock-provider probe so the fake provider outcome
is still captured with the same seed.

Keep this harness local/manual only. Do not add CI, scheduled jobs, or GitHub Actions for it.

## Real-Code Boundary [#real-code-boundary]

The simulator is intentionally not a full server-in-a-process. Its current real-code boundary is:

| Path               | Boundary                                                                                                                                                                                                                                   |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| History            | Real `MemoryHistoryStore` reservations, appends, reads, cursors, stream inspection, purges, and resets.                                                                                                                                    |
| Versioned messages | Real `MemoryVersionStore` delivery reservations, version appends, replay, latest reads, paged chain reads, and `latest_by_history`.                                                                                                        |
| Presence history   | Real `MemoryPresenceHistoryStore` transition recording, dedupe, reads, cursors, stream inspection, snapshot reconstruction, and resets; the simulator separately models active connection counts for first-join/last-leave edge decisions. |
| Push durable state | Real `MemoryPushStore` device, subscription, schedule, status, publish-log, and idempotency APIs, with publish acceptance through `PushPipeline::accept_publish`.                                                                          |

Still modeled: live socket fanout, horizontal transport timing, replay-buffer hot recovery,
operator routing, push worker queues after accept, provider dispatch, provider feedback, and repair
timing. Those modeled paths exist to make deterministic faults cheap and replayable; they are
checked against the durable stores above rather than replacing them.

## Deterministic IO Model [#deterministic-io-model]

The simulator keeps deterministic IO under `crates/sockudo-simulator/src/io.rs`. The production
server does not use these wrappers; they are simulator-local boundaries around modeled IO and real
memory-store calls.

| Wrapper                       | Controls                                                                                                                                                                                                                                  |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DeterministicClock`          | The logical tick and durable timestamps. Timer advancement during quiesce jumps through due fanout and push work instead of sleeping wall-clock time.                                                                                     |
| `DeterministicFaultScheduler` | Seeded random choices, operation trace entries, and named fault rolls such as `fanout_drop`, `queue.ack_lost`, and `storage.push_publish.write_after_commit`.                                                                             |
| Schedule harness              | A separate seed domain for simulator-executed task ordering: top-level tick tasks, due fanout deliveries, due push schedules, and ready push queue items.                                                                                 |
| `DeterministicNetwork`        | Modeled live fanout delivery order, delivery tick, dropped messages, duplicates, and delayed delivery.                                                                                                                                    |
| `DeterministicQueue`          | Modeled push worker queue order, delayed delivery, redelivery, lease timeout duplication, and repair requeueing.                                                                                                                          |
| `DeterministicStorage`        | Storage read/write behavior around real memory-store calls: dropped writes before reservation, torn multi-record writes, fail-after-commit, lost responses, stale reads, corrupted reads, delayed commit visibility, and backend outages. |

The final JSON report includes a bounded `io_trace` with the most recent operation, fault,
scheduling, and logical-timer decisions. That trace is intentionally compact: it is for replay
orientation, not a full event log. Use the seed, tick count, mode, fault flags, and workload
weights as the canonical replay input.

To inspect scheduler and timer decisions for a seed:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 42 \
  --ticks 10000 \
  --json | sed '1d' | jq -r '.io_trace[]'
```

To replay a saved failure capsule and print the same trace window:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file /tmp/sockudo-sim-failure.json \
  --json | jq -r '.[0].io_trace[]'
```

Current limitations:

| Limitation                                 | Why                                                                                                                                                                |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| No real sockets or multi-process transport | Fanout is a deterministic in-process queue so drops, duplicates, and delays are exactly replayable.                                                                |
| No wall-clock timer waits                  | Logical ticks make timer behavior reproducible and keep shrink/replay fast.                                                                                        |
| No real provider calls                     | Provider responses are seeded outcomes so retry, invalid-token, and lost-response paths can be covered without external services.                                  |
| No real production storage backend faults  | The simulator wraps real memory stores and injects storage behavior at the simulator boundary; Redis/Postgres/disk failures still need integration or chaos tests. |
| Bounded trace memory                       | `recent_trace`, `push.recent_trace`, and `io_trace` keep recent context only; failure capsules and replay commands should preserve the full seed/config.           |

## VOPR-Style Maturity [#vopr-style-maturity]

The simulator has three layers:

| Layer          | Command                   | Purpose                                                                                                                                |
| -------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| Fixed safety   | `make simulator`          | Replays one fixed profile. Use this for direct bug reproduction.                                                                       |
| Swarm safety   | `make simulator-swarm`    | Randomizes topology, workload weights, page sizes, retention, and fault distributions from the seed before checking safety invariants. |
| Swarm liveness | `make simulator-liveness` | Runs the same deterministic swarm profile, then asserts bounded convergence after workload generation stops.                           |

Swarm profiles are deterministic: the seed generates both the simulator run and the profile
distribution. A failing swarm seed can be replayed without recording a separate config.

Safety mode answers: did accepted Sockudo-side work remain correct? Liveness mode adds: after
faults stop, did durable work become available and drain within a bounded number of ticks?

Use failure capsules when saving CI failures:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 42 \
  --ticks 50000 \
  --swarm \
  --failure-artifact /tmp/sockudo-sim-failure.json
```

Use shrink mode to reduce the replay before debugging. The shrinker first proves the original
config still fails, then deterministically tries smaller tick, operation, and fault prefixes,
topology reductions, and workload/fault profile simplifications. It accepts only candidates that
still fail and writes a smaller failure capsule:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file /tmp/sockudo-sim-failure.json \
  --shrink-failure \
  --shrink-output /tmp/sockudo-sim-failure.shrunk.json
```

The shrink command prints the exact replay command, and the artifact stores the same command in
`replayCommand`:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file /tmp/sockudo-sim-failure.shrunk.json
```

When shrinking a `--swarm` run directly from seed flags, the artifact also includes
`seedDerivedProfile`. That preserves the original seed-generated topology, workload, and fault
profile before any shrink simplifications, while replay continues to use the explicit shrunk
`config` from the capsule. Do not pass `--swarm` when replaying a failure capsule.

## Rolling Upgrade Replay [#rolling-upgrade-replay]

The simulator has an opt-in upgrade risk profile for production-style rolling changes. It keeps the
run deterministic while modeling:

| Upgrade surface           | Model                                                                                                                                                                                                 |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Rolling restarts          | Nodes are restarted one at a time into the target generation, with at least one live node left serving traffic. Durable recovery oracles run after each upgraded node comes back.                     |
| Mixed config/features     | Legacy nodes reject feature-gated versioned-message writes while target nodes accept them after the schema gate opens. Ordinary history, recovery, and push traffic continue through the mixed fleet. |
| V1/V2 compatibility       | Mixed V1/V2 simulated clients keep receiving live and recovered deliveries. V1 projections must strip V2-only fields, and V2 projections must preserve continuity and upgrade metadata.               |
| Durable before/after data | History, versioned messages, and push publishes are counted by `before_feature_change`, `during_rolling_change`, and `after_feature_change` phases in the final JSON.                                 |
| Schema/version gates      | Versioned writes are rejected before schema activation. The upgrade oracle fails if any version record appears in the pre-activation phase.                                                           |
| Push status semantics     | Push publish status/log/idempotency oracles continue to run across all upgrade phases, and target nodes must use the target push-status semantics.                                                    |

Run the manual local wrapper:

```bash
make simulator-upgrade SIM_SEED=48879 SIM_TICKS=1200
```

Or replay the same profile directly and inspect upgrade coverage:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 48879 \
  --ticks 1200 \
  --upgrade-risk-profile \
  --upgrade-require-coverage \
  --json | sed '1d' | jq '.upgrade, .protocol_oracles, .push'
```

Use a compressed schedule when reducing a failure or forcing the rollout to happen early in a short
manual run:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 43981 \
  --ticks 320 \
  --nodes 4 \
  --upgrade-risk-profile \
  --upgrade-require-coverage \
  --upgrade-schema-prepare-tick 8 \
  --upgrade-start-tick 12 \
  --upgrade-schema-activate-tick 18 \
  --upgrade-restart-duration-ticks 2 \
  --upgrade-interval-ticks 8 \
  --json
```

To preserve a failing upgrade run for exact replay:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 48879 \
  --ticks 1200 \
  --upgrade-risk-profile \
  --upgrade-require-coverage \
  --failure-artifact /tmp/sockudo-upgrade-failure.json

cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file /tmp/sockudo-upgrade-failure.json \
  --json
```

The replay command printed on invariant failure includes the upgrade timing flags. The full failure
artifact remains the canonical replay input when workload weights, fault probabilities, topology, or
retention settings were changed in addition to the upgrade flags.

## Fault Model [#fault-model]

The default workload is a weighted deterministic generator, not an ad hoc random branch in the
runner. Each tick samples one action from these weights:

| Flag                        | Default | Action                                                      |
| --------------------------- | ------: | ----------------------------------------------------------- |
| `--weight-publish`          |    `35` | Append a durable history message and fan it out to clients. |
| `--weight-create-versioned` |    `15` | Create a new mutable Protocol V2 message.                   |
| `--weight-mutate-versioned` |    `20` | Update, append to, or delete an existing mutable message.   |
| `--weight-presence`         |    `20` | Record a presence join/leave transition.                    |
| `--weight-recovery`         |     `7` | Probe durable recovery for one simulated client/channel.    |
| `--weight-purge`            |     `2` | Purge an older retained history prefix.                     |
| `--weight-push-register`    |     `8` | Register or update a push device.                           |
| `--weight-push-delete`      |     `3` | Delete a push device and its subscriptions.                 |
| `--weight-push-subscribe`   |     `8` | Subscribe a device to a channel.                            |
| `--weight-push-unsubscribe` |     `5` | Unsubscribe a device from a channel.                        |
| `--weight-push-publish`     |    `12` | Accept a push publish and drive durable fanout.             |
| `--weight-push-scheduled`   |     `4` | Store and release a scheduled push publish.                 |
| `--weight-push-feedback`    |     `2` | Replay duplicate provider feedback.                         |
| `--weight-push-repair`      |     `2` | Run durable push queue repair.                              |
| `--weight-oracle`           |     `1` | Force a full oracle sweep outside the periodic cadence.     |

`--swarm` replaces these fixed defaults with seed-derived distributions. Use fixed weights when
reproducing a concrete bug and swarm weights when hunting for new combinations.

Set a weight to `0` to disable that action family. At least one action weight must be non-zero.
The final JSON report includes the weights and selected action counts so coverage is visible for a
seed.

The default fault model injects:

| Flag                               |  Default | Meaning                                                                                                     |
| ---------------------------------- | -------: | ----------------------------------------------------------------------------------------------------------- |
| `--drop-prob`                      |   `0.08` | Probability that a live fanout message is dropped before reaching a simulated client.                       |
| `--duplicate-prob`                 |   `0.03` | Probability that a fanout message is duplicated.                                                            |
| `--max-delay-ticks`                |     `12` | Maximum deterministic fanout delay.                                                                         |
| `--crash-prob`                     |  `0.002` | Per-tick probability that one live node crashes.                                                            |
| `--restart-prob`                   |  `0.020` | Per-tick probability that one crashed node restarts.                                                        |
| `--pause-prob`                     |  `0.002` | Per-tick probability that one live node pauses without losing durable state.                                |
| `--resume-prob`                    |  `0.030` | Per-tick probability that one paused node resumes.                                                          |
| `--partition-prob`                 |  `0.003` | Per-tick probability that one live node is isolated from traffic.                                           |
| `--heal-prob`                      |  `0.020` | Per-tick probability that one partitioned node heals.                                                       |
| `--slow-prob`                      |  `0.003` | Per-tick probability that one node adds deterministic fanout delay.                                         |
| `--stale-prob`                     |  `0.002` | Per-tick probability that one node rejects traffic as stale until it catches up.                            |
| `--stream-reset-prob`              | `0.0005` | Per-tick probability of an operator history/presence stream reset.                                          |
| `--storage-drop-write-prob`        |  `0.004` | Probability a core durable write is dropped before taking a reservation or changing the store.              |
| `--storage-torn-write-prob`        |  `0.002` | Probability a multi-record versioned-message operation commits history but tears before the version record. |
| `--storage-stale-read-prob`        |  `0.006` | Probability a recovery read sees an older visible history prefix.                                           |
| `--storage-corrupt-read-prob`      |  `0.002` | Probability a recovery read is treated as corrupted and ignored fail-closed.                                |
| `--storage-delayed-commit-prob`    |  `0.006` | Probability a committed core durable write is hidden from recovery reads until a later deterministic tick.  |
| `--storage-max-commit-delay-ticks` |      `8` | Maximum delayed commit visibility window for core durable writes.                                           |
| `--queue-produce-lost-prob`        |  `0.015` | Probability that a push queue produce disappears after durable state was written.                           |
| `--queue-ack-lost-prob`            |  `0.010` | Probability that a consumed push queue item is redelivered because ack was lost.                            |
| `--queue-lease-timeout-prob`       |  `0.010` | Probability that a queue lease times out and duplicates work.                                               |
| `--write-fail-before-commit-prob`  |  `0.006` | Probability a push/device/subscription write fails before commit.                                           |
| `--write-fail-after-commit-prob`   |  `0.004` | Probability a write commits durably but reports failure.                                                    |
| `--response-lost-prob`             |  `0.006` | Probability a committed write loses its response.                                                           |
| `--read-stale-prob`                |  `0.010` | Probability a push worker sees a stale/empty read and fails closed.                                         |
| `--provider-retryable-prob`        |  `0.100` | Probability a fake provider returns quota/timeout/transient failure.                                        |
| `--provider-reject-prob`           |  `0.040` | Probability a fake provider permanently rejects a delivery.                                                 |
| `--provider-invalid-token-prob`    |  `0.025` | Probability a fake provider reports an invalid or expired token.                                            |
| `--provider-lost-response-prob`    |  `0.020` | Probability a fake provider accepted externally but Sockudo lost the response.                              |

Rejected operations are part of the model: a workload request may route to a crashed or partitioned
node and fail before it reaches durable storage. The shadow only advances after durable Sockudo
stores accept the operation.

Storage faults are injected at simulator-local boundaries. Dropped core writes are rejected before
serial reservations so the model does not create artificial gaps. Torn writes are limited to
multi-record operations where a safe prefix can exist, such as a versioned-message history row
committing before its version row. Delayed commits are tracked as per-channel visible prefixes; a
recovery read may lag, but it must never expose a gap or invalid cursor. Corrupted reads fail closed
and are retried by later recovery probes.

## Push Oracles [#push-oracles]

Push disasters are checked against Sockudo-side invariants:

| Oracle               | Meaning                                                                                                                    |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Durable acceptance   | Every accepted push publish has a durable status row and durable publish-log event.                                        |
| Idempotency          | Duplicate publish keys map back to the same logical publish and do not create a second effect.                             |
| Queue repair         | Lost publish-log and delivery queue messages are recreated from durable state by idempotent repair scans.                  |
| Provider convergence | Each planned eligible target eventually becomes accepted, retryable, rejected, expired, cancelled, or dead-lettered.       |
| Status transitions   | Modeled publish lifecycle transitions never move backward, never leave a terminal state, and agree with terminal counters. |
| Retry bounds         | Retryable outcomes schedule bounded retries and then converge.                                                             |
| Token invalidation   | Invalid/unregistered/expired tokens are marked terminal and removed from future eligible sends.                            |
| Counters             | Status counters never exceed planned logical deliveries and terminal statuses match terminal counters.                     |
| Safety after removal | No provider send is allowed after the model-visible state says the device is unsubscribed, deleted, or invalid.            |

## CI Profile [#ci-profile]

A good pull-request smoke profile is:

```bash
cargo test -p sockudo-simulator
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --seed 12648430 \
  --ticks 5000 \
  --json
```

To replay the checked-in seed corpus:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file crates/sockudo-simulator/corpus/disaster-seeds.json \
  --ticks 5000 \
  --json
```

To replay the storage-fault corpus:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file crates/sockudo-simulator/corpus/storage-faults.json \
  --json
```

For nightly burn-in, run multiple fixed seeds and keep any new failing seed as a regression test:

```bash
for seed in 1 2 3 4 5 12648430 3735928559; do
  cargo run -p sockudo-simulator --bin sockudo-sim -- --seed "$seed" --ticks 50000
done
```

For nightly swarm burn-in:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file crates/sockudo-simulator/corpus/swarm-seeds.json \
  --ticks 50000 \
  --swarm
```

For liveness-focused nightly burn-in:

```bash
cargo run -p sockudo-simulator --bin sockudo-sim -- \
  --corpus-file crates/sockudo-simulator/corpus/swarm-seeds.json \
  --ticks 50000 \
  --mode liveness \
  --swarm
```

## Reading Failures [#reading-failures]

Invariant failures include the seed and simulator tick:

```text
sockudo-sim: FAILED - reproduce with --seed 42
simulator invariant failed at tick 812 (seed=42): history serial gap on sim-channel-1: 41 then 43
replay command:
  cargo run -p sockudo-simulator --bin sockudo-sim -- --seed 42 --ticks 5000 --mode safety
config_json:
  {"seed":42,"ticks":5000,...}
recent trace:
  tick=807 push queue produce lost for PublishLog { publish_id: "push-00000000000000000042" }
  tick=811 node 2 restarted
```

Failure artifacts written with `--failure-artifact` include the full config and can be replayed
directly with `--corpus-file`. Shrunk artifacts add a `shrink` block with the original and reduced
tick, operation, and fault counts plus each accepted simplification step.

Treat these as product bugs unless the invariant is deliberately stronger than the subsystem's
documented contract. If the contract changes, update the simulator shadow and this page in the same
change.

## JSON Shape [#json-shape]

The final JSON includes top-level core counters, a `protocol_oracles` object with V1/V2 delivery,
cursor, identity, monotonicity, rewind, and presence-edge check counts, storage fault counters,
restart recovery check counts, a nested `push` object with durable status/log counts, status
transition counts, provider results, queue losses, repair counts, phase counts for upgrade runs, a
nested `upgrade` object with rolling restart/schema gate/oracle coverage counters, recent push trace
entries, and top-level `io_trace` entries for recent deterministic operation/fault decisions. Keep sample
outputs with the exact seed and tick count that produced them so they remain replayable. A compact
example generated with `--seed 12648430 --ticks 16 --json` lives at
`crates/sockudo-simulator/examples/disaster-report.example.json`.

## Extending It [#extending-it]

Add new workloads by following the existing pattern:

1. Generate the operation through the deterministic scheduler or workload generator.
2. Apply it to the real Sockudo store or subsystem first.
3. Advance the shadow only after the real operation succeeds.
4. Add a cheap per-run oracle and a quiesce oracle.
5. Add or pin a seed that fails before the fix and passes after it.


# MCP server (/docs/server/mcp)



Sockudo ships a Model Context Protocol (MCP) server so agents such as Claude can inspect and
operate a deployment through a typed, permissioned tool surface instead of raw HTTP calls. The
protocol layer is the official [`rmcp`](https://crates.io/crates/rmcp) SDK; Sockudo adds signed
API access, scopes, tools, resources, and prompts.

Two deployment shapes share the same code:

| Shape                                      | Transport to Sockudo                                                       | MCP transport                                        | Use when                                      |
| ------------------------------------------ | -------------------------------------------------------------------------- | ---------------------------------------------------- | --------------------------------------------- |
| Embedded (`sockudo` binary, `mcp` feature) | In-process: the server drives its own API router with self-signed requests | Streamable HTTP on `/mcp` (shared or dedicated port) | Hosted agents, shared team access, production |
| Standalone (`sockudo-mcp` binary)          | Signed HTTP to a remote deployment                                         | stdio or Streamable HTTP                             | Claude Desktop, Claude Code, local IDEs       |

Every tool call passes through the same validation, idempotency, rate limits, metrics, feature
gates, and role restrictions as an external HTTP API caller. Tool results are the documented HTTP
API response shapes, so anything learned from the [HTTP API](/docs/server/http-api) applies.

## Enable the embedded server [#enable-the-embedded-server]

Build with the feature and configure `[mcp]`:

```bash
cargo build -p sockudo --release --features "v2,mcp,redis,postgres"
```

```toml
[mcp]
enabled = true
path = "/mcp"
# port = 6100                 # optional dedicated listener (host defaults to the server host)
allowed_hosts = []            # empty = accept any Host (fine behind a trusted proxy)
allowed_origins = []          # browser origins; empty disables Origin checks
rate_limit_per_minute = 600   # per token; 0 disables
request_timeout_ms = 30000
session_ttl_seconds = 1800

[[mcp.tokens]]
name = "ops-agent"
token = "${SOCKUDO_MCP_OPS_TOKEN}"   # >= 16 chars; use env interpolation
scopes = ["read", "write"]
apps = ["*"]

[[mcp.tokens]]
name = "readonly-dashboard"
token = "${SOCKUDO_MCP_RO_TOKEN}"
scopes = ["read"]
apps = ["app-1"]
```

Validation refuses to start when `enabled = true` without tokens unless `allow_anonymous = true`
(development only). `MCP_TOKEN`, `MCP_TOKEN_NAME`, `MCP_TOKEN_SCOPES`, and `MCP_TOKEN_APPS` add a
token from the environment without touching the file; see
[Environment variables](/docs/reference/environment-variables#mcp).

Clients connect with a bearer token:

```bash
curl -sS http://127.0.0.1:6001/mcp \
  -H "Authorization: Bearer $SOCKUDO_MCP_OPS_TOKEN" \
  -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'
```

Claude Code:

```bash
claude mcp add --transport http sockudo https://rt.example.com/mcp \
  --header "Authorization: Bearer $SOCKUDO_MCP_OPS_TOKEN"
```

Claude API (MCP connector):

```json
{
  "mcp_servers": [{ "type": "url", "url": "https://rt.example.com/mcp", "name": "sockudo",
                    "authorization_token": "<token>" }],
  "tools": [{ "type": "mcp_toolset", "mcp_server_name": "sockudo" }]
}
```

## Standalone binary [#standalone-binary]

```bash
cargo install --path crates/sockudo-mcp --features cli
# stdio for Claude Desktop / Claude Code
sockudo-mcp --url https://rt.example.com --app app-1:key:secret --scopes read,write
claude mcp add sockudo -- sockudo-mcp --url https://rt.example.com --app app-1:key:secret
# Streamable HTTP with tokens
sockudo-mcp --transport http --listen 127.0.0.1:6100 \
  --token 'ops/read+write=<32+ char token>' --url https://rt.example.com --app app-1:key:secret \
  --metrics-url http://rt.example.com:9601/metrics
```

Credentials may also come from `SOCKUDO_URL`, `SOCKUDO_MCP_APPS` (`id:key:secret,...`), or
`SOCKUDO_APP_ID` / `SOCKUDO_APP_KEY` / `SOCKUDO_APP_SECRET`. Logs go to stderr; stdout is the
protocol channel.

## Scopes and safety [#scopes-and-safety]

| Scope   | Grants                                                            | Examples                                                                                                                                                      |
| ------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `read`  | Inspection                                                        | `sockudo_list_channels`, `sockudo_get_history`, `sockudo_server_stats`, `sockudo_server_metrics`                                                              |
| `write` | Publishing and mutation (implies `read`)                          | `sockudo_trigger_event`, `sockudo_update_message`, `sockudo_publish_annotation`, `sockudo_push_publish`, `sockudo_sign_channel_auth`                          |
| `admin` | Destructive and connection-affecting operations (implies `write`) | `sockudo_terminate_user_connections`, `sockudo_reset_history`, `sockudo_purge_history`, `sockudo_revoke_capability_tokens`, `sockudo_replay_push_dead_letter` |

* `tools/list` only returns tools the token may call; calling a hidden tool returns JSON-RPC
  `-32003`.
* Destructive tools require `confirm: true` and a `reason`, which the server records.
* Tokens carry an app allow-list; requests for other apps fail with `-32003`.
* App secrets never leave the server. `sockudo_list_apps` and `sockudo_get_app` return keys and
  sanitized policy only; webhook headers are redacted.
* Every call emits an audit log line on target `sockudo_mcp::audit` with principal, tool, app,
  channel, outcome, and latency.
* Upstream API errors are returned as `isError` results carrying the server's JSON error body so
  the agent can read `code` and `error` and recover.

## Tools [#tools]

| Area                   | Tools                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Discovery and server   | `sockudo_list_apps`, `sockudo_get_app`, `sockudo_server_info`, `sockudo_server_health`, `sockudo_server_accept_traffic`, `sockudo_server_stats`, `sockudo_server_usage`, `sockudo_server_metrics`                                                                                                                                                                                                                                        |
| Channels               | `sockudo_list_channels`, `sockudo_get_channel`, `sockudo_get_presence_users`                                                                                                                                                                                                                                                                                                                                                             |
| Publish                | `sockudo_trigger_event`, `sockudo_trigger_batch_events`                                                                                                                                                                                                                                                                                                                                                                                  |
| Durable history        | `sockudo_get_history`, `sockudo_get_history_state`, `sockudo_reset_history`, `sockudo_purge_history`                                                                                                                                                                                                                                                                                                                                     |
| Versioned messages     | `sockudo_get_message`, `sockudo_list_message_versions`, `sockudo_update_message`, `sockudo_delete_message`, `sockudo_append_message`                                                                                                                                                                                                                                                                                                     |
| Annotations            | `sockudo_list_annotations`, `sockudo_publish_annotation`, `sockudo_delete_annotation`                                                                                                                                                                                                                                                                                                                                                    |
| Presence history       | `sockudo_get_presence_history`, `sockudo_get_presence_history_state`, `sockudo_get_presence_snapshot`, `sockudo_reset_presence_history`                                                                                                                                                                                                                                                                                                  |
| Connections and tokens | `sockudo_terminate_user_connections`, `sockudo_force_reconnect_user`, `sockudo_revoke_capability_tokens`                                                                                                                                                                                                                                                                                                                                 |
| Push                   | `sockudo_push_publish`, `sockudo_push_batch_publish`, `sockudo_push_publish_status`, `sockudo_list_push_devices`, `sockudo_get_push_device`, `sockudo_list_push_channel_subscriptions`, `sockudo_list_push_subscription_channels`, `sockudo_list_push_dead_letters`, `sockudo_replay_push_dead_letter`, `sockudo_delete_push_scheduled_job`, `sockudo_list_push_credentials`, `sockudo_list_push_templates`, `sockudo_get_push_template` |
| Auth helpers           | `sockudo_sign_channel_auth`, `sockudo_sign_user_auth`, `sockudo_verify_webhook_signature`                                                                                                                                                                                                                                                                                                                                                |

Hide tools with `disabled_tools = ["sockudo_purge_history"]`. Tools that hit a feature the server
does not enable (for example push without the `push` feature) return the server's
`feature_disabled` or `404` error rather than failing at startup.

## Resources and prompts [#resources-and-prompts]

Resources use the `sockudo://` scheme: `server/info`, `server/health`, `server/stats`, `apps`,
`apps/{app_id}`, `apps/{app_id}/channels[/{channel}[/history|/presence|/messages/{serial}]]`,
plus embedded references `docs/http-api`, `docs/channels`, and `docs/operations`. Prompts
`sockudo_debug_channel`, `sockudo_incident_triage`, `sockudo_design_realtime_feature`, and
`sockudo_audit_app_security` walk an agent through common workflows. Argument completion is
available for `app_id` and `channel`.

## Observability [#observability]

Prometheus families (with the configured prefix):

| Metric                 | Labels                    | Meaning                                                                                             |
| ---------------------- | ------------------------- | --------------------------------------------------------------------------------------------------- |
| `mcp_requests_total`   | `port`, `outcome`         | Protocol requests by outcome (`tools_call`, `resources_read`, `rate_limited`, `unauthorized`, ...). |
| `mcp_tool_calls_total` | `port`, `tool`, `outcome` | Tool calls by tool and outcome (`ok`, `upstream_4xx`, `forbidden_scope`, `timeout`, ...).           |
| `mcp_tool_latency_ms`  | `port`, `tool`            | Tool execution latency.                                                                             |

## Testing [#testing]

* `cargo test -p sockudo-mcp --all-features` runs protocol, signing, catalog, and Streamable HTTP
  transport tests; `cargo test -p sockudo --features mcp mcp::` drives the in-process transport
  through the real API middleware.
* `make mcp-smoke` builds both binaries, starts a scratch server from `tests/mcp/config.toml`, and
  runs `tests/mcp/smoke_http.py` (embedded endpoint, admin and read-only tokens) and
  `tests/mcp/smoke_stdio.py` (standalone binary over stdio).
* For interactive checks use `npx @modelcontextprotocol/inspector` against `/mcp` with the
  `Authorization: Bearer` header, or register the endpoint in Claude Code and call
  `sockudo_server_info`.

## Performance notes [#performance-notes]

* The embedded transport never touches a socket: requests are signed and dispatched to the router
  with `tower::Service::oneshot`. Cost per call is one HMAC-SHA256, one MD5 for POST bodies, and the
  normal handler work.
* `tools/list` responses are prebuilt once per scope combination and served as an `Arc` clone.
* Responses are passed through as the server's JSON bytes; only object bodies are parsed once more
  to populate `structuredContent`.
* Sessions are managed by `rmcp` with idle expiry (`session_ttl_seconds`); the `2026-07-28`
  protocol revision is served statelessly.


# Mutable messages (/docs/server/mutable-messages)



Mutable messages are an existing Protocol V2 subsystem. They let one logical message evolve while
preserving an ordered version log for history and recovery.

## Actions [#actions]

| Action    | Event                     | Meaning                                                               |
| --------- | ------------------------- | --------------------------------------------------------------------- |
| `create`  | `sockudo:message.create`  | Create the logical message and reserve its original history identity. |
| `update`  | `sockudo:message.update`  | Shallow patch `name`, `data`, `extras`, or clear fields.              |
| `delete`  | `sockudo:message.delete`  | Mark/delete visible fields while preserving the version chain.        |
| `append`  | `sockudo:message.append`  | Append a string fragment to existing string data.                     |
| `summary` | `sockudo:message.summary` | Emit a reduced summary/projection event.                              |

Wire constants live in `sockudo-protocol/src/versioned_messages.rs`; store semantics live in
`sockudo-core/src/versioned_messages.rs` and `sockudo-core/src/version_store.rs`.

At commit time Sockudo also records a protocol-neutral message envelope in the
version record and new history payloads. It preserves the client message ID,
publisher identity and timestamp, typed stream/history/delivery positions,
complete encoding chain, user-visible extras, and operation metadata. Older
stored payloads remain readable through the envelope migration fallback.

## Aggregation [#aggregation]

Sockudo stores each operation as a version. Latest reads and history substitution return the latest
visible state, not a raw operation log, when versioned messages are enabled.

* `create` establishes `message_serial` and original `history_serial`.
* `update` and `delete` use tri-state field semantics: keep, clear, replace.
* `append` concatenates string fragments and rejects empty/non-string append payloads.
* `latest_by_history` returns one latest version per logical message sorted by original history
  serial.
* Replay continuity requires contiguous delivery serials.

Every create or mutation commits through one `VersionStore` transaction. The
transaction validates the exact predecessor, reserves the next delivery
position, writes the version, and stores any idempotency receipt together. A
failed transaction exposes neither a version nor a delivery gap. Concurrent
appends retry from the newest aggregate, so each accepted fragment appears once
and in commit order. Accumulated-byte, append-count, open-stream, and terminal
state checks are part of that transaction; in-memory counters are observability
only.

`op_id` is scoped by app, channel, message, and action. Replaying the same ID
with the same canonical operation returns the original acknowledgement without
fanout. Reusing it with a different operation is an idempotency conflict.
Memory, PostgreSQL, MySQL, DynamoDB, ScyllaDB, and SurrealDB implement the same
compare-and-apply contract.

## HTTP surfaces [#http-surfaces]

Signed app HTTP auth can create mutable messages through `/events`. Mutation and read surfaces are:

| Method | Path                                                                 |
| ------ | -------------------------------------------------------------------- |
| `POST` | `/apps/{appId}/channels/{channel}/messages/{messageSerial}/update`   |
| `POST` | `/apps/{appId}/channels/{channel}/messages/{messageSerial}/delete`   |
| `POST` | `/apps/{appId}/channels/{channel}/messages/{messageSerial}/append`   |
| `GET`  | `/apps/{appId}/channels/{channel}/messages/{messageSerial}`          |
| `GET`  | `/apps/{appId}/channels/{channel}/messages/{messageSerial}/versions` |

The optional Ably compatibility facade projects the pinned SDK shape over the
same service:

```http
PATCH /channels/{channel}/messages/{messageSerial}
Content-Type: application/json
```

```json
{
  "serial": "message-serial",
  "action": "message.append",
  "data": " fragment",
  "version": {
    "serial": "optional-version-serial",
    "clientId": "verified-or-server-attributed-client",
    "timestamp": 1710000000000,
    "description": "stream fragment",
    "metadata": { "source": "agent" }
  }
}
```

`message.update` and `message.delete` use the same envelope. JSON and MsgPack
requests, successes, and errors all use the negotiated compatibility encoder.
The path and body serial must identify the same logical message.

Mutation bodies may include `socket_id` and `client_id` to bind the operation to a connected V2
actor. Without `socket_id`, signed app-key HTTP requests are privileged server operations.
In `server_role = "api"` mode, `socket_id` cannot be verified (no local WebSocket connections)
and the request is rejected with HTTP 400. See [Server role](/docs/reference/configuration#server-role).

## Authorization [#authorization]

Capabilities are channel-pattern maps:

```json
{
  "message_append_own": ["private-ai:user-42:*"],
  "message_update_own": ["private-ai:user-42:*"],
  "message_delete_own": ["private-ai:user-42:*"],
  "history": ["private-ai:user-42:*"]
}
```

`*_any` grants are checked first. `*_own` grants require an identified actor, an identified
original creator, and a constant-time match between actor `client_id` and original creator
`client_id`. Trusted app-key server requests bypass as privileged operations.
For token-authenticated compatibility clients, an operation-supplied `clientId`
is metadata only after the authenticated actor and capability check succeeds;
it is never identity proof.

## Limits [#limits]

| Limit                                                  | Default                        |
| ------------------------------------------------------ | ------------------------------ |
| `versioned_messages.max_page_size`                     | `100`                          |
| `versioned_messages.retention_window_seconds`          | `0`, no expiry                 |
| `versioned_messages.purge_batch_size`                  | `1000`                         |
| `ai_transport.max_accumulated_message_bytes`           | `1048576`                      |
| `ai_transport.max_appends_per_message`                 | `4096`                         |
| `ai_transport.max_open_streaming_messages_per_channel` | `1024`                         |
| serial length                                          | `1..=128` bytes, no whitespace |

## Error parity [#error-parity]

Mutation errors use the same HTTP envelope as other app APIs:

```json
{ "error": "Connection is not allowed to append message on channel 'private-ai:x'", "code": "auth_failed", "status": 401 }
```

Feature-disabled, not-found, malformed-input, payload-too-large, and authorization failures are
kept distinct so SDKs can preserve behavior across languages.


# Observability (/docs/server/observability)



Observability is part of the runtime contract. A realtime system should tell operators when it is connected, degraded, delayed, retrying, or dropping work.

## Metrics endpoint [#metrics-endpoint]

```bash
curl http://127.0.0.1:9601/metrics
```

Scrape the endpoint with Prometheus and label metrics by environment, region, node, adapter, and app where possible. Sockudo emits metrics through the `metrics-rs` recorder and exposes them through the Prometheus exporter by default.

## TCP metrics exporter [#tcp-metrics-exporter]

For live debugging or sidecar consumers, Sockudo can also fan out metric events over TCP using `metrics-exporter-tcp`. This exporter streams protobuf-encoded metric events to connected clients; it is useful for local inspection and custom collectors, but Prometheus scraping should remain the primary production monitoring path.

```toml
[metrics.tcp_exporter]
enabled = true
host = "127.0.0.1"
port = 5000
buffer_size = 1024
```

The TCP exporter has bounded buffering by default. When buffers fill, event samples can be dropped to avoid blocking the server, so do not use it as the only source for alerts or SLO dashboards.

## Core signals [#core-signals]

| Area          | Watch                                                                       |
| ------------- | --------------------------------------------------------------------------- |
| Connections   | active sockets, connection attempts, disconnect reasons, heartbeat failures |
| Subscriptions | subscribe successes, auth failures, presence joins and leaves               |
| Publish       | accepted, rejected, idempotency hits, payload-too-large failures            |
| Fanout        | adapter publish latency, adapter receive latency, duplicate suppression     |
| Recovery      | resume successes, resume failures, replay counts, buffer misses             |
| History       | writes, reads, retention purges, cursor errors                              |
| Webhooks      | queued, delivered, failed, retried, dead-lettered                           |
| Push          | accepted, scheduled, dispatched, provider errors, publish status outcomes   |

When `ably-compat` is enabled, two lock-free operator snapshots supplement Prometheus:

* `/operator/stats/ably-runtime` reports queued messages/bytes, overflow and continuity loss,
  total encoded frames, shared data-projection encodes (`data_encoded`), fanout counts, replay
  source and recovery backend calls, duplicate suppression, backend/degraded/reset state, expiry,
  and derived-filter cache pressure. `data_encoded` excludes ACK, heartbeat, connection, and
  channel-control frames, so it is the counter used to verify one encode per active data format.
* `/operator/stats/aggregation` reports the bounded stats queue capacity, backlog, accepted,
  flushed, dropped, and failed observations.

These endpoints expose counters only. They never return credentials, device tokens, channel
payloads, or raw encrypted values.

## AI Transport [#ai-transport]

AI Transport observability is domain-blind. Sockudo reads only the well-known `extras.ai.transport` headers and never labels metrics by channel, run ID, message ID, or client ID.
Headers are validated once and then passed as a typed borrowed view to metric and webhook
classification. Empty `run-client-id` and `step-client-id` unknown-owner sentinels remain
unchanged on the wire but are treated as absent when deriving an identity.

Metrics exposed at `/metrics` include:

* `sockudo_ai_runs_started_total`
* `sockudo_ai_runs_ended_total{reason}`
* `sockudo_ai_cancel_signals_total`
* `sockudo_ai_active_streams`
* `sockudo_ai_stream_duration_seconds`
* `sockudo_ai_stream_bytes_total`
* `sockudo_ai_messages_rejected_total{code}`
* `sockudo_ai_messages_unparseable_total`
* `sockudo_appends_received_total`
* `sockudo_appends_delivered_total`
* `sockudo_rollup_ratio`
* `sockudo_flush_latency`
* `sockudo_history_recovery_success_total{source}`
* `sockudo_history_recovery_failures_total{code}`
* `sockudo_versioned_message_mutations_total{action,result}`
* `sockudo_versioned_message_retrieval_total{surface,result}`

The four production signals are run outcomes, stream rate, rejects by code, and rollup efficiency. Use `docs/public/grafana/ai-transport-observability.json` as the starting Grafana dashboard.

Active-stream gauges use exact atomic insertion/removal deltas. Tracker entries that never receive
a terminal event expire at the configured AI orphan TTL and decrement the same gauge exactly.
AI lifecycle webhooks are awaited into the configured bounded webhook queue, which supplies
backpressure and retry handling; Sockudo does not spawn an unbounded Tokio task per event or retain
AI events in the optional in-process batching vector.

Dashboard starter panels:

| Panel             | Metric                                                                                        |
| ----------------- | --------------------------------------------------------------------------------------------- |
| Active AI streams | `sockudo_ai_active_streams`                                                                   |
| Run outcomes      | `sockudo_ai_runs_started_total`, `sockudo_ai_runs_ended_total`                                |
| Rejects by code   | `sockudo_ai_messages_rejected_total`                                                          |
| Rollup efficiency | `sockudo_appends_received_total` vs `sockudo_appends_delivered_total`, `sockudo_rollup_ratio` |
| Rollup latency    | `sockudo_flush_latency`                                                                       |
| Recovery health   | `sockudo_history_recovery_success_total`, `sockudo_history_recovery_failures_total`           |
| Durable state     | `sockudo_history_degraded_channels`, `sockudo_history_reset_required_channels`                |
| Push backlog      | push queue/status/provider metrics from the push dashboard                                    |

For support escalation, capture the channel, wall-clock time window, verified `clientId`, and first error code before collecting logs. Do not ask customers for provider payloads unless the codec layer explicitly requires them.

## Persist completed AI runs [#persist-completed-ai-runs]

For Ably-style production persistence, persist completed runs from your backend instead of making Sockudo a domain store:

1. Enable the `ai_run_ended` webhook for the app.
2. On `reason: "complete"`, query the history endpoint for the channel around the webhook time.
3. Select messages with the same `extras.ai.transport.run-id`.
4. Store the reduced transcript in your application database.

```ts
app.post("/sockudo/webhooks", async (req, res) => {
  for (const event of req.body.events) {
    if (event.name !== "ai_run_ended" || event.reason !== "complete") continue;

    const history = await sockudo.channelHistory(event.channel, {
      limit: 1000,
      direction: "newest_first",
    });

    await storeCompletedRun({
      runId: event.run_id,
      channel: event.channel,
      items: history.items.filter((item) => {
        return item.extras?.ai?.transport?.["run-id"] === event.run_id;
      }),
    });
  }
  res.sendStatus(200);
});
```

## Logs [#logs]

Use structured logs for events that operators need to investigate:

```json
{
  "level": "warn",
  "target": "sockudo_push",
  "app_id": "app-id",
  "publish_id": "pub_123",
  "provider": "apns",
  "error": "BadDeviceToken"
}
```

Avoid logging secrets, raw auth signatures, provider tokens, or encrypted payloads.

## Grafana dashboards [#grafana-dashboards]

Recommended panels:

* active connections by node
* connection churn
* publish rate by app
* fanout latency histogram
* subscription auth failure rate
* recovery success ratio
* replay buffer pressure
* queue depth for webhooks and push
* push provider failure rate by provider
* APNs, FCM, Web Push latency by outcome
* AI run outcomes by reason
* AI stream duration and active streams
* AI reject codes and unparseable headers
* AI append rollup efficiency

## Alerts [#alerts]

Alert on symptoms operators can act on:

* readiness failures
* adapter connection loss
* rising publish failures
* high auth rejection rate after deploy
* recovery success ratio dropping
* history write failures
* webhook retry backlog
* push queue backlog
* push provider credential failures
* push delivery status callback failures

## Push status workflow [#push-status-workflow]

Push publishes are asynchronous. Store the `publish_id` returned by the API when a business workflow needs support visibility.

```ts
const response = await sockudo.publishPush({
  recipients: [{ type: "channel", channel: "orders" }],
  payload: { title: "Order updated", body: "Packed" },
  sync: false,
});

console.log(response.publish_id);
```

Then inspect status:

```bash
curl "https://realtime.example.com/apps/app-id/push/publish/pub_123/status"
```

Status records should be retained long enough for customer support and incident review.


# Push notifications (/docs/server/push-notifications)



Push notifications are a core part of Sockudo. WebSockets deliver realtime messages to connected clients; push notifications reach devices that are offline, backgrounded, rate-limited by the OS, or outside the active channel session.

```mermaid
flowchart LR
  D[Mobile/browser device] -->|provider token| B[Your backend]
  B -->|device registration| S[Sockudo push API]
  B -->|channel subscription| S
  A[Application event] -->|push publish or push rule| S
  S --> Q[(Push queue)]
  Q --> F[FCM]
  Q --> AP[APNs]
  Q --> W[Web Push]
  Q --> H[HMS]
  Q --> N[WNS]
  F --> D
  AP --> D
  W --> D
  H --> D
  N --> D
```

## Concepts [#concepts]

| Concept              | Meaning                                                                                     |
| -------------------- | ------------------------------------------------------------------------------------------- |
| Device registration  | A device record tied to a provider token and optional `client_id`.                          |
| Activation           | A safe workflow for clients to register or update devices through your backend.             |
| Channel subscription | A mapping between a device and a realtime channel for push targeting.                       |
| Credential           | Provider configuration for FCM, APNs, Web Push, HMS, or WNS.                                |
| Publish              | A push request accepted by Sockudo and fanned out asynchronously.                           |
| Publish status       | The operational record for accepted, scheduled, dispatched, failed, or cancelled push work. |

## Provider support [#provider-support]

| Provider | Platforms                                   | Credentials                                                | Notes                                                                              |
| -------- | ------------------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| FCM      | Android, web, cross-platform app backends   | Service account JSON and optional project ID               | Best default for Android and Firebase-backed apps.                                 |
| APNs     | iOS, iPadOS, macOS, watchOS, tvOS, visionOS | Team ID, key ID, bundle ID, `.p8` private key, environment | Use sandbox for development tokens and production for App Store/TestFlight tokens. |
| Web Push | Browsers and installed PWAs                 | VAPID subject, public key, private key                     | Payload size and browser support vary; store endpoint metadata with the device.    |
| HMS      | Huawei devices                              | App credentials                                            | Use when shipping to Huawei Mobile Services environments.                          |
| WNS      | Windows apps                                | WNS credentials                                            | Use for Microsoft Store/Windows notification channels.                             |

Sockudo normalizes admission, idempotency, queueing, retries, status retention, and feedback. Provider-specific limits still apply after the request leaves Sockudo.

## Admission readiness [#admission-readiness]

Sockudo fails closed before accepting publish work it already knows cannot be processed safely. The
push publish API requires a healthy push queue, production-safe storage, local pipeline workers, and
local provider-worker capability. Raw FCM/APNs/Web Push/HMS/WNS recipients require the matching
provider worker in this process. Channel, client, and device targets require at least one active
provider worker without scanning large channel subscription sets during admission; shard-path
publishes also require a local shard worker. The planner still resolves exact devices
asynchronously.

In `mode = "production"`, `storage_driver = "memory"` and `queue_driver = "memory"` are rejected
unless `allow_memory_drivers = true` or `PUSH_ALLOW_MEMORY_DRIVERS=true` is set explicitly. Memory
drivers are node-local and lose state on restart or dead-node loss.

Readiness failures return `503`. Queue pressure returns `429` with `Retry-After`. Quota rejection
persists a terminal `quota_exceeded` publish status and idempotency record without appending publish
log work or enqueueing the pipeline.

## Cluster queue delivery guarantees [#cluster-queue-delivery-guarantees]

Push queue delivery is at-least-once. Accepted publish work is durable before the API returns, and
every delivery result is idempotently applied by feedback workers. A provider send can still be
duplicated if a worker crashes after the provider accepts the request but before `push.results.v1`
is produced or before the original delivery batch is acknowledged. Sockudo keeps the duplicate
window bounded with stable delivery idempotency keys, deterministic retry context, and duplicate
feedback suppression, but it does not claim exactly-once provider delivery.

In monolith deployments, each node consumes only stages it can process. Planner, shard, feedback,
retry, and dead-letter stages are consumed only by nodes with the matching local worker enabled.
Provider delivery stages such as `push.delivery.fcm.v1` are consumed only by nodes whose provider
worker is active and credential-ready at boot. Changing provider capability or credentials requires
worker restart; stored credentials are reloaded when a supervised provider worker restarts.

External queue adapters pull broker messages into a bounded node-local ready/pending handoff before
Sockudo workers ack, nack, or dead-letter them. If the local worker does not finish before the local
lease timeout, the adapter requeues the envelope with backoff. Queue health and lag reported by the
generic queue manager adapter distinguish this node's local pull-ahead depth and oldest actionable
handoff age from broker-wide backlog, so operators can alert on Sockudo-owned worker staleness with
`sockudo_push_queue_oldest_age_seconds` and use backend-native tools for deeper broker visibility.
The monolith repair worker also scans durable publish-log entries that remain `queued` past
`push.repair_min_age_secs` and recreates missing `push.publish.v1` work if an accepted publish
lost its queue message before planning began.

## Retry and dead letters [#retry-and-dead-letters]

Provider throttling, 5xx responses, and transport failures are classified as retryable provider,
quota, or network failures. Feedback workers schedule retry entries on `push.retry.v1` with the
original delivery job context, a deterministic retry idempotency key, the next attempt number, the
first-attempt timestamp, retry deadline, and the provider error class. Retry scheduler workers
consume only due entries, respect provider `Retry-After` deadlines when configured, and otherwise
use bounded exponential backoff with deterministic jitter.

Retries stop at the earliest of `max_attempts`, `max_elapsed_secs`, or the publish/job
`expires_at_ms`. Exhausted retry work is emitted to `push.deadletters.v1` and the publish status
moves to `dead_lettered` unless some deliveries already succeeded, in which case it becomes
`partially_succeeded`. If the publish expiry passes before a retry can run, the publish moves to
`expired`. Pending retry work increments `retryScheduled`; retry dispatch increments
`retryAttempted`; `dispatched` counts terminal provider outcomes so retry attempts do not inflate
completion accounting.

Publish timing is enforced at delivery time. `notBeforeMs` delays provider dispatch; provider
workers split mixed due/future batches so due jobs are not blocked behind future jobs. `expiresAtMs`
prevents late sends; expired delivery jobs are recorded as `expired` and are not sent to providers.
Retry scheduling preserves the original timing window and does not run past publish or job expiry.

Operators inspect aggregate status with `GET /apps/{appId}/push/publish/{publishId}/status`, retry
and dead-letter queue lag through the health/backpressure surfaces, and metrics such as
`sockudo_push_queue_oldest_age_seconds{stage=...,state=...}`,
`sockudo_push_retry_scheduled_total`, `sockudo_push_retry_attempted_total`,
`sockudo_push_retry_dead_lettered_total`, `sockudo_push_retry_malformed_total`, and
`sockudo_push_provider_failures_total{failure_class=...}`.

Lifecycle safety checks expose `sockudo_push_invariant_violations_total{invariant=...}`. The bounded
`status_transition` label means a stale, non-monotonic status write was ignored. The metric does
not include payloads, device tokens, or provider credentials.

Publish-status mutations use storage-level compare-and-swap in every backend. Short-lived write
contention increments `sockudo_push_status_cas_conflicts_total{component=...}` and is retried with
a fixed bound. `sockudo_push_status_cas_exhausted_total{component=...}` means that bound was
exhausted; the worker returns a structured error instead of silently accepting a lost update.
Component labels come from a fixed internal set and neither metric includes payloads, tokens,
credentials, app IDs, or publish IDs.

All installations must stop old push admission and worker processes before starting
revision-aware workers; do not run old blind status writers alongside them. PostgreSQL and MySQL
must apply the push schema-version-2 migration while workers are stopped. DynamoDB, SurrealDB, and
ScyllaDB store the revision in the status document envelope and need no table migration, but still
require the same drain-and-replace deployment boundary.

Upgrade note: provider workers now enqueue internal `deliveryFeedback` payloads on
`push.results.v1` so feedback workers can schedule retries with replayable job context. Older
`deliveryResult` payloads are still accepted, but retryable legacy results without job context are
dead-lettered instead of being retried blindly.

## Retention and cleanup [#retention-and-cleanup]

The cleanup worker runs in monolith deployments when `push.cleanup_interval_secs > 0`. It removes
terminal publish statuses older than `push.publish_status_ttl_days`, delivery events and operator
invalidation events older than `push.analytics_retention_days`, expired idempotency records, and
expired scheduler locks. It does not delete active scheduled jobs, active retry work, credentials,
templates, or active device registrations.

Cleanup is incremental: each tick is bounded by `push.cleanup_batch_size` per category and
`push.cleanup_max_deleted_per_tick` overall. See [Push operations](/docs/server/push-operations)
for metrics, alerts, and outage runbooks.

## Device invalidation safety [#device-invalidation-safety]

Sockudo deletes or terminally invalidates a device only for device-specific terminal failures:

* FCM `UNREGISTERED`, `registration-token-not-registered`, or an `INVALID_ARGUMENT` response that
  specifically identifies the registration token as invalid.
* APNs `BadDeviceToken` or `Unregistered`.
* Web Push, HMS, and WNS expired subscription/channel responses such as `404` or `410`.

Provider-wide failures do not increment device failure counts and do not delete devices. This
includes FCM `SENDER_ID_MISMATCH` or project mismatch, APNs topic/environment/auth failures, Web
Push VAPID/auth failures, provider `429` quota responses, provider `5xx` responses, DNS/TLS/network
transport failures, and caller payload errors. These outcomes still update publish status, schedule
retry or dead-letter work according to the retry policy, and emit classified metrics.

Operators should treat `credential_auth` and project/topic mismatch failures as credential or
configuration incidents: rotate or fix the provider credential, project id, APNs topic/environment,
or VAPID configuration, then inspect `GET /apps/{appId}/push/deadLetters` and replay entries marked
`replayable` or republish from the original source as appropriate. Do not purge device registries
for these classes. The `sockudo_push_provider_failures_total{failure_class=...}` metric separates
`device_terminal`, `provider_transient`, `provider_quota`, `credential_auth`, `caller_payload`,
`network_transport`, and `unknown` failures. The `sockudo_push_token_invalidation_guard_total`
metric and `token-invalidation-guard` meta event fire when a publish crosses the built-in
invalidation spike threshold.

## Configure providers [#configure-providers]

```toml
[push]
storage_driver = "postgres"
queue_driver = "redis"
allow_memory_drivers = false
fcm_enabled = true
apns_enabled = true
publish_status_ttl_days = 30
analytics_retention_days = 30
scheduler_interval_secs = 5
cleanup_interval_secs = 300
cleanup_batch_size = 1000
cleanup_max_deleted_per_tick = 100000

# Runtime env for FCM monolith workers:
# PUSH_FCM_ENABLED=true
# PUSH_FCM_SERVICE_ACCOUNT_JSON_PATH=/run/secrets/fcm-service-account.json
# PUSH_FCM_PROJECT_ID is optional when the service account JSON has project_id.

# Runtime env for APNs monolith workers:
# PUSH_APNS_ENABLED=true
# PUSH_APNS_TOPIC=com.example.app
# PUSH_APNS_PRIVATE_KEY_PATH=/run/secrets/AuthKey_KEYID12345.p8
```

## Register devices [#register-devices]

Clients should call your backend. Your backend authenticates the user, validates ownership, and forwards to Sockudo with server credentials.

```ts
app.post("/api/push/devices", async (req, res) => {
  const user = requireUser(req);

  const result = await sockudo.activateDevice({
    device_id: req.body.device_id,
    client_id: user.id,
    platform: req.body.platform,
    provider_token: req.body.provider_token,
  });

  res.json(result);
});
```

Device IDs should be stable per installation. Provider tokens can rotate; update the registration
when the platform SDK gives you a new token.

## Subscribe devices to channels [#subscribe-devices-to-channels]

```ts
await sockudo.upsertChannelPushSubscription({
  device_id: "ios-device-1",
  client_id: "user-42",
  channel: "orders",
});
```

Use the same authorization model as realtime channel subscription. If the user cannot subscribe to `private-orders` over WebSocket, they should not subscribe a device to `private-orders` for push.

```mermaid
sequenceDiagram
  participant App as Client app
  participant API as Your backend
  participant S as Sockudo
  participant P as Push provider

  App->>API: current provider token
  API->>API: verify user and device
  API->>S: register device
  API->>S: subscribe device to channel
  API->>S: publish push
  S->>P: provider request
  P-->>S: accepted or failed
  S-->>API: publish status
```

## Publish [#publish]

```ts
const response = await sockudo.publishPush({
  recipients: [
    { type: "channel", channel: "orders" },
    { type: "client", client_id: "user-42" },
  ],
  payload: {
    title: "Order updated",
    body: "Order ord_123 is packed",
    data: {
      order_id: "ord_123",
      channel: "orders",
    },
  },
  idempotency_key: "push-order-ord_123-packed",
  sync: false,
});

console.log(response.publish_id);
```

Use `sync: false` for most production traffic. Async admission returns quickly, then workers fan out
to providers and update publish status records.

### Payload fidelity, templates, and overrides [#payload-fidelity-templates-and-overrides]

Publish responses include `renderedPayloads`, one provider-specific preview for FCM, APNs, Web
Push, HMS, and WNS. The same renderer is used by admission and dispatch. Provider workers send the
previewed provider payload, except for runtime-only fields such as provider tokens, APNs device
tokens, generated authorization headers, and provider request IDs.

`providerOverrides` are provider-specific complete payloads. When an override exists for a
provider, it replaces the generic payload mapping for that provider. APNs override headers become
APNs request headers, Web Push `headers.ttl`, `headers.urgency`, and `headers.topic` become Web
Push request headers, and FCM/HMS/WNS overrides are sent as the provider body with runtime recipient
tokens added by the worker. Overrides are validated and size-checked before dispatch.

Templates are resolved during publish admission. A `payload.templateId` must reference an existing
template; missing templates fail the publish request before work is accepted. Sockudo selects the
template locale from `payload.templateData.locale`, falling back from exact locale to language and
then the template's `defaultLocale`. Template fields fill missing payload fields, while explicit
payload fields win. Template provider overrides are applied first and request `providerOverrides`
win when both define the same provider. The resolved payload is stored in the accepted publish work,
so later template edits do not change queued delivery or retry attempts.

Template placeholders use bounded `{{ data.path }}` substitutions against `payload.templateData`.
Missing placeholders fail admission with a precise invalid-template error; objects and arrays are
not coerced into strings.

## Delayed delivery and cancellation [#delayed-delivery-and-cancellation]

```ts
const delayed = await sockudo.publishPush({
  notBeforeMs: Date.parse("2026-05-19T18:00:00Z"),
  expiresAtMs: Date.parse("2026-05-19T18:10:00Z"),
  recipients: [{ type: "client", client_id: "user-42" }],
  payload: { title: "Reminder", body: "Your room starts soon" },
});
```

The public HTTP API does not expose a separate schedule-create endpoint. Use
`POST /apps/{appId}/push/publish` with `notBeforeMs` for delayed delivery. Persisted scheduled jobs
created by internal scheduler/store integrations can be cancelled before emission with
`DELETE /apps/{appId}/push/scheduled/{jobId}`.

## Channel-triggered push rule [#channel-triggered-push-rule]

Push rules convert normal channel events into push work. This is useful for product events and AI
agent completion notifications.

```toml
[[push_rules]]
enabled = true
channel_pattern = "notifications:*"
event_filter = ["agent-complete", "order-ready"]
rate_limit_per_second = 100

[push_rules.payload_mapping]
title_field = "title"
body_field = "body"
template_data_field = "data"
include_remaining_fields = true
```

```json
{
  "name": "agent-complete",
  "channel": "notifications:user-42",
  "data": {
    "title": "Agent finished",
    "body": "Your answer is ready",
    "data": {
      "session_id": "sess_01J"
    }
  }
}
```

## Capacity planning [#capacity-planning]

Push fanout has a different bottleneck profile than WebSocket fanout:

* provider rate limits
* token invalidation churn
* per-platform payload size limits
* queue depth and worker concurrency
* status retention writes
* scheduled publish scans
* provider callback volume

Before a campaign, test admission and provider dispatch separately. A healthy API admission rate does not prove provider delivery capacity.

## Benchmarking [#benchmarking]

Use the repository push scripts for repeatable scenarios:

```bash
node scripts/push-benchmark.mjs \
  --host http://127.0.0.1:6001 \
  --app-id app-id \
  --key app-key \
  --secret app-secret \
  --devices 10000 \
  --channels orders \
  --publish-rate 100
```

Track accepted publishes, queue depth, provider dispatch latency, provider error classes, and status write latency.

The Criterion benchmark `push_retry_scheduler/memory_drain_100k_due_retry_entries_budget_lt_5s`
seeds and drains 100k due retry entries through the memory queue/store path as a regression guard
for retry scheduling overhead.

## Troubleshooting [#troubleshooting]

| Symptom                            | Check                                                                                                                                                                 |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `202` returned but no notification | Inspect publish status and provider errors.                                                                                                                           |
| APNs rejects token                 | Verify bundle ID, environment, topic, and token freshness.                                                                                                            |
| Web Push rejected                  | Verify VAPID subject and key pair.                                                                                                                                    |
| FCM unauthorized                   | Verify service account and project.                                                                                                                                   |
| Channel push misses users          | Verify channel push subscriptions and `client_id` mapping.                                                                                                            |
| Large push is rejected             | Check platform payload limits and remove nonessential data.                                                                                                           |
| Push publish returns `503`         | Check push queue health, memory-driver production guard, provider feature flags, provider credentials, and dispatch worker count.                                     |
| Push publish returns `429`         | Check publish-log, shard, delivery result, retry, dead-letter, active provider delivery queue depth, and `sockudo_push_queue_oldest_age_seconds`.                     |
| Publish remains `dispatching`      | Check retry scheduler workers, `push.retry.v1` lag, dead-letter lag, and provider retry-after deadlines.                                                              |
| Publish becomes `dead_lettered`    | Inspect `GET /apps/{appId}/push/deadLetters`, retry metrics, provider outage history, and `max_attempts`/`max_elapsed_secs`; replay only entries marked `replayable`. |
| Cleanup failures increase          | Check `sockudo_push_cleanup_errors_total`, store connectivity, and batch limits.                                                                                      |

Push payloads should wake the app and include stable IDs. Fetch authoritative application state after the user opens the notification.


# Push operations (/docs/server/push-operations)



Push operations revolve around four questions: is work being admitted, is it moving through each
queue stage, are providers accepting it, can stale queued work be repaired, and is old operational
state being cleaned up.

## Retention and cleanup [#retention-and-cleanup]

The monolith push cleanup worker runs when `push.cleanup_interval_secs > 0`.

| State                                                   | Retention                                                          |
| ------------------------------------------------------- | ------------------------------------------------------------------ |
| Terminal publish statuses                               | `push.publish_status_ttl_days`                                     |
| Delivery events                                         | `push.analytics_retention_days`                                    |
| Operator invalidation events                            | `push.analytics_retention_days`                                    |
| Expired idempotency records                             | The record `expires_at_ms`; legacy non-epoch expiries are retained |
| Expired scheduler locks                                 | Removed after `expires_at_ms`                                      |
| Completed scheduled jobs                                | Deleted synchronously when emitted or cancelled                    |
| Active scheduled jobs and active retries                | Retained until they run, expire, or are cancelled                  |
| Credentials, templates, and active device registrations | Never removed by cleanup                                           |

Cleanup is bounded by `push.cleanup_batch_size` per category and
`push.cleanup_max_deleted_per_tick` overall. SQL stores use bounded deletes. Document stores clean
new writes through internal app and time indexes where available; families without a time index are
cleaned by bounded app-partition scans. Memory cleanup is for tests and local development only.

Dead-letter queue messages remain owned by the configured queue backend. Operators can inspect
queue-native dead-letter metadata through `GET /apps/{appId}/push/deadLetters`, filter by
`provider`, `sinceMs`, and `untilMs`, and page with `limit` plus `cursor`. Responses include safe
metadata only: dead-letter id, app/publish ids, provider when known, stage, key, reason, timestamp,
and `replayable`. They do not expose original push payloads, recipient tokens, endpoints, or
credential material.

Use `POST /apps/{appId}/push/deadLetters/{deadLetterId}/replay` to re-enqueue a replayable
dead-letter's original queue item after fixing the underlying incident. Marker-only dead letters
that were emitted without retained original queue payload stay inspectable but are returned as
`replayable: false`.

## Metrics [#metrics]

Watch these metrics by app, provider, stage, or bounded category labels:

| Signal              | Metrics                                                                                                                                                       |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Admission           | `sockudo_push_publish_accepted_total`, `sockudo_push_quota_acceptance_rejections_total`                                                                       |
| Queue lag           | `sockudo_push_publish_log_lag_seconds`, `sockudo_push_delivery_jobs_lag_seconds`, `sockudo_push_queue_oldest_age_seconds`, queue backend lag                  |
| Provider outcomes   | `sockudo_push_dispatched_total`, `sockudo_push_dispatch_duration_seconds`, `sockudo_push_provider_failures_total`                                             |
| Retries             | `sockudo_push_retry_scheduled_total`, `sockudo_push_retry_attempted_total`, `sockudo_push_retry_deferred_total`, `sockudo_push_retry_expired_total`           |
| Dead letters        | `sockudo_push_retry_dead_lettered_total`, dead-letter queue depth, `GET /apps/{appId}/push/deadLetters`                                                       |
| Repair              | `sockudo_push_repair_scanned_total`, `sockudo_push_repair_requeued_total`, `sockudo_push_repair_skipped_total`                                                |
| Credentials         | `sockudo_push_provider_failures_total{failure_class="credential_auth"}`                                                                                       |
| Device invalidation | `sockudo_push_token_invalidations_total`, `sockudo_push_token_invalidation_guard_total`                                                                       |
| Cleanup             | `sockudo_push_cleanup_scanned_total`, `sockudo_push_cleanup_deleted_total`, `sockudo_push_cleanup_errors_total`, `sockudo_push_cleanup_tick_duration_seconds` |
| Worker health       | `sockudo_push_worker_exits_total`                                                                                                                             |

Do not add labels for `publish_id`, `device_id`, tokens, endpoint URLs, or raw provider reasons.

## Alerts [#alerts]

Start with these alert shapes and tune them against normal traffic:

| Alert                  | Suggested condition                                                                                                                |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| Retry backlog age      | `sockudo_push_queue_oldest_age_seconds` for `retry_schedule` ready or inflight work exceeds the retry SLO                          |
| Dead-letter rate       | `sockudo_push_retry_dead_lettered_total` increases above baseline                                                                  |
| Provider auth failures | `sockudo_push_provider_failures_total{failure_class="credential_auth"}` is nonzero                                                 |
| Invalidation spike     | `sockudo_push_token_invalidation_guard_total` is nonzero or invalidation ratio jumps                                               |
| Dispatching age        | publish statuses remain `dispatching` beyond retry max elapsed plus provider timeout                                               |
| Queue lag              | critical stage depth crosses `PUSH_CRITICAL_QUEUE_MAX_LAG` or oldest actionable age crosses `PUSH_BACKPRESSURE_LAG_THRESHOLD_SECS` |
| Repair requeues        | `sockudo_push_repair_requeued_total` increases outside a queue-loss or worker-crash incident                                       |
| Cleanup failures       | `sockudo_push_cleanup_errors_total` increases                                                                                      |

## Runbooks [#runbooks]

### APNs outage [#apns-outage]

Confirm `sockudo_push_provider_failures_total{provider="apns"}` and APNs HTTP status family. If
failures are `credential_auth`, rotate or roll back the APNs key/topic/environment configuration and
restart provider workers so credentials reload. If failures are provider transient or quota, keep
devices intact, watch retry/dead-letter rate, and reduce campaign admission until `push.delivery.apns.v1`
lag drains.

### FCM outage [#fcm-outage]

Separate project/auth failures from provider `5xx` or quota responses using
`sockudo_push_provider_failures_total{provider="fcm",failure_class=...}`. For auth failures, fix
the service account or stored credential and restart workers. For provider outage, leave device
registrations in place, verify retries are scheduled, and throttle new fanout if retry lag or
publish-log lag crosses the configured backpressure thresholds.

### Credential rotation failure [#credential-rotation-failure]

Stop new publishes for the affected provider if admission is still accepting work. Restore the last
known-good credential or upload a corrected credential, then restart provider workers. Watch
`credential_auth` failures, provider worker exits, and `dispatching` publish age. Do not delete
devices for credential failures.

### Dead node with pending queue work [#dead-node-with-pending-queue-work]

Use the queue backend's visibility timeout and redelivery tools first. Sockudo queue adapters keep a
bounded local ready/pending handoff; if a node dies, broker redelivery depends on the selected
backend. After the replacement worker starts, verify `push.publish.v1`, `push.shards.v1`,
`push.results.v1`, `push.retry.v1`, and provider delivery stages are draining.

### Backlog or backpressure [#backlog-or-backpressure]

Check which stage crossed lag first. Publish-log lag points to planner capacity or queue health.
Shard lag points to channel fanout. Provider delivery lag points to provider throughput or
credentials. Retry lag points to provider outage or retry policy. Use
`sockudo_push_queue_oldest_age_seconds` to separate a shallow but stale stage from ordinary depth
growth. Dead-letter lag means operators need to inspect `GET /apps/{appId}/push/deadLetters`, fix
the underlying provider/credential/queue incident, then replay only entries marked `replayable` or
republish from the original source.

### Publish-log repair [#publish-log-repair]

When `push.repair_interval_secs > 0`, the monolith repair worker scans durable publish-log entries
that are still in `queued` state after `push.repair_min_age_secs` and re-enqueues the missing
`push.publish.v1` queue item. It is meant for accepted publish work whose queue message was lost or
acknowledged by a crashing worker before planning began. It does not recreate active retry,
provider-delivery, or terminal publish work.


# Realtime application patterns (/docs/server/realtime-application-patterns)



Sockudo starts Pusher-compatible and becomes Sockudo-native when you opt into Protocol V2 features.
The safest production pattern is to keep untrusted clients simple and put authority in your backend.

```mermaid
flowchart LR
  B[Browser or mobile SDK] -->|WebSocket subscribe| S[Sockudo]
  B -->|private/presence auth request| A[Your backend]
  A -->|signed auth response| B
  A -->|signed HTTP publish| S
  S -->|fanout| B
  S --> H[(History / version store)]
  S --> P[(Push queue)]
```

## Public feed [#public-feed]

Public channels require no channel auth. Use them for data that is safe for anyone with the app key
to receive.

```ts
import Sockudo from "@sockudo/client";

const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
});

const channel = client.subscribe("market:btc");
channel.bind("price.updated", (event) => {
  renderPrice(event.data);
});
```

Publish from a trusted backend:

```ts
await sockudo.trigger("market:btc", "price.updated", {
  symbol: "BTC",
  price: "104200.00",
});
```

## Private channel [#private-channel]

Private channels prove the user can subscribe before Sockudo admits the connection.

```ts
const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  channelAuthorization: {
    endpoint: "/api/sockudo/channel-auth",
    headers: { "x-csrf": csrfToken },
  },
});

const account = client.subscribe("private-account:user-42");
account.bind("invoice.paid", updateInvoice);
```

Backend auth should derive the channel from verified user state, not from a trusted-looking request
body.

```ts
app.post("/api/sockudo/channel-auth", async (req, res) => {
  const user = requireUser(req);
  const { socket_id, channel_name } = req.body;

  if (channel_name !== `private-account:${user.id}`) {
    return res.status(403).json({ error: "forbidden" });
  }

  res.json(sockudo.authorizeChannel(socket_id, channel_name));
});
```

## Presence room [#presence-room]

Presence channels add a current member list and join/leave events.

```ts
const room = client.subscribe("presence-room:incident-123");

room.bind("pusher:subscription_succeeded", (members) => {
  renderMembers(members);
});

room.bind("pusher:member_added", (member) => {
  addMember(member.id, member.info);
});

room.bind("pusher:member_removed", (member) => {
  removeMember(member.id);
});
```

Presence auth includes user data:

```ts
res.json(sockudo.authorizeChannel(socketId, channelName, {
  user_id: user.id,
  user_info: {
    name: user.name,
    role: user.role,
  },
}));
```

## Protocol V2 recovery and rewind [#protocol-v2-recovery-and-rewind]

Protocol V2 clients can recover missed messages and ask for recent history during subscribe.

```ts
const client = new Sockudo("app-key", {
  wsHost: "realtime.example.com",
  forceTLS: true,
  protocolVersion: 2,
  connectionRecovery: true,
});

const orders = client.subscribe("private-orders:user-42", {
  rewind: { count: 50 },
});

orders.bind("sockudo:rewind_complete", ({ historical_count, live_count }) => {
  console.log({ historical_count, live_count });
});

client.bind("sockudo:resume_success", ({ recovered, failed }) => {
  console.log("resume", { recovered, failed });
});
```

```mermaid
sequenceDiagram
  participant C as V2 client
  participant S as Sockudo
  participant H as Durable history

  C->>S: subscribe with rewind
  S->>H: read history until attach serial
  H-->>S: historical page
  S-->>C: historical messages
  S-->>C: buffered live messages
  S-->>C: sockudo:rewind_complete
  C--xS: network interruption
  C->>S: resume positions
  S->>H: cold replay if hot buffer expired
  S-->>C: sockudo:resume_success or resume_failed
```

Use `until_attach` for late joins: it prevents a history page from racing with live messages above
the attach serial.

## Mutable messages [#mutable-messages]

Mutable messages let a logical message be updated, deleted, appended, and summarized while keeping a
version history.

```ts
const latest = await channel.getMessage("msg_01J");

const versions = await channel.getMessageVersions("msg_01J", {
  limit: 20,
  direction: "oldest_first",
});
```

Trusted backends mutate through signed HTTP:

```http
POST /apps/{app_id}/channels/private-doc:123/messages/msg_01J/update
Content-Type: application/json
```

```json
{
  "data": { "title": "Final incident report" },
  "description": "user edit",
  "metadata": { "editor": "user-42" },
  "op_id": "edit-msg_01J-title-v2"
}
```

Append is the right primitive for streams:

```json
{
  "data": "next generated token ",
  "extras": {
    "ai": {
      "transport": {
        "run-id": "run_123",
        "stream-id": "text",
        "status": "streaming"
      }
    }
  },
  "op_id": "run_123:text:00042"
}
```

## Annotations [#annotations]

Annotations attach secondary state to a message without rewriting the message itself. Use them for
reactions, read receipts, moderation notes, delivery receipts, and summaries.

```http
POST /apps/{app_id}/channels/chat/messages/msg_01J/annotations
Content-Type: application/json
```

```json
{
  "type": "reactions:distinct.v1",
  "name": "thumbs-up",
  "client_id": "user-42",
  "count": 1
}
```

## Backend publishing checklist [#backend-publishing-checklist]

| Concern      | Recommendation                                                                    |
| ------------ | --------------------------------------------------------------------------------- |
| Secrets      | Keep app secrets and provider credentials server-side.                            |
| Idempotency  | Send `idempotency_key` for business events and `op_id` for mutations.             |
| Echo         | Use `socket_id` when the originating client already applied an optimistic update. |
| Payload size | Send stable identifiers and fetch large state from your app API.                  |
| Auth         | Sign private, presence, encrypted, and user auth from your backend.               |
| History      | Store opaque cursors unchanged. Do not parse cursor internals.                    |

## Scaling shape [#scaling-shape]

```mermaid
flowchart TB
  LB[Load balancer] --> N1[Sockudo node A]
  LB --> N2[Sockudo node B]
  LB --> N3[Sockudo node C]

  N1 <-->|adapter fanout| Bus[(Redis / NATS / Kafka / RabbitMQ / Pulsar / Iggy)]
  N2 <-->|adapter fanout| Bus
  N3 <-->|adapter fanout| Bus

  N1 --> Cache[(Shared cache)]
  N2 --> Cache
  N3 --> Cache

  N1 --> Store[(History + version store)]
  N2 --> Store
  N3 --> Store

  N1 --> Queue[(Webhook / push queue)]
  N2 --> Queue
  N3 --> Queue
```

For horizontal deployments, avoid memory-only cache, queue, history, and version stores. Memory is
fine for local development and single-node demos; production clusters need shared stores for
idempotency, recovery, push dispatch, and durable history continuity.

## Normal Sockudo vs AI Transport [#normal-sockudo-vs-ai-transport]

| Need                       | Normal Sockudo                    | AI Transport                        |
| -------------------------- | --------------------------------- | ----------------------------------- |
| Broadcast app events       | `trigger`, channels, private auth | Not needed                          |
| Presence/collaboration     | Presence channels                 | Used for agent/user state too       |
| Recover after reconnect    | Protocol V2 recovery              | Required for durable runs           |
| Persist channel history    | Durable history                   | Required for sessions               |
| Edit or append messages    | Mutable messages                  | Used for streamed output            |
| Wake offline users         | Push notifications                | Used for background agents          |
| Model/provider integration | Your app code                     | Provider adapters and run lifecycle |

Start with normal Sockudo when you are building realtime product events. Add AI Transport when model
runs need to outlive one HTTP request, one browser tab, or one device.


# Scaling (/docs/server/scaling)





Sockudo scales horizontally when every node shares the dependencies that carry cross-node state: adapter, cache, queue, app manager, and optional history store.

AI Transport has a stricter horizontal matrix because streaming recovery depends on durable history, versioned-message state, and shared orphan ownership. For horizontal adapters (`redis`, `redis-cluster`, `nats`, `pulsar`, `rabbitmq`, `google-pubsub`, `kafka`, or `iggy`), configure shared non-memory history and version-store backends plus Redis or Redis Cluster cache. Startup rejects AI Transport with process-local memory history, version stores, or cache on horizontal adapters.

## Architecture [#architecture]

<img alt="Horizontal scaling diagram" src="__img0" />

At minimum, a production cluster has:

* a load balancer with WebSocket upgrade support
* multiple Sockudo nodes
* a shared adapter for cross-node fanout
* shared cache for rate limits, idempotency, and coordination
* shared queue for webhooks and push delivery
* shared app manager for dynamic app credentials
* metrics and logs from every node

## Adapter choices [#adapter-choices]

| Adapter        | Strength                                                       |
| -------------- | -------------------------------------------------------------- |
| Redis          | Simple, common, low-latency local and regional deployments.    |
| Redis Cluster  | Higher Redis scale and shard-aware deployments.                |
| NATS           | Lightweight pub/sub with strong operational ergonomics.        |
| Kafka          | Durable stream backbone and high-volume integration pipelines. |
| RabbitMQ       | Enterprise messaging and routing patterns.                     |
| Pulsar         | Multi-tenant stream workloads.                                 |
| Google Pub/Sub | Managed GCP fanout.                                            |
| Apache Iggy    | High-throughput persistent log workloads.                      |

### NATS notes [#nats-notes]

NATS is a good fit for lightweight cross-node fanout, but Sockudo clusters should avoid turning every client subscribe or unsubscribe into a distributed request/reply round trip. Keep room-switch churn on local channel state where possible, and reserve cluster-wide socket counts for post-ack meta events, webhooks, HTTP inspection, and operator views.

For subscription-churn or room-switch benchmarks, prefer gossiped aggregate counts so count reads stay local on each node:

```bash
ADAPTER_ENABLE_SOCKET_COUNTING=false
ADAPTER_AGGREGATE_COUNTS=true
```

Presence first-join and last-leave checks remain strict by default because they drive member
webhooks and presence history. For workloads that prefer maximum churn throughput over immediate
transition consistency, opt into the replicated presence-registry fast path:

```bash
ADAPTER_FAST_PRESENCE_TRANSITIONS=true
```

For Kubernetes NATS clusters, prefer explicit StatefulSet pod DNS entries over a single headless service URL when you want better initial client spread:

```json
{
  "adapter": {
    "driver": "nats",
    "nats": {
      "servers": [
        "nats://sockudo-nats-0.sockudo-nats-headless:4222",
        "nats://sockudo-nats-1.sockudo-nats-headless:4222",
        "nats://sockudo-nats-2.sockudo-nats-headless:4222"
      ],
      "request_timeout_ms": 5000,
      "connection_timeout_ms": 5000,
      "subscription_capacity": 131072,
      "client_capacity": 131072,
      "max_reconnects": 60,
      "no_echo": true
    }
  }
}
```

If `nodes_number` is set, it means expected Sockudo nodes, not NATS server replicas. Use it only when the Sockudo replica count is fixed or injected from deployment automation; otherwise leave discovery enabled.

## Load balancing [#load-balancing]

Sockudo does not require sticky sessions for basic pub/sub when the adapter is shared. Sticky sessions can still reduce reconnect churn and preserve local buffers during rolling updates.

Use:

* WebSocket upgrade headers
* idle timeouts longer than heartbeat intervals
* health and readiness checks
* draining before pod termination
* disruption budgets for production clusters

For Kubernetes probes, use `/live` for liveness and startup probes. `/up` checks configured apps and shared dependencies, so it is appropriate for readiness but too expensive for liveness under backend pressure. A slow app manager, cache, queue, or adapter should make the pod unready, not restart it.

```yaml
livenessProbe:
  httpGet:
    path: /live
    port: http
startupProbe:
  httpGet:
    path: /live
    port: http
readinessProbe:
  httpGet:
    path: /up/<app-id>
    port: http
```

## Duplicate delivery [#duplicate-delivery]

Distributed realtime systems must tolerate retries and duplicate delivery. Sockudo features that help:

* HTTP `idempotency_key`
* V2 `message_id`
* client-side message deduplication in native SDKs
* adapter-level duplicate suppression visibility
* recovery continuity checks

Consumers should treat application event IDs as stable and idempotent.

## Recovery across nodes [#recovery-across-nodes]

V2 recovery uses stream continuity. A reconnect to a different node can recover only if the required replay state is available through the configured shared backend or still present in a valid buffer.

Fail closed if continuity cannot be proven. Do not display a recovered state unless the server returns a successful resume.

## Push fanout at scale [#push-fanout-at-scale]

Push notification fanout is queue-oriented. A realtime publish and a push publish may target the same logical event, but they have different latency, retry, and delivery semantics.

Operational recommendations:

* keep push `sync` false in production
* use idempotency keys for publish retries
* partition high-volume channel pushes by tenant or campaign
* set publish status retention high enough for support workflows
* alert on provider error rates and queue backlog
* store provider credentials in secrets, not config maps
* use capacity planning before large campaigns

## Rolling deploys [#rolling-deploys]

1. Mark the node unready.
2. Stop accepting new connections.
3. Let existing connections drain or close with a reconnect-friendly code.
4. Keep adapter and cache dependencies available.
5. Watch reconnect, resume, and missed-message metrics.
6. Roll nodes in small batches.

## Metrics to watch [#metrics-to-watch]

* active connections by node
* subscription count by channel class
* publish accepted and failed counters
* fanout latency and adapter errors
* recovery success and failure counters
* replay buffer pressure
* AI stream orphan cancellations and `ai_stream_orphaned` webhook delivery
* webhook queue depth and failure count
* push publish accepted, dispatched, failed, and scheduled counters
* push provider latency and error labels

## Capacity checklist [#capacity-checklist]

Before increasing traffic, follow [Capacity planning and benchmarks](/docs/deployment/capacity-planning)
and run a scenario that covers:

* peak connected clients
* high-frequency channel fanout
* private and presence auth throughput
* durable history writes
* recovery after node restart
* AI stream orphan closure after node death
* pause/unpause partition checks with `scripts/ai-transport-jepsen-lite.sh`
* push burst admission
* push provider throttling
* webhook retry behavior

For room-switch workloads, `benches/subscription-churn.js` models connected sockets that periodically unsubscribe and subscribe to a new room. Churn rate is approximately `VUS / (ROOM_SWITCH_INTERVAL_MS / 1000)`. For example, 10,000 users switching every 20 seconds produces about 500 unsubscribe+subscribe cycles per second:

```bash
k6 run \
  -e WS_HOSTS=wss://example.com/app/app-key \
  -e VUS=10000 \
  -e CHANNEL_COUNT=4000 \
  -e ROOM_SWITCH_INTERVAL_MS=20000 \
  -e SOCKET_LIFETIME_MS=600000 \
  -e DURATION=10m \
  benches/subscription-churn.js
```


# Security (/docs/server/security)



Sockudo security depends on clear trust boundaries. Client SDKs connect and subscribe; server SDKs hold secrets, sign auth responses, publish events, validate webhooks, and manage push.

## Secrets [#secrets]

Keep these server-side only:

* app secret
* encryption master key
* webhook signing tokens
* database credentials
* Redis or adapter credentials
* push provider credentials
* APNs private key
* FCM service account JSON
* Web Push VAPID private key

Use secret managers or Kubernetes Secrets. Do not bake secrets into images or public environment bundles.

## Origin policy [#origin-policy]

Configure allowed origins for browser clients. Do not use wildcard origins for authenticated apps.

```toml
[[app_manager.array.apps]]
id = "app-id"
key = "app-key"
secret = "app-secret"

[app_manager.array.apps.policy.channels]
allowed_origins = [
  "https://app.example.com",
  "https://admin.example.com"
]
```

## Client messages [#client-messages]

Client events allow subscribed clients to publish events to private or presence channels. Disable them unless the product explicitly needs peer-to-peer client events.

```toml
[app_manager.array.apps.policy.features]
enable_client_messages = false
```

When enabled, validate channel authorization carefully and use event names that are easy to audit.

## Auth endpoints [#auth-endpoints]

Auth endpoints must:

1. authenticate the current user
2. validate `socket_id`
3. validate `channel_name`
4. enforce tenant and resource policy
5. sign with the server SDK
6. avoid logging full signatures or secrets

## Three-layer auth model [#three-layer-auth-model]

Sockudo separates client authority into three layers:

| Layer                  | Who uses it                   | What it controls                                                                                                                      |
| ---------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Capability token       | Protocol V2 WebSocket clients | Connection identity plus `publish`, `subscribe`, `history`, and `presence` permissions by channel pattern.                            |
| Customer auth endpoint | Your backend and client SDK   | Private, presence, encrypted channel auth, and short-lived capability-token issuance.                                                 |
| SDK callbacks          | Your app code                 | Product-specific actions such as cancel buttons, input validation, and UI-level workflow checks before sending a permitted operation. |

Capability tokens are additive. They do not replace Pusher-compatible HMAC channel auth, signed app HTTP APIs, or per-app mutation policies. A token-authenticated mutation must have `publish` for the channel and must also satisfy the existing own/any mutable-message policy.

Revocation is stored through the configured Sockudo cache by `jti` or `client_id`. Use a shared cache such as Redis when multiple Sockudo nodes must observe revocations consistently. A node that receives `POST /apps/{appId}/revocations` closes matching local token-authenticated sockets immediately; other nodes reject the revoked token on the next validation or refresh once the cache entry is visible.

## Encrypted channels [#encrypted-channels]

Encrypted channels protect payload contents from the transport. They do not replace channel authorization.

Rules:

* channel names must start with `private-encrypted-`
* only one encrypted channel should be targeted per encrypted publish
* keep the encryption master key server-side
* rotate keys with a planned client migration window

## Webhook validation [#webhook-validation]

Validate webhooks with the raw body:

```ts
const webhook = sockudo.webhook({ rawBody, headers });
if (!webhook.isValid()) {
  return res.status(401).end();
}
```

Support key rotation by accepting old and new webhook tokens during the migration window.

## Push security [#push-security]

Push is a privileged subsystem. Treat it like an outbound messaging platform:

* device registration must be tied to an authenticated user or device identity flow
* provider tokens should be updateable and revocable
* channel push subscriptions must obey the same authorization model as realtime channels
* provider credentials must never be exposed to clients
* push publish APIs should be rate limited per tenant and workflow
* scheduled push should have cancellation and audit trails

## Rate limiting [#rate-limiting]

Rate limit:

* connection attempts by IP
* auth requests by user and IP
* client events by socket
* HTTP publish by app
* push registration and publish by tenant
* webhook retries

Use shared Redis-backed limits in multi-node deployments.

## Operational hardening [#operational-hardening]

* terminate TLS at a trusted proxy or enable TLS directly
* set proxy idle timeouts above heartbeat intervals
* use readiness checks during deploys
* emit audit logs for denied auth and push admin operations
* alert on authentication failures, signature failures, and provider credential errors


# Token streaming and rollup (/docs/server/token-streaming-rollup)



Sockudo persists every versioned `message.append` operation exactly as received. Append rollup only changes WebSocket egress: the version store, durable history, recovery, and latest-message reads still observe the unrolled mutation log.

Enable the `ai-transport` Cargo feature and runtime AI Transport config before using rollup:

```toml
[ai_transport]
enabled = true

[[ai_transport.channels]]
prefix = "private-ai-"

[ai_transport.rollup]
enabled = true
default_window_ms = 40
min_window_ms = 0
max_window_ms = 500
orphan_ttl_ms = 1000
wheel_tick_ms = 5
shards = 64
```

The WebSocket query parameter `append_rollup_window` is accepted only for Protocol V2 and must be one of the locked values below. Current server v1 semantics use server-wide per `(app_id, channel, message_serial)` coalescing at `[ai_transport.rollup].default_window_ms`; the query parameter is validated for SDK compatibility but does not allocate per-subscriber rollup state.

| `append_rollup_window` | Behavior                                                      |
| ---------------------- | ------------------------------------------------------------- |
| `0`                    | Disable coalescing; every append fan-out counts individually. |
| `20`                   | Short coalescing window for lower added latency.              |
| `40`                   | Default; caps a steady stream near 25 deliveries per second.  |
| `100`                  | Heavier coalescing for overloaded subscribers.                |
| `500`                  | Maximum supported coalescing window.                          |

For a stream, the first append is delivered immediately. Later appends inside the fixed window are held and the latest append wins on egress. A terminal append with `extras.ai.transport.status` of `complete` or `cancelled`, plus `message.update` or `message.delete`, flushes pending append state before delivering the terminal operation.

The scheduler keeps sharded flush and orphan deadline heaps. A tick peeks one heap per shard and
only pops due entries; it does not scan active streams. Generation tokens make rescheduled and
terminally removed deadlines harmless. The first append retains timing metadata only after its
immediate fanout. Pending payload state is allocated when a later append actually needs
coalescing.

Deadline discovery and delivery are two phases. The worker enters the existing per-channel publish
ordering gate before claiming a generation token, so a concurrent terminal mutation either flushes
the pending append itself or waits behind the claimed delivery. Deferred context retains the app,
channel, excluded socket/echo decision, full-message versus delta choice, original envelope, and
continuity position.

The worker processes at most 4,096 due or retry entries per tick and at most 16 channels
concurrently. Messages within one channel remain sequential. App lookup and ordering-gate
backpressure use a bounded 4,096-entry retry queue. Once a due delivery is claimed, its three
fanout attempts stay inside the same channel ordering permit so a terminal mutation cannot
overtake a retry. Exhaustion is logged with `outcome="reset_required"`; it is never treated as a
successful delivery. Scheduler deadlines use a monotonic runtime clock and are unaffected by
wall-clock corrections. Graceful shutdown stops deadline intake, drains pending content in bounded
chunks, finishes bounded retries, and only then allows connection teardown.

Prometheus exposes low-cardinality rollup metrics per app:

* `sockudo_appends_received_total`
* `sockudo_appends_delivered_total`
* `sockudo_rollup_ratio`
* `sockudo_active_streams`
* `sockudo_flush_latency`

Billing, rate limits, durable history, version storage, webhooks, and push accounting count original
create/update/delete/append requests. Rollup metrics count both original append receipt and
coalesced egress delivery, so operators can see the reduction ratio without hiding ingress load.
Active-stream gauges are changed by exact insertion/removal deltas. Publishing a gauge does not
lock or scan scheduler shards.

Important edge cases:

* A terminal append flushes pending state before delivering the terminal operation.
* `message.update` and `message.delete` flush pending append state first.
* A late subscriber reconstructs state from history/version storage, not from rollup buffers.
* A node that sees a stale stream after `orphan_ttl_ms` claims it through shared cache and appends a
  normal cancellation update; the latest version is re-read before mutation to avoid cancelling a
  stream that advanced on another node.
* `append_rollup_window=0` disables egress coalescing but does not change persistence or rate-limit
  behavior.

The scheduler exposes `RollupEngine::tracking_overhead_bytes_per_stream()` for the inline tracking
footprint, excluding key and payload allocations. The permanent Criterion matrix measures 2,000
and 50,000 independent streams for empty tick, one-percent due, all due, and terminal-storm
workloads:

```bash
cargo bench -p sockudo-ai-transport --bench rollup_engine -- --noplot
```

`rollup_scheduler::at_least_999_per_mille_due_by_window_plus_five_ms_under_documented_load`
locks the deadline boundary at a 40 ms window, 5 ms tick allowance, and 2,000 independent streams.

Run `scripts/ai-rollup-load-test.mjs` against a local server to exercise a synthetic `200 tok/s` stream and inspect delivery rate, final content, and append mutation latency.


# Webhooks (/docs/server/webhooks)



Webhooks let Sockudo notify your backend about lifecycle and operational events. They are delivered asynchronously and should be idempotent.

## Configure [#configure]

```toml
[webhooks]
enabled = true
batching_enabled = true
max_batch_size = 50
flush_interval_ms = 500
timeout_ms = 5000
```

## Validate signatures [#validate-signatures]

Use the raw request body:

```ts
app.post("/sockudo/webhooks", rawBodyMiddleware, (req, res) => {
  const webhook = sockudo.webhook({
    rawBody: req.rawBody,
    headers: req.headers,
  });

  if (!webhook.isValid()) {
    return res.status(401).send("invalid");
  }

  for (const event of webhook.getEvents()) {
    handleWebhookEvent(event);
  }

  res.send("ok");
});
```

## Event handling [#event-handling]

Webhook handlers should:

* deduplicate by event ID or stable business ID
* return quickly
* enqueue expensive work
* tolerate retries
* preserve raw payloads for audit when compliance requires it

Presence lifecycle events are:

* `member_added`
* `member_updated`
* `member_removed`

`member_updated` is emitted for Protocol V2 `sockudo:presence_update` frames. Its payload includes `channel`, `user_id`, and `user_info` with the latest member data.

## Push webhooks [#push-webhooks]

Push provider status callbacks and push lifecycle webhooks are core to operating notifications. Use them to reconcile provider outcomes with Sockudo publish status.

## AI Transport and message webhooks [#ai-transport-and-message-webhooks]

Enable event types per app webhook. AI Transport event types are:

* `ai_run_started`
* `ai_run_ended`
* `ai_cancel_requested`
* `ai_stream_orphaned`

Legacy `ai_turn_started` and `ai_turn_ended` event types are still accepted for existing webhook
consumers, but new integrations should subscribe to `ai_run_started` and `ai_run_ended`.

Versioned-message and annotation event types are:

* `message_version_created`
* `annotation_created`
* `annotation_deleted`

`ai_run_ended` includes `reason` and optional `error_code`. `ai_cancel_requested` includes both
`run_id` and the legacy `turn_id` alias. `ai_stream_orphaned` is emitted when the distributed
orphan janitor closes a stale streaming message after `ai_transport.rollup.orphan_ttl_ms`:

```json
{
  "name": "ai_stream_orphaned",
  "channel": "private-ai-chat",
  "message_serial": "00000000000000000001:node:00000000000000000001",
  "reason": "orphan_timeout"
}
```

`message_version_created` includes `channel`, `message_serial`, `version_serial`, and `action` (`message.create`, `message.update`, `message.delete`, or `message.append`). Annotation webhooks include `channel`, `message_serial`, `annotation_serial`, and `annotation_type`; delete events also include `deleted_annotation_serial`.

Typical push events include:

* publish accepted
* publish dispatched
* provider accepted
* provider rejected
* device token invalidated
* scheduled push cancelled
* delivery status callback received

Example handler:

```ts
function handleWebhookEvent(event: SockudoWebhookEvent) {
  switch (event.name) {
    case "push.publish_failed":
      alertPushFailure(event.data.publish_id, event.data.provider);
      break;
    case "push.device_invalidated":
      deactivateDevice(event.data.device_id);
      break;
    default:
      recordEvent(event);
  }
}
```

## Retry behavior [#retry-behavior]

Return a non-2xx status only when you want Sockudo to retry. If the event is valid but not useful, record it as ignored and return success.

## Security [#security]

* accept webhooks only over HTTPS
* validate signature before parsing business logic
* rotate tokens with overlap
* avoid logging provider tokens or encrypted payloads
* rate limit webhook endpoints separately from public API endpoints


# .NET (/docs/server-sdks/dotnet)



## Install [#install]

Install the published NuGet package:

```bash
dotnet add package SockudoServer --version 2.2.0
```

Or add it directly to your project file:

```xml
<PackageReference Include="SockudoServer" Version="2.2.0" />
```

## Configure [#configure]

```csharp
using Sockudo;

var options = new SockudoOptions
{
    Host = "127.0.0.1",
    Port = 6001,
    Encrypted = false,
};

var sockudo = new Sockudo(APP_ID, APP_KEY, APP_SECRET, options);
```

Register one `Sockudo` instance as a singleton and reuse it. `Host` overrides
`Cluster`; `Encrypted` selects HTTPS; `RestClientTimeout` bounds requests; and
`BatchEventDataSizeLimit` can reject oversized event data before sending it.
Use HTTPS when the API crosses an untrusted network.

## Publish [#publish]

```csharp
ITriggerResult result = await sockudo.TriggerAsync(
    "orders",
    "order.created",
    new { id = "ord_123" },
    new TriggerOptions
    {
        IdempotencyKey = "order-created-ord_123",
    }
).ConfigureAwait(false);
```

Publish to several channels or send a batch:

```csharp
await sockudo.TriggerAsync(
    new[] { "tenant-42:orders", "user-7:orders" },
    "order.updated",
    new { id = "ord_123", status = "paid" }
).ConfigureAwait(false);

await sockudo.TriggerAsync(new[]
{
    new Event
    {
        Channel = "orders",
        EventName = "order.created",
        Data = new { id = "ord_124" },
    },
    new Event
    {
        Channel = "orders",
        EventName = "order.paid",
        Data = new { id = "ord_124" },
    },
}).ConfigureAwait(false);
```

Set `TriggerOptions.SocketId` to exclude a sender that already applied the
change. Use a stable `IdempotencyKey` for every publish that may be retried.
Set `BatchEventDataSizeLimit` to the server's event-size limit to fail early.

## Auth [#auth]

```csharp
var privateAuth = sockudo.Authenticate("private-orders", socketId).ToJson();

var channelData = new PresenceChannelData
{
    user_id = "user-42",
    user_info = new { name = "Ada" },
};

var presenceAuth = sockudo.Authenticate("presence-lobby", socketId, channelData).ToJson();
```

Authentication methods sign a response; your endpoint must first authenticate
the application session, authorize the exact channel, and derive presence
identity from trusted state.

For `private-encrypted-*`, set a 32-byte `EncryptionMasterKey` in
`SockudoOptions`. The SDK encrypts published payloads and derives the channel
shared secret used during auth. Keep the master key in a secret manager and
continue to use TLS.

## State [#state]

```csharp
var channel = await sockudo.FetchStateForChannelAsync<object>("orders")
    .ConfigureAwait(false);

var users = await sockudo.FetchUsersFromPresenceChannelAsync<object>("presence-lobby")
    .ConfigureAwait(false);
```

Read bounded history and forward opaque cursors unchanged:

```csharp
var page = await sockudo.FetchHistoryForChannelAsync<object>(
    "orders",
    new { limit = 50, direction = "newest_first" }
).ConfigureAwait(false);

var presencePage = await sockudo.FetchPresenceHistoryForChannelAsync<object>(
    "presence-lobby",
    new { limit = 50, direction = "newest_first" }
).ConfigureAwait(false);

var snapshot = await sockudo.FetchPresenceSnapshotForChannelAsync<object>(
    "presence-lobby",
    new { at_serial = 4 }
).ConfigureAwait(false);
```

Application state is an operational snapshot and should not be used as an
authorization database.

## Webhooks [#webhooks]

Preserve the original body and validate before deserializing business data:

```csharp
var webhook = sockudo.ProcessWebHook(receivedSignature, rawBody);
if (!webhook.IsValid)
{
    return Results.Unauthorized();
}

await queue.EnqueueAsync(webhook.Events, cancellationToken);
return Results.Accepted();
```

Do not normalize or log the raw body or signature. Make downstream processing
idempotent.

## User connection management [#user-connection-management]

```csharp
await sockudo.TerminateUserConnectionsAsync("user-42")
    .ConfigureAwait(false);

await sockudo.ForceReconnectUserAsync("user-42")
    .ConfigureAwait(false);
```

Terminate for immediate revocation; force reconnect when clients should obtain
new auth or routing state.

## Signed push request [#signed-push-request]

If your .NET SDK version does not expose typed push helpers, generate a signed request for the push endpoint and send it with your preferred HTTP client.

```csharp
var requestBody = JsonSerializer.Serialize(new
{
    recipients = new[]
    {
        new { type = "channel", channel = "orders" },
    },
    payload = new
    {
        title = "Order updated",
        body = "Order ord_123 is packed",
    },
    idempotency_key = "push-order-ord_123-packed",
    sync = false,
});

var request = authenticatedRequestFactory.Build(
    PusherMethod.POST,
    "/push/publish",
    requestBody: requestBody
);

request.Headers.Add("X-Sockudo-Push-Capability", "push-admin");
```

Prefer a first-class push helper when available in your SDK version; the signed request shape is the same.

## Errors, retries, and application lifetime [#errors-retries-and-application-lifetime]

Inspect `ITriggerResult` and `IGetResult<T>` before using response data.
Validation failures such as `EventDataSizeExceededException` should not be
retried unchanged. Retry only transient HTTP failures, `429`, and suitable
server errors with bounded backoff and the original idempotency key.

* Register the SDK client as a singleton and reuse its HTTP resources.
* Set `RestClientTimeout` below the ASP.NET request or worker deadline.
* Pass cancellation through surrounding application operations where the SDK
  overload supports it.
* Keep credentials, event bodies, signed requests, webhooks, and provider
  tokens out of logs.
* Test payload limits, duplicate attempts, webhook retries, credential
  rotation, and graceful host shutdown.


# Go (/docs/server-sdks/go)



## Install [#install]

Install the v2 Go module. Go resolves this package through the module path and the `v2.2.0` Git tag:

```bash
go get github.com/sockudo/sockudo/server-sdks/sockudo-http-go/v2
```

## Configure [#configure]

```go
package main

import sockudo "github.com/sockudo/sockudo/server-sdks/sockudo-http-go/v2"

var client = sockudo.Client{
    AppID:  "app-id",
    Key:    "app-key",
    Secret: "app-secret",
    Host:   "127.0.0.1",
    Port:   "6001",
    Secure: false,
}
```

Reuse the client for the lifetime of the process. Configure a custom
`http.Client` to set the overall deadline and preserve connection pooling:

```go
client.HTTPClient = &http.Client{
    Timeout: 5 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 20,
        IdleConnTimeout:     90 * time.Second,
    },
}
```

The default client timeout is five seconds. Set `Secure: true` for HTTPS in
production. `ClientFromURL` and `ClientFromEnv("SOCKUDO_URL")` are convenient,
but connection URLs contain credentials and must not be logged.

## Publish [#publish]

```go
data := map[string]string{"id": "ord_123"}
key := "order-created-ord_123"

_, err := client.TriggerWithParams(
    "orders",
    "order.created",
    data,
    sockudo.TriggerParams{IdempotencyKey: &key},
)
if err != nil {
    panic(err)
}
```

Publish the same event to multiple channels with `TriggerMulti`, batch
independent events with `TriggerBatch`, and exclude an originating socket with
`TriggerParams.SocketID`:

```go
socketID := "123.456"
key := "order-paid:ord_123:v3"

_, err := client.TriggerWithParams(
    "orders",
    "order.updated",
    map[string]string{"id": "ord_123", "status": "paid"},
    sockudo.TriggerParams{
        SocketID:       &socketID,
        IdempotencyKey: &key,
    },
)
if err != nil {
    return err
}
```

Use a stable idempotency key for any publish a worker or HTTP handler may retry.
Do not generate a new key for each attempt.

## Auth endpoint [#auth-endpoint]

```go
func sockudoAuth(res http.ResponseWriter, req *http.Request) {
    params, _ := io.ReadAll(req.Body)
    if !currentUserCanAccess(req.Context(), requestedChannel(params)) {
        http.Error(res, "forbidden", http.StatusForbidden)
        return
    }
    response, err := client.AuthorizePrivateChannel(params)
    if err != nil {
        http.Error(res, "unauthorized", http.StatusUnauthorized)
        return
    }
    fmt.Fprint(res, string(response))
}
```

## Presence auth [#presence-auth]

```go
member := sockudo.MemberData{
    UserID: "user-42",
    UserInfo: map[string]string{
        "name": "Ada",
    },
}

response, err := client.AuthorizePresenceChannel(params, member)
```

The SDK signs the form body; your handler still owns session authentication and
channel authorization. Populate `MemberData` from trusted application state.
Use `AuthenticateUser` for user-targeted events, then
`TerminateUserConnections` or `ForceReconnectUser` when access changes.

## State and history [#state-and-history]

```go
prefix := "presence-"
info := "user_count"
channels, err := client.Channels(sockudo.ChannelsParams{
    FilterByPrefix: &prefix,
    Info:           &info,
})
if err != nil {
    return err
}

users, err := client.GetChannelUsers("presence-lobby")

limit := 50
direction := "newest_first"
page, err := client.ChannelHistory("orders", sockudo.HistoryParams{
    Limit:     &limit,
    Direction: &direction,
})
```

State is an operational snapshot, not an authorization source. History cursors
are opaque and must be passed to the next call unchanged.

## Webhooks [#webhooks]

Read and preserve the exact body bytes for signature validation:

```go
body, err := io.ReadAll(req.Body)
if err != nil {
    http.Error(res, "invalid body", http.StatusBadRequest)
    return
}

webhook, err := client.Webhook(req.Header, body)
if err != nil {
    http.Error(res, "invalid signature", http.StatusUnauthorized)
    return
}

enqueueWebhookEvents(webhook.Events)
res.WriteHeader(http.StatusAccepted)
```

Do not decode the JSON first or log the raw body and signature.

## End-to-end encrypted channels [#end-to-end-encrypted-channels]

Set `EncryptionMasterKeyBase64` to a base64-encoded 32-byte key and publish only
to `private-encrypted-*` names. The server SDK encrypts payloads and produces
the shared secret during channel auth. Keep the master key in a secret manager
and do not mix encrypted and unencrypted channels in one logical publish.

## Push notifications [#push-notifications]

```go
_, err := client.ActivateDevice(sockudo.PushDeviceDetails{
    "deviceId":      "android-device-1",
    "clientId":      "user-42",
    "platform":      "fcm",
    "providerToken": "provider-token",
})

_, err = client.UpsertChannelPushSubscription(sockudo.PushChannelSubscription{
    "deviceId": "android-device-1",
    "clientId": "user-42",
    "channel":  "orders",
}, "")

accepted, err := client.PublishPush(sockudo.PushPublishRequest{
    "recipients": []sockudo.PushRecipient{
        {"type": "channel", "channel": "orders"},
    },
    "payload": map[string]interface{}{
        "title": "Order updated",
        "body":  "Order ord_123 is packed",
    },
    "idempotency_key": "push-order-ord_123-packed",
})

_ = accepted
```

The Go push helpers sign `/push/*` requests and force async publish behavior.

## Errors and production guidance [#errors-and-production-guidance]

Every network operation returns an `error`; check it before using the response.
Retry only transient network errors, `429`, and suitable server failures with
bounded exponential backoff. Because an HTTP error can be ambiguous after the
server accepted a request, every retried publish needs an idempotency key.

* Reuse one `Client` and one tuned `http.Client`.
* Set a timeout below the enclosing request or job deadline.
* Pass request-scoped cancellation through your own surrounding workflow.
* Keep credentials, signed URLs, provider tokens, and raw webhook content out
  of logs.
* Bound channel-history page sizes and validate all auth endpoint input.
* Test duplicate attempts, key rotation, network interruption, and graceful
  worker shutdown.


# Server SDKs (/docs/server-sdks)



Server SDKs are trusted backend libraries. They hold app secrets, sign HTTP requests, sign private and presence channel auth responses, validate webhooks, publish realtime events, and manage push notifications.

Install server SDKs from the package names below. The old per-SDK repositories should be treated as
archived/legacy mirrors.

## Official SDKs [#official-sdks]

| Language | Package                                                     | Guide                              |
| -------- | ----------------------------------------------------------- | ---------------------------------- |
| Node.js  | `sockudo`                                                   | [Node.js](/docs/server-sdks/node)  |
| Python   | `sockudo-http-python`                                       | [Python](/docs/server-sdks/python) |
| PHP      | `sockudo/sockudo-php-server`                                | [PHP](/docs/server-sdks/php)       |
| Ruby     | `sockudo`                                                   | [Ruby](/docs/server-sdks/ruby)     |
| Go       | `github.com/sockudo/sockudo/server-sdks/sockudo-http-go/v2` | [Go](/docs/server-sdks/go)         |
| Rust     | `sockudo-http`                                              | [Rust](/docs/server-sdks/rust)     |
| Java     | `io.sockudo:sockudo-http-java`                              | [Java](/docs/server-sdks/java)     |
| .NET     | `SockudoServer`                                             | [.NET](/docs/server-sdks/dotnet)   |
| Swift    | `Sockudo` via SwiftPM                                       | [Swift](/docs/server-sdks/swift)   |

## Framework integrations [#framework-integrations]

| Framework     | Package           | Guide                                |
| ------------- | ----------------- | ------------------------------------ |
| Laravel 12–13 | `sockudo/laravel` | [Laravel](/docs/server-sdks/laravel) |

The Laravel package registers a native `sockudo` broadcasting connection and
exposes the PHP server SDK through Laravel's container and a facade. Laravel
Echo continues to use Sockudo's Pusher-compatible Protocol V1 connection.

These packages share the Sockudo/Pusher HTTP signing protocol but differ in
their typed helpers. When a new Sockudo endpoint is not yet wrapped by a
language SDK, use its generic signed-request helper or generated signed URI
rather than implementing HMAC signing independently.

## Realtime client or server SDK? [#realtime-client-or-server-sdk]

Use a server SDK in a trusted web application, API, worker, or job process. It
does not maintain a WebSocket subscription. Use a
[realtime client](/docs/clients) in browsers, mobile apps, desktop apps, or
backend consumers that need to receive events continuously.

| Task                                 | Realtime client                  | Server SDK         |
| ------------------------------------ | -------------------------------- | ------------------ |
| Subscribe and receive events         | Yes                              | No                 |
| Publish application events           | Client events only where allowed | Yes                |
| Hold app secret                      | Never                            | Yes                |
| Authorize private/presence channels  | Calls your backend               | Signs the response |
| Validate webhooks                    | No                               | Yes                |
| Query history and application state  | Through a trusted proxy          | Directly           |
| Manage push credentials and delivery | Through a trusted proxy          | Directly           |

## What belongs on the server [#what-belongs-on-the-server]

* event publishing and batch publishing
* idempotency keys for safe retries
* private, presence, encrypted, and user authentication
* webhook signature validation
* history, versioned messages, annotations, and presence history reads
* push device registration, channel push subscriptions, provider credentials, publish, scheduling, status, and delivery callbacks

## Configure one reusable client [#configure-one-reusable-client]

Every SDK needs the app ID, public key, secret, HTTP host, port, and TLS mode.
Load credentials from a secret manager or environment at process startup,
validate them once, and reuse the configured client. Reuse preserves HTTP
keep-alive pools and avoids per-request setup overhead.

For a typical deployment:

| Setting     | Local development | Production                                 |
| ----------- | ----------------- | ------------------------------------------ |
| Host        | `127.0.0.1`       | Internal service DNS or HTTPS API hostname |
| Port        | `6001`            | Service port or `443`                      |
| TLS         | Optional          | Required across untrusted networks         |
| Timeout     | Short, explicit   | Explicit and below the caller's deadline   |
| Credentials | Local test app    | Secret manager, scoped per environment     |

Do not log connection URLs containing credentials, signed query strings,
authorization responses, provider tokens, raw webhooks, or encryption keys.

## Publishing model [#publishing-model]

Use a single publish for one logical event, a multi-channel publish when the
same event must reach several channels, and a batch when several independent
events can share one HTTP request. Respect the per-SDK and server limits shown
in each language guide.

Exclude the originating `socket_id` when the sender has already applied its own
change optimistically. Attach a stable idempotency key derived from the
business operation whenever a request may be retried:

```text
order.created:tenant-42:order-123:version-1
```

An idempotency key identifies a logical operation, not an HTTP attempt. Reusing
one key for different payloads can hide a real update.

## Authorization endpoints [#authorization-endpoints]

A correct private or presence authorization handler performs this sequence:

1. Authenticate the application's user session.
2. Validate the `socket_id` and requested `channel_name`.
3. Check tenant and resource access for that exact channel.
4. Derive presence `user_id` and `user_info` from trusted application data.
5. Ask the server SDK to sign the response.
6. Return only the signed response to the client.

Signing is not authorization by itself. Never implement an endpoint that signs
every requested channel, and never accept presence identity directly from the
untrusted client.

For `private-encrypted-*`, configure the SDK's encryption master key and return
the derived channel shared secret. Store and rotate the master key like any
other high-value application secret.

## Webhook handling [#webhook-handling]

Validate the signature against the exact raw request bytes before JSON
decoding. Body parsing, whitespace normalization, or character re-encoding can
change the signed bytes. Return a non-success response for an invalid signature
and avoid logging the raw body.

After validation, enqueue business processing and acknowledge quickly.
Webhook handlers should be idempotent because delivery may be retried.

## Errors, timeouts, and retries [#errors-timeouts-and-retries]

Retry only transient transport failures, `429`, and appropriate `5xx`
responses. Do not retry validation, authentication, or authorization failures
without changing the request. Use exponential backoff with jitter and a bounded
attempt count.

Publish retries require an idempotency key. Keep the SDK timeout shorter than
the enclosing HTTP request or job deadline so the caller has time to handle the
result. Record status code, operation, latency, and retry count without logging
credentials or message content.

## State, history, and pagination [#state-history-and-pagination]

Application-state endpoints are operational snapshots, not a transactional
database. Use them for occupied-channel and presence diagnostics rather than
authorization decisions.

History and presence history use bounded pages and opaque cursors. Pass cursors
back unchanged; do not parse or construct them. A client performing a gap-free
late join should subscribe first and request history through its attach serial.

## Push as a backend concern [#push-as-a-backend-concern]

Client apps may collect device tokens, but server SDKs should register devices and publish push notifications. This keeps app secrets and provider credentials away from browsers and mobile apps.

```ts
await sockudo.publishPush({
  recipients: [{ type: "channel", channel: "orders" }],
  payload: {
    title: "Order updated",
    body: "Order ord_123 is packed",
  },
  idempotency_key: "push-order-ord_123-packed",
});
```

## Shared production pattern [#shared-production-pattern]

1. Instantiate one SDK client per process and reuse it.
2. Use environment variables or secret managers for app credentials.
3. Add `idempotency_key` to retryable publish and push workflows.
4. Validate private and presence channel access before signing.
5. Validate webhook signatures before parsing business logic.
6. Treat push publish responses as admission records and inspect status asynchronously.
7. Set explicit connect/request timeouts and bounded retries.
8. Monitor request latency, response status, rate limits, and webhook failures.
9. Test credential rotation, duplicate delivery, Sockudo restarts, and network
   interruption before production rollout.


# Java (/docs/server-sdks/java)



## Install [#install]

Install the published package from Maven Central:

```xml
<dependency>
  <groupId>io.sockudo</groupId>
  <artifactId>sockudo-http-java</artifactId>
  <version>2.2.0</version>
</dependency>
```

```kotlin
implementation("io.sockudo:sockudo-http-java:2.2.0")
```

## Configure [#configure]

```java
Sockudo sockudo = new Sockudo("app-id", "app-key", "app-secret");
sockudo.setHost("127.0.0.1");
sockudo.setPort(6001);
sockudo.setEncrypted(false);
```

Create one instance and reuse it; the client is thread-safe and keeps a pooled
HTTP connection manager. You can also configure it from
`http://key:secret@host:port/apps/app-id`, but that URL contains credentials
and must not be logged.

Use `setEncrypted(true)` for HTTPS in production. Configure the synchronous or
asynchronous HTTP client when you need an outbound proxy, connection pool
limits, or request timeouts.

## Publish [#publish]

```java
Map<String, Object> data = new HashMap<>();
data.put("id", "ord_123");

Result result = sockudo.trigger(
    "orders",
    "order.created",
    data,
    null,
    "order-created-ord_123"
);
```

The SDK also supports multi-channel and batch publishing:

```java
sockudo.trigger(
    List.of("tenant-42:orders", "user-7:orders"),
    "order.updated",
    Map.of("id", "ord_123", "status", "paid")
);

List<Event> batch = List.of(
    new Event("orders", "order.created", Map.of("id", "ord_124")),
    new Event("orders", "order.paid", Map.of("id", "ord_124"))
);
sockudo.trigger(batch);
```

Use the overload accepting a socket ID to exclude the originating connection.
The five-argument overload accepts both `socketId` and `idempotencyKey`; each
batch `Event` can also carry those values. Attach a stable key to any request
that a job or HTTP handler may retry.

All requests return `Result`. Inspect `getStatus()` and `getMessage()` rather
than assuming a completed call succeeded:

```java
Result result = sockudo.trigger(
    "orders",
    "order.created",
    data,
    null,
    "order-created-ord_123"
);
if (result.getStatus() != Status.SUCCESS) {
    throw new IllegalStateException("Sockudo publish failed: " + result.getStatus());
}
```

## Auth [#auth]

```java
String privateAuth = sockudo.authenticate(socketId, "private-orders");

Map<String, String> userInfo = new HashMap<>();
userInfo.put("name", "Ada");

String presenceAuth = sockudo.authenticate(
    socketId,
    "presence-lobby",
    new PresenceUser("user-42", userInfo)
);
```

Authenticate the application session and authorize the exact channel before
signing. Build `PresenceUser` from trusted server-side identity. Use
`authenticateUser` for user-targeted events, then
`terminateUserConnections` or `forceReconnectUser` when permissions change.

## Async [#async]

```java
SockudoAsync async = new SockudoAsync("app-id", "app-key", "app-secret");
async.setHost("127.0.0.1");
async.setPort(6001);

CompletableFuture<Result> result =
    async.trigger("orders", "order.created", Collections.singletonMap("id", "ord_123"));
```

Compose the returned future and handle errors explicitly:

```java
async.trigger(
        "orders",
        "order.created",
        data,
        null,
        "order-created-ord_123"
    )
    .thenAccept(response -> recordStatus(response.getStatus()))
    .exceptionally(error -> {
        scheduleRetry(operationId, error);
        return null;
    });
```

Tune the async client rather than blocking on every future. Async execution is
not a durable queue; persist work separately when it must survive process
failure.

## State and history [#state-and-history]

```java
Result channels = sockudo.get(
    "/channels",
    Map.of("filter_by_prefix", "presence-", "info", "user_count")
);
Result users = sockudo.get("/channels/presence-lobby/users");

Result page = sockudo.getChannelHistory(
    "orders",
    Map.of("limit", "50", "direction", "newest_first")
);
Result next = sockudo.getChannelHistory(
    "orders",
    Map.of("cursor", opaqueCursor)
);
```

Presence history and snapshots have corresponding helpers. State is an
operational snapshot; keep pages bounded and forward cursors unchanged.

## Webhooks [#webhooks]

Validate the signature against the exact raw body before parsing:

```java
Validity validity = sockudo.validateWebhookSignature(
    xPusherKey,
    xPusherSignature,
    rawBody
);
if (validity != Validity.VALID) {
    throw new SecurityException("invalid Sockudo webhook");
}

Webhook webhook = sockudo.parseWebhook(rawBody);
enqueueWebhookEvents(webhook.getEvents());
```

The parser preserves unknown event names and fields for forward compatibility.
Do not log the raw body or signature.

## End-to-end encrypted channels [#end-to-end-encrypted-channels]

Pass a base64-encoded 32-byte encryption master key to the SDK constructor and
publish only to `private-encrypted-*` names. The SDK encrypts the payload and
derives the shared secret returned during auth. Do not mix encrypted and
unencrypted channels in one publish, and keep the master key in a secret
manager.

## Push notifications [#push-notifications]

```java
Map<String, Object> device = new HashMap<>();
device.put("deviceId", "android-device-1");
device.put("clientId", "user-42");
device.put("platform", "fcm");
device.put("providerToken", "provider-token");
sockudo.activateDevice(device);

Map<String, Object> subscription = new HashMap<>();
subscription.put("deviceId", "android-device-1");
subscription.put("clientId", "user-42");
subscription.put("channel", "orders");
sockudo.upsertChannelPushSubscription(subscription);

Map<String, Object> payload = new HashMap<>();
payload.put("title", "Order updated");
payload.put("body", "Order ord_123 is packed");

Map<String, Object> request = new HashMap<>();
request.put("recipients", List.of(Map.of("type", "channel", "channel", "orders")));
request.put("payload", payload);
request.put("idempotency_key", "push-order-ord_123-packed");

sockudo.publishPush(request);
```

Push helpers set `sync` to `false` and include the required Sockudo push capability header.

## Concurrency, errors, and production guidance [#concurrency-errors-and-production-guidance]

The synchronous client is thread-safe. Its default connection pool is small;
increase `PoolingHttpClientConnectionManager.setDefaultMaxPerRoute` for
high-concurrency services and set explicit timeouts.

Retry only transient network errors, `429`, and suitable server failures with
bounded exponential backoff. A retried publish must retain its original
idempotency key. Authentication and validation errors require a request or
configuration change.

* Reuse the sync or async client.
* Keep the SDK deadline shorter than the caller's deadline.
* Validate and rate-limit auth/history endpoints.
* Keep credentials, signed URIs, raw webhooks, payloads, and provider tokens
  out of logs.
* Test connection-pool saturation, duplicate attempts, credential rotation,
  and graceful application shutdown.


# Laravel (/docs/server-sdks/laravel)



The official Laravel integration registers a `sockudo` broadcasting connection
backed by the PHP server SDK. Existing Laravel broadcast events keep their
normal semantics, while trusted backend code can use Sockudo-native APIs from
the service container or facade.

## Install [#install]

```bash
composer require sockudo/laravel
php artisan sockudo:install
```

The service provider is auto-discovered. Configure credentials through your
environment or secret manager:

```bash
BROADCAST_CONNECTION=sockudo

SOCKUDO_APP_ID=app-id
SOCKUDO_APP_KEY=app-key
SOCKUDO_APP_SECRET=app-secret
SOCKUDO_HOST=127.0.0.1
SOCKUDO_PORT=6001
SOCKUDO_SCHEME=http
```

Use `https` when the signed HTTP API crosses an untrusted network. Confirm the
configuration and API connectivity without exposing credentials:

```bash
php artisan sockudo:check
```

Use `php artisan sockudo:check --config-only` when Sockudo is intentionally
unreachable during image builds.

The install command warns when Laravel's `routes/channels.php` broadcasting
scaffold is absent. Install Laravel broadcasting routes before using private or
presence channels; the Sockudo package supplies the backend connection but
does not silently rewrite application route authorization.

## Broadcast events [#broadcast-events]

Laravel's existing event contracts work unchanged:

```php
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;

final class OrderUpdated implements ShouldBroadcast
{
    public function __construct(public readonly string $orderId) {}

    public function broadcastOn(): PrivateChannel
    {
        return new PrivateChannel("orders.{$this->orderId}");
    }
}
```

The driver supports queued broadcasting, `ShouldBroadcastNow`, `toOthers()`,
public channels, private channels, presence channels, encrypted private
channels, and user authentication. Define authorization callbacks in
`routes/channels.php` as usual:

```php
use Illuminate\Support\Facades\Broadcast;

Broadcast::channel('orders.{order}', function ($user, $order) {
    return $user->can('view', $order);
});
```

Signing proves that Laravel approved a subscription; your channel callback is
still responsible for tenant and resource authorization.

## Connect Laravel Echo [#connect-laravel-echo]

Laravel Echo uses the Pusher connector for Sockudo Protocol V1:

```js
import Echo from 'laravel-echo';
import Pusher from 'pusher-js';

window.Pusher = Pusher;

window.Echo = new Echo({
  broadcaster: 'pusher',
  key: import.meta.env.VITE_SOCKUDO_APP_KEY,
  wsHost: import.meta.env.VITE_SOCKUDO_HOST,
  wsPort: Number(import.meta.env.VITE_SOCKUDO_PORT ?? 6001),
  wssPort: Number(import.meta.env.VITE_SOCKUDO_PORT ?? 443),
  forceTLS: (import.meta.env.VITE_SOCKUDO_SCHEME ?? 'https') === 'https',
  enabledTransports: ['ws', 'wss'],
});
```

Echo sends private and presence authorization requests to Laravel's standard
`/broadcasting/auth` endpoint. Never expose `SOCKUDO_APP_SECRET` to JavaScript.

Protocol V2 recovery, rewind, filters, deltas, and mutable-message projections
require a Sockudo-native realtime client rather than Laravel Echo.

## Native APIs [#native-apis]

The facade proxies calls to the configured PHP server SDK client:

```php
use Sockudo\Laravel\Facades\Sockudo;

$history = Sockudo::getChannelHistory('orders', ['limit' => 50]);

Sockudo::updateMessage('orders', $messageSerial, [
    'data' => ['status' => 'paid'],
]);

Sockudo::publishAnnotation('orders', $messageSerial, [
    'type' => 'reaction',
    'name' => 'confirmed',
]);

$publish = Sockudo::publishPush([
    'recipients' => [['type' => 'channel', 'channel' => 'orders']],
    'payload' => ['title' => 'Order updated'],
    'idempotency_key' => 'order-updated:ord-123:v4',
]);
```

Inject `Sockudo\SockudoInterface` when constructor injection is preferable. The
binding resolves the connection named by `SOCKUDO_CONNECTION`, which defaults
to `sockudo`.

## Multiple Sockudo apps [#multiple-sockudo-apps]

Define additional `driver => sockudo` entries in
`config/broadcasting.php`, then select one explicitly:

```php
$client = Sockudo::connection('sockudo-eu');
$client->trigger('orders', 'order.updated', ['id' => 'ord-123']);
```

Each connection has its own credentials, host, TLS options, and HTTP client
settings. Keep all secrets out of committed configuration and logs.

## Deployment notes [#deployment-notes]

* Run broadcasts on Laravel queues when request latency should not depend on
  the realtime API.
* Keep the SDK timeout below the queue-job timeout.
* Restart long-lived queue workers after changing cached configuration.
* Use stable, business-derived idempotency keys for retryable native publishes.
* Test public, private, presence, encrypted, `toOthers()`, and queued flows
  before switching production traffic.


# Node.js (/docs/server-sdks/node)



## Install [#install]

Install the published package from npm:

```bash
npm install sockudo
# or: bun add sockudo
# or: pnpm add sockudo
# or: yarn add sockudo
```

## Configure [#configure]

```ts
import { Sockudo } from "sockudo";

const sockudo = new Sockudo({
  appId: "app-id",
  key: "app-key",
  secret: "app-secret",
  host: "127.0.0.1",
  port: 6001,
  useTLS: false,
});
```

The package supports Node.js 16+ and ships TypeScript declarations. Create one
instance at application startup and reuse it so HTTP connections can be pooled.

| Option    | Meaning                         | Typical production value               |
| --------- | ------------------------------- | -------------------------------------- |
| `host`    | HTTP API host without a scheme  | Internal service DNS or API hostname   |
| `port`    | HTTP API port                   | Service port or `443`                  |
| `useTLS`  | Use HTTPS                       | `true` outside a trusted local network |
| `timeout` | Request timeout in milliseconds | Lower than the route or job deadline   |
| `agent`   | Custom Node HTTP(S) agent       | Keep-alive, proxy, and pool settings   |

You can also parse a connection URL:

```ts
const sockudo = Sockudo.forURL(
  "https://app-key:app-secret@realtime-api.example.com/apps/app-id",
);
```

Keep that URL in a secret manager and never write it to logs.

## Publish [#publish]

```ts
await sockudo.trigger("orders", "order.created", { id: "ord_123" }, {
  idempotency_key: "order-created-ord_123",
});

await sockudo.triggerBatch([
  { channel: "orders", name: "order.created", data: { id: "ord_124" } },
  { channel: "orders", name: "order.paid", data: { id: "ord_124" } },
]);
```

Publish the same event to several channels with an array, and exclude an
originating socket when the sender already updated optimistically:

```ts
await sockudo.trigger(
  ["tenant-42:orders", "user-7:orders"],
  "order.updated",
  { id: "ord_124", status: "paid" },
  {
    socket_id: "123.456",
    idempotency_key: "order-paid:ord_124:v3",
  },
);
```

A multi-channel publish supports up to 100 channels; a batch supports up to 10
events. Use a stable idempotency key for every publish that a queue worker or
HTTP handler may retry. Passing `idempotency_key: true` asks the SDK to generate
a UUID, but a business-derived key is better when separate attempts may run in
different processes.

## Auth [#auth]

```ts
app.post("/sockudo/auth", express.urlencoded({ extended: false }), (req, res) => {
  const { socket_id, channel_name } = req.body;
  const user = requireUser(req);

  if (channel_name.startsWith("presence-")) {
    return res.json(sockudo.authorizeChannel(socket_id, channel_name, {
      user_id: user.id,
      user_info: { name: user.name },
    }));
  }

  res.json(sockudo.authorizeChannel(socket_id, channel_name));
});
```

Call application authorization before `authorizeChannel`; signing alone grants
access. For user-targeted events, expose a separate authenticated endpoint:

```ts
app.post("/sockudo/user-auth", express.urlencoded({ extended: false }), (req, res) => {
  const user = requireUser(req);
  res.json(sockudo.authenticateUser(req.body.socket_id, {
    id: user.id,
    user_info: { name: user.name },
  }));
});
```

Disconnect or force reconnect all sockets for a compromised or changed user:

```ts
await sockudo.terminateUserConnections("user-42");
await sockudo.forceReconnectUser("user-42");
```

## Webhooks [#webhooks]

Signature verification requires the exact raw body. In Express, install a raw
parser on the webhook route before a global JSON parser consumes it:

```ts
app.post(
  "/sockudo/webhook",
  express.raw({ type: "application/json" }),
  (req, res) => {
    const webhook = sockudo.webhook({
      rawBody: req.body.toString(),
      headers: req.headers,
    });

    if (!webhook.isValid()) {
      return res.status(401).send("invalid signature");
    }

    enqueueWebhookEvents(webhook.getEvents());
    return res.sendStatus(202);
  },
);
```

During credential rotation, `isValid()` can accept additional old key/secret
pairs for a bounded overlap window. Do not log the raw webhook or signature.

## State and history [#state-and-history]

```ts
const channels = await (await sockudo.get({ path: "/channels" })).json();
const users = await (await sockudo.get({ path: "/channels/presence-lobby/users" })).json();
const history = await sockudo.channelHistory("orders", {
  limit: 50,
  direction: "newest_first",
});
```

State queries describe current occupancy and should not be used as an
authorization database. History cursors are opaque: pass `next_cursor` back to
the next call unchanged and bound each page.

## Push notifications [#push-notifications]

```ts
await sockudo.activateDevice({
  deviceId: "ios-device-1",
  clientId: "user-42",
  platform: "apns",
  providerToken: "provider-token",
});

await sockudo.upsertChannelPushSubscription({
  deviceId: "ios-device-1",
  clientId: "user-42",
  channel: "orders",
});

const accepted = await sockudo.publishPush({
  recipients: [{ type: "channel", channel: "orders" }],
  payload: {
    title: "Order updated",
    body: "Order ord_123 is packed",
    data: { order_id: "ord_123" },
  },
  idempotency_key: "push-order-ord_123-packed",
});

const status = await sockudo.getPublishStatus(accepted.publish_id);
```

Push helpers force async delivery with `sync: false` for production-safe fanout.

## Errors, retries, and shutdown [#errors-retries-and-shutdown]

HTTP failures reject with `RequestError`, which exposes a status code for
classification. Retry transient network errors, `429`, and suitable `5xx`
responses with bounded backoff; do not retry `401`, `403`, or invalid payloads
unchanged.

```ts
try {
  await sockudo.trigger("orders", "order.created", payload, {
    idempotency_key: operationId,
  });
} catch (error) {
  if (error instanceof Sockudo.RequestError && error.statusCode === 429) {
    await retryLater(operationId);
  } else {
    throw error;
  }
}
```

Reuse the client instead of constructing it per request, set a timeout below
the caller's deadline, and drain application jobs during graceful shutdown.
Never log `error.url` or `error.body` without redaction because they can contain
signed parameters or application data.


# PHP (/docs/server-sdks/php)



## Install [#install]

```bash
composer require sockudo/sockudo-php-server:^2.0
```

## Configure [#configure]

```php
use Sockudo\Sockudo;

$sockudo = new Sockudo(
    'app-key',
    'app-secret',
    'app-id',
    [
        'host' => '127.0.0.1',
        'port' => 6001,
        'useTLS' => false,
    ],
);
```

Create the client once in your framework service container and reuse it. The
important options are:

| Option              | Default          | Purpose                            |
| ------------------- | ---------------- | ---------------------------------- |
| `host`              | `127.0.0.1`      | Sockudo HTTP API host              |
| `port`              | `6001`           | HTTP API port                      |
| `useTLS` / `scheme` | `false` / `http` | Select HTTPS in production         |
| `timeout`           | `30` seconds     | Bound each request                 |
| `path`              | none             | Optional reverse-proxy path prefix |

Inject a configured Guzzle client when you need keep-alive, proxy, middleware,
or retry policy:

```php
$http = new GuzzleHttp\Client([
    'timeout' => 5.0,
    'connect_timeout' => 2.0,
]);

$sockudo = new Sockudo(
    'app-key',
    'app-secret',
    'app-id',
    ['host' => '127.0.0.1', 'port' => 6001, 'useTLS' => false],
    $http,
);
```

## Publish [#publish]

```php
$sockudo->trigger('orders', 'order.created', ['id' => 'ord_123'], [
    'idempotency_key' => 'order-created-ord_123',
]);

$sockudo->triggerBatch([
    ['channel' => 'orders', 'name' => 'order.created', 'data' => ['id' => 'ord_124']],
    ['channel' => 'orders', 'name' => 'order.paid', 'data' => ['id' => 'ord_124']],
]);
```

Publish the same event to multiple channels with an array, and exclude the
originating connection when it already applied the change:

```php
$sockudo->trigger(
    ['tenant-42:orders', 'user-7:orders'],
    'order.updated',
    ['id' => 'ord_123', 'status' => 'paid'],
    [
        'socket_id' => '123.456',
        'idempotency_key' => 'order-paid:ord_123:v3',
    ],
);
```

A batch accepts up to 10 events. Use `triggerAsync` or `triggerBatchAsync` when
the surrounding runtime can compose Guzzle promises; calling `wait()` makes the
operation synchronous again. Every retryable publish needs a stable,
business-derived idempotency key.

## Auth [#auth]

```php
$private = $sockudo->authorizeChannel('private-orders', $socketId);

$presence = $sockudo->authorizePresenceChannel(
    'presence-lobby',
    $socketId,
    'user-42',
    ['name' => 'Ada']
);
```

These methods sign a response; they do not decide whether a user may access a
channel. Authenticate the session, validate the exact channel, and derive
presence identity before calling them. Use `authenticateUser` for
user-targeted events and `forceReconnectUser` when a user's permissions change.

## Webhooks [#webhooks]

Pass the exact request headers and raw body to the SDK before JSON decoding:

```php
try {
    $webhook = $sockudo->webhook($requestHeaders, $rawBody);
    foreach ($webhook->get_events() as $event) {
        enqueueWebhookEvent($event);
    }
} catch (\Sockudo\SockudoException $error) {
    return response('invalid signature', 401);
}
```

Signature failure throws. Do not normalize or log the body before validation.

## State and history [#state-and-history]

```php
$channels = $sockudo->getChannels([
    'filter_by_prefix' => 'presence-',
    'info' => 'user_count',
]);
$users = $sockudo->getPresenceUsers('presence-lobby');

$page = $sockudo->getChannelHistory('orders', [
    'limit' => 50,
    'direction' => 'newest_first',
]);
$next = $sockudo->getChannelHistory('orders', [
    'cursor' => $page->next_cursor,
]);
```

Treat application state as an operational snapshot. Keep history pages bounded
and pass cursors back unchanged.

## Laravel broadcasting [#laravel-broadcasting]

For a first-class `sockudo` broadcasting connection, auto-discovered service
provider, configuration check, and native SDK facade, install the
[Laravel integration](/docs/server-sdks/laravel):

```bash
composer require sockudo/laravel
php artisan sockudo:install
```

The Pusher-compatible connection below remains supported for existing
applications that do not want the Laravel integration package:

```php
'pusher' => [
    'driver' => 'pusher',
    'key' => env('SOCKUDO_APP_KEY'),
    'secret' => env('SOCKUDO_APP_SECRET'),
    'app_id' => env('SOCKUDO_APP_ID'),
    'options' => [
        'host' => env('SOCKUDO_HOST', '127.0.0.1'),
        'port' => env('SOCKUDO_PORT', 6001),
        'scheme' => 'http',
        'useTLS' => false,
    ],
],
```

Store these values in Laravel's environment/secret configuration and enable
TLS when the HTTP API crosses an untrusted network. Laravel broadcasting still
uses your application authorization routes for private and presence channels.

## Push notifications [#push-notifications]

```php
$sockudo->activateDevice([
    'deviceId' => 'web-device-1',
    'clientId' => 'user-42',
    'platform' => 'webpush',
    'providerToken' => $subscription,
]);

$sockudo->upsertChannelPushSubscription([
    'deviceId' => 'web-device-1',
    'clientId' => 'user-42',
    'channel' => 'orders',
]);

$accepted = $sockudo->publishPush([
    'recipients' => [
        ['type' => 'channel', 'channel' => 'orders'],
    ],
    'payload' => [
        'title' => 'Order updated',
        'body' => 'Order ord_123 is packed',
    ],
    'idempotency_key' => 'push-order-ord_123-packed',
]);

$status = $sockudo->getPublishStatus($accepted->publish_id);
```

Push publishing is asynchronous by default. Use status APIs and provider callbacks for delivery diagnostics.

## Logging, errors, and retries [#logging-errors-and-retries]

The client implements `Psr\Log\LoggerAwareInterface`, so attach the
application's PSR-3 logger with `setLogger()`. Do not log request bodies,
provider tokens, signed URLs, or credentials.

Catch `ApiErrorException` for non-success API responses and
`SockudoException` for SDK validation/configuration failures. Retry only
transient network failures, `429`, and suitable `5xx` responses with bounded
backoff. Retried publishes must retain their original idempotency key.

* Reuse the SDK and Guzzle clients from the service container.
* Keep the SDK timeout below the PHP request or queue-job timeout.
* Move large batches and push fanout to queue workers.
* Rate-limit and authorize auth/history endpoints.
* Test duplicate attempts, queue retries, credential rotation, and worker
  shutdown.


# Python (/docs/server-sdks/python)



`sockudo-http-python` is the trusted HTTP SDK for APIs, workers, and web
frameworks. It publishes and signs requests; it does not maintain a WebSocket
subscription. Use [`sockudo-python`](/docs/clients/python) for an asyncio
realtime consumer.

## Install [#install]

```bash
pip install sockudo-http-python
```

## Configure [#configure]

```python
from sockudo_http_python import Sockudo, SockudoOptions

sockudo = Sockudo(
    "app-id",
    "app-key",
    "app-secret",
    options=SockudoOptions(host="127.0.0.1", port=6001, use_tls=False),
)
```

The package supports Python 3.9+. Create one client per process, reuse its
HTTP pool, and close it during shutdown. Important options are:

| Option             | Default       | Purpose                                          |
| ------------------ | ------------- | ------------------------------------------------ |
| `timeout`          | `4.0` seconds | Bound each HTTP operation                        |
| `max_retries`      | `3`           | Retry eligible network/server failures           |
| `retry_base_delay` | `0.1` seconds | Base for retry backoff                           |
| `auto_idempotency` | `False`       | Generate keys when the caller omits one          |
| `http2`            | `True`        | Allow HTTP/2 where supported                     |
| `verify_tls`       | `True`        | Verify the server certificate or use a CA bundle |

Use `use_tls=True` in production. Do not disable certificate verification to
work around a deployment problem.

## Publish [#publish]

```python
from sockudo_http_python import TriggerOptions

sockudo.trigger(
    "orders",
    "order.created",
    {"id": "ord_123"},
    TriggerOptions(idempotency_key="order-created-ord_123"),
)
```

Inspect the returned `Result` rather than assuming a non-throwing call
succeeded:

```python
result = sockudo.trigger(
    ["tenant-42:orders", "user-7:orders"],
    "order.updated",
    {"id": "ord_123", "status": "paid"},
    TriggerOptions(
        socket_id="123.456",
        idempotency_key="order-paid:ord_123:v3",
    ),
)

if not result.ok:
    raise RuntimeError(f"Sockudo publish failed: {result.status}")
```

Use batch publishing for independent events that can share one request. A
business-derived idempotency key is preferable to auto-generation when
separate workers may retry the same logical operation.

## Async [#async]

```python
from sockudo_http_python import AsyncSockudo, SockudoOptions

async with AsyncSockudo(
    "app-id",
    "app-key",
    "app-secret",
    options=SockudoOptions(host="127.0.0.1", port=6001, use_tls=False),
) as sockudo:
    await sockudo.trigger("orders", "order.created", {"id": "ord_123"})
```

Use the synchronous client in traditional WSGI code and `AsyncSockudo` in
ASGI/asyncio code. Do not call the synchronous client directly on an event
loop. The context manager closes `httpx` resources; call `sockudo.close()` for a
long-lived synchronous instance during process shutdown.

## Auth [#auth]

```python
from sockudo_http_python import PresenceUser

private_body = sockudo.authenticate("123.456", "private-orders")

presence_body = sockudo.authenticate(
    "123.456",
    "presence-lobby",
    PresenceUser("user-42", {"name": "Ada"}),
)

user_body = sockudo.authenticate_user("123.456", {"id": "user-42"})
```

Authentication helpers only produce signatures. Your route must first
authenticate the application session and authorize the exact channel. Derive
presence and user identity from trusted server-side data.

Configure `encryption_master_key_base64` to authorize and publish to
`private-encrypted-*` channels. Keep that key in a secret manager and never
return it to a client; the SDK returns only the derived shared secret.

## State, user controls, and pagination [#state-user-controls-and-pagination]

```python
from sockudo_http_python import ChannelsParams, PresenceHistoryParams

channels = sockudo.list_channels(
    ChannelsParams(
        filter_by_prefix="presence-",
        info=["subscription_count", "user_count"],
    )
)
users = sockudo.get_channel_users("presence-lobby")
presence_page = sockudo.get_channel_presence_history(
    "presence-lobby",
    PresenceHistoryParams(limit=50, direction="newest_first"),
)

sockudo.terminate_user_connections("user-42")
sockudo.force_reconnect_user("user-42")
```

Treat state as an operational snapshot. Pass history cursors back unchanged and
keep pages bounded.

## History and annotations [#history-and-annotations]

```python
from sockudo_http_python import HistoryParams, MessageMutation, PublishAnnotationRequest

sockudo.get_channel_history("orders", HistoryParams(limit=50, direction="newest_first"))
sockudo.get_message("orders", "42")
sockudo.update_message("orders", "42", MessageMutation(data={"status": "paid"}))
sockudo.publish_annotation(
    "orders",
    "42",
    PublishAnnotationRequest(
        type="reactions:distinct.v1",
        name="like",
        client_id="user-42",
        count=1,
    ),
)
```

Versioned update, append, and delete operations should use stable operation
identities in the surrounding workflow. Preserve message and version serial
order in downstream state. Annotation names and client IDs are application
data; authorize them before publishing.

## Webhooks [#webhooks]

Validate the signature against the exact raw request bytes before decoding
JSON:

```python
validity = sockudo.validate_webhook_signature(
    request.headers["X-Pusher-Key"],
    request.headers["X-Pusher-Signature"],
    raw_body,
)

if validity.value != "valid":
    raise PermissionError("invalid Sockudo webhook")

webhook = sockudo.parse_webhook(
    request.headers["X-Pusher-Key"],
    request.headers["X-Pusher-Signature"],
    raw_body,
)
enqueue_events(webhook.events)
```

Configure your framework route so middleware does not replace or normalize the
raw body before validation. Do not log the raw body or signature.

## Push notifications [#push-notifications]

```python
from sockudo_http_python import PushSubscriptionParams

sockudo.activate_device({
    "deviceId": "android-device-1",
    "clientId": "user-42",
    "platform": "fcm",
    "providerToken": "provider-token",
})

sockudo.upsert_channel_push_subscription({
    "deviceId": "android-device-1",
    "clientId": "user-42",
    "channel": "orders",
})

accepted = sockudo.publish_push({
    "recipients": [{"type": "channel", "channel": "orders"}],
    "payload": {
        "title": "Order updated",
        "body": "Order ord_123 is packed",
        "data": {"order_id": "ord_123"},
    },
    "idempotency_key": "push-order-ord_123-packed",
})

sockudo.list_channel_push_subscriptions(PushSubscriptionParams(device_id="android-device-1"))
```

Push helper methods force `sync = False` and return the API result for status inspection.

## Errors and production guidance [#errors-and-production-guidance]

Ordinary HTTP operations return `Result` with `Status.SUCCESS`,
`CLIENT_ERROR`, `AUTHENTICATION_ERROR`, `SERVER_ERROR`, or `NETWORK_ERROR`.
Only retry when `result.status.should_retry()` is true, or when your policy
explicitly handles `429`. Retried publishes must carry an idempotency key.

* Reuse the sync or async client and close it cleanly.
* Keep the SDK timeout shorter than the request or job deadline.
* Use a trusted CA bundle through `verify_tls` for private PKI.
* Keep app secrets, signed URLs, webhook bodies, and provider tokens out of
  logs.
* Rate-limit auth and history proxy routes.
* Test duplicate publish attempts, token/key rotation, network interruption,
  and graceful worker shutdown.


# Ruby (/docs/server-sdks/ruby)



## Install [#install]

```ruby
gem "sockudo", "~> 2.0"
```

```bash
bundle install
```

For one-off scripts, install the gem directly:

```bash
gem install sockudo -v "~> 2.0"
```

## Configure [#configure]

```ruby
require "sockudo"

sockudo = Sockudo::Client.new(
  app_id: "app-id",
  key: "app-key",
  secret: "app-secret",
  host: "127.0.0.1",
  port: 6001,
  use_tls: false
)
```

Create one client in an initializer and reuse it. `use_tls` defaults to `true`;
a custom `port` takes precedence. You can also load
`SOCKUDO_URL=http://KEY:SECRET@HOST:PORT/apps/APP_ID` with
`Sockudo::Client.from_env`, but treat the full URL as a secret and never log it.

For applications behind an outbound proxy, configure `Sockudo.http_proxy`.
Keep the SDK request timeout below the Rails request or job deadline.

## Publish [#publish]

```ruby
sockudo.trigger(
  "orders",
  "order.created",
  { id: "ord_123" },
  { idempotency_key: "order-created-ord_123" }
)

sockudo.trigger_batch([
  { channel: "orders", name: "order.created", data: { id: "ord_124" } },
  { channel: "orders", name: "order.paid", data: { id: "ord_124" } },
])
```

Publish the same event to several channels with an array and exclude an
originating connection when it already applied the update:

```ruby
sockudo.trigger(
  ["tenant-42:orders", "user-7:orders"],
  "order.updated",
  { id: "ord_123", status: "paid" },
  {
    socket_id: "123.456",
    idempotency_key: "order-paid:ord_123:v3"
  }
)
```

Multi-channel and batch calls accept up to 10 entries. Use a stable
idempotency key for every publish that Active Job or another worker may retry.

## Auth [#auth]

```ruby
private_auth = sockudo.authenticate("private-orders", params[:socket_id])

presence_auth = sockudo.authenticate(
  "presence-lobby",
  params[:socket_id],
  user_id: "user-42",
  user_info: { name: "Ada" }
)
```

Signing is the last step of authorization. Authenticate the application
session, validate the exact requested channel, and derive `user_id` from
trusted server-side state before calling `authenticate`.

Use `authenticate_user` for user-targeted events. When access changes,
`terminate_user_connections` disconnects a user immediately and
`force_reconnect_user` closes sockets with a reconnect instruction.

## Webhooks [#webhooks]

```ruby
webhook = sockudo.webhook(request)

if webhook.valid?
  webhook.events.each { |event| handle_event(event) }
  head :ok
else
  head :unauthorized
end
```

Build the webhook from the original `Rack::Request` so validation sees the raw
body and headers. Validate before processing, enqueue the events, and return
quickly. Do not log the request body or signature.

## State and history [#state-and-history]

```ruby
channels = sockudo.channels(filter_by_prefix: "presence-")
info = sockudo.channel_info("presence-lobby", info: "user_count")
users = sockudo.channel_users("presence-lobby")

page = sockudo.channel_history(
  "orders",
  limit: 50,
  direction: "newest_first"
)
next_page = sockudo.channel_history("orders", cursor: page[:next_cursor])

presence_page = sockudo.channel_presence_history(
  "presence-lobby",
  limit: 50
)
snapshot = sockudo.channel_presence_snapshot(
  "presence-lobby",
  at_serial: 4
)
```

State is an operational snapshot. Keep history pages bounded and pass opaque
cursors back unchanged.

## Async requests [#async-requests]

`trigger_async` and `get_async` use EventMachine when its reactor is active and
otherwise return a threaded `HTTPClient::Connection` immediately:

```ruby
sockudo.trigger_async(
  "orders",
  "order.created",
  { id: "ord_123" },
  { idempotency_key: "order-created:ord_123" }
).callback do
  Rails.logger.info("sockudo publish accepted")
end.errback do |error|
  retry_publish_later(error)
end
```

Use background jobs for work that must outlive a web request; an async method
alone is not a durable queue.

## Push notifications [#push-notifications]

```ruby
sockudo.activate_device(
  deviceId: "ios-device-1",
  clientId: "user-42",
  platform: "apns",
  providerToken: "provider-token"
)

sockudo.upsert_channel_push_subscription(
  deviceId: "ios-device-1",
  clientId: "user-42",
  channel: "orders"
)

accepted = sockudo.publish_push(
  recipients: [{ type: "channel", channel: "orders" }],
  payload: {
    title: "Order updated",
    body: "Order ord_123 is packed"
  },
  idempotency_key: "push-order-ord_123-packed"
)

status = sockudo.get_publish_status(accepted[:publish_id])
```

Use Rails jobs for push fanout workflows that also touch your product database.

## Errors, logging, and retries [#errors-logging-and-retries]

All errors inherit from `Sockudo::Error`:

```ruby
begin
  sockudo.trigger(
    "orders",
    "order.created",
    payload,
    idempotency_key: operation_id
  )
rescue Sockudo::AuthenticationError
  # Fix credentials; do not retry unchanged.
rescue Sockudo::HTTPError => error
  retry_publish_later(error)
rescue Sockudo::Error => error
  report_sdk_error(error)
end
```

Set `Sockudo.logger = Rails.logger` to use the application logger, but redact
credentials, signed URLs, provider tokens, and event content. Retry only
transient network failures, `429`, and suitable `5xx` responses with bounded
backoff and the original idempotency key.

* Reuse the client from a Rails initializer.
* Keep request deadlines shorter than the web or job timeout.
* Authorize and rate-limit auth/history endpoints.
* Use jobs for fanout and push workflows.
* Test duplicate delivery, credential rotation, webhook retries, and worker
  shutdown.


# Rust (/docs/server-sdks/rust)



## Install [#install]

```toml
[dependencies]
sockudo-http = "2.2.0"
tokio = { version = "1", features = ["full"] }
sonic-rs = "0.5"
```

## Configure [#configure]

```rust
use sockudo_http::{Config, Sockudo};

let config = Config::builder()
    .app_id("app-id")
    .key("app-key")
    .secret("app-secret")
    .host("127.0.0.1")
    .port(6001)
    .use_tls(false)
    .build()?;

let sockudo = Sockudo::new(config)?;
```

The default TLS backend is rustls and encryption support is enabled by default.
Create one `Sockudo` value at startup and share it with application state; its
HTTP client is designed for reuse.

Important builder options include `timeout`, `pool_max_idle_per_host`,
`enable_retry`, and `max_retries`. Use TLS outside local development and set
the SDK timeout below the surrounding request or job deadline.

## Publish [#publish]

```rust
use sockudo_http::{events::TriggerParams, Channel};
use sonic_rs::json;

let channels = vec![Channel::from_string("orders")?];
let params = TriggerParams::builder()
    .idempotency_key("order-created-ord_123")
    .build();

sockudo
    .trigger(&channels, "order.created", json!({ "id": "ord_123" }), Some(params))
    .await?;
```

Exclude an originating socket and attach a stable idempotency key with
`TriggerParams`:

```rust
let params = TriggerParams::builder()
    .socket_id("123.456")
    .idempotency_key("order-paid:ord_123:v3")
    .build();

sockudo
    .trigger(
        &channels,
        "order.updated",
        json!({ "id": "ord_123", "status": "paid" }),
        Some(params),
    )
    .await?;
```

Use a business-derived idempotency key for any operation that may be retried.
Do not generate a new key on each attempt.

## Batch and tags [#batch-and-tags]

```rust
use sockudo_http::events::BatchEvent;
use sonic_rs::json;
use std::collections::HashMap;

let mut tags = HashMap::new();
tags.insert("tenant".to_string(), "acme".to_string());

let event = BatchEvent::new("order.created", "orders", json!({ "id": "ord_124" }))
    .with_tags(tags);

sockudo.trigger_batch(vec![event]).await?;
```

Tags are strings and require server-side tag filtering to be enabled. Avoid
putting secrets or high-cardinality unbounded data in tags.

## Channel and user authentication [#channel-and-user-authentication]

Perform application authorization before asking the SDK to sign:

```rust
use sockudo_http::{Channel, SockudoError};
use sonic_rs::json;

fn authorize_private(
    sockudo: &Sockudo,
    socket_id: &str,
    channel_name: &str,
) -> Result<sockudo_http::SocketAuth, SockudoError> {
    let channel = Channel::from_string(channel_name)?;
    sockudo.authorize_channel(socket_id, &channel, None)
}

let presence = Channel::from_string("presence-lobby")?;
let member = json!({
    "user_id": "user-42",
    "user_info": { "name": "Ada" }
});
let auth = sockudo.authorize_channel("123.456", &presence, Some(&member))?;
```

The HTTP handler must derive channel access and presence identity from its
authenticated session. For user-targeted events, use `authenticate_user` and
`send_to_user`; use `terminate_user_connections` or `force_reconnect_user`
when access changes.

## History and pagination [#history-and-pagination]

```rust
use sockudo_http::HistoryParams;

let page = sockudo
    .channel_history_with_name(
        "orders",
        Some(&HistoryParams {
            limit: Some(50),
            direction: Some("newest_first".to_owned()),
            ..Default::default()
        }),
    )
    .await?;

if let Some(cursor) = page.next_cursor {
    let next_page = sockudo
        .channel_history_with_name(
            "orders",
            Some(&HistoryParams {
                cursor: Some(cursor),
                ..Default::default()
            }),
        )
        .await?;
    process_history(next_page)?;
}
```

Keep pages bounded and treat the cursor as opaque. Presence history and
snapshots use the corresponding typed parameter structs.

## Webhooks [#webhooks]

Validate the signature against the unchanged request body before processing
events:

```rust
use std::collections::BTreeMap;

fn verified_events(
    sockudo: &Sockudo,
    headers: &BTreeMap<String, String>,
    raw_body: &str,
) -> Result<Vec<sockudo_http::WebhookEvent>, SockudoError> {
    let webhook = sockudo.webhook(headers, raw_body);
    if !webhook.is_valid(None) {
        return Err(SockudoError::Validation {
            message: "invalid webhook signature".to_owned(),
        });
    }
    Ok(webhook.get_events()?)
}
```

Do not deserialize, normalize, or log the body before verification. Enqueue
validated events and acknowledge quickly.

## End-to-end encrypted channels [#end-to-end-encrypted-channels]

Set `encryption_master_key_base64` on the config builder and publish to a
`private-encrypted-*` channel. The SDK encrypts the event and returns a derived
shared secret during channel auth. Keep the 32-byte master key in a secret
manager and continue to use TLS.

## Push notifications [#push-notifications]

```rust
use sonic_rs::json;

sockudo.activate_device(&json!({
    "deviceId": "ios-device-1",
    "clientId": "user-42",
    "platform": "apns",
    "providerToken": "provider-token"
})).await?;

sockudo.upsert_channel_push_subscription(&json!({
    "deviceId": "ios-device-1",
    "clientId": "user-42",
    "channel": "orders"
}), None).await?;

let response = sockudo.publish_push(&json!({
    "recipients": [{ "type": "channel", "channel": "orders" }],
    "payload": {
        "title": "Order updated",
        "body": "Order ord_123 is packed"
    },
    "idempotency_key": "push-order-ord_123-packed"
})).await?;
```

Use Rust SDK push helpers for backend services that already own transactional state and need typed error handling around notification fanout.

## Errors, retries, and graceful shutdown [#errors-retries-and-graceful-shutdown]

All fallible methods return `Result<T, SockudoError>`. Match the error category
when policy differs:

```rust
match sockudo
    .trigger(&channels, "order.created", payload, Some(params))
    .await
{
    Ok(response) => record_status(response.status()),
    Err(SockudoError::Request(error)) => schedule_retry(error)?,
    Err(SockudoError::Validation { message }) => {
        tracing::warn!(error = %message, "sockudo publish rejected");
    }
    Err(error) => return Err(error),
}
```

Retries must be bounded and use an idempotency key. Share the SDK client rather
than constructing it per request, do not hold application locks across
`.await`, and use your service's cancellation mechanism to stop accepting new
jobs before shutdown. Never log signed URLs, app secrets, event payloads, or
raw webhook content.


# Swift (/docs/server-sdks/swift)



## Install [#install]

Install through Swift Package Manager from the Sockudo HTTP Swift mirror. SwiftPM resolves this from
the `v2.2.0` Git tag and the mirror repository's root `Package.swift` manifest:

```swift
.package(url: "https://github.com/sockudo/sockudo-http-swift", from: "2.2.0")
```

```swift
.product(name: "Sockudo", package: "sockudo-http-swift")
```

The package manifest supports Swift 5.3+, iOS 13+, macOS 10.15+, tvOS 13+, and watchOS
6+. It also exports a `Pusher` compatibility product for existing integrations;
new code should use `Sockudo`.

## Configure [#configure]

```swift
import Sockudo

let sockudo = Sockudo(options: try SockudoClientOptions(
    appId: 123456,
    key: "app-key",
    secret: "app-secret",
    host: "127.0.0.1",
    port: 6001,
    useTLS: false
))
```

`useTLS` defaults to `true`. Keep it enabled in production and use an HTTPS API
hostname. Construct the client once in service/application state and reuse it.
Initializers validate options and throw; propagate or handle those errors
instead of using `try!`.

## Publish [#publish]

```swift
let event = try Event(
    name: "order.created",
    data: ["id": "ord_123"],
    channel: Channel(name: "orders", type: .public),
    idempotencyKey: "order-created-ord_123"
)

sockudo.trigger(event: event) { result in
    switch result {
    case .success(let summaries):
        print(summaries)
    case .failure(let error):
        print(error)
    }
}
```

Publish to multiple channels or batch independent events:

```swift
let multi = try Event(
    name: "order.updated",
    data: ["id": "ord_123", "status": "paid"],
    channels: [
        Channel(name: "tenant-42:orders", type: .public),
        Channel(name: "user-7:orders", type: .public),
    ],
    idempotencyKey: "order-paid:ord_123:v3"
)

let created = try Event(
    name: "order.created",
    data: ["id": "ord_124"],
    channel: Channel(name: "orders", type: .public)
)
let paid = try Event(
    name: "order.paid",
    data: ["id": "ord_124"],
    channel: Channel(name: "orders", type: .public)
)

sockudo.trigger(event: multi) { result in
    handlePublishResult(result)
}
sockudo.trigger(events: [created, paid]) { result in
    handlePublishResult(result)
}
```

Use an event's `socketId` to exclude the originating connection. Batches accept
up to 10 events. Any publish that may be retried needs a stable
business-derived idempotency key.

## Auth [#auth]

```swift
let privateChannel = Channel(name: "orders", type: .private)

sockudo.authenticate(channel: privateChannel, socketId: "123.456") { result in
    print(result)
}

let userData = PresenceUserAuthData(userId: "user-42", userInfo: ["name": "Ada"])
let presenceChannel = Channel(name: "lobby", type: .presence)

sockudo.authenticate(channel: presenceChannel, socketId: "123.456", userData: userData) { result in
    print(result)
}
```

The SDK signs after your handler authenticates the session and authorizes the
exact channel. Derive presence identity from trusted application state. Use
`authenticateUser` for user-targeted events.

## Webhooks [#webhooks]

Pass the original request to `verifyWebhook` before processing its body:

```swift
sockudo.verifyWebhook(request: receivedRequest) { result in
    switch result {
    case .success(let webhook):
        enqueue(events: webhook.events)
    case .failure:
        rejectWebhook()
    }
}
```

Do not deserialize, normalize, or log the raw body before signature
verification. Make downstream processing idempotent.

## State and history [#state-and-history]

```swift
sockudo.channels(
    withFilter: .presence,
    attributeOptions: .userCount
) { result in
    handleChannels(result)
}

let orders = Channel(name: "orders", type: .public)
sockudo.history(
    for: orders,
    options: .init(limit: 50, direction: "newest_first")
) { result in
    handleHistory(result)
}

let presence = Channel(name: "lobby", type: .presence)
sockudo.presenceSnapshot(
    for: presence,
    options: .init(atSerial: 4)
) { result in
    handleSnapshot(result)
}
```

Current state is an operational snapshot. Bound history pages and pass opaque
cursors back unchanged.

## End-to-end encrypted channels [#end-to-end-encrypted-channels]

Pass a base64-encoded 32-byte `encryptionMasterKey` in
`SockudoClientOptions`, then use a channel with type `.encrypted`. The SDK
encrypts payloads and derives the auth shared secret. A single event cannot mix
encrypted and unencrypted channels. Keep the master key in a secret manager
and continue to use TLS.

## User connection management [#user-connection-management]

```swift
sockudo.terminateUserConnections(userId: "user-42") { result in
    handleUserControl(result)
}

sockudo.forceReconnectUser(userId: "user-42") { result in
    handleUserControl(result)
}
```

Terminate for immediate revocation; force reconnect when the user should
obtain refreshed auth or routing state.

## Push notifications [#push-notifications]

```swift
sockudo.activateDevice(body: [
    "deviceId": "ios-device-1",
    "clientId": "user-42",
    "platform": "apns",
    "providerToken": "provider-token",
]) { result in
    print(result)
}

sockudo.upsertChannelPushSubscription(body: [
    "deviceId": "ios-device-1",
    "clientId": "user-42",
    "channel": "orders",
]) { result in
    print(result)
}

sockudo.publishPush(request: [
    "recipients": [["type": "channel", "channel": "orders"]],
    "payload": [
        "title": "Order updated",
        "body": "Order ord_123 is packed",
    ],
    "idempotency_key": "push-order-ord_123-packed",
]) { result in
    print(result)
}
```

Use Swift server SDKs in trusted services, not inside untrusted client apps with app secrets.

## Errors, concurrency, and production guidance [#errors-concurrency-and-production-guidance]

Every asynchronous operation completes with `Result`. Handle both cases and
avoid force-unwrapping or `try!` in production. Retry only transient transport
errors, `429`, and suitable server responses with bounded backoff. Retried
publishes must retain their original idempotency key.

* Reuse the configured client and its URL session.
* Keep the SDK operation deadline below the enclosing request or job deadline.
* Dispatch completion work to the appropriate actor or queue for your
  application.
* Keep credentials, signed URLs, raw webhooks, event data, and provider tokens
  out of logs.
* Test duplicate attempts, key rotation, webhook retries, and graceful service
  shutdown.