Architecture

NotifKit is eight services communicating through Redis Streams, with PostgreSQL as the system of record. Every service is stateless and horizontally scalable; all coordination happens in Redis. This page is the contract underneath that.

The streams

Priority is expressed as separate physical streams rather than a score, so a worker draining a low-priority backlog physically cannot be in front of a critical message.

Each consumer reads through a named group — notifkit:group:enricher, notifkit:group:engine, notifkit:group:delivery, and so on — with consumer set to {service}-{pid}. Adding a process adds a consumer to the group and Redis divides the work; no configuration, no rebalance step.

Sorted sets and locks

Delivery guarantees

At-least-once. A message is acknowledged only after process() returns successfully. If a worker dies mid-flight the message stays in the group's pending list and another consumer claims it.

Not exactly-once. If a provider accepts a send but its response never arrives — a timeout on the return path — NotifKit records a failure and retries. The user receives it twice. No amount of internal deduplication can prevent this; it is inherent to calling a remote API without a two-phase commit.

Ordering is not guaranteed. Messages are processed concurrently, up to WORKER_CONCURRENCY per worker. Two sends to the same user may arrive in either order. If order matters, use a workflow, which is serialised per instance.

Where deduplication happens

Marks are provisional: a stage that fails after claiming a key releases it, so a retry is not swallowed as a duplicate.

Retries and the dead-letter queue

There is no exponential backoff. Recovery is driven by Redis pending-entry reclamation on a fixed interval, which makes the timing predictable and the mechanism crash-safe:

Failure classes

Backpressure

Streams are trimmed to QUEUE_MAX_LEN (10 million entries by default) as they are written. That is a safety valve against unbounded memory growth, not a flow-control mechanism — past the cap, the oldest entries are discarded.

Real backpressure comes from the consumer side. Workers read a bounded batch and process at most WORKER_CONCURRENCY messages at once, so when a provider slows down the Delivery worker pulls fewer messages, its pending list grows, and the upstream stages accumulate depth rather than overrunning anything. Stream depth is the number to alert on.

Deployment topologies

Single process

new NotifkitServer({ services: ["all"] });

Everything in one process. Correct for development, and fine in production up to the point where one service's load starts interfering with another's.

Split by role

The services array is the only thing that differs between deployments. Point them at the same Redis and Postgres and they form one system.

// Deployment 1 — behind your load balancer, scaled on request rate.
new NotifkitServer({
  services: ["api"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
});

// Deployment 2 — the pipeline, scaled on stream depth.
new NotifkitServer({
  services: ["enricher", "engine", "scheduler", "events"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
  autoMigrate: false,           // exactly one deployment should own migrations
  workerConcurrency: 25,
});

// Deployment 3 — the part that talks to providers, scaled on their limits.
new NotifkitServer({
  services: ["delivery"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
  autoMigrate: false,
  providers: [new ResendTransport({ apiKey: RESEND_KEY, from: "hi@acme.com" })],
});

What scales on what

The data model

Migrations run on startup unless you pass autoMigrate: false. With several deployments sharing one database, let exactly one of them own migrations.

Multi-tenancy

Every query is scoped by projectId, which is derived from the API key rather than from anything in the request body — a caller cannot address another tenant's data by changing a field. Keys are hashed with SHA-256 and cached for 60 seconds; revoking one publishes an invalidation so every API process drops it immediately instead of waiting out the TTL.

The admin key is the exception: it can act on behalf of any project, but must name one with an x-project-id header or a projectId query parameter. Requests without it are rejected rather than defaulted.