Partitioning Queues by Tenant ID

Partitioning by tenant is the most common sharding key in a multi-tenant job system, and the one most likely to produce skew. This guide applies the models in Queue Partitioning Strategies, part of Queue Fundamentals & Architecture, to that specific key.

Problem Statement

Your single job queue serves 4,000 tenants. Ordering matters within a tenant — a "create account" job must run before "send welcome email" — but not across them. A single FIFO queue gives you ordering you do not need globally and no isolation you do need, so one tenant's 200,000-row import delays everyone. You want ordered processing per tenant, parallelism across tenants, and a plan for the one customer who is fifty times larger than the median.

Prerequisites

• A tenant identifier available at enqueue time and carried in the message envelope.
• A broker that supports multiple queues or partitions: Redis keys, RabbitMQ queues, Kafka partitions, or SQS queues.
• Per-tenant volume data — the distribution decides the scheme far more than the technology does.
• Idempotent consumers, since rebalancing can redeliver in-flight work.

Step 1 — Decide What Partitioning Must Guarantee

Partitioning buys two things that are frequently conflated: ordering within a partition, and isolation between them. Which you need changes the design.

If ordering is not genuinely required — and it often is not, because idempotent handlers and explicit dependencies remove the need — per-tenant fairness over one shared queue is far cheaper than per-tenant partitioning. That path is covered in preventing tenant starvation with weighted queues.

Step 2 — Hash Tenants onto a Fixed Partition Count

One queue per tenant does not scale to thousands: brokers charge per queue in connections, memory, and monitoring surface. Hash instead onto a fixed number of partitions.

import hashlib

PARTITIONS = 32

def partition_for(tenant_id: str) -> int:
    # A stable hash, not Python's salted hash(): the mapping must survive restarts.
    digest = hashlib.blake2b(tenant_id.encode(), digest_size=8).digest()
    return int.from_bytes(digest, "big") % PARTITIONS

def queue_for(tenant_id: str) -> str:
    return f"jobs.p{partition_for(tenant_id):02d}"

Using a language's built-in hash() is a real bug here: Python salts it per process, so two producers compute different partitions for the same tenant and ordering silently breaks.

Step 3 — Handle the Whale

Tenant volume is almost always power-law distributed, so hashing spreads tenants evenly and load very unevenly. Override the hash for known outliers.

# Explicit assignments beat the hash. Dedicated partitions for the largest
# tenants; everyone else hashes over the remaining range.
DEDICATED = {"acme-corp": 0, "globex": 1}
SHARED_START, SHARED_COUNT = 2, 30

def partition_for(tenant_id: str) -> int:
    if tenant_id in DEDICATED:
        return DEDICATED[tenant_id]
    digest = hashlib.blake2b(tenant_id.encode(), digest_size=8).digest()
    return SHARED_START + int.from_bytes(digest, "big") % SHARED_COUNT

Dedicating a partition to the whale gives it a full worker's throughput and stops it from harming neighbours, at the cost of a mapping table that must be reviewed as customers grow. Automate the review: alert when any single tenant exceeds a threshold share of its partition's volume.

Step 4 — Rebalance Without Breaking Ordering

Changing PARTITIONS remaps most tenants, and a tenant whose partition changes can have jobs in flight on both the old and new queue simultaneously — losing ordering exactly when it matters.

def drain_and_switch(tenant_id: str, old_q: str, new_q: str) -> None:
    """Ordered migration: pause, drain, switch, resume. The pause is what
    guarantees no tenant has work on two partitions at once."""
    routing.pause(tenant_id)                    # producers buffer or block
    wait_until(lambda: depth(old_q, tenant_id) == 0, timeout=300)
    routing.set_override(tenant_id, new_q)      # atomic in the routing store
    routing.resume(tenant_id)

Consistent hashing reduces how many tenants move when the partition count changes — with a hash ring, adding one partition remaps roughly 1/N of tenants rather than nearly all of them — but it does not remove the need to drain the ones that do move.

Step 5 — Size the Partition Count

def partition_count(peak_jobs_per_min: float, p50_job_seconds: float,
                    per_partition_concurrency: int = 1) -> int:
    """Partitions cap concurrency when each is consumed by one worker."""
    jobs_per_worker_min = 60 / p50_job_seconds * per_partition_concurrency
    return max(8, math.ceil(peak_jobs_per_min / jobs_per_worker_min * 1.5))

partition_count(12_000, 0.4)     # 120 partitions — probably too granular
partition_count(12_000, 0.4, 8)  # 15  — with 8-way concurrency per partition

Where strict ordering is only needed per tenant rather than per partition, per-partition concurrency is safe as long as one tenant's jobs never run concurrently — enforce that with a short per-tenant lock rather than by collapsing concurrency to one.

Step 6 — Monitor Skew Directly

# Skew ratio — the single number that tells you whether the scheme still works
max(rate(jobs_processed_total[10m])) by (partition)
  / quantile(0.5, rate(jobs_processed_total[10m]))

Step 7 — Know What Partitioning Does Not Fix

Partitioning isolates queueing. It does nothing about shared downstream resources: if every partition's worker writes to the same database, a whale saturating that database still degrades everyone, and the queue metrics will look perfectly balanced while doing it. The same applies to a shared rate-limited API, a shared cache, and a shared connection pool.

The practical consequence is that partitioning should be paired with a per-tenant limit at the resource, not only at the queue. A per-tenant concurrency cap on database work, or a token bucket per tenant in front of the third-party API — the mechanisms in rate limiting and throttling jobs — is what converts queue-level isolation into genuine end-to-end isolation.

Partitioning also does not survive a workload whose ordering requirement is broader than one tenant. If a job for tenant A must run after a job for tenant B — a cross-tenant aggregation, say — no per-tenant partitioning scheme can express that, and the dependency belongs in the job graph rather than in the queue topology. Encode it as a parent-child relationship, as in BullMQ flows for parent-child jobs, rather than trying to obtain it from partition ordering.

Step 8 — Keep the Routing Table Somewhere Both Sides Can Read

The partition mapping is shared state between producers and consumers, and hard-coding it in application config guarantees they will disagree during a deploy. Put it behind a small service or a cached lookup so both sides read the same version.

class Routing:
    """Partition assignments with a short cache. Overrides live in the database
    so a whale can be moved without a deploy."""
    def __init__(self, store, ttl: int = 30):
        self._store, self._ttl, self._cache, self._at = store, ttl, {}, 0.0

    def partition_for(self, tenant_id: str) -> int:
        if time.time() - self._at > self._ttl:
            self._cache = self._store.load_overrides()   # {tenant: partition}
            self._at = time.time()
        if tenant_id in self._cache:
            return self._cache[tenant_id]
        digest = hashlib.blake2b(tenant_id.encode(), digest_size=8).digest()
        return SHARED_START + int.from_bytes(digest, "big") % SHARED_COUNT

The thirty-second cache is a deliberate compromise: it keeps the hot path free of a network call while bounding how long a producer can publish to a stale partition after an override changes. That window is exactly why the migration procedure in step 4 pauses the tenant rather than switching instantly — with the pause, a stale producer blocks; without it, a stale producer writes to the old partition while consumers have already moved, and the tenant's ordering breaks in a way that is very hard to reproduce afterwards.

Verification

1. The mapping is stable. Compute the partition for a tenant in two separate processes; the results must match. This catches salted hashes immediately.
2. Ordering holds per tenant. Enqueue a numbered sequence for one tenant under load and assert the consumer observed them in order.
3. Skew is bounded. Max-to-median partition throughput should stay under about 3× outside of migrations.
4. A migration preserves order. Move a tenant with the drain procedure while enqueueing continuously; the observed sequence must have no reordering.

Gotchas & Edge Cases

Salted or unstable hashes. Python's hash(), Java's String.hashCode() across versions, and anything seeded per process will silently split a tenant across partitions. Use an explicit stable hash.

Empty partitions still cost. Each is a queue with a consumer, connection, and dashboard. A hundred partitions for 4,000 tenants is usually more operational surface than it is worth.

Partition count changes are migrations, not config edits. Treat them with the same care as a schema change: plan, drain, verify.

Deleted tenants leave stale queues. Sweep partitions with no traffic for a long period, or the monitoring surface grows monotonically.

Per-partition consumers make deploys slower. With one consumer per partition, a rolling restart touches every partition in turn. Group partitions per worker process so a deploy handles several at once.

Related

• Queue Partitioning Strategies — partition keys, ordering models, and sizing in general.
• Scaling queue partitions in AWS SQS — the same problem with SQS's constraints.
• Preventing tenant starvation with weighted queues — fairness without partitioning, when ordering is not required.
• Rate Limiting & Throttling Jobs — isolating the shared resources partitioning cannot protect.