Design a Unique ID Generator

Architecture

Capacity Estimation

Snowflake

Twitter Snowflake: a 64-bit ID composed as (timestamp_ms || machine_id || sequence) with bit allocations like 41 + 10 + 12. Properties:

• Time-ordered — sortable by creation time. Indexes append at the tail (no random insert on B-tree).
• Coordination only at startup — each machine reserves a unique machine_id from ZooKeeper / etcd; runtime allocation is in-process.
• 4096 IDs/ms/machine — sequence overflow blocks until the next millisecond.
• 69 years of timestamp — with a custom epoch, you push the rollover problem far enough out.

Failure mode: clock goes backward (NTP step, leap second). Snowflake's standard handling: refuse to generate IDs until the clock catches up (block briefly) or alarm and shut down. Without this guard, two IDs can collide at the same (timestamp, machine_id, sequence).

UUIDv4 vs v7

• UUIDv4 — 122 random bits. Extremely unlikely collision (2⁶¹ generations before 1% probability). No coordination, no time component.
• UUIDv7 — 48-bit Unix millisecond timestamp + 74 random bits. Time-ordered like Snowflake but no machine_id. RFC 9562 (2024).

The case for v7 over v4: B-tree indexes on UUIDv4 keys fragment because new IDs land in random pages. UUIDv7's leading timestamp gives append-mostly inserts, dramatically reducing index bloat in Postgres and MySQL. Use v7 by default in any new system.

KSUID

Segment's K-Sortable Unique ID: 32-bit timestamp (seconds, custom epoch 2014) + 128 random bits, encoded as 27-character base62. Pros:

• Sortable lexicographically — works as a sortable text column without binary-aware tooling.
• Compact text representation — 27 chars vs UUID's 36.
• No coordination — pure-process generation.

Best when IDs flow as text (REST APIs, URLs) and you want time-ordering. Less efficient than 64-bit Snowflake when you index on it as binary.

ULID

Universally Unique Lexicographically Sortable Identifier: 48-bit Unix millisecond + 80 random bits, encoded base32 (Crockford's). 26 characters. Properties:

• Time-ordered with millisecond precision.
• Monotonic within the same millisecond — if two IDs generate in the same ms, the second increments the random portion. Strict same-process monotonicity, eventual cross-process.
• Crockford base32 avoids ambiguous characters (no 0/O, 1/I/L).

ULID and UUIDv7 occupy similar territory; UUIDv7 is the standard answer (it has an RFC). ULID is fine and existed first.

The Clock Skew Problem

Any time-prefix scheme depends on monotonic time. Failure modes:

• NTP step backward — clock jumps to past time; new IDs collide with already-issued ones. Defense: detect step (compare wall vs monotonic clock); refuse to issue or shift to a sequence-only mode until reconciled.
• VM live migration pause — clock jumps forward by seconds. Mostly harmless (gaps in timestamps); becomes a problem only if you exceed the timestamp encoding width unexpectedly.
• Leap second — depending on OS, clock either stalls or steps. Stall is fine; step is not. Modern Linux smear-leap-second avoids the worst.

Operationally: monitor NTP drift, alert at 100 ms divergence; ensure all hosts use the same sync source; consider PTP for sub-millisecond consistency in tight clusters.

Distributed Coordination Tradeoffs

• None (UUIDv4) — trivial; no time-order; large.
• Startup-only (Snowflake, machine_id assignment) — ZooKeeper/etcd at boot; runtime is in-process.
• Per-batch allocator — service grabs a range of IDs (1000) from a Postgres SEQUENCE; replenishes when low. Coordination amortized.
• Per-ID allocator — every generation is a network call to a central counter. Highest accuracy of monotonicity, lowest throughput.

Most production systems pick "startup-only" or "per-batch": 100× the throughput of per-ID with negligible monotonicity loss.

Failure Modes

• Two machines with the same machine_id — ZK lease race during deploy. Issue lease via etcd CAS; verify on each ID generation; refuse if invalidated.
• Sequence overflow — high-burst service exceeds 4096/ms budget. Snowflake stalls until next ms; fix by widening sequence bits or sharding the workload.
• Database key fragmentation from UUIDv4 — switch to v7 / Snowflake.
• Replicated DB with conflicting IDs — two regions both generate ID X. Use machine_id partitioning to ensure region-disjoint ID spaces.

FAQ