MongoDB Internals

Architecture Overview

Key Numbers

Why MongoDB Exists

WiredTiger Storage Engine

WiredTiger has been MongoDB's default engine since 3.2 (2015), replacing the old MMAPv1 mmap-based engine. It is a standalone B+ tree library that MongoDB embeds; it shares the basic shape of a Postgres B-tree but with several important differences tuned for document workloads.

• Per-document concurrency. WiredTiger uses MVCC with a per-page write lock. Two writers updating different documents on different pages do not block each other. The old MMAPv1 engine held a collection-level write lock — the single biggest reason teams migrated.
• Page size 32 KB. Pages split when they exceed this and are checkpointed every 60 seconds (or 2 GB of journal, whichever comes first). The page cache is sized at 50% of (RAM - 1 GB) by default.
• Block compression. Each block is compressed before write. Snappy is the default (fast, ~2x ratio). Zstd (since 4.2) gives ~3-4x at higher CPU cost. Indexes use prefix compression — a sorted run of keys with shared prefixes only stores the suffix.
• Two trees per collection. One B+ tree holds the documents keyed by a hidden RecordId (an 8-byte int). Each secondary index is its own B+ tree mapping {indexed_field, RecordId} → null. This is why secondary index lookups always require a final fetch back to the data tree — they are not covering by default.

// WiredTiger config (mongod.conf)
storage:
  engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 16              # explicit cache size
      journalCompressor: snappy    # snappy | zstd | zlib | none
    collectionConfig:
      blockCompressor: zstd        # per-collection override
    indexConfig:
      prefixCompression: true      # default true

BSON: The Document Format

MongoDB does not store JSON. It stores BSON — a binary-encoded superset of JSON that adds typed fields (Int32, Int64, Decimal128, Date, ObjectId, Binary, UUID) and length-prefixed strings/documents so the parser can skip fields without scanning byte by byte. BSON is what travels over the wire and what WiredTiger writes to disk.

// JSON the user writes
{ "_id": "alice", "age": 30, "tags": ["admin", "ops"] }

// BSON on the wire (annotated)
\x3a\x00\x00\x00              // total document length: 58 bytes
  \x02 _id\x00 \x06\x00\x00\x00 alice\x00      // 0x02 = string, length 6, value
  \x10 age\x00 \x1e\x00\x00\x00                // 0x10 = int32, value 30
  \x04 tags\x00 \x1f\x00\x00\x00               // 0x04 = array, length 31
    \x02 0\x00 \x06\x00\x00\x00 admin\x00
    \x02 1\x00 \x04\x00\x00\x00 ops\x00
    \x00
  \x00                        // document terminator

The 16 MB document size limit is not arbitrary — it's the largest size that fits comfortably in a single network message under the 48 MB max message size, with headroom for the wrapping OP_MSG envelope. Documents that need to grow beyond that should split into a parent doc plus a "bucket" collection (the time-series collections in 5.0+ formalize this pattern).

Indexes: B-trees on Document Paths

MongoDB indexes are WiredTiger B+ trees keyed on the value at a JSON path. They support compound keys, multikey (per-array-element) entries, partial filters, TTL expiry, sparse mode, and 2dsphere/text/hashed variants. The query planner picks one index per query stage and can intersect indexes only as a last resort — practically you design the compound index to match the query.

// Compound index on (status, createdAt DESC)
db.orders.createIndex(
  { status: 1, createdAt: -1 },
  { partialFilterExpression: { archived: false } }   // partial: only non-archived
);

// Multikey index over an array path
db.posts.createIndex({ "comments.author": 1 });
// Stores one entry per array element. A post with 50 comments
// produces 50 index entries — bloat scales with array length.

// TTL index
db.sessions.createIndex(
  { expiresAt: 1 },
  { expireAfterSeconds: 0 }   // background reaper deletes when expiresAt < now()
);

// Hashed index (used for hashed sharding)
db.users.createIndex({ userId: "hashed" });

The ESR rule for compound index field order: Equality first, then Sort, then Range. A query find({status: "open"}).sort({created: -1}) against an index {status:1, created:-1} can serve both predicate and sort without an in-memory sort. Reverse the order and the planner blocks on a SORT_KEY_GENERATOR stage that materializes the full result set in memory (and fails at 100 MB with the dreaded OperationFailed: Sort exceeded memory limit).

Sharding: Chunks, Balancer, mongos

Sharding is how MongoDB scales writes horizontally. It introduces three moving pieces: a shard key (one or more fields, immutable after collection sharding), chunks (contiguous ranges of the shard key, default 128 MB), and the balancer (a background process on the config server primary that migrates chunks to even out storage).

A query enters via mongos, which holds a cached chunk map from the config replica set. If the query includes the shard key, mongos routes it to exactly the shards owning matching chunks (a "targeted" query). Without the shard key, mongos broadcasts to every shard and merges results — a "scatter-gather" that scales linearly with shard count in the wrong direction.

// Two sharding strategies
sh.shardCollection("app.orders", { customerId: 1 });          // range
sh.shardCollection("app.events", { _id: "hashed" });          // hashed

// Compound shard key — better for query targeting
sh.shardCollection("app.events", { tenantId: 1, ts: 1 });

// Inspect chunk distribution
db.orders.getShardDistribution();
// Shard shard0000 at rs0/...:  Data 412.3 MiB docs : 1 200 000 chunks : 4
// Shard shard0001 at rs1/...:  Data 419.8 MiB docs : 1 220 000 chunks : 4
// Shard shard0002 at rs2/...:  Data 408.1 MiB docs : 1 180 000 chunks : 3

Replica Sets & the Oplog

Each shard is itself a replica set: 3-50 mongod processes (max 7 voting) that share a capped collection called the oplog. The primary writes operations to the oplog as part of every write; secondaries tail the oplog asynchronously and apply ops in order. The protocol is a derivative of Raft ("PV1" — protocol version 1) with leader leases.

// Sample oplog entry (local.oplog.rs)
{
  ts: Timestamp(1714857600, 1),                  // hybrid logical clock
  t: NumberLong(7),                              // election term
  h: NumberLong("-3829110238472..."),            // op hash
  v: 2,
  op: "u",                                       // u=update, i=insert, d=delete
  ns: "app.users",
  o2: { _id: ObjectId("66...") },                // query selector
  o:  { $v: 2, diff: { u: { lastSeen: ISODate("...") } } }
}

Election flow. Heartbeats every 2s. After electionTimeoutMillis (default 10 000) of missed heartbeats, a secondary increments its term and requests votes. To win, the candidate needs (a) a majority of voting members, (b) the most recent oplog entry among voters. End-to-end failover is typically 10-12 seconds. Tightening electionTimeoutMillis below 5s on cloud VMs almost always backfires — GC pauses and noisy neighbors trigger spurious elections that churn through terms.

Oplog sizing. The oplog is capped at 5% of free disk by default. Its purpose is to give a temporarily disconnected secondary enough time to catch up before falling off — once an op is gone from the oplog, the secondary requires a full resync. For high write rates, size the oplog to cover at least 24 hours of operations.

Read & Write Concerns

MongoDB exposes durability as a per-operation knob. Pick the right concern for the operation, not the deployment.

Aggregation Framework

Aggregation is MongoDB's answer to "we still need analytical queries." A pipeline is an ordered list of stages, each a transformation. The planner can push $match and $sort down to use indexes, and recent versions push $group with hashed partitioning across shards ($mergeCursors).

db.orders.aggregate([
  { $match: { status: "shipped", createdAt: { $gte: ISODate("2024-01-01") } } },
  { $lookup: { from: "customers", localField: "customerId",
               foreignField: "_id", as: "customer" } },
  { $unwind: "$customer" },
  { $group: { _id: "$customer.country",
              revenue: { $sum: "$total" },
              orderCount: { $sum: 1 } } },
  { $sort: { revenue: -1 } },
  { $limit: 20 }
], { allowDiskUse: true });   // spill to disk if > 100 MB per stage

The $lookup stage is a left-outer join, but it is not distributed: it runs on the shard executing the pipeline and issues per-document queries to the foreign collection. For high cardinality joins this is a footgun. Materialized views via $merge / $out are the production answer.

Multi-Document Transactions

Available since 4.0 (replica sets) and 4.2 (sharded clusters). They use WiredTiger snapshot isolation underneath; cross-shard transactions add a two-phase commit coordinator. The hard cap is 60 seconds — long-running txns abort with TransactionExceededLifetimeLimitSeconds.

const session = client.startSession();
try {
  session.startTransaction({
    readConcern:  { level: "snapshot" },
    writeConcern: { w: "majority" }
  });

  await accounts.updateOne({ _id: "a" }, { $inc: { bal: -50 } }, { session });
  await accounts.updateOne({ _id: "b" }, { $inc: { bal:  50 } }, { session });

  await session.commitTransaction();
} catch (e) {
  await session.abortTransaction();
  throw e;
} finally {
  session.endSession();
}

The MongoDB team's own guidance is to design schemas so transactions are rarely needed. If your aggregate fits in one document, you get atomic writes for free — every single-document update is atomic without any transaction wrapping.

Schema Validation

Since 3.6, collections accept a JSON Schema validator. The validator runs on every insert/update; validationAction decides whether violations warn (logged, accepted) or error (rejected). validationLevel chooses whether existing documents (pre-validator) are checked on update.

db.createCollection("orders", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["customerId", "total", "status"],
      properties: {
        customerId: { bsonType: "objectId" },
        total:      { bsonType: "decimal", minimum: 0 },
        status:     { enum: ["pending", "shipped", "cancelled"] },
        items:      { bsonType: "array", minItems: 1 }
      }
    }
  },
  validationLevel: "strict",      // strict | moderate | off
  validationAction: "error"       // error | warn
});

Tradeoffs & When To Use

FAQ