Building a Low-Latency
Order Book from Scratch

Chapter 01C++ Foundations You Actually Need

You don't need all of C++ to build high-performance systems. You need a specific subset, and you need to understand it deeply. This chapter covers the pieces that matter for our order book.

Types and Their Sizes

Every variable in C++ has a type, and every type has a fixed size in bytes. In systems programming, you must know exactly how much space each thing occupies.

Use int32_t and int64_t (from <cstdint>) for guaranteed sizes. Plain int could be 2, 4, or 8 bytes depending on the platform. In systems code, "it depends" is unacceptable.

Pointers

A pointer stores a memory address — literally a number that says "the data I point to lives at location 0x7ff4a2b3c000." That's all it is.

Pointers are the foundation of everything in this project: linked lists, memory pools, and direct data access without copying.

Structs — Grouping Data Together

A struct groups related fields into one unit. The compiler lays them out sequentially in memory:

struct Order {
    int64_t  id;        // 8 bytes, offset 0
    int64_t  price;     // 8 bytes, offset 8
    int64_t  quantity;  // 8 bytes, offset 16
    Side     side;      // 1 byte,  offset 24
    Order*   prev;      // 8 bytes, offset 32 (after padding!)
    Order*   next;      // 8 bytes, offset 40
};

Notice the gap: side ends at byte 25, but prev starts at byte 32. The compiler inserts 7 invisible padding bytes to align the pointer to an 8-byte boundary. We'll exploit this in Chapter 8.

Templates

Templates let you write code that works with any type. The compiler generates a specialized version for each type you use — zero runtime cost.

template <typename T>
class MemoryPool {
    T* allocate();          // returns a T-sized slot
    void deallocate(T*);    // reclaims a slot
};

MemoryPool<Order> pool(10000);   // compiler generates MemoryPool_Order
Order* o = pool.allocate();       // returns an Order-sized slot

static_assert — Compile-Time Safety Nets

If the condition is false, the build fails. Encode your assumptions so they can never silently break:

static_assert(sizeof(Order) == 64, "Order must be exactly one cache line");
static_assert(alignof(Order) == 64, "Order must be cache-line aligned");

If someone adds a field and Order grows past 64 bytes, the compiler catches it immediately. Compare this to a runtime bug that only surfaces in production at 3 AM.

RAII — Automatic Cleanup

Acquire resources in constructors, release them in destructors. Since C++ guarantees destructors run when objects leave scope, cleanup is automatic:

class MemoryPool {
    MemoryPool(size_t n) { storage_ = allocate_slab(n); }  // acquire
    ~MemoryPool()        { free(storage_); }                // release
};

No try/finally. No garbage collector. No "did I forget to call close()?" Just scoping rules.

Chapter 02What Is an Order Book?

Every exchange — NYSE, NASDAQ, CME — runs an order book at its core. It's the data structure that matches buyers with sellers.

The Basic Idea

Traders submit limit orders: "I want to buy 100 shares at $50.03" or "I want to sell 200 shares at $50.05." The order book collects these and organizes them by price.

Bids are sorted highest-first. If you're selling, you want the buyer paying the most. Asks are sorted lowest-first. If you're buying, you want the cheapest seller. The spread is the gap between them.

Price-Time Priority (FIFO)

When multiple orders sit at the same price, who gets filled first? Whoever arrived first. This is called FIFO matching.

Three Operations — That's All

An order book processes exactly three event types. On a busy exchange, these arrive millions of times per second.

Chapter 03Representing Money — Fixed-Point Arithmetic

Your first instinct for a price like $50.03 is double price = 50.03;. This is a trap.

Why Doubles Lie

The Fix: Multiply, Store as Integer

Chapter 04STL Containers — What They Hide from You

The C++ Standard Library gives you std::map, std::vector, std::deque. They're correct and general-purpose. That last part is the problem.

std::map — The Elegant Trap

A std::map<Price, PriceLevel> seems perfect: keys are sorted, lookup is O(log N). Under the hood, it's a red-black tree where each node is a separate heap allocation, scattered across memory.

What We Replaced Them With

Chapter 05Memory: The Stack, the Heap, and Why new Is Slow

Two Places to Store Data

What Happens Inside new

Chapter 06Building a Memory Pool

If every object is the same size, you don't need a general-purpose allocator. Pre-allocate a slab, hand out fixed-size slots.

How the Free List Works

Chapter 07Intrusive Data Structures

Each price level is a FIFO queue. With std::deque, cancelling an order means scanning to find it — O(K). We need O(1).

Non-Intrusive vs. Intrusive

The Cancel Operation — Three Pointer Rewires

Chapter 08Cache Lines and Struct Layout

Your CPU doesn't read individual bytes from RAM. It reads 64-byte chunks called cache lines. Understanding this changes how you design data structures.

The Memory Hierarchy

Cache Line Alignment

Chapter 09Flat Arrays vs. Trees

Stock prices exist on a fixed grid. A stock between $49.00 and $51.00 with $0.01 ticks has exactly 201 possible levels. Why use a tree when you can use an array?

Chapter 10Putting It All Together — The Order Book

Operation Summary

Chapter 11Benchmarking — Proving It Works

Every optimization was driven by a benchmark, not a hunch. "I think this will be faster" is worthless. "This benchmark shows 217x improvement" is a fact.

Common Traps

Our Results

Chapter 12System Architecture — Why HTTP Doesn't Belong Here

The Cost of HTTP

The Pipeline Model — No Network Between Components

Chapter 13Binary Protocols & Zero-Copy Parsing (ITCH)

The order book has a clean C++ API. But a real exchange never calls your function — it sends you bytes over a wire. This chapter is the translator: raw NASDAQ ITCH bytes in, book.add_order(...) out, with no parsing, no copying, and no allocation.

Why binary, and why it changes everything

A JSON "add order" is ~200 bytes of text you must parse — walk each character, find the quotes, convert the string "50.03" into a number. The ITCH equivalent is 36 bytes with zero parsing: the number is already sitting there as raw bytes in the exact layout your CPU uses.

That's the whole trick. If the bytes on the wire match the memory layout of a C++ struct, you don't parse — you point a struct at the bytes. Making that safe forces you to understand four things cold, because interviewers drill every one.

1. Struct layout — padding is the enemy

The compiler aligns each field to a "nice" address and fills the gaps with invisible padding. A wire message has no padding. Point a padded struct at an unpadded message and every field past the first reads garbage.

2. Endianness — which end of the number goes first

The number 500,300 needs 4 bytes. In what order? Your x86 CPU stores least-significant-byte-first (little-endian). Network protocols traditionally send most-significant-first (big-endian). Read big-endian bytes as little-endian and the value is scrambled. Only multi-byte fields need care; a single byte has no order.

// C++20 lets us swap only when the host disagrees with the wire — at compile time.
inline std::uint32_t from_be(std::uint32_t v) {
    if constexpr (std::endian::native == std::endian::little) return __builtin_bswap32(v);
    else return v;   // already big-endian host: nothing to do
}

3. Zero-copy — and the trap inside it

The naive move is reinterpret_cast<const AddOrder*>(buffer): no copy, the struct just overlays the bytes. It's genuinely how fast systems work — and it's technically undefined behavior under the strict aliasing rule, which says you may only read memory through a pointer of its real type (with char/std::byte the exception). Those bytes were never "born" an AddOrder.

4. Framing & dispatch — where does one message end?

The wire is a continuous stream; nothing visually separates messages. Three strategies exist: fixed size per type (ITCH: the type byte implies the length), length prefix (a 2-byte field says "I am N bytes" — ITCH's transport layer), and delimiters (text protocols only). Our parser reads the 2-byte big-endian length, slices one message, looks at the type byte, and switches to the right book call. Unknown types are skipped by their length — which is why framing matters.

while (off + 2 <= len) {
    std::uint16_t msg_len = read_be16(data + off);          // 1. frame
    if (off + 2 + msg_len > len) break;                      //    (truncated tail?)
    dispatch(data + off + 2, msg_len);                       // 2. type byte -> book
    off += 2 + msg_len;                                      // 3. advance
}

The interview scorecard

Verified: the parser sustains ~7.15 million ITCH messages/second (236 MiB/s) through the full frame → swap → dispatch → book pipeline.

Chapter 14Measuring the Tail — Latency Histograms & Replay

This is the feature that changes the project's category. Anyone can benchmark an average. The showcase is: replay a full trading day and publish the tail-latency distribution — p50, p99, p99.9, max — because in trading the mean is a lie. What matters is the worst message, since that's the one that loses money.

Why the average hides the danger

The histogram: HdrHistogram in ~2 KB

Values are bucketed logarithmically. Each power-of-two "octave" is split into 32 linear sub-buckets, so relative error is bounded to ~3% across the whole range from 1 ns to years, using a fixed array and an O(1) record(). Percentiles are a single walk of the cumulative counts.

Timing a 10 ns operation honestly

You cannot measure a ~10 ns book operation with a ~30 ns std::chrono call — the clock would dominate. So the replay uses the CPU timestamp counter (rdtsc, ~6 ns), calibrated against steady_clock at startup to convert ticks to nanoseconds. That calibration is itself a good interview line.

The result — a real measure → diagnose → fix arc

Replaying a synthesized million-step day, the first run showed a median of 162 ns but a hideous tail. Diagnosis: the order-id unordered_map was rehashing as it grew to ~470k entries. Fix: pre-reserve the map to pool capacity in the constructor. Re-measured:

Chapter 15Lock-Free Programming — The SPSC Ring Buffer

To hand data from the network thread to the processing thread, you cannot use a mutex: a lock costs ~25 ns uncontended and blocks the thread entirely when contended — a stall the hot path can't afford. The answer is a lock-free single-producer / single-consumer ring buffer. This is the classic HFT interview topic, so understand every word.

The structure

Two free-running counters, head (consumer) and tail (producer). The slot is counter & (Capacity-1) (capacity is a power of two, so the mask is one AND). Size is tail - head. No locks — coordination is two atomics.

Memory ordering — the crux

Modern CPUs and compilers reorder memory operations for speed. Left unchecked, the consumer could see the new tail before the producer's write to the slot lands — and read a half-written message. Acquire/release ordering forbids exactly that.

Each thread reads its own counter relaxed (it's the only writer) and the other's counter with acquire. The producer's release store pairs with the consumer's acquire load to create a happens-before edge — the guarantee that makes the whole thing correct.

False sharing — the silent killer

Cache coherence works at 64-byte cache-line granularity. If head and tail shared a line, every producer write to tail would invalidate the consumer's cached copy of head (and vice versa) — two cores ping-ponging one line, turning a lock-free design into a cache-thrash bottleneck. The fix is one keyword:

alignas(64) std::atomic<std::size_t> head_{0};   // its own cache line
alignas(64) std::atomic<std::size_t> tail_{0};   // its own cache line

The interview scorecard

Chapter 16The Pipeline — UDP and Threads

Now assemble the whole thing the way a real feed handler is built: the network at the edge, lock-free handoffs in the middle, the hot processing thread running undisturbed.

Why UDP, not TCP

Market data is UDP multicast: connectionless, unordered, best-effort. That's the right choice — a lost packet is stale a microsecond later anyway, so you never want TCP stopping the world to retransmit ancient prices. The cost is that you must tolerate loss (real feeds add sequence numbers and a gap-fill channel).

Chapter 17Finding the Tail — Profiling

You can't optimize what you can't see. Profiling is how you turn "it feels slow" into "the unordered_map rehash is 60% of the p99.9." The tools you should be able to name and use:

What actually causes tails

• Rehashing / reallocation — a hash map doubling its buckets copies everything. Fix: pre-reserve. (We did — it cut p99.99 by ~45%.)
• Cold-page faults — first touch of freshly-allocated memory traps into the kernel. Fix: pre-fault at startup.
• Cache misses — pointer-chasing scattered memory. Fix: contiguous layout (our flat arrays and pool).
• Scheduling jitter — the OS deschedules your thread. Fix: pin to an isolated core.

Chapter 18Production Hardening

The difference between a benchmark and a system is what happens at the edges: startup, shutdown, failure, and observability — without ever slowing the hot path.

Warm up before you trade

The first message pays for everything cold: unmapped pages, empty caches, an untrained branch predictor. Real systems run a warmup pass before the market opens. We pre-fault the memory pool at construction (commit every page up front), moving that cost off the latency-critical path. It's opt-in so tests stay fast.

Never log on the hot path

A single blocking printf or file write in the processing loop can cost microseconds and stall everything behind it. The discipline: the hot path writes structured events into a lock-free buffer (like our ring); a separate low-priority thread drains and formats them. The trading thread never touches I/O.

Graceful shutdown & backpressure

Threads coordinate teardown through atomic flags with release/acquire ordering — the producer signals "done," the consumer drains the ring, then exits. Under overload, backpressure (the receiver spinning when the ring is full) is a deliberate policy choice: block the slow source rather than silently lose data mid-book.

Chapter 19The Interview — The Grill Sheet

The consolidated toughest questions across the whole project, with crisp answers. If you can defend every row from memory, the project is bulletproof.

Data structures & memory

Correctness & money

Systems & concurrency

The one meta-question

Chapter 20What's Next

The Road So Far

Chapter 21Why This, Not That — The Decision Log

Interviewers rarely ask "what did you build." They ask "why did you build it that way, and what else did you consider?" Every choice below has a rejected alternative. If you can defend the whole table, you can defend the project.

1. Price = fixed-point int64

Interview trap: "Why ×10000 specifically?" → four decimal places covers sub-penny US equity ticks with room to spare; it's a convention, not a law.

2. Price levels = flat array, not a tree

A flat array indexed by (price−min)/tick is O(1), contiguous, and the "next best" is a short cache-friendly scan. The cost is memory for the full price grid — fine for one symbol.

Interview trap: "What if the price range is huge or unknown (crypto)?" → then you'd use a hybrid: a hash map of active levels plus a coarse index, or a bounded window that recenters. The flat array is the right call for a bounded-tick equity; naming its limit is the point.

3. Order queue = intrusive doubly-linked list

Embedding prev/next in the Order gives O(1) cancel with zero extra allocation. The trade-off: the Order is coupled to one list (fine — it lives at exactly one level).

4. Allocation = free-list pool

Interview trap: "What happens when the pool is full?" → add_order returns false; the caller decides (reject / grow / alert). We never silently allocate and blow the latency budget.

5. Order lookup = unordered_map<id,Order*>, pre-reserved

By-id cancel/execute needs a fast id→Order map. A hash map is O(1) average; we reserve it to pool capacity so it never rehashes mid-session (that was the measured p99.9 spike). Alternative — a sorted structure — buys nothing since we never iterate ids in order.

6. Deserialization = memcpy, not the cast

std::memcpy into a real struct is well-defined and compiles to the same load; C++20 std::bit_cast is the modern spelling.

7. Thread handoff = lock-free SPSC ring

8. Transport = UDP

TCP guarantees delivery by retransmitting — but a retransmitted price is stale garbage. UDP's best-effort delivery is correct for market data; you handle loss with sequence numbers + a gap-fill channel, not by blocking. Shared memory beats UDP for same-box feeds (and real colocated systems use it), which is exactly the "what's next" direction.

9. Timing = calibrated rdtsc

std::chrono::steady_clock is ~20–30 ns per call — it would dominate a 10 ns measurement. rdtsc is ~6 ns; we calibrate it against steady_clock at startup to get ns. clock_gettime is portable but still a function call. The honest caveat: even rdtsc can't cleanly resolve a single 10 ns op, which is why the histogram's value is the tail, not the median.

10. Histogram = HdrHistogram-style log buckets

11. Language = C++

Deterministic destruction (no GC pauses — the reason Java is a hard sell for the hot path), manual memory control, zero-overhead abstractions, and direct hardware access (alignas, intrinsics). Rust offers the same performance with memory-safety guarantees and is a genuine modern alternative; C is lower-level but you lose RAII and templates. C++ is the incumbent because the ecosystem and the control are both there.

12. Build = header-only lib + Docker Alpine

Header-only keeps the whole engine inlinable across translation units (the optimizer sees everything) and the project trivially buildable. Docker/Alpine makes it run identically on any machine with no toolchain install — the reason you can clone this and get numbers in one command. Bare metal with core pinning is where you'd go for real latency work (Chapter 17).

Chapter 22C++ From Scratch — Every Concept This Project Uses

A ground-up reference for every language and library feature the code touches, basic to advanced. If a term in the project is unfamiliar, it's defined here — with why it appears and the nuance an interviewer probes.

A. Types & values

• Fixed-width integers (<cstdint>: int64_t, uint32_t, uint8_t) — exact, guaranteed sizes. Plain int is "at least 16 bits," which is unacceptable when a struct must match a wire layout byte-for-byte.
• Type aliases (using Price = std::int64_t;) — give a domain meaning to a raw type. Zero runtime cost; pure readability and single-point-of-change.
• enum class Side : uint8_t — a scoped enum with an explicit 1-byte underlying type. Scoped means Side::Buy can't silently convert to int (catches bugs); the uint8_t keeps it one byte for packing.
• std::size_t — the unsigned type for sizes/indices; matches what containers return. Mixing it with signed int triggers warnings (we compile with -Werror, so we're forced to be correct).
• Designated initializers (C++20: BookConfig{.min_price=400000}) — name fields at construction for clarity and safety against field reordering.

B. Memory model & layout

• Stack vs heap — stack: instant, auto-freed, limited (~8 MB); heap: flexible, manual, slow to allocate. Our 11 MB ring buffer must live on the heap (stack overflow otherwise — a bug we actually hit).
• Alignment (alignof, alignas) — a type's required address multiple. alignas(64) pins an object to a cache-line boundary. Used on Order (one line per order) and on the ring's counters (false-sharing avoidance).
• sizeof / padding — the compiler inserts gaps to satisfy alignment. Understanding this is why wire structs need #pragma pack(1) and why our Order has spare bytes we repurpose.
• Cache lines (64 bytes) — the unit the CPU actually fetches. Contiguous data is prefetched for free; scattered data misses. The single biggest performance lever in the project.
• #pragma pack(1) — disables padding so a struct matches an external byte layout exactly. Essential for the ITCH wire structs; static_assert(sizeof(...) == N) proves it worked.

C. Pointers, references, const

• Pointer — a variable holding an address (8 bytes on 64-bit). The backbone of intrusive lists (prev/next) and the pool.
• Reference — an alias for an existing object; can't be null or reseated. We pass the OrderBook& into the parser to avoid copying and signal "this must exist."
• const-correctness — marking read-only methods/params const lets the compiler enforce intent and enables optimization. get_bbo() const promises it won't mutate the book.

D. Classes & object lifetime

• Constructor / destructor — run on creation/destruction. The foundation of RAII.
• RAII — "Resource Acquisition Is Initialization": acquire in the constructor, release in the destructor, cleanup is automatic on scope exit. The pool frees its slab, the socket closes its fd — no leaks, no finally.
• Rule of five / deleted copies — a class managing a resource must define (or delete) its copy/move operations. We = delete copy/move on the pool, ring, and sockets so nobody accidentally double-frees a slab or fd.
• explicit — stops a constructor from being used for silent implicit conversions. Small safety habit throughout.

E. Templates & compile-time

• Class templates (MemoryPool<T>, SpscRingBuffer<T,N>) — the compiler stamps out a specialized version per type, with the exact size/layout baked in. Generic and zero-overhead.
• Variadic templates + perfect forwarding (allocate(Args&&...) → ::new(slot) T(std::forward<Args>(args)...)) — forward any constructor arguments through the pool to placement-new, preserving value categories.
• constexpr / if constexpr — computed at compile time. if constexpr (std::endian::native == little) compiles the byte-swap in or out with zero runtime branch.
• static_assert — a compile-time test. We pin wire sizes and the power-of-two ring capacity so a mistake fails the build, not production.

F. Dynamic allocation internals

• new-expression vs operator new — new T() does two things: get memory, then construct. Separating them is the key idea behind the pool.
• Placement new (::new(ptr) T(...)) — construct an object in memory you already own. How the pool builds an Order in a pre-allocated slot with no allocation.
• Explicit destructor call (ptr->~T()) — the matching teardown for placement new; run the destructor without freeing the memory (the slot returns to the free list).

G. Casting & undefined behavior

• static_cast — checked, related-type conversions (e.g. uint32→int64 price). The cast you should reach for by default.
• reinterpret_cast — "trust me, treat these bytes as that type." Powerful and dangerous; the source of the strict-aliasing trap.
• Strict aliasing — the rule that you may only access an object through a pointer of its real type (char/std::byte excepted). Breaking it is UB the optimizer may exploit. memcpy / bit_cast are the safe path.
• Undefined behavior — the standard imposes no constraints; the program may do anything. The catalog we actively avoid: aliasing violations, misaligned access, signed overflow, reading uninitialized memory, data races.

H. Bit manipulation

• Masks & power-of-two — x & (N-1) equals x % N when N is a power of two, in one instruction. The ring buffer's wrap.
• Byte swap (__builtin_bswap32) — reverse byte order for endianness conversion, one instruction.
• Count-leading-zeros (__builtin_clzll) — find the highest set bit to compute a value's magnitude; the histogram's log-bucket index.

I. Concurrency (the deep end)

• Data race — two threads touch the same memory, at least one writing, with no synchronization. UB. The thing atomics exist to prevent.
• std::thread — an OS thread; join() waits for it. The pipeline runs three.
• std::atomic — a type whose operations are indivisible and synchronizing. The ring's head/tail.
• Memory order — how much reordering around an atomic is allowed. relaxed: atomicity only. acquire (loads): nothing after it moves before. release (stores): nothing before it moves after. An acquire that reads a release's value establishes happens-before: the reader sees everything the writer did prior. That pairing is what makes the ring correct. (seq_cst, the default, is stronger and slower — we don't need it.)
• False sharing — two threads writing different variables that share a cache line, forcing the line to ping-pong between cores. Fixed with alignas(64).
• Lock-free vs wait-free — lock-free: the system as a whole always progresses (our ring). Wait-free: every thread finishes in bounded steps (stronger, rarer). Neither uses a mutex.

J. Standard library containers used

K. Time & randomness

• std::chrono — type-safe durations/clocks. steady_clock (monotonic, for measuring intervals) vs system_clock (wall time, can jump). Always steady_clock for latency.
• std::mt19937_64 — a seeded, deterministic PRNG. A fixed seed makes the synthetic day reproducible — same input every run, so numbers are comparable.

L. Error handling

• Exceptions (std::bad_alloc, std::runtime_error) — used at startup/setup (pool allocation, socket bind), never on the hot path (throwing is expensive and unpredictable).
• Return codes on the hot path — add_order returns bool, execute_order returns filled quantity. The caller branches; no exception cost per message.
• assert — a debug-build sanity check (compiled out in Release). Guards invariants like a positive tick size.

M. Tooling as correctness

• -Wall -Wextra -Wpedantic -Werror — every warning is a build failure. This is why the project has zero warnings: they can't rot into bugs. (It's also why -Waddress-of-packed-member forced us to read packed fields by value, not by address.)
• CMake + FetchContent — declarative build; pulls GoogleTest/Benchmark at configure time so there's nothing to install.
• static_assert as a test — the cheapest test there is: it runs at compile time and costs nothing at runtime.

Built as part of the HFT Order Book Learning Project — June 2026
C++20 · CMake · GoogleTest · Google Benchmark · Docker