Hardware Memory Models: Store Buffers, Memory Barriers, and Out-of-Order Execution

01Why Modern CPUs Do Not Execute Instructions Sequentially

In high-performance systems engineering, the single biggest misconception is assuming that the CPU executes machine instructions in the exact order written in source code. To maximize instruction throughput, modern superscalar CPUs execute instructions Out-of-Order (OoO).

CPUs contain reorder buffers (ROB), register renaming units, and speculative branch predictors that execute independent instructions simultaneously. Furthermore, the optimizing C++ compiler itself rearranges instructions during compilation to minimize register pressure and hide memory latency.

In a single-threaded program, the hardware and compiler maintain the 'as-if' rule: observable behavior is identical to sequential execution. In multi-threaded systems, however, hardware reorderings become directly visible to other CPU cores, causing bizarre memory consistency bugs.

02Store Buffers and Store-Load Reordering

Writing data to L1 cache takes several clock cycles. To prevent the CPU execution engine from stalling on every store instruction, modern processors write values to an internal FIFO queue called the Store Buffer.

The CPU continues executing subsequent instructions immediately while the store buffer drains to cache in the background. If a subsequent instruction is a read from a different address (a Load), the CPU reads from cache before the earlier store has arrived in cache!

This causes Store-Load Reordering: to other CPU cores, the Load appears to execute BEFORE the earlier Store.

03Strong vs Weak Hardware Architectures

Hardware architectures exhibit fundamentally different memory consistency models:

x86 / x86_64: Total Store Order (TSO). A strongly ordered architecture. Loads are not reordered with loads; stores are not reordered with stores. ONLY Store-Load reordering is permitted.

ARM64 / Apple Silicon / POWER: Weakly Ordered Architectures. Hardware allows almost all reorderings: Load-Load, Load-Store, Store-Store, and Store-Load! Code that appears to work on x86 will crash with subtle race conditions when ported to ARM unless explicit memory barriers are used.

04Hardware Memory Model Comparison

Permitted hardware memory reorderings across processor architectures:

The C++ memory model abstracts these hardware differences, providing uniform cross-platform atomic guarantees.

+-----------------------+---------------------+-------------------+-------------------------+| Architecture          | Store-Load Reorder  | Load-Load Reorder | Store-Store Reorder     |+-----------------------+---------------------+-------------------+-------------------------+| x86 / x86_64 (TSO)    | YES (Store buffer)  | NO                | NO                      || ARM64 / AArch64       | YES                 | YES               | YES                     || POWER                 | YES                 | YES               | YES                     || C++ Abstract Machine  | Governed by std::memory_order enum semantics                          |+-----------------------+---------------------+-------------------+-------------------------+

05Production Systems Architecture & Hardware Trade-Offs

High-performance multi-threaded systems engineering demands a rigorous mental model of hardware memory architectures. Modern multi-socket servers feature Non-Uniform Memory Access (NUMA), where memory access latency varies depending on whether a thread accesses local or remote CPU socket memory controllers. Without careful thread affinity and memory pinning, cross-socket bus contention can degrade throughput by orders of magnitude.

Furthermore, when building lock-free pipelines with atomic variables, developers must recognize the cost of hardware memory fences. While relaxed memory orderings compile to raw unconstrained CPU instructions, acquire-release orderings synchronize cache lines across cores with minimal pipeline stalls, providing the ideal foundation for ultra-low-latency asynchronous message passing.