Computer Architecture

Computer Architecture

Every line of code you write eventually becomes electrical signals in silicon. The path from high-level language to executed instruction involves compilers, instruction set architectures, and microarchitectural mechanisms that are invisible but determinative — the difference between code that runs in 10ms and code that runs in 10 seconds.

Instruction Set Architecture (ISA)

The ISA is the contract between software and hardware: the set of instructions, registers, and memory model that a CPU supports. Programs compiled for x86-64 run on any chip implementing that ISA, regardless of internal microarchitecture.

RISC vs CISC: Reduced Instruction Set Computing (ARM, RISC-V) uses simple, fixed-length instructions that execute in one cycle, relying on compilers to combine them. Complex Instruction Set Computing (x86) has hundreds of complex, variable-length instructions. Modern x86 processors internally translate CISC instructions into RISC-like micro-operations (μops), combining the software ecosystem of CISC with the microarchitectural advantages of RISC.

Registers are the CPU's fastest storage — 16-32 general-purpose registers, each holding 64 bits. Accessing a register is a single cycle. The goal of compiler register allocation is keeping frequently used values in registers rather than memory.

The Pipeline

A modern CPU executes instructions in stages: Fetch (read instruction from memory/cache), Decode (parse opcode and operands), Execute (perform arithmetic/logic in ALU), Memory (read/write data memory if needed), Writeback (store result in register).

Pipelining overlaps these stages — while instruction 5 executes, instruction 6 decodes and instruction 7 fetches. A 5-stage pipeline can theoretically execute 5 instructions per cycle (CPI of 1). Modern out-of-order superscalar CPUs handle 4-8 instructions per cycle by executing independent instructions simultaneously.

Hazards disrupt the pipeline. Data hazards: instruction B needs a result instruction A hasn't produced yet — solved by forwarding (passing results directly between stages) or stalling. Control hazards: branch instructions don't know the next PC until execution — solved by branch prediction. Structural hazards: two instructions need the same hardware unit simultaneously — solved by duplicating functional units.

Branch Prediction

Modern CPUs are deeply pipelined (10-20+ stages). A branch misprediction means flushing all in-flight instructions and restarting — a 15-20 cycle penalty. With branch prediction, the CPU guesses the branch outcome and continues filling the pipeline speculatively.

Modern predictors (tournament, TAGE) achieve >99% accuracy on typical workloads. The remaining 1% of mispredictions still dominate runtime in branch-heavy code. This is why sorting data before searching can speed up code: predictable branch patterns reduce mispredictions (the "branch prediction warm-up" effect in benchmarks).

Spectre and Meltdown (2018) demonstrated that speculative execution of mispredicted branches can access and leak privileged memory through cache side channels — the most severe hardware security vulnerabilities in decades.

Memory Hierarchy

The gap between CPU speed and DRAM latency is enormous: a register read takes 0 cycles; an L1 cache hit takes 4 cycles; L2 takes 12 cycles; L3 takes 40-50 cycles; DRAM takes 200-300 cycles. A cache miss to DRAM costs 200x more than an L1 hit.

Caches exploit locality. Temporal locality: recently accessed data will be accessed again. Spatial locality: nearby data will be accessed soon. Caches are organized into cache lines (64 bytes on x86) — the unit of transfer. Accessing any byte in a cache line loads the whole line.

Cache associativity balances simplicity and hit rate. Direct-mapped caches map each address to one cache slot — fast but prone to conflict misses. Fully associative caches allow any address in any slot — optimal hit rate but expensive. N-way set-associative (4-way, 8-way) balances both.

Cache coherence in multi-core systems ensures all cores see a consistent memory state. The MESI protocol assigns states to cache lines: Modified (dirty, exclusive), Exclusive (clean, exclusive), Shared (clean, multiple caches have it), Invalid (stale). When one core writes a Shared line, it sends an invalidation to other cores — a cache-to-cache transfer that takes ~50 cycles but is far faster than going to DRAM.

Writing Cache-Friendly Code

The fastest code manipulates data in patterns that maximize cache hits. Key techniques:

Sequential access: iterate through arrays element by element, not with large strides. A stride of 64 (skipping every cache line) reduces effective bandwidth by 64x.

Structure of Arrays vs Array of Structures: for processing one field across many objects, Structure of Arrays (separate arrays per field) has better cache behavior than Array of Structures (objects with all fields together).

Cache blocking (tiling): matrix multiplication naively has poor cache behavior. Tiling the computation into blocks that fit in cache can improve performance 10x.

SIMD and Parallelism

Modern CPUs support SIMD instructions: AVX-512 on x86 can add 16 float-32s in a single instruction. Compilers auto-vectorize simple loops; critical inner loops may need manual SIMD intrinsics or libraries that use them. NumPy, OpenBLAS, and deep learning frameworks are largely SIMD-optimized C/C++ libraries.

Hardware threads (SMT): Intel Hyper-Threading lets one physical core appear as two logical cores, interleaving two threads' instructions to hide memory latency. This improves throughput for memory-bound code but not for compute-bound code (both threads share the same execution units).