Abstract
The Memory Hierarchy structures storage in hierarchical layers to address three fundamental design constraints: high operational speed, low power consumption, and timing predictability. Because no single physical storage technology can simultaneously offer infinite bandwidth, zero leakage power, low cost, and deterministic single-cycle access, architectures combine small, fast volatile caches with dense main memory and persistent mass storage.
System Requirements & The Architectural Trade-Off
Modern processor architectures balance three competing memory demands:
- Speed (Low Latency & High Bandwidth): Minimizing CPU stall cycles during data fetch and store operations.
- Low Power: Reducing static leakage power in dense on-chip arrays and dynamic switching power during off-chip bus transfers.
- Predictability: Guaranteeing hard upper bounds on memory access latencies for real-time interrupt handlers and embedded control loops.
flowchart TD TOP["▲ <b>Fastest / Lowest Latency / Highest Cost per Bit</b>"] L1["<b>Registers</b> (< 1 ns)"] L2["<b>L1 / L2 / L3 Caches</b> (1 - 30 cycles)"] L3["<b>Main Memory / DRAM</b> (100 - 300 cycles)"] L4["<b>Non-Volatile Flash / SSD</b> (25,000 - 2,000,000 cycles)"] L5["<b>Secondary Disk Storage</b> (> 5,000,000 cycles)"] BOT["▼ <b>Slower / Higher Density / Lowest Cost per Bit</b>"] TOP ---> L1 L1 ---> L2 L2 ---> L3 L3 ---> L4 L4 ---> L5 L5 ---> BOT TOP <--- L1 L1 <--- L2 L2 <--- L3 L3 <--- L4 L4 <--- L5 L5 <--- BOT
Latency & Density Spectrum
| Component | Diagram | Hierarchy Level | Primary Technology | Performance & Density Trade-Off | Access Latency (Cycles) |
|---|---|---|---|---|---|
| Cache |  | Level 1 / L2 / L3 | SRAM | Maximum speed; low density; high cost per bit. | |
| Main Memory |  | System RAM | DRAM | High density; medium latency; requires periodic refresh. | |
| Solid State Drive |  | Mass Storage | NAND Flash | Non-volatile mass storage; block-addressable. | |
| Hard Disk Drive |  | Secondary Storage | Magnetic Media | Maximum capacity; lowest cost per bit; mechanical delays. |
Predictable Real-Time Memory: ARMv8 & TCM
Standard cache systems introduce latency non-determinism due to unpredictable cache misses. Real-time system architectures incorporate Tightly Coupled Memory (TCM) to ensure predictable execution.
ARMv8 Memory Hierarchy featuring Tightly Coupled Memory
- Tightly Coupled Memory (TCM): On-chip SRAM mapped directly into the physical address space alongside main memory. Unlike caches, TCM bypasses tag-matching logic, providing deterministic, zero-wait-state single-cycle access for time-critical interrupt service routines (ISRs) and safety algorithms.