Memory Hierarchy and Locality

Most computers have a memory hierarchy :

• Fastest memory is most expensive and smallest, e.g. 1 KB registers.

• On-chip memory is expensive and small, e.g. 1 MB cache.

• Medium-speed memory is larger, e.g. 1 GB main memory.

• Slowest memory is cheapest and largest, e.g. 1 TB disk.

Operations can only be performed on registers. There is a continual migration of data between slower memories and faster memories where it can be processed.

Computer hardware and software is often optimized to run faster when there is memory locality , i.e. successive accesses to memory are in nearby locations by memory address.

Algorithms that step through an array using a for loop to access adjacent locations are likely to be faster than algorithms that jump around.