If you’ve been looking to buy a new laptop lately, chances are you’ve heard this term. A friend or tech-savvy relative may warn you against using cheaper Celeron chips, citing a lack of “cache”. This seems to be a small detail in the specifications. it is not.
What is a cache? It is a high-speed data storage layer located between the processor and the main memory. Without it, modern computing would crawl.
Computers use cache everywhere. Inside the processor is L1, L2 and often L3 cache. Next, operating systems have memory caches, page caches, and disk caches. Virtual memory also acts as a kind of cache. Speed up tasks and reduce hardware costs. How this works in practice.
Librarian analogy: no cache
To understand the logic, let’s strip away the silicon and use the librarian. Imagine a library where you can’t touch the books yourself. you asked the librarian who ran to the back storage room to get the book.
The first customer arrives. He wanted Moby Dick. The librarian went into the storage room, took out a book and handed it to him. The customer returns the book later. The librarian took it back to the storage room. He returned to his desk and waited.
The next customer wants Moby Dick again. The librarian has to go around the entire storage room again. Even for popular titles, he does the same inefficient job every time.
This is how a cacheless computer works. Uses main memory (storage) for each request regardless of frequency.
Librarian analogy: using cache
Now give that librarian a backpack to the librarian. He can carry up to 10 books. This backpack is a cache.
A new day has begun. The backpack is empty. The first customer wanted Moby Dick. The librarian went to the storeroom, took the book and handed it over. The customer returns the book. Instead of taking it back to the storage room, the librarian places it in his backpack. He remained seated at his desk.
Another customer arrives. I want Moby Dick. The librarian first looks for a backpack. The book is there. He handed it over immediately. No trip to the storeroom. Customers get faster service.
This is a cache hit. This product was found in fast storage.
Cache Misses and Latency
What if the customer asks for a book that is not in the backpack? The librarian must first search for the bag. Only then did he realize it was missing. Then he ran to the warehouse.
This is a cache miss.
Does this make the librarian less efficient? Technically yes. He wasted time checking the bag first. In computing, however, this delay is negligible. Modern hardware checks for cache availability in nanoseconds. In comparison, the path to “storage” (main memory) is massive. The penalty for missing is small compared to the penalty for not caching.
Important caching rules
From this simple model, we see the core mechanics.
- Speed and Size: Caches use faster, smaller memory to speed up slower, larger memory. It works with operating systems, web servers and hardware.
- Hit and Miss: Check the cache first. Did you find it? Hit. Not found? Miss. The system must then retrieve the data from a larger, slower pool.
- Size limitation: The cache memory is always smaller than the main memory. You can’t put the whole library in a backpack.
- Multiple Layers: These can be stacked. Our librarian could have a shelf (larger caches) and backpacks (smaller, faster caches) behind the counter. He checks the backpack, then the shelves, then the storage room. This is a multi-level cache.
This hierarchy makes computers feel nimble, even when it’s technically juggling large amounts of data. The data you use most stays close to the processor. The rest are waiting in the background.
How L1, L2 and L3 caches improve CPU speed
Time is the enemy of computing. Measures processor time in nanoseconds. When the microprocessor accesses RAM, it waits about 60 nanoseconds. It feels very slow. For a chip with a short cycle time of 2 nanoseconds, 60 nanoseconds is an eternity.
The solution is speed classes. Engineers have built a special memory bank on the motherboard. It’s small. It’s fast. Access takes about 30 nanoseconds. This is twice as fast as main memory. This is the second level or L2 cache.
Then there is the first level cache or L1. They put this small, high-speed memory directly on the microprocessor chip. Data is now transferred at the speed of the processor itself instead of the memory bus. In a 233 MHz Pentium, L1 cache is 3.5 times faster than L2 cache, which is twice as fast as RAM.
Some newer chips are even more advanced. They embedded two cache layers inside the silicon die. When this happens, the motherboard cache is pushed to level 3 or L3. It creates a ladder of speed.
The fastest thing in a computer is the microprocessor. L1 caches L2. L2 cache main memory. Main memory then caches slower peripherals such as hard drives and CD-ROMs. Hard drives can also cache your Internet connection.
Browser cache: why web pages load faster
Internet access has become a bottleneck. This is the slowest link in the chain. Web browsers such as Chrome and Safari are aware of this. They use your hard drive as temporary storage.
When you access an HTML page for the first time, your browser loads the page and saves a copy in a specific folder on your disk. Render the page. You see it.
The next time you click the URL, your browser will check the timestamp. Compares the cached file date on disk with the Web server date. If the server file is not new, the browser will skip the download. It serves the local copy.
A smaller, faster memory system is the hard disk. The larger and slower system is the Internet.
Experimenting with hard disk cache and floppy drives
Caches aren’t just in your CPU or browser. It lives on the peripherals. Modern hard drives have fast memory. Typically around 512 KB. The hard disk controller manages this memory rather than the computer directly managing it.
When the system requests data, the driver first checks this internal cache. This is done before moving the mechanical head of the drive. Mechanical movement is slow. Access to memory is immediate. If the data is cached, the controller returns immediately. The head never moves. You save time.
You can check the behavior of this cache yourself. Try an experiment with a floppy drive. Floppy disks are very slow.
Open a large file, such as a 300KB text file. The floppy disk light turns on. Wait. It takes about 20 seconds to load while the drive physically searches for the data.
Now close the editor. Open the same file again immediately. Don’t wait 30 minutes. Do not fill the memory with other tasks. The floppy light will not turn on. The file loads instantly.
The operating system checks the memory cache for floppy data. It found it.
The difference is obvious. One access to a floppy disk takes 120 milliseconds. Accessing the main memory takes about 60 nanoseconds. That is a massive gap in speed. This test can be performed on a hard drive, but the results are not as noticeable. The floppy makes the latency visible.
Cache hierarchy: speed and size
To understand the whole system, look at the numbers.
- L1 cache : Cache memory at the full speed of the microprocessor. The access time is about 10 nanoseconds. The size varies from 4 KB to 16 KB.
- L2 cache : SRAM based access. It takes 20-30 nanoseconds. The size is usually between 128 and 512 KB.
- Main Memory : Normal RAM. Access takes about 60 nanoseconds. The size varies from 32 MB to 128 MB.
- HDD : Mechanical and slow. Access takes about 12 milliseconds. The size is from 1 GB to 10 GB.
- Internet : Incredibly slow. Latency ranges from 1 second to days. The size is practically unlimited.
L1 caches L2. L2 cache main memory. Main memory cache disk. The disk caches the network.
This is a relay race where each runner is faster than the last. However, data is always transferred from the slower side to the faster side. If the cache misses, you wait. If it hits, you don’t. The overall performance of the machine is determined by how often it falls between fast, small memory and slow, large storage.
Imagine a world where every byte of your computer’s RAM ran as fast as your CPU’s L1 cache. That would be lightning. It’s going bankrupt too.
This is a fundamental trade-off in computer architecture. You need speed, but you also need to make sure your bank account doesn’t disappear when you buy a new computer. Caching is the compromise. Use small amounts of expensive, fast memory to speed up large amounts of cheap, slow memory.
This is the math behind the madness.
Modern 500 MHz chips complete 500 million cycles per second. This is one cycle every 2 nanoseconds. If your CPU has to wait for RAM, you’ll hit a wall. Accessing the main memory takes about 60 nanoseconds. This is about 30 cycles wasted. 30 cycles. Gone. Waste occurs when the processor waits for data to arrive from slow storage.
Without L1 and L2 cache, your computer would sit idle for most of its life, waiting for memory to catch up.
The magic of a small cache
Consider a 256KB L2 cache managing 64MB of RAM. This roughly equates to a ratio of 256,000 bytes of cache to 64,000,000 bytes.
How can such a small, fast memory manage large amounts of data? It works on a principle called locality of reference.
Although it sounds academic, the concept is intuitive. In fairly complex programs, you won’t use all the code at the same time. Only a small part is reused. Even if the executable file is 10 megabytes in size, only a few hundred bytes are active per second. And those specific bytes? They appear again and again.
The CPU does not have to fetch code and data from slow main memory every time it is needed. Retrieve from cache. The cache is hit almost every time because the same bits keep getting reused.
Locality of Reference in Action
Let’s look at a simple logical flow to see why this is so. Imagine a program that asks you to enter a number between 1 and 100.
- The program is waiting for input.
- Store your number in variable X.
- Set variable Y to 100.
- Set the variable Z to 1.
- Loop: Divide Z by X.
- If the remainder is zero, print “Z is a multiple of X.”
- It adds 1 to Z.
- Repeat step 5 until Z reaches Y.
Lines 1 to 4 are executed only once. The loop (lines 5 to 8) runs 100 times.
The processor executes loop instructions repeatedly. These instructions remain cached so they execute at full speed. What about the rest of the program? It runs once and then disappears.
This is locality of reference in plain English. Approximately 95% of a program’s execution time is spent executing a small set of repetitive code. Word processors do the same thing. They spend 95% of their time waiting for you to type or update the screen. The code that handles the interaction is cached. The remaining 5% do the heavy lifting: formatting, spell checking and saving.
By keeping those 5% hot in fast memory, your CPU will feel like it’s working instantly. Users find it responsive. Manufacturers save money by not having to use SRAM (Static Random Access Memory) to build 64 MB RAM modules. It uses DRAM (Dynamic RAM) for bulk and SRAM for cache.
It’s not a perfect balance. Sometimes the cache misses and the CPU still has to wait. But 95% of the time it works. That’s why we don’t bother building computers in fast memory, We can’t afford to. Get 95% efficiency at a fraction of the cost.
Frequently asked questions
What is the use of cache?
Caching stores data in a temporary, fast location so it can be retrieved faster than accessing main storage. Reduces latency and improves system performance.
What is memory cache?
In-memory caching stores frequently accessed data in reserved locations (caches) that can be accessed faster than main RAM or disk storage. This allows the processor to fetch data without main memory latency.
What is caching and how does it work?
Caching is a technique that takes advantage of locality of reference. It predicts what data the CPU will need next and stores it in fast memory. When the processor requests this information, it immediately finds it in the cache instead of waiting for it to be delivered from the slower main memory.
