The Cache Memory

Let’s try to tell the Cache story in the person of our known George, who is now responsible for a great company archive.
Each time someone needed a document asked it to George and he went into the archive aisle after aisle, he entered into the proper room, he looked for the proper bookshelf, he checked for the proper shelf, he fetched for a ladder in order to get access to the shelf, he sought for the proper folder and finally he took from that folder the requested document,he took a photocopy of it and put it back to place.
Then he ran all the way back handed the photocopy to whom requested it and, as he had more people waiting, he was going to follow identical paths to satisfy the next requests.
Thus George came to the end of each working day extremely tired from running during all the day the same paths. For that reason George thought of a way to not get so tired at the end of the day. It started by taking note of the places where he went to get the documents and verified that he was frequently going to the same places to get documents that were in proximity of each other or even the same documents several times.
So, he decided to take two photocopies of the requested document and also one photocopy  of the documents that were close to it (in its proximity). And he kept all these photocopies in a large drawer of his desk.
Having done so, George noticed that now he had to move from his desk, since he often had already a photocopy of  the asked documents in his desk drawer, only having to take a photocopy of this one in a photocopier that was near him. He also noticed that the more documents he had in his desk drawer, the less he had to go into the archive.
In the meanwhile, as the documents in his desk drawer were already a lot, he organized them within tabs identified by their address on the archive, i.e. the room, the bookshelf, the shelf and the folder.
From there on, when George was asked for some document he checked its address in the archive and searched the drawer tab for that same address. Then he checked if the asked document was there. If it wasn’t, George went into the archive to search for it , he took the ​​two photocopies from it and one from the documents in proximity, he brought them with him  and kept them in the desk drawer, creating the tab for them if it didn’t already exist.
The result was positive, as they were increasingly less the times that George had to go into the archive. But a new problem arose. The desk drawer was completely full. But the method was working. His legs and back were a lot less painful at the end of the day where he got a lot less tired. It was not the time to stop.
So being, George imagined an improvement to its solution. He enlisted the help of a few colleagues, fetched for a bookshelf that was unused and placed it behind his desk. Now, he took always another photocopy of any asked document to put on the shelf. So when it needed to take documents from his desk drawer due to the lack of space he always kept them close to him, stored on the bookshelf.
And how did George know what documents to remove from the desk drawer when he needed room for new ones? Having this doubt, George started registering in the kept documents the number of times they were ordered. This way George could always know the least requested one within each tab and took the option to remove these.
As the bookshelf was getting filled less were the times George had to go into the archive. George was proud of its strategy and bragged from it to his friends. When the bookshelf got filled the times that George had to go into the archive were very few during the all working day.

And this is the story of the Cache, told in the person of George.

The Cache is  the space where George kept his copies. The drawer was the L1 cache (Level 1) and the bookshelf was the L2 Cache (level 2). The persons who asked for the documents represent the CPU.
Therefore, the Cache memory is one more strategy found by the human imagination to establish the best technical/economic commitment to get the best possible performance from the CPU.
When we say that all the working elements of the CPU (running program instructions and data) must be in the main memory for the CPU to use them, we complement this statement replacing the Main Memory by the Cache. This only complements the previously stated, since all the records in the Cache are duplicated from records that are in the Main Memory. Because the fetch in the main memory can become very slow, as we could observe in the previous chapter, we found a workaround where, playing with the principles of locality and temporality data blocks are collected from the Main Memory and placed in the Cache, trying to reduce the number of accesses required to the Main Memory during a program execution. And the results already exceed 95% access reductions.
Therefore all needed elements for CPU to work must be in the L1 Cache, although CPU will request them by their addresses in the Main Memory, the only valid to the CPU. It’s in the L1 cache that the CPU requests are fetched inwill look for the running program instructions and for the data it needs and where it will write temporary or definitive results of running programs. The CPU requests are fetched in L1 Cache, i.e. in the 2 L1 caches, as in opposition to the older Von Neuhman architecture the L1 cache is separated into data and instructions. But only in the L1 Cache memory. Thereafter all returns to the normal and all values are brought together to the only same memory. We’ll see how.
Cache memory cells are static , i.e. they don’t require refreshing, maintaining the stored values ​​while powered. However they are the volatile, such as main memory. The static memory cells are obtained by logical circuits Flip Flop D we’ve seen before. We will see how , in an imaginary cache we will try to reproduce.
Because the CPU requests values by their main memory addresses, which obviously do not have correspondence in the cache memory, the data to be stored in the cache will be so in a way that it can be found directly at any demand by those addresses without any need to fetch them. For this purpose various types of caches are used, ranging from the highest performance (with huge logic volume associated) to the lowest performance (with much less logic associated) .
We will try to show on circuits the several types of address mappings (from the main memory to the cache memory), imaginary circuits that will have nothing to do with reality but which meet this exposition intended purposes.
Finally we will analyze how a last generation processor (the Intel Core i7 with 4 cores) cache operates, where the management of competition between the 4 Cores for the data in Cache is now a matter of huge complexity.

See the global synopsis of this work

We introduce here the table of contents of the Paper Book to describe the themes approached in this Chapter