Why are Multicore CPUs So Hot?

Sujay V Sarma

These days it is common to have processor codenames and architecture names used with abandon. The names can confuse anyone, and after a while, it becomes nearly impossible to tell them apart. This month, we take you through this jargon jungle and unveil the secrets of how modern x86 processors work. There are two parts of the story. 

One is the micro-architecture part, which is a combination of multiple technologies. For instance, when Intel says 'Core 2' or AMD says 'AM2', it denotes that a particular combination of technologies is at work to produce a processor that can be called a 'Core 2' or an 'AM2' product.

The second part to our story is the architecture itself, which is known by a specific codename (say 'Conroe' or 'Manchester'). This month, we look at technologies behind the different micro-architectures and next month, we'll see what each combination (the architectures themselves) is about.

Macro Fusion

A term coined by Intel, it refers to the processor's ability to combine several instructions into one, thus optimizing it and making for a faster execute. Thus, if values from two different memory locations already in the processor cache are to be compared but the instruction set first loads them elsewhere and then compares them, the Macro Fusion technique would enable you to directly compare them, skipping a step. Spread across an entire application or thread, this can significantly boost execution times.

Handling L2 Cache

When you have more than one core on a processor die, selecting the right place to have the L2 cache is key. L1 is not touched here since it is a small-sized cache that contains instructions being immediately fed to the processing core. One method uses a common L2 cache for multiple cores (Intel) while the other seeks a dedicated cache location per core (AMD). There are pros and cons to both sides of the coin.

When you go the AMD way, you have two L2 caches in a dual core die and four of them on a quad core. While each core gets its own dedicated cache, the drawback is that when some cores are more active than the others, their caches will be overflowing and hence those cores suffer a performance hit even though the core waits for a fetch from the main memory. 

Intel's shared core (what it calls an 'Advanced Smart Cache') uses shared L2 caches between two cores, letting cores be better utilized when one of the cores is relatively less loaded. But, this method introduces the headache of having a memory controller that manages the memory between the two cores (who gets to put what where and so forth). 

The memory bandwidth Intel promises with its way out peaks at 96 GB per second at 3 GHz. Obviously it is hard to take a call on which one is better, as each method compensates for a different scenario.

Intel Smart Memory Access

So, when you have a shared L2 cache like Intel has with its Advanced Smart Cache technology, the headache of managing the cache between two cores falls on a memory controller. This is the bundle they call 'Smart Memory Access.' Along with memory management, SMA also resolves memory locations and adjusts internal pointers so that when the core finds an instruction to jump elsewhere in the code or fetch an instruction or data that's already been cached, it can be directly fetched from the L2 or main memory location instead of reloading it. This speeds up out-of-order execution. 

The speed arises from the fact that instructions that are independent of each other can be executed as soon as that instruction has been decoded, without having to wait for the sequence of instructions before it to finish. Thus, if a block of code loads two values and stores them elsewhere without any processing with other logical code in between can be reordered and the LOAD/STORE can be executed in independent threads and many clock cycles earlier.

AMD Direct Connect: AMD solves the problem of managing L2 cache in multi core or multi processor systems, by doing away with the need for an FSB and talking directly to the various components through something called the HyperTransport Link. Taken together this architecture is called 'Direct Connect' (DCA). Each processor built around DCA has an integrated memory controller and is HyperTransport enabled. Each processor with DCA is linked to specific portions of memory. 

When one CPU needs to access data that is in the memory linked to the other processor, it will use HyperTransport to link to that processor/memory. This HyperTransport linkage is called Coherent HyperTransport.

Memory access speeds for processor transactions are boosted when you put the controller on the processor die itself. But in a multi-processor system, when you add more processors, you also add more memory controllers. To solve the problems with access overlaps and violations and race conditions, AMD uses something called NUMA (Non-Uniform Memory Access) that is similar to Intel's SMP (Symmetric Multi-Processing). NUMA deviates from the SMP in that NUMA is asymmetric. 

Although the original use for NUMA and DCA involve multiple processors rather than cores, it can be applied to a multi-core system just as easily.

Multimedia instructions

With the Pentium-1 class of processors, we had the SSE (or AMD's equivalent of 3DNow!), SSE-2 and SSE-3 that themselves are descendants of the now defunct MMX instruction set. SIMD is one more evolution forwards from there. 128-bit SSE instructions traditionally take two clock cycles each to execute. SIMD allows for these instructions to complete in one clock, doubling their throughput. 

In addition to all that, SIMD adds about 70 new instructions to process packed floating-point data, control memory without cache pollution and extend the MMX instruction set. This lets your application do really parallel threads with independent control flow. This expanded instruction set is also called the 'Streaming SIMD extensions' because it works best for streaming data (into the processor) and letting you do video processing like encoding video and other multimedia data, faster.

Intel vPro

If you've heard the talk about having system level partitions with each partition being able to run isolated operating systems and software, and your IT department being able to manage your system remotely using hidden or access restricted areas of your system while you were busy working with your section of it... vPro is the technology that is set to make that a reality.

With vPro, your IT department can remotely monitor, diagnose and repair your desktop even it is switched off. Also, the system can send out its configuration information like what cards are installed, how the BIOS has been configured and so on. This also improves asset tracking and inventory since one no longer needs to go to each computer to audit this and neither do they need to install specific software or open (desktop) firewall ports to do this. 

Additionally, vPro also has the benefit of lower power usage by turning off processor functions when they are not in use. Yes, vPro can turn off specific portions inside the processor when they are not being used rather than turning an entire core on or off.

New in virtualization

This is a big topic outside the discussion on processors as well. The reason this is hot is well known-the more the number of applications you can put into a single box (most usually by cramming more virtual machines into that box) the lower your cost of ownership and running costs and better your cost efficiency. But the biggest limiting factor so far to that argument has been that while virtualization has been possible in the non-x86 world for some time now, it has been primarily software based in the x86 realms. 

The VMware, Xen and Virtual Servers/PCs of the world have ruled this area. However, the performance gained by doing that while the hardware did not inherently support virtualization is not that great. The traditional way to boost virtual machine performance is by jacking up the amount of RAM and the processor speed.

Intel's vPro and VT (Virtualization Technology) and AMD's AMD-V are rather big steps to taking the x86 into a fully virtualizible environment. We have discussed vPro above, let's take some time to understand VT and AMD-V.

Intel VT: Codename 'Vanderpool', it enables support for virtualization software layers to control processing actually being done inside a virtual machine. This allows such software to monitor virtual systems and marshal their resource (processor and memory) usage. Intel VT is in use not just in x86 desktop processors, but also affects Itanium 32 and Itanium 64 bit families. 

Three new features for the 32-bit VT (called VT-x) are: a more coordinated way of dealing with blocked NMI (Non Maskable Interrupts) when VMs (guest OS in a VM) are exiting, setting up virtual processor IDs by VMMs and then use these IDs to translate buffer addresses in memory and; instead of having a common memory page tables for host and guest OS VMs now have their own page tables. 

The Xen Hypervisor gets extra support from the Intel VT architecture, letting it expose resources and configuration to its guest software that return better virtualized performance. This includes presenting all the real processor information bits (CPUID) to the guest, with the exception of the VMX and MCA.

AMD-V: This is a step further from AMD's 64-bit architecture (AMD64). AMD-V adds two modes of operation to the systems it runs on: Host mode and Guest mode. Also added is a new instruction called VMRUN that lets virtual machines along with the guest OS and its applications work a little faster inside a VM. An AMD-V processor will initially boot up with its new VM capabilities disabled (the 'guest' mode) until compatible VMM (VM Managers) are detected. 

Once such a VMM is detected, the processor switches to 'host' mode and turns on all its capabilities. An AMD-V processor in host mode features a number of interesting capabilities like setting up at the hardware level itself the kinds of resources the system software is allowed to access. Using these instructions one can even assign different VMs exclusive access to different resources (like network interfaces).

As we continue adding cores to processors and add more of such processors to our computer systems, it is natural that their technologies will advance to let multiple processor cores access what so far has been a resource dedicated to one processor core. Next month, we will be looking to demystify the various processor code names in the multi-core domain.

Source: PCQuest