3. Introduction
• Hyper-Threading technology makes a single processor appear as
two logical processors.
• It was first implemented in the Prestonia version of the Pentium®
4 Xeon processor on 02/25/02.
4. Traditional Approaches (I)
• High requirements of Internet and
Telecommunications Industries
• Results are unsatisfactory compared the gain
they provide with the cost they cause
• Well-known techniques;
• Super Pipelining
• Branch Prediction
• Super-scalar Execution
• Out-of-order Execution
• Fast memories (Caches)
5. Traditional Approaches (II)
• Super Pipelining:
• Have finer granularities, execute far more instructions within a
second (Higher clock frequencies)
• Hard to handle cache misses, interrupts and branch mispredictions
• Instruction Level Parallelism (ILP)
• Mainly targets to increase the number of instructions within a cycle
• Super Scalar Processors with multiple parallel execution units
• Execution needs to be verified for out-of-order execution
• Fast Memory (Caches)
• To reduce the memory latencies, hierarchical units are using which
are not an exact solution
6. Traditional Approaches (III)
Same silicon technology
Normalized speed-ups with
Intel486™
microarchitecture, has improved
integer performance five- or six-
fold1
die size has gone up
fifteen-fold, a three-times-higher rate
power increased almost eighteen-fold
during this period1.
7. Thread-Level Parallelism
• Chip Multi-Processing (CMP)
• Put 2 processors on a single die
• Processors (only) may share on-chip cache
• Cost is still high
• Single Processor Multi-Threading;
• Time-sliced multi-threading
• Switch-on-event multi-threading ( well for server application)
• Simultaneous multi-threading
9. Hyper-Threading (HT)
Technology
• Provides more satisfactory solution
• Single physical processor is shared as
two logical processors
• Each logical processor has its own
architecture state
• Single set of execution units are shared
between logical processors
• N-logical PUs are supported
• Have the same gain % with only 5% die-
size penalty.
• HT allows single processor to fetch and
execute two separate code streams
simultaneously.
10. Firstimplementationon the intel xeon
processorfamily
• Several goals were at the heart of the microarchitecture:
• Minimize the die area cost of implementing
• One logical processor is stalled the other logical processor
could continue to make forward progress.
• Allow a processor running only one active software thread
to run at the same speed
21. The Execution Trace Cache
• Primary or Advanced form of L1 Instruction Cache.
• Delivers 3 μops/clock to the out-of-order execution logic.
• Most instructions fetched and decoded from the Trace Cache.
• caches the μops of previously decoded instructions here, so it
bypasses the instruction decode
• Recovery time for a mispredicted branch is much shorter in
compression of re-decode the IA-32 instruction
22. Execution Trace Cache (TC) (I)
• Stores decoded instructions called “micro-
operations” or “uops”
• Arbitrate access to the TC using two IPs
• If both PUs ask for access then switch will occur in the next
cycle.
• Otherwise, access will be taken by the available PU
• Stalls (stem from misses) lead to switch
• Entries are tagged with the owner thread info
• 8-way set associative, Least Recently Used (LRU)
algorithm
• Unbalanced usage between processors ( shared
nature of TC)
24. Microcode Store ROM (MSROM) (I)
• Complex instructions (e.g. IA-32) are decoded into more than
4 uops
• TC sends a microcode-instruction pointer to MSROM
• Shared by the logical processors
• Independent flow for each processor(Two microcode
instruction pointers)
• Access to MSROM alternates between logical processors as in
the TC
25. Microcode Store ROM (MSROM) (II)
The Microcode ROM controller
then fetches the uops needed
and returns control to the TC
26. TLB
• processors have been working not with physical memory
addresses, but with virtual addresses
• Advantages:
• More memory be allocated
• Keeping only necessary data
• Disadvantages:
• Virtual addresses need to translate to physical address
• Table gets so large and can’t be stored on chip ( paged)
• Translation stage need for each memory access, too much slow
27. Translation look aside buffer
(TLB)(II)
• A small cache memory directly on the processor that stored
the correspondences for a few recently accessed addresses.
• Until Core 2 used two level cache:
• Level 1 TLB: small but very fast( for loads only) – small (16 entries)
• Level 2 TLB: handled load missed( 256 entries)
28. Translation look aside buffer
(TLB)(III)
• Nehalem:
• First level - data TLB:
• Shared between data and instruction
• Stores 64 entries( small pages 4k) or 32 entries( large pages 2M/4M)
• First level - instruction TLB:
• Shared between data and instruction
• Stores 128 entries( small pages) and 7( large pages)
• Second level (unified):
• Stores up to 512 entries ( only small pages)
29. Branch Predictors
• A branch breaks the parallelism
• Branch prediction determines whether or not a branch will be
taken and if it is, quickly determines the target address for
continuing execution
• complicated techniques are needed
• needed is an array of branches—the Branch Target Buffer (BTB)
• and an algorithm for determining the result of the next branch
Intel hasn’t provided details on the algorithm used for their new
predictors
30. ITLB and Branch Prediction (I)
• If there is a TC miss, bytes need to be loaded from L2
cache and decoded into TC
• ITLB gets the “instruction deliver” request
• ITLB translates next Pointer address to the physical
address
• ITLBs are duplicated for each logical processors
• L2 cache arbitrates on first-come first-served basis while
always reserve at least one slot for each processor
31. ITLB and Branch Prediction (II)
• Branch prediction structures are either duplicated or shared
• If shared owner tags should be included
• Return stack buffer is duplicated
• Very small structure
• call/return pairs are better predicted for software threads
independently.
• branch history buffer tracked independently for each logical
processor
• large global history array is a shared structure tagged with a logical
processor ID.
35. Allocator
• The out-of-order execution:
• Re-ordering
• Tracing
• sequencing
• Allocates many of the key machine buffers;
• 126 re-order buffer entries
• 128 integer and floating-point registers
• 48 load, 24 store buffer entries
• Allocator logic takes uops from the qeue
• Resources shared equal (partition) between processors
• Limitation of the key resource usage, we enforce fairness and
prevent deadlocks over the Arch.
• For every clock cycle, allocator switches between uop queues
• If there is “stall” or “HALT”, there is no need to alternate
between processors
36. Register Rename
• The register rename logic renames the architectural IA-32
registers onto the machine’s physical registers
• The 8 general usage IA-32 register expand to 128 available
physical register.
• Involves with mapping shared registers names for each
processor
• Each processor has its own Register Alias Table (RAT)
• The register renaming process is done in parallel to the
allocator logic
• Uops are stored in two different queues;
• Memory Instruction Queue (Load/Store)
• General Instruction Queue (Rest)
• Queues are partitioned among PUs
37. Instruction Scheduling
• Schedulers are at the heart of the out-of-order execution
engine
• There are five schedulers which have queues of size 8-12
• Collectively, they can dispatch up to six uops each clock cycle
• Scheduler is oblivious when getting and dispatching uops
• It ignores the owner of the uops
• Dependent inputs and availability of execution resources
• It can get uops from different PUs at the same time
• To provide fairness and prevent deadlock, there is a limit on the
number of active entries of each PU qeue
38. Execution Units & Retirement
• Execution Units are oblivious when getting and
executing uops
• Since resource and destination registers were renamed earlier,
during/after the execution it is enough to access physical registries
• After execution, the uops are placed in the re-order
buffer which decouples the execution stage from
retirement stage
• The re-order buffer is partitioned between PUs
• Uop retirement commits the architecture state in
program order
• Once stores have retired, the store data needs to be written into
L1 data-cache, immediately
39. Memory Subsystem
• Totally oblivious to logical processors
• Schedulers can send load or store uops without regard to PUs and
memory subsystem handles them as they come
• Memory types;
• DTLB:
• Translates addresses to physical addresses
• 64 fully associative entries; each entry can map either 4K or 4MB page
• Shared between PUs (Tagged with ID)
• L1, L2 and L3 caches
• The L1 data cache is virtually addressed and physically tagged.
• Cache conflict might degrade performance ( share data in the cache)
• Using same data ( same data) might increase performance (more mem.
hits) – common in server application code
40. System Modes (I)
• Two modes of operation;
• single-task (ST)
• When there is one software thread to execute
• multi-task (MT)
• When there are more than one software threads to execute
• ST0 or ST1 where number shows the active PU
• HALT command was introduced where resources are combined after the
call
• Reason is to have better utilization of resources
41. System Modes (II)
• HALT transitions the processor from MT-mode to ST0- or ST1-
mode
• In ST0- or ST1-modes, an interrupt sent to the HALTed
processor would cause a transition to MT-mode.
42. Operating system and
Applications
• Hyper-Threading:
• operating system and application software as having twice the
number of processors
• OS optimizations:
1. use the HALT instruction if one logical processor is active and
the other is not.
• Not use HALT -> Idle loop !
2. scheduling software threads to logical processors
44. OS Support for HT
• Native HT Support
• Windows XP Pro Edition
• Windows XP Home Edition
• Linux v 2.4.x (and higher)
• Compatible with HT
• Windows 2000 (all versions)
• Windows NT 4.0 (limited driver support)
• No HT Support
• Windows ME
• Windows 98 (and previous versions)
45. Conclusion
• Measured performance (Xeon) showed performance gains of
up to 30% on common server applications.
• HT is expected to be viable and market standard from Mobile
to server processes.
-Since the logical processors share the vast majority of microarchitecture resources and only a few small structures were replicated, the die area cost of the first implementation was less than 5% of the total die area.-A logical processor may be temporarily stalled for a variety of reasons, including servicing cache misses, handling branch mispredictions, or waiting for the results of previous instructions.
-When a complex instruction is encountered, the TC sends a microcode-instruction pointer to the Microcode ROM.-The Microcode ROM controller then fetches the uops needed and returns control to the TC.-Two microcode instruction pointers are used to control the flows independently if both logical processors are executing complex IA-32 instructions.
-keeping only the data necessary at a given moment in actual physical memory with the rest remaining on the hard disk-This means that for each memory access a virtual address has to be translated into a physical address, and to do that an enormous table is put in charge of keeping track of the correspondences-The problem is that this table gets so large that it can’t be stored on-chip—it’s placed in main memory, and can even be paged (part of the table can be absent from memory and itself kept on the hard disk).
-The return stack buffer, which predicts the target of return instructions, is duplicated.--The branch prediction structures are either duplicated or shared. The return stack buffer, which predicts the target of return instructions, is duplicated because it is avery small structure and the call/return pairs are better predicted for software threads independently.
-The allocator logic takes uops from the uop queue and allocates many of the key machine buffers needed to execute each uop-Some of these key buffers are partitioned such that each logical processor can use at most half the entries.
The register rename logic renames the architectural IA- 32 registers onto the machine’s physical registers. This allows the 8 general-use IA-32 integer registers to be dynamically expanded to use the available 128 physical registers.-The renaming logic uses a Register Alias Table (RAT) to track the latest version of each architectural register to tell the next instruction(s) where to get its input operands.
-The schedulers determine when uops are ready to execute based on the readiness of their dependent input register operands and the availability of the execution unit resources.
-The DTLB translates addresses to physical addresses-Each logical processor also has a reservation register to ensure fairness and forward progress in processing DTLB misses.
-An operating system that does not use HALT optimization would execute on the idle logical processor a sequence of instructions that repeatedly checks for work to do