Speculative execution refers to the scenario where instructions are executed in the ex- pectation that they will be needed in actual program execution. The main motivation, of course, is to improve performance. There are two main reasons to speculatively execute
Chapter 7 • Itanium Architecture 115
instructions: to keep the pipeline full and to mask memory access latency. We discuss two types of speculative execution supported by the Itanium: one type handles data de- pendencies, and the other deals with control dependencies. Both techniques are compiler optimizations that allow the compiler to reorder instructions. For example, we can spec- ulatively move high-latency load instructions earlier so that the data are available when they are actually needed.
Data Speculation
Data speculation allows the compiler to schedule instructions across some types of am- biguous data dependencies. When two instructions access common resources (either reg- isters or memory locations) in a conflicting mode, data dependency exists. A conflicting access is one in which one or both instructions alter the data. Depending on the type of conflicting access, we can define the following dependencies.
• Read-After-Write (RAW): This dependency exists between two instructions if one instruction writes into a register or a memory location that is later read by the other instruction.
• Write-After-Read (WAR):This dependency exists between two instructions if one instruction reads from a register or a memory location that is later written by the other instruction.
• Write-After-Write (WAW):This dependency exists between two instructions if one instruction writes into a register or a memory location that is later written by the other instruction.
• Ambiguous: Ambiguous data dependency exists when pointers are used to access memory. Typically, in this case, dependencies between load and store instructions or store and store instructions cannot be resolved statically at compile time because we don’t know the pointer values. Handling this type of data dependency requires run-time support.
There is no conflict in allowing read-after-read (RAR) access. The first three dependencies are not ambiguous in the sense that the dependency type can be statically determined at compile/assembly time. The compiler or programmer should insert stops (;;) so that the dependencies are properly maintained. The example
sub r6=r7,r8 ;;
add r9=r10,r6
exhibits a RAW data dependency onr6. The stop after thesubinstruction would allow theaddinstruction to read the value written by thesubinstruction.
If there is no data dependency, the compiler can reorder instructions to optimize the code. Let us look at the following example:
sub r9=r10,r6 // cycle 2
ld8 r4=[r5] ;;
add r11=r12,r4 // cycle 4
Because there is a two-cycle latency to the first-level data cache, the addinstruction is scheduled two cycles after scheduling the ld8instruction. A straightforward optimiza- tion involves moving theld8instruction to cycle 1 as there are no data dependencies to prevent this reordering. By advancing the load instruction, we can schedule theaddin cycle 3 as shown below:
ld8 r4=[r5] // cycle 1
sub r6=r7,r8 ;;
sub r9=r10,r6 ;; // cycle 2
add r11=r12,r4 // cycle 3
However, when there is ambiguous data dependency, as in the following example, instruc- tion reordering may not be possible:
sub r6=r7,r8 ;; // cycle 1
st8 [r9]=r6 // cycle 2
ld8 r4=[r5] ;;
add r11=r12,r4 ;; // cycle 4
st8 [r10]=r11 // cycle 5
In this code, ambiguous dependency exists between the first st8and ld8because r9 andr5could be pointing to overlapped memory locations. Remember that each of these instructions accesses eight contiguous memory locations. This ambiguous dependency will not allow us to move the load instruction to cycle 1 as in the previous example.
The Itanium provides architectural support to move such load instructions. This is facilitated by advance load (ld.a) and check load (ld.c). The basic idea is that we initiate the load early and when it is time to actually execute the load, we will make a check to see if there is a dependency that invalidates our advance load data. If so, we reload; otherwise, we successfully advance the load instruction even when there is ambiguous dependency. The previous example with advance and check loads is shown below:
1: ld8.a r4=[r5] // cycle 0 or earlier
. . .
Chapter 7 • Itanium Architecture 117
3: st8 [r9]=r6 // cycle 2
4: ld8.c r4=[r5]
5: add r11=r12,r4 ;;
6: st8 [r10]=r11 // cycle 3
We inserted an advance load (line 1) at cycle 0 or earlier so thatr4would have the value ready for theaddinstruction on line 5 in cycle 2. However, we have to check to see if we can use this value. This check is done by the check load instruction ld8.con line 4. If there is no dependency between the store on line 3 and load on line 4, we can safely use the prefetched value. This is the case if the pointers in r9and r5are different. On the other hand, if the load instruction is reading the value written by the store instruction, we have to reload the value. The check load instruction on line 4 automatically reloads the value in the case of a conflict.
In the last example, we advanced just the load instruction. However, we can improve performance further if we can also advance all (or some of) the statements that depend on the value read by the load instruction. In our example, it would be nice if we could advance theaddinstruction on line 5. This causes a problem if there is a dependency between the store and load instructions (on lines 3 and 4). In this case, we not only have to reexecute the load but also theaddinstruction. The advance check (chk.a) instruction provides the necessary support for such reexecution as shown in the following example.
ld8.a r4=[r5] // cycle -1 or earlier
. . . add r11=r12,r4 // cycle 1 sub r6=r7,r8 ;; st8 [r9]=r6 // cycle 2 chk.a r4,recover back: st8 [r10]=r11 recover: ld8 r4=[r5] // reload
add r11=r12,r4 // reexecute add
br back // jump back
When the advanced load fails, the check instruction transfers control to recover to reload and reexecute all the instructions that used the value provided by the advanced load.
How does the Itanium maintain the dependency information for use by the check instructions? It keeps a hardware structure called the Advanced Load Address Table (ALAT), indexed by the physical register number. When an advanced load (ld.a) in- struction is executed, it records the load address. When a check is executed (eitherld.c
orchk.a), it checks ALAT for the address. The check instruction must specify the same register that the advanced load instruction used. If the address is present in ALAT, exe- cution continues. Otherwise, a reload (in the case ofld.c) or recovery code (in the case ofchk.a) is executed. An entry in ALAT can be removed, for example, by a subsequent store that overlaps the load address. To determine this overlap, the size of the load in bytes is also maintained.
Control Speculation
When we want to reduce latencies of long latency instructions such as load, we advance them earlier into the code. When there is a branch instruction, it blocks such a move because we do not know whether the branch will be taken until we execute the branch instruction. Let us look at the following code fragment.
cmp.eq p1,p0 = r10,10 // cycle 0
(p1) br.cond skip ;; // cycle 0
ld8 r1 = [r2] ;; // cycle 1
add r3 = r1,r4 // cycle 3
skip:
// other instructions
In the above code, we cannot advance the load instruction due to the branch instruction. Execution of thethen branchtakes four clock cycles. Note that the code implies that the integer compare instruction that sets the predicate registerp1and the branch instruction that tests it are executed in the same cycle. This is the only exception; in general, there should be a stop inserted between an instruction that is setting a predicate register and the subsequent instruction testing the same predicate.
Now the question is: how do we advance the load instruction past the branch instruc- tion? We speculate in a manner similar to the way we handled data dependency. There is one additional problem: because the execution may not take the branch, if the specu- lative execution causes exceptions, they should not be raised. Instead, exceptions should be deferred until we know that the instruction will indeed be executed. The Not-a-Thing bit is used for this purpose. If a speculative execution causes an exception, the NaTbit associated with that register is set. When a check is made at the point of actual instruction execution, if the deferred exception is not present, speculative execution was successful. Otherwise, the speculative execution should be redone. Let us rewrite the previous code with a speculative load.
ld8.s r1 = [r2] ;; // cycle -2 or earlier
// other instructions
cmp.eq p1,p0 = r10,10 // cycle 0
(p1) br.cond skip // cycle 0
chk.s r1, recovery // cycle 0
Chapter 7 • Itanium Architecture 119 skip: // other instructions recovery: ld8 r1 = [r2] br skip
The load instruction is moved by at least two cycles so that the value is available inr1by the time it is needed by theaddinstruction. Because this is a speculative load, we use the ld8.sinstruction. And, in place of the original load instruction, we insert a speculative check (chk.s) instruction with the same registerr1. As in the data dependency example, if the speculative load is not successful (i.e., theNaTbit ofr1is set), the instruction has to be reexecuted. In this case, the check instruction branches to therecoverycode.