TDT4255 Lecture 7: Hazards and exceptions

Transcription

1 TDT4255 Lecture 7: Hazards and exceptions Donn Morrison Department of Computer Science

2 2 Outline Section 4.7: Data hazards: forwarding and stalling Section 4.8: Control hazards Section 4.9: Exceptions

3 3 Review Real-world examples of pipelining Effect of pipelining on latency Effect of pipelining on throughput What are the three types of hazards in a processor pipeline? What are the five stages of the MIPS pipeline?

4 4 Datapath with control

5 5 Data hazard example There are data dependencies in the sequence Last four instructions use register $2 Assume that $2 is 10 before the sub instruction and that $1-$3 is -20 The and instruction should use -20 for $2, but reads 10 from the register file The or instruction also reads 10 from the register file The add and sw instructions read the correct value -20 from the register file sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

6 6 Data hazard example walkthrough

7 7 CC 1 sub $2, $1, $3 $2 = 10

8 8 CC 2 and $12, $2, $5 sub $2, $1, $3 $2 = 10

9 9 CC 3 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = 10, and instruction reads 10 for $2, stores in ID/EX

10 10 CC 4 add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = 10, or instruction reads 10 for $2, stores it in ID/EX, and instruction uses 10 in ALU operation

11 11 CC 5 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = (10/)-20, add reads new value -20 for $2 from register file, or uses 10 in ALU op

12 12 CC 6 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 $2 = 20, sw reads new value -20 for $2 from reg file, add uses -20 in ALU op, and writes register $12 with value calculated with $2 = 10

13 13 When is data produced/needed? sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

14 14 CC 1 sub $2, $1, $3 $2 = 10

15 15 CC 2 and $12, $2, $5 sub $2, $1, $3 $2 = 10

16 16 CC 3 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = 10, and instruction reads 10 for $2, stores in ID/EX

17 17 CC 4 add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = 10, or instruction reads 10 for $2, stores it in ID/EX, and instruction can use -20 from EX/MEM.AluOut

18 18 CC 5 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 $2 = (10/)-20, add reads new value -20 for $2 from register file, or can use -20 from MEM/WB.AluOut

19 19 How do we know when to forward? add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3

20 20 Detecting data hazards Notation EX/MEM.RegisterRd (Register Rd in EX/MEM register) EX/MEM.AluOut (Regsiter ALUOut in EX/MEM register) Hazard conditions 1a EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegisterRd = ID/EX.RegisterRt

21 21 Name that data hazard CC 4 add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 1a EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegisterRd = ID/EX.RegisterRt sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

22 22 Name that data hazard CC 4 add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 1a 1a EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegisterRd = ID/EX.RegisterRt sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

23 23 Name that data hazard CC 5 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 1a EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegisterRd = ID/EX.RegisterRt sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

24 24 Name that data hazard CC 5 sw $15, 100($2) add $14, $2, $2 or $13, $6, $2 and $12, $2, $5 sub $2, $1, $3 2b 1a EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegisterRd = ID/EX.RegisterRt sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

25 25 Detecting data hazards continued Detection performed in the EX state There is no hazard if the previous instruction will not write the result RegWrite for the earlier instruction must be asserted $0 is always 0 and a write to $0 will not create dependencies 1a EX/MEM.RegWrite and EX/MEM.RegisterRd!= 0 and EX/MEM.RegisterRd = ID/EX.RegisterRs 1b EX/MEM.RegWrite and EX/MEM.RegisterRd!= 0 and EX/MEM.RegisterRd = ID/EX.RegisterRt 2a MEM/WB.RegWrite and MEM/WB.RegisterRd!= 0 and MEM/WB.RegisterRd = ID/EX.RegisterRs 2b MEM/WB.RegWrite and MEM/WB.RegisterRd!= 0 and MEM/WB.RegisterRd = ID/EX.RegisterRt

26 26 Sequence with forwarding Dependence between pipeline registers and the inputs to the ALU Required data exists in time for later instructions It is possible to supply the inputs to the ALU needed by the and instruction and or instruction by forwarding the results found in the pipeline registers

27 27 Adding forwarding logic If inputs to the ALU can be taken from any pipeline register, proper data can be forwarded By adding multiplexers to the input of the ALU, the pipeline can be run at full speed in presence of data dependencies

28 28 Forwarding unit with ALU input select

29 29 Forwarding unit control lines Mux control Source Explanation ForwardA = 00 ID/EX Alu operand A comes from the register file ForwardA = 10 EX/MEM Alu operand A comes from previous cycle ALU result ForwardA = 01 MEM/WB Alu operand A comes from previous cycle memory read or earlier ALU result ForwardB = 00 ID/EX Alu operand B comes from the register file ForwardB = 10 EX/MEM Alu operand B comes from previous cycle ALU result ForwardB = 01 MEM/WB Alu operand B comes from previous cycle memory read or earlier ALU result 1a if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 10 1b if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRt)) ForwardB <= 10 2a if (MEM/WB.RegWrite and (MEM/WB.RegisterRd 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 01 2b if (MEM/WB.RegWrite and (MEM/WB.RegisterRd 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB <= 01

30 30 Single register access Summing a vector in a single register Result of instruction in WB stage Result of instruction in MEM stage Source of operand in ALU stage add $1, $1, $2 add $1, $1, $3 add $1, $1, $4 add $1, $1, $5...

31 31 CC 1 add $1, $1, $2

32 32 CC 2 add $1, $1, $3 add $1, $1, $2

33 33 CC 3 add $1, $1, $4 add $1, $1, $3 add $1, $1, $2 add $1, $1, $3 reads old value from register file

34 34 CC 4 add $1, $1, $5 add $1, $1, $4 add $1, $1, $3 add $1, $1, $2 add $1, $1, $4 reads old value from register file, add $1, $1, $3 gets the forwarded value from EX/MEM 1a) if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 10

35 35 CC 5 add $1, $1, $6 add $1, $1, $5 add $1, $1, $4 add $1, $1, $3 add $1, $1, $2 1a) if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 10 2a) if (MEM/WB.RegWrite and (MEM/WB.RegisterRd 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 01

36 36 Forwarding unit control lines Mux control Source Explanation ForwardA = 00 ID/EX Alu operand A comes from the register file ForwardA = 10 EX/MEM Alu operand A comes from previous cycle ALU result ForwardA = 01 MEM/WB Alu operand A comes from previous cycle memory read or earlier ALU result ForwardB = 00 ID/EX Alu operand B comes from the register file ForwardB = 10 EX/MEM Alu operand B comes from previous cycle ALU result ForwardB = 01 MEM/WB Alu operand B comes from previous cycle memory read or earlier ALU result 1a if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 10 1b if (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRt)) ForwardB <= 10 2a if (MEM/WB.RegWrite and (MEM/WB.RegisterRd 0) and not (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0)) and (EX/MEM.RegisterRd ID/EX.RegisterRs) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA <= 01 2b if (MEM/WB.RegWrite and (MEM/WB.RegisterRd 0) and not (EX/MEM.RegWrite and (EX/MEM.RegisterRd 0)) and (EX/MEM.RegisterRd ID/EX.RegisterRt) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB <= 01

37 37 Datapath modified to resolve hazards What about stores following r-type instructions: add $2, $1, $3 sw $2, 100($3)...or store following loads: lw $2, 100($3) sw $2, 100($4) Forwarding unit in EX-stage (with MUXes) Operand register numbers are passed on from ID stage via ID/EX pipel. reg. Some details are left out, like sign-extending unit

38 38 Data hazards and stalls Forwarding cannot avoid stalling the pipeline when an instruction tries to read a register following a load instruction that writes the same register The data is still being read from memory in clock cycle 4 while the ALU is performing the operation for the following instruction Something must stall the pipeline for the combination of load followed by an instruction that reads its result Hazard detection needed lw $2, 20($1) and $4, $2, $5 or $8, $2, $6 add $9, $4, $2 slt $1, $6, $7

39 39 CC 1 lw $2, 20($1) $2 = 10

40 40 CC 2 and $4, $2, $5 lw $2, 20($1) $2 = 10

41 41 CC 3 or $8, $2, $6 and $4, $2, $5 lw $2, 20($1) $2 = 10, and instruction reads 10 for $2, stores in ID/EX

42 42 CC 4 add $9, $4, $2 or $8, $2, $6 and $4, $2, $5 lw $2, 20($1) $2 = 10, or instruction reads 10 for $2, stores it in ID/EX, and instruction needs new $2 value, but it is not yet available

43 43 CC 5 slt $1, $6, $7 add $9, $4, $2 or $8, $2, $6 and $4, $2, $5 lw $2, 20($1) $2 = (10/)-20, add reads new value -20 for $2 from register file, or uses 10 in ALU op

44 44 Data hazard detection ID stage must: 1. Test to see if previous instruction is a load 2. Check if source registers match the destination register of the load if (ID/EX.MemRead and ((ID/EX.RegisterRt = IF/ID.RegisterRs) or (ID/EX.RegisterRt = IF/ID.RegisterRt))) Stall pipeline

45 45 Stall insertion CC2 - and is fetched and lw is decoded CC3 - and is decoded, or is fetched and lw is executed CC4 - and is decoded, or is fetched, a nop is executed and lw is in the MEM stage The nop can be achieved by setting harmless control signals

46 46 Stall / no operation Both the instructions in the ID and IF stages must be stalled so that we do not lose the fetched instructions Preventing these two instructions from making progress is accomplished simply by preventing the PC register and the IF/ID pipeline register from changing The back half of the pipeline starting with the EX is executing instructions that have no effect: nops, which act like bubbles Deasserting all 9 control signals in the EX, MEM and WB stages will create a do nothing or nop instruction - No registers or memories are written

47 47 Pipeline w/ control, forwarding, hazard detection

48 48 The big picture Although the compiler generally relies upon the hardware to resolve hazards, the compiler must understand the pipeline to achieve the best performance. Otherwise, unexpected stalls will reduce the performance of the compiled code. (page 374)

49 49 Control and branch hazards The decision whether to branch or not is taken in the MEM stage Without intervention the three sequential instructions following the branch will be fetched and begin execution Although less frequent than data hazards, a three instruction flush is costly

50 50 Assume branch not taken Stalling until the branch is complete is too slow Improvement: Assume the branch will not be taken and continue execution. If it is taken the instructions that are being fetched and decoded must be discarded. If branches are untaken half the time, and if it costs little to discard the instructions, this optimization halves the cost of control hazards To discard instructions: change the original control values to 0s Also change the three instructions in the IF, ID and EX stages when the branch reaches the MEM-stage Discarding instructions means we must be able to flush instructions in the IF, ID and EX stages of the pipeline

51 51 Reducing branch delay If branch execution is moved earlier in the pipeline, fewer instructions need to be flushed (so far we have assumed that the next PC for a branch is selected in the MEM stage) Many branches can rely on simple tests which do not require a full ALU operation Moving the branch decision up requires two actions to occur earlier: computing the branch target address and evaluating the branch decision

52 52 Reducing branch delay - early branch detection We already have the PC and the immediate field in the IF/ID pipeline register, so we just move the branch adder from the EX stage to the ID stage BEQ; we would compare the two registers (simple logic) read during the ID stage Moving the branch test to the ID stage implies additional forwarding and hazard detection hardware, since a branch dependent on a result still in the pipeline must still work properly with this optimization

53 53 Example (page 378) 36 sub $10, $4, $8 40 beq $1, $3, 7 # PC relative branch to and $12, $2, $5 48 or $13, $2, $6 52 beq $14, $4, $2 56 and $15, $6, $ lw $4, 50($7) Assumes that the pipeline is optimized for branches not taken and branch execution is moved to the ID stage The ID stage of CC3 determines that a branch must be taken, so 72 is selected as the next PC address and zeros the instruction fetched for the next CC. CC4 shows the instruction at location 72 being fetched and the single bubble or nop instruction as a result of the taken branch

54 54 Dynamic branch prediction With deeper pipelines (> 5 stage MIPS) a simple static prediction scheme will probably waste too much performance. Dynamic branch prediction uses runtime information to decide where to begin fetching new instructions A branch prediction buffer or branch history table is a small memory indexed by the lower portion of the address of the branch instruction. The memory contains one bit indicating whether the branch was recently taken or not A problem is that we don t know if the prediction is the right one, and it may have been put there by another branch with the same low-order bits Another shortcoming: even if a branch is almost always taken, we will predict incorrectly twice, rather than once, when it is not taken

55 55 2-bit prediction scheme By using 2 bits rather than 1, a branch that strongly favors taken or not taken will be mispredicted only once

56 56 Other branch handling strategies Delayed branch Always execute the following instruction Compilers and assemblers try to fill in the following instruction with one without dependencies Loses effect on long pipelines and multiple issue pipelines Branch target buffer Store the expected jump address in a buffer Global dynamic prediction Use the global branch behaviour to determine prediction Effective if combined with local branch prediction

57 57 Final datapath

58 58 Exceptions add $1, $2, $1 suppose overflow. We must: Transfer control to exception routine immediately after this instruction Flush the instructions following the add from the pipeline and begin fetching instructions from the new address Employ the same mechanism as for taken branches, but with the exception deasserting the control lines

59 59 Datapath w/ exception handling controls ID.Flush is ORed with the stall signal from the hazard detection unit To flush instructions in EX-stage: EX.Flush causing MUXes to zero the control lines Additional input to PC is added to be able to fetch instructions from hex, which is the exception location for overflow

60 60 Causes of exceptions 1. I/O device request 2. Hardware malfunction 3. Invoking an operating system service from a user program 4. Using an undefined instruction 5. Overflow 1 & 2 are not associated with a special instruction, so the implementation has some flexibility as to when to interrupt the pipeline, using the mechanism used for other exceptions In case of simultaneous multiple exceptions, the normal solution is to prioritize the exceptions

61 61 Example exception 40 sub $11, $2, $4 44 and $12, $2, $5 48 or $13, $2, $6 4C add $1, $2, $1 50 slt $15, $6, $7 54 lw $16, 50($7) Instructions to be invoked: sw $25, 1000($0) sw $26, 1004($0) Overflow for add in EX stage forced into PC CC7 shows that add and following instructions are flushed and the first instruction of the exception code is fetched. Address of the instruction following add is saved: 4C + 4 = 50. and and or complete

62 62 CC 6

63 63 CC 7

64 64 Hardware / software interface (1/2) HW + OS works in conjunction so exceptions behave as expected HW contract is normally to: Stop the offending instruction in midstream Let all prior instructions complete Flush all following instructions Set a register to show the cause of the exception Save the address of the offending instruction, and jump to the prearranged address

65 65 Hardware / software interface (2/2) The OS contract is to look at the cause of the exception and act appropriately: Undefined instruction, hw failure, overflow: kills the program and returns an indicator of the reason I/O request or OS service call: Saves state of program, performs desired task, restores the program to continue exec. On of the most important and frequent uses of exceptions is handling page faults and translation lookaside buffer (TLB) exceptions (Chapter 5)

66 66 Summary Data hazards: forwarding and stalling Control hazards Exceptions

67 67 Next week No lecture. Following week (week 42): Guest lecture by Marius Grannæs, Silicon Labs Note: Starts at 1115