Outline Pipeline, Pipelined datapath Dependences, Hazards Structural, Data - Stalling, Forwarding Control Hazards Branch prediction

Transcription

1 Control Hazard

2 Outline Pipeline, Pipelined datapath Dependences, Hazards Structural, Data - Stalling, Forwarding Control Hazards Branch prediction

3 Control Hazard Arise from the pipelining of branches and other instructions that change the PC

4 Control Hazard Arise from the pipelining of branches and other instructions that change the PC Also called Branch Hazards

5 Branch Evaluation

6 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw lw $4, $4, 50($7) Control Hazard

7 beq Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles)

8 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF and

9 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX and or IF ID IF

10 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard Branch Taken lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX and or IF ID IF

11 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard NPC Updates lw Time lw $4, $4, 50($7) (clock cycles) beq and IF ID EX MEM IF ID EX or add IF ID IF

12 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard NPC Updates lw Time lw $4, $4, 50($7) (clock cycles) beq and IF ID EX MEM IF ID EX or add IF ID IF Insert NOP NOP

13 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 Branch add add $14, $2, $2, $2 target $2 enters the the pipeline lw Time lw $4, $4, 50($7) (clock cycles) beq and or IF ID EX MEM WB IF ID EX MEM IF ID EX add IF ID Target: lw IF

14 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq and or add IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM IF ID EX Target: lw IF ID Target+1 IF

15 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX MEM WB and IF ID EX MEM WB or IF ID EX MEM WB add IF ID EX MEM WB Target: lw IF ID EX MEM WB Target+1 IF ID EX MEM

16 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) Target: beq and or add lw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Penalty = 3 cycles

17 Control Hazard Branch evaluation occurs in EX stage. PC updates in MEM stage. Branch target loads in the next cycle The delay in determining the proper instruction to fetch is called a Control Hazard or Branch Hazard.

18 Reducing Pipeline Branch Penalties

19 Reducing Pipeline Branch Penalties Freeze the pipeline

20 Reducing Pipeline Branch Penalties Freeze the pipeline Time (clock cycles) beq nop nop nop lw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

21 Reducing Branch Delay

22 Reducing Branch Delay Most branches rely on simple tests (equality or sign) Move the branch decision up

23 Reducing Branch Delay Most branches rely on simple tests (equality or sign) Move the branch decision up Compute NPC earlier Evaluate branch at the earliest

24 Reducing Branch Delay

25 NPC Reducing Branch Delay

26 Reducing Branch Delay NPC Branch Delay: Delay between Branch instruction fetch and Branch Target fetch of of a taken branch.

27 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD BEQ IF ID IF

28 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX BEQ IF ID SUB IF

29 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX BEQ IF ID SUB IF PC = PC + Offset

30 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM BEQ IF ID EX SUB IF NOP ID Branch Target IF

31 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM WB BEQ IF ID EX MEM SUB IF NOP ID NOP EX Branch Target IF ID

32 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM WB BEQ IF ID EX MEM WB SUB IF NOP ID NOP EX MEM NOP NOP WB Branch Target IF ID EX MEM WB

33

34

35 Branch Delay Slot Time (clock cycles) Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Delay Slot

36 Branch Hazards Time (clock cycles) Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Target IF IF ID EX MEM WB 1 stall cycle for every branch yields a performance loss of 10% to 30%!

37 Example Branch Taken

38

39

40 Branch Evaluation in ID Steps Are beq inputs ready?

41 Branch Evaluation in ID Steps Decode the instruction Decide whether to bypass/forward to the equality unit Forwarding Logic has to be modfied Complete the comparison If branch taken, set the PC to the branch target address.

42 Branch Evaluation Forwarding New forwarding logic required Source operands can come from ALU/MEM or MEM/WB If source operands are not ready, data hazard can occur and a stall will be needed add add $1, $1, $2, $2, $2 $2 beq beq $1, $1, $3, $3,

43 Reducing Pipeline Branch Penalties Freeze the pipeline Static Prediction Predict Untaken, Predict Taken Delayed Branch Fill Branch Delay Slot

44 Predict Untaken Scheme Time (clock cycles) Untaken Branch IF ID EX MEM WB Instruction i + 1 Instruction i + 2 Instruction i + 3 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Time (clock cycles) Taken Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

45 Reducing Pipeline Branch Penalties beq $1, $3, 28 and and $12, $12, $2, $2, $5 $5 or or $13, $13, $6, $6, $2 $2 add add $14, $14, $2, $2, $2 $2 lw lw $4, $4, 50($7) 50($7) Predict Taken beq $1, $3, 28

46 Reducing Pipeline Branch Penalties Predict Taken Great if prediction is correct beq beq $1, $1, $3, $3, and and $12, $12, $2, $2, $5 $5 or or $13, $13, $6, $6, $2 $2 add add $14, $14, $2, $2, $2 $2 lw lw $4, $4, 50($7) 50($7)

47 Reducing Pipeline Branch Penalties Predict Taken Great if prediction is correct. Penalty on misprediction only. Time (clock cycles) beq beq $1, $1, $3, $3, and and $12, $12, $2, $2, $5 $5 or or $13, $13, $6, $6, $2 $2 add add $14, $14, $2, $2, $2 $2 lw lw $4, $4, 50($7) 50($7) beq lw and IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

48 Reducing Pipeline Branch Penalties sll... srl srl $14, $14, $2, $2, $2 $2 beq beq $1, $1, $3, $3, and and $12, $12, $2, $2, $5 $ Fill the branch delay slot sll...

49 Reducing Pipeline Branch Penalties Fill the branch delay slot sll sll srl srl $14, $14, $2, $2, $2 $2 beq beq $1, $1, $3, $3, and and $12, $12, $2, $2, $5 $ Time (clock cycles) beq srl IF ID EX MEM WB IF ID EX MEM WB branch target IF ID EX MEM WB

50 Branch Delay Slot Predict taken Predict untaken

51 Static Branch Prediction

52 Dynamic Branch Prediction 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }}

53 Dynamic Branch Prediction 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

54 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

55 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T T T N T T T T T T T T T T T T Wrong Predictions BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

56 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T T T N T T T T T T T T T T T T Wrong Predictions BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program Can Can we we do do better?

57 Dynamic Branch Prediction 0010 Branch Prediction Bits

58 Dynamic Branch Prediction 2-bit predictors Branch Prediction Bits 2-bit Bimodal Saturating Counter

59 Dynamic Branch Prediction 2-bit predictors Branch Prediction Bits 2-bit Bimodal Saturating Counter

60 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

61 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

62 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

63 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

64 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

65 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

66 Dynamic Branch Prediction 2-bit predictors T T T N T T T T T T T T...

67 Dynamic Branch Prediction n-bit saturating counters

68 Branch Predictors Without Branch Predictor With With Branch Predictor

69 Observations? Dynamic Branch Prediction

70 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) {

71 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Outcome of the previous branch X 0010 Branch Prediction Buffer

72 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Two-level predictors (1,2) predictor Outcome of the previous branch X 0010 Branch Prediction Buffer

73 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Two-level predictors (1,2) predictor Outcome of the previous branch X 0010 Branch Prediction Buffer Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

74 Correlating Branch Predictors if (aa == 2) aa = 0; Outcomes of the previous 2 branches Branch Prediction Buffer if (bb == 2) bb = 0; if (aa!=bb) { XX 0010 A 4096 bit, (2,2) buffer supports how many branch instructions? Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

75 Correlating Branch Predictors if (aa == 2) aa = 0; Outcomes of the previous 2 branches Branch Prediction Buffer if (bb == 2) bb = 0; if (aa!=bb) { XX 0010 A 4096 bit, (2,2) buffer supports how many branch instructions? (m,n) BPB bits=2 m n No. of prediction entries Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

76 Correlating Branch Predictors Yeh and Patt, ISCA, 1992.

77 Local Predictors Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

78 Local Predictors A history of branch behaviour is recorded Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

79 Local Predictors A history of branch behaviour is recorded One for each possible combination of outcomes for the last n occurrences of this branch Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

80 Local Predictors Example 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T N T T T T N T T T T N T T

81 Local Predictors Example 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T N T T T T N T T T T N T T Local History Prediction NT NT T T T T T T T T NT NT

82 Tournament Predictors

83 Tournament Predictors Branch PC PC Local predictor Global predictor Predictor Selector Prediction

84 Tournament Predictors Use multiple predictors: Global, local or mix Branch PC PC Local predictor Global predictor Predictor Selector Prediction

85 Tournament Predictors Use multiple predictors: Global, local or mix Combine them with a selector 2 bit saturating counter to select the right predictor for the branch (global vs. local) Branch PC PC Local predictor Global predictor Predictor Selector Prediction

86 Tournament Predictors

87 Outline Pipeline, Pipelined datapath Dependences, Hazards Structural, Data - Stalling, Forwarding Control Hazards Branch prediction