Outline. Pipeline, Pipelined datapath Dependences, Hazards. Control Hazards Branch prediction. Structural, Data - Stalling, Forwarding

Transcription

1 Control Hazard

2 Outline Pipeline, Pipelined datapath Dependences, Hazards Structural, Data - Stalling, Forwarding Control Hazards Branch prediction

3 Control Hazard Arise from the pipelining of branches and other instructions that change the PC

4 Control Hazard Arise from the pipelining of branches and other instructions that change the PC Also called Branch Hazards

5 Branch Evaluation

6 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw lw $4, $4, 50($7) Control Hazard

7 beq Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles)

8 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF and

9 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX and or IF ID IF

10 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard Branch Taken lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX and or IF ID IF

11 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard NPC Updates lw Time lw $4, $4, 50($7) (clock cycles) beq and IF ID EX MEM IF ID EX or add IF ID IF

12 beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $2 Control Hazard NPC Updates lw Time lw $4, $4, 50($7) (clock cycles) beq and IF ID EX MEM IF ID EX or add IF ID IF Insert NOP NOP

13 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 Branch add add $14, $2, $2, $2 target $2 enters the the pipeline lw Time lw $4, $4, 50($7) (clock cycles) beq and or IF ID EX MEM WB IF ID EX MEM IF ID EX add IF ID Target: lw IF

14 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq and or add IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM IF ID EX Target: lw IF ID Target+1 IF

15 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX MEM WB and IF ID EX MEM WB or IF ID EX MEM WB add IF ID EX MEM WB Target: lw IF ID EX MEM WB Target+1 IF ID EX MEM

16 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) Target: beq and or add lw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Penalty = 3 cycles

17 Control Hazard Branch evaluation occurs in EX stage. PC updates in MEM stage. Branch target loads in the next cycle The delay in determining the proper instruction to fetch is called a Control Hazard or Branch Hazard.

18 Reducing Pipeline Branch Penalties

19 Reducing Pipeline Branch Penalties Freeze the pipeline

20 Reducing Pipeline Branch Penalties Freeze the pipeline Time (clock cycles) beq nop nop nop lw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

21 Reducing Branch Delay

22 Reducing Branch Delay Most branches rely on simple tests (equality or sign) Move the branch decision up

23 Reducing Branch Delay Most branches rely on simple tests (equality or sign) Move the branch decision up Compute NPC earlier Evaluate branch at the earliest

24 Reducing Branch Delay

25 NPC Reducing Branch Delay

26 Reducing Branch Delay NPC Branch Delay: Delay between Branch instruction fetch and Branch Target fetch of of a taken branch.

27 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD BEQ IF ID IF

28 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX BEQ IF ID SUB IF

29 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX BEQ IF ID SUB IF PC = PC + Offset

30 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM BEQ IF ID EX SUB IF NOP ID Branch Target IF

31 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM WB BEQ IF ID EX MEM SUB IF NOP ID NOP EX Branch Target IF ID

32 Branch Delay ADD BEQ SUB... R2, R3,R4 R2, R4, loop R5, R5,R4 Time (clock cycles) ADD IF ID EX MEM WB BEQ IF ID EX MEM WB SUB IF NOP ID NOP EX MEM NOP NOP WB Branch Target IF ID EX MEM WB

33 Branch Delay Slot Time (clock cycles) Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Delay Slot

34 Branch Hazards Time (clock cycles) Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Branch Target IF IF ID EX MEM WB 1 stall cycle for every branch yields a performance loss of 10% to 30%!

35 Example Branch Taken

36

37

38 Branch Evaluation in ID Steps Are beq inputs ready?

39 Branch Evaluation in ID Steps Decode the instruction

40 Branch Evaluation in ID Steps Decode the instruction Decide whether to bypass/forward to the equality unit Forwarding Logic has to be modfied

41 Branch Evaluation in ID Steps Decode the instruction Decide whether to bypass/forward to the equality unit Forwarding Logic has to be modfied Complete the comparison

42 Branch Evaluation in ID Steps Decode the instruction Decide whether to bypass/forward to the equality unit Forwarding Logic has to be modfied Complete the comparison If branch taken, set the PC to the branch target address.

43 Branch Evaluation Forwarding New forwarding logic required Source operands can come from ALU/MEM or MEM/WB add add beq beq $1, $1,$2, $2,$2 $2 $1, $1,$3, $3,28 28

44 Branch Evaluation Forwarding New forwarding logic required Source operands can come from ALU/MEM or MEM/WB If source operands are not ready, data hazard can occur and a stall will be needed add add $1, $1, $2, $2, $2 $2 beq beq $1, $1, $3, $3,

45 Reducing Pipeline Branch Penalties Freeze the pipeline Static Prediction Predict Untaken, Predict Taken Delayed Branch Fill Branch Delay Slot

46 Predict Untaken Scheme Time (clock cycles) Untaken Branch IF ID EX MEM WB Instruction i + 1 Instruction i + 2 Instruction i + 3 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

47 Predict Untaken Scheme Time (clock cycles) Untaken Branch IF ID EX MEM WB Instruction i + 1 Instruction i + 2 Instruction i + 3 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Time (clock cycles) Taken Branch IF ID EX MEM WB Instruction i + 1 Branch Target Branch Target + 1 IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB

48 Static Prediction Predict Taken

49 Reducing Pipeline Branch Penalties Fill the branch delay slot sll sll srl srl beq beq and and $14, $14,$2, $2,$2 $2 $1, $1,$3, $3,28 28 $12, $12,$2, $2,$5 $5

50 Reducing Pipeline Branch Penalties Fill the branch delay slot sll sll srl srl $14, $14, $2, $2, $2 $2 beq beq $1, $1, $3, $3, and and $12, $12, $2, $2, $5 $ Time (clock cycles) beq srl IF ID EX MEM WB IF ID EX MEM WB branch target IF ID EX MEM WB

51 Branch Delay Slot

52 Branch Delay Slot

53 Branch Delay Slot

54 Branch Delay Slot Predict taken

55 Branch Delay Slot Predict taken

56 Branch Delay Slot Predict taken Predict untaken

57 Static Branch Prediction

58 Static Branch Prediction Reduce mispredictions?

59 Dynamic Branch Prediction 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }}

60 Dynamic Branch Prediction 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

61 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

62 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T T T N T T T T T T T T T T T T Wrong Predictions BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program

63 Dynamic Branch Prediction Branch Target Buffer Single bit predictors (1-bit bimodal predictor) Change prediction with branch behaviour No. of wrong predictions? 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T T T N T T T T T T T T T T T T Wrong Predictions BRANCH TARGET BUFFER PC Prediction Target 0x x128 0x x080 0x x Addresses of branches in the program Can Can we we do do better?

64 Dynamic Branch Prediction 0010 Branch Prediction Bits

65 Dynamic Branch Prediction 2-bit predictors Branch Prediction Bits 2-bit Bimodal Saturating Counter

66 Dynamic Branch Prediction 2-bit predictors Branch Prediction Bits 2-bit Bimodal Saturating Counter

67 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

68 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

69 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

70 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

71 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

72 Dynamic Branch Prediction 2-bit predictors... T T T N T T T T T T T T...

73 Dynamic Branch Prediction 2-bit predictors T T T N T T T T T T T T...

74 Dynamic Branch Prediction n-bit saturating counters

75 Branch Predictors Without Branch Predictor

76 Branch Predictors Without Branch Predictor With With Branch Predictor

77 Dynamic Branch Prediction

78 Dynamic Branch Prediction

79 Observations? Dynamic Branch Prediction

80 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) {

81 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Outcome of the previous branch X 0010 Branch Prediction Buffer

82 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Two-level predictors (1,2) predictor Outcome of the previous branch X 0010 Branch Prediction Buffer

83 Correlating Branch Predictors eqntott code code if (aa == 2) aa = 0; if (bb == 2) bb = 0; if (aa!=bb) { Two-level predictors (1,2) predictor Outcome of the previous branch X 0010 Branch Prediction Buffer Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

84 Correlating Branch Predictors if (aa == 2) aa = 0; Outcomes of the previous 2 branches Branch Prediction Buffer if (bb == 2) bb = 0; if (aa!=bb) { XX 0010 A 4096 bit, (2,2) buffer supports how many branch instructions? Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

85 Correlating Branch Predictors if (aa == 2) aa = 0; Outcomes of the previous 2 branches Branch Prediction Buffer if (bb == 2) bb = 0; if (aa!=bb) { XX 0010 A 4096 bit, (2,2) buffer supports how many branch instructions? (m,n) BPB bits=2 m n No. of prediction entries Yeh and Patt, Alternative implementations of two-level adaptive branch prediction, ISCA, 1992.

86 Correlating Branch Predictors Yeh and Patt, ISCA, 1992.

87 Local Predictors Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

88 Local Predictors A history of branch behaviour is recorded Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

89 Local Predictors A history of branch behaviour is recorded One for each possible combination of outcomes for the last n occurrences of this branch Previous outcomes of the same branch Branch Prediction Buffer XXXXXXXXXX K entries

90 Local Predictors Example 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T N T T T T N T T T T N T T

91 Local Predictors Example 0x0100 0x0100 while(1) {{ for(i=0; i<count; i++) i++) {{ }} BRANCH BRANCH }} Branch instruction behaviour T T N T T T T N T T T T N T T Local History Prediction NT NT T T T T T T T T NT NT

92 Tournament Predictors

93 Tournament Predictors Branch PC PC Local predictor Global predictor Predictor Selector Prediction

94 Tournament Predictors Use multiple predictors: Global, local or mix Branch PC PC Local predictor Global predictor Predictor Selector Prediction

95 Tournament Predictors Use multiple predictors: Global, local or mix Combine them with a selector 2 bit saturating counter to select the right predictor for the branch (global vs. local) Branch PC PC Local predictor Global predictor Predictor Selector Prediction

96 Tournament Predictors

97 Outline Pipeline, Pipelined datapath Dependences, Hazards Structural, Data - Stalling, Forwarding Control Hazards Branch prediction

98 Xtra Slides

99 Pipelined vs. Nonpipelined Implementation Ratio of execution times between the two? For 10 6 instructions? Pipelining increases the instruction throughput opposed to individual instruction execution time. IF ID EX MEM WB

100 Throughput of the Pipeline Number of tasks completed in unit time (one second) w=η f f: frequency of operation

101 Efficiency of the Pipeline Percentage of stages accomplishing tasks related to the instruction in execution η= No. of Instructions Instruction Execution Time η= n k+(n 1) n: number of instructions in the program. k: number of pipeline stages

102 Reducing Pipeline Branch Penalties Freeze the pipeline Predict Taken Predict Untaken Fill Branch Delay Slot Time (clock cycles) i BEQ IF ID EX MEM WB i-1 AND IF ID EX MEM WB i+16 Branch Successor IF ID EX MEM WB i+17 Branch Successor + 1 IF ID EX MEM WB

103 Control Hazard Branch Penalty = 3 cycles

104 Branch Prediction Buffer BIMODAL PREDICTOR Branch PC PC A 12 bits Counter Target PC PC Branch PC PC B Two Two different branch PCs PCs may may map map to to the the same same entry entry in in the the BPB BPB Given limited buffer space, how can can one maximize buffer entries while minimizing aliases? entries

105 BPB Global Predictor GLOBAL PREDICTOR Branch PC PC A Counter Target PC PC 10 bits 2 bits Concatenate 12 bits Global History Aliases may may appear entries

106 BPB Aliases GLOBAL PREDICTOR Branch PC PC A 12 bits XOR Counter Target PC PC 12 bits Global History May May reduce aliases entries

107 BPB Local Predictor LOCAL PREDICTOR 12 bits Branch PC PC A Counter Target PC PC 6 bits 12b 12b entries May May reduce aliases entries

108 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 WRONG!!! or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq and or add IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM IF ID EX Target: lw IF ID Target+1 IF

109 Control Hazard beq beq $1, $1, $3, $3, and and $12, $2, $2, $5 $5 or or $13, $6, $6, $2 $2 add add $14, $2, $2, $2 $ lw Time lw $4, $4, 50($7) (clock cycles) beq IF ID EX MEM WB and IF ID EX MEM WB or IF ID EX MEM WB add IF ID EX MEM WB Target: lw IF ID EX MEM WB Target+1 IF ID EX MEM

110 Early Branch Evaluation Evaluate branch at the earliest beq beq $1, $1, $3, $3, When are the source operands available? After Register Read

111 BPB Local Predictor LOCAL PREDICTOR Branch PC PC A Counter Target PC PC 6 bits 12b 12b entries entries