AUTOMATED DEBUGGING. Session Leader: Zongxu Mu

Transcription

1 AUTOMATED DEBUGGING Session Leader: Zongxu Mu

2 THE FIRST COMPUTER BUG Source: 10/22/2013 Automated Debugging - Zongxu Mu 2

3 A RANDOM COMPUTER BUG Srouce: 10/22/2013 Automated Debugging - Zongxu Mu 3

4 IF MICROSOFT MADE CARS Occasionally your car would just die for no reason, you'd have to restart it. For some strange reason, you would just accept this. Source: 10/22/2013 Automated Debugging - Zongxu Mu 4

5 MOTIVATION Motivation as a Computer Science Student For proficient debuggers For those who hate debugging Relationship with the whole course Machine Learning Optimization Automated Algorithm Design 10/22/2013 Automated Debugging - Zongxu Mu 5

6 AUTOMATED DEBUGGING Bug/fault isolation/localization Use tools to find the likely region/lines Examine by programmers Correct it! (Automatically?) 10/22/2013 Automated Debugging - Zongxu Mu 6

7 A WARM-UP EXERCISE Question: how do you debug? 10/22/2013 Automated Debugging - Zongxu Mu 7

8 #include <stdio.h> #include <stdlib.h> /* This function is buggy - Let's start debugging ;) * Given a year, the function returns whether it is a leap year. * A leap year is a year that can be dividable by 4, with the exception that * if the year is several hundreds, it should be dividable by 400. * So, what is wrong here? * Need help? Check the test cases! */ bool isleapyear(int year) { if (year & 3!= 0) // Line 1; & is the "bit-wise and" operator return false; // Line 2 else if (year%100 == 0 && year%400!= 0) // Line 3 return false; // Line 4 else // Line 5 return true; // Line 6 } int main(int argc, char* argv[]) { printf("%d\n", isleapyear(atoi(argv[1]))); return 0; } 10/22/2013 Automated Debugging - Zongxu Mu 8

9 SOME TEST CASES Command Result Correct/Wrong./leap (False) Correct./leap (True) Correct./leap (False) Correct./leap (True) Wrong./leap (True) Correct./leap (False) Correct 10/22/2013 Automated Debugging - Zongxu Mu 9

10 TRADITIONAL FAULT LOCALIZATION Program Snapshots Deduction Memory dump Insert print statements printf("wtf.."); Debugger-like tools Break points Source: 10/22/2013 Automated Debugging - Zongxu Mu 10

11 AUTOMATED FAULT LOCALIZATION Execution Slice-based Program Spectrum-based Statistics-based * Program State-based * Machine Learning-based Adapted from [Wong & Debroy, 2009] - the second secondary reading (link) 10/22/2013 Automated Debugging - Zongxu Mu 11

12 BP NEURAL NETWORK FOR FAULT LOCALIZATION A Machine Learning-based Approach 10/22/2013 Automated Debugging - Zongxu Mu 12

13 A STRAIGHT-FORWARD RATIONALE There is a relationship between Whether a statement is executed, and Whether the execution result is successful To capture that relationship, we use 10/22/2013 Automated Debugging - Zongxu Mu 13

14 BP NEURAL NETWORK Inputs Output XX nn mm (1) Θ (mm+1) h (1) Θ (mm+1) h aa (1) aa (2) aa (3) yy nn 10/22/2013 Automated Debugging - Zongxu Mu 14

15 BPNN-BASED FAULT LOCALIZATION Prepare test coverage data Build a BP neural network Train the neural network with data Use virtual data to obtain suspiciousness 10/22/2013 Automated Debugging - Zongxu Mu 15

16 PREPARE TEST COVERAGE DATA Statement Coverage 10/22/2013 Automated Debugging - Zongxu Mu 16

17 BUILD A BP NEURAL NETWORK mm input-layer neurons mm executable statements The coverage of each statement is a feature 1 hidden layer of h neurons 1 output neuron ssssssssssssss xx = 1 1+ee xx 10/22/2013 Automated Debugging - Zongxu Mu 17

18 TRAIN THE NEURAL NETWORK Randomize Weights (Θ) Forward Propagation aa nn+1 = ssssssssssssss Θ nn aa nn Back Propagation Adjust Θ based on computer h Θ xx 10/22/2013 Automated Debugging - Zongxu Mu 18

19 VIRTUAL CASES SUSPICIOUSNESS How to get the suspiciousness of a statement? A virtual case with only ONE statement covered Suspiciousness level of that statement Is this a kind of Transfer Learning? 10/22/2013 Automated Debugging - Zongxu Mu 19

20 ENHANCEMENTS & DISCUSSIONS # of candidate statements The fault is related to statements covered by failed cases SS II = SS tt ff1 SS tt ff2 SS tt ffnn kk SS MM = SS tt ffmm with the fewest statements # of hidden-layer neurons h Multiple bugs? Clustering failed tests Expandability coverage at module level 10/22/2013 Automated Debugging - Zongxu Mu 20

21 Source: LET S TRY IT! Search for TODO in nncostfunction.m and complete the program 10/22/2013 Automated Debugging - Zongxu Mu 21

22 RESULTS Pred = The BUG is here! These 3 lines are even higher! 10/22/2013 Automated Debugging - Zongxu Mu 22

23 WEAKNESS Program Structures Branching Looping Direct and indirect causal statements 10/22/2013 Automated Debugging - Zongxu Mu 23

24 DELTA DEBUGGING TO ISOLATE CAUSE-EFFECT CHAINS FROM PROGRAMS A Program State-based Approach 10/22/2013 Automated Debugging - Zongxu Mu 24

25 CAUSE-EFFECT CHAIN variable v1 was x1, and then variable v2 became x2, thus variable v3 became x3,..., and at last, the program failed vv 1 = xx 1 vv 2 = xx 2 vv 3 = xx 3 Failure 10/22/2013 Automated Debugging - Zongxu Mu 25

26 DELTA DEBUGGING Is about isolating Relevant input Divide and conquer Delta (δδ) debugging Relevant program states The cause-effect chain, and then The error 10/22/2013 Automated Debugging - Zongxu Mu 26

27 DELTA-DEBUGGING RELEVANT INPUT Divide-and-conquer What is the smallest difference between A passing input and a failed input Delta (δδ) debugging δδ = δδ 1 δδ 2 δδ nn of atomic differences φφ = cc ff cc ff cc ss cc ss = δδ 1, δδ 2,, δδ nn cc ss cc ff is 1-minimal 10/22/2013 Automated Debugging - Zongxu Mu 27

28 DELTA-DEBUGGING PROGRAM STATES Minimized input difference Large program state difference Delta-debug program states Adapted from Zeller s presentation at Microsoft Research, October 2003 (link) 10/22/2013 Automated Debugging - Zongxu Mu 28

29 DELTA-DEBUGGING PROGRAM STATES Memory Graphs An address in rr ss can be meaningless in rr ff Node variables Edge references Pointer dereferencing Struct/Class member access Array element access Adapted from [Cleve and Zeller, 2005] (link) 10/22/2013 Automated Debugging - Zongxu Mu 29

30 DELTA-DEBUGGING PROGRAM STATES Structural graph differences Difference in the set of variables Detect changes by computing a large common subgraph Small graph: the largest common subgraph Large graph: a large common subgraph Difference State deltas (δδ s) 10/22/2013 Automated Debugging - Zongxu Mu 30

31 ISOLATING CAUSE-EFFECT CHAINS Compare executions at certain locations vv 1 = xx 1 vv 2 = xx 2 vv 3 = xx 3 Failure 10/22/2013 Automated Debugging - Zongxu Mu 31

32 ISOLATING THE ERROR To break the cause-effect chain Causality standpoint: all break points equivalent Programmer viewpoint: a most desired fix/break The fault is in the eye of the beholder Adapted from Zeller s presentation at Microsoft Research, October 2003 (link) 10/22/2013 Automated Debugging - Zongxu Mu 32

33 DELTA DEBUGGING An automated debugging system requiring Automated testing At least one case passes as reference A debugger tool Retrieve variables and their values Alter the values 10/22/2013 Automated Debugging - Zongxu Mu 33

34 FUTURE WORK Cause transitions [Cleve and Zeller, 2005] Search in the time domain When a cause changes from one variable to another 10/22/2013 Automated Debugging - Zongxu Mu 34

35 ML & CAUSE-EFFECT CHAIN Jiang, L., & Su, Z. (2005). Automatic isolation of cause-effect chains with machine learning. Tech. rep., Technical Report CSE , University of California, Davis. 10/22/2013 Automated Debugging - Zongxu Mu 35

36 PAST AND FUTURE Deduction Observation Induction Experimentation Adapted from Zeller s presentation at Microsoft Research, October 2003 (link) 10/22/2013 Automated Debugging - Zongxu Mu 36

37 THANK YOU! 10/22/2013 Automated Debugging - Zongxu Mu 37

38 REFERENCES Delta Debugging Project. ( Jiang, L., & Su, Z. (2005). Automatic isolation of cause-effect chains with machine learning. Tech. rep., Technical Report CSE , University of California, Davis. Wong, W. E., & Debroy, V. (2009). A survey of software fault localization. University of Texas at Dallas, Tech. Rep. UTDCS Wong, W. E., & Qi, Y. (2009). BP neural network-based effective fault localization. International Journal of Software Engineering and Knowledge Engineering, 19(04), Zeller, A. (2002). Isolating cause-effect chains from computer programs. In Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering (pp. 1-10). 10/22/2013 Automated Debugging - Zongxu Mu 38