Diagnostic quality problem solving

Transcription

1 Diagnostic quality problem solving Strategies for identifying causes of problems Karmiel, 2 June 2016 Jeroen de Mast University of Amsterdam j.demast@uva.nl Jeroen de Mast Personal introduction Prof. of Methods & Statistics for Operations Management Amsterdam Business School, University of Amsterdam Principal consultant, much experience in high-tech, manufacturing, healthcare, services, logistics industries. Research interests: Statistical evaluation of measurement and testing systems. Appointment scheduling Operations improvement in healthcare Lean, Six Sigma, problem-solving. 2

2 Electrical instabilities High voltage electrical device Connector / cable Power supply Problem: Electrical instabilities 3 Diagnostic problem solving Electrical instabilities : From week 12 (2008) onwards. Product and connector completely destroyed when connected to power supply for 1 out of 8 products (12%). Only affects product type TA. Similar product TB not affected. Diagnostic problem solving: What are the causes of the problem? 4

3 Check for anomalies in the production process: Dimensional variation in the connectors Color differences Contamination of tools with chemicals. Electrical instabilities Brainstorming: Main suspect is the connector. Various explanations and causes were suggested. Result: > 45 potential causes, to be investigated in a statistically designed experiment 5 Sort 45 potential causes by multi-voting Design experiment for top 15 Electrical instabilities % Three months later: design: Still 1 out 32 runs of 8 products destroyed. Geometry of connector Tool coating Operator handling of connector Process tool setup Connector depth Process control Amount of grease Insulator geometry 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 6

4 Brainstorming & DOE approach: Case: conclusion - Extensive search space causes not enumerable or multitudinous laborious - Extensive search space problem solver may get bogged down in wrong part of the search space - Raised candidate causes are too unspecific to allow useful experimentation The Brainstorming & DOE approach is effective only after sufficient focus has been achieved. Needed for diagnosis in complex situations: strategies for achieving focus (= narrowing down the search space). 7 Diagnostic quality problem solving Conceptual framework: Diagnosis is a search through a state-space

5 Quality engineering: A study of diagnostic strategies Juran: diagnostic and remedial journey Six Sigma: rather incoherent and poorly structured collection of tools (5 Why, C&E-diagram, exploratory data-analysis, ) Shainin, Kepner & Tregoe: branch-and-prune strategy Scientific literature: Troubleshooting of devices Artificial intelligence Medical diagnosis - Conceptual framework - Six strategies for efficient diagnosis 9 Conceptual framework Accepted: Under consideration: State space Rejected: H 5 H 7 H H 1 3 H 8 H H 2 4 H 6 generation testing evaluation Domain knowledge: Physical structure Functional structure Operations context Normal behavior General knowledge Fault theory: Fault dictionaries Pathologies Taxonomies Pattern recognition procedures Observations: Observational data Tests and experiments Findings Domain knowledge and observations Conceptual framework for the process of diagnosis: Based on concepts from Artificial Intelligence As a search through a state space until a goal state is reached. 10

6 Conceptual framework: hypotheses Find a causal explanation for the observed problem. Potential explanations: hypotheses. Knife chipped H 1 Cutting process malfunctions Many rejects Product stretches when cut Dimensional variation Many rejects H 2 Hypotheses can be global and vague, or specific and detailed. 11 Diagnosis: search through a state space Accepted: Under consideration: State space Rejected: H 5 H 7 H H 1 3 H 8 H H 2 4 H 6 Hypotheses can be under consideration, accepted or rejected. These determine the states in the search process. Goal-state: a sufficiently specific hypothesis is accepted. 12

7 Diagnosis: search through a state space Accepted: Under consideration: State space Rejected: H 3 H 1 H 2 H 4 generation testing evaluation Domain knowledge: Physical structure Functional structure Operations context Normal behavior General knowledge Fault theory: Fault dictionaries Pathologies Taxonomies Pattern recognition procedures Observations: Observational data Tests and experiments Findings Domain knowledge and observations 13 Diagnosis: search through a state space The search progresses from state to state by three operators: generation: invent a new hypothesis Status of new hypothesis: under consideration Driven by domain knowledge and observations testing: collect new observations and knowledge for evaluating a hypothesis Guided by hypotheses under consideration evaluation: determine whether a hypothesis is accepted or rejected Driven by domain knowledge and observations 14

8 Diagnosis: search through a state space Brainstorming Accepted: Under consideration: Design of State space Rejected: H 5 H 7 experiments H H 1 3 H 8 H H 2 4 H 6 Statistical inference generation testing evaluation Domain knowledge: Physical structure Functional structure Operations context Normal behavior General knowledge Fault theory: Fault dictionaries Pathologies Taxonomies Pattern recognition procedures Observations: Observational data Tests and experiments Findings Domain knowledge and observations 15 Diagnostic strategies Accepted: Under consideration: State space Rejected: H 5 H 7 H 8 H 6 H 1 H 3 H 4 H 2 Diagnostic strategy (how, when, in what order?) generation testing evaluation Domain knowledge and observations Diagnostic strategy: makes the search process more efficient. 16

9 Diagnostic quality problem solving Six diagnostic strategies: Search tactics that make diagnosis more efficient Six strategies for diagnosis Tools for diagnosing quality problems: C&E matrix, 6xWhy, Shainin System, brainstorming, DOE, pairwise comparisons, IS vs IS-NOT analysis, Strategy: the thinking patterns on which they are based: 1. Blind trial & error 2. Branch-and-prune tactics 3. Known problem 4. Proximate causes 5. Syndrome-based search 6. Funneling strategy 18

10 1. Blind trial & error 2. Branch-and-prune tactics 3. Known problem 4. Proximate causes 5. Syndrome-based search 6. Funneling strategy Six strategies for diagnosis 1. Blind trial & error search: - Randomly try out candidate causes until you find the one that causes the problem. - Effectively the least efficient search strategy possible. - Finite number of causes expected trials Branch-and-prune 2. Branch-and-prune tactics: - Divide the space of all possible causes in high-level sub-classes ( branching ) - On the basis of the structure of the product or process or based on generic structures such as time and space. - Next, by observation and testing try to rule out whole branches at once ( pruning ). - Zoom-in on the retained branches, studying them in more detail. - Eliminate-and-zoom-in 20

11 2. Branch-and-prune Component swapping Power supply and connector used for TA s adjusted such that they can be used for the TB s. TB TA Product Connector Power supply Result: power supply and connector give no problems when used for the TB s, but they do give problems when used for the TA s. Conclusion: problem is in the product itself, not in the connector or power supply Cause of instabilities Product & production history Connector Power supply Branch-and-prune Branch-and-prune based on time - Problems emerged suddenly in week Conclusion: cause must be something that changed in wk P Chart of Instabilities Proportion of Instabilities UCL= _ P= LCL=0 Cause of instabilities Week Tests performed with unequal sample sizes Events before wk. 28 Events wks. 28,29 Events after wk

12 Branch-and-prune based on product type - TA s are affected, TB s are not. 2. Branch-and-prune - Conclusion: cause must be something that distinguishes the production processes or product design of the TA s from those of the TB s. - Zoom-in on differences main difference turns out to be the soldering process. TA TB Other techniques for branch-and-prune: Half-split technique (bisection) 4W2H questions (who, where, what, when, how, how much) Branch-and-prune based on functional structure Known problem Symptoms Possible explanations Looking on the internet whether a problem is known 24

13 3. Known problem 3. Known problem Study whether the problem is known. Examine defective products closely, listing the observed symptoms. What does the damage look like? Can you find abnormal measurements? Can you disassemble a product to examine what went wrong ( autopsies )? Use the list of symptoms as a query in a search through a knowledge base, where earlier experiences with similar problems are stored: Consult experts or the expert literature. Do a search on internet. Discuss the observed symptoms with colleagues ( brainstorming ). Study analogous processes / products / problems Known problem Literature search for known issues with similar electrical devices: Dust Contamination with salts Contamination with metal particles Enclosures of air bubbles Salt Surfaces of some TAs cleaned, and residue analyzed sodium chloride ( table salt ) present on the surfaces. 26

14 4. Proximate causes strategy 4. Proximate causes Try to reason backwards from the observed symptoms to their immediate ( proximate ) causes. Thus, one moves the problem description one step upwards in the chain of cause-and-effect. This gives a more focused problem definition. Technique: 5xWhy? (Ask why? five times). Salt deposit Short circuit Instabilities From observations, the immediate cause of the instabilities turns out to be: a short circuit caused by a salt residue on the product s surface. Problem definition recast from What causes the instabilities?, to: Where does the salt residue come from? 27 TA TB Combining clues Where does the salt come from? Soldering process Soldering flux - Used for the TAs but not for the TBs. - Introduced recently (week 28?). - Turns out to contain sodium chloride. Soldering flux discontinued - 4 weeks later: 0 instabilities - Problem solved! 28

15 1. Blind trial & error 2. Branch-and-prune tactics 3. Known problem 4. Proximate causes 5. Syndrome-based search 6. Funneling strategy 5. Syndrome-based search Observe a number of occurrences of the problem, and compare them to normal behavior (BOB vs WOW comparison). Try to find a pattern of concomitant symptoms and characteristics that occur together with the problem This pattern is the syndrome. The pattern may reveal characteristics of the causal mechanism that may help in ruling out options Syndrome-based search Example: eccentricity of pins on cell phone components Histogram of Eccentricity Frequency Eccentricity Syndrome: eccentricity has a bi-modal distribution. cause of eccentricity must have two distinguishable states. Injection molding: 2 molds, one of them worn-out. 30

16 6. Funneling strategy 6. Funneling strategy Generate a list of specific and detailed hypotheses by brainstorming. Design an efficient testing strategy (for example: DOE). Only efficient after the search space has been narrowed down (focus). 31 Diagnostic quality problem solving A general strategy for problem diagnosis

17 Electrical instabilities: search process Brainstorming Focus on connector No convincing results Swapping test Focus on product itself Comparison of production processes of TA and TB Focus on soldering process Literature search Found salt residue Possible explanation Salt residue causes short circuits Study soldering process in detail, looking for origin of salt residue Soldering flux identified as culprit Funneling Branch & prune Branch & prune Known problem Proximate causes Funneling 33 Think strategically Think strategically Use pruning principles to focus the search. but do not follow a single strategy rigidly At each stage in the search, reassess the situation and its tactical consequences. 1. Known problem 2. Proximate causes 3. Branch-and-prune 4. Syndrome-based strategy Known problem? Achieve focus on the relevant part of the problem space 5. Funneling Efficient testing of detailed hypotheses 34

18 Problem solving in quality engineering Joseph Juran: Universal sequence for quality improvement Diagnostic journey from symptoms to causes. Remedial journey from causes to remedy. Shainin and Kepner & Tregoe systems Well structured strategies for problem diagnosis Limited to pruning strategies Poor grounding in science Six Sigma s DMAIC model Analyze (DMAIC): identification of causes that determine the CTQ s behavior Many tools and techniques for hypothesis generation: Brainstorming Cause & Effect diagram 5xWhy? Exploratory data analysis Strong techniques for testing the effects of hypothesized causes 35 No or only rudimentary strategic structure Examples: Pyzdek and Breyfogle: Six Sigma Incoherent collection of tools for hypothesis generation and testing No strategy or structure Example: Pande et al. ( Root cause analysis circle ) Analyze data / process Develop causal hypotheses (one or more) Analyze data / process Refine or reject hypotheses After a few cycles: Confirm and select vital few causes Limited to funelling strategy Examples: George et al, Gitlow et al. Brainstorming and fishbone diagram: list of candidate causes Regression analysis: narrow down candidate causes until the vital few remain 36

19 Lessons from AI, medical diagnosis, troubleshooting: Conclusions Problem diagnosis = search through a space of potential explanations ( hypotheses ) Search progresses towards goal state by hypothesis generation, testing and evaluation. Diagnostic strategies structure theses tasks and make them efficient. Opportunity: Courses on Six Sigma s DMAIC fall short Organize tools in strategic structure 37 38