ERT: Early Reliability Test

Size: px

Start display at page:

Download "ERT: Early Reliability Test"

Claude Stevenson
6 years ago
Views:

ERT: Early Reliability Test This shows how to test for reliability and lifetime, by starting before product completion. This exploits strong correlation between generations.

1 ERT: Early Reliability Test This shows how to test for reliability and lifetime, by starting before product completion. This exploits strong correlation between generations. Qualitative test can be done early and fruitfully, even though quantitative test becomes feasible only later. This offers: earlier feedback, easier and smarter engineering, better quality, better tests for reliability & lifetime, fewer late problems. Mike Silverman and Arthur Zingher Ops A La Carte, LLC; Santa Clara CA Early Reliability Testing Page 1

2 ERT Intro 1 Industries compete on reliability. Companies need to develop more reliable products faster Reliability tests and improvements often are performed when time is short development is nearly complete engineering corrections are difficult and costly the product is nearly frozen Early Reliability Testing Page 2

3 ERT Intro 2 Instead, Early Reliability Test (ERT) is a development tactic that offers earlier feedback and thus enables more lead time smarter engineering better reliability and quality less total cost & risk Early Reliability Testing Page 3

4 ERT Intro 3 ERT needs to overcome potential challenges samples from immature manufacturing low test coverage too few samples immature designs parallel / concurrent designs (can t test until integrated) This will show how to overcome these challenges in many cases. Early Reliability Testing Page 4

5 ERT Intro 4 This presentation describes concepts & techniques for ERT potential benefits and cost-savings how to overcome potential challenges relevant case studies Early Reliability Testing Page 5

6 When to test reliability 1 Fig 1: Typical Development Phases When should we test for reliability?? During P1, when engineering changes are easier?? During P2, when engineering is more mature, so tests are more complete? Early Reliability Testing Page 6

7 When to test reliability 2 Best reliability test is done at all three phases: P1 test for early reliability feedback P2 test for cleaner specimens, and better coverage P3 test to validate the design Also, start Lifetime & Reliability Demo during P2, so this is completed before the end of P3. Early Reliability Testing Page 7

8 When to test reliability 3 For a project that develops subunits in parallel test each subunit as early as it is available (P1) rather than waiting for final system test (P3) when it is painfully late. Fig 2 shows when to test. Early Reliability Testing Page 8

9 When to test reliability 4 Figure 2: Reliability Testing during product development Early Reliability Testing Page 9

10 Correlated generations 1 Development typically includes successive generations: prior product preliminary prototype intermediate prototype final new product A test specimen can be a complete product complete prototype sub-assembly significant component Early Reliability Testing Page 10

11 Correlated generations 2 Pivotal: Successive generations are usually strongly correlated in defects, wear, fatigue, failures, mechanisms and root causes. Previously, this correlation was not sufficiently appreciated Early Reliability Testing Page 11

12 Correlated generations 3 This correlation typically applies at many levels Technologies & resources for reliability test Mechanisms for reliability and lifetime effects Results of reliability tests This correlation enables earlier start for testing smarter testing later fewer compromises driven by a tight schedule. Early Reliability Testing Page 12

13 Correlated generations 4 This correlation facilitates: Use earlier generation(s) to develop test technology & test resources For later generation(s) reuse this test technology & test resources Thus reliability work on successive generations incurs little more investment than reliability work solely on final generation Early Reliability Testing Page 13

14 Correlated generations 5 Suppose reliability & lifetime early tests examine just a small population but learn its defects early These defects usually are correlated with defects in later test of a larger population and defects in later generation By early testing & understanding of even a few specimens, we can often gain useful early qualitative feedback Early Reliability Testing Page 14

15 Correlated generations 6 These correlations enable smarter tactics: In parallel with early development, use a few specimens from an early generation to develop test technology & resources such as test apparatus test methods test analysis test acceleration techniques test monitoring methods Early Reliability Testing Page 15

16 Correlated generations 7 ERT Early Reliability Test often can provide earlier understanding of causes and mechanisms for defects, wear, fatigue and failure that otherwise would degrade the final generation Early Reliability Testing Page 16

17 Correlated generations 8 Also, ERT test enables much more lead time for reliability work longer test runs milder acceleration easier extrapolation minimized schedule-driven compromises easier and smarter follow-on engineering Early Reliability Testing Page 17

18 Correlated generations 9 Most reliability test investment is for up front test development To design, construct, program the test apparatus, fixtures, software To develop acceleration techniques To develop monitoring instruments (Also at ASTR07, please see our presentation on Leading Indicators, Life Test... ) To learn to interpret test results Early Reliability Testing Page 18

19 Correlated generations 10 Earlier investments provide physical and intellectual technology assets for successive tests Successive tests can use operating methods and instruments that are already understood, available, automated Successive tests thus can incur relatively little labor even with considerable operating time Early Reliability Testing Page 19

20 Correlated generations 11 Reliability testing on successive generations typically incurs only moderately more investment, compared to reliability testing on just the final generation This moderate extra investment is rewarded by earlier feedback hence better development cycle hence better final product Early Reliability Testing Page 20

21 Challenges to overcome ERT needs to overcome big challenges Immature mfg processes Low test coverage Few samples Immature designs Concurrent / parallel designs (can t test until integrated) Early Reliability Testing Page 21

22 Overcome immature mfg 1 Early generation specimens typically are just a few specimens made with immature manufacturing process. On so few specimens, often we can afford to augment this test with simple examination (e.g.: naked eye or simple microscope) or even Failure Analysis. This enables us: to identify the cause of each failure to exclude failures probably restricted to immature mfg to include failures probably significant for mature mfg Early Reliability Testing Page 22

23 Overcome immature mfg 2 Start test early. Distinguish probably relevant failures versus probably irrelevant failures Relevant failures found early may be little gold nuggets. These may forewarn what could happen with a mature process. Thus overcome fear & paralysis due to irrelevant failures and immature manufacturing. Early Reliability Testing Page 23

24 Overcome immature mfg 3 CASE STUDY Very early during design, we tested Gigabit Fiber Channel Found a tolerance/rubbing issue between the housing and a component near the edge of the board This early feedback facilitated early board re-spun facilitated moving this component. Early Reliability Testing Page 24

25 Overcome immature mfg 4 CASE STUDY Figure 3: Gigabit Fiber Channel Product Early Reliability Testing Page 25

26 Overcome low coverage 1 During early development, in-house test coverage typically is low and specimens are few Therefore test for qualitative (gross) issues and postpone test for quantitative (fine) issues This early test may be very worthwhile May save considerable cost Compared to late learning & late engineering Such as late board spin or late chassis changes Early Reliability Testing Page 26

27 Overcome low coverage 2 Sometimes, it is allowable to start with commercially available test equipment This temporarily bypasses custom test programs & scripts that won't become available until later. Often this is good enough for worthwhile early test, even though complete coverage is postponed to later phase Early Reliability Testing Page 27

28 Overcome low coverage 3 CASE STUDY We tested an internet appliance product well before the diagnostics were ready. Therefore we just purchased some off-the-shelf software to exercise the memory, hard drive, and a few other components. Coverage wasn t complete Coverage was good enough for useful early feedback Early Reliability Testing Page 28

29 Overcome low coverage 4 Overcoming Low Test Coverage CASE STUDY Figure 4: Internet Appliance Product Early Reliability Testing Page 29

30 Overcome few samples 1 In an early generation, typically only a few specimens are available Therefore engineers previously avoided early testing. Instead, from even a few specimens, we can use a test / analyze / fix method and gain early qualitative feedback Early Reliability Testing Page 30

31 Overcome few samples 2 During early development with few specimens, it is quite feasible & worthwhile to test for qualitative (gross) defects & failures to gain early qualitative feedback and hence to stimulate early engineering changes During later development with numerous specimens, it is also required to test for quantitative (fine) defects and failures to gain quantitative feedback and hence to prove product lifetime There is useful synergism between these two tests Early Reliability Testing Page 31

32 Overcome few samples 3 Typically samples will fail within a fairly tight distribution Therefore Highly Accelerated Life test (HALT) can be used to trade test margins for a size of specimen population Thus even with few samples, HALT test of the outer edge of this distribution will tell about product performance Early Reliability Testing Page 32

33 Overcome few samples 4 CASE STUDY We tested a $100K cooling cabinet with multiple subunits. These were separated and tested as individual subunits. Spares were used only for a a few of the critical subunits rather than as second copy of the entire system. Early Reliability Testing Page 33

34 Overcome few samples 5 CASE STUDY Figure 5: Storage Cabinet separated into subunits Early Reliability Testing Page 34

35 Overcome immature design 1 Again use HALT for early discovery of qualitative design defects. This will accelerate design maturation. The goal of this reliability test is qualitative learning to uncover problems rather than quantitative learning to pass final generation The earlier we test and uncover defects, the more time and money we will save which can be used partially to allow more time and money for later quantitative tests Early Reliability Testing Page 35

36 Overcome immature design 2 During an early generation, often it is more sensible to test the product margins rather than to test for manifest failure and product lifetime In particular, beware of life test on generation P0, because: P0 typically is NOT built with final materials, design, process and thus P0 defects, wear, fatigue and failure may be NOT relevant to later generations Early Reliability Testing Page 36

37 Overcoming immature design 3 CASE STUDY For an electro-mechanical medical device during P0, we knew that life test was premature Instead during P0, we tested margin and characterization to prove design repeatability During P0, this was more feasible and valuable than testing until manifest failure Early Reliability Testing Page 37

38 Overcome immature design 4 CASE STUDY Set up a high-speed camera on the mechanical assembly Tested during hundreds of runs on several products Thus measured repeatability of the mechanical design Early Reliability Testing Page 38

39 Overcoming immature design 5 CASE STUDY Figure 6: Vision System used to capture repeatability Early Reliability Testing Page 39

40 Overcome parallel development 1 Parallel or concurrent design & development Requires mating two or more subunits as prerequisite for meaningful test data This impedes test prior to integration Nevertheless, we still can test earlier, although not as less early as serial development of subunits Early Reliability Testing Page 40

41 Overcome parallel development 2 Start test as soon as subunits are ready for integration Don t wait for SW or diagnostics to be complete. Just make sure you have a way to functionally test unit. Worst case is inability for functional test of two subunits. Nevertheless, once subunits are ready for physical integration, we can test these non-operationally. This less desirable than full-functional test Nevertheless we still can perform vibration tests to find resonant frequencies. This may point out many things, including component interference issues, mounting issues, and board layout issues. Early Reliability Testing Page 41

42 Overcome parallel development 3 CASE STUDY We tested a Neutrino Telescope for the National Science Foundation (NSF) This product could be tested with full functions only if all the pieces were working. Rather than waiting for that, we elected to start with some non-operational testing on some fixtures we custom-designed. Early Reliability Testing Page 42

43 Overcome parallel development 4 CASE STUDY Fig 7: Neutrino Telescope Final Assembly Fig 8: Board Fixture for Telescope Early Reliability Testing Page 43

44 ERT Conclusion 1 Early Reliability Testing (ERT) can be done in parallel with product development by an outside group without depleting the time of the in-house development team. Early Reliability Testing Page 44

45 ERT Conclusion 2 ERT is a development tactic that can enable earlier feedback, smarter engineering, less total cost, lower risk for reliability. Thus ERT offers better final product and better reliability Early Reliability Testing Page 45

How-To Evaluate a Veterinary Digital Radiography System A SPECIAL REPORT

How-To Evaluate a Veterinary Digital Radiography System A SPECIAL REPORT INTRODUCTION The more information you have, the easier decisions are to make. Experiencing a live hands-on demonstration will take