Erik van Veenendaal. Improve Quality Services BV. Onafhankelijk en objectief Innovatief en trendsettend (EU-projects/TU

Size: px

Start display at page:

Download "Erik van Veenendaal. Improve Quality Services BV. Onafhankelijk en objectief Innovatief en trendsettend (EU-projects/TU"

Jodie Davis
6 years ago
Views:

Risk Based Testing in Practice Never speculate on that which can be known for certain Erik van Veenendaal (eve@improveqs.

1 Risk Based Testing in Practice Never speculate on that which can be known for certain Erik van Veenendaal PRISMA 1 Improve Quality Services BV Dienstverlenende organisatie op het gebied van Testen en Kwaliteitsmanagement Advisering, Detachering en Opleidingen Gespecialiseerd en gekwalificeerd (CISA / ISTQB) Onafhankelijk en objectief Innovatief en trendsettend (EU-projects/TU projects/tu-e) Opgericht in jan 98 - ± 30 medewerkers Improve Quality Services BV TestEvent - 1

Erik van Veenendaal Oprichter en DGA Internationaal erkend testepert Co-Auteur Testen volgens TMap Auteur The Testing Practitioner Vice President International Software Testing Qualifications Board

2 Erik van Veenendaal Oprichter en DGA Internationaal erkend testepert Co-Auteur Testen volgens TMap Auteur The Testing Practitioner Vice President International Software Testing Qualifications Board (ISTQB) Vice Chair TMMi Foundation Keynote spreker EuroSTAR en STAREast 3 What is Risk? A factor that could result in a future negative consequence; usually epressed as impact and likelihood (ISTQB Glossary) Testers only have the responsibility to identify the risks and provide information on their status to dare to undertake management attitude and style Improve Quality Services BV TestEvent - 2

3 Testing = Risk Management Objective: most feasible coverage effective usage of limited resources Resources» staffing» infrastructure» time!».. the right level and type of coverage on the right parts at the right time 5 The challenge. if only we knew!! Improve Quality Services BV TestEvent - 3

4 Risk Based Testing Risk identification looks at ways of establishing what the risks are and where they are Risk analysis looks into the critical, comple and potential error prone areas Then we build tests to mitigate the risk Subsequently we track, monitor and report regarding the risks 7 Risk Identification Split up in functional and/or technical items Higher level test according to requirements Lower levels test according to architecture May also be based on a brainstorm session Maimum number of appr. 35 risk items Risk item 1 Risk item 2 Risk item 3 Risk item 4 Functionality Security Functionality Interoperability Improve Quality Services BV TestEvent - 4

5 Risk Analysis Risk = impact likelihood What is the impact for the business? What is the likelihood that there are defects? Determine factors based on previous projects, e.g. defect patterns Likelihood technical risk Impact business risk You You already know this this!! Eercise: Risk Risk Factors 9 Factors From Practice defect patterns / history Likelihood Impact compleity user importance new development ( selling item ) (level of re-uses) financial (or other) interrelationship damage (e.g. safety) (# interfaces) usage intensity size eternal visibility technology cost of rework geographical spread ineperience (of Customization Weightings development team) needed can be be applied Likelihood Improve Quality Services BV TestEvent - 5

6 Stakeholders Involvement Identify Stakeholders Internal (likelihood) and eternal (impact) Assign factors for them to score individually 9 :: Critical Critical 5 :: High High 3 :: Moderate 1 :: Low Low 0 :: None None they shall make choices Item 1 Item 2 Item 3 Item 4 Item 5 User importance Usage intensity Safety 11 Consensus Meeting Discuss issue list - first defects found!! Result may influence development Item 1 Item 2 Item n Compleity 5 2 Likelihood New development 3 1 Interfacing 2 2 Technology 1 1 Eperience level Impact User imprortance Usage intensity 4 3 Safety Improve Quality Services BV TestEvent - 6

7 The Product Risk Matri MoSCoW priorities Likelihood Could Test Must Test focus of X1 development I II testing focus of system X2 III level IV testing 4 Won t Test Should Test Impact 13 Eample Development Testing Statement coverage 20 70% Pair inspection Likelihood 12 I III Formal test specification Boundary Value Analysis Statement coverage 90% Full code inspection II IV Statement coverage 70% Free 4format Impact Improve Quality Services BV TestEvent - 7

8 Eample System Level Testing Use Cases (basic 20 flow) Use Cases (incl. alternatives) Equivalence Partitioning Decision Table Testing I II Likelihood 12 III Use IV Cases (incl. alternatives) Equivalence Partitioning Use Cases 4(basic flow) Impact 15 Differentiated Test Approach!! Reviews & inspection Test design start-up meetings Reviews of test design Eit criteria Level of independence Most eperienced person Priority setting Re-testing Regression testing without this risk management doesn t make much sense!! Improve Quality Services BV TestEvent - 8

9 Practical Guideline Test level Quality Attribute Low Risk Shall be company specific Medium Risk High risk Acceptance test Functionality Security Isolation re-test Basic flow UC Testers Isolation re-test Use cases Testers Full re-test Use cases Domain eperts System test Functionality.. Equivalence Partitioning No testware reviews Equivalence Partioning Review test designs Decision Table testing Review test design 17 Recognize this...? After months of testing the system finally goes live and.. fails Test manager says: we already knew this would happen Who is at fault? Risk based testing = Risk based reporting The major Test deliverable Management Information!!!! Improve Quality Services BV TestEvent - 9

10 Communication Levels No, just can I release!! 19 Risk Based Reporting (1) TS1 TS2 TS3 TS4 TS5 TS6 TS7 TS8 Risk item 1 Risk item 2 Risk item 3 Risk item 4 Risk item Improve Quality Services BV TestEvent - 10

11 Can we release the product? TS1 TS2 TS3 TS4 TS5 TS6 Management view TS7 TS8 Risk item 1 Risk item 2 Risk item 3 Risk item 4 Risk item 5 21 Survey Results average Ease-of of-use 6,5 7 (large σn) it s s simple but not easy median Usefulness 7,6 8 Efficiency 7,4 7 Effectiveness 7, Improve Quality Services BV TestEvent - 11

a risk based differentiated test approach Provide risk based management reporting it

12 Supporting tool 23 Key learning points A structured and practical approach for risk based testing is available Re-discuss the risk assessment on a regular basis Define a risk based differentiated test approach Provide risk based management reporting it doesn t t stop at the planning stage Improve Quality Services BV TestEvent - 12

13 Improve Quality Services BV TestEvent - 13

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING. CP 7026-Software Quality Assurance Unit-I. Part-A

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CP 7026-Software Quality Assurance Unit-I 1. What is quality? 2. What are the Views of Quality? 3. What is Quality Cost? 4. What is Quality Audit? 5. What