Developing Bioanalytical Methods Balancing the Statistical Tightrope

Size: px

Start display at page:

Download "Developing Bioanalytical Methods Balancing the Statistical Tightrope"

Francis Sims
5 years ago
Views:

1 Developing Bioanalytical Methods Balancing the Statistical Tightrope

2 Lee: can I use this number? Process Development GSK,

3 it s about 40 about 40? probably... 3

4 Enlightenment?

5 5

6 Unconscious Conscious Blooms Taxonomy the 4 stages of competence Incompetent Competent Consciousness 6

7 Me A Statistical God

8 Using Statistics

9 Why? Six Reasons 1. Potency assays are key in making medicines 2. Bioassays are very variable 3. Statistics will help you understand your data 4. Understanding your data will reveal if control exists 5. Your level of control allows you to judge RISK 6. Regulators globally require it 9

10 The Regulator & Assay Control Regulators have been asking for this for years! QbD 1. Pharmaceutical cgmps for the 21st Century 2. PAT 3. ICH Q2: Validation of Analytical Procedures 4. ICH Q8: Pharmaceutical Development 5. ICH Q9: Quality Risk Management 6. ICH Q10: Quality Pharmaceutical Systems 10

11 Statistics The complete solution?

12 Or this? Your assay? 12

13 Or this? or your assay? 13

14 Statistics - an Amazing Transition 14

15 Bioassays will always be variable You can improve it - by understanding it - Focusing effort in right places - This brings control - You can manage expectations - This is understood by regulators 15

16 Why assay variation matters? product variation + assay variation + inaccuracy A few unsatisfactory batches may even pass specification due to a combination of assay method and process variability Many satisfactory OOS batches likely to fail (potentially costing Ms) because of combination of assay method & process inaccuracy & variation 16

17 Our Control Strategy Define What does the scientist need to achieve? i.e. selectivity, accuracy, precision linearity Measure Identify & prioritise analytical CNX parameters Analyse Control parameters Fix & control Noise parameters e.g., MSA, Precision experimental parameters e.g., DoE Regression Input into Improve Method Ruggedness Method Robustness Control Method Control Strategy & reduce Risk prior to Validation Routine Use & Continuous Improvement 17

18 Generating Bioassay Data 18

19 The Rules 1. Speak with your statistician before generating data 2.See Rule 1 19

20 Lot s data Value 20

21 21

22 Statistics are a tool 22

23 QC Which Tools? UCL LCL Stage 4 QC Tools CELLULA, Shewhart chart, CUSUM YES Technology Transfer NO TIME Stage 3: Validation Tools Nested, CELLULA Precision Accuracy Linearity etc. Stage 2: Development Tools DX8, JMP, Minitab Design Stage 1: Qualification Tool Fishbone, Minitab

24 What s Appropriate Knowledge? Learning takes time Will you use it often enough? It s not an academic pursuit Activities must add value do what s necessary 24

25 Scope & Design

26 Define & Scope How is the assay performing? Prec/TOL 2-sided = 6 x =

27 Parameters (e.g. 15) pdna NaCl ph Tube Length Time Seeding Density Ratio of Transfection Temperature Agitation and level Vector type, conc Addition Order

28 Q. How Many parameters? Q. Which parameters? Q. What ranges? A. Existing knowledge A. Common sense A. Practical limits

29 Define & Scope Drill down - map out assay - build understanding & scope Assay Flow 29

30 Define & Scope Drill down & map out assay to build understanding & scope Attention is focused toward key steps and the parameters involved in these steps Cause & Effect Diagram (Fishbone) helps think your assay through Identify & prioritise analytical CNX parameters 30

31 Scope & Screen Scope ranges with simple experiments Scoping Experiments Explore mildest to most forcing conditions 31

32 Revealing the Big Hitters 32

33 Temptation

34 OFAT Building Understanding Provides estimates of effects at set conditions of the other factors and no interaction effects. NaCl ph pdna 34

35 Building Understanding Factorial Design Estimates effects at different conditions to estimate interactions Design of Experiments DOE

36 Optimisation work towards a Robust Optimum Optimise the parameters that survived the initial screening 36

37 Simulations The tools allow you to simulate scenarios using the data you ve built up Visual simulation of expected performance relative to specification 37

38 Is the Model Correct? 38

39 Method stretch what if? Validate & Verify The evaluation of robustness should be considered during the development phase and should show the reliability of an analysis with respect to deliberate variations in method parameters ICH Q2B, 1994 Ideal Settings Control Space Design Space 39

40 Assay Control: control the parameters inside boundaries 40

41 Working within the control boundaries will keep the assay under control Even if you go outside the control boundaries, the assay will have enough flexibility to deal with it without an OOS 41

Summary - Data Driven Development Scope Screen Optimize

into routine use Explore mildest to most forcing conditions

narrow ranges Estimate & utilize interactions to move

42 Summary - Data Driven Development Scope Screen Optimize Verify QC/TT Transfer to QC to validate on batches & bring into routine use Explore mildest to most forcing conditions Identify few potential key parameters Focus on vital few & narrow ranges Estimate & utilize interactions to move towards optimum conditions Rattle the cage to deliver a design space

43 43

44 Precision It may be considered at three levels: 1. Repeatability 2. Intermediate precision 3. Reproducibility ICH Q2A, 1994

45 Repeatability 1 analyst in 1 laboratory on 1 day injecting 6 times Summary Statistics Number of Standard Coefficient Lower 95% CI Upper 95% Values Mean Deviation of Variation for Mean CI for Mean t30 PS %

46 1 analyst in 1 laboratory on 1 day injecting 6 samples each tested 6 times Intermediate Precision As well as sample variation, this study still provides information on repeatability 46

Intermediate Precision So we compare the

47 Intermediate Precision So we compare the mean values for each sample (over replicate results per sample) Variance Components Factor df Variance % Total Sample % Repeat % % Standard Mean Deviation RSD % 47

48 and the others..? Precision within a laboratory but with different analysts, on different days, with different equipment reflects the real conditions within one laboratory ICH Q2A

49 Peak Area Intermediate Precision Data collect using several analysts using several instruments over several days: Y Sample 49

50 Peak Area Intermediate Precision Potentially misleading: large analyst-to-analyst variation present: Y Sample Analyst 1 Analyst 2 Analyst 3 50

51 Intermediate Precision better examined looking at multiple sources of variation within an assay want to understand major sources of variation such as sample, prep, analyst etc. 51

52 Intermediate Precision 52

53 Intermediate Precision Can also perform Unbalanced designs One operator performs multiple injections on single preparation; Two operators perform single injections on multiple preparations 53

54 Reproducibility multiple laboratories; typically run as an interlaboratory cross-over study, with each participating lab sending samples to every other lab and analysing all samples (including own). sent to and analysed by other lab Samples from laboratory: A B C A B C 54

55 Reproducibility Can use analysis of variance (ANOVA) to look for differences or biases between labs Alternatively look for analytical equivalence

56 Risk Management The level of effort, formality and documentation....should be commensurate with the level of risk ICH Q9 Evaluation of the risk to quality should be based on scientific knowledge & ultimately link to the protection of the patient Is the bioassay fit for purpose and under control? 56

57 Before & After How is the assay performing? P/TOL 2-sided = 6 x =

58 Before & After Better P/TOL 2-sided = 6 x =

59 Risk Management Method Understanding, Control and Capability (MUCC) Understand impact of variation upon risk Capable? Risk Management Loop Understanding? Capability & Precision Control? Statistical Process Control (SPC) Charts 59

60 Risk Management Understanding? Capable? Capable? P/TOL 2-sided = 6 x = 1.01 Understanding? Control? Capability & Precision 60

t30 PS Risk Management P/TOL 2-sided = 6 x 6.

I Chart Investigate out-of-control points. Yes 0% 0.

61 t30 PS Risk Management P/TOL 2-sided = 6 x I-MR Chart of t30 PS Summary Report = 0.42 Is the process mean stable? Evaluate the % of out-of-control points. I Chart Investigate out-of-control points. Yes 0% 0.0% > 5% No UCL= _ X= Comments The process mean is stable. No data points are out of control on the I chart Observation LCL=

62 Summary 1.Build a good basic understanding of stats but don t need to become guru 2.Involve a statistician, at least at the beginning 3.Build understanding of your bioassay (QbD) it s a must 4.Get to grips with Bioassay Variability 62

63 Lee: can I use this number? 63

64 Yes it s with 95% Confidence for the statisticians in the audience 64

65 Acknowledgments Dr. Paul Nelson Prism TC Ltd Pictures from The Cartoon Guide to Statistics Larry Gonick & Woollcott Smith 65

Method Development and Validation for Nutraceuticals

White Paper Method Development and Validation for Nutraceuticals Maud Silvent Technical Specialist David Neville Technical Specialist Method Development and Validation for Nutraceuticals Introduction Nutraceutical