Development of Manuals of Operating Procedures

Transcription

1 Development of Manuals of Operating Procedures Wendy Weber, ND, PhD, MPH (Laura Lee Johnson, Ph.D.) NIH Introduction to the Principles and Practice of Clinical Research January 28,

2 Objectives Define several parts of a manual of operating procedures document Identify resources to assist in development of study designs and documents 2

3 Application vs. Protocol vs. MOP Application NIH limits 6-12 pages, concise justification and design of clinical study Protocol Detailed plan of the clinical study (no page limit) Manual of Operations and Procedures (MOP) Finely detailed information describing the conduct and operation of a clinical study 3

4 Protocols and Manuals of Procedures Studies vary Single detailed protocol Protocol + Manual of Procedures (MOP) Multi-site studies Different institutions may have varying protocol requirements Using study MOP very useful Complicated or long procedures: MOP may be more useful Documents have to all agree! 4

5 Manual of Operations and Procedures (MOP) Operationalize your Protocol Can another investigator step into your study (or reproduce it) at any time? Can anyone you might hire for any position follow entire document and run study? Could study continue if you lose key staff? Data and Safety Monitoring Activities 5

6 6

7 7

8 Manual of Operations and Procedures (MOP) Transforms Study Protocol Guidelines Study organization, operational definitions Training plan, communications plan Outline screening and eligibility criteria Participant recruitment, screening, enrollment, randomization, retention Case report forms, data collection forms Data collection methods, data flow, quality control, SOPs, data & statistical analyses 8

9 Examples of Chapters in MOP Overview Recruitment Eligibility General procedures Informed consent Screening Randomization Follow-up visits Retention Intervention ECG Blood collection Physical assessment Fitness testing DXA scanning Health events Participant safety Adverse events Data management Quality control Interviewing Study organization Study website 9

10 Subject Recruitment aka The Facts of Life Early estimates unrealistically high Takes a major effort People presumed eligible for study during planning will disappear mysteriously as soon as the study starts Recruitment will be more difficult, cost more, and take longer than planned 10

11 11

12 Prepare! Collect reliable data to estimate participant availability If matching, allow for screening twice as many controls per discrete variable matched upon Decide on general recruitment approach Outline steps in recruitment process Establish necessary recruitment contacts 12

13 Recruitment Mistakes/Problems Competing with private physicians for patients Providing basic care rather than referring patient back to primary care physician Failure to maintain adequate contact with referring physician 13

14 Recruitment Mistakes/Problems Attempting recruitment without the support of colleagues Taking access to medical records for granted Failing to secure enthusiasm and commitment of staff Inadequate publicity 14

15 Recruitment Mistakes/Problems Unnecessary inclusion/exclusion criteria Overly restrictive Study plans made without reliable data Pilot recruitment verify participant availability Recruitment timetable Unrealistic goals Specify recruitment metrics Sample size calculations Make them consistent with recruitment realities 15

16 Recruitment: Why So Hard? Early estimates unrealistically high Requires monumental effort Potential recruits presumed eligible for study during planning Disappear mysteriously when study starts Inadequate publicity 5 passes More difficult, more costly, takes more time than planned 16

17 Examples of Chapters in MOP Overview Recruitment Eligibility General procedures Informed consent Screening Randomization Follow-up visits Retention Intervention ECG Blood collection Physical assessment Fitness testing DXA scanning Health events Participant safety Adverse events Data management Quality control Interviewing Study organization Study website 17

18 Specifics of Interventions Training Handouts Props Scripts for behavioral interventions Detailed drawings of movements, exercises to be performed 18

19 Document Uniformity of Intervention Important elements of the intervention Training program for instructors Instructor training programs, retreats, internships, practicums Internal study training What will be done by whom & when in each session Time for each part of each session Tailoring allowed and how Handouts 19

20 Control usually costs money = larger sample sizes Control everything except the smallest element of the intervention that you want to test Big effect on outcome collected in order to see a statistically significant difference between groups Consequences: smaller the difference in outcomes between the non-t and treatment conditions More control you impose Less sensitive or precise the outcome measure 20

21 No placebo/control = problems Patients tend to do better by receiving some treatment, even placebo or standard of care/treatment as usual (SOC/TAU) Comparing a patient on treatment to baseline does not take this into account 21

22 Additional Problems Researchers tend to interpret findings in favor of the new treatment Investigator/participant bias Impossible to distinguish the effect of time from treatment effects Confounding 22

23 What is the Right Control Group? Waitlist control Placebo control Remove active components May still have an impact on outcomes May be called Attention control Active controls Standard of care Alternative intervention Fewer parts of multi-part intervention Different study population (eg, case-control) No universally-appropriate control group 23

24 Placebo, Standard of Care, and Attention Controls 8-week meditation class with monthly booster classes If for weight loss maintenance Weight Watchers? Health Education Program? Yoga A form of exercise or stretching? Cooking classes? Book club? 24

25 Examples of Chapters in MOP Overview Recruitment Eligibility General procedures Informed consent Screening Randomization Follow-up visits Retention Intervention ECG Blood collection Physical assessment Fitness testing DXA scanning Health events Participant safety Adverse events Data management Quality control Interviewing Study organization Study website 25

26 Specifics of Laboratory Methods Enzyme determinations: laboratory methods? Chest x-rays PA and/or lateral Supine or erect Clinical measurements Blood Pressure (BP) Supine or standing How many times? Average? Rest periods between measurements? Feet on floor? Arm is where? Cuff size? Heart sounds in left lateral decubitus 26

27 Collecting Data on People: Case Report Forms (CRFs) Paper ecrf Transcribed from paper Direct entry SMS Computer Try to avoid open ended questions Providing answer choices to questions allows more efficient summarization 27

28 CRFs Do Design the form to collect relevant data In accord with the protocol Facilitate efficient and complete data collection, processing, and analysis Facilitate pooling data across studies Provide units of measure (e.g. pounds, kilos) Include instructions for completion, participant ID number, Study name and contact information 28

29 Spreadsheet From Hell 29

30 Much Nicer 30

31 Examples of Chapters in MOP Overview Recruitment Eligibility General procedures Informed consent Screening Randomization Follow-up visits Retention Intervention ECG Blood collection Physical assessment Fitness testing DXA scanning Health events Participant safety Adverse events Data management Quality control Interviewing Study organization Study website 31

32 Masking/Blinding Specify whom to be masked, why, how, and to what Assess effectiveness of masking Specify criteria for unmasking, whom to be unmasked Mask determination of outcome so that reviewers are unaware of treatment assignment; provide information on "need to know" basis 32

33 Blinding & Random Allocation Clarify the blinded personnel Endpoint assessment Who allocates people to a study group? Exactly what mechanism are they using? Stratifying or balancing or matching? Data verified before assignment Lots of programming and oversight time When exactly are study participants randomized/allocated? 33

34 Randomization Reminders Randomizing too early = Trouble Not actually randomizing (when the study design states it is a randomized study) = Worse Basic principles are important Randomize to minimize selection bias If the method of randomization used in a study does not accomplish this task or can easily be undermined, there is a problem 34

35 Interim Analyses After every 50 in each arm completed 2 cycles (or had an event?) 50% of events have occurred 75% of events have occurred Something else Efficacy, Futility, Safety, anything else? As the study goes on this might change based on the Data Monitoring Committee This also impacts the Sample Size 35

36 Informed Consent: Approach Find proper setting: quiet, private Provide adequate time Encourage potential participant to discuss with others (family members, physician), ask questions Ensure participant's competence to give consent Provide copy of signed consent In unblinded studies, must be willing to participate regardless of random assignment 36

37 Mistakes in Consent Process Inadequate time Failure to specify required procedures Inadequate documentation Vague or inaccurate statements Making commitments that cannot be met Use of untruths to protect study design Consent after the fact Speaking for the patient ("I understand that...") 37

38 Commonly Missing Study Design Elements Choosing the right dose Designing the right control group(s) Information on randomization, blinding, data capture, quality control 38

39 Consistency Across Documents Matters Protocol Manual of Procedures Informed Consent Form Data and Safety Monitoring Plan Case Report Forms Database Responses to comments from funder, regulatory agencies, IRB/IACUC/IBC/SRC, etc 39

40 Write the MOP So anyone you might hire for any position can follow then entire document and run the study Undergrad, post doc, statistician, anyone Vital for multi-site studies 40

41 Document Changes & Problems Changes in procedures necessary Changes in inclusion or exclusion Changes in data collection procedures Revisions as needed, dated, with replacement pages in MOP; versions of protocol Document if find Drift in measurements Change in health and treatment patterns or practices within the community 41

42 Conclusion Make sense Statistical methods and sample size flow with the rest of the proposal Correctly staffed Implementable Practical Rigorous Testable hypotheses that address aims and goals of the research 42

43 Questions 43

44 NIH Resources Templates for Protocols, Manual of Operating Procedures, Data Safety Monitoring Plans, Regulatory Binder Documents, Informed Consents, DSMB Charters & Conflict of Interest NIAID Clinical Research Toolbox (templates) toolkit/pages/default.aspx NCCAM Clinical Research Toolbox (templates) NIDCR Toolkit for Clinical Researchers 44

45 Resources: General Books Hulley et al (2001) Designing Clinical Research, 2 nd ed. LWW Rosenthal (2006) Struck by Lightning: The curious world of probabilities Bland (2000) An Introduction to Medical Statistics, 3rd. ed. Oxford University Press Armitage, Berry and Matthews (2002) Statistical Methods in Medical Research, 4th ed. Blackwell, Oxford 45

46 More Books Statistical Reasoning in Medicine: The Intuitive P-Value Primer by Lemuel Moye Designing Clinical Research: An Epidemiologic Approach, edited by Stephen Hulley Critical Appraisal of Epidemiological Studies and Clinical Trials by Mark Elwood 46

47 And More Books Data Monitoring Committees in Clinical Trials: A Practical Perspective by Ellenberg, Fleming, DeMets. Fundamentals of Clinical Trials by Friedman, Furberg, DeMets The Statistical Evaluation of Medical Tests for Classification and Prediction by Margaret Sullivan Pepe 47

48 Resources: Articles Simon R. Optimal two-stage designs for phase II clinical trials. Controlled Clinical Trials. 10:1-10, Thall, Simon, Ellenberg. A two-stage design for choosing among several experimental treatments and a control in clinical trials. Biometrics. 45(2): ,

49 Regulatory Guidances ICH E9 Statistical principles ICH E10: Choice of control group and related issues ICH E4: Dose response ICH E8: General considerations US FDA guidance and draft guidance on drug interaction study designs (and analyses), Bayesian methods, etc. ndustry/ucm htm 49

50 REDCap Research Electronic Data Capture Started by Vanderbilt University 450+ active institutional partners (US+) Secure web application Build and manage online surveys and databases quickly and securely projects 50

51 AssessmentCenter Online research management tool Create study-specific websites Capture participant data securely PROMIS instruments (short forms, CATs, profiles) are a central feature Automated accrual reports Capture endorsement of online consent 51

52 Resources: URLs Sample size calculations simplified Statistics Guide for Research Grant Applicants, St. George s University of London www-users.york.ac.uk/~mb55/guide/guide.htm Software: nquery, EpiTable, SeqTrial, PS ( n/powersamplesize) 52