Academic excellence for business and the professions How has acceptability of healthcare interventions been defined and assessed? An overview of Systematic Reviews Mandeep Sekhon*, Martin Cartwright, & Jill J Francis UKSBM 10 th Annual Scientific Meeting Nottingham, 4 th December 2014 *PhD Candidate School of Health Sciences
Outline Rationale Methods Results Discussion Next Steps Acknowledgements
Rationale MRC framework - outlines essential and appropriate methods researchers should adopt to design and evaluate complex interventions (MRC, 2000) Evaluations are often undermined by problems of acceptability (Craig et al., 2008)
Rationale In the MRC Guidance reference to acceptability is made 4 times
Research Questions 1) How has acceptability been defined? 2) How has acceptability been assessed? Define 3) Does acceptability relate to theoretical constructs that illustrate the relationship between another variable? E.g. adherence Assess Theorise
RQ: How has the acceptability of healthcare interventions been defined and assessed? Methods Overview of systematic reviews designed to compile evidence from multiple systematic reviews of interventions into one accessible and usable document (Becker & Oxman, 2008 p. 607) Search Strategy 1) Systematic reviews published between May 2000 and January 2014: Acceptability appearing in the title within the Cochrane Database of Systematic Reviews 2) Systematic review filters, combined with acceptab* appearing in the title only within:medline, Embase, PsycINFO,PsycArticles and CINHAL Data Extraction: 3 Main Sections Definitions of Acceptability = Consensus Group Task Assessments of acceptability Whether/ how acceptability has been theorised Analysis Descriptive
Included Eligibility Screening Identification Results: Included Reviews Records identified through database (n = 1,470) Records after duplicates removed (n = 1279) Records screened (n = 1279 ) Full-text articles assessed for eligibility (n = 41) Reviews included in Overview (n = 33) Records excluded (n = 1238) - Not a systematic review or critical synthesis of a healthcare intervention Full-text articles excluded, with reasons (n = 8 ) - Full article not available in English (1) - Quality and acceptability of patient reported outcome measures (1) - Social validity of treatment acceptability measures (1) - Acceptability of sampling methods (1) - Duplicate review articles (n=2) - Author opinion reviews (n = 2)
Results: Characteristics of Included Reviews Out of 33 Reviews Sample n Patients 26 Patients and health care providers 2 Health care providers 2 User 1 Public 1 Children 1 Design of Studies n Cross sectional 1 RCTS 25 Mixed methodology 6 qualitative methods (interviews) 1 Quality Assessment n Cochrane 15 Jade Scale 1 CASP 2 Grade 1 Authors judgement 5 not assessed 9
Results: Defining Acceptability Consensus Group Task: Frequencies of 4 or higher needed, to be classed as a definition - Out of 33 Reviews: Conceptual Definition n = 1 Author Intervention Conceptual Definition N Newman (2010) HIV Vaccine HIV acceptability, that is potential users judgements of satisfactoriness of future HIV vaccines and the willingness to use them Operational Definitions n = 20 Examples Author Intervention Operational Definition n Allen (2010) HPC Vaccine Items were categorised as vaccine acceptability if they included the word 'would' as in 'would you agree' to vaccination or would you accept the vaccine No consensus in the literature on a definition of acceptability Hanwella (2011) Drugs (ADHD) All cause discontinuation was used as a measure of acceptability 7 Ciprini (2011) Drugs (Anti-manic) Treatment discontinuation (acceptability) was defined as the number of patients who left the study early for any reason during the first 2 weeks of treatment of the total number of patients randomly assigned to each treatment group 5 7 7 Lewis (2012) Self help intervention anxiety disorders Acceptability was assessed in terms of any formalised measure of satisfaction 7 Andrews (2010) Computer therapy (Anxiety and depressive disorders) measures: acceptability to participants (percent adherent to full course, percent satisfied) Adherence and patient satisfaction are indicators of acceptability of computerised CBT to patients 6 Skapinakis (2010) Drugs (depression in Parkinson's) we also measured the total number of dropouts in each arm to assess the acceptability 6
Results: Assessing Acceptability Measures of observed behaviour n Drop-out rates 7 All cause discontinuation rates 5 withdrawal rates 2 discontinuation and removal rate 1 Discontinuation, attrition, adherence, non compliance 1 Willingness to participate / declining to participate 1 rates of uptake, adherence, completion of exercise 1 uptake 1 side effects 1 healthcare interventions total n of reviews 20 Behavioural and Self report measures n Adherence and satisfaction 2 Dropout rates, take up rates, surveys ( cognitive reasons for dropping out and satisfaction 1 measure) Wound healing time, acceptability and satisfaction measure 1 dropout rates, reasons for discontinuation 1 response rates, patients perspectives and user experience 1 total n of reviews 6 Self report measures Surveys (hypothetical scenarios) 2 Satisfaction measure 1 Attitude 1 Perceptions/ experience 1 Perceptions/ experience and attitudes No formal validated measure applied to assess acceptability of open ended questions 1 total n of reviews 7 None of the reviews provided pre-specified criteria for judging the acceptability of the intervention Delivery of measure (relative to start of Intervention) Pre intervention 1 During intervention 6 Post intervention 16 Not applicable 4 Unclear 1 not reported 5 total n of reviews 33 n 1 n
Results: Has theory of acceptability been reported? Evidence from this Overview indicates that at the review level: No theory applied to definitions of acceptability No theory applied to measures of acceptability
Discussion There is a gap within the literature No consensus on a definition or assessment of acceptability Limitations Reviews in our sample do not capture all studies on acceptability BUT Possible that if a study has assessed acceptability very likely to be in one of the included systematic reviews Systematic review filters applied Most feasible way of answering the research question
Discussion MRC Framework advises to consider acceptability within the piloting stage of an intervention e.g. before a trial Researchers follow guidelines (e.g. CONSORT) for reporting a set criteria of outcomes at the end of a trial e.g. number of dropouts, attrition rates Questions to consider: 1. Assess acceptability pre- or post-intervention? 2. Differences between prospective and retrospective acceptability? 3. Experiential vs. hypothetical acceptability?
Next Steps Theorise Develop theory Use theory to create a measure of acceptability Two steps: 1) Search primary studies of review papers assessment approaches (e.g. questionnaires, interviews) Hypothesize number of items will be small, as literature is immature 2) What other theories or components of theories can be applied to understand acceptability? Consider items in theory based questionnaires?
Thank you & Acknowledgements Professor Jill Francis, City University London Dr. Martin Cartwright, City University London Research Group, School of Health Sciences, City University London E-mail: Mandeep.Sekhon.1@city.ac.uk Twitter: @Mandy_sekhon
References Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ: British Medical Journal, 321(7262), 694. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj, 337. Newman, P. A., & Logie, C. (2010). HIV vaccine acceptability: A systematic review and metaanalysis. AIDS, 24(11), 1749-1756. Allen, J. D., Coronado, G. D., Williams, R. S., Glenn, B., Escoffery, C., Fernandez, M.,... & Mullen, P. D. (2010). A systematic review of measures used in studies of human papillomavirus (HPV) vaccine acceptability. Vaccine, 28(24), 4027-4037. Hanwella, R., Senanayake, M., & de Silva, V. (2011). Comparative efficacy and acceptability of methylphenidate and atomoxetine in treatment of attention deficit hyperactivity disorder in children and adolescents: a meta-analysis. BMC psychiatry, 11(1), 176. Lewis, C., Pearce, J., & Bisson, J. I. (2012). Efficacy, cost-effectiveness and acceptability of self-help interventions for anxiety disorders: systematic review. The British journal of psychiatry, 200(1), 15-21. Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010). Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PloS one, 5(10), e13196. Skapinakis, P., Bakola, E., Salanti, G., Lewis, G., Kyritsis, A. P., & Mavreas, V. (2010). Efficacy and acceptability of selective serotonin reuptake inhibitors for the treatment of depression in Parkinson's disease: a systematic review and meta-analysis of randomized controlled trials. BMC neurology, 10(1), 49.