Validity of Self-reported Periodontal disease: a systematic review

Transcription

1 Validity of Self-reported Periodontal disease: a systematic review Research protocol July 2015 Hadeel M. Abbood 1 Dr. Tatiana V. Macfarlane 1 Dr. George Cherukara 1 1 University of Aberdeen, Aberdeen Dental School and Hospital, AB25 2ZD

2 Table of contents Introduction... 3 Rationale: Why it is important... 3 Aim... 4 Objectives... 4 Method... 4 STUDY SELECTION... 4 SEARCH STRATEGY... 4 INCLUSION AND EXCLUSION CRITERIA... 4 DATA EXTRACTION... 5 QUALITY ASSESSMENT... 5 Milestone... 6 References:... 7 Table of appendixes Appendix 1 Map of searching terms in MEDLINE... 9 Appendix 2 Flow diagram of study selection process Appendix3 Data extraction form Appendix 4 Quality assessment tool (AMSTAR) Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy

3 Introduction Periodontal disease is a group of diseases of the periodontal tissues that lead to attachment loss and alveolar bone destruction. The prognosis of periodontal disease in some but not all patients may results in tooth loss, especially if there is no treatment (Löe et al. 1986). Diagnosis depends on the findings from the history and clinical examination of the patient (Clerehugh, Tugnait & Genco 2013), which can includes several separate indices. These indices include Gingival Index (GI), Plaque Index (PLI), bleeding on probing (BOP), loss of attachment (LOA), furcation involvement, and tooth mobility. The information gathered will allow the clinician to decide on a logical basis whether additional special tests are required, such as radiographic examination and tests to detect biomarkers in the saliva and the gingival crevicular fluid. (Chapple, Gilbert 2002). Self-reporting is easy to apply, cost-effective, and provides immediate results. It can be relied upon for national public health surveys (Tourangeau, Yan 2007). Self-report has gained popularity as an important method in screening and motivational interviewing (Lundahl, Burke 2009). Self-rated health assessment and utilization of healthcare services are important determinants of health, and have particular relevance for public health (Chakraborty et al. 2003). Self-report is a potent, skilled, and accepted means of assessing many diseases, such as assessing cancer, cardiovascular disease (Newell et al. 1999), and juvenile rheumatoid arthritis (Wright et al. 1994). It can also be used to assess risk factors for disease, such as diet (Willett 1990, Rimm et al. 1992), physical activity (Wolf et al. 1994), high blood pressure (Tormo et al. 2000), and general health (Sheridan, Mulhern & Martin 1998). An example for self-reported questionnaires survey is the Queensland preventive health surveys that are designed to make the respondent self-assess their own health (Queensland Government 2015). The Scottish health survey (SHeS) is another example of self-reported health. It provides a detailed picture of the health of the Scottish population in private households. It is designed to make a major contribution to the monitoring of health in Scotland. SHeS also monitor oral health, especially the periodontal health and the oral hygiene using a self-reported questionnaire (the Scottish Government 2015). Rationale: Why it is important For public health interventions against periodontal disease to be developed, implemented, and evaluated, an adequate monitoring of the disease is required. Clinical assessment of periodontal disease is extremely need more resources and facilities, and cannot be used in several state-based surveillance systems. The use of valid, economical, and low-resource selfreported measures of periodontal disease would be of great benefit in a variety of ways. Firstly, it would aid in studying periodontal disease epidemiologically at a larger scale than is feasible with the present clinical measures, because using surveys can incorporate a larger population than clinical examination. Secondly, questions regarding periodontal disease could easily be added to ongoing studies to evaluate associations with other diseases and conditions. The use of self-report would allow for an easier and low-cost method of obtaining data for research and would support the evolution of oral health programs (Siegal, Martin & Kuthy 1988, Kallio 1996). Finally, self-reported measures would allow for surveillance of the periodontal condition of populations over time, in national, state, or regional surveillance programs. 3

4 It has been over 10 years since the last systematic review of the validity of self-reported periodontal disease was published (Blicher et al 2005). In this time, several studies using selfreported periodontal disease have been reported. It is the time now to conduct another systematic review, and further assess the validity of self-reported periodontal disease. Such a review will update the previous systematic review by Blicher et al. Aim To update the systematic review published in 2005 by Blicher et al. which reviewed literature up to June (2 nd week) Objectives 1. Summarise new findings and combine them with the previous review by Blicher et al. 2. Identify the most effective means of self-reporting of periodontitis. Method Study selection The research aimed to analyse all the papers that assessed the validity of self-reported gingival and periodontal diseases by comparing with a clinical gold standard. Search strategy MEDLINE search strategy was adopted from review by Blicher et al and will be updated to include studies until Embase database will be added to the electronic search. The search strategy terms will be grouped into three categories. Group I terms will be related to gingivitis and periodontal disease (gingivitis, gingival, gingival disease, periodontal, periodontal disease, periodontitis, tooth mobility, loss of attachment, bleeding gum); group II will include terms associated with self-reporting (questionnaire, self-assessment, self-report, self-reported), and group III terms concerning process of validation (comparison, compared, validity, validation) (see appendix 1). Intra-group terms will be combined with the Boolean commands OR, and Inter-group terms will be combined with AND. Two journals (Journal of Clinical Periodontology and Periodontology 2000) will be searched manually from Publications in languages other than English will not be included. Inclusion and exclusion criteria We will include studies in English language only; any other languages will be excluded. The studies to be included should use the clinical gold standards for each questionnaire. Studies that used self-report but did not validate these measures or validate them in overall or in combination of more than one oral health measure will be excluded. 4

5 Data extraction After completing the database search and manual search, two independent reviewers will screen the title and the abstract of each study following the inclusion criteria. If disagreement occurs between the two reviewers, a third reviewer will be asked consulted. Data extraction will be done by two independent reviewers. They will extract the required data from each study. These information include: study type, population description (sample size, mean age, gender, method of recruitment), question used for self-report, clinical gold standard, and time of data collection. A standardised data extraction form will be used for data extraction (see appendix 3). The data extracted will be used to construct a 2*2 table to calculate the sensitivity and specificity. Predictive values will also be used. If some information is missing we will contact the authors of the studies to provide us with the required information. The data from research studies will be used to create tables with group questions asked by subject and contain information regarding specific phrasing, reference gold standard. Validity of self-assessment will be quoted as reported by the authors: percentage of agreement, sensitivity/specificity, p-value, predictive values, correlation/regression coefficient or descriptive methods as reported by the authors, We will calculate an additional statistics based on the data provided in the manuscript when needed and possible. Good validity level will be arbitrarily defined as sum of either: positive and negative values, or specificity and sensitivity, exceeding or equal to 120%. This value was arbitrary chosen by Blicher et al. (2005). If a study compared a self-reported question against multiple clinical measures, only the most relevant one will be chosen. Quality assessment The studies will be analysed according to the guidelines for systematic reviews of diagnostic studies (Devillé et al. 2002). Each publication will be reviewed using specially designed data collection sheets (see appendix 3) with respect to: population profile, selection criteria, type of self-reporting, nature of questions, reference clinical gold standard, and results of validation. Quality assessment tool (QUADAS-2) will be used to evaluate the risk of bias and applicability of primary diagnostic accuracy for each study. QUADAS-2 is designed to assess the quality of primary diagnostic accuracy studies; the tool consists of four key domains regarding patient selection, index test, reference standard, flow and timing. The tool is completed in four phases: 1) state the review question; 2) develop review specific guidance; 3) review the published flow diagram for the primary study or construct a flow diagram if none is reported; 4) judgement of bias and applicability. Each domain is assessed in terms of the risk of bias and the first three are also assessed in terms of concerns regarding applicability. To help reach a judgement on the risk of bias, signalling questions are included. These flag aspects of study design related to the potential for bias and aim to help reviewers make risk of bias judgments. Risk of bias is judged as low, high, or unclear. If all signalling questions for a domain are answered yes then risk of bias can be judged low. If any signalling question is answered no this flags the potential for bias. Review authors then need to use the guidelines developed in phase 2 to judge risk of bias. The unclear category should be used only when insufficient data are reported to permit a judgment. QUADAS-2 is available on the link below: QUADAS-2 is available on the link below: 5

6 Quality assessment tool for systematic review (AMSTAR) will be used to evaluate the previous systematic review by Blicher et al. in yes/no fashion (Shea et al. 2007)(see appendix 4). Study protocol will be registered with PROSPERO We aim to publish this review in a peer reviewed journal. We will follow Prisma guidelines for reporting of systematic reviews We will register our study in systematic review data repository (SRDR) Milestone Protocol finalised - August 2015 Search strategy- August to September 2015 Study selection -September Data extraction September - October Quality assessment-october vember Summary of results vember Writing final report -vember Circulation of report for comment December Submission for publication January

7 References: Blicher, B., Joshipura, K. & Eke, P. 2005, "Validation of self-reported periodontal disease: a systematic review", Journal of dental research, vol. 84, no. 10, pp Chakraborty, N., Islam, M.A., Chowdhury, R.I., Bari, W. & Akhter, H.H. 2003, "Determinants of the use of maternal health services in rural Bangladesh.", Health Promot Int, vol. 18, no. 4, pp Chapple, I.L.C. & Gilbert, A.D. 2002, "Understanding Periodontal Diseases: Assessment and Diagnostic Procedures in Practice." in Quintessence Publishing Co.,, pp Clerehugh, V., Tugnait, A. & Genco, J.R. 2013, Periodontology at a glance, 4th edn, Wiley- Blackwell. Devillé, W.L., Buntinx, F., Bouter, L.M., Montori, V.M., de Vet, H.C., van der Windt, D.A. & Bezemer, P.D. 2002, "Conducting systematic reviews of diagnostic studies: didactic guidelines.", BMC Med Res Methodol, vol. 3, no. 2, pp. 9. Kallio, P. 1996, "Self-assessed bleeding in monitoring gingival health among adolescents.", Community Dent Oral Epidemiol, vol. 24, pp Löe, H., Anerud, A., Boysen, H. & Morrison, E. 1986, "Natural history of periodontal disease in man. Rapid, moderate and no loss of attachment in Sri Lankan laborers 14 to 46 years of age", Journal of Clinical Periodontology, vol. 13, no. 5, pp Lundahl, B. & Burke, B.L. 2009, "The effectiveness and applicability of motivational interviewing: A practice-friendly review of four meta-analyses.", Journal of Clinical Psychology, vol. 65, no. 11, pp Newell, S.A., Girgis, A., Sanson-Fisher, R.W. & Savolainen, N.J. 1999, "The accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease in the general population: a critical review.", American journal of preventive medicine, vol. 17, pp Queensland Government 2015, 12 May, 2015-last update, Queensland preventive health surveys. Available: [2015, 07/22]. Rimm, E.B., Giovannucci, E.L., Stampfer, M.J., Colditz, G.A., Litin, L.B. & Willett, W.C. 1992, "Reproducibility and validity of an expanded self-administered semiquantitative food frequency questionnaire among male health professionals.", American Journal of Epidemiology, vol. 135, pp Shea, B.J., Grimshaw, J.M., Wells, G.A., Boers, M., Andersson, N., Hamel, C., Porter, A.C., Tugwell, P., Moher, D. & Bouter, L.M. 2007, "Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews", BMC Medical Research Methodology, vol. 7, no

8 Sheridan, C.L., Mulhern, M. & Martin, D. 1998, "Validation of a self-report measure of somatic health.", Psychol Rep,, pp Siegal, M.D., Martin, B. & Kuthy, R.A. 1988, "Usefulness of a local oral health survey in program development.", J Public Health Dent, vol. 48, pp the Scottish Government 2015, Wednesday, March 04, 2015-last update, Scottish Health Survey. Available: [2015, 07/22]. Tormo, M.J., Navarro, C., Chirlaque, M.D. & Barber, X. 2000, "Validation of self diagnosis of high blood pressure in a sample of the Spanish EPIC cohort: overall agreement and predictive values. EPIC Group of Spain.", J Epidemiol Community Health, vol. 54, pp Tourangeau, R. & Yan, T. 2007, "Sensitive questions in surveys.", Psychological Bulletin, vol. 133, no. 5, pp Willett, W. 1990, Nutritional epidemiology. Oxford University Press, New York. Wolf, A.M., Hunter, D.J., Coblitz, G.A., Manson, J.E., Stampfer, M.J., Corsano, K.A. & et al. 1994, "Reproducibility and validity of a self-administered physical activity questionnaire.", International journal of epidemiology, vol. 23, pp Wright, F.V., Law, M., Crombie, V., Goldsmith, C.H. & Dent, P. 1994, "Development of a self-report functional status index for juvenile rheumatoid arthritis.", Journal of Rheumatology, vol. 21, pp

9 Appendix 1 Map of searching terms in MEDLINE AND All the groups results will be combined with AND AND Only Group I and II will be combined with AND for more results (some studies do not use the validity terms in their titles) Group III terms regarding validity Group I terms regarding periodontal disease Group II terms regarding self-report 9

10 Included Eligibility Screening Identification Appendix 2 Flow diagram of study selection process Records that will be identified through database searching (n=?) Additional records that will be identified through hand-searching journals (n =?) Records screening (n =?) Excluding records due to language or validity (n =?) Records after screening (n =?) Removal of Duplication (n =?) Full-text articles assessment for eligibility (n =?) Studies that will be included in qualitative synthesis (n =?) Full-text articles excluded (n =?) - PD - self-reported - Neither PD nor selfreported - t looking at validity of selfreporting 10

11 Appendix3 Data extraction form ID: First author s sure name: Title: Country of study: Year of publication Type of study Participants Number: Male Female Age range: Data extraction Self-reported Question Clinical Gold Standard Validity test DOR (Diagnistic odds retio) 11

12 Appendix 4 Quality assessment tool (AMSTAR) 1. Was an a priori design provided? The research question and inclusion criteria should be established before the conduct of the review. 2. Was there duplicate study selection and data extraction? There should be at least two independent data extractors and a consensus procedure for disagreements should be in place. 3. Was a comprehensive literature search performed? At least two electronic sources should be searched. The report must include years and databases used (e.g. Central, EMBASE, and MEDLINE). Key words and/or MESH terms must be stated and where feasible the search strategy should be provided. All searches should be supplemented by consulting current contents, reviews, textbooks, specialized registers, or experts in the particular field of study, and by reviewing the references in the studies found. 4. Was the status of publication (i.e. grey literature) used as an inclusion criterion? The authors should state that they searched for reports regardless of their publication type. The authors should state whether or not they excluded any reports (from the systematic review), based on their publication status, language etc. 5. Was a list of studies (included and excluded) provided? A list of included and excluded studies should be provided. 6. Were the characteristics of the included studies provided? In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes. The ranges of characteristics in all the studies analyzed e.g. age, race, sex, relevant socioeconomic data, disease status, duration, severity, or other diseases should be reported. t applicable t applicable t applicable t applicable t applicable t applicable 7. Was the scientific quality of the included studies assessed and documented? A priori methods of assessment should be provided (e.g., for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo controlled studies, or allocation concealment as inclusion criteria); for other types of studies alternative items will be relevant. t applicable 12

13 8. Was the scientific quality of the included studies used appropriately in formulating conclusions? The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the review, and explicitly stated in formulating recommendations. 9. Were the methods used to combine the findings of studies appropriate? For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity (i.e. Chi-squared test for homogeneity, I²). If heterogeneity exists a random effects model should be used and/or the clinical appropriateness of combining should be taken into consideration (i.e. is it sensible to combine?). 10. Was the likelihood of publication bias assessed? An assessment of publication bias should include a combination of graphical aids (e.g., funnel plot, other available tests) and/or statistical tests (e.g., Egger regression test). 11. Was the conflict of interest stated? Potential sources of support should be clearly acknowledged in both the systematic review and the included studies. t applicable t applicable t applicable t applicable 13

14 Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy Methodology Checklist 5: Studies of Diagnostic Accuracy S I G N This checklist is based on the work of the QUADAS2 team at Bristol Univeristy ( Study identification (Include author, title, reference, year of publication) Guideline topic: Key Question : Before completing this checklist, consider: 1. Is the paper really a study of diagnostic accuracy? It should be comparing a specific diagnostic test against another, and not a general paper or comment on diagnosis. 2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist.. Reason for rejection: Reason for rejection: 1. Paper not relevant to key question 2. Other reason (please specify): Checklist completed by: All the questions in the following sections have associated footnotes providing short explanations behind each of the questions. Users who want more detailed explanations should consult the QUADAS-2: Background Document. DOMAIN 1 PATIENT SELECTION Risk of bias In a well conducted diagnostic study 1.1 A consecutive sequence or random selection of patients is enrolled. i Is that true in this study? 1.2 Case control methods are not used. ii 1.3 Inappropriate exclusions are avoided. iii Applicability 1.4 The included patients and settings match the key question. iv DOMAIN 2 INDEX TEST Risk of bias In a well conducted diagnostic study 2.1 The index test results interpreted without knowledge of the results of the reference standard. v Is that true in this study? 2.2 If a threshold is used, it is pre-specified. vi 14

15 Applicability 2.3 The index test, its conduct, and its interpretation is similar to that used in practice with the target population of the guideline. vii DOMAIN 3 REFERENCE STANDARD Risk of bias In a well conducted diagnostic study 3.1 The reference standard is likely to correctly identify the target condition. viii 3.2 Reference standard results are interpreted without knowledge of the results of the index test. ix Applicability 3.3 The target condition as defined by the reference standard matches that found in the target population of the guideline. x Is that true in this study? DOMAIN 4 FLOW AND TIMING Risk of bias In a well conducted diagnostic study 4.1 There is an appropriate interval between the index test and reference standard. xi Is that true in this study? 4.2 All patients receive the same reference standard. xii 4.3 All patients recruited into the study are included in the analysis. xiii SECTION 5: OVERALL ASSESSMENT OF THE STUDY 5.1 How well was the study done to minimise bias? Code as follows: xiv High quality (++) Acceptable (+) Low quality (-) Unacceptable reject What is your assessment of the applicability of this study to our target population? Directly applicable Some indirectness (Please explain in the following section for tes) 5.2 tes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question. 15

16 i Studies should enrol either all eligible patients suspected of having the target condition during a specified period, or a random sample of those patients. The essential point is that investigators should have no freedom of choice as to which individual patients are or are not included. ii There is evidence that studies comparing patients with known disease with a control group without the condition tend to exaggerate diagnostic accuracy. iii Inappropriate exclusions may result in either overestimates (eg by excluding difficult to diagnose patients) or underestimates (eg by excluding patients with red flags suggesting presence of disease) of the degree of diagnostic accuracy. iv Patients included in the study should match the target population of the guideline in terms of severity of the target condition, demographic features, presence of differential diagnosis or co-morbidity, setting of the study and previous testing protocols. v This is similar to the question of blinding in intervention studies. The index test should always been done first, or by a separate investigator with no knowledge of the outcome of the reference test. vi Bias can be introduced if a threshold level is set after data has been collected. Any minimum threshold should be specified at the start of the trial. vii Variations in test technology, execution, or interpretation (eg use of a higher ultrasound transducer frequency) may affect estimates of diagnostic accuracy. viii Estimates of test accuracy are based on the assumption that the reference standard is 100% sensitive (=accurately diagnoses the target condition). ix This is the similar to question 2.1, but in this case relates to making sure the reference standard is applied without any prior knowledge of the outcome of previous tests. x The definition of the target condition used when testing the reference standard may differ from that used by the NHS in Scotland. eg threshold levels used in laboratory cultures may differ. xi The index test and reference standard should be performed as close together in time as possible, otherwise changes in the patients condition is likely to invalidate the results. xii In some cases the choice of reference standard may be influenced by the outcome of the index test or the urgency of the need for diagnosis. Use of different reference standards is likely to lead to overestimates of both sensitivity and specificity. xiii t including all patients in the analysis may lead to bias as there may be some systematic difference between those lost to follow-up and those analysed. xiv Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (-): Either most criteria not met, or significant flaws relating to key aspects of study design. Conclusions likely to change in the light of further studies. 16