2. Are data reliable and complete?

Transcription

1 2. Are data reliable and complete? Tom Hiatt WHO/HQ/Stop TB TB monitoring and evaluation unit

2

3 Data quality: what is it? 1. Data Quality: The Accuracy Dimension Jack E. Olson "Data has quality if it satisfies the requirements of its intended use. It lacks quality to the extent that it does not satisfy the requirement. In other words, data quality depends as much on the intended use as it does on the data itself. To satisfy the intended use, the data must be accurate, timely, relevant, complete, understood, and trusted." Definition of Data Quality US Census Bureau "Fitness for use. To be fit for use, data products must possess all three attributes of quality defined by the Office of Management and Budget: 1) Utility - refers to the usefulness of the information for its intended users. 2) Objectivity - refers to whether information is accurate, reliable, and unbiased, and is presented in an accurate, clear, and unbiased manner. 3) Integrity - refers to the security or protection of information from unauthorized access or revision."

4 Intended use of TB notification data Estimate of TB incidence Estimate of trend in TB incidence over time

5 Afghanistan Bahrain Djibouti Egypt Iran Iraq Jordan Kuwait Lebanon Libya Morocco Oman Pakistan Qatar Saudi Arabia 15 Somalia Sudan Syria Tunisia UAE WB and GS Yemen Saudi Arabia, Tunisia, Egypt, Iraq, Libya, Sudan, Oman, Kuwait, Pakistan, Somalia, Syria, UAE, Djibouti, Lebanon, Morocco, Yemen, Bahrain, Jordan, Afghanistan, Qatar, West Bank and Gaza, Iran

6 2.1 Are TB data complete? Do the nationally aggregated TB notifications include all the data/reports from the reporting units that were expected to report to NTP? Were any notification reports missing from the lowest admin levels at any time? 2.2 Are TB data reliable? Are reported TB cases actually TB cases? Are TB cases classified correctly? e.g. new cases are not classified as re-treatment or vice versa. Or smear unknown cases are not classified as smear-negative.

7 Assessing quality: system vs. data System (M&E, HIS) Data M & E strengthening tool» (MEASURE Evaluation) Performance of Routine HIS tool» (JSI) HIS assessment tool» (Health Metrics Network) Record reviews (DQA tools by Measure and GAVI) Analytical approaches Population based data Commodities Models Patterns and trends

8 In god we trust, all others bring data! Rex Davenport

9 Comparison of reports received versus expected & 2.1.2: Do you have a system to monitor the completeness of reporting from admin 1 to national level? If yes, since when)?

10 Comparison of reports received versus expected & 2.1.4: Do you have a system to monitor the completeness of reporting from admin 2 to admin 1 level? If yes, since when? & 2.1.6: Do you have a system to monitor the completeness of reporting from admin 3 to admin 2 level? If yes, since when?

11 Identification of unusual fluctuations 2.1.7: Were there unusual fluctuations in the time series? Pakistan Iraq??? Egypt? Djibouti

12 Identification of unusual fluctuations 2.1.7: Were there unusual fluctuations in the time series?

13 Identification of unusual fluctuations 2.1.8: If yes, can you list the reasons for these unusual fluctuations in time? Sudden improvements or disruptions in the recording and reporting system Inclusion of data from new reporting units Sudden changes in TB diagnostic capacity Changes in notified case definitions Don't know Other. Please specify.

14 Identification of unusual fluctuations Do you think that the fluctuations were driven by a certain case type?

15 Identification of unusual fluctuations Do you think that the fluctuations were driven by a certain case type? Yes, it was manly driven by fluctuations in the number of SS+ pulmonary TB cases Yes, it was manly driven by fluctuations in the number of SS- pulmonary TB cases Yes, it was manly driven by fluctuations in the number of extra-pulmonary TB cases No, I don't believe the fluctuations were driven by a certain case type. Don't know Other. Please specify.

16 Variation in notification rates of new TB cases across admin Is there a lot of variation between notification rates of new (all and by smear) TB cases across admin1? ~ 140 ~ 0

17 Variation in notification rates of new TB cases across admin If yes, what are the main reasons to explain this variation? True differences in TB epidemic sub-nationally Differences in TB diagnostic capacity Differences in the recording and reporting system Don't know Other. Please specify.

18 Variation in notification rates of new TB cases across admin Do you think that the fluctuations found for the national data were driven by certain admin 1 areas?

19 Comparison notifications workshop template versus reported in WHO TB database Was there a difference between the number of notifications reported in the workshop template and those reported in the WHO Global TB database?

20 Comparison notifications workshop template versus reported in WHO TB database If yes, can you list the reasons for this difference? Case definition understood as different in each database Inclusion of reports that arrived late Don't know Other. Please specify.

21 Comparison national and subnational notification data in the workshop template & : Was there a difference between the national aggregated data with the subnational data reported in the workshop template? If yes, can you explain the reasons for this difference?

22 2.2 Are TB data reliable? Are reported TB cases actually TB cases? Are cases classified correctly?

23 Relapse Retreat / National Pulm New SS+ Pulm / / New All /

24 A - Proportion of all TB cases that are new In the last year, how does the proportion compare with the global and regional average? / New All

25 A - Proportion of all TB cases that are new If the proportion is higher or smaller than the global average, how do you explain this? Factors that interfere with the number of retreatment cases, including differences in risk factors, TB control efforts, proportion of drug-resistant TB Misclassification problems (i.e. retreatment cases classified as new cases) Don't know Other causes - please specify / New All

26 A - Proportion of all TB cases that are new & 2.2.4: Are there significant variations over time? If yes, how do you explain these variations over time? Variations in the factors that interfere with the number of retreatment cases, including TB control efforts, proportion of drug-resistant TB Reduction of misclassification problems (i.e. retreatment cases no longer classified as new cases) Don't know Other causes - please specify / New All

27 B - Proportion of new cases that are pulmonary In the last year, how does the proportion compare with the global and regional average? / Pulm New

28 B - Proportion of new cases that are pulmonary If the proportion is higher or smaller than the global average, how do you explain this? Differences in extra-pulmonary TB diagnostic capacity Differences in the age structure of TB cases Differences in HIV prevalence Differences in notification policy or practice Misclassification problems Don't know / Pulm New

29 B - Proportion of new cases that are pulmonary & 2.2.8: Are there significant variations over time? If yes, how do you explain these variations over time? Variations in extra-pulmonary TB diagnostic capacity Variations in the age structure of TB cases Variations in HIV prevalence Variations in notification policy/practice Variations in misclassification problems (introduction of measures to correct the misclassification problem) Don't know Other causes - please specify / Pulm New

30 C - Proportion of all pulmonary cases that are smear positive In the last year, how does the proportion compare with the global and regional average? / SS+ Pulm

31 C - Proportion of all pulmonary cases that are smear positive / SS+ Pulm

32 C - Proportion of all pulmonary cases that are smear positive If the proportion is higher or smaller than the global average, how do you explain this? Differences in capacity to perform smear examination Differences in the age structure of TB cases Differences in HIV prevalence Differences in notification policy or practice Misclassification problems Don't know Other causes - please specify / SS+ Pulm

33 C - Proportion of all pulmonary cases that are smear positive & : Are there significant variations over time? If yes, how do you explain these variations over time? You can select more than one. Variations in diagnostic capacity for smear positive cases Variations in the age structure of TB cases Variations in HIV prevalence Variations in notification policy/practice Variations in misclassification problems (introduction of measures to correct the misclassification problem) Don't know Other causes - please specify / SS+ Pulm

34 D - Proportion of all re-treatment cases that are 1) relapse, 2) treatment-after-failure, 3) treatment-after-default 4) other retreatment In the last year, which of the categories contributed most to the total number of retreatment cases? Relapse Treatment-after-failure Treatment-after-default Other re-treatment / Relapse Retreat

35 D - Proportion of all re-treatment cases that are 1) relapse, 2) treatment-after-failure, 3) treatment-after-default 4) other retreatment & Where there significant changes over time in the contribution of each of the categories to the total number of retreatment cases? If yes, how do you explain these changes over time? Variations in the factors that drive the TB epidemic, including TB control efforts and TB treatment regiments Variations in the prevalence of drug-resistance TB Variations in notification policy/practice in these categories over time Variations in the amount of misclassification between the categories over time Don't know Other causes - please specify

36 Relapse Retreat / Subnational New All Pulm New SS+ Pulm / / /

37 A - Proportion of all TB cases that are new In the last year, how do the proportions for each of the admin 1 level areas compare with the national average? / New All

38 A - Proportion of all TB cases that are new If you identified a big variation, how do you explain that? Differences in factors that drive the TB epidemic across admin 1 areas, including risk factors, TB control efforts, proportion of drug-resistant TB Misclassification problems across admin 1 areas (i.e. retreatment cases wrongly classified as new cases) Don't know Other causes - please specify / New All

39 B - Proportion of new cases that are pulmonary In the last year, how do the proportions for each of the admin 1 level areas compare with the national average? / Pulm New

40 B - Proportion of new cases that are pulmonary If you identified a big variation, how do you explain that? Differences in extra-pulmonary diagnostic capacity across admin 1 areas Differences in the age structure of TB cases across admin 1 areas Differences in HIV prevalence across admin 1 areas Differences in the notification policy or practice across admin 1 areas Differences in misclassification problems across admin 1 areas Don't know Other causes - please specify / Pulm New

41 C - Proportion of all pulmonary cases that are smear positive In the last year, how do the proportions for each of the admin 1 level areas compare with the national average? / SS+ Pulm

42 C - Proportion of all pulmonary cases that are smear positive If you identified a big variation, how do you explain that? Differences in capacity to perform smear examination across admin 1 areas Differences in the age structure of TB cases across admin 1 areas Differences in HIV prevalence across admin 1 areas Differences in the notification policy or practice across admin 1 areas Differences in misclassification problems across admin 1 areas Don't know Other causes - please specify / SS+ Pulm

43 2.3 Do you have data on TB- HIV co-morbidity cases?

44 2.3 Do you have data on TB-HIV co-morbidity cases? & 2.3.2: Is there a national TB-HIV surveillance system? If yes, since when? Yes, data on the results of HIV testing of TB patients is collected as part of the main TB surveillance system Yes, data on the results of HIV testing of TB patients is collected in a parallel system (e.g. HIV sentinel surveillance system) No, there is no system to record results of HIV testing of TB patients Don't know

45 2.3 Do you have data on TB-HIV co-morbidity cases? 2.3.3: If yes, have there been variations in the proportion of registered TB patients with HIV+ results over the last 5 years? Yes, and they are mainly due to real changes in the proportion of coinfected patients Yes, and they are mainly due to changes in the proportion of TB patients that are tested for HIV Yes, and they are mainly due to changes in the recording of this information in the system Yes, and they are due to a combination of the above mentioned causes No, the proportion has not varied much Don't know

46 2.3 Do you have data on TB-HIV co-morbidity cases? & 2.3.5: Have you ever done a national survey for the prevalence of HIV positive patients among a representative sample of your registered TB patients? What was the result of your last national survey? Yes, one survey Yes, more than one survey No Don't know

47 2.4 Do you have data on MDR-TB cases?

48 2.4 Do you have data on MDR-TB cases? & 2.4.2: Is there a national MDR-TB surveillance system? If yes, since when? Yes, data on MDR-TB patients is collected as part of the main TB surveillance system Yes, data on MDR-TB patients is collected in a parallel system No, there is no system to record MDR-TB patients data Don't know

49 2.4 Do you have data on MDR-TB cases? 2.4.3: If yes, have there been variations in the proportion of registered TB patients that have MDR-TB over the last 5 years? Yes, and they are mainly due to real changes in the proportion of MDR-TB patients Yes, and they are mainly due to changes in the proportion of TB patients that have access to culture and/or drug sensitivity testing Yes, and they are mainly due to changes in the recording of this information in the system Yes, and they are due to a combination of the above mentioned causes No, the proportion has not varied much Don't know

50 2.4.4 & 2.4.5: Have you ever done a national survey for the prevalence of MDR-TB patients among a representative sample of your registered TB patients? What was the result of your last national survey? Yes, one survey Yes, more than one survey No 2.4 Do you have data on MDR-TB cases? Don't know

51 How useful are the data? Continue on Thank you!