Day 2 MER Analytics Exercise PMTCT Data Analysis

Transcription

1 Background It is important to note that reviewing PMTCT data requires more than just comparing results to targets. This exercise will guide the user to review the PMTCT & EID cascade and coverage rates for pregnant women and HIV exposed infants to provide a fuller picture of the progress of the PMTCT program. This will familiarize the user with 1) analyzing PMTCT data at the OU/SNU level and Partner level using Panorama or DATIM; 2) analytic questions to consider when reviewing PMTCT data; 3) shaping follow up analyses, questions for partners, and/or programmatic actions to take based on data review. Questions this exercise will explore: Day 2 MER Analytics Exercise PMTCT Data Analysis What are the key priority questions to ask about coverage and positivity when reviewing the MER PMTCT and EID cascade? Which partners are excelling based on key indicators, such as proxy EID testing coverage? What can we learn from them? Which subnational areas and implementing partners are struggling, and why on STAT coverage, ART coverage, proxy EID coverage? o What should we ask our partners to further understand the context for their performance? o What other data should we review? What are the assumptions, data limitations, and caveats associated with MER indicator PMTCT & EID indicator calculations? E.g. Which partners are over achieving with coverages >100%? Is this due to low proxy denominators, actual performance, or data quality issues? Directions Choose Option 1, 2, or 3 to complete the exercise. 1. Option 1: Complete this exercise using the example country in screenshots contained in this document. There is an answer key for this at the end of the exercise. 2. Option 2: Complete this exercise using your country of choice. However, there is no answer key for this option. Parts of the answer key contained here could help support your analysis. 3. Option 3: Use DATIM pivot table favorites to export data and copy into Excel for further review. Ask the instructor for more information. Materials & Data Access Needed 1) Handout Tip Sheet included at end. Tip sheet is a reference for PMTCT cascade calculations, analysis questions, and list of data assumptions, limitations, and caveats. 2) Data system: For Option 2 above, need Panorama access. For Option 3 above, need DATIM access. Contents Part 1: Assess key PMTCT cascade metrics at Operating Unit (e.g. Country) Level... 2 Part 2: Compare PMTCT Coverage Indicators across Subnational Units (SNUs) and Partners... 3 Part 3. Deciding on next steps... 7 PMTCT Data Analysis Exercise Answer Key... 8 PMTCT Indicator Tips... 9 Priority analytical questions for the PMTCT clinical cascade... 9 Deeper analytical questions... 9 DATIM Favorites to support logic checks... 9 PMTCT MER Indicator Assumptions and Limitations for FY

2 Part 1: Assess key PMTCT cascade metrics at Operating Unit (e.g. Country) Level a. You can use the screenshots provided below to answer the questions. If you choose to review another country in Panorama, start by selecting OU & SNU Analysis >> PMTCT Cascade Visuals. b. Review the PMTCT Indicator Tip Sheet to understand the key cascade calculations for STAT coverage, STAT positivity, ART coverage, and EID testing coverage. Note that the PMTCT Cascade table contains % Coverage and % Positive calculations at the bottom of the table, underneath % Achievement of target. 1. For cumulative FY17 Q3 (FY17Q1+2+3): a. STAT coverage: What percent of pregnant women who attended ANC knew their HIV status? b. STAT positivity: What percent of women who knew their status were positive? 2. For cumulative FY17 Q3, what is the coverage for PMTCT_ART? a. What is the denominator used to determine ART coverage? b. How does PMTCT_ART coverage compare to % achievement of the PMTCT_ART target, and what does each of these calculations tell us about the performance of the PMTCT program? 3. Looking at the % coverages, which indicator(s) might you want to review first to better understand the performance issues? 2

3 Part 2: Compare PMTCT Coverage Indicators across Subnational Units (SNUs) and Partners Go to Partner & Mechanism Analysis > Mech by SNU. Wait for the page to load. Then, under the same menu, select PMTCT Cascade Visuals. Click on the PMTCT Program Coverage view. Select the PSNU/Partner button. The view displayed here is cumulative Fiscal Year 2017 (FY17) data in this case, FY17Q1+Q2+Q3. You can change which quarters are selected under Select Filters. The questions in this exercise focus on cumulative FY17 data. In view to the right, you also see FY17Q4 period selected, but because no data has been submitted yet to the headquarters level, the view does not actually contain Q4 data. 3

4 4. If you see a cell highlighted in this color purple for infant testing coverage what does it mean? Hint: Hover over the Legend button at the top right of the visual. 5. With so much data to review, it can help to prioritize our analysis in some way. One way to do this is to narrow our review to a subset of items, such as reviewing highest volume values. Which SNU and partner have the highest volume of HIV positive pregnant women reported through PMTCT_STAT_POS? Hint: First, hover your cursor over the dark blue header cell of interest. Then, sort in descending order. 4

5 6. Among the priority SNUs with the highest volume, which PSNU partner combinations would you want to further investigate? See sub questions a, b, and c below. a. Which have <90% of pregnant women in ANC with a known HIV status? What are reasons we might see very low performance on this indicator? b. Which have PMTCT_ART coverage <90%? What are reasons we might see lower coverage? Reasons for overly high coverage? c. Which have proxy infant testing coverage <60% by 2 months? What are reasons we might see lower coverage? Which partner(s) would you want to follow up with? 5

6 7. Let s look at this in another way by selecting a different column to sort. In the view below, columns are sorted from high to low % for proxy infant testing coverage by 2 months of age. Why might we see coverage over 100%? Is this a problem? 6

7 Part 3. Deciding on next steps Let s turn our review into follow up actions. 8. How else would you want to explore this data, or use other MER and non MER data sources, to further investigate the issues? 9. What are the data quality and/or program performance discussion points you would like to cover with implementing partners? 10. What questions or comments would you have for your highest achieving partner in EID? 11. What questions or comments would you have for your lowest achieving partners in EID? 7

8 PMTCT Data Analysis Exercise Answer Key 1. A. 94% B. 5% 2. 97% a. PMTCT_STAT_Pos or 43,580 b. Target Achievement 67%. Coverage of 97% is best indication of good program performance. However, should investigate variation at lower subnational levels for potential areas with low coverage. Check target setting assumptions from COP16 to see if target setting can be improved. 3. Proxy infant testing coverage (PMTCT_EID / STAT_POS) 4. EID 12 mo coverages less than 80%. Key: 5. PSNU: Kinonconi MC; Prime Partner: 6. a. STAT coverage: Geita DC, Nyamagana DC b. ART coverage: Geita DC. Reasons for low % on ART: poor linkage or women choose not to initiate. Overly high % on ART women identified in early quarter start ART in the next quarter. c. See Tip Sheet for discussion of use of PMTCT_STAT_POS in EID Testing Coverage calculation. 7. See PMTCT Indicator Tip Sheet and Assumptions & Limitations for Proxy Infant Testing Coverage (PMTCT_EID / PMTCT_STAT_POS) 8 11: For discussion. Many possible answers. Ideas: a. Partner MER indicator narratives and partner quarterly reports i. Are the site level data accurate? ii. If data are accurate, has the partner provided information in the indicator narratives about reasons for the lower coverage whether there were stock outs, whether staff were not sending off samples that were collected? b. Non MER data: supply chain stock levels; detailed data on turn around time; cohort data. SIMS. c. Other ways to explore the MER data (not limited to EID): i. Quarter by quarter trend analysis: is the IP improving across quarters? Has the IP reported consistently per MER guidance across quarters? ii. Site level data review. iii. Disaggregate PMTCT_STAT by newly diagnosed vs. known positive at entry; disaggregate PMTCT_ART by new vs. already on life long ART. Calculate proxy linkage for newly identified positive pregnant women on ART. Review geographic variations. iv. MER Guidance explore Lab and HRH indicators at Q4 and possible link with results in a particular geographic area? Can these annual indicators help highlight locations needing follow up and potential areas to intervene? v. PMTCT_FO (Final Outcome) mother infant cohort tracking. vi. Retention among pregnant and post partum HIV+ women (TX_RET if data quality is good; otherwise, is non MER data on retention available?) 8

9 PMTCT Indicator Tips Priority analytical questions for the PMTCT clinical cascade 1. How many pregnant women know their HIV status? What percent? % STAT Coverage = PMTCT_STAT numerator / PMTCT_STAT denominator 2. How many pregnant women who know their HIV status are HIV positive? What percent? % Positive = PMTCT_STAT_POS / PMTCT_STAT numerator 3. How many HIV positive pregnant women are on ART? What percent? % ART Coverage = PMTCT_ART / PMTCT_STAT_POS 4. What percent of HIV exposed infants (HEI) had a first virologic HIV test by 2 months of age? By 12 months of age? Proxy EID Testing Coverage by 2 months of age = PMTCT_EID_2mo / PMTCT_STAT_POS Proxy EID Testing Coverage by 12 months of age = [PMTCT_EID_12mo (sum 0 2mo plus 2 12mo)] / PMTCT_STAT_POS *See assumptions and limitations. 5. How many HIV exposed infants were HIV infected by 2 months? By 12 months? What percent? EID % Positive 2mo = PMTCT_EID_2mo_POS / (PMTCT_EID_2mo_POS + PMTCT_EID_2mo_NEG) EID % Positive 12mo = PMTCT_EID_12mo_POS / (PMTCT_EID_12mo_POS + PMTCT_EID_12mo_NEG) This calculation excludes tests for which an EID sample was collected for a first virologic test, but for which result is unknown. *See assumptions and limitations. This applies to FY17 only; FY18 will use new, distinct indicator, PMTCT_HEI_POS. 6. How many HIV infected infants were linked to ART? What percent? PMTCT_EID_12mo_POS / TX_NEW <1 = % linked to ART *See assumptions and limitations. This applies to FY17 only; FY18 will use new, distinct indicator, PMTCT_HEI_POS. 7. How many HIV exposed infants receive a final outcome 6 weeks after cessation of breastfeeding? What percent? How many were negative? How many were positive? How many died? How many were lost to follow up? [PMTCT_FO numerator / PMTCT_FO denominator = % of HIV exposed infants with final outcome] Deeper analytical questions 1. Which partners are excelling along key priority questions, such as proxy EID testing coverage? What can we learn from them to share with other partners? 2. Which partners are struggling, where, and why on STAT coverage, ART coverage, proxy EID coverage? (by PSNU, site level, age disaggregates) What should we be asking our partners to further understand the context for their performance? 3. Which partners are over achieving with coverages >100%? Is this due to low denominators, actual performance, or data quality issues? 4. What are the assumptions, data limitations, and caveats associated with a. Using PMTCT_STAT_POS as a proxy for estimated number of HIV exposed infants? b. Calculating infant positivity with PMTCT_EID? c. Calculating infant linkage to treatment using PMTCT_EID_POS and TX_NEW age<1? DATIM Favorites to support logic checks FY17 Q3 PMTCT_STAT_POS, PMTCT_ART and TX_NEW_V4 web pivot/index.html?id=gq2zkszjcak MOZ FY17 Q3 PMTCT_STAT AGE and STATUS _V1 web pivot/?id=wca3od1yuq7 Moz FY17 Q3 PMTCT_EID (includes Q1 Q3) web pivot/index.html?id=xindkr2ez8w FY17 Q3 PMTCT_STAT Vs. HTS ANC V4 web pivot/?id=hhhzq5uv4ut 9

10 PMTCT MER Indicator Assumptions and Limitations for FY17 Proxy Infant Testing Coverage (PMTCT_EID / PMTCT_STAT_POS) Infant testing coverage relies on PMTCT_STAT_POS as a proxy denominator for the total number of HIV exposed infants (HEI). Reviewing infants with a first virologic test (N) against PMTCT_STAT_POS results (D) should be done carefully, keeping in mind the assumptions and limitations below. Review of outlier percentages for testing coverage by age band is recommended (e.g. review high and low outliers for 0 2 month testing coverage disaggregate). Assumptions We would not expect all the women reported under PMTCT_STAT_POS to have given birth to the infants reported under PMTCT_EID based on the time periods we are comparing. For example, the pregnant women in Q3 reported as PMTCT_STAT_POS will not all have given birth to the same infants reported as having a first sample collected for a virologic test in PMTCT_EID Q3. However, despite that time period mismatch, one assumption is that the total number of HIV positive women (estimated HEI) does not vary significantly quarter by quarter, so it is reasonable to compare infants tested to the STAT_POS denominator from the same reporting time period. Limitations PMTCT_STAT_POS likely underestimates the number of HEI because it includes only women who are HIV+ at ANC1 for the current pregnancy. It does not include women who attend ANC1 and are HIV+ but are not diagnosed; women who never attended ANC; or any woman who seroconverts after ANC 1, during delivery, or breastfeeding. Proxy EID testing coverage of over 100% may indicate that a program is doing well at (re)testing women throughout their infants risk exposure period per national guidelines. For example, testing of women in labor and delivery and post partum would not be captured under PMTCT_STAT_POS, but the first EID test of those women s infants would be reported in PMTCT_EID, leading to over 100% testing coverage. On the other hand, PMTCT_STAT_POS could overestimate the number of HEI that should be tested, because not all pregnancies may come to term. EID Positivity and % of Test Results Unknown For FY17, the MER guidance included a requirement to disaggregate first EID virologic tests collected in the quarter by the test result received in the quarter: positive, negative, or result unknown. USG should understand how partners are reporting and whether they are following the MER guidance. Is the partner reporting on all the samples collected in the quarter and providing data on the number of test results that are unknown? Or, are they only reporting on tests and test results for which the result was returned? Positivity calculated from PMTCT_EID results in FY17 should be done with caution. Positivity could be responsibly assessed if is sure that partners are reporting on all tests sent and correctly results as negative, positive, and unknown. Even if results are correctly reported, but there is a high percent of test results that are unknown, calculating positivity through EID disaggregates is not recommended. Infant Linkage to ART (TX_NEW age <1 / PMTCT_EID_POS (0 12mo)) One limitation of the EID indicator in FY17 is that if infant results are reported as unknown in one quarter, they will not show up as positive or negative result in the next quarter. This makes it difficult to know the true number of HIV positive infants. The EID_POS result is likely under representing the total number of positive infants identified. For this reason, comparing EID_POS to TX_NEW <1 as a linkage calculation should be done with caution; linkage could appear to be higher than it is in reality. 10