MARA referral M&E Framework with recommended indicators & sources of data Health Goal Aims (1) Reduce infections (HIV, Hep B & C, STI) (2) Increase ART treatment, for MARA boys & girls Recommended Indicators (<18, by sex & risk factor) HIV prevalence STI prevalence Hep B & C prevalence No. on ART % on ART adhering to treatment Data Collection National AIDS Authority statistics Bio-Sentinel surveillance Occasional prevalence studies (e.g. CDC) Nat l ART monitoring systems - NOT ANALYSED IN PROGRAMME - No direct control by programme Effects: Changes in Behaviours OUTPUTS Changes in Determinants INTERVENTION ACTIVITIES (1) MARA clients seek appropriate treatment from medical & social services (2) MARA receive comprehensive medical treatment according to identified needs MARA motivated to seek appropriate treatment Service providers friendly and non-discriminatory toward MARA Consent available for <18 year olds to receive treatment at point of referral % MARA accessing referral level treatment services in previous 12 months % MARA undergoing HIV testing No. and % HIV+ MARA undergoing ART eligibility screening No. and % MARA tested (& treated if positive) for STI, Hep B & C All services compliant with relevant health policies, e.g. AIDS strategy, STI programme No. referrals of MARA to medical services No. referred MARA accompanied by staff % referred vouchers collected at hospital, by referral organisation MARA clients aware & satisfied with services Service providers demonstrate motivation & positive attitudes to working with MARA No. MARA referred to child protection contact % MARA referred to child protection accessing medical services Information contributed to advocacy for legal change BSS, when available MARA study baseline-follow up surveys (ideally comparing intervention & comparison sites) Occasional audit of hospital medical records to assess treatment provided Documentation of how national guidelines, policies, and programmes are implemented on the ground (collected by NGO or advocacy organisations (?)) Routine project data from each referring organisation (voucher collection & analysis) Routine project data from each referral site (voucher collection & analysis) Client interviews (focus on MARA perceptions) Provider interviews (both referring & referral) Quality of care spot checks at referral sites Occasional documentation of case studies from project experience (1) Partnerships formed with medical services to provide friendly services & facilitate MARA access (2) Partnership formed with child protection authorities to ensure access to medical services where parental consent not available (3) Voucher system established for direct referrals to medical services, through staff accompaniment, and via child protection contacts Number of contracts signed with medical facilities; contact persons identified & trained Number of contracts signed with child protection; contact persons identified & trained Functioning referral system in place, including distribution & collection of vouchers Services follow project protocols and conform to quality standards Project records Training records content, attendance Documentation of partnership (meetings, correspondence) Routine quality monitoring (supplies, facility checklists, staff supervision, vouchers checked e.g. correctly printed, code system in use) Direct control by programme
Indicators & Methods Explained Definitions Quantitative indicators: Express a numerical value, such as number of clients receiving a service, or the proportion of staff who have been trained in a new skill. Can also enumerate aspects of quality, such as percent of medical providers expressing empathy for MARA during interviews.. Qualitative indicators: Capture feelings, perceptions, and characteristics of quality, such as how a clinic is youth-friendly, or why clients choose one VCT centre over another. Useful for interpreting quantitative information. Routine data: Information collected continuously, and compiled at specific intervals, e.g. monthly & quarterly. Often drawn from service statistics such as client registers or records. In this project, records of referrals, accompanied visits to medical facilities, and child protection records are all routine data. Vouchers will help verify these records (both the half kept by referring institutions, and the half given in at the referral site). Non-routine data: Additional information required on an occasional basis and therefore collected at rare (but regular) intervals (yearly, baselineendline) or selected as part of a special one-off study (such as audits of hospital records, interviews with clients and providers ). Effects-level indicators Indicator Data Collection Who? % MARA accessing referral level treatment services in previous 12 months Measures proportion of MARA who report having attended any STI or HIV service in the previous year, as well as the proportion who report attending the specific medical facilities participating in the referral intervention, disaggregated by sex, age group, and risk profile (IDU, SW) % MARA undergoing HIV testing Measures proportion of MARA who report having undergone an HIV test. Will be separated into 3 indicators: % who have ever tested for HIV; % tested in past 12 months; % know their status. Disaggregated by sex, age group, and risk. No. and % HIV+ MARA undergoing ART eligibility screening Measures whether referred MARA receive necessary treatment after seeking it. This indicator will show how many HIV+ young people accessed VCT at the participating facilities, and what % of them Baseline / Follow-up surveys Questions will be added for specific facilities Baseline / Follow-up surveys Hospital record audits at the start of the intervention (month 1-3) and at the end UNICEF/LSHTM 2ndary analysis of BSS if available, and same populations included UNICEF/LSHTM 2ndary analysis of BSS if available, and same populations included Commissioned by UNICEF; data collection by a member of the
who test positive are then referred into the ART programme. Although not all young people testing for HIV will necessarily be MARA, this can be used as a proxy indicator, since undergoing an HIV test implies a perception of risk (a risk assessment is used during VCT counselling to establish whether it is advisable for an individual to test.) No. and % MARA tested (& treated if positive) for STI, Hep B & C Measures whether referred MARA receive necessary treatment after seeking it. This indicator will show how many HIV+ young people accessed tests for STI, Hep B & C at the participating facilities, and what % of them who test positive are received appropriate treatment. Although not all young people testing for HIV will necessarily be MARA, those testing positive for Hep B & C can be used as a proxy indicator. STI testing and treatment cannot be taken as a proxy for MARA, as sexually active young people with no experience of IDU or selling sex are likely to seek these services. This will still provide an indication of service quality. All services compliant with relevant health policies, e.g. AIDS strategy, STI programme Assesses functioning of the treatment system at policy/ programme level, e.g. whether Syphilis testing & treatment available free of charge at point of delivery. (month 10-12). Records from the 6 months prior to data collection to be audited (between 3-12, depending on number of eligible files) Hospital record audits at the start of the intervention (month 1-3) and at the end (month 10-12). Records from the 6 months prior to data collection to be audited (between 3-12, depending on number of eligible files) Comparison on national policy & planning documentation with actual health system function. 2-3 times throughout project, information to be gathered from a comprehensive list of NGO with experience of national programmes Institute of Public Health with access to hospital records. Commissioned by UNICEF; data collection by a member of the Institute of Public Health with access to hospital records. UNICEF to conduct policy review and gather standardised feedback.
Output-level indicators Indicator Data Collection Who? No. referrals of MARA to medical services Measures coverage of referral project. This indicator will measure the number of clients reached by ARAS or any other referring organisation, who were referred to any specific medical service with which a partnership has been established. Numbers should be disaggregated by age, sex, risk factor, and which facility (and service) the referral was made. No. referred MARA accompanied by staff Similar to above, this indicator will demonstrate how many referred MARA have been accompanied to the medical facility. This should be disaggregated by age, sex, risk factor, facility and service referred to, and ideally why the individual was accompanied. Can also be expressed as a % of the overall referrals. % referred vouchers collected at hospital, by referral organisation This indicator will measure the extent to which the referral system motivates and facilitates MARA s access to appropriate medical services. Will validate other records kept of referrals. MARA clients aware & satisfied with services This indicator is qualitative, and monitors perceptions & attitudes toward various services (i.e. in reference to quality, access, content, acceptability and basic friendliness ). Service providers demonstrate motivation & positive attitudes to working with MARA This indicator is qualitative, and monitors providers perceptions of the project, how it is working, and will elicit some demonstration of how providers interact with and view MARA and their needs. Client Interviews 12-15 interviews conducted 2 times during the project; in months 4-6 and 10-12. Interviews to be conducted with MARA boys and girls, with a mix of risk profiles, facilities referred to, health needs, etc. Provider Interviews 1-2 staff from each referring or referral institution interviewed 2 times during the project; in months 4-6 ARAS staff member(s) responsible for M&E of the project ARAS to ensure data collection systems in place in all referring and referral project sites. ARAS to collect from all referring and referral sites. Analysis by ARAS and UNICEF Organised by UNICEF. Likely to use same university fieldworkers who conducted baseline qualitative research. Access to MARA to be facilitated by ARAS and other NGO. Organised by UNICEF. Likely to use same university fieldworkers who conducted baseline qualitative research.
No. MARA referred to child protection contact Measures coverage of referral project. This indicator will measure the number of clients reached by ARAS or any other referring organisation, who have been referred to child protection due to inability to gain parental consent for required medical services. Numbers should be disaggregated by age, sex, risk factor, and which child protection authority the referral made. No. and % MARA referred to child protection accessing medical services This indicator will measure the extent to which referrals through the child protection system successfully facilitates MARA s access to appropriate medical services. Care must be taken to ensure the Child Protection services keep good records. Information contributed to advocacy for legal change Evidence that the experience of the project has been disseminated to advocacy efforts for legislative reform. and 10-12. Spot checks for quality of services also required; 1-2 times in first 2-3 months, then at 6 months and 10 months, using standardised checklists Routine Service statistics & vouchers collected at participating medical services Occasional qualitative studies, to elicit case studies illustrating barriers to accessing services faced by MARA UNICEF ARAS staff member(s) responsible for M&E of the project ARAS to ensure data collection systems in place in all referring and referral project sites. ARAS to collect vouchers from all referring and referral sites. Analysis by ARAS and UNICEF UNICEF, in collaboration with participating NGO Indicators at the level of Intervention Activities self-explanatory and rely on good records kept throughout the project; important that all components of the intervention documented in as much detail as possible to assist in monitoring & evaluating the process of implementation.