SOCIAL MARKETING RESEARCH SERIES Romania (2007): HIV/AIDS TRaC Study Evaluating the Effect of a POL-type Program among Men who have Sex with Men in Bucharest Third Round The PSI Dashboard Bucharest, Romania June 2007 PSI s Core Values Bottom Line Health Impact * Private Sector Speed and Efficiency * Decentralization, Innovation, and Entrepreneurship * Long-term Commitment to the People We Serve
Mercury Research 95 Siret Street, 1 st Floor, District 1, 012152 Bucharest, Romania Research Division Population Services International 1120 Nineteenth Street NW, Suite 600 Washington, D.C. 20036 Romania (2007): HIV/AIDS TRaC Study Evaluating the Effect of a POL-type Program among Men who have Sex with Men in Bucharest Third Round Mercury Research, Romania Population Services International, 2007 Contact Information: Clayton Davis, Country Representative Str. George Calinescu Nr. 13, et. 3-4 Sector 1, Bucharest Romania Phone +4021 230.72.25 Fax +4021 230.72.33 Email cdavis@psi.ro Varja Lipovsek, Regional Researcher, PSI Bristol, UK Email varjalipovsek@yahoo.com Anca Serbanescu, Senior Research Consultant Mercury Research 95 Siret Street, 1 st Floor, District 1, 012152 Bucharest, Romania Phone +4021 224.66.00 Fax +4021 224.66.11 Email anca_serbanescu@mercury.ro
Summary Summary Acknowledgements We thank the owners of Queens and Impact, the interviewers who worked with PSI on this study, Mercury Supervisors. We also thank Varja Lipovsek, Regional Researcher, for her valuable advice and assistance. Background & Research Objectives Since November 2005, PSI/Romania has been implementing a 1.5-year self-funded program targeting men who have sex with men in Bucharest, Romania. The purpose of the program is to increase safer sexual behavior among MSM who frequent gay clubs in Bucharest through interpersonal communication (using the Popular Opinion Leader approach) and using a high coverage social marketing (SM) strategy to increase access to and availability of condoms and appropriate testing services. The current study is part of this program portfolio of PSI/Romania. It is the third and final round of surveys with the primary objective to evaluate the effect of the Popular Opinion Leader (POL) intervention piloted in two MSM clubs in Bucharest, Romania: Queens and Impact. Research objectives include measuring the implementation of the POL intervention, measuring exposure to the intervention, and correlating exposure with desired changes in health behaviors (primarily protected sexual intercourse and use of health services for STI and HIV testing) and selected determinants of these behaviors. Description of Intervention The intervention leverages the dynamic of social diffusion by recruiting POLs, or informal leaders, to advocate for sexually responsible behavior with their peers during normal everyday conversations. POLs attend a series of four two-hour trainings where they learn how to reduce their own risk and how to engage at-risk friends in persuasive conversations regarding condom use and STI/HIV testing. This model has been implemented in other countries with various target groups and usually aims to recruit and train 10-15% of the total target population. This percent roughly corresponds to the expected percent of the general population that is considered to be innovators or early adopters of new behavior. First phase of the intervention December 2005 - end of May 2006 consisted of a training program for 61 MSM POLs, recruited from the Queens bar in Bucharest. There were also two two-hour maintenance or reunion meetings of 30 POLs during this period. In addition to the - 1 -
Summary POL activities, the intervention also included a series of five brochures focusing on sexual health topics. In the second phase, beginning September 2006, PSI Romania started to recruit leaders from the new gay club, Impact, as the initial implementation venue, Queens, had closed down in May. This group included initial Queens' customers as well as new Impact customers that had not frequented Queens before. The leaders participated in trainings with a slightly different content from Phase 1, content/approach changes being implemented following the mid-term impact evaluation follow-up survey (conducted in June 2006) and trainers experience gained during the first phase of training sessions. The main change was a higher focus on the motivational dimension through the following objectives: motivate participants to come up with personal solutions to counteract barriers to safer behaviors, increase the level of motivation of the participants to endorse safer sex messages with their friends, concentrate on participants skills to explore and enhance the motivation of their peers for behavior change In this second phase, PSI/Romania trained 32 POLs (additionally: 6 other people only attended 1 or 2 sessions), between October to December 2006. There was also one maintenance session with about 18 POLs and another special maintenance session with about 15 POLs trained in the 1 st phase. Methodology During the time of intervention implementation and research, only two suitable venues where MSM congregate existed in Bucharest: clubs Impact and Queens. Therefore, the study employed a take-all approach without sampling; all eligible patrons of the clubs being surveyed were invited to participate in the survey. A baseline and two follow-up surveys were conducted (baseline and first follow-up report are available at www.psi.org/research/). The data collection during baseline took place over a period of five weekends, whereas the first and second follow-up surveys were carried out in six weekends in an effort to increase the number of study participants. The majority of eligible and willing respondents were captured. During the baseline, only the club Queens was open; therefore, the research was limited to this location. During the project implementation phase, the club Impact opened (and Queens closed for some months), and the program reached individuals frequenting Impact. At the time of the second follow-up, both clubs were open. The eligibility criteria in the third round therefore - 2 -
Summary included individuals who had frequented either the clubs Queens or Impact from January 2006. The recruitment ensured capturing respondents who had been exposed to the intervention in either club Impact or Queens. Exposure to the program (see the evaluation analysis on page 17) was defined as exposure to conversations about safe sex and getting tested for STI or HIV, not exposure to PSI-led trainings. However, those trained as peer leaders were also included in the exposed group at evaluation analysis. Main Findings As the analysis of changes over time indicates, it seems the program peaked at follow-up 1 and then maintained or even decreased (on STI testing in past two months) its performance. This hypothesis is sustained by the following indicators: unprotected anal sexual acts, and STI testing in past two months. A number of contextual factors which may be relevant in explaining this outcome. In the first phase of the intervention STI testing was free of charge (possible effects: testing in groups, more people that get tested). The first group of peer leaders trained included people who fit best the POL profile visible in their group of friends, well established leaders, etc. whereas people in the second phase possessed these characteristics to a lesser extent. Findings do suggest that people talk more about testing, as this indicator increased from baseline to follow-up 2. However, based on results of the current study, it was not possible to determine a clear effect of the POL intervention. Results on the STI testing indicator STI testing in the past two months registers higher values among exposed respondents than among non-exposed; but there was no difference between baseline and follow-up. This can be interpreted as a targeting effect. This means that the program potentially targeted people who were pre-disposed to being tested. In other words, it cannot be stated that exposure to the program caused the increase in testing; rather, it is possible that people who were more likely to be tested were also more likely to be exposed (or drawn) to the program. An overall reduction of anal and oral sexual acts over time was noted (see the monitoring analysis). However, this was not attributable to the program, as no significance was showed between not-exposed and exposed respondents. - 3 -
Summary Programmatic Recommendations It should be noted that the key behaviors were already quite high at baseline: 62% reported consistently using condoms, 77% reported ever having an STI test, and 75% reported ever having an HIV test. The key behaviors can therefore not be considered innovations and in this light, the program s main approach social diffusion of innovation may not have been the best strategy. Although testing behaviors were quite high, it seems to be easier to promote testing than behavior change related to condom use. This could be due to the fact that testing requires a single action (i.e., getting a test), which does not need to include the partner. Condom use, particularly consistent condom use, requires a long-time commitment to behavior change and the cooperation and consent of both partners. Thus, further inquiry into what drives and sustains condom behavior might be worthy. Among drivers that may constitute possible choices for changing behavior, the research identified: talking about safer sex (with a significant influence on both condom use and STI testing); ability related barriers: self-efficacy in insisting on safer sex with someone with whom one has had unprotected sex before; and social support: believing that friends think it is important to use a condom with every anal sex act (see more details in the segmentation analysis on page 12 and 13). The program should target those with multiple partners, this being a group with a high risk level: people that have multiple partners seem to be more likely to engage in risky oral sex than those with one or no partners (see segmentation analysis on page 11). - 4 -
Monitoring Table Baseline and follow-up (1 and 2) results regarding condom use for anal sex, risky oral sex, health service utilization, and related determinants among men who have sex with men, Risk Group: Men who have sex with men frequenting the bars Queens and Impact in Bucharest, Romania Behaviors: Condom use in anal sexual acts in past 60 days; non-risky oral sexual acts in the past 60 days; getting STI test in the past 12 months, and ever tested for HIV Baseline Follow-up1 Follow-up2 MONITORING TABLE Sign. (n=117) (n=147) (n=266) SEXUAL CHARACTERISTICS and RISK % or mean % or mean % or mean In the past year, sexual partners were only men (vs. both men and 60.0% ab 70.1% a 54.1% b ** women) Proportion respondents who had multiple partners in the last 60 days 64.6% a 58.3% a 53.3% a (vs. one or no partners) Mean number of different partners (anal sex) in the last 60 days; 3.6 (m) a 4.4 (m) b 3.0 (m) a range 0-30 Within the last year, has received money or other compensation for 11.2% a 10.8% a 10.0% a sex SEXUAL BEHAVIORS Mean number of anal sexual acts in the last 60 days; range 0-60 11.0 (m) a 9.5 (m) a 7.0 (m) b ** Mean number of unprotected anal sexual acts in the last 2 months; 4.5 a 2.2 b 2.7 ab range 0-60 Among those with multiple partners: Of the number of anal sexual acts in the last 60 days, the estimated mean number that were 8.1 a 9.1 a 7.0 a protected (with a condom) Proportion of protected anal sex, of all anal sex acts 75.1% a 74.5% a 72.2% a Proportion respondents who reported using a condom at all anal sex 61.7% a 63.5% a 63.0% a acts in the last 60 days (consistent condom use) Proportion respondents who reported using a condom at last anal sex -- -- 73.1% with a man Mean number of oral sex acts in the last 60 days; range 0-60 11.5 ab 13.2 b 9.0 a * Of the number of oral sex acts in the last 60 days, the estimated mean number that were risky (i.e., ejaculation in either partner s mouth); 4.1 a 4.0 a 3.0 a range 0-50 Among those with multiple sexual partners: Of the number of oral sex acts in the last 60 days, the estimated mean number that were 3.6 a 2.8 a 3.5 a risky (i.e., ejaculation in either partner s mouth) Proportion of unprotected oral sex acts, of all oral sex acts 30.3% a 28.0% a 31.4% a Proportion respondents who report engaging in no risky oral sex acts 53.6% a 49.8% a 57.4% a in the last 60 days BEHAVIOR: USE OF HEALTH SERVICES Has ever gone to a health service to get an STI test 77.4% a 72.5% a 77.5% a Has had one or more STI test in the past 2 months (of those who 42.8% a 58.7% b 41.3% a * have ever had an STI test) If ever had an STI test, has had one or more within the past year -- -- 74.4% (January 06 February 07) If ever had an STI test, FIRST TIME Jan 06 Feb 07 -- -- 30.9% 91.9% (n=19) 94.0% If ever had an STI infection, notified any relevant sexual partner 86.1% (n=56) a a (n=28) a Has ever gotten an HIV test 74.7% a 72.9% a 70.5% a OPPORTUNITY Peer norms Strongly agrees that friends think it s important to use a condom with 67.9% a 64.5% a 71.1% a every anal sex act Indicators below relate to peer norms and peer influences and are proxy indicators for being exposed to the POL intervention - 5 -
Monitoring Table MONITORING TABLE In a typical 2-months period, talks to friends about safe sex once a week or more (vs. less frequently) In the past 2 months, friends have started a conversation about safer sex Mean number of times friends have started a conversation about safer sex in the past 2 months (of those who have had any such conversation); range 1-27 In a typical 2-months period, talks to friends about getting tested for STI or HIV once a week or more (vs. less frequently) In the past 2 months, friends have started a conversation about getting tested for STI/HIV Mean number of times friends have started a conversation about getting tested for STI/HIV in the past 2 months (of those who have had any such conversation); range 1-20 In the past 2 months, friends have started a conversation about both safer sex and getting tested for STI or HIV ABILITY Self-efficacy Agrees it is difficult to insist on safer sex with someone he has known for a long time Agrees it is difficult to insist on safer sex with someone with whom he has had unprotected sex before Agrees it is difficult to insist on safer sex with someone for whom he has strong feelings MOTIVATION Baseline (n=117) Follow-up1 (n=147) Follow-up2 (n=266) 47.1% a 56.0% a 58.2% a 57.7% a 66.6% a 63.2% a 4.1 (m) a 5.1 (m) a 5.1 (m) a Sign. 46.2% a 57.9% ab 58.8% b 53.5% a 56.0% a 52.6% a 3.2 (m) a 3.4 (m) a 3.9 (m) a 40.4% a 47.6% a 44.1% a -- 36.8% a 39.2% a -- 31.3% a 38.6% a -- 37.7% a 34.6% a Perceived threat/risk Perceives self at some or high risk for an STI 56.0% a 50.7% a 49.0% a Perceives self at some or high risk for HIV 43.7% a 37.2% a 36.4% a EXPOSURE Has attended 3 or 4 of the 4 Popular Opinion Leader trainings -- 25.0% 17.8% POPULATION CHARACTERISTICS Mean age (range 18-42) 24.4 (m) a 25.3 (m) a 24.9 (m) a Completed university or more (vs. lower levels) 53.8% a 53.8% a 43.9% b Average monthly after-tax income is more than 1,051 RON (vs. less) 39.1% a 60.0% b 55.2% b ** Visited the club Queens/Queens or Impact most or every weekend in the past 3 months 46.2% a 39.5% a 48.5% a (m) = mean a,b = Means or proportions marked by the same letter are not different from each other; means and proportions marked by different letters are different from each other. *=p<0.05; **=p<0.01; ***=p<0.001; = p<0.10 All behavior, risk and determinant variables are adjusted by age, education, income, primary type of sexual partners (all males or males and females), and frequency of attending the club Queens/Impact. Population characteristics are unadjusted. Question asked at follow-up 2 only; proportions are unadjusted. - 6 -
Monitoring Analysis Monitoring Analysis The preceding monitoring table was prepared in accordance with PSI s behavior change framework, PERForM 1, and presents change over time in risk variables, key behaviors, and in opportunity, ability, and motivation factors related to safe sexual practices. Population characteristics indicate that the baseline and follow-up samples were not different in terms of age composition and being a frequent visitor of the club Queens or Impact. However, compared with baseline, the follow-up 2 population seems to be significantly less educated: at follow-up 2, 43.9% of the sample reported completing at least university (as compared to 53.8% at both baseline and follow-up 1). Income is another differentiator between baseline and follow-up population: at follow-up 2, 55.2% reported earning more than 1,051 RON per month (as compared to 39.1% at baseline). Looking at the sexual characteristics and risk variables measured, the populations investigated at both follow-up surveys appear different from each other in terms of having mostly men as sexual partners, but both are similar to the baseline description on this variable. At the same time, the mean number of partners declined from follow-up 1 to follow-up 2. In all three waves, similar percentages of respondents reported that they had multiple partners in the past 2 months between approximately 55%-65% (no significant difference, despite the decreasing trend from baseline to follow-up 2) or having engaged in transactional sex in the past year approximately 11%. Regarding sexual behaviors, the results show that respondents reported a decrease in the mean number of anal sexual acts from 11.0 at baseline, to 9.5 at follow-up 1, and to 7.0 at follow-up 2. A similar trend is noticed on the mean number of oral sex acts in the last 2 months. The mean number of unprotected sexual acts also decreased from 4.5 at baseline to 2.2 at follow-up 1, but maintain a similar value of 2.7 at follow-up 2. Analysis of change over time shows stability on the remainder of indicators of sexual behaviors. Thus, no change was detected on consistent condom use, the proportion of respondents engaging in no risky oral sex, mean number of risky oral sexual acts or the proportion of risky oral sex. At follow-up 2, the international (UNAIDS) indicator relevant to MSM was added: 73.1% of respondents reported using a condom at last anal sex with a man. Health service utilization was another area considered. The analysis of change over time shows that the proportion of respondents who have ever had an STI or a HIV test did not change (77.4%, 72.5%, 77.5% - 7 -
Monitoring Analysis for STIs and 74.7%, 72.9%, 70.5% for HIV). However, STI testing in the past two months peaked at follow-up 1 and then decreased, back to the baseline value: increase from 42.8% at baseline to 58.7% at follow-up 1 and then down to 41.3% at follow-up 2. As measured in the second follow-up, the proportion of respondents who have had at least one STI test within the past year (January 2006 February 2007 being the period affected by the program) is relatively high at 74.4%, but comparable with the proportion of respondents ever tested. Finally, the proportion of respondents who have had an STI and notified a sexual partner maintained high values in all three survey rounds (91.9%, 94.0%, 86.1%) (with no significant change). Opportunity was measured through items relating to peer norms and peer influences, which also act as proxy indicators for the reach of the POL intervention (the model hinges on conversations occurring naturally in social groups). 71.1 % mentioned in the third round that their friends believed it is important to use a condom at every anal sexual act, the percentages being comparable with those observed at baseline or follow-up 1 (67.9%, 64.5%). The data show no significant changes in peer norms, nor in talking to friends about safe sex (47.1%, 56.0%, 58.2%). There was however a marginally significant change (at p<0.10) in talking to friends about STI testing: it increased from 46.2% at baseline to 58.8% at follow-up 2. When talking about safer sex and talking about testing are combined into one indicator, there is no significant change over time in respondents who reported having had both conversations in the past 2 months (40.4%, 47.6%, 44.1%). Ability was measured through items relating to self-efficacy in using condoms in different situations. These items were added at follow-up 1 (because they were found to be significantly associated with condom use in an online survey conducted among MSM population in the end of 2005), so the analysis considers changes from this point to follow-up 2. No significant changes over time were identified on the ability items: at both follow-ups, approximately one third of the sample agreed that it would be difficult to insist on safe sex with a person the respondent has known for a long time, with a person with whom he has had unprotected sex before, and with a person for whom he has strong feelings. Motivation refers to items relating to perceived risk. There was no change over time on perceiving oneself to be at risk for STI and HIV: in the third round, 49.0% perceived themselves at some or high risk for an STI (no significant change from 56.0% at baseline, 50.7% at follow-up 1); 36.4% did the same in case of HIV (no significant change from 43.7% at baseline, 37.2% at follow-up 1). - 8 -
Monitoring Analysis Exposure to POL was analyzed through attendance of POL trainings; a participant was considered a POL if he attended at least 3 out of 4 meetings. This proportion is comparable across the two follow-up measurements: 25% at follow-up 1, and 17.8% at follow-up 2. - 9 -
Segmentation Analysis (1 of 4) Segmentation Analysis 1 Consistent condom use Segmentation is the process of dividing a heterogeneous population into a homogenous audience; in other words, the population is divided into those who practice the desired behavior and those who do not, e.g., condom users and non-users. Then each group is profiled according to the barriers to behavior change that were captured in the surveys, which, in turn, can allow programmers to design and implement behavior change interventions. The segmentation analyses shown in this report pertain to the follow-up 2 sample only. For the behavioral outcome consistent condom use, a segmentation table could not be constructed as only one predictor was identified as having a significant influence on consistent condom use. Consistent condom users were different from non-consistent users only in regard to one ability factor agreement that it is difficult to insist on safer sex with someone with whom one has had unprotected sex before. - 10 -
Segmentation Table and Analysis (2 of 4) Segmentation Table 2: Risky oral sex among men who have sex with men, follow-up survey, Risk Group: Men who have sex with men, and who frequented the bar Queens or Impact in Bucharest, Romania Behavior: Engaged in any risky oral sex acts in last 60 days (n=225) No risky SEGMENTATION TABLE oral sex (46.2%) SEXUAL CHARACTERISTICS and RISK % or mean Risky oral sex (53.8%) % or mean Proportion respondents who had multiple partners in the last 60 days (vs. one or no partners) 43.6% 65.0% 2.46 ** MOTIVATION Perceives self at some or high risk for an STI or HIV 49.6% 39.2% 0.59 Oral sex is termed risky when it ends in ejaculation in either partner s mouth Computed as a mean from Perceives self at some or high risk for an STI and Perceives self at some or high risk for HIV, variables that are highly correlated *=p<0.05; **=p<0.01; ***=p<0.001; = p<0.10. Hosmer and Lemeshow Test: Chi-square=0.671, df=4, Sig=0.955; Omnibus Test: Chi-square=12.7, df=2, p<0.01; R squares: Cox & Snell Rsquare=0.053, Nagelkerke Rsquare=0.07 Each variable is adjusted for all other variables in the model. ORs Sign. Segmentation Analysis 2 Risky oral sex Respondents who had engaged in risky oral sex are more likely to have had multiple partners in the last two months than respondents with no risky oral sex (65.0%, compared to 43.6%). In terms of odds ratios, people that have multiple partners seem to be 2.46 times more likely to engage in risky oral sex than those with one or no partners. Risky oral sex and motivation are also related: respondents who did not engage in risky oral sex were more likely to perceive themselves to be at risk for an STI or HIV (49.6%, compared to 39.2%). This may suggest that high-risk individuals often do not perceive themselves to be at risk. - 11 -
Segmentation Table and Analysis (3 of 4) Segmentation Table 3: STI testing in the past two months among men who have sex with men, follow-up survey, Risk Group: Men who have sex with men, and who frequented the bar Queens in Bucharest, Romania Behavior: Has had one or more STI tests in the past 2 months (n= 194) SEGMENTATION TABLE No test 1+ tests ORs (57.7%) (42.3%) Sign. SEXUAL CHARACTERISTICS and RISK % or mean % or mean Within the last year, has received money or other compensation for sex 6.1% 16.1% 2.99 * OPPORTUNITY In a typical 2-months period, talks to friends about safer sex once a week or more (vs. less frequently) 55.1% 72.3% 2.14 * *=p<0.05; **=p<0.01; ***=p<0.001; = p<0.10. Hosmer and Lemeshow Test: Chi-square=0.045, df=1, Sig=0.832; Omnibus Test: Chi-square=10.65, df=2, p<0.01; R squares: Cox & Snell Rsquare=0.053, Nagelkerke Rsquare=0.072 Each variable is adjusted for all other variables in the model. Segmentation Analysis 3 STI testing in the last two months STI testing seems to be associated with transactional sex, as respondents who had one ore more STI tests in the previous two months were more likely to have received money or other compensation for sex than those not tested (16.1%, compared to 6.1%). In terms of odds ratios, people that received money or other compensation for sex are 2.99 times more likely to get tested in the past two months than those who had not engaged in transactional sex. In terms of social support, respondents who had one or more STI test reported that their friends talked to them about safer sex more often than in case of respondents that were not tested (72.3%, compared to 55.1%). Respondents that talked about safer sex with their friends once a week or more (in a typical two months period) are 2.14 times more likely to test themselves than those not talking or talking less frequently with their friends about this subject. - 12 -
Segmentation Table and Analysis (4 of 4) Segmentation Table 4: Condom use at last anal sexual act among men who have sex with men, follow-up survey, Risk Group: Men who have sex with men, and who frequented the bar Queens in Bucharest, Romania Behavior: Used condom at last anal sexual act (n= 243) SEGMENTATION TABLE Non-users Users ORs (25.1%) (74.9%) Sign. OPPORTUNITY % or mean % or mean In a typical 2-monh period, talks to friends about safer sex once a week or more (vs. less frequently) 46.6% 63.5% 1.98 * ABILITY Agrees it is difficult to insist on safer sex with someone with whom he has had unprotected sex before 59.8% 31.6% 0.31 *** MOTIVATION Agrees their friends think it is important to use a condom with every anal sex act. 57.4% 76.9% 2.40 * POPULATION CHARACTERISTICS Visited the club Queens and/or Impact most or every weekend in the past 3 months 61.6% 44.2% 0.50 ** *=p<0.05; **=p<0.01; ***=p<0.001; = p<0.10. Hosmer and Lemeshow Test: Chi-square=3.168, df=8, Sig=0.923; Omnibus Test: Chi-square=30.26, df=4, p<0.001; R squares: Cox & Snell Rsquare=0.117, Nagelkerke Rsquare=0.173 Each variable is adjusted for all other variables in the model. Segmentation Analysis 4 Condom use at last anal sexual act As segmentation table 4 indicates, condom users at last anal sexual act differ from non-users in terms of opportunity, ability, and motivational factors, as well as frequency of visits to Queens or Impact (as a group characteristic). Condom users at last sexual act show a higher social support than non-users: higher frequency of talking to their friends about safer sex (63.5%, compared to 46.6%). Respondents that talked about safer sex with their friends once a week or more (in a typical two months period) are 1.98 times more likely to use condom at last anal sexual act than those not talking or talking less frequently with their friends about this subject. Another form of social support with motivational potential differentiates between condom users at last anal sex and non-users: more respondents of the former category reported their friends think it is important to use a condom with every anal sex act (76.9%, compared to 57.4%). Respondents that agreed their friends think it is important to use a condom with every anal sex act - 13 -
Segmentation Table and Analysis (4 of 4) are 2.40 times more likely to use a condom at last anal sexual act than those not agreeing with this form of social support. At the same time, there were fewer condom users that agreed it is difficult to insist on safer sex with someone with whom one has had unprotected sex before (31.6%, compared to 59.8%). - 14 -
Evaluation Table Evaluation results of the POL intervention among men who have sex with men, Romania, 2007 Risk Group: Men who have sex with men frequenting the bar Queens or Impact in Bucharest, Romania Exposure has 3 levels: 0 = Baseline: All respondents 1 = Follow-up not-exposed: Those who did not report their friends giving them advice in the past two months about safe sex and getting tested for STI & HIV 2 = Follow-up exposed: Those who report that in the past two months friends gave advice about safe sex and getting tested for STI & HIV and those trained as peer leaders Behaviors: Condom use in anal sexual acts in past 60 days; non-risky oral sexual acts in the past 60 days; getting STI test in the past 12 months, and ever tested for HIV Follow-up: Follow-up: Baseline EVALUATION TABLE not exposed exposed Sign. (n=117) (n= 124) (n=137 ) SEXUAL CHARACTERISTICS and RISK % or mean % or mean % or mean Mean number of different partners (anal sex) in the last 60 days; range 0-30 Proportion respondents who had multiple partners in the last 60 days (vs. one or no partners) Mean number of anal sexual acts in the last 60 days; range 0-60 SEXUAL BEHAVIORS The mean number of unprotected anal sexual acts in the last 2 months; range 0-60 The proportion of all sexual acts that were protected with a condom in the last 2 months Proportion respondents who reported using a condom at all anal sex acts in the last 60 days (consistent condom use) The mean number of risky oral sex acts in the last 2 months (i.e., which ended with ejaculation in either partner s mouth); range 0-50 The proportion of all oral sex acts that were risky, in the last 2 months Proportion respondents who report engaging in no risky oral sex acts in the last 60 days BEHAVIOR: USE OF HEALTH SERVICES 3.6 a 3.0 a 3.3 a 65.7% a 52.0% b 56.8% ab 11.2 a 5.5 b 8.1 b ** 4.5 a 1.9 b 3.1 ab 75.4% a 73.5% a 72.0% a 62.2% a 62.5% a 63.4% a 4.3 a 3.1 a 2.9 a 30.2% a 29.3% a 34.2% a 52.7% a 56.9% a 58.6% a Has ever gone to a health service to get an STI test 77.0% a 74.4% a 81.4% a Has had one or more STI test in the past 2 months (of those who have ever had an STI test) 43.4% a 30.4% b 53.0% a ** Has ever gotten an HIV test 74.6% ab 64.9% b 75.8% a OPPORTUNITY Peer norms Strongly agrees that friends think it s important to use a condom with every anal sex act ABILITY 67.7% a 67.2% a 75.9% a - 15 -
Evaluation Table EVALUATION TABLE Self-efficacy Agrees it is difficult to insist on safer sex with someone he has known for a long time Agrees it is difficult to insist on safer sex with someone with whom he has had unprotected sex before Agrees it is difficult to insist on safer sex with someone for whom he has strong feelings MOTIVATION Baseline (n=117) Follow-up: not exposed (n= 124) Follow-up: exposed (n=137 ) -- 38.7% a 39.7% a -- 41.2% a 37.0% a -- 39.8% a 30.5% a Sign. Perceived threat/risk Perceives self at somewhat or high risk for an STI 55.8% a 53.0% a 46.9% a Perceives self at somewhat or high risk for HIV 43.8% a 38.6% a 37.0% a a,b = Means or proportions marked by the same letter are not different from each other; means and proportions marked by different letters are different from each other. *=p<0.05; **=p<0.01; ***=p<0.001; = p<0.10 All behavior, risk and determinant variables are adjusted by age, education, income, primary type of sexual partners (all males or males and females), and frequency of attending the club Queens/ Impact. - 16 -
Evaluation Analysis Evaluation Analysis The evaluation table presents key behaviors and determinants by different exposure categories. The variable which measures exposure to the POL intervention is based on two items: conversations among peers on safe sexuality and getting tested, and being a leader (i.e., individuals who were trained as part of the Popular Opinion Leader intervention). In addition, the variable also includes the baseline data. The categories of the variable are: 0 = baseline (all respondents); 1 = follow-up, not exposed to POL (i.e., respondents who did not report having conversations with their friends about safe sex and testing in the past two months); 2 = follow-up, exposed (i.e., respondents who reported having conversations with their friends about safe sex and testing in the past two months, and respondents who participated in any of the POL trainings). The analysis indicates some differences in terms of sexual risks between exposure categories: a lower proportion of non-exposed respondents with multiple partners than baseline respondents, but no difference between exposed and the other two categories (baseline 65.7%, not exposed 52.0%, exposed 56.8%); the mean number of anal sexual acts in the last two months was highest among those surveyed at baseline (mean of 11.2 anal sexual acts, the difference is statistically different), compared with non-exposed and exposed, that registered comparable values: mean of 5.5, respectively 8.1. Regarding the sexual behaviors, the table shows that the mean number of unprotected sexual acts in the last two months is lower among follow-up not exposed than at baseline. However, as no difference was identified between follow-up exposed and either of the other groups, we cannot state an effect of the POL intervention based on this indicator. On other sexual behaviors analyzed, there are no significant changes across exposure categories: no change on consistent condom use (baseline 62.2%, not exposed 62.5%, exposed 63.4%), no change on proportion of all sexual acts that were protected (75.4%, 73.5%, 72.0%). A similar situation was noticed in the case of oral sex indicators: no differences between exposure categories, regarding proportion of unprotected oral sex acts (baseline 30.2%, not exposed 29.3%, exposed 34.2%) or proportion of respondents reporting engaging in no risky oral sex in the past two months (52.7%, 56.9%, 58.6%). - 17 -
Evaluation Analysis HIV and STI testing illustrates the following pattern: differences between non-exposed and exposed category, with higher values in case of the last category, on both STI testing in past two months (not exposed 30.4%, exposed 53.0%), and ever tested for HIV (not exposed 64.9%, exposed 75.8%), However, as there was no difference between the baseline and the follow-up exposed category, is not possible to determine an effect of the POL intervention from these indicators. Opportunity and ability factors show no significant differences across exposure categories. A similar proportion of respondents consider it important to use a condom with every anal sex act (baseline 67.7%, not exposed 67.2%, exposed 75.9%). Respondents in different exposure categories appreciate similarly their ability to insist on safer sex in various situations: with someone he has known for a long time (not exposed 38.7%, exposed 39.7%), with someone with whom he has had unprotected sex before (not exposed 41.2%, exposed 37.0%), with someone for whom he has strong feelings (not exposed 39.8%, exposed 30.5%). No significant difference was noticed on motivation-related factors: perceives self at somewhat or high risk for an STI (baseline 55.8%, not exposed 53.0%, exposed 46.9%) or for HIV (baseline 43.8%, not exposed 38.6%, exposed 37.0%). - 18 -
Effectiveness Summary Effectiveness Summary Men who have sex with men frequenting the bars Queens and Impact in Bucharest, INDICATORS SEXUAL BEHAVIORS MONITORING TABLE (Baseline to follow-up 2) EVALUATION TABLE (Exposed vs Non-exposed at Follow-up 2) CONCLUSION Mean number of anal sexual acts in the last 60 days; range 0-60 - NS No impact BEHAVIOR: USE OF HEALTH SERVICES Has had one or more STI test in the past two months (of those who have ever had an STI test) NS + No impact (targeting effect) For an overall image of the effectiveness of the program, the following aspects were considered: the change between baseline and follow-up 2 in the monitoring table and the difference between exposed and non-exposed in the evaluation table, as this complements the monitoring results. The table above presents relevant indicators, i.e., indicators that either showed a change in the monitoring table or were found significant in the evaluation table. There were only two indicators, one for the sexual behavior and the other regarding the use of health services. The mean number of anal sexual acts declined between baseline and follow-up 2. However, there was no difference between exposed and non-exposed at follow-up. The overall conclusion is that no impact of the program can be claimed in this concern. The use of the health services in the past two months was higher for the exposed vs. non-exposed. However, since it was not significant in the monitoring table, no impact can be claimed on this indicator either. The interpretation for this is the targeting effect: i.e. there is a doubt about the direction of the cause-effect relation between testing and exposure either exposure leaded to testing, or those tested in the past two months were pre-selected by the program (intervention targeted those who were already predisposed to change). These are two equally possible alternatives. Another indicator that changed between baseline and follow-up 2 in the monitoring table was: In a typical two month period talks to friends about getting tested for STI or HIV once a week or more (vs. less frequently). Yet, this cannot be used for the evaluation of the program s impact because it is one of the items that make up the exposure. - 19 -
Programmatic Recommendations Programmatic Recommendations Results on the STI testing indicator - STI testing in the past two months registers higher values among exposed respondents than among non-exposed; but there is no difference between baseline and follow-up can be interpreted as a targeting effect. This means we can not clearly state if the exposure caused testing, or those that get tested (already predisposed to desired outcomes) simply talked more about it and thus, are more likely to recall the intervention. Thus, there is a doubt about the direction of the cause-effect relation between testing and exposure and we can not conclude that the intervention brought about a change in behaviors or determinants. Besides the fact that changing behavior among this population proved difficult, another potential explanation for the lack of research proven impact might be that the key behaviors (condom use and testing) were already quite high at baseline. These behaviors can therefore not be considered innovations and in this light, the program s main approach social diffusion of innovation may not have been the best strategy. The research showed some findings that people talk more about testing. It seems to be easier to promote and train to encourage testing than behavior change related to condom use. Thus, further inquiry into what drives and sustains the condom behavior might be worthy. As talking about safer sex does influence the actual condom use, as well as testing ( respondents that talked about safer sex with their friends once a week or more, in a typical two months period, are more likely to use condom at last anal sexual act and to test themselves for STI than those not talking or talking less frequently with their friend about this subject (see more details in the segmentation analysis on pages 12 and 13), sustaining conversations about safer sex should be considered. Overcoming ability-related barriers (self efficacy in insisting on safer sex with someone with whom one has had unprotected sex before) proved also to have a significant influence on condom use. The program should target those with multiple partners, this being a group with a high risk level: people that have multiple partners seem to be more likely to engage in risky oral sex than those with one or no partners (see more details in the segmentation analysis on page 11). - 20 -
Annex 1: Methodology Methodology Sample Characteristics The sample was composed of men at the Queens club (at baseline), club Impact at follow-up 1 and clubs Queens and Impact (only 4 out of 6 weekends, the population saturation point being reached). Respondents recruited at follow-up 2, similar to the other two measurements, met the following criteria: (1) had at least some male sexual partners in the past year; (2) frequented the club Queens and/or Impact at least some of the weekends in the period between January 2006 March 2007 (previous 3 months at baseline; and in the period between December 2005 and April 2006 at follow-up 1); (3) was at least 18 years old; (4) lived in Bucharest for the previous 12 months or longer (6 months at baseline and follow-up 1). Sampling Methodology: A take-all approach If at baseline and follow-up 1, the universe from which the sample could be drawn was restricted to one club - Queens, respectively Impact at follow-up 2 both clubs were included. Apart from these two clubs, currently there are no other suitable MSM venues in Bucharest where the program as well as the research could be implemented. Therefore, a take-all approach was used. That is, all eligible and willing respondents were asked to participate in the study over a period of six weekends (having the objective to obtain a bigger sample, compared to baseline and follow-up 1, when the data collection period was five weekends). The weekends (Friday and Saturday evening) were chosen since those were the busier days in the clubs. Interviews with key informants in both clubs in January 2007 suggested the following estimations of maximum number of clients: Table 1 Population estimates Friday and Saturday, at peak time Location Day / peak time Maximum number of clients Queens Fri / 23.30-02.30h 90 Queens Sat / 23.30-02.30h 300 Impact Fri / 23.30-02.30h 80 Impact Sat / 23.30-02.30h 230 Total 700 Discounting was made for over-estimation of this figure (10%) and presence of female and heterosexual patrons (10%). At the same time, it is assumed there are not too many repeats on a given weekend; i.e., most people are loyal to one club and one night on which to go out. - 21 -
Annex 1: Methodology However, it is estimated that a large proportion of individuals are regular clients that is, they will be repeat clients over the subsequent weekends. Being fairly closed networks, it is likely that relatively few new clients would be present at either club on a subsequent weekend. Furthermore, the maximum number of interviews which could be conducted was estimated at 25-30 per weekend. Given these restrictions, a desired sample size of 120 was set at baseline; a desired sample of 140 was set at follow-up 1; and 275 at follow-up 2. At all measurements, the following situation was encountered: at the end of the allocated fieldwork period the number of eligible respondents had greatly diminished, suggesting that the majority individuals in the target audience who were willing to participate in the study had done so. A table detailing the total number of contacts, refusals and non-eligible contacts is shown below. Table 2: Refusals and non-eligible contacts Baseline Follow-up 1 Follow-up 2 Total number of contacts 303 355 912 Reasons Refusal to participate from the beginning 25 49 95 Foreigners 14 19 38 Already participated in the survey 60 42 322 Does not pass age filter 4 2 3 Does not pass living in Bucharest filter 26 30 50 Does not pass the Queens visit frequency filter 39 54 38 Does not pass the sexual partners' filter 8 10 34 Refusals at the end of recruitment questionnaire 0 2 14 Other refusals 10 0 38 Does not pass the POLs filter (Once a maximum of 13% has been reached, they were screened out to avoid over-sampling of POLS) - - 12 Self-completed questionnaires 117 147 268 Data Collection Procedure Data was collected over a period of five weekends at both baseline and follow-up 1 and over a period of six weekends at follow-up 2 (only four weekends for Impact, because the MSM population s saturation point was reached). The method used for selecting respondents was the same at all measurements: two interviewers approached every client while they were in the club, on Friday and Saturday night (club Queens at baseline, club Impact at follow-up 1, both clubs at follow-up 2). Each willing respondent was first given a brief questionnaire to establish if he was eligible to participate; those who passed the filter and were willing to take part in the study were included in the sample. - 22 -
Annex 1: Methodology Survey instrument The survey was a self-completed quantitative questionnaire with approximately 30 items, covering the following topics: population characteristics; communication with friends about safe sex and testing; items relating to self-efficacy to use condoms in various situations; personal risk perception for HIV and STI; sexual behaviors, including anal sex and oral sex, the use of condoms and practicing non-risky oral sex; use of health services for HIV and STI testing; and exposure to the PSI POL program. The survey took about 10-15 minutes to complete. In addition, a limited number of new questions were added, focusing primarily on having been part of the opinion leaders trainings that were implemented as part of the PSI intervention. Another instrument was a brief, 5-question screening questionnaire which determined each respondent s eligibility to participate in the study, and included items relating to the criteria as described in the section on Sample Characteristics, above. Analytic technique The data were analyzed using SPSS (version 15.0). Data was examined through univariate statistics (frequencies, distribution). There were no scales. The monitoring tables present adjusted proportions (adjusting for available socio-demographics: age, living in Bucharest vs. other locations, education level and level of income). For segmentation tables, correlation matrices were examined first; then logistic regression for survey data was used to obtain final models, and the proportions presented in the tables were calculated using the adjusted proportions command, (adjusting for all other variables in the model). - 23 -
Annex 2: Performance Framework for Social Marketing Performance Framework for Social Marketing HEALTH STATUS QUALITY OF LIFE HALO AND SUBSTITUTION EFFECT USE RISK-REDUCING BEHAVIOR RISK COVERAGE, QUALITY, ACCESS, EQUITY OF ACCESS, EFFICIENCY OPPORTUNITY ABILITY MOTIVATION POPULATION CHARACTERISTICS IMPACT, EQUITY AND COST EFFECTIVENESS EXPOSURE SOCIAL MARKETING INTERVENTION PRODUCT PRICE PLACE PROMOTION This study design is guided by PSI s PERForM framework. PERForM describes the social marketing research process, identifies key concepts important for designing and evaluating social marketing interventions and mirrors the four levels and concepts in the logical framework. The top level consists of the goal of social marketing for any health promotion intervention, namely improved health status and/or for interventions relating to coping with sickness or disability, quality of life. The second level consists of the objectives of social marketing stated as product or service use on the left side and/or other risk-reducing behaviours that do not involve the use of a product or service on the right side. The adoption or maintenance of these behaviours in the presence of a given risk or need for health services is causally antecedent to improving or maintaining health and or quality of life. The third level consists of the determinants of PSI Behaviour Change framework summarised in terms of opportunity, ability and motivation that may differ by population characteristics such as age and sex. The fourth level consists of the characteristics of the social marketing intervention. - 24 -