WALES BOWEL CANCER SCREENING PILOT: EVALUATION REPORT PROJECT DATES: FEBRUARY TO APRIL 2015

Transcription

1 WALES BOWEL CANCER SCREENING PILOT: EVALUATION REPORT PROJECT DATES: FEBRUARY TO APRIL 2015 Published August 2016

2 ACKNOWLEDGEMENTS This pilot was developed in partnership with Public Health Wales, and funded and project managed by Cancer Research UK (CRUK). Public Health Wales Screening Division (PHW) shared insight, supported with stakeholder engagement and conducted analysis of the uptake data, with support from Cancer Research UK. The mailed interventions were fulfilled and distributed by DST Output UK, the mailing house contracted to manage the bowel cancer screening invitation process for Public Health Wales. This report was compiled by Becky White, Rosie Hinchcliffe, Helen Clayton, Nick Ormiston-Smith, Kathryn Weir, and Hayley Heard. 2

3 CONTENTS SUMMARY... 4 BACKGROUND... 5 EVALUATION METHODS... 7 RESULTS DISCUSSION CONCLUSIONS END NOTES APPENDICES

4 SUMMARY Cancer Research UK (CRUK) worked in partnership with Public Health Wales Screening Division (PHW) to run a Bowel Cancer Screening (BCS) service improvement pilot across Wales, from February to April This project built upon learning gained from an earlier BCS pilot, which ran in London in It aimed to encourage men and women aged to complete and return their guaiac Faecal Occult Blood Test (gfobt) kit when it is received, by reducing barriers to completion. The primary outcome measure was change in gfobt uptake at 12 weeks after invite (i.e. the date the invitee is sent the NHS test kit). Analysis showed that a personalised CRUK endorsement letter (intervention A) sent three days following the NHS test kit i increased uptake amongst all screening history groups, with the largest impact in those being invited for the first time (9.1% absolute increase). The CRUK endorsement letter plus a kit enhancement pack containing three pairs of gloves (intervention B) also increased uptake in first-timers ii (5.5% increase) and previous nonresponders (2.3% increase), yet it had the greatest impact in more deprived non-responders (3.2% increase), particularly more deprived male non-responders (3.9% increase). The CRUK endorsement letter plus the full kit enhancement pack (containing gloves & poo catcher) (intervention C) increased uptake amongst non-responders (2.4% increase) only. Out of home advertising (posters, pharmacy bags, press articles etc.) on its own (intervention D) did not increase uptake in any of the screening history groups, and uptake among previously screened invitees was significantly lower than controls. However, secondary qualitative research commissioned to further understand these results did not identify any elements of the advertising that discouraged participation, suggesting that the observed decrease in uptake could be caused by factors that could not be accounted for in the uptake analysis. Nevertheless, some potential improvements to the posters were highlighted by focus group participants. In conclusion, the CRUK endorsement letter (A) worked effectively across the full screening population, potentially because it was personalised (making it highly relevant to the individual), and functioned as an additional cue to prompt participation, since it was received following an NHS test kit. Results also suggest that a targeted approach would be more beneficial in helping to reduce inequalities in Wales, by sending the CRUK endorsement letter (A) to first-timers only, and the CRUK endorsement letter plus gloves (B) to non-responders in more deprived groups. i The CRUK endorsement letter was designed to arrive approximately 2-3 days following the NHS test kit ii First-timers were defined as those being invited to screening for the first time, previous non-responders were those who had been invited before but had never returned a used test kit, and previously screened were those who had been invited before and returned a used test kit. 4

5 BACKGROUND Bowel screening in Wales Bowel cancer is the fourth most common cancer in Wales and more than 900 people die from it each year. 1 Bowel screening aims to detect bowel cancer at an early stage when treatment is more likely to be effective. In Wales, a bowel screening guaiac-based Faecal Occult Blood Test (gfobt) kit is sent via post every two years to men and women aged Unfortunately uptake of the test in Wales remains lower than target at around 51%. 2 However, it is estimated that the NHS Bowel Cancer Screening programme will cut deaths from the disease by 25 per cent among those who complete the test at least once 3. By 2025 it is predicted that gfobt will save over 2,000 lives from bowel cancer each year in the UK 4. In Wales, a number of psychosocial barriers to participation have been identified, including: Fear and denial around the test outcome (i.e. preferring not to participate for fear it will be a positive result) 5-8 6, 8-10 Belief that it isn t necessary to complete the test if you don t have any symptoms Belief that the test is difficult, unpleasant or unhygienic 7-11 Poor knowledge of benefits and eligibility 6-12 Various interventions have been shown to increase uptake of gfobt bowel screening. 13 Cancer Research UK (CRUK) and Public Health Wales (PHW) Bowel Screening Programme worked in partnership to pilot four such interventions in Wales. Project aims The primary aim of the project was to increase participation in gfobt by raising awareness of the NHS bowel cancer screening programme (i.e. benefits/eligibility) amongst the eligible population, and reducing barriers to participation: Increase uptake of gfobt by 2-6% iii Raise awareness of the effectiveness of the test, along with benefits (and risks) of the test Reduce beliefs that the test is difficult to complete Interventions An initial pilot project implemented in Greater London (2014) found that a CRUK endorsement flyer (A5) included within an NHS test kit mailing failed to have an impact on gfobt bowel screening uptake. However, when the flyer was combined with either a separate kit enhancement pack mailing, containing gloves and poo catchers (arriving approximately 2 days after the NHS test kit), or with advertising, there were small but significant increases in uptake. When the CRUK flyer, kit enhancement pack, and advertising were all combined together, uptake increased by 6.1 percentage points among year olds, and 7.3 percentage points among year olds 14. iii The estimated increase in uptake was based on evidence from the London pilot. Uptake was expected to vary depending on screening history, age, gender and level of deprivation 5

6 The pilot in Wales aimed to build on these learnings by re-formatting the flyer into a personalised endorsement letter and sending it separately from the NHS test kit (posted 3 days following iv ), with a view to understanding whether an additional contact had more impact. The individual impact of the most effective intervention in London (i.e. flyer, kit enhancement pack (gloves & poo catcher) and advertising) was explored further in Wales by posting three different versions of direct mail (see below) and running advertising alone. The contents of the kit enhancement packs (pack) were also split out to establish the relative cost effectiveness of each. The interventions were: Intervention A: CRUK endorsement letter A4 personalised CRUK endorsement letter, outlining the benefits (and risks) of participating in the NHS bowel cancer screening programme and providing a recommendation from CRUK to complete the test. Posted three days following an NHS test kit with the aim of it arriving 2-3 days following an NHS test kit. Intervention B: CRUK endorsement letter plus pack with gloves CRUK endorsement letter plus x1 kit enhancement pack containing x3 pairs of latex free gloves. Posted three days following an NHS test kit with the aim of it arriving 2-3 days following an NHS test kit. Intervention C: CRUK endorsement letter plus pack with gloves & poo-catchers CRUK endorsement letter plus x3 kit enhancement packs; each containing x1 pair of gloves and x1 poo catcher. Posted three days following an NHS test kit with the aim of it arriving 2-3 days following an NHS test kit. Intervention D: Advertising only Advertising only; consisting of posters on bus stops, posters on phone boxes, posters on the interior, exterior and backs of buses, posters in pubs and shopping malls, on pharmacy bags and press articles in regional press titles. See Appendix 1 for examples of the intervention materials. For intervention costs, see Table 1 (summary) & Appendix 2 (detailed). TABLE 1: SUMMARY OF INTERVENTION COSTS Intervention A Intervention B Intervention C Intervention D Total sample that 9,300 7,300 7,300 11,094 received intervention Total sample included 8,875 6,995 6,981 11,094 i in final analysis Total cost: 7, , , , Inclusions: Set up fees, print production, fulfilment, envelopes, Welsh translations for letters and mailing costs As per intervention A, plus purchase and delivery of kit enhancement packs containing gloves only As per intervention A, plus purchase and delivery of kit enhancement packs containing gloves & poo catchers Design and artwork fees, print production and paid media costs i Sample receiving intervention D was defined at the analysis stage as those people identified as having been exposed to advertising. iv The CRUK endorsement letters were designed to arrive approximately 2 days following the arrival of NHS kits, as NHS kits were sent on a Friday, and the CRUK letters were sent 3 days later on the following Monday. NHS kits would take an extra day to arrive due to the postal service not running on a Sunday, so were likely to arrive Monday or Tuesday, whilst the CRUK letters were likely to arrive the following Wednesday or Thursday. 6

7 EVALUATION METHODS Objectives The final primary outcome measure was uptake at 12 weeks after invitation, amongst those aged who were eligible for the programme. An earlier initial analysis, also referenced in this document, measured uptake at 6 weeks after invitation. The analysis of PHW bowel cancer screening uptake data aimed to: 1. Determine whether the interventions significantly increased gfobt uptake at 6 and 12 weeks after the invitation date, with all other factors (for which there is data) taken into account. 2. Identify which intervention groups had the largest increases in uptake compared to controls, in order to better identify the most effective interventions. 3. Identify demographic groups where the interventions had the greatest impact. Secondary evaluation measures were used to further understand the results. After the uptake analysis, follow-up qualitative research (focus groups) was commissioned to understand the observed impact of advertising on uptake in previously screened invitees, and to identify any improvements that could be made to the advertising. The methods and brief findings are detailed in appendix 12, with insights incorporated into the discussion. Summary of uptake analysis The uptake analysis aimed to identify the impact of the interventions on uptake, by comparing uptake amongst invitees in each intervention group to controls, who received normal care. Logistic regression models were used to predict the probability of screening uptake in each intervention group (A, B, C or D) compared to records in the control group. The models controlled for other demographic variables that were found to affect uptake (deprivation, gender, age, and an ethnicity proxy). Data for some factors known to be associated with bowel screening uptake such as invitee s ethnicity and household size were not available, however an area-based (Lower Super Output Areas) proxy for ethnicity was available for the 12 week analysis. The main effects of each intervention on the probability of screening uptake were modelled separately for the three screening history groups; previously screened, first-timers (those being invited for the first time) and non-responders (those who have been invited before but have not returned a kit). This was due to large differences in underlying uptake as well as differing intervention effects between the three groups. For each of the three stratified models, further subgroup analyses were conducted to compare the impact in the interventions in more deprived (Welsh Index of Multiple Deprivation (WIMD) quintiles 4 & 5) and less deprived (quintiles 1,2 & 3) invitees, and specifically amongst more deprived (quintiles 4 & 5) males. 7

8 Intervention & control group allocation Table 2 gives an overview of the interventions and the dates and areas where they were implemented. Due to the complexity of the campaign, interventions were not randomly allocated on an individual basis, but were allocated by week (see Appendix 3 for a breakdown of the intervention group allocations by week). This approach was suitable for this project because the majority of variation in uptake over time is driven by factors such as screening history and age of the respondent, which could be accounted for at the analysis stage. Posted interventions (A, B and C) were allocated by the week that the recipients were invited. All people invited in the first week of the intervention received that intervention. NHS test kits are sent out once a week, on Fridays. The posted interventions in this pilot were then sent out the following Monday. Where the numbers invited in the first week were lower than the target sample size for the intervention (7,300), invitees from a second overspill week were also sent the posted intervention. The target sample size for each intervention was 7,300. This was larger than the minimum 4,356 cases needed to detect an increase in uptake of at least 3% (see Appendix 4 for power calculations), as we anticipated that up to 40% of cases may need to be excluded at the analysis stage due to contamination by advertising (see page 9 for more information on contamination). However, it was possible at the results stage to account for contamination in the model, and a sensitivity analysis also showed that contamination did not have a major impact on results, so the full sample of up to 7,300 cases were included in the results. The majority of advertising activity ran in Cardiff & Vale and Aneurin Bevan Health Boards from 16 th March to 12 th April 2015, with some bus activity (i.e. posters on buses) continuing for a further two weeks until 26 th April. Some of the advertising (i.e. bus activity) also spilled over into the neighbouring Health Board of Cwm Taf. Intervention D was defined as those people sent kits from Friday 20 th March to Friday 17 th April 2015 (inclusive) in Aneurin Bevan and Cardiff & Vale Health Boards. There was an error on the endorsement letters sent in the first week of intervention A (intervention posted 30/01/15), where addressees first names and last names were in the incorrect order. Intervention A was therefore rerun after the advertising had finished (intervention posted 20/04/15), to ensure an adequate sample of invitees sent the correctly addressed endorsement letters. During the rerun week, the intervention was not sent to those invited in areas that had just received advertising, i.e. Aneurin Bevan, Cardiff & Vale, and the advertising overspill area Cwm Taf. Uptake in each of the intervention groups (A, B, C or D) was compared to uptake in a control group. Invitees were identified as controls if they were sent gfobt kits between Friday 16 th January 2015 and Friday 29 th May 2015 and received normal care (i.e. they did not receive any of the posted interventions (A, B, or C) and were not identified as having been invited in an advertising area during the advertising campaign (intervention D)). 8

9 TABLE 2: INTERVENTION & CONTROL GROUP ALLOCATIONS Intervention group Weekly kit sent dates included 1 Dates intervention mailed/ advertising ran Location Total sample 2 Intervention A CRUK endorsement letter Intervention B CRUK endorsement letter + pack (gloves only) Fri 30 th Jan 15 Fri 6 th Feb 15 Fri 17 th April 15 Fri 13 th Feb 15 Fri 20 th Feb 15 Mon 2 nd Feb 15 Mon 9 th Feb 15 Mon 20 th April 15 Mon 16th Feb 15 Mon 23rd Feb 15 Pan Wales 3 8,875 Pan Wales 6,995 Intervention C CRUK endorsement letter + pack (gloves & poo-catcher) Intervention D Advertising only Controls: All other people sent invites from during study period, who received normal care (i.e. did not receive any of the mailed interventions and were not invited in one of the areas that received advertising during the campaign period) Fri 27 th Feb 15 Fri 6 th March 15 Fri 20 th March Fri 27 th March Fri 3 rd April Fri 10 th April Fri 17 th April Selected invitees from Fri 16 th Jan to Fri 29 th May Mon 2nd Mar 15 Mon 9th Mar 15 Advertising ran from Mon 16 th Mar 15 to Mon 26 th Apr 15 Pan Wales 6,981 2 Health Boards in SE Wales: Aneurin Bevan & Cardiff and Vale 11,094 N/A Pan Wales 67,761 1 Kits are sent each Friday. The call run file (i.e. details of those to be sent a kit) is generated the Friday before. 2 Total sample included in analysis, after exclusions. Includes contaminated cases. 3 Rerun week for intervention A (kits sent 17th April) did not include those sent kits in Aneurin Bevan, Cardiff & Vale, and Cwm Taf. Definition of uptake An invitee has been counted as screened if they return a used test kit within 6 weeks (initial results) or 12 weeks (final results) after the date they are originally sent their kit. This measure (returned a used test kit) is consistent with PHW s definition of uptake, but is different to the definition used in the London Bowel Cancer Screening Campaign improvement pilot. In the London pilot an invitee had to have returned a valid test kit, or have reached a definitive test result of either normal or abnormal, in order to be counted as adequately screened. Further analysis of the London pilot results showed no significant differences in the impact of the interventions if the definition of uptake was similarly widened to include anyone who sent back a test kit, due to the relatively small numbers of people who returned spoilt kits in London. Eligibility and exclusions In Wales, bowel cancer screening kits are routinely sent every two years to eligible men and women aged between years, beginning at their 60 th birthday. Whilst the process to identify invitees is 9

10 the same in England and Wales, it should be noted that there are differences in the subsequent process to identify the eligible population for the purposes of uptake analyses. For this particular analysis, PHW excluded individuals who would not have been able to receive the interventions, such as prisoners, and those who would have been sent a FIT (Faecal Immunochemical Test) kit in this particular round, due to an inconclusive result in their last screening round. These numbers are negligible (only 53 people). In the 12 week analysis, a further 1,055 people were excluded because they met other ineligibility criteria at the analysis stage. 3,761 were then excluded because they could not be linked to a WIMD score. Unfortunately, 14 people were erroneously included in the analysis despite missing Health Board of residence. Variables included in dataset The following variables were included in the dataset; the invitee s age group, gender, screening history, deprivation quintile (Welsh Index of Multiple Deprivation (WIMD)), Health Board of residence, the week they were sent their kit, their allocated intervention group, whether they may have been contaminated by the advertising campaign (i.e. seen the advertising when it was not intended), whether they were eligible, and whether they returned a used test kit within 12 weeks after they were sent their kit, or within 6 weeks for the initial analysis (see Appendix 5 for metadata). Not all variables were included in the final logistic regression models. In the final 12 week analysis, we also included an area level estimate of the individual s likely ethnicity, sourced from the ONS 2011 census. It is a proxy estimate, based on the percentage of residents in the individual s Lower Super Output Area (LSOA) that are white versus non-white. Univariate analysis Univariate analysis identified variables that impacted uptake, including other variables that we would need to account for when modelling the effect of the intervention group. The univariate analysis was conducted at the 6 week analysis, and not repeated at 12 weeks, except for ethnicity, which was added to the model at 12 weeks. See Appendix 6 for univariate results. Multivariate analysis development A multivariate logistic regression model was developed to account for confounding factors that were known to affect uptake (where data was available). Beginning with the intervention variable, other explanatory variables were added to the main model; age group, screening history, deprivation (WIMD quintile), gender, and ethnicity quintile. These variables were included in the final multivariate model because they were significant predictors of uptake after controlling for the other predictors (p <0.05), and were not closely interrelated with other explanatory variables. The invitee s Health Board of residence was not included in the final multivariate analysis, as this would have introduced collinearity with the intervention variable. This is because intervention D was mostly defined by the invitee s Health Board of residence, as the advertising campaign ran in only two Health Boards (Cardiff & Vale and Aneurin Bevan). Week of invitation was not included in the final multivariate analysis. Further descriptive analysis showed that a large proportion of variation in uptake by week could be explained by the proportion 10

11 of invitees each week who had been screened before (as uptake is substantially higher amongst previously screened people than previously unscreened people). As interventions were defined by week, introducing week of invitation as another variable could also have introduced collinearity. Models split by screening history Model development showed that when screening history was added to the model, there were notable changes in the effect of the intervention groups on uptake. Whilst the univariate model for intervention group showed that uptake was higher in intervention groups A, B and D compared to controls, the model with both intervention group and screening history showed no significant differences in uptake between intervention groups and controls, with the exception of A (see Table 3, row: Version 1) All invitees, contamination not accounted for, column: B) Multivariate model ). Screening history is likely to be the dominant driver of variation in bowel screening uptake, and the proportion of people invited who have been screened before varies greatly week by week. This would mask the comparatively smaller effects of the interventions, which were allocated on a weekly basis. The decision was therefore made to stratify the main multivariate logistic regression model, and report on the main effects of the interventions in different screening history groups; previously screened invitees, first-timers, and non-responders. Table 3 shows a sensitivity analysis of the results for the interventions at 6 and 12 weeks, using these different approaches to modelling screening history (see row: Version 1) All invitees, contamination not accounted for, columns: di, dii, diii). Stratification also enabled us to model interactions between screening history and other demographic variables that affect uptake, without the added complexity of introducing interaction variables into the models. For example, there was a different relationship between age and uptake in previously screened invitees, compared to first-timers, and non-responders. There was also an interaction between gender and deprivation occurring in previously screened invitees only. This was omitted from the final model results for consistency between the three stratified models. 11

12 TABLE 3: SENSITIVITY ANALYSIS OF INTERVENTION GROUP RESULTS, BY APPROACH TO SCREENING HISTORY AND CONTAMINATION Rows show results from model versions using different approaches to contamination. Columns show results from univariate & multivariate models: a) without accounting for screening history, b) accounting for screening history, c) accounting for screening history and all other significant predictor variables in the main model, and d) stratifying the main model by screening history. Results are shown both for initial 6 week and 12 week uptake analyses. Intervention group a) Univariate model (intervention group only) b) Multivariate model (intervention group + screening history) c) Main model (intervention group + screening history + all other predictor variables*) di) Stratified model: Previously screened dii) Stratified model: First-timers diii) Stratified model: Non-responders 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks Version 1) All invitees, contamination not accounted for Intervention A 1.36* 1.39* 1.19* 1.24* 1.19* 1.23* * 1.49* 1.46* 1.47* 1.43* Intervention B 1.43* 1.48* * 1.26* 1.45* 1.38* Intervention C * * 1.39* Intervention D 1.08* 1.10* * 0.92* Version 2) Completely excluding contaminated invitees Intervention A 1.36* 1.42* 1.20* 1.28* 1.21* 1.27* 1.11* 1.18* 1.44* 1.45* 1.41* 1.40* Intervention B 1.42* 1.46* * * 1.50* Intervention C * 1.45* Intervention D 1.07* 1.08* *

13 Intervention group a) Univariate model (intervention group only) b) Multivariate model (intervention group + screening history) c) Main model (intervention group + screening history + all other predictor variables*) di) Stratified model: Previously screened dii) Stratified model: First-timers diii) Stratified model: Non-responders 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks Version 3) All invitees, with contaminated invitees moved to a separate contaminated intervention group within the intervention group variable Intervention A 1.36* 1.42* 1.20* 1.28* 1.20* 1.27* 1.11* 1.18* 1.44* 1.45* 1.40* 1.39* Intervention B 1.42* 1.46* * 1.50* Intervention C * 1.46* Intervention D 1.07* 1.08* Contaminated intervention group 1.03* * 1.05* 1.08* 1.08* 1.09* 1.08* 1.09* 1.09* Version 4) All invitees, with contaminated invitees included in original intervention groupings, plus a separate contamination variable Intervention A 1.36* 1.39* 1.19* 1.24* 1.19* 1.23* * 1.49* 1.46* 1.47* 1.43* Intervention B 1.43* 1.48* * * 1.25* 1.45* 1.38* Intervention C * * 1.39* Intervention D 1.08* 1.10* * Separate contamination variable: Contaminated (ref: 0.98 i 0.95* 1.04* * 1.04* 1.09* 1.07* uncontaminated) *P value < 0.05 i Results for version 4 of the univariate model are for the intervention group and contamination variables modelled separately in different univariate models 13

14 Contamination sensitivity analysis During the 6 week analysis a possible contamination was identified, whereby invitees in the posted intervention groups (A, B and C) from Health Boards where outdoor advertising (D) was run (Cardiff & Vale, Aneurin Bevan, and spilling over into Cwm Taf) potentially could have been exposed to the advertising, despite being in another intervention group. Seeing the advertising could have potentially increased their likelihood of returning the kits and therefore impacted the results for interventions A, B, C, and controls. For example, an invitee in intervention group C in Cardiff & Vale who was sent their gfobt kit on 27th February 2015 could have delayed returning their kit until after the advertising campaign began in the area on 16 th March. They could have seen the advertising, and then returned their kit in time to be counted as screened (i.e. within the 6 weeks following the date they were sent the kit). A similar situation applies to those invited in the advertising Health Boards before or after the advertising campaign was run, who would otherwise have been included in the control group. At the 6 week analysis, contamination was accounted for by including contaminated invitees in their original intervention groups, and adding a separate binary contamination variable to account for any additional effect on uptake caused by being contaminated by advertising (see Table 3, model versions 4i, 4ii, and 4iii). However, at the final 12 week analysis, model results showed that when ethnicity was included, the binary contamination variable was not significant in two of the three models. Moreover, ethnicity and the contamination variable were correlated (see Appendix 11). The two were likely collinear, as Health Boards where contamination was possible (those where advertising took place or spilled into Cardiff & Vale, Aneurin Bevan, and Cwm Taf) are also Health Boards with lower percentages of white residents. This suggested that the contamination variable was not working as effectively as intended, and was removed from the final models in favour of the ethnicity variable, despite its limitations as a proxy. Table 3 shows a sensitivity analysis of the possible 12 week models, which all include ethnicity as a predictor variable. Model version 1) used all invitees, including contaminated invitees. In version 2) we re-ran the model but excluded contaminated invitees from the analysis completely. Contaminated invitees were defined as any people who were invited from the two advertising campaign Health Boards (Cardiff & Vale and Aneurin Bevan), unless they were part of intervention D (i.e. sent kits during the advertising campaign weeks from 20th March to 24th April 2015). People invited at any time in the advertising overspill Health Board (Cwm Taf) were also defined as contaminated. Version 3) included all invitees in the model, but the contaminated invitees identified in version 2) were moved to a separate contaminated intervention group. Finally, version 4) also included all invitees in the model, but the contaminated invitees were kept in their original intervention groups as in 1) and 2). Meanwhile, a separate binary contamination variable was included in the model to account for any additional effect on uptake of being contaminated by advertising. Removing the contamination variable had limited impact on the overall results for the interventions, so final 12 week results have used models 1di, 1dii and 1diii. 14

15 RESULTS Final analyses of uptake at 12 weeks after invitation (using models 1di, 1dii and 1diii shown in Table 3) show that, in Wales, the interventions had substantially different impacts in different screening history groups. See Table 4 for results, also Appendix 7 for odds ratios): Overall, intervention A (CRUK endorsement letter) had the largest impact of all of the interventions in each of the screening history groups; first-timers (9.1% absolute increase),previous non-responders (2.6% increase), and previously screened (1.4% increase). Intervention B also increased uptake in first-timers (5.5% increase) and non-responders (2.3% increase). Further segmentation of results showed that amongst more deprived non-responders, intervention B (CRUK endorsement letter + kit enhancement pack with gloves) was the intervention with the largest impact on uptake (3.2% increase) (see Appendix 8 & 9 for odds ratios). The impact of intervention B was larger still amongst more deprived non-responders who were male (3.9% increase) (see Appendix 10 for odds ratios). Intervention C increased uptake in non-responders (2.4% increase) only. Intervention D (advertising only) did not increase uptake in any of the screening history groups, and uptake was significantly lower in intervention D amongst previously screened invitees (1.6% decrease). TABLE 4: INITIAL 6-WEEK AND FINAL 12-WEEK UPTAKE RESULTS AMONGST YEAR OLDS, BY SCREENING HISTORY GROUP Intervention Previously screened First-timers Non-responders 6 Weeks 12 Weeks 6 Weeks 12 Weeks 6 Weeks 12 Weeks Modelled uptake in controls 70.1% 76.7% 32.3% 36.1% 5.9% 6.6% Percentage point increase in uptake compared to controls: Intervention A CRUK endorsement letter Intervention B CRUK endorsement letter + pack (gloves) Intervention C CRUK endorsement letter + pack (gloves & poo-catcher) Intervention D Advertising only +1.0% +1.4%* +9.3%* +9.1%* +2.5%* +2.6%* -1.5%* -0.9% +5.3%* +5.5%* +2.4%* +2.3%* -1.0% -1.2% +3.6%* +3.1% +2.0%* +2.4%* -1.5%* -1.6%* +1.1% +1.9% 0.2% +0.7% Table shows the modelled underlying uptake amongst controls, followed by the absolute (percentage point) increase in uptake in each intervention group compared to controls. * indicates that the difference was statistically significant at a 95% confidence level. 15

16 DISCUSSION Results suggest that a personalised CRUK endorsement letter (A) can increase gfobt uptake across all screening history groups in Wales. It was particularly effective amongst those being invited for the first time (9.1% absolute increase), potentially by providing this group with further information and context, and an additional prompt to complete the NHS test kit. CRUK endorsement of bowel screening was substantially more successful in Wales than in the previous pilot in London, where it was sent as a flyer in the same mailing as the NHS FOBT kit, rather than as a separate personalised letter. The flyer may have been less noticeable than the letter as it was included in the same mailing as the NHS FOBT kit alongside the other materials routinely sent 10. It should also be noted that at the time in Wales, invitees did not receive pre-invitation letters before the actual NHS test kit, as was the case in London. This could increase the value of adding another point of contact or reminder to the process in Wales, however further research is needed to check whether the CRUK endorsement letter is still effective in other contexts, e.g. where people are sent pre-invitation letters. The CRUK endorsement letter plus gloves (B) increased uptake amongst first-timers (5.5% absolute increase) and non-responders (2.3% absolute increase). Amongst non-responders from deprived areas, it was also the intervention with the largest impact on uptake (3.2% absolute increase). The gloves could therefore be potentially addressing some of the barriers to completion of the test for people who have not completed it before; such as perceptions that handling faeces will be unpleasant or disgusting. 3 Further research could confirm and explore how the enhancement pack with gloves may address barriers specific to more deprived non-responders. The CRUK endorsement letter plus gloves and poo catchers (C) increased gfobt uptake amongst non-responders only (2.4% absolute increase), and did not have a bigger impact than sending the CRUK endorsement letter plus gloves only (B). This suggests that including the poo catchers in the kit does not have an additional impact on invitees. Advertising alone (D) did not increase uptake in any screening history group, and uptake was lower amongst previously screened invitees in the advertising intervention group compared to controls (1.6% absolute decrease). Follow-up qualitative research was commissioned by CRUK and conducted by Beaufort to further investigate this (see appendix 12 for more information). Focus groups with previously screened participants (as well as with non-responders and those yet to be invited, i.e year olds) did not identify any elements of the advertising that discouraged participation. However, some suggestions were made for improvements to the posters: A more direct, personal call-to-action that goes further to addressing barriers, particularly amongst non-responders (e.g. worry about the results, or feelings that the kit will be unpleasant to complete). However, these improvements would need to be considered alongside the need to support an individual s informed choice. A focus on communicating essential information about the kit and programme to raise awareness and knowledge (requested by those yet to be invited). A possible reason for the observed decrease in uptake amongst previously screened invitees in the advertising intervention group could be due to factors that could not be accounted for in the analysis. Controls were sourced from across Wales, whereas the advertising ran only in Aneurin Bevan and Cardiff & Vale Health Boards. Whilst other factors such as deprivation, ethnicity (proxy), gender and age were controlled for in the analysis, it is possible that the characteristics of people living in these Health Boards may further differ from the rest of Wales (e.g. proficiency in English, social grade, education, differences between urban and rural services). Furthermore, whilst the effect of contamination of controls by advertising over time has been examined, the effect of 16

17 contamination over space has not been controlled for (i.e. controls from other parts of Wales may have travelled into Cardiff or Newport and been impacted by advertising). It is important to note that other local contexts can also influence the effectiveness of out of home advertising campaigns; such as the number of opportunities for adverts (e.g. number of bus stops), population density and commute patterns, newspaper readership, and saturation of the advertising landscape by competing campaigns. 17

18 CONCLUSIONS The CRUK endorsement letter (A) was the most effective intervention, increasing uptake across all the screening history groups. The results suggest that a targeted approach would be most beneficial in helping to reduce inequalities in Wales: The CRUK endorsement letter (A) could be sent to first-timers only, as it had a large impact in this group (+9.1%). The CRUK endorsement letter plus gloves (B) could be sent to non-responders in more deprived groups, as it had the largest impact of any intervention in this hard-to-engage group (+3.2%). Improvements could also be made to the CRUK advertising materials (D); for example by striving to focus messaging towards the information needs of first timers and non-responders specifically. 18

19 END NOTES 1. Based on the annual average number of bowel cancer (ICD10 C18-20) cases (2013) and deaths (2014) in Wales 2. Bowel Screening Wales Annual Statistical Report, Hewitson P et al (2007) Screening for colorectal cancer using the faecal occult blood test Hemmoccult. Cochrane Database System Review. 4. D. M. Parkin, P. Tappenden, A. H. Olsen, J. Patnick, and P. Sasieni (2008) Predicting the impact of the screening programme for colorectal cancer in the UK, Journal of Medical Screening, vol.15, no. 4, pp Miles, A., Rainbow, A. & von Wagner, C. (2011) Cancer fatalism and poor self-rated health mediate the association between socioeconomic status and uptake of colorectal cancer screening in England. Cancer Epidemiol Biomarkers Prev. 6. Palmer, C., Thomas, M., Von Wagner, C. & Raine, R. (2014) Reasons for non-uptake and subsequent participation in the NHS bowel cancer screening programme: a qualitative study. BJC. 7. Ekberg M, Callender M, Hamer H & Rogers S (2014) Exploring the decision to participate in the National Health Service Bowel Cancer Screening Programme. Eur J Cancer Prevention. 8. Chapple, A., Ziebland, S., Hewitson, P. & McPherson, A. (2008) What affects the uptake of screening for bowel cancer using a faecal occult blood test (FOBt): a qualitative study. Soc Sci Med. 9. O Sullivan, I. & Orbell, S. (2004) Self-sampling to reduce mortality from colorectal cancer: a qualitative exploration of the decision to complete a faecal occult blood test (FOBT). Journal of Medical Screening. 10. Hall, N., Rubin, G., Dobson, C., Weller, D., Wardle, J., Ritchie, M. & Rees, C. (2013) Attitudes and beliefs of non-participants in a population-based screening programme for colorectal cancer. Health Expectations. 11. von Wagner, C., Good, A., Smith, S. & Wardle, J. (2011) Responses to procedural information about colorectal cancer screening using faecal occult blood testing: the role of consideration of future consequences. Health Expectations. 12. Bennett, K., von Wagner, C. & Robb, K. (2015) Supplementing factual information with patient narratives in the cancer screening context: a qualitative study of acceptability and preferences. Health Expectations. 13. Cancer Research UK: Evidence on increasing bowel cancer screening uptake B. White, E. Power, M. Ciurej, S. Hing Lo, K. Nash, N. Ormiston-Smith (2015) Piloting the Impact of Three Interventions on Guaiac Faecal Occult Blood Test Uptake within the NHS Bowel Cancer Screening Programme. BioMed Research International, Article ID

20 APPENDICES Appendix 1: Examples of intervention materials Intervention A: A4, 2pp endorsement letter, sent 3 days following the NHS test kit Intervention B: CRUK endorsement letter + x1 kit enhancement pack containing x3 pairs of latex free gloves 20

21 Intervention C: CRUK endorsement letter + x3 kit enhancement packs, each containing x1 pair of gloves and x1 poo catcher Intervention D: Advertising only 21

22 Appendix 2: Detailed breakdown of intervention costs Invoices 1 Invoice costs Intervention A: CRUK endorsement letter Intervention B: CRUK endorsement letter plus pack with gloves Intervention C: CRUK endorsement letter plus pack with gloves & poocatchers Intervention D: Advertising only Set up costs Design costs - updates to OOH creative, press and letters 2 6, , Additional layout and artwork of bus formats 1, , Additional layout and artwork of bus formats - extra Welsh translations Ongoing costs A3 & A4 posters Mediacom (press) 5, , Mediacom (press) 4, , Mediacom (bus panels) 8, , Mediacom (washrooms) 4, , Mediacom (6s & 48s) 38, , Set up, testing fees and print production of direct mail 9, , , , Full kits (gloves + poo catchers) x5,500 17, , Gloves only kits x5,500 5, Extra 6000 poo catchers 1, Additional full packs (gloves & poo catcher) and packs with gloves only (2000 of each type of pack = 4,000 packs in total) 3 1, Additional production & fulfilment associated with intervention A, week two re-print (2000) 4 1, ,

23 OS C5 envelopes (24,500) 1, Additional C5 envelopes (7000) Destroying incorrect intervention A mailings & re-print of week two letters Courier Postage (split across interventions is estimated) 5 6, , , , Total 108, , , , , Costs per engagement (for sample included in final analysis) Total sample that received intervention (i.e. mailed direct mail or estimated to have been exposed to advertising) 9,300 7,300 7,300 11,094 Total sample included in final analysis 6 8,875 6,995 6,981 11,094 1 Note the above table excludes evaluation costs 2 This is an estimated split of design costs across the interventions 3 Packs with gloves & poo catchers will have cost more than packs with gloves only. Exact cost breakdown unknown. 4 Excluded from cost effectiveness analysis as costs are due to an error 5 This is an estimated split of postage costs across the three interventions 6 Number for intervention D does not differ to no. receiving intervention, as the group identified as having been exposed to advertising is an estimate established at the analysis stage 23

24 Appendix 3: Intervention allocations by week Week number in analysis Call run (Friday) Kits sent (following Friday) Total invitees sent kit in week Intervention group allocation Date mailed intervention/ advertising ran Notes 8527 Control week NA No invitees sent interventions /01/ /01/ /01/ /01/ Control week NA above /01/ /01/ Intervention A (error week) i /01/ /02/ Intervention A + controls /02/ /02/ Intervention B /02/ /02/ Intervention B + controls /02/ /02/ Intervention C /02/ /03/ Intervention C + controls 02/02/2015 All invitees sent intervention A 09/02/2015 Additional invitees sent intervention A to reach sample size. All other invites were controls. 16/02/2015 All invitees sent intervention B 23/02/2015 Additional invitees sent intervention B to reach sample size. All other invites were controls. 02/03/2015 All invitees sent intervention C 09/03/2015 Additional invitees sent intervention C to reach sample size. All other invites were controls /03/ /03/ Controls NA No invitees sent interventions /03/ /03/ Intervention D (advertising) + controls /03/ /03/ Intervention D (advertising) + controls /03/ /04/ Intervention D (advertising) + controls /04/2015 (due to bank holiday) 10/04/ Intervention D (advertising) + controls 16/03/ /04/15 16/03/ /04/15 16/03/ /04/15 16/03/ /04/15 Advertising in ABU & CVUii. Invites in all other Health Boards were controls. As above As above As above 24

25 338 10/04/ /04/ Intervention A rerun/ Intervention D (advertising) + controls /04/ /04/ Intervention D (advertising) + controls Intervention A sent on 20/04/15. Advertising 16/03/ /04/15 in ABU & CVU 16/03/ /04/15 Additional invitees were sent intervention A to reach sample size, due to an error in the first intervention A week (kits sent 30/01/15).i Intervention A was not sent to ABU, CVU Health Boards, where advertising ran, and CTUii where there was some overspill of advertising. Those sent kits in ABU & CVU Health Boards were allocated to intervention D (advertising) as with previous weeks. Advertising in ABU & CVUii. Invites in all other Health Boards were controls /04/ /05/ Controls NA No invitees sent interventions /05/ /05/ Controls NA No invitees sent interventions /05/ /05/ Controls NA No invitees sent interventions /05/ /05/ Controls NA No invitees sent interventions /05/ /05/ Controls NA No invitees sent interventions i There was an error on the endorsement letters sent in the first week of intervention A (intervention mailed 02/02/15), where addressees first names and last names were in the incorrect order. Intervention A was therefore rerun after the advertising had finished (intervention mailed 20/04/15), to ensure an adequate sample of invitees sent the correctly addressed endorsement letters. ii Aneurin Bevan (ABU), Cardiff & Vale (CVU), Cwm Taf (CTU). 25

26 Appendix 4: Power calculations for intervention target sample sizes Calculations are based on this online tool: Calculations assume a baseline uptake of 50%. If the impact of the campaign were to be X% with significance of 5% and a power of 80% and a two sided test then the sample size would need to be: Baseline uptake Difference aiming to see Uptake after the campaign Minimum sample size required 50% 1% 51% % 3% 53% % 3.25% 53.25% % 5% 55% % 7% 57% % 10% 60% 388 Therefore in order to detect a 3% increase in uptake it is necessary that each intervention group has a minimum of 4,356 observations. 26

27 Appendix 5: Metadata for variables included in dataset Variable name Type Description Categories AgeInvite Integer Individual s age at invite (i.e. when sent kit). 0: 60 Max value: 74 ageinvite_cat_3 Categorical As above 0: : : gender_cat Categorical Gender 0: M (Male) 1: F (Female) 2: U (Indeterminate) sxhistory_incid ent Binary Screening history of individual whether they have been adequately screened before sxhistory_cat Categorical Screening history of individual previously screened, first-timer, non responder HB_cat Categorical Individual s Health Board of residence when invited WeekRecall Integer The week that the individual was sent a kit 0: Prevalent (never been screened before) 1: Incidence (previously screened at least once) 0: I (Incident i.e. previously screened at least once) 1: P1 (First Prevalent Round i.e. first-timer) 2: P2 (Subsequent Prevalent Round i.e. non-responder) 0: ABMU (Abertawe Bro Morgannwg) 1: ABU (Aneurin Bevan) 2: BCU (Besti Cadwaladr) 3: C&VU (Cardiff& Vale) 4: CTU (Cwm Taf) 5: HDU (Hywel Dda) 6: PT (Powys) 7: Unknown 0: 325 (first week included in data, i.e. kits sent 16/01/2015 Max value: 344 (final week included in data, i.e. kit sent 29/05/2015) INT_cat Categorical Allocated intervention group 0: Control 1: Intervention A 2: Intervention B 3: Intervention C 4: Intervention D (if living in C&VU or ABU and invited between weeks 333 and 339 i.e. the advertising period) Contaminated_ cat_hbonly WIMDscore Binary Flag to state whether the individual was sent kit at a time and place when they could have been contaminated (i.e. exposed to advertising). Individuals allocated to intervention D are not marked as contaminated. Variable performs similar function to intervention_contam. Continuous Area level estimate of the individual s Welsh Index of Multiple Deprivation (WIMD) rank, based on their LSOA (Lower Super Output Area) of residence. imd_quintile Categorical Area level estimate of the individual s Welsh Index of Multiple Deprivation 0: No (not contaminated) 1: Yes (contaminated - any case or control living in CTU, or invited from C&VU or ABU and not invited between weeks 333 and 339). 1: Most deprived LSOA in Wales 1897: Least deprived LSOA in Wales 1: Rank 1517 to 1897 (least deprived quintile) 27