Being Evidence Based. Ruth E Mann, PhD ATSA 2018

Transcription

1 Being Evidence Based Ruth E Mann, PhD ATSA 2018

2 What gives me the right?

3 Evidence based rehabilitation in prisons What does evidencebased mean? The evidence base for programmes that treat sexual offending Getting the context right: Rehabilitative culture in prisons

4 1. What does it mean to be evidence based?

5

6

7

8 Best available research 1. Randomized controlled trials 2. Quasi experiments 3. Systematized clinical observation 4. Consensus among recognized experts 5. Opinion Clinical expertise competencies Assessment, formulation and treatment planning Implementation and monitoring Interpersonal expertise Continual self reflection Use of research evidence Understanding the influence of individual and cultural differences Seeking consultation and alternative services Having a cogent rationale for clinical strategies.

9 Continual self reflection The ability to reflect on one s own experience, knowledge, hypotheses, inferences, emotional reactions and behaviours, and to use that reflection to modify one s practices. A recognition of the heuristics and biases both cognitive and affective that can affect clinical judgement. Taking explicit action to limit the effects of those biases

10 A cogent rationale for clinical strategies Knowledge of relevant research and the organization provided by theoretical conceptualizations [Especially for] problem constellations, patient populations and clinical situations for which treatment evidence is sparse

11 It sounds so straightforward!

12 But have you ever noticed that observers of a sports event may describe it very differently depending on what team they favour?

13 Confirmation bias: A common threat to evidence based practice

14 Confirmation bias means Being less receptive to counter indicative information than to supportive information. Overweighting the confirmatory evidence and underweighting the negative disconfirmatory evidence. Requiring less hypothesis consistent evidence to accept a hypothesis, and more hypothesis inconsistent evidence to reject a hypothesis By the way it also happens in clinical practice

15 Combating confirmation bias Typically there are several plausible hypotheses to account for a specific observation. What would happen if we stated all the alternative hypotheses and sought to confirm them as lawyers do? 1. State your hypothesis 2. State the disconfirming or alternative hypothesis/es 3. Force yourself to give equal attention to finding evidence for the alternative hypotheses Also remember the base rates

16 Correctional quackery another threat to evidence based practice Note: the term does not necessarily imply wilful rejection of research findings. In many cases where people advocate strongly for programs, we don t actually know if they work or not

17 When intuition goes wrong There are several examples of interventions or projects where policy makers believed strongly that the intervention would work, but where research found the effect was the opposite to intended. That is, the intervention increased reoffending rather than decreased it. These examples teach us that we need to be cautious in assuming that our intuition or common sense is always correct.

18 Cambridge Somerville Youth Project Even rebellious youth from ghastly families could be steered away from a delinquent career through consistent emotional support, friendship and timely guidance. Boys at risk of delinquency were identified and randomly allocated to caseworker support or no intervention. Caseworkers visited on average twice a month for over five years. Boys & their families received friendly guidance, parental counselling, practical assistance, specialist referrals, encouragement to join youth groups and participate in creative & sporting activities, assistance with finding jobs, academic tutoring, and opportunities to attend summer camps. The boys were largely identified as having become adjusted.

19 Cambridge Somerville What happened at the end of the Project? Almost equal numbers of control and treatment group did better than anticipated. A slightly larger number of boys in the treatment group had been in court and charged with a slightly larger number of offences.

20 Cambridge Somerville Project: What was happening by middle age (98%)? Two thirds said the project had helped them. They gave specific examples and testimonials, said the project had put them on the right track, and had improved their values. Those who had been in the treatment group were more likely to have been convicted for crimes, had died an average of five years earlier, and were more likely to have been diagnosed with serious mental health problems.

21 Cambridge Somerville Project: What about the boys who benefited most? 38 boys were identified as having benefited the most 22 were no better or worse than their matched controls 4 were better 12 were worse

22 Another example: Juvenile Awareness Programs Scared Straight and other juvenile awareness programmes are based on the premise that exposing teenagers to the consequences of crime for them will deter them from committing crime. Typical programme components included visiting/being locked in a cell, discussion sessions with prisoners, some of which were very confrontational.

23 9 Randomised Controlled Trials. The projects varied in how confrontational they were.

24 Outcomes of juvenile awareness programmes Interviews and surveys of participants and their parents and teachers indicated unanimous support for the programmes. None of the programmes prevented delinquency. The prisoners who took part were also very positive about the programmes. One of them (SS) increased it, and another increased it for the most at risk youth despite apparently improving attitudes towards crime. The highly positive feedback from all those involved, and apparent attitudinal change, was therefore misleading.

25 Why is participant feedback misleading? When it comes to evaluating the effectiveness of an intervention, I advocate a don t ask, can t tell policy researchers should not assess the impact of a program by asking people how much they benefited from it.

26 Nickerson: [A robust body of evidence] has revealed a tendency for people to respond to questions in a way that in effect acquiesces to the hypothesis the interrogator is entertaining Responders may inadvertently provide evidence for a working hypothesis by behaving in a way that is consistent with that assumption. Wilson: Perhaps they knew that the [program] didn t help much but they did not want to rain on the researchers parade.

27 Wilson: There is a more fundamental issue: they often don t know the answer We develop theories about the causes of our feelings, but few of us are perfect at knowing exactly why we feel the way we do, because life is not a controlled experiment in which only one thing varies at a time. To answer this question, they would have to know how they would be feeling if they had not participated in the program.

28 Note that the caution about asking people applies only to attempts to determine outcome. Otherwise, according to Wilson: Asking people about their feelings, attitudes, opinions and knowledge can be quite valuable.

29 Heraclitus of Ephesus, weeping philosopher: No man steps in the same river twice One might be tempted to conclude that forensic systems worldwide are highly innovative places, constantly evolving with the aim to prevent criminality as much as possible (Schmidt & Mann, 2018)

30 Evidence Based Cognitive Skills Programs Hierarchical therapeutic communities for substance misusers Procedural Justice Education in prison Correctional Quackery Juvenile Awareness Programmes Zero Tolerance approaches Drug and alcohol awareness We don t know (untested) Sports Greening programs Victim awareness programs Mindfulness Animals Arts in prison

31 2. Programs that treat sexual offending

32

33 This study covered 12 years of treatment delivery and involved a very large sample size The target group in the this study were 2,551 men who had started CORE in prison between 2000 and 2012 Their reoffending outcomes were compared with the outcomes of 13,219 men with sexual index offences who were not recorded as attending

34 The study achieved a strong match between treatment and control PSM Matching The study achieved a strong match on 80+ variables but not motivation or ability to engage, denial, and dynamic measures of deviant sexual interest. Just 28% of CORE had full risk factor analysis for matching. Both binary and frequency reoffending outcomes Different types of sexual offending at both index and reoffence Sensitivity analyses outcomes for different sets of participants

35 Results Over an average follow up of 8.2 years the overall binary reoffending rates for everyone in the study (unweighted) were 38.3% (any offence) and 7.5% (any sexual offence excluding breach). The sexual reoffending rate (excl breach) for those 2,551 men study who started the Core SOTP in prison between 2000 and 2012, after an average of 8 years, was 10%. That is, 255 men were convicted for a sexual reoffence. The comparable rate for the matched comparison group was 8% a 2 percentage point difference/ 25% increase. This difference could alternatively be expressed as 90% survival rate versus a 92% survival rate. There was no significant difference in the sexual reoffending of the two groups until 5 years after release

36

37 Who got worse? This overall difference seems to be largely driven by child image reoffending (Table A8 ) which was the only sub type of sexual reoffending where the difference between the groups was statistically significant. There were no differences in the rates of adult serious or child contact reoffending.

38 As our clients know only too well, it is difficult to explain a bad outcome without justifying or excusing it Selfabasement Sorrow & making good Justifying & Intellectualizing

39 What went wrong? The research? The design? The implementation? The context? Compare: SOTP, SOTEP, Progams with best outcomes

40 The design? Focus on offence accounts? Victim empathy? Lack of skills practice? Lack of focus on sexual interests? Overly manualised? Lack of individualisation?

41 The implementation? Lack of agency? Use of paraprofessional therapists? Poor programme integrity? Dose? Timing? Throughcare/ resettlement?

42 The context? Target driven organisational culture? Rehabilitative prison culture?

43 Or were we seduced by confirmation bias? Usual hypothesis treatment works Null hypothesis it doesn t work Alternative hypothesis treatment makes people worse Alternative hypothesis treatment isn t necessary

44 Schmucker & Losel, 2017 Sex offending outcomes

45 Sexual offence prevention is a field where a) the emotional involvement of the public is high, b) the felt need for politicians and policymakers is pressing, c) the evidence for effective interventions is rather weak, absent, or even indicates some detrimental effects, d) empirical research is particularly difficult due to low (and potentially shrinking) recidivism rates (Hanson et al., 2016; Mews et al., 2017), e) while the most prolific treatment theory (RNR) indicates that treating low risk people is largely ineffective (or even detrimental) (Bonta & Andrews, 2017) f) RNR principles are still largely ignored in routine practice (McGrath et al., 2010; Bonta & Andrews, 2017) We do not criticise those who bow to pressures a) and b); instead we have to sympathise with the dificulty of knowing what the right or best thing to do is, given the problems of c) to e)

46 3. Getting the context right

47 Hierarchical components of a rehabilitative prison Re settle Address attitudes & thinking Address drug & alcohol problems Rehabilitative culture Safety, Decency, Procedural Justice

48 A Rehabilitative Culture is the ideal prison climate for rehabilitative activities and programmes to flourish and have maximum benefit. A Rehabilitative Culture also enables identity change in its own right. A Rehabilitative Culture makes a prison safer as well as reducing reoffending.

49 Rehabilitative culture: the importance of supportive and encouraging custodial staff A study across four German prisons found significant correlations between the extent to which the prisoners found the custodial staff to be supportive and the extent to which they felt able to participate in therapeutic programmes. A study in the Netherlands found that the most salient predictor of many positive outcomes was custodial staff having a motivational treatment style stimulating prisoners to take part in therapy and plan for their futures.

50 Rehabilitative culture for men convicted of sexual offending: Dr Nick Blagden s research Mixed methods case studies of three prisons plus a qualitative longitudinal study: The crucial importance of relational dynamics between prisoners and staff in the desistance process as they enable recognition, validation and reflection of personal change The biggest impediment seems to be relational ambivalence for staff, this dialectical between them being compliant prisoners who they need to work with vs personal disgust. Also perceived unfairness. The chain of connection is reciprocity, relationships, other centeredness and a sense of ownership over their own rehabilitation There has to be the latitude to construct viable identities, but importantly there needs to be the opportunity to enact these new selves, to live them, like behavioural experiments. Having that validation, the recognition of change is so important

51 There needs to be the opportunity to enact these new selves, to live them, like behavioural experiments

52

53 The take home points Being evidence based is harder than it sounds. The science is not good enough. We have to disentangle intuition from evidence. Confirmation bias is everywhere you look. We all want to believe that treatment works. Don t stop believing but remember to do the confirmation bias thought experiments. Prisons are often places of pain. Let s see if we can make them places of hope.

54 Thank you for