The Effectiveness of ASAP Education and Rehabilitation Programs

The Effectiveness of ASAP Education and Rehabilitation Programs James L. Nichols 1 EDUCATIONAL PROGRAMS B ackground and D escription Educational programs for convicted drinking drivers have recently received much attention in both the United States and Canada. Such programs are based on the premise that knowledge concerning alcohol will reduce the frequency of driving after using excessive amounts of alcohol. Thus educational programs are, by definition, aimed primarily at transmitting information. In actuality, such programs differ considerably in their content and approach. On the average, however, such a program will constitute 8-12 hours of instruction over a 2-4 week interval and will cover such subjects as: (1) the physiological effects o f alcohol; (2) the social and psychological effects of excessive alcohol use; (3) legal aspects of drinking and driving; and (4) alternatives to drinking and driving behaviour. Such programs have been implemented on a more extensive scale than that on which they have been evaluated. In the United States, where more than a quarter of a million persons have been assigned to attend such schools as part of the Alcohol Safety Action Projects (ASAPs) many of such efforts have not been rigorously evaluated. This is even more so the case with non-asap DWI education programs in the U.S.A. and the situation is similar in other nations im plem enting alcohol education programs for drinking drivers. Even when valid experimental designs have been developed for evaluating educational programs, local opposition to procedures such as random assignment and no-treatment control groups has resulted in compromised and equivocal results. Another major problem which plagues such evaluation studies is the lack of adequate numbers of persons in both the education and no-education groups so that small effects could be documented if in fact they existed. P roject L evel Evaluation o f ASAP E ducational Programs The criteria for evaluating the effectiveness of ASAP alcohol education programs have ranged from (1) increased knowledge levels and (2) improved attitudes towards drinking and driving to (3) reductions in alcohol-related violations a n d/o r (4) changes in subsequent crash involvement for persons exposed to such programs. Changes in knowledge or attitude levels appear to be the most easily and frequently used criteria for measuring the effects of these programs. With regard to changes in knowledge levels, thirty-one studies of such ASAP school effects were submitted to the U.S. Departm ent of Transportation in the three-year period from 1972 through 1974. These studies were primarily of the before-and-after variety in which paper and pencil tests designed to measure knowledge about alcohol and its effects were adm inistered before and after persons were exposed to an educational program. Some studies included control groups, who were not exposed to the programs but who were given pre-post tests National Highway Traffic Safety Administration, U.S. Department of Transportation. 622

The Effectiveness o f ASAP Education and Rehabilitation Programs 623 during sim ilar intervals. Virtually all o f the studies received indicated that statistically significant im provem ents in k now ledge levels resulted from persons being exposed to the ed u cational programs. W hile som e studies were m ethodologically less sound than others, it appears safe to conclude that educational program s can change the level o f know ledge con cerning alcohol am ong a group o f persons exposed to them. A similar, yet som ewhat more equivocal, situation exists with regard to the effects o f such programs on attitudes towards drinking and driving. Over the same three-year period, at least tw enty-one such studies were conducted in various ASAP localities. Seventeen (81 percent) o f these before-and-after studies reported im proved attitudes towards drinking and driving follow ing exposure to the programs. One primary reason why these studies are less interpretable than the know ledge-orientated studies is that most o f the persons being tested probably assumed that they were still under the control o f the court and were understandably prone to say what they perceived the program officials wanted to hear. A nother major problem involves the inherent difficulty involved in defining and m easuring attitudes. Som ew hat m ore closely related to the ultim ate program objective o f reducing crash in volvem ent is the m easurem ent o f im pact in terms o f reductions in subsequent alcohol-related violations. In order to adequately assess such program impact, the post-program violation records o f convicted drinking drivers w ho are exposed to educational programs are com pared with the subsequent records o f persons not so exposed. O bviously, the group o f individuals sent to the education program should have the sam e pre-program probability o f rearrest as control persons not so assigned. O ne way to obtain this equality is by randomly assigning these persons to the education and no-education groups. Few o f the studies available to date have utilised such rigorous procedures. As an alternative, local evaluators have som etim es attempted to com pensate for the lack o f random assignm ent by com paring the profiles o f education and no-education groups relative to variables know n to be related to subsequent rearrest probabilities (e.g. prior arrest records) and then have attempted to control for any observed differences by statistical m eans. M ore often than not, how ever, significant d ifferences in the initial characteristics o f the education and no-education groups have not been sought out and thus have not been controlled for. From 1972 through 1974 there were fifty-four analytic studies conducted on the effects o f ASAP educational schools in terms o f reduced violations for the exposed groups. A central A SAP evaluator reviewed these analytic studies with regard to their m ethodology as well as their results. The studies were then categorised as being either m ethodologically weak (w) or m ethodologically sound (s). As Figure 1 illustrates, thirty-six (67 percent) o f these studies were judged by the central evaluator as being m ethodologically weak. O f these studies, sixteen (44 percent) reported that the groups exposed to the educational programs had fewer subsequent alcohol-related driving arrests than did the non-exposed control groups. O f the eighteen studies judged as being m ethodologically sound, four (22 percent) reported positive results for the educational program. The finding that the num ber o f positive findings are indirectly proportional to the am ount o f experim ental control exercised, has been previously reported with regard to evaluations in the general alcohol rehabilitation area. Thus in the case o f drinking driver educational programs, approxim ately one-fifth o f the well controlled studies have reported beneficial effects potentially attributable to the program. A few studies have reported negative program effects. Studies which have attem pted to assess the effectiveness o f alcohol education schools in terms o f reducing crashes have been m ethodologically similar to those which have exam ined subsequent arrests. Thus the problem s and the steps taken to rem ove the effects o f such problems have also been similar. As with violation studies, adequate steps have only infrequently been taken to counteract potential biases. In the ASAP experience twelve crash studies were received from 1972 to 1974. Three (25 per cent) o f these studies were judged as being suitably controlled. N one o f the controlled studies reported statistically significant reductions in subsequent crashes for the educational groups. Sm all sam ple sizes were ob viou sly an inherent problem confronting these studies.

624 J. L. Nichols 20-19- 18-17- 16-15- 14-1 J POSITIVE RESULTS 1 NO EVIDENCE OF EFFECT W - WEAK STUDIES S - SOUND STUDIES to 13' UJ 5 12H w m oio- 9 -i m I 8- z 7-6 - 5-4- 3-111 I 8 2-1- 11 w s 1972 W 1973 s W S 1974 Figure 1 Effects o f ASAP schools in terms o f reducing alcohol related arrests. Program L evel E valuation Findings Prior to being directed to attend alcohol education programs (or any other rehabilitation program) most clients receive a diagnostic exam ination intended to classify them as either a problem or a non-problem (social) drinker. W hile tim e and space do not permit a description o f this process, it can be reported that persons d iagnosed as problem drinkers have characteristically had twice the num ber o f subsequent alcohol related driving arrests as those diagnosed as n on-problem drinkers. In order to evaluate the effectiveness o f education programs across a number o f project sites, each project subm itted to the U.S. D epartm ent o f Transportation a quarterly tabulation (for each drinker type) o f the proportion o f persons who were assigned to various education and non-education m odalities at their site. Included in these tabulations was the proportion o f persons in each group who were subsequently re-arrested for alcohol related driving violations. This inform ation was then aggregated and analysed by the central ASAP rehabilitation evaluator.1 In addition to the above data, the central evaluator collected inform ation from the projects concerning various m easurable characteristics o f their schools (and therapy procedures). Som e o f these characteristics included: (a) the proportion o f program time spent lecturing; (b) the proportion o f tim e the client spends actively participating; (c) program

The Effectiveness o f ASAP Education and Rehabilitation Programs 625 length; and (d) the number of clients in each session. With this information, the central evaluator was able to conduct a series of factor and cluster analyses in order to classify the various educational schools along a meaningful dimension ranging from relatively interaction (or therapy) oriented (type 1) schools to more lecture-oriented (type 3) schools. A comparison was then made between the various school types regarding the rearrest rates of persons exposed to them. These comparisons were made for both problem drinkers and non-problem (social) drinkers. Early results of these efforts were reported previously by Nichols and Reis.1 Since that time further analyses have been completed which have replicated and strengthened the earlier studies. The most important finding of these studies was that problem drinkers exposed to the large, lecture oriented schools had significantly higher rearrest rates than those attending more interaction oriented programs. For social drinkers, the characteristics of the school appeared to make little difference. Figure 2 shows the results of the problem drinker comparisons in terms of proportions not rearrested (i.e. surviving ). These findings suggest that problem drinkers probably should not be exposed to lecture type educational programs. 1 2 3 4 6 8 QUARTERS AFTER ENTRY Figure 2 Rearrest survival rates o f problem drinkers exposed to various ASAP education programs.

626 J. L. Nichols O T H E R R E H A B IL IT A T IO N P R O G R A M S B ackground There are a variety of program types to which convicted drinking drivers may be sentenced by the courts. They range from outpatient programs (e.g. school, group therapy, individual therapy, and chemotherapy) to long-term in-patient programs. While most of the referrals at any ASAP location were to education programs, the effectiveness of other alternatives have also been examined. Project L evel E valuations Unfortunately, most of the project level analytic studies did not distinguish between the types of therapy included in their assessments of overall rehabilitation program effectiveness. A num ber of studies were conducted wherein clients were grouped relative to whether they had been assigned to any form of rehabilitation (e.g. school, group therapy, chemotherapy, etc.), versus whether they had not been so assigned. Thus, a large component of the effort exam ined by such studies also included education programs. O f the thirty-five studies subm itted by the projects concerning the effectiveness o f overall rehabilitation in reducing subsequent rearrests, only ten (29 per cent) were considered methodologically sound. Four (40 per cent) of these relatively sound studies suggested that the overall decision of sending a person to some form of rehabilitation resulted in subsequently lower arrest rates for such persons. Nearly sixty per cent of the twenty-five studies judged to be methodologically poor reported favorable results for the decision to refer drivers to rehabilitation programs. Figure 3 shows these results. 15-14- 13-12- 11 - w m L L IU Q 3 9- co L L 8 - O cc 7 - UJ CD 5 6 - z> z 5-4- POSTIVE RESULTS 3 NO EVIDENCE OF EFFECT W - WEAK STUDIES S - SOUND STUDIES 3-2 - 1 - W W 1973 1974 Figure 3 Effects o f overall rehabilitation in terms o f reducing alcohol related arrests.

The Effectiveness o f ASAP Education and Rehabilitation Programs 627 On a more specific note, there was one quite sophisticated study which suggested that the use of disulfiram (chemotherapy) as a treatment component resulted in significantly fewer subsequent rearrests and/or crashes over at least a 6-month period than did the use of no rehabilitation at all. In this study, treatment and control groups were not randomly assigned but rather were controlled by means of statistical covariance techniques. Such studies, if replicated, would provide invaluable inform ation for decision making. Program L evel E valuation On a program level, the central rehabilitation evaluator aggregated data for clients referred to and clients not referred to rehabilitation and compared the recidivism rates of the two groups. The most significant findings of these comparisons were that social drinkers referred to rehabilitation had lower subsequent rearrest rates than did social drinkers not referred to such programs. There were no significant differences between problem drinkers referred or not referred. Thus it appears that not only do non-problem (social) drinkers generally have lower subsequent rearrest rates than problem drinkers, they may also be more susceptible to positive program effects. Figure 4 illustrates the results of the non-problem (social) drinker comparisons. -r----------1---------- 1---------- 1--------------------- 1---------------------r!--------------------- r 1 2 3 4 6 8 10 12 QUARTERS AFTER ENTRY Figure 4 Rearrest survival rates o f non-problem drinkers exposed to rehabilitation vs. those not exposed to rehabilitation. As was suggested earlier, the previous comparisons included a substantial educational component and thus do not tell us much about the effects of non-educational rehabilitation programs. In order to get a better evaluation of the effects of various types of non-school programs in the ASAP effort, programs were first designated as being of the non-school category if less than fifty per cent of the program s time was spent in didactic information transmission. These non-school programs were then subjected to factor and cluster analysis procedures similar to those used for the educational programs. This process resulted in the identification of two major types of non-school rehabilitation efforts in existence across the various ASAP sites.1

628 J. L. Nichols Briefly, the first type (type 1) modalities were slightly more intensive, longer session group therapies involving fewer persons (8) in each session. Type 2 modalities consisted of a larger num ber of sessions (22), shorter in duration (l'a hours) and a larger number of persons per session (18). Subsequent comparisons of the two therapy types suggested that problem drinkers referred to the type 1 program s had fewer subsequent alcohol related rearrests than did those persons assigned to the type 2 modalities. Figure 5 illustrates this finding. r r i i i 1 i----------------------------------------1---------------------------------------- 1 1 2 3 4 6 8 QUARTERS AFTER ENTRY Figure 5 Rearrest survival rates fo r problem drinkers exposed to two non-school therapy types DISCUSSION Although there are suggestions that generalised rehabilitation referrals may be beneficial in terms of reducing subsequent violations, there is little hard evidence available to date to show that such programs are having a measurable crash reduction impact. Effects in terms of reductions in other alcohol problem areas have not yet been determined but are presently being looked at. Still, on the basis of the results already gained using driving related criteria some trends are apparent. First off, it appears that persons diagnosed as non-problem (or social) drinkers are less likely to be rearrested for drunken driving than are persons diagnosed as problem drinkers. This is a relatively stable finding in the ASAP experience. More specifically, profile analyses have suggested that convicted drinking drivers who have lower BAC s at time o f arrest, fewer prior convictions for drinking related offences, larger incomes and

The Effectiveness o f ASAP Education and Rehabilitation Programs 629 white collar jobs have a lower initial probability of being rearrested than do persons with converse characteristics. In addition to these lower initial rearrest probabilities there is some evidence from the program level analyses to suggest that social drinkers may be somewhat more responsive to treatment than are their problem drinker counterparts and that the type of treatment to which social drinkers are exposed (i.e. lecture versus interactive or extensive versus short-term) matters little. Thus, it can be hypothesised that the most economically efficient programs should be suitable for social drinkers. These probably would constitute short-term, didactic, educational programs. This hypothesis should now receive greater evaluative attention. For persons diagnosed as problem drinkers, there is little evidence that rehabilitation programs have any substantial effect on improving their rearrest or crash records (at least not in the measurable short-term). More specific analyses, however, suggest that diagnosed problem drinkers exposed to interactive (rather than lecture) type schools and more personable yet intensive therapies have significantly fewer subsequent alcohol related rearrests than those referred to lecture oriented or shorter programs. For such persons there is evidence from one methodologically sound study to suggest that the use of the drug disulfiram (chemotherapy) may aid the rehabilitation process by maintaining sobriety while the process of rehabilitation is getting underway. Due to the lack of no-treatment control groups it is not possible to conclude that brief lecture oriented educational programs are having an adverse effect on the subsequent records of diagnosed problem drinkers. However, strong suggestions to that effect are provided by the program level analyses. Indications o f treatm ent characteristics most suitable for various sex, age, social or educational levels are not presently forthcoming from these drinking driver studies. There are some indications in the general alcohol rehabilitation literature that program characteristics such as length and intensity are more im portant than is theoretical orientation.2 Some of the data presented here appear to support such findings. Studies o f non-driving related treatment programs have more often reported more favourable results than have studies using driving related criteria. This is probably due to some extent to at least two factors: (1) lesser concern over experim ental rigour and (2) the use of life change criteria (other than driving problems) which may be more sensitive to change than are driving related criteria (e.g. reduced quantity of drinking, improved health, improved socio-economic status, etc.). Drinking driver programs in the United States are now beginning to use these criteria (in addition to crashes and violations) to evaluate program effectiveness. The employment of such criteria will also contribute to a more useful assessment of program characteristics which are most suitable for various client types. R E F E R E N C E S 1. Nichols, J. L., and Reis, R. E., One Model for the Evaluation of ASAP Rehabilitation Efforts, Proceedings of the Sixth International Conference on A lcohol, Drugs, and Traffic Safety, Addiction Research Foundation of Ontario, Toronto, Ontario, Canada. 1975. 2. Ruggels, W. L., Armor, D. J., Polich, J. M. Mothershead, A., and Stephen, M., A Follow-Up Study of Clients at Selected Alcoholism Treatment Centers Funded by NIAAA, National Institute on Alcohol Abuse and Alcoholism, U.S. Department of Health, Education, and Welfare, Rockville, Maryland, U.S.A., May, 1975.