Efficiency in psychiatric treatment the case of treating personality disorders

Transcription

1 Research project: Efficiency in psychiatric treatment the case of treating personality disorders Vidar Halsteinli, SINTEF Health 1. Introduction In this project we want to study efficiency in treatment of psychiatric patients by utilising measures of inputs and outcomes on the individual level. The objective is to analyse variation in efficiency between treated patients and treatment units, and second to analyse the association between characteristics of patients and the organisational environment and efficiency. From principles of production economics, efficiency can be defined as observed productivity compared to the level of productivity which is obtainable, e.g. measured as best practice (1). Productivity can be defined as the ratio of outcome and resources used as input. Treatment personnel and administrative personnel together with medication and equipments are the major inputs in the psychiatric treatment process, but patient characteristics can be seen as inputs as well to take account of quality aspects. The objective of psychiatric treatment is improved health status for the patient (or prevent worsening), and the product is the health effect of the treatment process. Inefficiency may be a result of (a) too many resources used when achieved health effect is taken into account (and the ratio of input and outcome are compared to best practice), (b) the composition of inputs in the treatment process are not at optimum or (c) if there is decreasing returns of scale, treatment given in smaller units would have increased the outcome for the given level of input (an alternative is increasing return to scale). Factors explaining why inefficiency exist may be related to (a) the general organisational environment of the treatment units like the financing system or whether there is qualified treatment personnel available, (b) leadership performance and the way treatment units are organised like the qualification and composition of personnel and cooperation, or (c) the fact that best practice is yet not known. In our project we will study the case of treating patients with personality disorders. It is shown that intensive time-limited day hospital treatment followed by a phase of outpatient continuation therapy, has a positive effect for the patients (2-4). The question whether treatment of personality disorders could be performed with higher efficiency has not been analysed to the same extent. Karterud et al (2) though discuss questions related to efficiency. They do not find any positive correlation between treatment intensity (measured as number of hours per week) and treatment effect, which indicate that inefficiency in treating personality disorders might exist. At the same time they find differences in effect between treatment units, and this observation is connected to differences in treatment atmosphere inside units. This finding indicates that the internal organisational environment might influence efficiency. Efficiency in the production of health services is analysed for different types of patients and services, but to a large extent on an institutional level (5-11). Studies of efficiency in psychiatric treatment are limited. With reference to Norway efficiency of outpatient clinics for children are analysed by Halsteinli et al (10, 11). There are two major challenges when analysing efficiency of health 1

2 services. Both large hospitals and small treatment units have multiple and different tasks (in production theory called multiple and heterogeneous products), and second the identification of outputs are difficult and associated with difficulties both methodological and with respect to availability of data. When information about health effect on the patient level is missing, intermediate outputs like number of treated patients or consultations are used to a large extent. A way to alleviate this measurement problem is to study less aggregated and more homogenous units and use information about effect on the patient level from clinical registrations. En example of this is Puig-Junoy (12) who has analysed efficiency in the clinical management of critically ill patients. In Norway treatment of personality disorders given as intensive time-limited day hospital treatment and followed by a phase of outpatient continuation therapy, is widely accepted. The treatment includes a thorough pretreatment assessment phase. The outpatient phase following the intensive period includes group therapy with individual therapy in addition, according to individual needs. A group of nine treatment units are organised in a research network (13). Among the units the time-limited day treatment vary with respect to geographical place, institutional affiliation, size/capacity, treatment intensity (hours per week) and to some extent treatment time (number of weeks) (2). There are also some differences between units with respect to the outpatient period. In the research network data has been systematically collected for several years including information about patients, treatment and outcomes. This study will use data from the network, and in addition collect separate information about organisational factors from the different units. 2. Research questions The objective of this project is to study efficiency variation in psychiatric treatment based on outcome measures on the patient level, and next to study how organisational factors might explain efficiency variation. In other words we want to answer the question which factors should be given attention when increased efficiency is in focus. Based on data from treatment of personality disorders, the research questions are: 1) A methodological question in what way and to what extent can health effect measured by common psychometric instruments be utilised as outcomemeasures in the study of efficiency? 2) Which model of the treatment process (technology) and results (product) is best suited to identify efficient and less efficient treated patients? 3) Are there large differences between efficient and less efficient treated patients and are these differences associated to patient characteristics like social status, diagnosis and patient history or to specific treatment units? 4) Are efficiency/inefficiency associated to the organisational environment of the treatment unit like capacity, institutional affiliation and financing system? 5) Are efficiency/inefficiency associated to characteristics of internal leadership and organisation? 3. Data The study will use data from the Network of day treatment units in Norway for the period The material includes information about patients treated in this period at eight different units (number nine has just recently joined the network). Data is collected both for patients who have completed the program 2

3 and for those classified as drop-outs. The following instruments related to patients diagnose, symptoms and functioning are used: SCID-II Structured Clinical Interview for Personality Disorders DSM-IV Global Assessment of Functioning Scale, split version (GAF-S and GAF-F) Symptom Checklist 90 (SCL 90-R) Short version of Inventory of Interpersonal Problems-Cirkumplex (IIP-C) called CIP Data is collected on three different moments (T1, T2 and T3) during the treatment process: Start of intensive day hospital treatment (T1): Questionnaire answered by the patient about social and economic status, quality of life, health problems etc, SCL-90R, CIP Questionnaire answered by staff including SCID-II, GAF-S and GAF-F. End of intensive day hospital treatment (T2): Questionnaire answered by the patient about quality of life, outcome of the day treatment program, SCL-90R, CIP Questionnaire answered by staff including GAF-S and GAF-F. One year after end of intensive day hospital treatment (T3): Questionnaire answered by the patient about social and economic status, quality of life, health problems, outcome of outpatient treatment, SCL- 90R, CIP Questionnaire answered by staff including GAF-S and GAF-F and information about participation in outpatient group treatment and individual treatment. In addition the patients, answers a questionnaire with regard to ward atmosphere six weeks after start of intensive day hospital treatment (a modified version of Ward Atmosphere Scale, WAS). At the end of 2003 the material consists of 2572 patients where 2432 have finished intensive day hospital treatment (T2). The number of patients with questionnaires completed one year later (T3) was 1162 (61 per cent responded). The cohorts within the different units vary from 28 to 422 patients, partly because the units have joined the network at different moments. The material is well described by Karterud et al (2) and Wilberg et al (3). In addition to this some information about the treatment units are available with respect to treatment programs: Number of days and hours per week Number of weeks for intensive day treatment program Capacity number of patients and staff As a part of the project information about treatment units will be collected as a supplement to the existing material. Questionnaire (with closed alternatives) about organisational environment and internal organisation: What type of hospital is the treatment unit a part of? Characteristics of organisational independency of the unit 3

4 Financial sources Size and capacity Characteristics of treatment program and staff 4. Methodology We will use Data Envelopment Analysis (DEA) to identify efficient and a nonefficient treated patients, while regression will be used to analyse the link between efficiency and explanation variables. As a non-stochastic non-parametric mathematical programming method, DEA estimates the frontier of the technical feasible production set as the piecewise linear envelopment of the best practice observed units (patients). Every unit receives a relative efficiency score between 0 and 1 based on their distance from the front, where units on the front have the score of one. An advantage of DEA is that it is possible to model multiple outputs and multiple inputs at the same time, and compared to parametric methods (9), an assumption of specific functional form is not required. On the other hand there are limitations with DEA in terms of assumptions of no measurement error or random fluctuation in input-output measures. In our case the possibility of multiple outputs and no specific functional form are considered of major importance. Efficiency measures for treatment units may be presented as average scores for patients treated at the specific unit, with the possibility of analysing average scores for groups of patients depending on patient characteristics like severity of illness. The DEA/method is applied to a large extent to measure efficiency in the production of health care services (5, 7-8, 10-12, 15-16). There are specific methodological challenges concerning the use of outcome measurement on the patient level like change in GAF-rating, indexes from SCL- 90R, CIP and other types of patient information. This challenge is both concerning how to interpret changes observed, the weighting between different outcome measures, and discriminating between different input-output models. 5. Papers as part of the thesis This project will lead to a doctoral thesis based on the following papers of which one is accepted and one is submitted: 1) Paper accepted (10): Efficiency in Norwegian Psychiatric Outpatient Clinics for children. Panel data for the period of including observations from 47 outpatient clinics, are analysed, and observations consists of different types of personnel (full time equivalents), number of consultations/interventions, hours spent on interventions and number of patients treated. Utilising the DEA method variation in relative efficiency is analysed in the light of the official stipulated goal of 50 per cent increased productivity (14). Tests of different input-output models lead to model with two inputs (the number of university-educated staff and other staff), two outputs (number of hours spent on direct and indirect intervention) and variable returns to scale. The average of estimated clinic efficiencies is 71 % which indicate considerable room for improved performance. Mean productivity is 64 %, but many large clinics have considerable lower performance due mainly to scale inefficiencies. The estimated efficiency indicates a potential not far from the official stipulated goal. 4

5 2) Paper submitted (11): The paper aims to estimate change in productivity among outpatient clinics for children, and whether any change is related to personnel-mix, budget growth or financial incentives. Observations from 67 outpatient clinics for the period of consist of different types of personnel, number of interventions and number of patients treated. Utilising the DEA method a Malmquist output-based productivity index is calculated, decomposed in technical efficiency change (catching up), scale efficiency change and frontier shifts (the good becoming better). The results indicate increased overall productivity, with important contribution from increased technical efficiency. Personnel growth has negative influence on productivity growth, while a growth in the share of university educated personnel improves productivity. The financial reform in 1997 which gave greater weight for interventions per patient lead to lower productivity growth in the subsequent period for those that had initial budgetary gain from the reform. 3) Paper new: The objective is to investigate experiences concerning utilisation of instruments measuring effect in psychiatric treatment in general, and concerning patients with personality disorders in general. The work will be based on review of international literature on this topic. Measures of effect with relevance for analysis of efficiency will be our point of departure, and the questions to be answered are the question of validity and reliability, methodological strength and weakness (for instance ordinal or cardinal scale), how to interpret changes (for instance if changes on the individual or group level is possible or only group level) and the connection between decease specific instruments and health related quality of life instruments in general. 4) Paper new: The objective is to find the best possible empirical model for the treatment process (technology) and results (product) when treating patients with personality disorders, and to analyse variation in relative efficiency. The DEA method will be utilised with data from the network of day hospital treatment units in Norway together with supplementary data from the current project. The numbers of variables are high and to be able to distinguish between efficient and inefficient treated patients, the challenge is to modify the number of input variables and their specification (for instance by aggregating two or more variables) in a meaningful way. It is also a challenge how changes in effect or outcome should be interpreted, and given multiple outputs how the weighting-issue should be solved. To discriminate between different models, paper 1 (10) will be a point of departure. 5) Paper new: Based on estimated relative efficiency scores the objective is to analyse the association between efficiency and (a) patient characteristics and (b) organisational/environmental characteristics. The model utilised for estimating efficiency scores will be based on results from paper 4 and the DEA-method, while regression will be applied in the next step. The question is how patient characteristics and organisational characteristics are associated with efficiency. One hypothesis is that patient characteristics affect efficiency through the composition of the patient group. If a treatment unit has a majority of patients whom are assumed to have minor improvement in health status from the treatment program, this might affect the outcome for those patients whom are assumed to have major improvement in health status. Among organisational or environmental characteristics ownership (15), financing system (16) size, quality aspects and leadership (5) are used as explanation variables in efficiency analysis. Hypothesises are derived from theoretical behavioural models for hospitals and from organisation theory, but a large number of papers are to some extent empirical in their approach. 5

6 Conflicting views in theoretical literature on direction and magnitude of effects might be one explanation for this (15). Our focus is both on characteristics of the environmental organisation of psychiatric treatment and the internal organisation of treatment units, and at the same time seek theoretical foundation for our hypothesis. The objective is to achieve knowledge about which factors seems to require special attention when improved efficiency is wanted. 6. Practical value of the project Increased efficiency is an important objective for national health policy in general in Norway, psychiatric treatment included (14, 17). This project will provide knowledge of how instruments measuring treatment effects on the patient level and in a clinical setting, can be utilised in studies of efficiency in psychiatric treatment. Of special interest is the use of GAF since this instrument is used on a routine basis by Norwegian hospitals and data are collected by the Norwegian Patient Register every year. The project will provide methodological experience with regard to outcome-measures in studies of efficiency in psychiatric treatment, and hopefully contribute to the use of improved outcome-measures. The project will also provide knowledge about variation in efficiency in psychiatric treatment and factors explaining variation, which is assumed to be of interest for regional health authorities responsible for existing treatment units and for establishing new units. 7. National and international cooperation A co-operation with the unit for personality disorders at Ullevål University Hospital HF and professor, Sigmund Karterud is established (appendix 1). There is also an ongoing research co-operation between SINTEF Health, Health Economic Research Program at the University of Oslo (HERO) and Institute of Public Health (ISM) at the Faculty of Medicine at Norwegian University of Technology and Science (NTNU) in Trondheim. Ph. d. and Research Economist Sverre A C Kittelsen, Frisch Centre (UiO) and HERO and Associate Professor Jon Magnussen ISM NTNU are willing to participate with scientific guidance (appendix 2 and 3). They both have documented scientific experience in health economics. An application for the doctoral program at NTNU will be sent. Vidar Halsteinli is a project participant in the EU-network project EU Project on Mental Health Economics with Professor Martin Knapp, London School of Economics/Director of Centre for the Economics of Mental Health (CEMH) at Institute of Psychiatry, King s College, London as the project leader (appendix 4). 18 countries participate and economic evaluation is one of the themes in the project. 8. Time table This project will start January the 1. in 2005 and be finished by December the The different parts of the project will be like this: 6

7 Spring Autumn Spring Autumn Spring Autumn Doctoral course x x x Preparation of data: Data from network x Supplemental data x x Paper 3 x x Paper 4 x x Paper 5 x x Doctoral thesis x References 1 Hoel M, Moene KO: Produksjonsteori. Universitetsforlaget AS Bateman A, Fonagy P. Effectiveness of partial hospitalization in the treatment og borderline personality disorder: A randomized controlled trial. American Journal of Psychiatry, 156, , Karterud S, Pedersen G, Bjordal E, Braband J, Friis S, Haaseth Ø, Haavaldsen G, Irion T, Leirvåg H, Tørum E, Urnes Ø: Day treatment of patients with personality disorders: Experiences from a Norwegian treatment research network. Journal of Personality Disorders, 17(3), , Wilberg T, Karterud S, Pedersen G, Urnes Ø, Irion T, Braband J, Haavaldsen G, Leirvåg H, Johnsen K, Andreasen H, Hedmark H, Stubbhaug B: Outpatient group psychothearpy following day treatment for patients with personality disorders. Journal of Personality Disorders, 17(6), , Kooreman P: Nursing home care in the Netherlands: A non-parametric efficiency analysis. Journal of Health Economics 1994: 31: Byrnes P, Valdemanis V: Analyzing technical and allocative efficiency of hospitals. In: Charnes A, Cooper W, Lewin A W, Seiford L M (eds.). Data envelopment analysis: theory, methodology and applications. Boston: Kluwer, 1994: Magnussen J. Efficiency measurement and the operationalization of hospital production. Health Services Research, 1996; 31: Widstrøm E, Linna M, Niskanen T. Productive efficiency and its determinants in the Finnish Public Dental Service. Community Dent Oral Epidemiol 2004; 32: Efficiency in provision of public dental services in Norway. Grytten J, Rongen G. Community Dent Oral Epidemiol 2000; 28: Halsteinli V, Magnussen J, Kittelsen S A C. Scale, efficiency and organization in Norwegian psychiatric outpatient clinics for children. Journal of Mental Health Policy and Economics, 4 (2) : (2001). 11 Halsteinli V, Magnussen J, Kittelsen S A C. Productivity growth in Norwegian outpatient clinics. HERO Working Paper 2003:9. 7

8 12 Puig-Junoy J: Technical efficiency in the clinical management of critically ill patients. Journal of Health Economics 1998; 7: Internettadresse: 14 St prp nr 63 ( ) Om opptrappingsplanen for psykisk helse Endringer i statsbudsjettet for Sosial- og helsedepartementet, Oslo. 15 Burgess JF, Wilson PW. Hospital Ownership and Technical Inefficiency. Management Science/Vol. 42, No.1, January Hagen TP, Iversen T, magnussen J. ISF og sykehusenes effektivitet. Erfaringer fra 1997 til HERO Skriftserie 2000:1, UiO. 17 Helsedepartmentet. Styringsdokument for de regionale helseforetakene, desember 2001, Oslo. 8