Evidence-based operational and strategic behavior of

Transcription

1 Closing the Loop: Optimizing Physicians Operational and Strategic Behavior Paul T. Sergeant, MD, PhD, and Eugene H. Blackstone, MD Katholieke Universiteit Leuven, Leuven, Belgium, and The Cleveland Clinic Foundation, Cleveland, Ohio Clinical databases are essential elements in optimizing medical care. They are no finality by themselves, but essential elements in the generation of knowledge. Optimal medical care starts with optimal care based on existing knowledge. This care continues with the registration of the variability in morbidity, comorbidity, and therapy, but also the registration of the early and late outcome. This should then allow the generation of structured inferences based on this registration and the closure of the loop, by treating patients according to this newly created evidence. (Ann Thorac Surg 1999;68:362 6) 1999 by The Society of Thoracic Surgeons Presented at the Thirty-fourth Annual Meeting of The Society of Thoracic Surgeons, New Orleans, LA, Jan 26 28, Address reprint requests to Dr Sergeant, Department of Cardiac Surgery, Gasthuisberg University Hospital, Herestraat, 3000 Leuven, Belgium; Evidence-based operational and strategic behavior of clinicians should be based on knowledge generated by following patients during their treatment, but also beyond their hospital stay, and preferably for as long as possible, until the end of the patient s life. This knowledge should include the interaction of the patient s morbidity and comorbidity, the specificity of the disease and the intervention, and environmental factors. The purpose of this manuscript and presentation is to follow the path of these data, from the generation of information up to the generation of knowledge useful for an individual patient or a population. The generation of information has to follow methodological rules and constraints. The first of these is a methodology for overcoming the absence in the late 20th century of complete, longitudinal, patient records of all health care information. Important morbid events may be forgotten if a patient is contacted at infrequent intervals, or if only vital status information is available from national databases. Therefore, when the K.U. Leuven Coronary Surgery Database was established, a doublecore data system for patient follow-up was incorporated. First, there was an unformalized update by a regular stream of follow-up reports by referring cardiologists. Because most of them trained at our University Hospital, we have encouraged them to send reports after each regular visit. Visits to the cardiologists are financially neutral for the patient in the Belgian Health Care system. In addition to this real-time update, two (1987 and 1993) cross-sectional follow-up studies were performed of known surviving patients using the very strict common closing date methodology. This methodology fulfills the theoretical assumptions of actuarial methods. Others have implemented similar multitiered follow-up systems, but using an anniversary follow-up scheme for obtaining systematic information. This scheme also fulfills the theoretical assumptions of actuarial methods. This cross-sectional follow-up is managed and monitored by a self-developed information management system, directly integrated in the K.U. Leuven Coronary Surgery Database. Figure 1 describes the different loops of step 1, taking place after defining the common closing date. Figure 2 describes some of the different loops of steps 2 and 3, when the patient is identified as deceased or if events have taken place in follow-up. The loops of step 2 stop only when the senior researcher (a cardiac surgeon) has spoken with a person who was present at the time and place of death of the patient. This is very important for a correct classification of the mode of death, just as important as a complete insight into the events and status of the patient in the weeks and months before death. Death is always sudden and unexpected for the close relatives of a patient. A more profound analysis frequently corrects this information. Systematic follow-up is time consuming and expensive. For us, the second cross-sectional follow-up of 9600 patients took more than 24 months from the direct mail date of the first letter to the patient. These first letters were sent in the week after the common closing date. At that time, all preoperative, operative, and hospital data had been completed. The information management system performed well, and complete follow-up was obtained of 99.9% of the patients. Generally, systems optimized for data gathering, information management, and data updating and verification are not optimized for sophisticated statistical data analysis. Generally, a snapshot is needed of the data relevant to the analysis. Importantly, all data corrections discovered during analysis are fed back to the primary database. In our case, data were transferred from the database computing system into the data analysis computing system of the University of Alabama at Birmingham. Even in an ideal future setting of existing computerized longitudinal patient medical records, a data 1999 by The Society of Thoracic Surgeons /99/$20.00 Published by Elsevier Science Inc PII S (99)00627-X

2 Ann Thorac Surg SERGEANT AND BLACKSTONE 1999;68:362 6 CLOSING THE LOOP 363 Fig 1. Step 1 in the generation of information: obtaining the status of the patient at the common closing date. analysis phase would require extraction of values for variables across multiple patient records at some specific point in time. Fig 2. Step 2 (a deceased patient) and step 3 (an event, symptom, or procedure) in the generation of information, once the status of the patient has been identified. Data analysis must also be approached purposefully, methodically, and with clear questions in mind. The analysis begins with assessing the quality, completeness, and relevance of the study endpoints (outcomes) and variables. These are then organized into medically meaningful clusters. Formalization of this step has become popularly known as an important aspect of data mining. This is followed by investigation of the relation of each variable in isolation to outcomes. For ordinal (ordered categorical information) and continuous variables, transformations of scale must be sought so as to best fulfill the assumptions of the underlying statistical models. Table 1 lists some of the steps in data exploration preceding and finally including the generation of patientspecific predictions and nomograms usable for optimizing therapeutic decision making. This process is labor intensive and requires a close and permanent collaboration between the data analyst and the clinician. Only then are the data ready for initial multivariable analysis. The creation of interaction variables and the grouping of categorical variables should be based on extensive clinical experience and previous literature review. The K.U. Leuven Coronary Surgery database started off with many hundreds of variables, describing the patient, procedural, and institutional variability. The optimization process of the variables through transformation and grouping increases this number even further. Unexplored domains and relations become visible and disappear during the mandatory data exploration, and the foundations of the multivariable analysis are laid. The generation of knowledge starts when the parametric multivariable model has been created [1]. One reason for use of parametric modeling is that the model can be explored graphically (which we have termed nomo-

3 364 SERGEANT AND BLACKSTONE Ann Thorac Surg CLOSING THE LOOP 1999;68:362 6 Table 1. The Different Steps in Data Exploration and Analysis Leading to the Generation of Patient-Specific Predictions and Nomograms Univariate Time-related Grouping of categorical variables Univariate Non-time-related Analysis for whole interval Univariate Non-time-related Analysis for predefined intervals Univariate Time-related Analysis (stratified actuarial curves) Univariate Time-related Logit plots Univariate Time-related Transformation of continuous variables Time-related Creation of parametric hazard function (patient (procedure (patient procedural (institutional (patient procedural institutional Multivariable Time-related Patient-specific predictions Multivariable Time-related Generation of nomograms grams ) to discover the shape of relations, the impact of risk factors on one another, and the appearance and disappearance of evident differences, all of which go into drawing helpful clinical inferences, suggesting hypotheses needing basic research, and developing strategies to optimize therapy in the interim. This laborious, costly, time-consuming activity of generating new knowledge contrasts with the present intensive scrutiny of in-hospital outcomes. Governments, societies, institutions, and the media use hospital mortality as an indicator of quality of care in cardiac surgery. The information is based on aggregated data for a surgeon or for a center, and most frequently for all surgical interventions combined, uncorrected for type of intervention, pathology, or comorbidity. It is assumed by many that the operative (30-day) mortality for primary coronary artery bypass grafting (CABG) and for aggregated data should be 2%. Very few authors have actually complete follow-up, even to the 30-day limit; this creates an evident bias in all data without follow-up beyond hospital stay. Figure 3 is the decomposition of the different phases of the hazard of mortality after CABG [2]. The hazard of dying after CABG induced by the preprocedural condition and the procedure stays active for 2 months beyond the data of the procedure. There is no scientific basis to restrict the Fig 3. Decomposition of the three hazard phases (solid lines) composing the overall instantaneous hazard (dotted line) for mortality by any mode at any time after CABG. study of the periprocedural mortality to the hospital stay. Hazard-phase analysis for a particular event identifies the appropriate interval for the study of the periprocedural hazard for that event, for that particular pathology and intervention. The assumption of the 2% risk without extensive correction for pathology and comorbidity is only valid if the risk is similar for every individual patient and close to 2%. The assumption is also valid in the presence of different risks for different patients but with a similar patient mix for every center, surgeon, or institution. This last condition is certainly invalid with the annual changes in patient mix, even within one individual center. A patient-specific prediction uses a generated equation and uses the patient-specific values for every variable in the equation predicting that particular event. Figure 4 presents a patient specific prediction of the 30-day mortality after CABG, calculated for 9600 consecutive patients (K.U. Leuven ) and for the last 1000 Fig 4. Patient-specific prediction of the 30-day mortality after CABG, calculated for 9600 consecutive patients (K.U. Leuven ) and for the last 1000 patients of that same data set, sorted by increasing mortality.

4 Ann Thorac Surg SERGEANT AND BLACKSTONE 1999;68:362 6 CLOSING THE LOOP 365 Fig 5. Patient-specific prediction of the 20-year survival after CABG, calculated for 9600 consecutive patients (K.U. Leuven ), sorted by decreasing survival. Only the lines representing the minimum, the maximum, and the different 1%, 5%, 10%, 25%, 50%, and 75% are defined. patients of that same data set, sorted by increasing mortality. The information presented in this plot is comprehensive. There is a 100-fold spread in 30-day mortality of the 9600 primary CABG patients. This spread was reduced to 50-fold in the last 1000 patients of that particular data set. The hospital mortality for most CABG patients has been extremely low (2400 patients, or 25%, had an operative mortality of less than 0.4%), and most of the mortality is concentrated in few patients with horrendous pathology and comorbidity. Some of these patients combined cardiac arrest at the start of surgery with old age and other comorbidity. Any statement concerning an average 30-day mortality is therefore groundless. Any national or institutional audit or quality control methodology without extensive correction for this variability is a scientifically invalid method. Recently, Osswald and colleagues have confirmed that it is the highest risk patients whose risk is most likely to be underestimated by 30-day mortality [3]. However, hospital information is readily available, relatively inexpensive to gather, particularly if confined to administrative and computerized laboratory data, and thus appealing to institutions and providers who tend to focus on short-term issues rather than the long-term care of patients. Another inference of Figure 4 concerns least invasive techniques of CABG. Aggregated data resulting in a 2% 30-day mortality might seem comparable with standard techniques for the uninformed but are similarly misleading, since the comorbidity of the patients undergoing less-invasive CABG is usually extremely low. The comparable 30-day mortality should be more in the 0.2% to 0.5% range. Randomized trials comparing less-invasive CABG techniques in low-risk patients will therefore have very large sample sizes. Informed consent to a patient is usually based on a medical cost-benefit calculation. The issue of the cost, expressed as the periprocedural mortality, has been addressed already. The benefit, expressed as the longterm survival, is usually based on an intuitive appreciation by the attending physician of the residual survival reserves. Figure 5 represents the wide spectrum of the survival of our CABG population. Similarly as in 30-day mortality, a very good long-term survival is identified in 25% of the patients, a good one in an additional 25%, and an acceptable one in another 25%. The worst survival is accumulated in the last 1% of the population. It should be clear that only patient-specific predictions, based on long-term follow-up studies, can provide the tools for appropriate medical decision making and informed consent. It is also obvious that honesty toward an audited individual or institution requires that audited interval should be as long as possible, thereby auditing the cost simultaneous with the benefit. Individual surgeons might have an increased periprocedural mortality but with a dramatic gain in late survival, more than compensating for the periprocedural risk. Similar models have been created for alternative therapies [4], and in computro predictions allow more precise comparisons between therapies. Intuition, even for the informed clinician, cannot correct for the complex interactions (Figure 6). Patient-specific nomograms will give a much better insight in differences in surgical strategy. The multivariate time-related parametric analysis [5] of the freedom from angina after primary CABG identified the influence of a single arterial graft (under various conditions) in the two active phases and the additional influence of increasing the number of arterial grafts beyond a single one in the early phase ( p ). Similarly, the same analysis detected major influences of younger age in both phases of the hazard. It is impossible for the clinician to understand the size of these influences, and even less how their respective influences interact with one another. Fig 6. Patient-specific nomogram of the 10-year survival after CABG, for a patient with median values for all risk factors (relevant for this event). The median patient is presented with an ejection fraction of 40%, with and without chronic atrial fibrillation and with changing 1-second expiratory volumes (expressed as a percent from normal).

5 366 SERGEANT AND BLACKSTONE Ann Thorac Surg CLOSING THE LOOP 1999;68:362 6 Figure 7 quantifies the benefit, on the freedom from angina after primary CABG, of using a second internal thoracic artery versus only one. It is obvious that the benefit will have very little clinical relevance either at 1 or 5 years and this for the whole spectrum of age at operation. An essential step in closing the loop in evidence-based medical care is the study of the validity of the generated equations on independent study samples. The conditions of this validity have been well identified in a study [6] of previously described equations on an independent population of 3720 patients followed for 5 years. Closing the loop, from treating patients, studying their early and late follow-up, and using that generated knowledge in the treatment of the future patients, is what medicine should be all about. The generated knowledge can cover many domains. Some of these domains are the variability in pathology, in treatment and in patient spectrum. The generated knowledge should help the attending physician in the actual care of an individual patient by solving the differential therapy problem, a more documented informed consent, optimizing therapeutic strategy at primary care, and at return of symptoms. Evidence-based medicine should similarly guide departmental management as well as departmental quality control. Fig 7. Patient-specific nomogram of the 1-year and 5-year difference in freedom from angina after primary CABG, for a patient with median values for all risk factors (relevant for this event). The median patient is a triple-vessel disease patient presented with changing age at operation. References 1. Blackstone E, Naftel D, Turner M. The decomposition of time-varying hazards into phases, each incorporating a separate stream of concomitant information. J Am Stat Assoc 1986;81: Sergeant P, Blackstone E, Meyns B, K.U. Leuven Coronary Surgery Program. Validation and interdependence with patient-variables of the influence of procedural variables on early and late survival after CABG. Eur J Cardiothorac Surg 1997;12: Osswald BR, Blackstone EH, Tochtermann U, Thomas G, Vahl CF, Hagl S. The meaning of early mortality after CABG. Eur J Cardiothorac Surg 1999;15: Kirklin J, Barratt-Boyes B. Patient-specific predictions and comparisons in ischemic heart disease. In: Kirklin J, Barratt- Boyes B, eds. Cardiac surgery II. New York: Churchill Livingstone, 1991: Sergeant P, Blackstone E, Meyns B. Is return of angina after coronary artery bypass grafting immutable, can it be delayed, and is it important? J Thorac Cardiovasc Surg 1998;116: Sergeant P, Blackstone E, Meyns B. Can the outcome of coronary bypass grafting be predicted reliably? Eur J Cardiothorac Surg 1997;11:2 9.