Influence of Orthopaedic Clinical Specialist Certification on Clinical Outcomes

Transcription

1 Journal of Orthopaedic & Sports Physical Therapy 2000;30(4) :83-93 Influence of Orthopaedic Clinical Specialist Certification on Clinical Outcomes Dennis I. Hart, PhD, PT Edward A. Dobrzykowski, MHS, Pr ATC2 Journal of Orthopaedic & Sports Physical Therapy Study Design: Effect of clinical specialization was studied in a retrospective analysis of a commercial outcomes database. Objective: To assess effectiveness of care as measured by changes in health status and efficiency as measured by visii duration of treatment episode, and net revenue between patients treated by clinicians with and without orthopaedic clinical specialist certification (OCS). Back@: Clinical specialization is becoming common in physical therapy, but there are no studies to support improved efficiency or effectiveness with advanced practitioner competencies. Methods and Measures: A total of 258 adults treated in practices participating in the Focus on Therapeutic Outcomes process during 996 comprised the data set. Seven physical therapists with OCS treated 29 patients (clinical specialist group). These patients were matched to 29 patients not treated by physical therapists with OCS (comparison group) randomly chosen from the aggregate data set. All patients completed a standardized health status questionnaire at initial evaluation and discharge. Standardized response means (SRMs) were calculated to measure change during treatment. Results: Therapists with OCS were more efficient than therapists without OCS, using fewer visits ( vs ) for less estimated cost ($949 + $736 vs $238 + $227) during the same treatment duration ( vs 35.4 t 25.6 days) and performed fewer treatment procedures. Overall, there was no difference in effectiveness as measured by change in health status, that is, unit of functional improvement per episode ( SRM for clinical specialists compared with SRM for comparison group). The OCS group had better value (unit of functional improvement per estimated dollar) and utilization (unit of functional improvement per visit) for the constructs of physical functioning (value: vs ; utilization: vs ) and role physical (value: vs ; utilization:. +.9 vs ) (SRMs for OCS group vs comparison group, respectively). Conclusim Our data support the conclusion that physical therapists with OCS are more efficient compared with clinicians without OCS. Study limitations in design, small sample size, and low number of clinicians are discussed. / Orthop Sports Phys 7her 2000;30: Key Words: health-related quality of life, orthopaedic clinical specialist, outcomes Director of Consulting and Research, Focus On Therapeutic Outcomes, Knoxville, Tmn. Director of Development, Focus On Therapeutic Outcomes, Knoxville, Tenn. Currently works for Fort Sanders Sevier Medical Center, Sevierville, Tenn. Partial support for this study received from the Orthopaedic Section of the American Physical Therapy Association Clinical Research Grant Program 997. This study was approved by the institutional Review Board for the Protection of Human Subjects of Focus On Therapeutic Outcomes. Send correspondence to Dennis L. Hart, FOTO, Inc, 0523 Brevity Drive, Great Falls, VA dhart988 4@aol.com C linical accountability of medical practitioners, including physical therapists," is considered the "third revolution in medical care."27 Clinicians quantify accountability by evaluating the effectiveness of clinical techniques. Effectiveness* has been defined by the outcomes of clinical services delivered by practitioners in the general community or in usual and customary practice settings.4 Many researchers5."l"5.".la20:~2-24,~99,4 and,-linician~l.~ emphasize patient self-report of function, health, and well-being as the preferred method of assessing the effectiveness of treatments compared with traditional measurements of impairment.l5ss Assessment of outcome effectiveness would be incomplete without assessment of resource utilization. Jette et all7 presented efficiency data (visits, duration of episode of care, charges) for patients with spinal disorders, and Dobnykowski and Nance7 presented efficiency data from an aggregate data set for patients with a variety of disorders. Neither study related the administrative data (efficiency) to patient outcome, which is increasingly important since resource utilization is reduced with * Effectiveness is defined in our study as the unit of functional improvement per episode of care.

2 managed care. Quality can be measured as a unit of functional improvement per episode of care. Value (unit of functional improvement per dollar cost to the payer) and utilization (unit of functional improvement per visit) are examples of measurements that represent the quality of treatment rendered by physical therapists. Becoming certified in a clinical specialty is increasingly popular in physical therapy. One of the earliest specializations created by the American Board of Physical Therapy Specialties was the orthopaedic clinical specialist certification (OCS). If the costs to the therapist for securing'this specialization or the costs to the employer hiring the specialist are passed on to the consumer, the cost of delivering physical therapy may increase. If the value of rehabilitation is to improve, resource utilization (visits, duration, cost) should decrease or clinical quality (unit of functional improvement) should increase or both. If treatment from a specialist costs more, the only way to improve value is to increase quality. If the specialist can increase the quality of care with lower resource utilization (decreased visits, duration, cost), the value of services of specialists will increase. This would be a strong reason for obtaining specialization. We could not find any published studies examining differences in effectiveness (outcomes) and efficiency (costs, visits, duration) of outpatient orthopaedic rehabilitation provided by physical therapists with an OCS certification compared with those without specialization certification. Therefore, the purpose of this retrospective study was to determine if there was a difference in () effectiveness as measured by changes in self-reported health status and (2) efficiency as measured by visits, duration, and costs to payers between patients treated by physical therapists with and without OCS. METHODS Subjects A total of 258 adults (9 men with a mean + SD age of years; 67 women with a mean 2 SD age of years) comprised the data set (Table ). These patients were retrospectively selected from 28,895 patients entered into the Focus On Therapeutic Outcomes (FOTO) aggregate data set during 996. The study patients were treated in acute orthopaedic outpatient centers and had completed a standardized health status questionnaire at TABLE. A priori variables by group: Treated by clinical specialist Patient groups Treated by nonspecialist Severityt Slight Moderate Severe 3 34 Very severe Age group (y) < Depression indicator Present Not present Employment at discharge Employed, working full duty Employed, working part-time Employed, but not working Not working, receiving disability Unemployed Retired Student Other Reason for discharge Goals met Goals not met, maximum benefit 7 8 Not compliant 9 6 Other 6 5 Impairment category Arm, excluding shoulder Cervical spine Lumbar spine Lower extremity Shoulder Not otherwise classified Numbers are frequencies of patients per group. No x2 statistics were significant across groups. t Severity indicates quartile of the sum of the admission health status scores of the 6 functional scales. A total of 29 patients (OCS treatment group) were identified as having been treated by 7 physical therapists with OCS in 996. Data from these patients were analyzed using 6 a priori variables (Table ). Severity,'" age,'ojnjy impairment category, pres- ence of a depression identifier,'hjy reason for discharge, and employment status'njy were selected, because these variables have been shown to influence health status outcomes. These variables were selected a priori and used to test the homogeneity of groups (Table ). Severity was defined as the quartile of the initial evaluation and discharge. The FOTO process sum of the admission health status scores of the 6 has been previously des~ribed.~-~.'~.'~~.'~~"-~~ From the functional scales. As severity increases, gains in clinician registration forms, 9 physical therapists health status increase.'" Patients were placed into imwith OCS were identified. Verification that the thera- pairment categories by the primary anatomical body pists held an OCS in 996 was obtained via tele- part identified as being treated by the evaluating phone interview. Seventeen physical therapists con- therapist. The categories were cervical spine, lumbar sented to participate. spine, arm (upper extremity excluding shoulder), J Orthop Sports Phy Ther.Volume 30.Number 4.April 2000

3 shoulder, hip, knee, and not otherwise classified. Although we are unaware of any studies that show that health status outcomes are different across impairment categories, the hierarchical structure of physical functioning has been shown to be different across impairment categories,i2 and it seems logical to classify patients by anatomical part. Therefore, we thought it was important to group patients by impairment categories. Younger individuals report greater gain in health status outcomes.lo~lhjy The presence of a depression identifier reduces health status change score^.^^^^^ Patients who remain on the job or return to work have better health status gains. There is a relationship between therapists indicating that their patients have met the treatment goals and higher gains in health status. One hundred twentynine patients (comparison group) were randomly selected from the aggregate data set and matched to the OCS group on the a priori variables. Homogeneity of the a priori variables across comparison and OCS treatment groups was tested using x2 statistics test for independence (P <.05). Four groups of patients were randomly selected from the database before a comparison group was found that did not differ from the OCS treatment group for all a priori variables. During the selection process, no consideration was made to the type of clinician (ie, physical therapist, occupational therapist, or assistant), because these variables were not identified a priori as related to changes in health status outcomes. The project was reviewed and approved by the FOTO Institutional Review Board for the Protection of Human Subjects (Knoxville, Tenn). Therapists Not all therapists completed all requested information at the time of clinician registration. From the data available, there were 7 physical therapists with OCS and 60 clinicians (53 physical therapists, 5 occupational therapists, and 2 physical therapist assistants) in the comparison group, none of which had OCS (Table 2). Most therapists (57.%) in the clinical specialist group earned baccalaureate degrees, and 7.2% of the clinicians in the comparison group earned baccalaureate degrees. The remainder of clinicians had master's degrees. On average, the specialists had more clinical experience as measured 2 ways: () in years of direct patient care ( years) compared with the clinicians of the comparison group ( years), but the difference was not significant (2-tailed t, = -.2, P =.56), and (2) in years from date of license acquisition ( ) compared with the clinicians of the comparison group ( ), but the difference was not significant (2-tailed b,, =.0, P =.7). The clinical specialists worked more hours per week in their facil- TABLE 2. Clinician characteristics.* Sext Men Women Highest degreet Baccalaureate Master of science Patient groups Treated by clinical Treated by Variable specialist nonspecialist Type of licenset Physical therapist Occupational therapist Physical therapist assistant Years of direct patient care* Years from license acquisition* Hours worked per weeks Numbers are frequencies of clinicians per group or mean t SD. Not all clinicians in the nonspecialist group completed highest degree or type of license data entry on the clinician registration form. t x2 test not significant; P >.05. * Two-sample t test not significant; P >.05. Two-sample t test significant; P <.05. ity ( hours) than the clinicians in the comparison group ( hours) (2-tailed b,, = -2.3, P =.048). The clinicians in both groups entered data during the last 6 months of 996. Procedures FOTO personnel examined the clinician registration database. We did not want to miss any therapist with OCS, so all clinicians identifying themselves as having a specialization were contacted regardless of type of specialty (n = 62). Each clinician who registered as having OCS was mailed a questionnaire. A total of 26 questionnaires were returned completed (42% response rate). Each therapist who had not returned his or her questionnaire was contacted via telephone to complete the questionnaire via oral interview. The purpose of the clinician contact was to determine the existence of the OCS in 996. No attempt was made to control for other factors, such as age, years of experience, highest educational level, and other specialties between groups. Data Collection At the end of the treatment episode, the treating clinician or staff member recorded on a discharge questionnaire net revenue (money expected to be received from payer, including cepay), the number of treatment visits, and treatment procedures used for each patient. Duration of treatment episode (number of calendar days between initial evaluation and discharge) was calculated via computer program J Orthop Sports Phy Ther.Volume 30oNumber 4.April

4 from dates of initial evaluation and discharge. Outcomes data consisted of the patient's responses to self-administered patient self-report surveys: one (intake) collected before initial evaluation and one collected toward the end of treatment or at discharge. Status surveys were completed during treatment at greater than I-week intervals. The timing of the completion of the status or discharge surveys varied and depended on clinical circumstances. Preferably, the status or discharge survey was completed on the day of discharge. If the patient's last visit was not predicted at the time of service and a status or discharge survey was not completed at discharge, the last survey obtained during treatment was used as the final survey. Information describing the clinicians and the facilities was collected at the time the clinician and facility were registered in the FOTO process. The intake and status and discharge surveys consisted of questions from the acute version of the 36 Item Short-Form Health Survey (SF-36).2"24.3R Questions from the 2-Item Short-Form Health Survey (SF-2) 35 are imbedded within the SF R The SF-36 and SF-2 have been shown to be reliable and valid j".-9n Six functional scales were calculated using questions from the SF-2 and SF-36: physical functioning (SF-36), bodily pain (SF-2), role physical (SF-2), mental health (SF-2), social functioning (SF-2), and vitality (SF-2). Scoring of the patient health status responses followed the published alge rithms, which transform the ordinal client responses to interval scores from 0 to 00 for each question.39 These transformed response scores were averaged to obtain the score for each scale. The higher the score, the better the patient's perception of their health, well-being, and functional abilities. Each functional scale represents an estimate of the current level of health and well-being and functional ability related to a specific construct. The transformed scores for the six functional scales on intake were averaged to produce a global health status measure on intake. The same calculation was used to produce a global health status measure on discharge. The change in global health status represents an average unit of functional change per episode. An episode is defined as the time between intake and discharge. Change scores using the standardized response means (SRMs) (discharge - intake/sd of the change scores)30 were calculated for each functional scale. An overall standardized change score in health status was calculated from the 6 functional scales by averaging the discharge minus intake change scores for each functional scale then dividing by the SD of the average change scores. The SRMs from the overall health status change score and the 6 functional scale change scores represent measures of quality (ie, the unit of functional improvement per treatment episode). Value was calculated by dividing each of the 7 SRMs by the net revenue (value = SRM/net revenue). Each calculation represents a measure of value (ie, unit of functional improvement per dollar cost to the payer). Utilization was calculated by dividing each of the 7 standardized response means by the number of visits (utilization = SRM/visits), each of which represents a measure of utilization (ie, unit of functional improvement per visit). Value and utilization were calculated on a patient-by-patient basis. There are no studies describing the reliability or validity of overall health status, value, or utilization change scores. The functional scales with 2 or more questions in the FOTO instrument have been shown to have good-toexcellent internal consistency reliability.3 In a study assessing a large sample (n = 63,474), the Cronbach a for the physical functioning scale was.9 on intake and.9 on discharge.3 The reliability coefficients for the bodily pain scale were.79 on intake and.84 on discharge. These reliability coefficients compare favorably to published reliability scores.39 The a for role physical was.76 on intake and.82 on discharge.3 The a for mental health was.63 on intake and.66 on discharge. There are no test-retest reliability studies published for the FOTO tool. The physical functioning scale was shown to have good construct validity, unidimensionality, and reproducibility across testing times and patients with various impairment categories.i2 These results compared favorably with previously published studies in unidimensionality of the SF-36 physical functioning scale for patients with chronic medical and psychiatric condition^.^ Data Analyses Differences between groups (OCS and comparison) and each dependent variable were tested using -way analyses of covariance (ANCOVAs) using the respective intake patient self-reported health status as the covariates. The ANCOVA models for efficiency data had visits, duration, and net revenue as dependent variables, clinician group as the independent variable, and intake self-reported global health status measure as the covariate. The analysis of visits, for example, used group as the independent variable and intake self-reported global health status measure as the covariate. ANCOVA models for change in health status used the discharge individual functional scale and discharge global health status scores as the dependent variables, clinician group as the independent variable, and respective intake self-reported health status measures as the covariates. The analysis of physical functioning measured at discharge, for example, used group as the independent variable and intake self-reported physical functioning measure as the covariate. ANCOVA models for change scores in value and utilization used the SRMs for individual functional scale and global value and utiliza- J Orthop Sports Phys Ther-Volume SO. Number 4.April2000

5 tion scores as the dependent variables, clinician group as the independent variable, and the intake self-reported global health status measure as the covariate. The analysis of SRM for the role physical value measure, for example, used group as the independent variable and intake self-reported global health status measure as the covariate. The analysis of SRM for global utilization measure used group as the independent variable and intake self-reported global health status measure as the covariate. The critical a level was set at.05 Physical agents, procedures, and exercises used by the clinicians were compared between the 2 treatment groups using x2 tests for independence. The a level was reduced to.o for the x2 tests on treatments to adjust for multiple analyses.25 Standardized deviates [(observed - expected) / (square root of expected)] were used to identify cells where the o b served values were much greater than expected. Retrospective power ( - p) analyses were conducted for dependent variables used in all ANCOVAs. Beta was defined as the probability of accepting the null hypothesis when we should have rejected it. The 2 statistical analyses analyzed differences across groups for overall health status, individual functional scales, overall value, individual value of functional scales, overall utilization, individual utilization of functional scales, visits, duration, and net revenue. This represents an increased risk of producing a type I error (rejecting the null hypothesis of no better group difference when in fact the null hypothesis is true). No adjustments in a values were made. RESULTS = 4.7, P =.03) There were no other differences in individual functional scales across groups. Overall value tended to be better for patients in the OCS group (Table 4), but the difference was not significant (ANCOVA F,,,,,, = 3.4, P =.066). Value for the 3 functional scales that represent physical components of health status tended to be better for the OCS group. The unit of functional improvement per estimated cost of treatment was better for patients treated by therapists with OCS for physical functioning (ANCOVA F,,,,,, = 4.2, P =.04) and role physical (ANCOVA F,.,,,,, = 4.7, P =.03). Bodily pain value tended to be better for patients treated by therapists with OCS, although the difference was not significant (ANCOVA Fl,,255 = 3.2, P =.076). The same trend seen with value was present with utilization (Table 5). Overall utilization was not different across groups (ANCOVA F,,,,,,, = 2.6, P =.Ill). The unit of functional improvement per visit for the physical component scales was better for patients treated by therapists with OCS for physical functioning (ANCOVA F..255 = 5.2, P =.023) and role physical (ANCOVA FI,,,,, = 6.0, P =.05) but not for bodily pain (ANCOVA F,,, = 3., P =.078). Statistical power was low (<0.7) for efficiency and effectiveness dependent variables analyzed with AN- COVAs and ANOVAs. It was estimated that a sample size of 400 patients would be needed for a power of 0.8. Using role physical utilization as an example, it was estimated that a sample size of 400 patients would provide a power of 0.8 with associated variance of 4.0 and effect size (difference between AN- COVA-adjusted group means) of 0.6 with a =.05. Treatment Procedures Efficiency Analyses The most prevalent treatment procedures used by Patients treated by therapists with OCS received both groups are listed in Table 6. Numbers represent fewer visits ( ) compared with patients treat- frequencies of patients in each group who received ed by therapists without OCS ( ) (mean + the treatment. Clinicians without a clinical specializa- SD) (analysis of variance [ANOVA] F,,255 = 5.32, P tion used more ice or compression, massage, body =.02). Estimated treatment cost was less for the mechanic training, postural exercises, flexibility exer- OCS group ($949 + $736) compared with the com- cises, functional training, strengthening exercises, parison group ($238 + $227) (ANOVA Fl.,255 = and home exercises compared with the therapists 4.6, P =.O3). There was no difference in duration with OCS. Highwelocity manipulation was used sparof treatment episode across groups (OCS, ingly by both groups. The odds ratios were used as a 48.3 days; comparison, days) (ANOVA post hoc analysis for the significant x2 test statistics. F,,,s55 = 0.06, P =.8). Patients in the comparison group were approximately 5.4 times more likely to receive ice or compression than patients in the OCS group. The most striking Effectiveness Analyses finding was that patients in the comparison group were 7.2 times more likely to receive functional Measures of health status (quality) are displayed in training (Table 6). Table 3. Discharge health status was not different across groups (ANCOVA Fl = 2.0, P =.l63). Pa- DISCUSSION tients treated by physical therapists with OCS reported more gain in physical functioning compared with The results of this retrospective study demonstratpatients in the comparison group (ANCOVA Fl,,255 ed that clinical outcomes (effectiveness) as measured "-"- ' J Orthop Sports Phys Ther.Volume SO Number 4 *April 2000

6 TABLE 3. Health status analysis.* Patients treated by certified orthopaedic Patients treated by nonspecialist clinical specialists In = 29) (n = 29) Raw scores Change scores Raw scores Change scores ANCOVAt Variable Min Max Mean SD Mean SD SRM Min Max Mean SD Mean SD SRM Source F df P Health status* Intake 5 Discharge 4.2 Physical functioning Discharge 0 Role physical Discharge 0 Bodily pain Discharge 0 Mental health Discharge 20.0 Vitality Discharge Intake 36.2 < I Intake 39.7 < Intake 84. < I Intake 36.8 < Intake 9. < Croup.8.I Intake 47.0 < Social functioning Intake 74.0 <.00 Discharge I95 Numbers represent minimum (min), maximum (max), unadjusted mean (mean), SD, and standardized response mean (SRM) of raw scores. t ANCOVA indicates analysis of covariance of discharge scores with the intake scores as the covariate. $ Health status at intake indicates average score from the 6 functional scores at intake; health status at discharge indicates average score from the 6 functional scores from discharge. TABLE 4. Value analyses.* Patients treated by Patients treated by certified nonspecialists orthopedic clinical specialists (n = 29) (n = 29) ANCOVAS Variablet Min Max Mean SD Min Max Mean SD Source F df P Value Intake 5.8 < Physical functioning value Intake < Role physical value Intake 6.5 <.00 Bodily pain value Intake 33.8 < Mental health value Intake <.00.I c Social functioning value Intake 53.8 <.00 Croup.8.I87 Vitality value Intake * Numbers represent standardized response means. Min indicates minimum; max, maximum; and mean, unadjusted means. t Value indicates average standardized response mean from the 6 functional scores divided by net revenue times 000. Physical functioning value, role physical value, etc. represent the value for each functional scale, which is calculated by specific scale standardized response mean divided by net revenue times 000. $ ANCOVA indicates analysis of covariance of standardized response means with the intake scores as the covariate. 88 J Orthop Sports Phys Ther-Volume 30oNumber 40April 2000

7 TABLE 5. Utilization analyses.' Patients treated by Patients treated by certified nonspecialists orthopaedic clinical specialists (n = 29) (n = 29) ANCOVAS Variablet Min Max Mean SD Min Max Mean SD Source F df P Utilization Intake 24.4 < Physical functioning utilization Intake 43.3.I < Role physical utilization Intake 25.5 <.00 Bodily pain utilization Intake <.00 Mental health utilization Intake C I2 Vitality utilization Intake 42.6 < Social functioning utilization Intake < Numbers represent standardized response means. Min indicates minimum; max, maximum; and mean, unadjusted means. t Utilization indicates average standardized response mean from the 6 functional scores divided by visits times 0. Physical functioning utilization, role physical utilization, etc. represent the utilization for each functional scale, which is calculated by specific scale outcome standardized response mean divided by visits times 0. $ ANCOVA indicates analysis of covariance of standardized response means with the intake scores as the covariate. by patient self-report health status, in general, were unaffected by the level of specialization of the practitioner. In general, there was no difference in the outcomes for patients treated by physical therapists with OCS compared with patients (randomly matched) who were treated by clinicians without OCS. Physical therapists with OCS were more efficient because they treated their patients with fewer visits, estimated cost was less per episode, and they used fewer treatment techniques than practitioners who treated patients in the comparison group. Therefore, the primary finding of our study is that therapists with OCS who treat patients in acute orthopaedic outpatient facilities obtain similar health status improvement with fewer patient visits at lower estimated cost compared with a comparison group. If these data can be confirmed through prospective studies, the findings of improved efficiency without degradation of the effectiveness of care support the specialization process in orthopaedic physical thera- PY. Effectiveness, defined as the unit of functional improvement per episode of care, is a measure of clinical quality. Dividing quality by estimated cost to the payer provides an approximate measure of value, and dividing quality by visits provides a measure that we define as utilization. There were no differences between groups for overall value or utilization. However, analyses of the specific functional scales support the finding that patients treated by clinicians with OCS report improved value and utilization for the physical constructs of health status, particularly physical functioning and role physical. Physical functioning refers to performance of physical activities normal for people in good physical health. Activities as- sessed included running, lifting, carrying, walking, climbing stairs, vacuuming, bending, kneeling, bathing, et~.~~:~~.~~-~~ Physical functioning was the only individual health status functional scale that was different across groups, with patients seeing physical therapists with OCS reporting higher gains. The construct of role physical concerns the ability to accomplish normal roles of daily life. Activities included maintaining activity levels at work and during leisure or tasks at h~me.~~~~."-~wthough not statistically significant, patients treated by physical therapists with OCS tended to report better value and utilization for bodily pain, a physical constr~ct.~~~~~~~~-~ Taken together, these findings support the acquisition of OCS to improve the value of clinical outcomes. We controlled for the patient's perception of the severity of their functional problems by using an AN- COVA with the intake self-reported health status scores as the covariate for the assessments of health status at discharge value, and utilization. Patients were treated for similar episode durations, which es sentially controlled for the effect of time on the perception of health status between the groups. Since the number of visits was less for the OCS group, controlling for intake health status scores appeared to allow discrimination of clinical effectiveness between groups on utilization for physical functioning and role physical. However, the power of the ANOVA statistics for the dependent variables was low. Without a control group that received no treatment, we cannot determine if the health status improvement was related to the treatment provided. However, because of the between-group differences in visits, net revenue, health status in physical functioning, value of physical functioning and role physi- J Orthop Sports Phys Ther-Volume SO. Number 4.April2000

8 TABLE 6. Treatment contingency tables.' Journal of Orthopaedic & Sports Physical Therapy Patients treated by Patients treated by certified orthopaedic nonspecialists clinical specialists Treatment (n = 29) (n = 29) Odds ratiot Physical agents Moist heat Electrical stimulation Ultrasound Ice-compression* Procedures Massage* Myofascial techniques High-velocity manipulation Mobilization Exercises Body mechanic training* Closed-chain exercises Postural exercises Endurance exercises Flexibility exercises* Stabilization exercises Functional training+ Strength exercises* Home exercises* Joint mobilization Numbers are frequencies of patients in each contingency table cell. t Odds ratios included for variables with significant nonzero ratios between group and treatment. * Yates corrected x2 statistic, P <.0, df =. 5 Standardized deviates greater than 2.0 [(observed - expected)/(square root of expectedll. cal, and utilization of physical functioning and role physical, these data support further studies of outcomes associated with advanced clinical specialization. The SRM was used to quantify responsivene~s.~~-~.~~ The SRM represents a standardized change score that is not affected by sample size and has been shown to be a conservative measure of change.42 The combination of the SRM and ANCOVA with the intake self-reported health status scores as the covariate allowed a conservative statistical assessment of differences between the groups. If one applies the interpretation of standardized change scores that CohenJ -UP proposed for effect size scores to the SRM, on a group basis, the standardized change scores were large (ie, for both OCS and non-ocs). The SRh4s for physical functioning scales that ranged from are considered moderate. The SRh4s for mental functioning scales that ranged from were also considered moderate. Future studies are needed to determine how much change is necessary to meet a threshold for minimal clinically important difference in functional abilities or health-related quality of life..m-3' An effort was made to ensure a randomly selected, balanced, and homogeneous data set to which the J Orthop Sports Phys Ther*Volume 30-Number 4.April2000

9 OCS treatment group could be compared. Unfortunately, there were only 7 therapists with OCS, and they entered only 29 patients into the database in 996. The number of days between date of initial evaluation and December 3, 996, was not different across groups (OCS, ; comparison, 76.3? 3.3; Stailed b, = -.6; P =.2). The patients randomly selected for the comparison group were treated by 60 clinicians, including physical and occupational therapists and physical therapist assis tants. This created a statistically balanced data set according to the a priori variables, but the data set was not balanced according to professional qualifications. We had not anticipated this distribution a priori. The difference in professional qualifications and skills might have influenced the clinical outcomes. We did not investigate this question, but because of the obvious importance of differences in clinician demographics, this question deserves exacting prospective investigation, which was not the purpose of this article. We know that the physical therapists with OCS worked more hours (Table 2) and tended to have more years of direct patient care (although not significant), which might have influenced our findings. The low number of patients receiving high-velocity manipulation was not surprising given previous results.'vhose authors reported percentages of manipulation use from , depending on impairment category and payer type.'" Their range was similar to ours (0-3.0) without regard to impairment category and payer type. The new data represent an apparent lack of change in treatment choice for the use of high-velocity manipulation from for the participants of the FOTO process. Although we would have examined through regression analyses the effect of treatment use on clinical outcomes, we felt the limited size of the data set reduced the power of those potential results. Therefore, x2, odds ratio statistics, and standardized deviates were used to examine the frequency distribution of treatment techniques across treatment groups. Since there was a tendency for the comparison group to use more treatment techniques, including physical agents, procedures, and exercises, without improvement in clinical outcome, the data suggest similar outcomes at reduced utilization with patients treated by therapists with OCS. The study design does not allow any analysis of effects of treatment techniques on outcome. The current study used generic health status questions to assess patients' perception of change in functional abilities and health and well-being. Questions related to improved responsiveness to clinical change between types of assessment tools have been raised.' Some authors reported improved responsiveness to clinical change using disease, condition, or patientspecific questi~nnaires.'.~-~~.~ However, White4' assessed differences between perceived disability assessed via the Oswestry questionnairer and health-re- lated quality of life assessed via the SF-36 questionnaire2j24.w using a FOTO data set.'6jr-20 The condition-specific tool allowed better description only for patients with chronic low back syndromes compared with patients with less chronic condition~.~' These findings support other reports'"2r that there is little advantage to condition-specific tools over the multidimensional generic tools. Conditionspecific and generic health status tools have their strengths and limitations and may overlap on several constructs, reducing the need for both.2h Because the various outcomes tools measure different constru~ts,'j'~~~~~~~~~ a combination of questions from generic health-related quality of life and condition-specific and patient-specific functional tools should improve the understanding of clinical change.' Limitations As previously described,'"'*m this type of study is observational. Although we randomly selected patients using a priori variables that have been shown to influence health status outcomes, there was no attempt to compare or control for the types of clinicians treating the patients in the comparison group, the results of treatment, or the randomization of patients in any way. In future studies, controlling for the type of clinician a priori may strengthened the study design. The current design precludes conclusions about the effectiveness of therapy. Both groups of patients reported improvement of health status, but without a control group who received no therapy, evaluating the effect of physical therapy is not possible. No attempt to differentiate, describe, influence, or track changes in treatment over time was made. More detail describing or classifying patients who received treatment was not available for this study. We grouped patients by impairment categories related to the anatomical body part treated, which may be related to misclassification of impairment categories. Classifying patients according to movement signs and symptoms has been shown to improve outcome pre- diction4 and to stratify out~omes.~wo attempt was made to take advantage of improved outcomes with more precise patient ~lassification.~~~ The results represent analyses on a clinical database where practitioners purchase the services of an independent data management company for the purpose of assessing outcomes in a standardized manner with national external benchmarks. The results of the analyses are used for clinical quality improvement, clinician management, accreditation, marketing, and business decisions. Because this is a clinical database from practices electing to participate for many reasons, there are threats to external validity, including lack of a priori planning, missing observations, selection bias, and referral bias.'~202" Because J Orthop Sports Phys Ther-Volume SOeNumber 4.April 2000

10 we could not monitor the effect of the training pre cess of data collection for the FOTO outcomes process, we have no assessment of reliability of the method of following the data collection process. Rigorous data quality procedures were in place and used for automated monthly data checking for completeness. There was no attempt made to assess the level of vigor with which the provider followed the proposed data quality standards and recommended implementation processes.i0 Internal consistency reliability has been tested for all multiquestion scales,lj and construct validity and reproducibility of the physical functioning scale2 has been tested, but further reliability and validity analyses need to be conducted on several aspects of the FOTO tool. The patients in this study came from a sample of practices in 4 geographic regions covering several clinical settings that contained physical and occupational therapists and physical therapist assistants with a wide diversity of clinical skills, techniques, and experiences. The patients represent a small sample with a wide variety of characteristics, including clinical, demographic, and administrative variables. The sample does not represent a welldefined population but rather a common patient population in rehabilitation settings. This is an inherent strength and a threat to external validity for this type of study. It was a concern that the therapists with OCS only treated 29 patients in this data set. A larger sample population is recommended for future statistical analyses to improve statistical power. Despite these limitations, the design allows for preliminary analyses of the differences in outcomes between the patients who were treated by clinicians with and without OCS. We hope the analyses will facilitate discussion, improved design, implementation, and interpretation of future studies on health and well-being in patients receiving rehabilitation. Therefore, these results need to be considered preliminary and need independent confirmation. CONCLUSION Patients who received rehabilitation in acute orthopaedic outpatient facilities were studied to determine if patients treated by orthopaedic clinical specialists had better outcomes than those treated by nonspecialists. Therapists with OCS were more efficient: they used fewer visits and cost less than clinicians without OCS. Overall health status was not different across clinician groups. Value (unit of functional improvement per estimated cost) and utilization (unit of functional improvement per visit) were greater for the domains of physical functioning and role physical for patients treated by therapists with OCS compared with clinicians without OCS. ACKNOWLEDGMENTS We greatly appreciate the statistical review and recommendations from Paul W. Stratford, MSc, PT, and Holly Hollingsworth, PhD. Their comments consolidated our thoughts relating to the statistical tests used. REFERENCES Binkley JM, Stratford PW, Lott SA, Riddle DL. The lower extremity functional scale (LEFS): scale development, measurement properties, and clinical application. Phys Ther. 999;79: Chatman AB, Hyams SP, Neel JM, et al. The Patient-Specific Functional Scale: measurement properties in patients with knee dysfunction. Phys Ther. 997;77: Cohen J. Statistical Power Analysis for the Behavior Sciences. 2nd ed. New York, NY: Academic Press Inc; 988. Delitto A, Cibulka MT, Erhard RE, Bowling RW, Tenhula ]A. Evidence for use of an extension-mobilization category in acute low back syndrome: a prescriptive validation pilot study. Phys Ther. 993;73: Di Fabio RP. Physical therapy for patients with TMD: a descriptive study of treatment, disability, and health status. Orofac Bin. 998;2: Di Fabio RP, Boissonnault W. Physical therapy and healthrelated outcomes for patients with common orthopaedic diagnoses. I Orthop Spom Phys Ther. 998;27: Dobrzykowski EA, Nance T. The Focus On Therapeutic Outcomes (FOTO) outpatient orthopaedic rehabilitation database: results of J Rehabil Outcomes Meas. 997; : Fairbank JCT, Couper J, Davies JB, OfBrian JP. The Oswestry low back pain disability questionnaire. Physiotherapy. 980;66: Haley SM, McHorney CA, Ware JE. Evaluation of the MOS SF-36 physical functioning scale (PF-lo), I: unidimensionality and reproducibility of the Rasch Item Scale. ] Clin Epidemiol. 994;47: Hart DL. Effect of risk-adjustment on disability tables. PT Magazine. November 997:36. Hart DL. Relation between three measures of function in patients with chronic work-related pain syndromes. J Rehabil Outcomes Meas. 998;2: - 4. Hart DL. Assessment of unidimensionality of physical functioning in patients receiving therapy in acute, orthopaedic outpatient centers. In: Smith RM, ed. Outcome Measurement in the Health Sciences. Vol. Chicago, Ill: MESA Press. In press. Hart DL, Dobrzykowski E. Effect of exercise history on outcomes. In: Schunk C, ed. Orthopaedic Physical Therapy Clinics of North America. Philadelphia, Pa: WB Saunden Co. In press. lezzoni LI. Risk and outcomes. In: lezzoni LI, ed. Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor, Mich: Health Administration Press; 994:l-28. Jette AM. Using health-related quality of life measures in physical therapy outcomes research..phys Ther. 993;73: Jette AM, Delitto A. Physical therapy treatment choices for musculoskeletal impairments. Phys Ther. 997;77: Jette AM, Smith K, Haley SM, Davis KD. Physical therapy episodes of care for patients with low back pain. Phys Ther. 994;74:0-5. Jette DU, Jette AM. Physical therapy and health outcomes J Orthop Sports Phys Ther.Volume SO. Number 4.April 2000

11 in patients with knee impairments. Phys Ther. 996;76: Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther. 996;76: Jette DU, Jette AM. Professional uncertainty and treatment choices by physical therapists. Arch Phys Med Rehabil. 997;78: Kane RL. Looking for physical therapy outcomes. Phys Ther. 994;74: Liang MH, Rossel AH, Larson MG. Comparisons of five health status instruments for orthoped evaluation. Med Care. 990;7: McHorney CA, Ware JE, Raczek AE. The MOS 36-item short-form health survey (SF-36, II: psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 993;3: McHorney CA, Ware JE, Lu JFR, Sherbourne CD. The MOS 36-Item Short-Form Health Survey (SF-36, Ill: tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 994;32: Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. N Engl) Med. 987;37: Pryor DB, Lee KL. Methods for the analysis and assessment of clinical databases: the clinician's perspective. Stat Med. 99;0: Relman A. Assessment and accountability: the third revolution in medical care. N Engl ) Med. l988;3 9: Riddle DL, Stratford PW. Use of generic versus regionspecific functional status measures on patients with cervical spine disorders. Phys Ther. 998;78: Riddle DL, Stratford PW, Binkley JM. Sensitivity to change of the Roland-Morris back pain questionnaire: part 2. Phys Ther. 998;78: Stratford PW, Binkley JM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 996;76: Stratford PW, Binkley JM, Riddle DL, Guyatt GH. Sensitivity to change of the Roland-Morris back pain questionnaire: part. Phys Ther. 998;78: Stratford PW, Gill C, Westaway M, Binkley JM. Assessing disability and change on individual patients: a report of a patient specific measure. Physiother Canada. 995;47: Tugwell P, Bombardier C, Buchanan W, et al. The MAC- TAR Patient Preference Disability Questionnaire: an individualized functional priority approach for assessing improvement in physical disability in clinical trials in rheumatoid arthritis. ) Rheumatol. 987;4: Ware JE, Brook RH, Davies AR, Lohr KN. Choosing measures of health status for individuals in general populations. Am) Public Health. 98;7: Ware JE, Kosinski M, Keller SD. A 2-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 996;34: Ware JE, Kosinski M, Keller SD. SF- 2: How to Score the SF- 2 Physical and Mental Health Summary Scales. Boston, Mass: The Health Institute, New England Medical Center; Ware JE, Kosinski M, Keller SD. SF-36 Physical and Mental Health Summary Scales: A User's Manual. Boston, Mass: The Health Institute, New England Medical Center; Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36, I: conceptual framework and item selection. Med Care. 992;30: Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 Health Survey Manual and Interpretation Guide. Boston, Mass: The Health Institute, New England Medical Center; Werneke M, Hart DL, Cook D. A descriptive study of the centralization phenomenon: a prospective analysis. Spine. 999;24: White LJ. Measuring outcomes in patients with low back pain [dissertation]. University of Illinois at Chicago, Chicago,, Wright JG, Young NL. A comparison of different indices of responsiveness. ) Clin Epidemiol. 997;50: J Orthop Sports Phys Ther*Volume SO-Number 4.April