Q Methodology to Measure Physician Satisfaction with Hospital Pathology Laboratory Services at a Midwest Academic Health Center Hospital

Transcription

1 Q Methodology to Measure Physician Satisfaction with Hospital Pathology Laboratory Services at a Midwest Academic Health Center Hospital Laurie A. Gillard, MS, MT(ASCP) SBB, 1 Veronica N. Lewis, MS, MT(ASCP) SBB, 1 Robert Mrtek, PhD, 1 Colleen Jarosz, MBA, MT(ASCP) SBB 2 ( 1 University of Illinois at Chicago, Chicago, IL, 2 Loyola University Medical Center, Maywood, IL) DOI: /GEME4J9J4N2963V0 Science Received Revisions Received Accepted Assorted opinion statements relating to laboratory services (Q sample) were ranked according to preference by the individual physician (Q sort). Information gathered from the by-person factor analysis and focused discussions after the Q sort process will assist laboratory administration to make effective changes to improve quality of service. Hospital physicians appreciated the Q sort opportunity for direct interaction with pathology laboratory personnel and felt laboratory administration was committed to addressing and ultimately resolving service issues. The purpose of this study was to establish a mechanism that reflects an accurate representation of physicians opinions of laboratory services. This process paved the way for action plan development to improve pathology services for this pathology laboratory. A physician satisfaction survey for this midwestern university medical center pathology laboratory was initially developed to fulfill requirements for accreditation by the College of American Pathologists (CAP) College of American Pathologists, July 2003 Laboratory General (Web Version) page 13:GEN Phase I Have the referring physicians or patients satisfaction with the laboratory service been measured within the past 2 years? 1 Up to this time, surveys performed at this pathology laboratory have been of the traditional type, using rating scales which base responses to questions with answers such as highly agree, agree, neutral, disagree, and highly disagree. Such surveys tend to lose valuable information by averaging scores across respondents or across opinionnaire items which result in making broad, often low-impact consensus generalizations regarding laboratory service. After literature review and discussion, Q methodology was chosen as the survey tool for the present physician satisfaction survey. It provided staff physicians with an opportunity to meet with a laboratory representative to express their individual opinions regarding their interactions with the laboratory. Q methodology is a research tool for subjective study of individuals points of view. It was developed by W. Stephenson in 1935 and has been used worldwide to conduct behavioral research. 2 The researcher compiles statements that the participant sorts according to a specific condition of instruction. Stephenson states Q emphasizes uniqueness. It makes no predictions and looks for no tests of hypotheses. It emphasizes the discovery of unexpected characteristics and gives these discoveries center stage. 2 Q methodology provided the laboratory administrators with an opportunity to see laboratory services through the eyes of the physicians. Since this survey includes a personal interview, Q methodology also ensures a high response rate, unlike mailed surveys that often result in minimal feedback. In this Q sort, the statements consisted of assorted laboratory services that the doctors were asked to rank according to their level of satisfaction, from most dissatisfied to most satisfied, and record their results on the Q sort response grid. Methods Subjective statements were collected from previous medical center customer satisfaction surveys, Web site surveys, and also survey questions from journal articles. This series of statements, or concourse, consisted of a large number of subjective feelings about topics about satisfaction with hospital laboratory service. From this concourse, a representative but not exhaustive subset, or Q sample, of 23 statements was selected and refined by a group of laboratory professionals based upon the most relevant pathology laboratory service issues (Table 1). Each of the 23 statements was randomly assigned a sequence number and printed individually on a 2 x 3 card. A Q sort response grid was designed to serve as a graphic aid and for each participant s documentation. It consisted of 23 rectangles laid out in columns of predetermined lengths to form a bell shaped curve into which respondents would write 1 statement number in each rectangle (Image 1). While thinking about the hospital pathology laboratory services, the participant read all the statements and sorted the cards into 3 piles; those statements he/she felt satisfied with, those he/she felt neutral about, and those he/she felt dissatisfied with. Then the participant was asked to rank the 2 statements from the satisfied pile as his/her MOST satisfied. Those cards were placed to the far right in front of the participant s work area. From the remainder, they were then asked to select the next three most satisfied statements from the pile and place those to the left of the first two cards. The statement ranking process was continued until all the satisfied statements had been ranked on the response grid columns. Using the pile of laboratory service statements with which the participant felt dissatisfied, the above sort order was reversed by first placing the 2 MOST dissatisfied cards at the far left; the next 3 dissatisfied cards were placed to the right of the first 2 cards, and so on until all the dissatisfied statement cards were arranged on the grid columns. Those statements the participant felt neutral towards were placed in the center area, between the satisfied and dissatisfied statements. The participant then transferred the numbers printed on the statement cards to the Q sort response grid, 1 statement number June 2005 Volume 36 Number 6 LABMEDICINE 361

2 Table 1_Q Sample for Physicians Number Item 1 The stat turnaround time is reasonable for my laboratory orders. 2 Critical values are called to me in an acceptable time frame. 3 The test menu selection is adequate for my patient population. 4 The routine test results are returned to me in a timely manner. 5 My patients are satisfied with laboratory phlebotomy. 6 The laboratory personnel are knowledgeable. 7 Transport of specimens to the laboratory is reliable. 8 The laboratory technologists are helpful when questions arise. 9 The laboratory services manual is easy to use. 10 Test results are easily accessible. 11 Management of the pathology laboratory is concerned with providing good customer service. 12 The laboratory has a positive image in this medical community. 13 Laboratory reports are clear and easy to read. 14 The quality of work is consistent with all shifts (24/7). 15 The incidence of laboratory error is within an acceptable range. 16 The correct ordering physician is on the laboratory reports. 17 Laboratory personnel act in a professional manner. 18 Pathologists are available to answer my questions. 19 Laboratory couriers are available when needed. 20 Misplaced or lost specimens are not a problem. 21 My patients can easily find the laboratory in order to have blood drawn for tests. 22 The people answering the phones are pleasant and helpful. 23 When blood products are ordered stat, the blood bank responds quickly. Image 1_Q sort response grid. per rectangle (Figure 1). After completing the sort, each participant was invited to respond to 3 open-ended questions: Why did you choose the 2 statements as your most satisfied? (Table 2) Why did you choose the 2 statements as your most dissatisfied? (Table 3) If we could do one thing to improve our pathology laboratory service to you, what would it be? (Table 4) To select the participants, the chairman of the Pathology Department, contacted the chairman of the following services: Hematology-Oncology, Primary Care, Emergency Medicine, Surgery, Anesthesiology, General Medicine and Cardiology requesting the participation of 2 attending and 2 resident physicians. Once the physicians were identified, they were scheduled individually or in a small group to participate in the Q sort. A representative of the laboratory was present during the Q sort to assist in the completion of the survey. The laboratory representative provided verbal and written explanations, as well as consent forms, to the physicians. Twenty-three physicians participated in the physician satisfaction survey and 21 surveys were completed. Two physicians were unable to complete the Q sorts. One physician felt he had no criticism of the laboratory; and therefore, he could not separate the statements into a dissatisfied pile. The second physician was repeatedly paged and could not stay for the complete survey. The Q sort required 15 to 20 minutes to complete. The data were analyzed using PQMethod software, version 2.11, available at PQ Method requires the application of 3 statistical procedures: development of a 21 by 21 intercorrelation matrix of pair wise Pearson correlation coefficients, factor analysis of the Q-sort matrix using persons as variates, and the computation of standardized normal factor scores. 1 An important feature of Q factor analysis is the intercorrelation matrix of Q-sorts for by-person factor analysis. It is people, not statements, who form clusters of shared subjectivity. The factor analysis is used to determine the number of groups (factors) contained in the structure of the intercorrelation matrix via factor extraction and simplification of factor structure by axis rotation. The calculation of factor scores is used to gauge the relative significance of each statement in the identified factor. 3 The program identifies relationships between the Q sorts and then groups the common responses together. Each column position of the Q sort response grid has a numerical value, ranging from +3 to -3. All the statements within 1 column receive the same score. When the statements are sorted and the numbers are recorded on the survey grid, the statements now have a unique distribution on the grid. Factor types are identified by shared opinion types. A Q methodological approach makes it possible to identify and classify physician opinion types by examining subjective preference issues that are both common and different to all participants. Those who sort items in similar ways will have prepared Q sorts that correlate. Each factor represents a common perspective of all participants who cluster on that factor. The ultimate goal of Q methodology is to reveal various models or viewpoints of thought not numerical distribution of each statement within a viewpoint nor even the relative prevalence of each opinion type within the larger health care provider community. Figure 1_Answer sheet. Results Q methodology provides a way of exhibiting the form and structure of opinions and views, which would otherwise remain hidden in more traditional attitude questionnaires. 4 Statements are ranked from most satisfied (+3) to most dissatisfied (-3) based on location in the distribution curve (Image 1). When the Downloaded 362 from LABMEDICINE Volume Number 6 June 2005 labmedicine.com

3 Table 2_Sample of Physician Responses to Why did you choose the two statements as your most satisfied? Statement #4: Routine test results are returned in a timely manner. Most CBC results are available to me when I am seeing the patient, frequently other results are also available in this time frame. I can always expect results the following day. Statement #6: Laboratory personnel are knowledgeable. I always feel that questions that arise are answered to my satisfaction in a professional and pleasant manner. Statement#13: Laboratory reports are clear and easy to read. Presentation in EMR (electronic medical record) for complicated blood tests is excellent. Statement #23: When blood products ordered STAT, blood bank responds quickly. Most of my dealings are with the blood bank in crisis situations. They are very helpful in insuring products and communicating problems with specimen attainment and distribution. Table 3_Sample of Physicians responses to Why did you choose the two statements as your most dissatisfied? Statement #1: STAT turnaround time. Troponins need more rapid TAT (this was mentioned by several physicians). Intraoperative coagulation testing for surgical tests important to monitor patients, (this was the response of several anethesiologists). Statement #4: Routine test results. If there is a delay in surgical pathology results, the surgeons would like pathology to contact them through or a phone call with an explanation (additional stains, second opinion, etc). Statement #20: Misplaced or lost specimens. Misplaced or lost specimens generate multiple phone calls and a lot of wasted time. Table 4_A Sample of Physicians responses to If we could do one thing to improve our service to you, what would it be? 1. Need to improve intraoperative coagulation testing, whether it be a new processing system or new point of care testing. 2. Less On-Hold time. Maybe create a physician s call line? 3. Move laboratory into the emergency department with point-of-care testing. 4. Clear chain of command for questions from laboratory tech to manager to pathologist, 24/7. 5. Computers in each OR room so that laboratory results could be accessed. 6. Attending physicians could be called whenever a path report is greater than 3 to 4 days with an explanation for the delay. This would alleviate much anxiety for the patient. data are analyzed, a matrix of correlation coefficients is generated. Correlation coefficients demonstrate the extent to which Q sorts agree or disagree with each other. Factor analysis identifies groups of Q sorts, which on the strength of their correlations, appear to match up as an opinion type. As a result of this Q sort matrix analysis, the physicians are correlated to each other and factored into groups according to degree of similarity in the way they rank the statements. This is considered as by person factor analysis. 3 The physicians who participated in this study displayed 6 distinct opinion types, and none of these viewpoints was highly correlated with any others. Factor 1 consisted of 3 physicians who were either anesthesiologists and/or surgeons. The predominate feelings of this group was that the laboratory was doing good work, the pathologists are available for consultation, and the laboratory reports are easy to read. The major dissatisfied statement was the availability of blood products and the turnaround time of both stat and routine test results. Factor 1_Anesthesiology/Surgery Group #1 15 Incidence of Laboratory Error Acceptable Pathologists Available for Questions Laboratory Reports easy to Read Quality of Work Consistent 24/ Patients Satisfied with Phlebotomy -1 1 Stat Turnaround Time -1 4 Routine Test Turnaround Time Blood Product Availability -2 Factor 2 was composed of 3 internal medicine physicians. They felt the laboratory was doing a good job of calling critical test results to the physician. They also liked the format of the laboratory reports. However, this group was dissatisfied with the test menu selection and availability of laboratory couriers to transport specimens. Factor 2_Medicine 2 Critical Values Called Laboratory Reports Easy to Read +2 4 Routine Tests Turnaround Time +1 3 Test Menu Selection Adequate Laboratory Courier Available Pathologists available for Questions -1 7 Specimen Transport Reliable -2 Factor 3 participants included a pediatrician and 2 internal medicine physicians. They were satisfied with the availability of pathologists for questions, the test menu selection, stat turnaround time, and the calling of critical test results. Lost or misplaced specimens, the inconsistent quality of work on all shifts, and blood product availability, dissatisfied them. June 2005 Volume 36 Number 6 LABMEDICINE 363

4 Factor 3_Medicine/Pediatrics 18 Pathologists Available for Questions +2 3 Test Menu Selection Adequate +1 1 Stat Turnaround Time +1 2 Critical Values Called Quality of Work Consistent 24/ Blood Product Availability Laboratory Couriers Available Misplaced or Lost Specimens -2 Factor 4 consisted of another group of 3 operating room physicians, either anesthesiologists and/or surgeons. This group shared the dissatisfied opinion that Stat turnaround time was too long, but interestingly, they were satisfied with the availability of blood products. They also felt the quality of work was not consistent on all shifts. Factor 4_Anesthesiology/Surgery #2 Number Statements Score 22 The people answering the phones are +2 pleasant and helpful. 23 Blood Product Availability +2 1 Stat Turnaround time Quality of Work Consistent 24/7-2 Factor 5 was composed of 2 internal medicine physicians who worked in both outpatient and inpatient settings. They thought the pathology laboratory had a positive image in this medical center community. They also thought the medical technologists were helpful and knowledgeable, test results were easy to access, and laboratory reports were easy to read. They felt strong dissatisfaction with the problem of misplaced or lost specimens as well as dissatisfaction with delays in receiving critical value results by phone. Factor 5_Ambulatory Medicine 8 Technologists Helpful with Questions Positive Laboratory Image Laboratory Reports Easy to Read +1 6 Knowledgeable Laboratory Personnel Pathologists Available for Questions The people answering the phones are -1 pleasant and helpful. 16 Correct Ordering Doctor on Report Misplaced or Missing Specimens -2 2 Critical Values Called -2 Factor 6 consisted of an emergency room physician and an internal medicine physician. They felt critical values were called promptly, the medical technologists were helpful, professional, and knowledgeable. Their major dissatisfaction was with the turnaround time of test results. Also, misplaced or lost specimens and test result accessibility were a concern. Factor 6_Medicine/Emergency Medicine 2 Critical Values Called +2 8 Technologists Helpful with Questions +2 6 Knowledgeable Laboratory Personnel +1 3 Test Menu Selection Adequate Misplaced or Lost Specimens Incidence of Laboratory Errors Acceptable Test Results Accessible -1 1 Stat Turnaround Time -2 Discussion The Q sorts of the physicians provide a way of viewing laboratory services from the perspective of the doctors, allowing all viewpoints to be acknowledged. Through the statistical use of correlation and factor analysis, factors are identified which recognize physicians that share a common viewpoint with regard to what laboratory services mean to them. Seventy-six percent of the participating physicians, (16 of the 21 Q sorts), loaded on 1 of the 6 factors. The remaining 5 physicians were mixed loaders, which meant they shared opinions with multiple factors. This survey tool demonstrates that various medical specialties have specific needs and priorities that are not always met by the laboratory. When the factors were identified, Factor 5 reported dissatisfaction with the correct ordering doctor on report. One group of internal medicine physicians discussed the problem of receiving patient laboratory reports of other doctors. This problem was also pointed out by 1 of the attending physicians because she would often receive the reports of patients her residents had seen. This is a significant issue that using Q methodology helped draw attention to. The cluster of opinion surrounding this issue emphasized the importance it held to this group of physicians and also identified deficiencies in the laboratory test ordering process. Another interesting viewpoint highlighted by using Q methodology involved the 2 groups of operating room physicians, Factor 1 and Factor 4. These 2 groups of physicians had conflicting views of stat blood availability. The Q sort demonstrated the individuality of the factors. Both groups agreed turnaround times for stat testing was too long. Factor 4 felt the availability of blood products was +2 satisfactory while the Factor 1 felt dissatisfied to the degree of grading it a -2. This addresses inconsistencies in the administration of blood products to the operating room but simultaneously provides useful information to laboratory administrators, where to focus service improvements to the surgical service. Using the Q sort survey tool resulted in an 82% response rate (23 doctors participated, 28 were initially requested to participate). This is a much higher response rate when compared to prior surveys. We suggest that having a representative of the laboratory present during the Q sort to assist in the completion of the survey gave the impression that the laboratory administrators were genuinely interested in what the physician had to say. In addition, we also propose that the presence of the laboratory representative facilitated effective communication between laboratory services and the physician. Q methodology assumes issues can take on different meanings and relevance depending upon experiences of individual physicians. (Recall as an example, the distinctly different viewpoints involving the operating room physicians.) The relative importance of the individual Q sort is revealed when the 364 LABMEDICINE Volume 36 Number 6 June 2005 labmedicine.com Downloaded from

5 Table 5_Sample of Ideas for Action Plan Development: Identifying Areas for Improvement Based on Physician Responses: 1. STAT and routine TAT: review and monitor STAT TAT and TAT for high volume test. 2. TAT for blood products: discuss with blood bank personnel possible solutions to reduce TAT. 3. Test menu selection: identified as an issue with 1 factor (general medicine physicians) communicate with group to identify tests which could be evaluated and added to the test menu. 4. Availability of laboratory couriers: increase courier runs to the outpatient laboratories. This could potentially impact (improve) TAT. 5. Correct ordering physician on report: with new physician order entry system implemented in 2005, this problem should be resolved. 6. Incidence of laboratory errors: Identify the incidence of corrected reports and analyze the data. physician expresses his/her satisfaction and dissatisfaction when answering the 3 questions on the survey form. The physician also has the option of discussing problems regarding quality service with the laboratory representative at the time of the survey. Limitations The Q sort forces the participant to prioritize their feelings. One physician could not sort the cards because he did not feel dissatisfied with any of the laboratory services. One of the limitations is the choice of statements has to be broad enough to include enough negative as well as positive statements to represent relevant opinions. The process of Q sorting gives the physician control in deciding what it is about a statement that is significant to them. It can allow for a more thorough expression of the viewpoint of the individual physician doing the Q sort. In the situation where the participant was not able to complete the Q sort, he was encouraged to offer comments at the bottom of the survey form to the best of his ability. Although the information collected was incomplete, it still provided the laboratory administrators with the abbreviated opinion of this physician. Conclusion Q methodology provides an approach of understanding laboratory service perspectives of a convenience sample of physicians who use the pathology laboratory services. This survey suggests that opinions of physicians towards laboratory services are related to and affected by their medical specialty; consequently improvements can be made strategically, focusing on areas identified as in need of service improvement (Table 5). Certain key issues such as stat test turnaround time and laboratory error remain an important topic for most of the physicians. However through the recognition of shared opinions which were displayed as factors, distinct problem areas were identified. This will enable laboratory administrators to efficiently address and improve quality of service to these areas, focusing the allocation of funds in a more cost effective manner, and monitor service satisfaction through continuing communication with the physicians and the support staff. LM 1. Sarewitz SJ. Laboratory accreditation checklists. College of American Pathologists. July 2003;GEN McKeown B, Thomas D. Q methodology. Newbury Park, CA: Sage Publications, 1988: Mrtek R, Tafesse E. Wigger U. Q-methodology and subjective research. J Social Admin Pharm. 1996;13: Cordingley L, Webb C, Hillier V. Q methodology. Nurse Researcher. 1997;4: Definitions Q methodology is a research method which uses a statistical approach to measure subjective feelings. This method creates a 3 dimensional representation of the participant making the Q sort. Concourse is a representative set of statements relating to the subject to be studied. Q sample is selected from the concourse. These statements must be opinions and viewpoints which embody the major issues relating to the subject being studied. Q sort is the actual arrangement of the Q sample statements by the participant. Likert scale is a method used to rank items via a 1-to-5 disagrees to agree scale. This is a unidimensional scaling method. By-person factor analysis is the definition of using the individual as the variable and the statements as the stimuli. Factor analysis involves people, therefore, the factor analysis identifies groups of individuals with similar viewpoints. Operant subjectivity is the communication of an individual s (the operant) subjective opinions or points of view. Factor loading represents the correlation of each participant s statement on the identified factors. Factors represent a cluster of similar points of view. Each factor corresponds to a different group of opinions. Suggested Reading List Valenta A, Wigger U. Q-methodology: Definition and application in health care informatics. J Am Med Informatics Assoc. 1997;4: Jahn M. Clinicians examine the lab, and they like what they see. MLO. 1997;4: Jahn M. Doctors speak their minds on lab profiles and problems. MLO. 1997;6: Steindel S, Howanitz P. Physician satisfaction and emergency department laboratory test turnaround time. Path Lab Med. 2001;125: Howanitz P, Hoffman F, Schifman R, et al. Nationwide quality assurance program can describe standards for the practice of pathology and laboratory medicine. Quality Assurance Health Care. 1992;4: Galloway M, Nadin L. Benchmarking and the laboratory. J Clin Path. 2001;54: Zimmermann PG. What can be done to enhance customer satisfaction? J Emerg Nursing. 1997;23: Zimmermann PG, Pierce B. Customer satisfaction. J Emerg Nursing. 1999;25: Zimmermann PG. Measuring customer satisfaction: How do you measure customer satisfaction? J Emerg Nursing. 1998;24: Mayer T, Zimmermann PG. ED customer satisfaction survival skills: One hospital s experience. J Emerg Nursing. 1999;25: HRSolutions, Inc. Physician Satisfaction Surveys. Available at: Accessed on November 15, June 2005 Volume 36 Number 6 LABMEDICINE 365