A Preliminary Evaluation of the Effectiveness of the Cystic Fibrosis Foundation Mentoring Program for Respiratory Care

Transcription

1 A Preliminary Evaluation of the Effectiveness of the Cystic Fibrosis Foundation Mentoring Program for Respiratory Care Kathleen M Richards RRT, Mary K Lester RRT, Melissa J Chin, and Bruce C Marshall MD BACKGROUND: In 2008 the Cystic Fibrosis (CF) Foundation launched the Respiratory Therapy Mentoring Program, which pairs a respiratory therapist (RT) relatively new to CF (apprentice) with a highly experienced RT (mentor) from a similar CF care center. We wished to determine if we had achieved our short-term goal of increasing CF-specific knowledge among the apprentices who participated in the program. METHODS: Selected apprentices were each matched with a mentor, based on characteristics of CF population, clinical setting, center size, and geographic location of their care centers. Apprentices completed a CF-specific RT knowledge self assessment tool prior to and after a site visit to their mentor s center. Mentors also completed a post site visit knowledge self assessment tool regarding their apprentice. RESULTS: Thirty-seven apprentices completed a pre and post site visit knowledge self assessment tool. The median pre and post site visit scores were 12 and 31 (P <.001) respectively. The mentors post site visit scores of their apprentices (median 29, P.07) did not significantly differ from the apprentices post site visit scores. CON- CLUSIONS: The results of this preliminary evaluation suggest that the RT mentoring program has achieved its short-term goal of increasing CF-specific knowledge among RTs relatively new to CF care. Key words: cystic fibrosis; respiratory therapy; mentoring; quality improvement; education. [Respir Care 2013;58(5): Daedalus Enterprises] Introduction Ms Richards is affiliated with the Pulmonary Function Laboratory, Primary Children s Medical Center, Salt Lake City, Utah. Ms Lester is affiliated with the Medical University of South Carolina, Charleston, South Carolina. Ms Chin and Dr Marshall are affiliated with the Cystic Fibrosis Foundation, Bethesda, Maryland. Supplementary material related to this paper is available at The authors presented a version of this paper at the North American Cystic Fibrosis Conference, held November 3 5, 2011, in Anaheim, California. Ms Richards has disclosed a relationship with Pharmaxis. Ms Lester has disclosed a relationship with the Cayston Allied Health Care. The other authors have disclosed no conflicts of interest. Correspondence: Kathleen M Richards RRT, Pulmonary Function Laboratory, Primary Children s Medical Center, 100 North Mario Capecchi Drive, Salt Lake City UT kathleen.richards2@imail.org. Cystic fibrosis (CF) is an autosomal recessive, multisystem chronic disease affecting an estimated 30,000 people in the United States. The pulmonary manifestations account for the majority of the morbidity and mortality. 1 Patients with CF have difficulty clearing mucus from their lungs, resulting in chronic pulmonary infections and inflammation, which leads to bronchiectasis. In order to maintain lung health, clearance of airway secretions is recommended. 2 Multidisciplinary care, aggressive pulmonary and nutritional management, and newly introduced therapies have resulted in significant improvements in the length and quality of life for people with CF. SEE THE RELATED EDITORIAL ON PAGE 889 The CF Foundation accredits more than 110 CF care centers across the United States, caring for approximately 25,000 patients. The care centers are comprised of multidisciplinary teams that include physicians (typically pul- DOI: /respcare RESPIRATORY CARE MAY 2013 VOL 58 NO 5

2 monologists serving as center directors), nurses (typically serving as clinical coordinators), respiratory therapists (RTs), dietitians, social workers, and other healthcare professionals. 3 The CF RT is an integral member of the team. The optimal care of a CF patient requires knowledge of the disease and its impact on the lungs and other organ systems, pulmonary function testing, airway clearance techniques, chronic medications for maintenance of lung health, pulmonary complications, and infection control issues. 2,4-6 Practicing RTs must also build a strong partnership with patients and families in order to customize care to the needs and preferences of the individual and promote selfmanagement strategies and adherence to the daily medical regimen. Because of the specialized nature of CF care, CF RTs in the United States gain most of their knowledge and experience through on-the-job training from other members of their care center team and annual participation in courses and workshops at the North American Cystic Fibrosis Conference. As part of the CF Foundation s ongoing quality improvement initiative, the Foundation launched the CF Respiratory Therapy Mentoring Program in The primary goal of this study was to assess the effectiveness of this new program by measuring CF-specific knowledge among apprentices with a CF-specific RT Knowledge Self Assessment Tool (RT KAT) designed for this study, before and after completion of the program. In addition, we conducted surveys of the apprentices and their center directors/clinic coordinators to collect subjective feedback on the program. Methods The CF Foundation, which provided administrative and financial support for the mentoring program, chose 3 highly experienced RTs to serve as facilitators for the program and to develop a CF Respiratory Therapy Resource Document (RT 101). This comprehensive document covered the fundamental knowledge thought to be important in delivering competent respiratory care to CF patients. It is available to all RTs on the CF Foundation s intranet. RT Knowledge Assessment Tool The RT 101 document served as the template for development of the RT KAT, an assessment tool that encompassed a wide range of topics pertinent to CF patient care, such as airway clearance techniques, infection control, equipment and devices, and inhalation therapy. It consisted of 13 questions, with 19 scored responses (see the supplementary materials at The selected mentors reviewed the RT 101 document and the RT KAT before launch of the mentoring program, and there was a consensus among these experts that the RT KAT QUICK LOOK Current knowledge The Cystic Fibrosis (CF) Foundation accredits more than 110 CF care centers across the United States, which care for approximately 25,000 CF patients. The CF respiratory therapist (RT) is an integral member of the team. Optimal care of a CF patient requires knowledge of the pathophysiology, pulmonary function testing, airway clearance techniques, lung health maintenance medications, pulmonary complications, and infection control issues. In the CF Foundation s respiratory therapy mentoring program, experienced mentors give RTs specialized training in CF respiratory care. What this paper contributes to our knowledge The CF Foundation mentoring program was found to effectively improve apprentice RTs knowledge of this disease. covered the key areas of CF-specific knowledge for RTs working at a CF care center ( face validity ). No formal validation or psychometric analyses were conducted on the RT KAT. Apprentices were asked to provide an honest self-assessment of their RT knowledge and skills at both the beginning and end of the program (pre and post site visit RT KAT). Each mentor was also asked to complete a post site visit RT KAT to assess their apprentice s knowledge. All the answers were scored using the following Likert rating scale: disagree ( 2), ( 1), (1), agree (2). The minimum and maximum scores were 38 and 38, respectively. Program Evaluation Surveys To further assess the effectiveness of the mentoring program, 2 surveys were conducted. First, a survey of the apprentice s center director and/or clinic coordinator was conducted 3 6 months after the site visit. This survey evaluated the apprentice s impact on their center s CF clinical practice. Second, in the fall of 2010, a survey was conducted of all former apprentices. The completion of this survey was anonymous, preventing the identification of the responders and non-responders. It assessed the apprentice s confidence in performing their role and perceived improvement in CF patient care at their center. Questions regarding continued engagement in CF care and amount of time spent caring for CF patients were also asked to assess the retention of RTs after participation in the program. Both surveys were conducted using Survey Monkey ( RESPIRATORY CARE MAY 2013 VOL 58 NO 5 765

3 Selection of Participants and Matching Starting with a successful pilot, from 2008 to 2010, the program consisted of 2 sessions per year, with approximately 10 apprentice/mentor matches per session. Of the 81 apprentices who applied, 44 were chosen to participate, based on criteria described below. In 2011 the program was changed to 1 session per year, with matches. A call for applications was sent out at the beginning of each session, via the CF RT Listserv and an communication to all CF care center directors. A program overview was provided, which outlined the roles and expectations of the mentors and apprentices. All applications were assessed objectively using a point system. Mentor applications were evaluated and scored based on years of CF experience (a minimum of 5 years is required), number of hours worked, percent of time devoted to CF care, and level of involvement in the CF respiratory therapy community (eg, participation at national conferences, consortia). Selected mentors agreed to participate in the program for a minimum of 2 sessions. Apprentice applications were evaluated and scored based on years of CF experience ( 3 years was prioritized), center size, number of hours worked, percent of time devoted to CF care, and the availability of other CF RTs at the center. For example, applicants with less CF experience, working at a high volume CF center and with no other experienced CF RTs, scored higher than those with more CF experience at a smaller center with the support of other CF RTs. Those with the highest point totals were selected to participate in the program. The mentors and apprentices were then matched based on the characteristics of the CF population they served (adults/pediatrics/both), their clinical setting (in-patient/ out-patient/both), center size, and the geographic location of their respective CF centers. Site Visit Process Mentors and apprentices were notified of their acceptance into the program and were provided contact information for their counterpart, a checklist of responsibilities, guidelines for developing learning objectives, and a copy of the CF RT 101 resource document. Apprentices were also asked to complete the pre-site-visit RT KAT. The completed RT KAT and the apprentice s application were provided to his/her mentor. Conference calls were then conducted between the facilitator and each mentor/apprentice pair to review expectations and site visit logistics. Each apprentice developed learning objectives for the site visit with input from his/her own CF care team and the assigned mentor. Once the objectives were finalized, the mentor prepared an agenda for the 1 2 day site visit. The visit typically included a meeting with the multidisciplinary Table 1. Characteristics of Apprentices and Mentors: CF team, a review of the RT 101 document and the mentor s CF respiratory specific educational and patient assessment materials, shadowing the mentor in out-patient clinic and/or during in-patient rounds, pre and post clinic conferences, and a debrief at the end of the visit to review the learning objectives. Within 2 weeks after the site visit, the apprentice and mentor completed a post site visit RT KAT. Outcome Measures The RT KAT score of the apprentices and mentors was the primary outcome measure. Apprentices who completed both a pre and post site visit RT KAT were included in the analysis. If an apprentice did not answer a specific question on either the pre or post site visit RT KAT, that question was excluded from the analyses. If an apprentice or mentor did not answer a specific question on the post site visit RT KAT, that question was excluded from the analyses. The total scores are presented as medians with minimum and maximum values. Statistical Analyses Apprentices Mentors Total number of participants Years of experience Median 20 Range 7 30 Program Pediatric 15 9 Adult 6 3 Both Number of CF patients Median Range Percent time devoted to CF Median Range Service Out-patient 11 3 In-patient 4 0 Both The pre and post apprentice scores for each question and for the sum totals were compared with a Wilcoxon signed-rank test. The apprentices and mentors post site visit RT KAT scores for each question and the sum totals 766 RESPIRATORY CARE MAY 2013 VOL 58 NO 5

4 Table 2. Distribution of Apprentices Knowledge Self Assessment Responses Pre Site Visit (%) Post Site Visit (%) Question P* Lung disease Pathogens Infection control Disinfecting respiratory equipment Airway clearance prescription Airway clearance techniques Percussion and postural drainage Active cycle of breathing Autogenic drainage PEP/airway oscillation device Intrapulmonary percussive ventilation High frequency chest wall compression Huff cough Medications - guidelines Medications - registry Medications - side effects Respiratory equipment Order of medications Education of patients/families Customization of airway clearance * P values from Wilcoxon signed-rank test. PEP positive expiratory pressure device were compared with a Mann-Whitney test. A P value (2 tailed).05 was considered statistically significant. Participants Results Since inception, 44 apprentices and 23 mentors have participated in the program. The apprentices were selected from among 81 applicants. As shown in Table 1, the mentors had a median of 20 years of experience in CF care (range 7 30 y). The mentors were from both pediatric and adult CF care programs and various center sizes, with a median of 170 patients (range patients). The median proportion of time that the mentors devoted to CF care was 55% (range %). Twenty of the 23 mentors worked in both the out-patient and in-patient setting, and only 3 of the 23 provided care exclusively in the out-patient setting. Of the apprentices who participated in the program, 33 of 44 had 3 years experience in CF care, and 20 had 1 year. The apprentices also represented pediatric and adult programs, with 23 of 44 working in both. The median center size among apprentices was 120 patients (range ) and the median time devoted to CF was 38% (range %). Twenty-nine of the 44 apprentices provided care in both the in-patient and out-patient setting. RT KAT Results Thirty-seven mentor/apprentice pairs (excluding the pilot session) completed a pre and post-site visit RT KAT. Table 2 shows the distribution of apprentice responses to each question in the pre and post site visit RT KAT. Of note, before the site visit, over half of the apprentices answered disagree when asked if they were able to discuss, demonstrate, and instruct patients using airway clearance techniques, such as active cycle of breathing, autogenic drainage, and intrapulmonary percussive ventilation, and over a third disagreed with questions concerning the their own knowledge of transmission of pathogens, medication use at their care center, adverse side effects of medications, and educational resources for patients and families. The median apprentice pre site visit total score was 12 (range 13 to 35) and the median apprentice post site visit total score was 31 (range 17 to 38), a statistically significant difference (P.001). The difference between pre and post site visit scores was also statistically significant for each individual question on the RT KAT (see Table 2). RESPIRATORY CARE MAY 2013 VOL 58 NO 5 767

5 Table 3. Apprentice Program Evaluation Survey Discussion The apprentices and mentors post site visit RT KAT scores did not significantly differ: median of 31 (range 17 to 38) and 29 (range 12 to 38), respectively (P.07). The apprentices and mentors post site visit scores also did not differ significantly for any of the individual questions (data not shown). Survey Results n % Total participants (44) 38 (86) Are you still working with CF patients in CF care? Yes 37 (97.4) No 1 (2.6) Has the amount of time that you spend at work caring for CF patients increased? Yes 20 (52.6) No 18 (47.4) Do you still have contact with your mentor? Yes 22 (57.9) No 16 (42.1) Do you feel comfortable contacting your mentor with clinical questions or concerns? Yes 37 (97.4) No 1 (2.6) Have you participated/presented work pertaining to CF at conferences, consortia, et cetera, since participation in the mentoring program? Yes 16 (43.2) No 21 (56.8) Skipped question 1 Do you feel the level of CF respiratory care has improved at your center since participation in the mentoring program? Yes 33 (86.8) No 5 (13.2) Have you been able to accomplish, and/or implement the goals that you set for this program? Yes 28 (73.7) No 10 (26.3) CF cystic fibrosis Twenty-six of 37 (70%) of the apprentices center directors/clinic coordinators completed the survey, and 72% responded that CF respiratory care had improved at their center since the RT participated in the program, and 80% responded that their RT was more knowledgeable and active in CF care. Among the 44 apprentices, 38 (86%) completed the program evaluation survey. The results are shown in Table 3. Of note, 33 of 38 (86.8%) reported that the level of respiratory care had improved at their center since participation in the program. The CF Foundation mentoring program provided an opportunity for an RT relatively new to the role as a member of the CF multidisciplinary care team to work closely with and learn from a highly experienced RT. In this study we found that apprentices self-assessed CF-specific knowledge significantly increased following participation in the mentoring program. Results of the program evaluation surveys completed by the apprentices and their center directors/clinic coordinators provide supportive evidence of the value of the program. These findings suggest that the mentoring program is meeting the short-term goal of increasing CF-specific knowledge among RTs who are relatively new to CF care. Anecdotal evidence pointed to additional unexpected benefits of the program. First, we believe the high retention rate reported among apprentices is an important finding that may be related to improved job satisfaction. An engaged and experienced workforce will increase the likelihood that we will achieve our overarching goal (ie, to improve the quality of respiratory care and healthcare outcomes for CF patients). Of note, several apprentices have now returned to the program as mentors, including one who is currently serving as a facilitator to oversee the entire program. Second, subjective feedback from mentors suggested that the program benefitted them as well. Some mentors reported that they increased their own knowledge and confidence in preparing for their apprentice. As Aristotle once said, Teaching is the highest form of understanding. Lastly, the program seemed to stimulate productive interactions between RTs at care centers across the country, fostering what Wenger and Snyder 7 have referred to as communities of practice. This we believe will be important in sustaining and further improving the CF care model. We are not aware of any similar mentoring programs for RTs in this or any other specialized care areas. The annual North American CF Conference has sessions tailored to RTs, including a day-long session focused on teaching airway clearance techniques. This session is popular among RTs who are new to CF care and is rated highly in postsession surveys. However, this session focuses on only one aspect of respiratory care. There are other networking sessions for RTs at the North American CF Conference for informal sharing of knowledge, as well as an active Listserv, on which CF RTs can post questions and comments. None of these opportunities have one-on-one interactions focused on improving knowledge and skills, like the mentoring program described in this study. While this preliminary evaluation of the effectiveness of the program is encouraging, several important limitations must be pointed out. The RT KAT was developed by highly experienced RTs and had face validity among their 768 RESPIRATORY CARE MAY 2013 VOL 58 NO 5

6 peers, but it is subjective in nature and not scientifically validated. Some key aspects of CF respiratory care may have been missed and/or others wrongly weighted in the scoring scheme. Another potential limitation is the risk of bias in the apprentice self assessments. However, the close agreement between the apprentice and mentor post site visit RT KATs lends credibility to the self assessments of the apprentices. It is also important to note that the mentoring process was not rigorously standardized. For example, there were no processes to ensure that all mentors administered and taught airway clearance techniques in the same manner. The variability in knowledge, skills, and teaching ability of the mentors could impact the fidelity of the knowledge transmitted to the apprentice, and the subsequent assessments of knowledge by apprentice and mentor. The program evaluation surveys also had limitations. They were subjective in nature, and the responses to the apprentice survey were collected from 1 month to 28 months post site visit. Because of the variable time elapsed between the mentoring session and completion of the survey, the results may be influenced by recall bias. 8 Nonetheless, the apprentice survey results (see Table 3) showed that many of them maintained contact with their mentors and felt comfortable contacting them with questions regarding the care of CF patients after completion of the program. Conclusions The CF Foundation Respiratory Therapy Mentoring Program is a unique opportunity for RTs relatively new to CF (apprentices) to receive specialized training in CF respiratory care from a highly experienced mentor. This preliminary assessment suggests that the program has been effective in meeting the short-term goal of improving apprentices RT specific knowledge. ACKNOWLEDGMENTS We thank Michael Mundorff MBA MHSA and Jeanette Seidelman RRT for their help in editing this paper; Karen Homa PhD for advising us on the analytical plan and conducting the analysis; Amanda Leonard MPH RD CDE and Teresa Schindler MSc RD, the facilitators of the dietitian s mentoring program (it is the insightfulness, enthusiasm, and passion of Amanda that was the inspiration for a mentoring program); and the facilitators CF Center directors, Barbara Chatfield MD and Patrick Flume MD, for their unending support of our endeavors in the pursuit of improved CF respiratory care. REFERENCES 1. Cystic Fibrosis Foundation. Patient Registry Report Registry-Report-2009.pdf. Accessed February 18, Flume PA, Robinson KA, O Sullivan BP, Finder JD, Vender RL, Willey-Courand DB, et al. Cystic fibrosis pulmonary guidelines: airway clearance therapies. Respir Care 2009;54(4): Quinton HB, O Connor GT. Current issues in quality improvement in cystic fibrosis. Clin Chest Med 2007;28(2): Flume PA, O Sullivan BP, Robinson KA, Goss CH, Mogayzel PJ, Willey-Courand DB, et al. Cystic fibrosis pulmonary guidelines: chronic medications for maintenance of lung health. Am J Respir Crit Care Med 2007;176(10): Saiman L, Siegel J; Cystic Fibrosis Foundation. Infection control recommendations for patients with cystic fibrosis: microbiology, important pathogens, and infection control practices to prevent patientto-patient transmission. Infect Control Hosp Epidemiol 2003; 24(Suppl 5):S6-S Flume PA, Mogayzel PJ, Robinson KA, Rosenblatt RL, Quittell L, Marshall BC; Clinical Practice Guidelines for Pulmonary Therapies Committee. Cystic fibrosis pulmonary guidelines: pulmonary complications hemoptysis and pneumothorax. Am J Respir Crit Care Med 2010;182(3): Wenger EC, Snyder WM. Communities of practice: the organizational frontier. Harvard Bus Rev 2000;78(1): Thorndyke LE, Gusic ME, Milner RL. Functional mentoring: a practical approach with multilevel outcomes. J Contin Educ Health Prof 2008;28(3): This article is approved for Continuing Respiratory Care Education credit. For information and to obtain your CRCE (free to AARC members) visit RESPIRATORY CARE MAY 2013 VOL 58 NO 5 769