Immunization Education Among Family Practice Residency Programs

Transcription

1 Residency Education Vol. 35, No Immunization Education Among Family Practice Residency Programs Mary Patricia Nowalk, PhD, RD; Richard K. Zimmerman, MD, MPH; Sanford R. Kimmel, MD; Douglas J. Bower, MD; Donald B. Middleton, MD; Roger A. Sherwood, CAE; Judith A. Troy, MS Background and Objectives: The dynamic nature of immunization schedules, shortages, and administration techniques makes keeping up to date with current national recommendations difficult and necessitates periodic evaluation of immunization teaching resources. Methods: This study surveyed family practice residency program directors in 1998 to assess their satisfaction with immunization teaching resources and interest in new resources. Subsequently, with funding from the Centers for Disease Control and Prevention, the Society of Teachers of Family Medicine Group on Immunization Education developed a series of educational materials devoted to educating family physicians about immunizations. In 2001, residency directors were surveyed again to evaluate the educational resources. Results: Most program directors reported satisfaction with resources currently available to them for teaching residents about childhood immunizations, but about half (41% in 1998 versus 55% in 2001) agreed that keeping up to date on childhood immunizations was difficult. The corresponding figures for adult immunizations were 27% in 1998 and 36% in Pocket-size immunization schedules were ranked highly important (53% in 1998 versus 45% in 2001). Many would use handheld computer immunization schedules (53%). Although satisfaction with available resources did not increase following introduction of a newly developed journal supplement and Web site, the limited response received was favorable. Conclusions: Immunization teaching materials developed by family physicians, especially those that make use of evolving technologies, can be useful resources for individuals teaching family practice residents and for keeping up to date on recommendations for immunizations. (Fam Med 2003;35(10):711-6.) Despite clear evidence of the effectiveness of immunizations in preventing disease, disability, and death, the use of immunizations for disease prevention is less than optimal, and certain vaccine-preventable diseases continue to devastate vulnerable populations. For example, influenza and pneumonia, together, are the fifth leading cause of death among the elderly in the United States, yet immunization rates for influenza (66%) and pneumococcal polysaccharide vaccine (55%) 1 among the elderly remain far from national goals of 90%. 2 From the Department of Family Medicine and Clinical Epidemiology, University of Pittsburgh (Drs Nowalk, Zimmerman, and Middleton and Ms Troy), the Department of Family Medicine, Medical College of Ohio (Dr Kimmel), the Department of Family and Community Medicine, Medical College of Wisconsin (Dr Bower), and the Society of Teachers of Family Medicine, Leawood, Kan (Mr Sherwood). Similarly, national immunization rates for childhood vaccines have reached a plateau at approximately 75% for the 4:3:1:3 series (four doses of DTaP, three doses of polio vaccine, one dose of MMR, and three doses of Hib) by age 2 years. 3 Failure to reach immunization goals may be partly attributed to changing immunization recommendations due to availability of new vaccines such as pneumococcal conjugate vaccine, development of new vaccine combinations, addition of vaccine boosters, broadening of target populations for certain vaccines, and increasingly commonplace temporary shortages of vaccines. Previous research has shown that physicians immunization practice patterns are determined in part by principles and practices learned during residency training. 4 This dependence on knowledge acquired in residency, coupled with continual change in immunization recommendations and restricted time for educational updates, increases the likelihood that physicians will

2 712 November-December 2003 Family Medicine not easily stay up to date on immunization practices. Moreover, gaps in immunization knowledge and practice between family physicians and pediatricians have been documented In response, the Group on Immunization Education (GIE) of the Society of Teachers of Family Medicine (STFM) conducted a needs assessment of family practice residency program directors in 1998, developed new educational materials to educate physicians about immunizations, and surveyed program directors again in 2001 to determine if needs had been met. This study reports on the two surveys and the evaluation of the new educational resources. Methods Overview In 1998, the GIE conducted a needs assessment of family practice residency program directors. Subsequently, a subgroup of the GIE, through the STFM Foundation, received a grant from the Centers for Disease Control and Prevention (CDC) to enhance immunization education for family physicians. The multifaceted proposal included development of a monograph that became the 2001 immunization supplement to the Journal of Family Practice 14 (circulation of >70,000), articles for the American Family Physician, 15,16 presentations at national family medicine conferences, creation of a new Web site (www. immunizationed.org), and development of electronic slide sets/presentations to be placed on the Web site for download and use by educators. In 2001, following the publication of the journal supplement, a new survey of the STFM family practice residency program directors was undertaken that involved a needs assessment and evaluation of the newly developed educational resources. Subjects and Questionnaires 1998 Needs Assessment. During the summer of 1998, a questionnaire developed by the GIE, reviewed and approved by the STFM Board of Directors and the Institutional Review Board of the Medical College of Ohio, was mailed to 467 family practice residency program directors in the United States. Completed surveys were returned to STFM. Nonrespondents were mailed a second copy of the survey after 8 weeks. The survey consisted of 50 questions that assessed (1) level of agreement with statements regarding experience with immunization education, (2) relative importance of methods of immunization education, (3) importance to teaching and satisfaction with various immunization information sources, (4) importance of immunization information typically used in residency family practice centers, and (5) likelihood of using specific educational resources, if available. Demographic information was not collected. Questions took several forms. When asked whether they agreed with a statement, a 4-point Likert scale (strongly agree, agree, disagree, strongly disagree) was used. When asked about the importance of an educational resource, a 3-point Likert scale (extremely important, important, slightly important) was used, with the additional option of Do not use. Satisfaction was assessed using a 4-point Likert scale (very satisfied, satisfied, not satisfied, very unsatisfied). The questionnaire was electronically scanned and tabulated Needs Assessment and Evaluation. In 2001, a survey was mailed to 466 family practice residency program directors in three waves, approximately 8 weeks apart. The needs assessment was similar to the 1998 survey in that it asked about satisfaction with current immunization information resources, types of teaching methods, and importance of specific educational resources, using the same question format as in This survey was reviewed by the STFM Research Committee and approved by the STFM Board of Directors and the Institutional Review Board of the University of Pittsburgh. In addition to the needs assessment, the 2001 survey evaluated two of the new educational resources: (1) a supplement to the Journal of Family Practice 14 dedicated to vaccines and immunization practices and issues and (2) a Web site develope d by the GIE ( The evaluation specifically asked respondents about the usefulness of each topic in the journal supplement using a 3-point Likert scale. Further, respondents were asked to rate the quality of various aspects of the supplement as a whole and the Web site using a 4-point Likert scale. Data Analysis The entire 2001 survey was manually entered into an electronic database and analyzed using SPSS 10.1 for Windows (SPSS, Inc, Chicago). Frequencies and measures of central tendency were calculated. Differences between responses from 1998 to 2001 were analyzed using the z statistic. Alpha was set at P<.05 for all statistical tests. Results Respondents A total of 278 family practice residency program directors returned the 1998 questionnaire, for a 60% response rate (278/467). The response rate in 2001 was 62% (290/466). Descriptive data were not collected in 1998; however, in 2001, basic information describing the size and location of the residency program was collected. Respondents represented the diversity of US residency programs (2 to 54 resident positions per program, mean=22±8; faculty-to-resident ratio ranged from 1:6 3:1, mode=1:4; number of hospital beds ranged from 30 1,979, mean=448±276), and respondents were from all geographic regions and working in a variety of settings.

3 Residency Education Vol. 35, No Family practice residency directors were first asked whether they had difficulty staying up to date on childhood and adult immunization practices. For childhood immunizations, in 1998, 41% of respondents agreed, and in 2001, 55% agreed (P<.001, for difference between years). For adult immunizations, the corresponding figures were 27% in 1998 and 36% in 2001 (P=.011). Most felt confident in their ability to explain recommendations for polio vaccine (81%) as an example of a childhood immunization and pneumococcal vaccine (94%) as an example of an adult immunization to their residents and patients. The latter questions were only asked in Educational Methods Table 1 shows the methods ranked most important for educating residents about both childhood and adult immunizations. Of those who reported using these listed techniques, family practice preceptor teaching was rated extremely important most frequently for adult immunizations, and immunization schedules were most frequently rated extremely important for childhood immunizations. These ratings did not change significantly from 1998 to Journal clubs were least often ranked extremely important for teaching about both types of immunizations. Satisfaction with resources available to them for teaching residents about childhood vaccinations was high, with 84% satisfied in 1998 and 87% satisfied in 2001 (P=.227). For adult vaccination teaching resources, 80% were satisfied in 1998 and 82% in 2001 (P=.230). Sources of Information When asked to rate the importance of various published immunization information sources used in the residents family practice center, the three most important sources cited were the Advisory Committee on Immunization Practices/American Academy of Pediatrics/ American Academy of Family Physicians (ACIP/AAP/ AAFP) Childhood Immunization Schedule, the ACIP/ Centers for Disease Control (CDC) recommendations published in the Morbidity and Mortality Weekly Report (MMWR), and the AAP Red Book (Table 2). For adults, the three most important sources cited were the ACIP/ CDC recommendations published in MMWR, the US Preventive Services Task Force, and the AAFP Periodic Health Examination Charts. The Standards for Pediatric Immunization Practices, Standards for Adult Immunization Practices, The American College of Physicians/Infectious Disease Society of America (ACP/ IDSA) Green Book, and Needle Tips, published by the Immunization Action Coalition, were infrequently used in residencies family practice centers in 1998 and were not included in the 2001 survey. Ratings of the AAP Red Book and recommendations in the MMWR declined significantly, while rating of the joint childhood immunization schedule increased significantly from 1998 to In 1998, two thirds of program directors responded that they would be very likely to use an abbreviated pediatric/adult immunization guidebook if STFM or the AAFP were to develop one. When asked to rank the three most important immunization topics for future articles, the four topics ranked most important by the largest percentage of respondents were recent vaccination schedule changes (41%), vaccine safety and adverse reactions (16%), methods to increase vaccination rates (14%), and learning and teaching resources on immunization (13%). In both surveys, residency program directors were asked to rate the importance of the audio/visual materials available or potentially available to assist them in teaching residents about immunizations (Table 3). In 1998, those materials believed to be most beneficial were pocket cards and preprinted handouts of immunization schedules. These distillations of immunization information were rated extremely important by 53% and 51% of respondents, respectively, compared with lecture notes, slides, case presentations, and audiotapes Table 1 Importance of Currently Available Immunization Teaching Methods, 1998 and 2001 Rated Extremely Important for Rated Extremely Important for Childhood Immunizations Adult Immunizations Immunization Teaching P Value** P Value** Method* (n=278) (n=290) (n=278) (n=290) Posted immunization schedules 70% 68% % 50%.390 Preceptor 64% 61% % 56%.302 Lectures 31% 32% % 25%.444 Pediatric/adult rotation 20% 27% % Case presentations 19% 10% % 13%.066 Journal club 4% 3%.113 4% 2%.075 * Respondents were asked, Based on your current curricula, please rate the importance of each method in educating residents about childhood (adult) immunizations. **P value for differences between 1998 and 2001 not asked in this survey

4 714 November-December 2003 Family Medicine Table 2 Sources of Information on Immunization, 1998 (n=278) and 2001 (n=290) Rated Extremely Important Do Not Use Source* P Value** Childhood immunizations AAP Red Book 46% 37% 7% 3%.011 ACIP/AAP/AAFP childhood immunization schedule 59% 70% <1% <1%.008 Standards for Pediatric Immunization Practices 10% 33% Childhood/adult immunizations ACIP-MMWR 46% 40% 3% 3%.040 US Preventive Services Task Force 35% 33% 5% 1%.258 AAFP Periodic Health Exam Charts 23% 30% 12% 12%.066 Needle Tips 8% 35% Adult immunizations Standards for Adult Immunization Practices 10% 32% ACP/IDSA Green Book 3% 59% * Respondents were asked, Please rate the importance of the following information sources used in your residents family practice clinical center. **P values for differences in percent of respondents rating each source extremely important from 1998 to not asked AAP American Academy of Pediatrics ACIP Advisory Committee on Immunization Practices AAFP American Academy of Family Physicians MMWR Morbidity and Mortality Weekly Report IDSA Infectious Disease Society of America (2% 24% rated extremely important ). In 1998, program directors were also asked how often they access the CDC/MMWR Web site to inquire about immunization. The majority (53%) responded that they do not use the Web site, only 30% use it one to two times a year, and the remaining 17% use it three or more times a year. However, when asked how frequently they might access a Web site developed by the STFM GIE, 75% responded that they would likely use it at least once a year. Evaluation of Journal Supplement and Web Site Based on the findings of the 1998 needs assessment, the GIE developed several new resources to facilitate providing up-to-date immunization information to faculty while teaching family practice residents. The GIE produced a journal supplement dedicated to immunization with articles on the topics identified in the needs assessment that was issued several months prior to the 2001 survey. Evaluations of the supplement are shown in Table 4. Overall, the supplement was well received by those familiar with it, though only 34% of respondents had read it. In May 2001, the GIE Web site, www. immunizationed.org, went on-line and was submitted to several search engines shortly thereafter. By the end of December 2001, 16,315 individual sessions had been registered on the Web site. However, only 29 respondents reported having seen the Web site. Of those, the majority rated its value favorably (Table 5). Since the 2001 survey, a special feature was added to the Web site, an electronic childhood immunization schedule titled Shots 2002, which can also be downloaded onto handheld computers. (In early 2003, the childhood immunization schedule was updated, and an adult schedule was added [ Shots 2003 ]). There have been more than 30,000 downloads of Shots 2002, and the program has been recommended or required by several medical and nursing education programs. Discussion Both before and after GIE s development of new immunization teaching resources, satisfaction among family practice residency program directors was high. However, the difficulty in keeping up to date on both childhood and adult immunizations increased between 1998 and One interpretation of these data is that the volume of material to be addressed during residency precludes devoting the same time and attention to immunization rationale and practical administration as are given to immunization schedules and indications. Another interpretation is that even informative resources are not sufficiently responsive to changes in recommendations due to developing immunization technologies such as combination vaccines, production shortages, and new scientific evidence. Further, of the currently available resources, preceptor teaching and pocket-size, standard-size, and posted immunization schedules were most frequently rated as extremely important. This importance is probably due to the need for information on indications and schedules at the point of care. Therefore, keeping up-to-date immunization schedules easily available to residents in clinics where they work and for those who teach them is essential. Electronic publication of immunization schedules is one method to accomplish this task. Electronic (handheld and desktop) versions of immunization

5 Residency Education Vol. 35, No Table 3 Importance of Teaching Resources If Available, 1998 (n=278) and 2001 (n=290) Rated Extremely Would Not/ Important Do Not Use Material/Resource* P Value** Pocket-sized immunization schedules 53% 45%.031 5% 9% Preprinted immunization schedule handouts 51% 53%.337 3% 2% Prepared overheads and slides 24% 16%.012 7% 8% Case-based scenarios 19% 17%.206 6% 13% Preprinted lecture outlines 18% 12%.024 9% 12% Prepared audiotapes 2% 2% % 51% Handheld computer version of immunization schedule 53% 5% Web site 22% 7% Electronic slide presentation 22% 13% Previous research has shown that family physicians lack a consensus on the best or most frequently used source of immunization information. 8,10 We found that ACIP/ AAP/AAFP immunization schedules, the MMWR, and the AAP Red Book were the most important immunization information resources but that the majority of respondents would be very likely to use an STFM immunization guidebook. Hence, a journal supplement with articles addressing three of the top four cited areas of interest was published. Response to the 2001 journal supplement was favorable, and an updated 2003 version has been published. 17 * Respondents were asked, Please rate the importance of those materials that you would use in educating residents about immunizations, if they were available to you. **P values for differences in percent of respondents rating each resource extremely important from 1998 to 2001 not asked Table 4 Evaluation of Journal Supplement on Immunizations (Respondents=98) Very Not Did Not Article Useful Useful Useful Read Vaccine Schedules and Procedures 39% 49% 2% 10% Routine Vaccines Across the Life Span 33% 52% 4% 11% Vaccines for Persons at High Risk 28% 60% 1% 11% Overcoming Barriers to Immunization 25% 56% 6% 13% Specific aspect of overall supplement Excellent Good Fair Poor Ease of use 32% 58% 7% 4% Color pictures 33% 57% 9% 1% Practicality 30% 60% 8% 2% Depth of information 31% 67% 2% 0% Length 20% 66% 12% 2% schedules and frequently updated Web sites allow for rapid access to continually changing immunization information. These technologies are likely to appeal to recent medical graduates whose education was integrated with evolving computer technology. Given that 75% of 1998 survey respondents reported that they would very likely use an STFM-developed Web site at least once per year, GIE produced an immunization Web site and an electronic immunization schedule that can be downloaded onto handheld devices. The initial response to the Web site has been extremely favorable. Strengths and Limitations Although the 2001 survey was based on the 1998 survey, it was slightly different in that questions with low responses in 1998 were eliminated from the 2001 version, and others were added to evaluate new teaching resources. These differences would not affect the results because only questions that were exactly the same were compared across the two time periods. How many of the individuals responde d to both sur ve ys a nd how nonrespondents differed from respondents is unknown. It is also unknown whether changes in responses represent differences in perceptions of the same individuals or of different ones. However, the respondents in 2001 represented a wide array of program types and settings, and there is no reason to believe that this was not the case in Thus, we believe that the findings are generally applicable. Although most questions specifically referred to teaching residents, there is some uncertainty over whether the respondents answered questions in relation to their own professional needs versus their needs as a teacher, leading to possible confusion by respondents. Finally, low numbers of respondents who had seen the 2001 journal supplement and Web site may have been due to the proximity with which the survey followed their release. We chose the timing of the survey to closely follow publication of the journal supplement, so as to get responses while the publication was more likely to be fresh in respondents minds. On the other hand, following the announcement of the availability of Shots 2002 for handheld computers, individual ses-

6 716 November-December 2003 Family Medicine Table 5 Evaluation of the STFM Group on Immunization Education Web Site, (Respondents=29) Aspect of Web site Excellent Good Fair Ease of locating site on server 31% 66% 3% Ease of navigation through site 24% 66% 10% Value of links 27% 69% 4% Value of slides 15% 73% 12% Value of ACIP reports 27% 62% 12% Value of pictures of disease 16% 68% 16% Overall value of Web site 22% 74% 4% ACIP Advisory Committee on Immunization Practice sions on the Web site have averaged 9,000 12,000 per month. This trade-off between getting immediate responses and allowing more exposure time to the new resources may have contributed to the failure to observe significant differences in satisfaction between the first and second survey. Conclusions Although more program directors in 2001 than in 1998 agreed that keeping abreast of changes in immunization recommendations was difficult, satisfaction with available resources stayed the same. Immunization practice will continue to change as new vaccines and expanded recommendations for older vaccines are developed. Those who teach family practice residents should have resources available to them that address their needs for up-to-date, easily accessible, accurate information in the three primary areas of immunization education: (1) schedules and indications available through Shots 2003 and other annual childhood and adult immunization schedules, (2) rationale, addressed in the GIE s journal supplement as well as by AAP and CDC resources, and (3) practical procedures, addressed in the journal supplement, by the CDC and by the Immunization Action Coalition (Needle Tips). The STFM GIE is using evolving technologies such as Shots 2003 to fulfill the needs of those who are teaching and learning immunization practices. Acknowledgments: This project was approved by the Institutional Review Boards of the University of Pittsburgh and the Medical College of Ohio and by the Board of Directors of the Society of Teachers of Family Medicine. This project was supported by the Centers for Disease Control and Prevention (CDC), National Immunization Program, through Cooperative Agreement U66/CCU to the Society of Teachers of Family Medicine Foundation. This paper s contents are the responsibility of the authors and do not necessarily reflect the official views of the CDC. Corresponding Author: Address correspondence to Dr Nowalk, University of Pittsburgh, School of Medicine, Department of Family Medicine and Clinical Epidemiology, 3518 Fifth Avenue, Pittsburgh, PA Fax: tnowalk@pitt.edu. immunization. REFERENCES 1. Centers for Disease Control and Prevention. Early release of selected estimates based on data from the first quarter of 2002 NHIS. gov/nchs/about/major/nhis/released htm. 2. US Department of Health and Human Services. Healthy People 2010 conference edition. Washington, DC: US Department of Health and Human Services, Centers for Disease Control and Prevention. National, state, and urban area vaccination coverage levels among children aged months United States, MMWR Morb Mortal Wkly Rep 2001; 50: Zimmerman RK, Janosky JE. Immunization barriers in Minnesota private practices: the influence of economics and training on vaccine timing. Fam Pract Res J 1993;13(3): Loewenson PR, White KE, Osterholm MT, MacDonald KL. Physician attitudes and practices regarding universal infant vaccination against hepatitis B infection in Minnesota: implications for public health policy. Pediatr Infect Dis J 1994;13: Szilagyi PG, Rodewald LE, Humiston SG, et al. Immunization practices of pediatricians and family physicians in the United States. Pediatrics 1994;94: Zimmerman RK, Mieczkowski TA. How important is early childhood hepatitis B vaccination? A survey of primary care physicians. J Fam Pract 1998;47: Zimmerman RK, Schlesselman JJ, Baird AL, Mieczkowski TA. A national survey to understand why physicians limit childhood immunizations. Arch Pediatr Adolesc Med 1997;151: LeBaron C, Massoudi M, Stevenson J, Lyons B. Vaccination coverage and physician distribution in the United States, Pediatrics 2001; 107:e Zimmerman RK, Mieczkowski TA, Mainzer HM, Medsger AR, Nowalk MP. Understanding physician agreement with varicella immunization guidelines. Prev Med 2002;35: Kimmel SR, Puczynski S, McCoy RC, Puczynski MS. Practices of family physicians and pediatricians in administering poliovirus vaccine. J Fam Pract 1999;48(8): Freed GL, Freeman VA, Clark SJ, Konrad TR, Pathman DE. Pediatrician and family physician agreement with adoption of universal hepatitis B immunization. J Fam Pract 1996;42: Freed GL, Bordley WC, Clark SJ, Konrad TR. Universal hepatitis B immunization of infants: reactions of pediatricians and family physicians over time. Pediatrics 1994;93: Zimmerman RK, Kimmel SR, Middleton DB. Vaccines across the life span, J Fam Pract 2001;50:S1-S Zimmerman RK. Pneumococcal conjugate vaccine for young children. Am Fam Physician 2001;63: Kimmel SR. Vaccine adverse events: separating myth from reality. Am Fam Physician 2002;66(11): Zimmerman RK, Kimmel SR, Middleton DB. Vaccines across the life span. J Fam Pract 2003;52:1S-72S.