715 An Evaluation of Immunization Education Resources by Family Medicine Residency Directors Mary Patricia Nowalk, PhD, RD; Richard K. Zimmerman, MD, MPH, Donald B. Middleton, MD; Roger A. Sherwood, CAE; Feng-Shou Ko, PhD; Sanford R. Kimmel, MD; Judith A. Troy, MS Background and Objectives: Immunization is a rapidly evolving field, and teachers of family medicine are responsible for ensuring that they and their students are knowledgeable about the latest vaccine recommendations. Methods: A survey was mailed to 456 family medicine residency directors across the United States to obtain their evaluation of immunization resources developed by the Society of Teachers of Family Medicine s Group on Immunization Education. Frequencies, measures of central tendency, and differences between responses from 2001 to 2005 were analyzed. Results: Directors of 261 (57%) family medicine residencies responded, with >80% reporting satisfaction with immunization teaching resources. The popularity of bound resources decreased from 2001 to 2005, while immunization Web sites increased in importance. The journal supplement, Vaccines Across the Lifespan, 2005 was less frequently read in 2005 than its predecessor published in 2001, but quality ratings remained high. Use of the Web site, www.immunizationed.org, and the Shots software for both desktop and handheld computers has increased since their creation. Conclusions: Electronic immunization teaching resources are increasingly popular among family medicine residencies. As the field continues to change, the use of electronic resources is expected to continue, since they are easily updated and, in the case of www.immunizationed.org and Shots software, are available free of charge. (Fam Med 2007;39(10):715-9.) Significant progress has been made toward achieving optimal levels of childhood immunizations. For example, in 2006, the vaccination rate for the 4:3:1:3:3:1 series (4 doses of DTaP, 3 doses polio vaccine, 1 dose of MMR, 3 doses of Hib, 3 doses of Hep B, and 1 dose of varicella vaccine) was 77% for children; 1 the Healthy People 2010 goal is 80%. 2 Similarly, rates of pneumococcal polysaccharide vaccination for adults ages 65 years and older have risen from 46% in 1998 to 63% in 2002. 2 Influenza vaccination for adults ages 65 years and older, however, has remained essentially unchanged (65.6% in 2004 versus 64% in 1998). 2,3 From the Department of Family Medicine and Clinical Epidemiology, University of Pittsburgh (Drs Nowalk, Zimmerman, Middleton, and Ko and Ms Troy), the Department of Behavioral and Community Health Sciences, University of Pittsburgh (Dr Zimmerman); the Society of Teachers of Family Medicine, Leawood, Kan (Mr Sherwood); and the Medical University of Ohio (Dr Kimmel). Physician factors are an integral part of efforts to maintain and improve vaccination rates among patients in all age groups. Previous research has shown that subsequent immunization practice patterns are determined in part by the principles and practices regarding vaccine-preventable diseases learned during postgraduate training. 4 As the number of recommended vaccines, the options for combination vaccines, and the groups for whom vaccines are recommended (eg, adolescents) continue to grow, so too will the need for immunization education resources. These resources should be current, easily accessible, and should use the formats most acceptable to practicing physicians. Since 2001, the Group on Immunization Education (GIE) of the Society of Teachers of Family Medicine (STFM) has been developing new educational materials to assist those who teach family medicine in their efforts to provide residents and medical students the most up-to-date immunization information available.
716 November-December 2007 Family Medicine The GIE has written three supplements to the Journal of Family Practice, published in 2001, 5 2003, 6 and 2005 7 and created an immunization education Web site that first appeared in early 2001 and is continually updated with the most-recent Centers for Disease Control and Prevention (CDC) childhood and adult immunization schedules. The resources also include PowerPoint slide presentations for use by educators, Advisory Committee on Immunization Practices (ACIP) reports, links to other immunization Web sites, and free Shots software for personal digital assistants (PDAs), which provides the CDC childhood, adult, and catch-up schedules, as well as detailed information about vaccine-preventable diseases, vaccines, risks, and contraindications. This study s purpose is to report on the periodic evaluation of these immunization education resources. Methods Subjects and Questionnaires In 2005, a survey was mailed to 456 family medicine residency program directors from across the country. Nonrespondents received two additional mailings, sent approximately 8 weeks apart. The mailing included an introductory letter from STFM and was mailed from its offices. The survey consisted of 62 questions that assessed (1) level of agreement with statements regarding experience with immunization education, (2) importance to teaching of, and satisfaction with, various vaccine information sources and teaching materials, (3) evaluation of the GIE s immunization resources, and (4) miscellaneous information. Demographic information was not collected. Questions took several forms. When asked whether they agreed with a statement, a 4-point Likert scale (strongly agree, agree, disagree, strongly disagree) was used. When asked about the importance of a resource, a 4-point Likert scale (extremely important, important, slightly important, not important/do not use) was used. Quality ratings of various resources were assessed using a 4-point scale (excellent, good, fair, poor). The questionnaire was double entered into an electronic database and analyzed using SAS 8.2 (SAS Institute, Research Triangle, NC). Frequencies and measures of central tendency are reported. Differences between responses from 2001 to 2005 were analyzed using chi-square test by SAS PROC FREQ. Alpha was set at P<.05 for all statistical tests. The 2005 survey was similar to one conducted with the same group of subjects (family medicine residency directors) in 2001. 8 Many questions were repeated so that responses could be compared between the 2 years. In addition, questions were asked about immunization resources developed since 2001. This survey was reviewed by the STFM Research Committee and approved by the STFM Board of Directors and the Institutional Review Board of the University of Pittsburgh. Results Of the 456 surveys mailed to residency directors, 261 were returned (57% response rate). Respondents represented family medicine residencies from a variety of settings and sizes. Directors reported spending an average of 41.3% (± 24.3%) of their time teaching. Satisfaction With Teaching Resources In 2005, less than half (46.3%) of the respondents reported that it is difficult to stay up to date with childhood immunizations, and this was significantly fewer than the percentage (54.7%) reporting this in 2001 (P<.036). There was no difference between responses in the 2 years regarding keeping up to date on adult immunizations (35.5% in 2001 and 36.8% in 2005). There were no differences between the two surveys on the directors reported level of satisfaction with immunization resources, with >80% agreement for both surveys. Over the 4-year interval, the percent of directors who rated printed and/or bound immunization resources as extremely important for teaching residents declined significantly (Table 1). These resources include the American Academy of Pediatrics (AAP) Red Book, ACIP recommendations in the Morbidity and Mortality Weekly Report (MMWR), ACIP/AAP/ American Academy of Family Physicians (AAFP) childhood immunization schedule, and printed or downloadable immunization schedules. At the same time, the importance of immunization Web sites as a teaching resource increased significantly. The increase in the rating of immunization software for handheld computers (PDAs) was not significant, but in 2005, the percentage who rated it as extremely important (58.3%) was second only to the ACIP/AAP/AAFP childhood immunization schedule (59.2%) in 2005. Supplement to the Journal of Family Practice, Vaccines Across the Lifespan, 2005 In 2005, only 59 (23%) of the responding residency directors reported having seen the journal supplement, Vaccines Across the Lifespan, 2005, a significant decrease from 2001 (34%; P=.004). Four of the articles were updates of those published in 2001. Two of the updated articles were more frequently rated very useful for teaching residents in 2005 than in 2001, while ratings for two articles were not significantly different. There was no change in the ratings of the quality of the journal supplement, with approximately one third of the respondents indicating that the length, depth, and practicality, as well as the color pictures and ease of use, were excellent. Approximately one quarter of the respondents (27%) believed that they would refer to the supplement in the future, whereas 72% believed that it should be published annually. Seventy-six percent of respondents would prefer to receive the supplement online as opposed to the print copy.
717 Table 1 Evaluation of Available Immunization Education Resources by Family Medicine Residency Directors* Sources of Immunization Information Responded Extremely Important 2001 Responded Not Important/ Do Not Use AAP Red Book 104 (36.9) 68 (26.2) 8 (2.8) 16 (6.2).024 ACIP/CDC recommendations in MMWR 113 (40.2) 83 (32.3) 7 (2.5) 17 (6.6).026 US Preventive Services Task Force Recommendations 92 (32.9) 77 (29.8) 3 (1.1) 7 (2.7).436 AAFP Periodic Health Exam Charts 83 (29.6) 59 (22.9) 34 (12.2) 31 (12.0).277 ACIP/AAP/AAFP Childhood Immunization Schedule 199 (69.8) 154 (59.2) 2 (0.7) 8 (3.1).024 Immunization software NA 51 (20.1) NA 111 (43.7) - Preprinted or downloadable immunization schedules 147 (52.9) 114 (42.2) 6 (2.1) 3 (1.1).034 Case-based scenarios 45 (16.5) 36 (14.0) 35 (12.8) 32 (12.4).722 Pocket card immunization schedules 127 (45.2) 95 (36.8) 24 (8.6) 24 (10.9).257 Pictures of vaccine-preventable diseases NA 30 (11.6) NA 36 (13.9) - Audio vaccine risk/safety communication materials NA 7 (2.7) NA 96 (37.2) - Printed vaccine risk/safety communication materials NA 30 (11.9) NA 20 (7.9) - MP3 audio file applications NA 3 (1.2) NA 144 (56.5) - Immunization Web site 60 (22.1) 84 (32.7) 20 (7.4) 13 (5.1).001 Computer slide presentations 60 (22.4) 54 (26.8) 36 (13.4) 23 (8.8).324 Immunization schedule for PDAs 139 (52.3) 151 (58.3) 14 (5.3) 5 (1.9).070 * 2001 n=290, 2005 n=261 NA=not asked in 2001 AAP American Academy of Pediatrics ACIP Advisory Committee on Immunization Practices CDC Centers for Disease Control and Prevention MMWR Morbidity and Mortality Weekly Report AAFP American Academy of Family Physicians PDA personal digital assistant 2005 2001 2005 P Value STFM Group on Immunization Education s Web Site, www.immunizationed.org At the time of the 2005 survey, the Web site, www. ImmunizationEd.org, had been operational for 4 years. Use of the Web site increased significantly from 2001 to 2005. In 2005, 93 residency directors had used the Web site, of whom one third rated its overall value excellent, and more than half (56.5%) rated the value of Shots 2005 On-Line (which can be viewed and downloaded from the Web site) as excellent. Not unexpectedly, the most-frequently used section of Shots was reported to be the childhood schedule (42%), followed by the catch-up schedule (38%) and the adult schedule (27%). Shots Software for PDA Applications Shots software has been available for PDAs (Palm OS) since 2000. In 2002, the GIE added color support and supplemented the content. Subsequently, the program was developed to allow.net for Windows users to also download the program, and adult, highrisk, and catch-up schedules were added. There were approximately 58,000 downloads of Shots to PDAs in 2005 (Troy, personal communication, 2006). The number of residency directors who have downloaded Shots since its availability on the GIE Web site has risen each year from 11% in 2002 to 28% in 2005. Of the 116 residency directors who reported using Shots, 82 (71%) reported regular (monthly, weekly, or daily) use. Shots software was rated excellent by 41% or more of respondents on five measures (Table 2). The majority of residency directors (64%) reported that their residents and medical students are encouraged or required to use PDAs, and 95% believe that the use of PDAs will continue to grow and expand over the next 5 years. While 82% of programs recommend or require a pharmacy database, 60% recommend or require Medical Calculator or clinical software for PDAs. In 2005, 36% of residency directors reported that they recommend or require Shots software to their students. Discussion Residency directors reported less difficulty keeping up to date with childhood immunizations in 2005 than
718 November-December 2007 Family Medicine Table 2 Evaluation of Shots Software 2005 Factor Rated Excellent Rated Poor Ease of downloading 62 (58.5) 1 (0.9) Ease of use 71 (67.6) 0 (0.0) Value of pictures of vaccine-preventable diseases (Pocket PC and on-line 11 (19.3) 2 (3.5) versions only) Timeliness of schedules 61 (58.1) 0 (0.0) Value of risk/safety communication 41 (40.6) 1 (1.0) Overall value of Shots 2005 73 (68.9) 0 (0.0) in 2001, in spite of the fact that there have been several changes to the recommended childhood schedule since 2001. For instance, influenza vaccine for 6 to 23- month-old children was added to the schedule in 2004 following 2 years of encouragement to vaccinate, and in 2006 vaccine recommendations were changed again to include 6 to 59-month-old children. Recommendations regarding Tdap, MCV4, and hepatitis A vaccine also changed. Presumably, their ability to keep up to date is due to the availability of appropriate immunization resources, such as the resources produced by the STFM GIE and others. With the recent licensure of human papillomavirus, rotavirus, and zoster vaccines, changes continue to occur. Printed and/or bound resources seem to be declining in popularity as the importance of electronic resources is increasing. Although print resources are less important, the journal supplement geared toward the needs of family physicians is still given high ratings on quality factors such as length and depth of articles. This resource has been and will continue to be made available online, given the growing number of respondents who would prefer to receive it in that manner. Probably the most important resource developed and expanded by the GIE is the vaccination software Shots. This vaccination schedule and enhanced information resource is available for desktop use and may be used in conjunction with various electronic health record programs, as well as for use with PDAs. Use of PDAs by health care providers, especially younger physicians and residents, is increasing, 9 with an adoption rate for personal use of 45% 85%. We found that nearly two thirds of residencies recommend or require use of PDAs by family medicine residents. Strengths and Limitations Although the 2005 survey was based on the 2001 survey, it was slightly different in that questions were added to evaluate new teaching resources. These differences would not affect the results because only questions that were exactly the same were compared across the two time periods. How many of the individuals responded to both surveys and how those who did not respond differed is unknown. Therefore, it is also unknown whether changes in responses represent differences in perceptions of the same individuals or perceptions of different residency directors. Further, these results reflect the opinions of residency directors and are not necessarily commensurate with those of practicing physicians. Conclusions Both immunization schedules and information technology are advancing at a rapid pace. Early in 2007, an adolescent schedule of three vaccines is expected to be added, and new combination vaccines are continually being developed, as are new vaccines against other diseases. The handheld computer industry continues to develop new products, and PDA software programs and their use are burgeoning. Those who teach family medicine residents should have resources available to them that address their needs for up to date, easily accessible, accurate information, using accepted electronic devices. The real advantage of electronic resources is the ability to update information rapidly and with little effort on the part of the user. Immunization resources that address the needs for clear, concise, and current information for rapid everyday access, as well as in-depth information on more-complex issues such as catch-up schedules, contraindications, and high-risk patients are valuable to those who teach and those being trained. Acknowledgments: This project was approved by the Institutional Review Board of the University of Pittsburgh and by the Board of Directors of the Society of Teachers of Family Medicine (STFM). This project was supported by the Centers for Disease Control and Prevention (CDC), National Immunization Program, through Cooperative Agreement Grant #U66/CCU723241 to the STFM Foundation. Its contents are the responsibility of the authors and do not necessarily reflect the official views of CDC.
719 This project was presented as a Research Forum at the 2007 STFM Annual Spring Conference in Chicago. Corresponding Author: Address correspondence to Dr Nowalk, University of Pittsburgh, Department of Family Medicine and Clinical Epidemiology, 3518 Fifth Avenue, Pittsburgh, PA 15261. 412-383-2355. Fax: 412-383-2306. tnowalk@pitt.edu. Web site: www.pitt.edu/~familymd/immunization. REFERENCES 1. Centers for Disease Control and Prevention. National, state, and local area vaccination coverage among children aged 19 35 months United States, 2006. http://www.cdc.gov/mmwr/preview/mmwrhtml/ mm5634a2.htm. Accessed October 5, 2007. 2. Increase the proportion of adults who are vaccinated annually against influenza and ever vaccinated against pneumococcal disease. www. healthypeople.gov/document/html/objectives/14-29.htm. 2006. 3. Centers for Disease Control and Prevention. Influenza vaccination levels among persons aged 65 years and among persons aged 18 64 years with high-risk conditions United States, 2003. MMWR 2005;54: 1045-9. 4. Zimmerman RK, Janosky JE. Immunization barriers in Minnesota private practices: the influence of economics and training on vaccine timing. Fam Pract Res J 1993;13:213-24. 5. Zimmerman RK, Kimmel SR, Middleton DB. Vaccines across the life span, 2001. J Fam Pract 2001;50:S1-S60. 6. Zimmerman RK, Kimmel SR, Middleton DB. Vaccines across the life span. J Fam Pract 2003;52:S1-S72. 7. Zimmerman RK, Middleton DB, Kimmel SR, eds. Vaccines across the life span 2005. J Fam Pract 2005;54:S9-S26. 8. Nowalk MP, Zimmerman RK, Kimmel SR, et al. Immunization education among family practice residency programs. Fam Med 2003;35: 711-6. 9. Garritty C, Emam KE. Who s using PDAs? Estimates of PDA use by health care providers: a systematic review of surveys. Journal of Medical Internet Research 2006;8:e7.