EDUCATION. Journal of Food Science

Transcription

1 Food Science Education Research... Journal of Food Science EDUCATION Development and Implementation of a Food Safety Knowledge Instrument Carol Byrd-Bredbenner, Virginia Wheatley, Donald Schaffner, Christine Bruhn, Lydia Blalock, and Jaclyn Maurer ABSTRACT: Little is known about the food safety knowledge of young adults. In addition, few knowledge questionnaires and no comprehensive, criterion-referenced measure that assesses the full range of food safety knowledge could be identified. Without appropriate, valid, and reliable measures and baseline data, it is difficult to develop and implement effective education efforts. Thus, the purpose of this study was to develop a comprehensive, valid, reliable food safety knowledge questionnaire. Questionnaire development followed this process: 1) use of published reports and input from experts in food safety and sanitation (n = 7) to identify key food safety concepts; 2) development of a question bank (n = 101) assessing knowledge of key concepts (i.e., cross contamination prevention/disinfection procedures; safe times/temperatures for cooking/storing foods; groups at greatest risk for foodborne disease; foods that increase risk of foodborne disease; and foodborne disease pathogens); 3) refinement of initial questions by experts; 4) questionnaire pretest with young adults (n = 180) and refinement; 5) questionnaire pilot test (n = 126) and refinement; 6) final expert review and refinement; and 7) conversion into an online survey. Young adults (n = 4343, mean age 19.9 ± 1.7SD years) from 21 universities and colleges across the country completed the questionnaire. Item analysis was used to determine the overall quality of the test and identify improvements needed. Livingston s coefficient of reliability for criterion-referenced tests was The questionnaire met or exceeded generally recognized standards of reliability and validity. This questionnaire could be useful in baseline assessment of food safety knowledge and measurement of knowledge gained after an educational intervention in adults. Introduction Foodborne disease caused by microbial pathogens remains a significant public health problem in the 21st century, so much so that it is one of the priorities in the Healthy People 2010 initiative (U.S. Dept. of Health and Human Services 2000). Data from several surveys, including the 1998 FDA/FSIS Consumer Food Safety Survey (Food Safety and Inspection Service and U.S. Dept. of Agriculture 1998c) as well as the Behavioral Risk Factor Surveillance System (BRFSS) (Food Safety and Inspection Service and U.S. Dept. of Agriculture 1998a; Yang and others 1998a, 1998b; Altekruse and others 1999), indicate that food mishandling is more acute in some consumer groups than in others. Specifically, young adults, men, and individuals with education beyond high school (Williamson and others 1992; Fein and others 1995; Klontz and others 1995; Food Safety and Inspection Service and U.S. Dept. of Agriculture 1998b; Altekruse and others 1999) are more likely to engage in risky food handling than others. Although it is not known why these groups tend to have risky food handling behaviors, researchers hypothesize it may be due to lack of food safety education, role models, and/or food-handling experience (Altekruse and others 1996). In fact, recent generations have had limited opportunities to learn about safe food handling for at least 2 reasons (Kastner 1995; U.S. Dept. of Agriculture and Food and Drug Administration 1998): changes in the educational system that once taught food safety in home economics classes in every secondary school have resulted in a reduction or even elimination of such courses over the past decade (Beard 1991; U.S. Dept. of Agriculture and Food and Drug Administration 1998); and increasing numbers of working mothers and growing reliance on convenience, take out, and restaurant foods have decreased opportunities for children to learn safe food handling via observation. As a result of these changes, a large proportion of young adults may lack the critical knowledge needed to proactively protect themselves and their families (Beard 1991; U.S. Dept. of Agriculture and Food and Drug Administration 1991; Williamson and others 1992; Knabel 1995; Partnership for Food Safety Education 1998; Altekruse and others 1999; Anonymous 2002; U.S. Dept. of Agriculture and Food Safety Inspection Service 2002). Food safety education efforts designed to solve this problem could be targeted to young adults; however, little is known about this population s food safety knowledge. In addition, few knowledge questionnaires and no comprehensive, criterion-referenced measure that assessed adults full range of food safety knowledge could be identified (Medeiros and others 2001a). Without appropriate, valid, and reliable measures and baseline data, it is difficult to develop and implement effective education efforts (Gronlund 1998; Contento and others 2002). The purpose of this study was to develop a comprehensive, valid, reliable food safety knowledge questionnaire. MS Submitted 9/8/2006, Accepted 2/22/2007. Author Byrd-Bredbenner, Wheatley, and Maurer are with Rutgers Univ., 26 Nichol Ave., 220 Davison, New Brunswick, NJ Author Schaffner is with Rutgers Univ., 65 Dudley Rd., New Brunswick, NJ Author Bruhn is with Univ. of California, Davis, CA Author Blalock is with Rutgers Univ., 71 Lipman Drive, New Brunswick, NJ Direct inquiries to author Byrd-Bredbenner ( Bredbenner@aesop.rutgers.edu). 46 JOURNAL OF FOOD SCIENCE EDUCATION Vol. 6, 2007 C 2007 Institute of Food Technologists

2 Food safety knowledge scales... Methods Questionnaire development This study was approved by the authors Institutional Review Boards. Knowledge questionnaire development was a 7-stage process. The 1st stage involved using published reports (Beard 1991; Partnership for Food Safety Education 1998; Natl. Restaurant Assn. 1999; Medeiros and others 2001b) and input from a panel composed of experts in food safety and sanitation (n = 7) to identify food safety concepts that are key to protecting consumer health. The 2nd stage was to construct a bank of objective questions to assess knowledge of key food safety concepts: cross contamination prevention/disinfection procedures; safe times/temperatures for cooking/storing foods; groups at greatest risk for foodborne disease; foods that increase the risk of foodborne disease; and foodborne pathogens. The items were derived from instruments used in previously reported studies (Beard 1991; Fein and others 1995; Altekruse and others 1996; Walker 1996; Kinnear 1997; Unklesbay and others 1998; Yang and others 1998b; Jay and others 1999; American Meat Inst. Undated) and highlighted in current educational campaigns and research (Knabel 1995; Partnership for Food Safety Education 1998; U.S. Dept. of Agriculture and U.S. Dept. of Health and Human Services 2000; U.S. Dept. of Health and Human Services 2000). Additional items also were constructed to supplement those derived from other sources. Guidelines for test item construction (Berk 1986; Gronlund 1998; Linn and Gronlund 2000) were carefully followed in the development and refinement of all food safety knowledge items. Items measuring the same food safety concept were grouped together in a scale to facilitate the identification of specific knowledge deficit areas that could be targeted later in educational interventions. All items in the original item bank were multiple choice with 3 to 5 answer choices, true/false items, or dichotomous answer series items. Dichotomous answer series items are those that have a question followed by a series of answers, with each answer requiring a response of correct or incorrect (for example, Compared to most people, which of these individuals are more likely to get sick or seriously ill from harmful germs in food? preschool children: yes or no; teenagers: yes or no; pregnant women: yes or no; and so on ). Each multiple choice and true/false item had 1 correct answer and could earn 1 point. Each answer in a dichotomous answer series item had a correct judgment (yes/no), with each answer earning 1 point if judged correctly. In stage 3, the initial question bank (101 items) was reviewed by the panel of experts in food safety, education, and/or tests and measurements to select the most salient questions, identify improvements needed in question phrasing to avoid ambiguity, assess suitability of the questions for the audience, determine completeness of the questions in measuring the breadth and depth of food safety knowledge, and identify items that were unnecessary because they duplicated the content of other items or were deemed to be outside the scope of the study. The review also assessed the content validity of each question in relation to the concept being measured in the scale to which it was assigned. Items were refined based on the panel s input and further reviewed to ensure that they adhered closely to the rules of test item construction. In stage 4, the 72 items remaining after the expert review were pretested with young adults from a variety of majors that were enrolled in freshman/sophomore level courses at a northeastern university (n = 180) during spring semester Results were analyzed to examine item difficulty (percent answering each question correctly) and distracter (incorrect answer choices) functioning. The questionnaire was further refined by deleting most items having difficulty levels less than 15% or greater than 85% because they were too difficult or easy, respectively. If deleting an item would have impaired the integrity of its scale in measuring the breadth and depth of knowledge associated with the scale, the question stem and/or its answer choices were revised to improve the item. To improve answer choice quality, poorly functioning distracters (those selected by few participants) were modified to improve their plausibility or highly functioning distracters and/or correct answer choices were modified to make poorly functioning distracters more attractive. Some questions were eliminated to further reduce duplication and shorten the instrument to reduce participant time burden. In stage 5, the refined instrument that contained 48 items was then pilot tested with a 2nd sample of young adults from a variety of majors that were enrolled in freshman/sophomore level courses at a northeastern university (n = 126) during fall semester 2004, none of whom participated in the pretest. Pilot test participants ranged in age from 18 to 27 (mean = 18.9 ± 1.3 SD); most were freshmen (67%), female (57%), white (55%), in good health (91%), had never held a job as a food preparer (75%), and did not have any food safety training certifications (98%). Pilot test data were analyzed to improve the questionnaire. Items with similar content were culled to reduce questionnaire length. Items that were too easy or difficult were eliminated or refinements were made to item distracters, correct answers, and/or question stem in cases where deleting the question would diminish the integrity of the scale. Despite refinements during the previous stage, many items related to foodborne pathogens continued to be difficult (answered correctly by only 5% to 10% of the participants). To preserve the breadth of the concepts measured by the questionnaire, the expert panel decided to retain these difficult items, and to better control for guessing on items related to foodborne pathogens, add the answer choice don t know even though this answer choice would be scored as incorrect. In stage 6, a final review by the expert panel was conducted to identify any other needed refinements, to verify the accuracy of the answer key, and to confirm the content validity of the 5 knowledge scales. The final knowledge questionnaire had 39 items: 33 multiple choice items with 4 or 5 answer choices, 1 true/false item, and 5 dichotomous answer series items (see Table 1). The test items corresponded to a 7th-grade reading level according to the Flesch Kincaid Grade Level score. The scales used in the final instrument and sample questions are shown in Table 2. In stage 7, this questionnaire was converted into an online survey using Perseus SurveySolutions software (Perseus Development Corp. 2004) and administered to young adults across the United States. Prior to administration, the online survey was tested extensively to ensure that there were no technical errors in functioning or recording of responses. Sample College and university instructors were invited via to recruit students in their introductory level general education courses to complete the online survey. Most participating instructors awarded extra credit points to students that completed the survey. From January to October 2005, 4548 individuals enrolled at 21 colleges and universities completed the knowledge questionnaire as part of a larger nationwide survey of young adults. The colleges and universities were located throughout the United States and included those ranging from community college to research institutions. Participants who were older than the age criterion set in this Available on-line at: Vol. 6, 2007 JOURNAL OF FOOD SCIENCE EDUCATION 47

3 JFSE: Journal of Food Science Education Table 1 Final knowledge questionnaire: item type, item difficulty, and scale reliability Item type Easy (difficult) items a Livingston Multiple Dichotomous Possible Mean Multiple Dichotomous reliability Scale name choice answer series points difficulty index choice answer series c coefficient b Scale 1: Cross contamination prevention & disinfection procedures Scale 2: Safe times/temperatures for cooking/storing food Scale 3: Groups at greatest risk for foodborne disease Scale 4: Foods that increase risk of foodborne disease Scale 5: Common food sources of foodborne disease pathogens 9 2 (with a series of 20 answers) 0 to (0) 8 (1) to (0) n/a (with a series of 8 answers) 0 to (0) 1 (0) d 1 (with a series of 0 to (0) 3 (0) answers) to (3) n/a 0.72 Total 35 4 (54 answers) 0 to (3) 12 (1) 0.92 a Easy items were those with a difficulty index of >0.82. Difficult items were those with a difficulty index of <0.18. b The calculation was computed using 50% as the cut-off passing score. c The last answer choice in the 4 dichotomous series questions was none of the above. Because this answer would not be selected by anyone who identified any of the 7 to 25 other answers as being true, the difficulty index of these 4 items exceeded 96%. d One item was true/false. study for young adults (that is, 17 to 26 y) were eliminated from the analysis; the final sample size was Results and Discussion The mean age of participants (n = 4343) was 19.9 ± 1.7 SD y. Most were female (65%), white (70%), never married (94%), had never held a job preparing food (76%), did not hold a food safety certification (94%), were in good health (88%), self-rated their food safety knowledge level as at least fair (94%), and had never completed a college course in nutrition, food science, or microbiology (64%). The sample was from a wide array of college majors with no major predominating. The responses to the knowledge questionnaire from the 4343 participants underwent item analysis to determine the overall quality of the test and identify improvements needed. An examination of the answer choices indicated that all were functioning. The mean rate at which participants selected the 265 possible answer choices in the questionnaire was 34% (range 1% to 98% of responses). Just 6 answer choices attracted 2% or less of the responses; modifying these responses to make them more plausible could further strengthen the questionnaire s quality. The mean difficulty index for all items combined was 0.61 (see Table 1), which is comparable to the ideal difficulty of 0.62 for the number of items and answer choices on this questionnaire (Ebel and Frisbie 1986). The difficulty index for the vast majority of items was in the acceptable range (0.18 to 0.82). Three of the 4 difficult items (difficulty index ranged from 0.11 to 0.14) were multiple choice and, as on the pilot test, were items related to foodborne pathogens (Scale 5). The answer choice don t know that was provided on Scale 5 accounted for 45% to 72% of participant responses to these very difficult items. The other difficult item was on Scale 1: Cross contamination prevention & disinfection procedures, and asked participants to judge whether a person with a cold should refrain from preparing food for other people in order to prevent food poisoning. All of the easy items (that is, difficulty index >0.82) were dichotomous answer series items. It is important to note that one-third of the easy items were the last answer choice in the 4 dichotomous series questions (that is, none of the above ). Because this answer would not be selected by anyone who identified any of the 7 to 25 other answer choices in the dichotomous series questions as being true, the difficulty index of these items exceeded Other items that were answered correctly by 83% to 91% of the participants were whether (1) those with diarrhea or vomiting should prepare food for others; (2) these foods increased a person s risk of food poisoning: raw oysters, clams, or mussels; or rare hamburgers; and (3) hands should be washed after touching one s face; a pimple; clean pots and pan; or a clean countertop. Guidelines for test item construction indicate that test items should be answered correctly by approximately 20% to 80% of participants. However, experts in test development maintain that item difficulty is less important on criterion-referenced tests like the one in this study and that items falling outside this range should be retained when the items are essential to ensuring that the full range of knowledge about a subject is assessed (Ebel and Frisbie 1986; Linn and Gronlund 2000). In this case, the few items that fell outside the range of 20% to 80% were retained because they measure important concepts related to food safety. Deleting them would diminish the integrity of the questionnaire. A discrimination index (the difference between the percent of participants in the upper quartile of scores that got the item correct and the percent in the lower quartile) was computed for each item. For the overwhelming majority of items, a greater percentage of participants scoring in the upper quartile got the items right compared to those in the lower quartile and, thus, discriminated in the desired manner. The only exceptions were for 7 distracters in 2 dichotomous answer series items. Specifically, when asked to identify all of the 26 food items in a list that increase a person s risk of food poisoning, those scoring in the upper quartile were more likely than those in the lower quartile to incorrectly identify these 5 items as increasing one s risk: meat cooked medium-well, frozen food with frost build-up on the package, food stored in a cabinet beside the oven, commercially canned vegetables eaten right out of the can without reheating them, and rice in a box that does not show a USDA inspection stamp. In addition, compared to those scoring in the lower quartile, those in the upper quartile were more likely to answer incorrectly that to prevent foodborne illness, a person with unexplained itching and a person with a cold should not prepare food for others. Items discriminating in the expected direction had an average discrimination index of 0.33 (range 0.01 to 0.67) indicating that the item discrimination level overall was good and in the range appropriate for a criterion-reference test (Ebel and Frisbie 1986). Items that discriminate in an unexpected direction generally should be revised to improve their quality (Ebel and Frisbie 48 JOURNAL OF FOOD SCIENCE EDUCATION Vol. 6, 2007 Available on-line at:

4 Food safety knowledge scales... Table 2 Questionnaire scales and sample items Scale Scale 1: Cross contamination prevention & disinfection procedures Scale 2: Safe times/temperatures for cooking/storing food Scale 3: Groups at greatest risk for foodborne disease Scale 4: Foods that increase risk of foodborne disease Scale 5: Common food sources of foodborne disease pathogens Sample item To prevent food poisoning, the best way to wash dishes is to.... Choice 1: wash and dry them in an automatic dishwasher Choice 2: soak them in the sink for several hours and then wash them in the same water Choice 3: hand wash them right after the meal and then let them air-dry Choice 4: hand wash them right after the meal and then dry them with a dish towel A. choice 1 or 2 B. choice 1 or 3 C. choice 2 or 4 D. choice 3 or 4 E. any choice is fine as long as the dishes look clean If a family member is going to be several hours late for a hot meal, how should you store the meal to keep it safe until this person is ready to eat it? A. store it in the refrigerator and reheat it when the person is ready to eat it B. store it in on the kitchen counter until the person is ready to eat it C. store it in a cool oven until the person is ready to eat it D. store it in a warm oven until the person is ready to eat it Compared to most people, which of these individuals are more likely to get sick or seriously ill from harmful germs in food? (Mark all that apply) A. preschool children B. teenagers C. pregnant women D. older people (age 60 and over) E. people who are HIV positive F. cancer patients G. people who frequently eat at restaurants or get take-out food often Eating which of these foods will increase a person s risk of food poisoning? (Mark all that apply). A. raw oysters, clams, or mussels B. home canned beans, carrots, peas or potatoes right from the jar C. unpasteurized milk D. rare hamburgers E. leftover soup reheated until warm, but not boiling F. fried eggs with a runny or soft yolk G. raw homemade cookie dough or cake batter H. sushi I. food right from the refrigerator that feels warm J. food stored under the sink K. picnic foods that were stored at room temperature for more than 2 h L. box of rice that does not show a USDA inspection stamp Listeria bacteria are mostly associated with which food? A. home canned foods B. raw or undercooked beef C. deli meats D. raw eggs and poultry E. don t know 1986). An examination of these items indicates that some would benefit by improvement (for example, substitute roasts and steaks for meat in the meat cooked medium well item to indicate this question is referring to large cuts of meat, not ground meat; insert canned or packaged at the beginning of the foods stored in a cabinet beside the oven item to indicate that this item refers to shelf-stable foods). Other poorly discriminating items, however, seem to indicate that young adults that are more knowledgeable about food safety may be overly cautious (for example, commercially canned foods do not need reheating prior to consumption; rice packages do not show USDA inspection stamps; food preparers with unexplained itching or colds do not need to curtail food preparation to avoid transmitting foodborne illness). To examine the effectiveness of the incorrect answer choices in attracting students, the frequency with which those scoring in the lower quartile selected incorrect answers was compared to the frequency those in the upper quartile selected them (Linn and Gronlund 2000). The incorrect answer choices of all of the items discriminated in the expected direction functioned well (Linn and Gronlund 2000). That is, the incorrect answer choices attracted about equal or more lower scoring students than high scoring students with a single exception: the incorrect answer choice anti-bacterial soap was commonly selected as the answer for the question stem The best way to keep from getting food poisoning from fresh fruits and vegetables is to wash them with. Thirteen percent more participants in the upper quartile selected this incorrect answer than those in the lower quartile, which, again, may indicate the greater apparent cautiousness of those that are more informed about food safety. The reliability of the final knowledge test was 0.92 as computed by Livingston s Coefficient for criterion-referenced tests using a passing score of 50% (Livingston 1972; Berk 1986; Coscarelli and Shrock 2002). Reliability coefficients for each Available on-line at: Vol. 6, 2007 JOURNAL OF FOOD SCIENCE EDUCATION 49

5 JFSE: Journal of Food Science Education scale ranged from 0.72 to 0.87 (see Table 1). The reliability coefficients for the total test and each scale indicate the measures are highly reliable (Ebel and Frisbie 1986; Gronlund 1998; Linn and Gronlund 2000). The complete questionnaire is found in the Appendix. Conclusions Overall, the food safety knowledge questionnaire developed in this study meets or exceeds standards of reliability and validity (Ebel and Frisbie 1986; Gronlund 1998; Linn and Gronlund 2000). Potential uses include baseline assessment of food safety knowledge as well as measurement of knowledge gain after an educational intervention. The strength of the individual scale reliability coefficients indicates the scales can be used independently in cases where researchers are focusing on a single concept rather than conducting a comprehensive evaluation. In addition, the 7th grade reading level indicates that this questionnaire likely is appropriate for use with a wide variety of (adolescent through adult) audiences, although researchers are cautioned to conduct a pilot test to confirm its appropriateness with these other audiences. A limitation of this study is that the sample was restricted to young adults enrolled in a small sampling of colleges and universities in the United States. However, these students were distributed across the nation and attending colleges and universities of varying sizes with widely varying admission requirements. This is one of the few studies of its kind (Medeiros and others 2004; Unklesbay and others 1998) and adds to the tools available to improve the development and assessment of the effectiveness of food safety education programs. Future research should investigate the usefulness of the food safety questionnaire in generating baseline data from other audiences as well as establish the usefulness of the questionnaire scales as a measure of food safety intervention effectiveness. Acknowledgment This research was funded by the U.S. Dept. of Agriculture, Natl. Food Safety Initiative, Grant nr Appendix Food Safety Knowledge Questionnaire Select the BEST answer Cross Contamination Prevention/Disinfection Procedures Scale 1. The best way to keep from getting food poisoning from fresh fruits and vegetables is to wash them with () regular soap. () hot water. () anti-bacterial soap. () an anti-bacterial sponge. () cool running water. 2. After you have used a cutting board to slice raw meat, chicken, or fish and need to cut other foods, which of these is the best way to prevent food poisoning? Choice 1: wipe the cutting board off with a paper towel Choice 2: rinse the cutting board under very hot water Choice 3: turn the board over and use the other side Choice 4: wash the cutting board with hot soapy water and rinse Choice 5: set the cutting board aside and use a different cutting board to cut other foods () Choice 1 or 3 () Choice 2, 3, or 5 () Choice 3 or 4 () Choice 4 or 5 () All of the choices 3. To prevent food poisoning, the best way to wash dishes is to: Choice 1: wash and dry them in an automatic dishwasher Choice 2: soak them in the sink for several hours and then wash them in the same water Choice 3: hand-wash them right after the meal and then let them air-dry Choice 4: hand-wash them right after the meal and then dry them with a dish towel () Choice 1 or 2 () Choice 1 or 3 () Choice 2 or 4 () Choice 3 or 4 () Any choice is fine as long as the dishes look clean 4. When should kitchen counters be washed, rinsed, and sanitized? () after each use () when you begin working with another type of food () at 4-hour intervals if the counter is in constant use () all of the above 5. Which procedure for cleaning kitchen counters is most likely to prevent food poisoning? () spray with a strong sanitizing solution () wash with a detergent, rinse, then wipe with a sanitizing solution () wipe with a sanitizing solution, then rinse with clean water and wipe dry () brush off any dirt or food pieces, then wipe with sanitizing solution 50 JOURNAL OF FOOD SCIENCE EDUCATION Vol. 6, 2007 Available on-line at:

6 Food safety knowledge scales To prevent food poisoning, how often should the kitchen sink drain in your home be sanitized? () daily () weekly () monthly () only when food is going to be thawed or washed in the sink 7. Which is the most hygienic way to wash your hands? () apply sanitizer, run water, rub hands together for 20 seconds, rinse hands, dry hands, rub on an antiseptic hand lotion () apply soap, rub hands together for 20 seconds, rinse hands under water, dry hands, apply sanitizer () run water, moisten hands, apply soap, rub hands together for 20 seconds, rinse hands, dry hands () run water, moisten hands, apply sanitizer, rub hands together for 20 seconds, rinse hands, dry hands, rub on antiseptic hand lotion 8. If you have a sore on the back of your hand, should you prepare food for other people? () Yes, if it isn t infected. () Yes, if you put a bandage on it. () Yes, if you wear a glove. () Yes, if you bandage the sore and wear a glove. () No, you should not prepare food until the sore heals. 9. Which should not be done when storing raw meat, fish, or poultry in the refrigerator? () place it in the coldest part of the refrigerator. () set it in a larger container before refrigerating. () place it on the lowest shelf in the refrigerator. () leave it in the package it came in () all should be done when storing raw meat, fish, or poultry 10. To prevent food poisoning, which of these individuals should not prepare food for other people? (Check all that apply) [] a person with diarrhea [] a person with severe acne [] a person with bandaged burns on his or her hands that are covered with gloves [] a person with a fever [] a person with unexplained itching [] a person who smokes [] a person with a sore throat [] a person with a cold [] a person with vomiting [] person with HIV [] none of these individuals 11. When preparing food, you should wash your hands after touching which of these? (Check all that apply) [] your face [] clean pots and pans [] utensils that are being used to prepare food [] fresh fruit [] dishes that came out of the dishwasher [] a pimple [] clean countertop [] clothing [] none of these foods Safe Times/Temperatures for Cooking/Storing Food Scale 1. Which practice is most likely to cause food poisoning? () leaving stuffing in a cooked turkey until it cools to room temperature () stuffing turkeys just before cooking them () cooking turkeys until the stuffing reaches 165 F () removing the giblet bag before cooking a turkey 2. When is it safest to place refrigerated foods in your cart when grocery shopping? () early in the shopping trip () about halfway through the shopping trip () near the end of the shopping trip () at the very end of the shopping trip, just before checking out () it doesn t matter when I place them in the cart 3. What is the recommended freezer temperature for preventing food poisoning? () 0 F( 18 C) () 18 F( 8 C) () 24 F( 4 C) () 32 F(0 C) Available on-line at: Vol. 6, 2007 JOURNAL OF FOOD SCIENCE EDUCATION 51

7 JFSE: Journal of Food Science Education 4. Imagine that your electricity went off and the meat, chicken, and/or seafood in your freezer thawed and felt warm. To prevent food poisoning, what should you do? () throw them away () cook them right away () see how they smell or look before deciding what to do () immediately re-freeze until solidly frozen, then cook them 5. Which of the following is considered the most important way to prevent food poisoning? () spray for pests in the kitchen area at least every week () rarely or never serve leftovers () keep foods refrigerated until it s time to cook or serve them () clean kitchen counters with sanitizing solutions weekly 6. For ground beef to be safe to eat, it needs to be cooked until its internal temperature reaches () 90 F(32 C) () 125 F(52 C) () 160 F(71 C) () 250 F (121 C) 7. What is the maximum temperature refrigerators should be to preserve the safety of foods? () 0 F( 18 C) () 25 F( 4 C) () 40 F(4 C) () 45 F(7 C) () 60 F(16 C) 8. If a family member is going to be several hours late for a hot meal, how should you store the meal to keep it safe until this person is ready to eat it? () store it in the refrigerator and reheat it when the person is ready to eat it () store it in on the kitchen counter until the person is ready to eat it () store it in a cool oven until the person is ready to eat it () store it in a warm oven until the person is ready to eat it 9. All foods are considered safe when cooked to an internal temperature of () 130 F(54 C) () 140 F(60 C) () 150 F(66 C) () 165 F(74 C) 10. Which method is the most accurate way of determining whether hamburgers are cooked enough to prevent food poisoning? () cut one to check the color of the meat inside () check the color of the juice to be sure it is not pink () measure the temperature with a food thermometer () check the texture or firmness of the meat () measure the length of time the hamburgers cook 11. Which food does not need to be refrigerated to prevent food poisoning? () fresh fruit salad () roasted ears of corn on the cob () open box of raisins () chocolate pudding () an open can of green beans 12. To prevent food poisoning, how long should leftover foods be heated? () until they are boiling hot () just until they are hot, but not too hot to eat right away () just until they are at least room temperature () reheating isn t necessary 13. What is the least safe method for thawing a frozen roast? () leave it in the refrigerator until it is thawed () leave it on the kitchen counter until it is thawed () put it in a microwave oven set to automatic defrost () put it in under running water for 1 hour 14. What is the safest method for cooling a large pot of hot soup? () put the soup in a shallow pan and refrigerate it right away () place the cooking pot filled with soup in the refrigerator right away () transfer the soup to clean, deep pot before refrigerating it () leave it on the counter until it cools to room temperature, then refrigerate it () add ice cubes to the soup 52 JOURNAL OF FOOD SCIENCE EDUCATION Vol. 6, 2007 Available on-line at:

8 Food safety knowledge scales... Foods that Increase Risk of Foodborne Disease Scale 1. Chilling or freezing eliminates harmful germs in food. () true () false 2. Which food is least likely to cause food poisoning? () slices of cantaloupe left on the counter overnight () baked potato that was left on the kitchen counter overnight () leftover turkey eaten cold () chocolate cake that was left on the kitchen counter overnight 3. Eating which of these foods will increase a person s risk of food poisoning? (Check all that apply). [] Raw oysters, clams, or mussels [] Home canned beans, carrots, peas or potatoes right from the jar [] Unpasteurized milk [] Rare hamburgers [] Leftover soup reheated until warm, but not boiling [] Fried eggs with a runny or soft yolk [] Raw homemade cookie dough or cake batter [] Grilled steak served on the same plate that held raw steak without washing the plate [] Sushi [] Food right from the refrigerator that feels warm [] Meat cooked medium-well [] Unpasteurized fruit juice [] Cooked shellfish that have unopened shells [] Sliced melon [] Foods prepared in a kitchen with a pet present [] Fresh fruit salad stored at room temperature [] Frozen foods with frost build up on the package [] Food stored in a cabinet beside oven [] Fresh raw milk cheese [] Soft food like jelly or sour cream after scraping off mold [] Soft scrambled eggs [] Raw sprouts (alfalfa, clover, radish) [] Commercially canned vegetables right out the can without re-heating them [] Picnic foods that were stored at room temperature for more than 2 hours [] Box of rice that does not show a USDA inspection stamp [] none of these foods Groups at Greatest Risk for Foodborne Disease Scale 1. People should be especially careful about not eating raw seafood, if they have () diabetes. () HIV infection. () cancer. () any of these diseases. 2. Which foods do pregnant women, infants, and children not need to avoid? () soft cheeses, cold smoked fish, cold deli salads () hot dogs and lunchmeats that have not been reheated () raw or undercooked eggs () canned vegetables, pasteurized fruit juice () These individual do not need to avoid any type of food 3. Compared to most people, which of these individuals are more likely to get sick or seriously ill from harmful germs in food? (Check all that apply) [] preschool children [] teenagers [] pregnant women [] older people (age 60 and over) [] people who are HIV positive [] cancer patients [] people who frequently eat at restaurants or get take-out food often [] none of these individuals Common Food Sources of Foodborne Disease Pathogens 1. Salmonella bacteria can cause food poisoning. How can a food be made safe if it has salmonella in it? () cook it thoroughly () wash it under extremely hot running water () freeze it for at least 3 days () the food cannot be made safe Available on-line at: Vol. 6, 2007 JOURNAL OF FOOD SCIENCE EDUCATION 53

9 JFSE: Journal of Food Science Education 2. Staph (Staphylococcus) bacteria that cause food poisoning are most likely associated with which food? () contaminated water from unfiltered mountain streams and lakes () food prepared by cooks with their bare hands and then left at room temperature () undercooked pork, especially bacon () raw or undercooked eggs and poultry 3. Botulism is a disease that is most likely associated with which food? () canned foods () food prepared by cooks with their bare hands and then left at room temperature () undercooked pork, especially bacon () raw eggs, or raw or undercooked poultry 4. Listeria bacteria are most likely associated with which food? () home canned foods () raw or undercooked beef () deli meats () raw eggs and poultry 5. Harmful E. coli bacteria are most likely associated with which food? () raw or undercooked pork () sliced lunch meat () soft cheeses like Brie () raw or undercooked beef 6. Trichinosis is most likely associated with which food? () deli meats () raw or undercooked pork () soft cheeses like Brie () raw or undercooked beef 7. Campylobacter bacteria are most likely associated with which food? () canned food () raw or undercooked pork () raw or undercooked poultry () raw or undercooked beef 8. You may contaminate the next food you touch with salmonella bacteria if you don t wash your hands after touching: () raw pork. () raw sprouts and lettuce. () raw beef. () raw chicken. Scoring Correct answers are in bold type. Each correct answer is awarded 1 point. Scale scores are calculated by summing total points earned in each scale. Scale scores are summed to compute the total food safety knowledge score. Possible Livingston reliability Scale name points coefficient Cross contamination prevention 0to & disinfection procedures Safe times/temperatures for 0to cooking/storing food Foods that increase risk of 0to foodborne disease Groups at greatest risk for 0to foodborne disease Common food sources of 0to foodborne disease pathogens Total 0 to The calculation was computed using data from 4343 young adults enrolled in college between April and October 2005; a 50% as the cut-off passing score. Developers Carol Byrd-Bredbenner, PhD, RD, FADA Virginia Wheatley, MPH, REHS Donald Schaffner, PhD Christine Bruhn, PhD, RD Lydia Blalock, PhD Jaclyn Maurer, PhD, RD For More Information, Contact: Carol Byrd-Bredbenner, PhD, RD, FADA Professor of Nutrition/Extension Specialist Rutgers Univ. 26 Nichol Ave., 220 Davison New Brunswick, NJ Tel.: Fax: bredbenner@aesop.rutgers.edu 54 JOURNAL OF FOOD SCIENCE EDUCATION Vol. 6, 2007 Available on-line at:

10 Food safety knowledge scales... References Altekruse S, Street D, Fein S, Levy A Consumer knowledge of foodborne microbial hazards and food-handling practices. J Food Prot 59(3): Altekruse S, Yang S, Timbo B, Angulo F A multi-state survey of consumer food-handling and food-consumption practices. Am J Prev Med 16(3): American Meat Inst. Undated. Report says education is missing link in food safety equation: cites major knowledge gap on ways to prevent foodborne illness. Available at: Accessed on January 9, Anonymous Checking on changes: consumer research. Food Safety Educ 7(1):1 2. Beard T HACCP and the home: the need for consumer education. Food Technol 45(6): Berk RA A consumer s guide to setting performance standards on criterion referenced tests. Rev Educ Res 56(1): Contento I, Randell J, Basch C Evaluation and design of nutrition knowledge measures. J Nutr Educ Behavi 34(1):2 25. Coscarelli W, Shrock S The two most useful approaches to estimating criterion-referenced test reliability in a single test administration. Perform Improv Q 15(4): Ebel R, Frisbie D Essentials of educational measurement. Englewood Cliffs, NJ: Prentice-Hall Inc. Fein S, Lin C-T, Levy A Foodborne illness: perceptions, experience, and preventive behaviors in the United States. J Food Prot 58: Food Safety and Inspection Service U.S. Dept. of Agriculture. 1998a. CDC reports: behavioral risk. Food Safety Educ 3(4). Available at: Accessed on November 28, Food Safety and Inspection Service U.S. Dept. of Agriculture. 1998b. FDA: consumers are changing. Food Safety Educ 3:4. Food Safety and Inspection Service U.S. Dept. of Agriculture. 1998c. New consumer research driving food safety education. Food Safety Educ 3(4):4. Gronlund N Assessment of student achievement, 6th ed. Boston: Allyn & Bacon. Jay L, Comar D, Govenlock L A national Australian food safety telephone survey. J Food Prot 62(8): Kastner C The real story about food safety. J Anim Sci 73(9): Kinnear J What s your candidate s food safety IQ? There s no better time or place to find out than during the interview. Food Manag 32:50 6. Klontz K, Timbo B, Fein S, Levy A Prevalence of selected food consumption and preparation behaviors associated with increased risks of food-borne disease. J Food Prot 58(8): Knabel S Foodborne illness: role of home food handling practices. Scientific status summary. Food Technol 49(4): Linn R, Gronlund N Measurement and assessment in teaching, 8th ed. Upper Saddle River, NJ: Merrill Co. Livingston SA Criterion-referenced applications of classical test theory. J Educ Meas 9(1): Medeiros L, Hillers V, Chen G, Bergmann V, Kendall P, Schroeder M Design and development of food safety knowledge and attitude scales for consumer food safety education. J Am Diet Assoc 104(11): Medeiros L, Hillers V, Kendall P, Mason A. 2001a. Evaluation of food safety education for consumers. J Nutr Educ Behavi 33(suppl 1):SS Medeiros LC, Kendall P, Hillers V, Chen G, DiMascola S. 2001b. Identification and classification of consumer food-handling behaviors for food safety education. J Am Diet Assoc 101(11): Natl. Restaurant Assn. EF ServSafe Coursebook. USA: Natl. Restaurant Assn. Partnership for Food Safety Education Fight bac! Snapshot of consumer behavior and attitudes. Available at: Accessed on December 29, Perseus Development Corp Perseus SurveySolutions. Version 6. Braintree, MA: Perseus Development Corp. U.S. Dept. of Agriculture, Food and Drug Administration Results of the Food and Drug Administration s 1988 health and diet survey food handling practices and food safety knowledge for consumers. Washington, DC: U.S. Dept. of Agriculture and Food and Drug Administration. U.S. Dept. of Agriculture, Food and Drug Administration USDA/FDA education initiative: evaluating the placement of food safety education in American Schools. Available at: vm.cfsan.fda.gov/ dms/fseduin2.html. Accessed on February 24, U.S. Dept. of Agriculture, Food Safety Inspection Service Changes in consumer knowledge, behavior, and confidence since the 1996 PR/HACCP Final Rule. Available at: haccpimpacts.pdf. Accessed on February 22, U.S. Dept. of Agriculture, U.S. Dept. of Health and Human Services Nutrition and your health: Dietary guidelines for Americans, 5th ed. Washington DC: U.S. Government Printing Office. U.S. Dept. of Health and Human Services Healthy people 2010: Volume 1. Available at: tableofcontents.htm. Accessed on November 28, Unklesbay N, Sneed J, Toma R College students attitudes, practices, and knowledge of food safety. J Food Prot 61(9): Walker A Food-safety in the home a survey of public awareness. London: Her Majesty s Stationary Office. Williamson D, Gravani R, Lawless H Correlating food safety knowledge with home food-preparation practices. Food Technol 46(5): Yang S, Leff M, McTague D, Horvath K, Jackson-Thompson J, Murayi T, Boeselager G, Melnik T, Gildemaster M, Ridings D, Altekruse S, Angulo F. 1998a. Multistate surveillance for food-handling, preparation, and consumption behaviors associated with foodborne diseases: 1995 and 1996 BRFSS food-safety questions. Mor Mortal Wkly Rev 47(SS-4): Yang S, Leff M, McTague D, Horvath K, Jackson-Thompson J, Murayi T, Boeselager G, Melnik T, Gildemaster M, Ridings D, Altekruse S, Angulo F. 1998b. Multistate surveillance for food-handling, preparation, and consumption behaviors associated with foodborne diseases: 1995 and 1996 BRFSS food-safety questions. Appendix. Mor Mortal Wkly Rev 47(SS-4): Available on-line at: Vol. 6, 2007 JOURNAL OF FOOD SCIENCE EDUCATION 55