Update on the National Board Dental Hygiene Examination

Transcription

1 Update on the National Board Dental Hygiene Examination Dr. David M. Waldschmidt Joint Commission on National Dental Examinations ADEA 2013 Annual Session Seattle, WA 2012 Joint Commission on National Dental Examinations. All rights reserved.

2 Overview and Agenda This is an informational session intended to provide an update on the National Board Dental Hygiene Examination. Introduction Joint Commission on National Dental Examinations (JCNDE) policy updates Constructing the National Board Dental Hygiene Examination Examination Trends Survey Feedback Additional information and resources

3 Mission Statement of the JCNDE The JCNDE develops and conducts highly reliable, state of the art cognitive examinations that assist regulatory agencies in making valid decisions regarding licensure of oral health care professionals, develops and implements policy for the orderly, secure, and fair administration of its examinations, and is a leader and resource in assessment for the oral health care profession. 3

4 Composition of the JCNDE Appointing Organizations and Current Appointees AADB (6) Guy Shampaine, DDS, Chair LeeAnn Podruch, DDS, JD Jerri Ann Donahue, DDS Conrad P. McVea, III, DDS Michael Reggie VanderVeen, DDS Open ADEA (3) Connie Drisko, DDS Marc E. Levitan, DDS Birgit J. Glass DDS, MS ADA (3) ADHA (1) ASDA (1) Public (1) Liaisons & Observers Lorin D. Peterson, DDS, Vice-Chair Peter S. Trager, DDS Robert A. Hersh, DDS Mary Lou Gerosky, RDH, MEd Ian R. Murray, BS Ms. Kelley Shannon Kenneth J. Versman, DDS, MS (ADA Board Liaison) Jiwon Lee (ASDA Observer) Liaisons and observers do not participate in voting 4

5 JCNDE Committees Committees of the JCNDE Committee on Administration Committee on Dental Hygiene Committee on Examination Development Committee on Research and Development Committee for an Integrated Examination (CIE) (ad Hoc) Test Construction Committees (TCCs) NBDE Part I (6) NBDE Part II (12) National Board Dental Hygiene Examination (6) More than half of the individuals on TCCs are dental educators Criteria for TCC membership are published in the JCNDE Standing Rules 5

6 Department of Testing Services Test Development Test Administration Research and Development/ Psychometrics Client Services and Special Projects

7 Examination Programs Joint Commission on National Dental Examinations (JCNDE) National Board Dental Examinations Part I National Board Dental Examinations Part II National Board Dental Hygiene Examination (NBDHE) Council on Dental Education and Licensure (CDEL) Dental Admission Test (DAT) Association of Schools and Colleges of Optometry (ASCO) Optometry Admission Test (OAT)

8 Annual Department Activities Administer 40,000+ examinations 75,000+ phone calls plus and fax correspondence DENTPIN requests/updates Examination guides, website updates Examination/test applications including fee waiver requests and testing accommodations Score reporting Fulfill additional score report requests (30,000+) Fulfill orders for NBDHE released item sets, National Board certificates, and the DAT practice test. Resolve issues/problems with testing (emergencies, ID problems, testing conditions)

9 Annual Department Activities Conduct 40+ test construction committee meetings Scoring and research services for contracted clients JCNDE and DAT publication and web site updates R&D staff research publications and presentations Activities conducted by professionally trained staff (currently 24 full-time employees)

10 JCNDE Software Infrastructure Upgrade Background Five testing programs, each operating completely independently. Difficult to maintain systems and implement new policies programmatically; data was also formatted differently. The JCNDE wanted to serve communities of interest more effectively, reduce costs, improve efficiency. The goal was to unify those programs including all historical data into a single system. No off-the shelf solution existed, needed to customize an existing product to meet JCNDE needs. In 2005, DTS submitted a request to initiate a project to address the above issues. 10

11 JCNDE Software Infrastructure Upgrade Service interruption scheduled through April 3 rd, affecting the following services: Updates/retrievals/requests involving personal information/dentpin Application processing Score report requests Score processing Communication regarding the interruption provided in December of 2012, follow-up communication in February of

12 JCNDE Software Infrastructure Upgrade A long, and arduous process Our existing systems are incredibly complex to program. The Joint Commission in its history has never undertaken a data/program conversion effort of this magnitude. A large number of issues (over 340) were identified and required substantial effort to address. The Aptify conversion effort impacted our regular release cycle for all exams EXCEPT dental hygiene. Staff are doing everything they can to mitigate delays and are working to restore services as soon as possible. 12

13 JCNDE Policy Updates: Reminders Application eligibility period is now 6 months. Score reports now include a history of scores for all examination attempts. As of January 1, 2012, candidates are limited to successful completion of an examination within five years of testing or five examination attempts, whichever comes first. Administrations prior to 2012 do not count as part of the five. Candidates are prohibited from taking the examination to practice or to obtain an advance review of the content. 13

14 JCNDE Policy Updates (2013) Examination results may be withheld or reported when forensic analyses provide compelling information indicating the presence of an irregularity. At its 2013 meeting, the Joint Commission will be reviewing the criteria for selecting dental hygiene test constructors. 14

15 JCNDE Policy Updates (2013) The goal of the IDEA Grant Program is to enhance the Joint Commission s testing program with respect to the evaluation of those seeking licensure to practice dentistry or dental hygiene. The Joint Commission adopted new procedures and timeframes for reviewing and approving proposals (e.g., letter of intent to precede formal submission), in hopes of receiving proposals that are more closely aligned with the goals of the Joint Commission and its IDEA Grant Program. 15

16 JCNDE Updates (2013) National Board examination fees are currently as follows: NBDE Part I: $355 NBDE Part II: $400 NBDHE: $390 Score report request fees are currently $

17 Constructing the National Board Dental Hygiene Examination Test construction begins with a statement of test purpose The purpose of the National Board Dental Hygiene Examination is to assist state boards in determining the qualifications of individuals seeking licensure to practice dental hygiene. 17

18 Examination Content Specifications Examination content specifications are based on the findings of validity studies involving practice analyses which are conducted periodically. The most recent practice analysis for the Dental Hygiene examination occurred in Test construction committees are responsible for recommending minor modifications during the interim period between practice analyses. The Joint Commission with its Committee on Dental Hygiene (NBDHE) approves changes to the content specifications for the NBDHE. 18

19 Item and Examination Development Test construction committees are responsible for the development of items and editions (i.e., forms) of the examinations, using Joint Commission guidelines for writing high-quality multiple-choice items. 19

20 Examination Content: NBDHE The NBDHE is a comprehensive, computer-based examination that is administered in 1 day and is usually taken when the individual is within 4 months of being issued a dental hygiene diploma. The NBDHE contains 350 items, 200 of which are discipline-based and 150 of which are case-based. The discipline-based items focus on three major areas: I. Scientific Basis for Dental Hygiene Practice II. Provision of Clinical Dental Hygiene Services III. Community Health / Research Principles. Case-based items can consist of the following: Patient dental/medical history Dental chart Radiographs Photographs. 20

21 Administration A high level of security is maintained on all examination materials. Strict precautions are in place at the Joint Commission s offices and at testing centers to ensure that test content is not compromised. Administrations are monitored for irregularities. The Joint Commission offers the NBDHE on computer at Pearson Testing Centers across the US. 21

22 Administration: Common Items

23 Administration: Security

24 Administration: Security

25 Scoring and Reporting Exams are scored based on the number of correct answers provided by the candidate. Number correct scores (i.e., raw scores) are then converted to scale scores. Results are reported as pass/fail based on performance standards set via standard setting activities. 25

26 Standard Setting The NBDHE is a criterion-referenced examination (not norm-referenced). Expert educators and practitioners identify standards (pass/fail points) following established procedures and criteria, not by the process sometimes known as grading on a curve. The most recent Dental Hygiene standard setting exercise occurred in 2010 and was implemented in Standards are maintained across examination forms through equating procedures that control for any subtle differences in difficulty across examination forms. 26

27 Trends in Administration Volume and Candidate Performance 2012 Joint Commission on National Dental Examinations, All rights reserved. 27

28 Examinee Volume and Performance Trends The following slides present information concerning examinee volume and performance for the 10 year period between 2003 and Examinee volume data include all individuals (first time, repeaters, accredited, non-accredited) completing the National Board Dental Hygiene Examination. Performance trend data include candidates enrolled in accredited schools in the U.S and Canada who took the examination for the first time. 28

29 NBDHE Administrations ( ) 9,000 8,000 7,000 6,000 5,000 4,000 3,000 2,000 1,000 - NBDHE Administrations ( ) Total Accredited Non- Accredited 2013 Joint Commission on National Dental Examinations. All rights reserved. 29

30 Exam Administration Trends 3000 NBDHE Monthly Volumes ( ) January February March April May June July August September October November December 30

31 NBDHE Failure Rates * * 2012 * A new standard was introduced this year, based on updated standard setting activities. Note: the failure rate for 2012 was updated subsequent to the ADEA meeting based on new data (the rate changed from 4.3% to 4.2%) Joint Commission on National Dental Examinations, All rights reserved. 31

32 Exam Administration Trends Testing Accommodations Examination Number of Accommodations Provided (2012) NBDHE 61 Accommodations have been provided for documented learning disabilities, medical conditions, sensory impairments, etc. Examples of accommodations include additional testing time, separate testing room, lighting and seating adjustments, etc. The JCNDE has seen a dramatic increase in the number of accommodations requested over the last few years. 32

33 Survey Feedback At the end of the testing session candidates are asked questions concerning their satisfaction with the testing experience. Changes to survey items occurred in this cycle: Rating Scale (5 pt): Strongly Agree to Strongly Disagree Revised survey questions to improve clarity. The following slides present the findings associated with the new survey items. 33

34 NBDHE Survey Feedback The tutorial at the beginning of the exam helped me learn how to navigate through the exam. (N=2,598) 59.5 Strongly Agree Agree Undecided Disagree Strongly Disagree Not Applicable 2013 Joint Commission on National Dental Examinations, All rights reserved. 34

35 NBDHE Survey Feedback I knew how to use the computer to take the exam. (N=2,586) Strongly Agree Agree Undecided Disagree Strongly Disagree 2013 Joint Commission on National Dental Examinations, All rights reserved. 35

36 NBDHE Survey Feedback The quality of the radiographs was sufficient to answer the questions. (N=2,587) Strongly Agree Agree Undecided Disagree Strongly Disagree 2013 Joint Commission on National Dental Examinations, All rights reserved. 36

37 NBDHE Survey Feedback The quality of the clinical photographs was sufficient to answer the questions. (N=2,587) Strongly Agree Agree Undecided Disagree Strongly Disagree 2013 Joint Commission on National Dental Examinations, All rights reserved. 37

38 NBDHE Survey Feedback 60 The quality of labels and arrows appearing with images was sufficient to answer the questions. (N=2,586) Strongly Agree Agree Undecided Disagree Strongly Disagree 2013 Joint Commission on National Dental Examinations, All rights reserved. 38

39 NBDHE Survey Feedback I had sufficient time to complete the exam. (N=2,587) Strongly Agree Agree Undecided Disagree Strongly Disagree 2013 Joint Commission on National Dental Examinations, All rights reserved. 39

40 Summary The Joint Commission continues to work to improve the National Board Dental Hygiene Examination, to better serve the purpose of the examination and the Dental Hygiene community. These efforts are focused in multiple areas such as the following: Program monitoring Updating policies Updating procedures On behalf of the Joint Commission, thank you for the opportunity to provide this update. 40

41 Additional Information and Resources Joint Commission on National Dental Examinations Integrated National Board Dental Examination National Boards (Examination Guides, FAQ s, DENTPIN Information, Score Report Requests) Part I: Part II: Dental Hygiene: Test Construction Committee Information Technical Reports, ADEA Presentations, Item Development Guides 41

42 Contact Information David M. Waldschmidt, Ph.D. Secretary, Joint Commission on National Dental Examinations and Director, Department of Testing Services Kathleen J. Hinshaw, L.D.H., Ed.D. Senior Manager, Test Administration Ellen Ryske, MBA Manager, Client Services and Special Projects Joint Commission on National Dental Examinations 211 E. Chicago Ave., Suite 600 Chicago, Illinois Fax: (312)

43 Questions and Discussion 43

44 Thank You 44