Problem-based learning (PBL), as it is known. Effectiveness of Problem-Based Learning in Chinese Dental Education: A Meta-Analysis

Transcription

1 Effectiveness of Problem-Based Learning in Chinese Dental Education: A Meta-Analysis Beilei Huang, D.D.S., M.S.; Liwei Zheng, D.D.S., M.S.; Chunjie Li, D.D.S., M.S.; Li Li, D.D.S., M.S.; Haiyang Yu, D.D.S., M.D., Ph.D. Abstract: This article provides a critical overview of problem-based learning (PBL) practice in dental education in China. Because the application of PBL has not been carried out on a large scale in Chinese dental education, this review was performed to investigate its effectiveness. Databases were searched for studies that met the inclusion criteria, with study identification and data extraction performed by two reviewers independently. Meta-analysis was done with Revman 5.1. Eleven randomized controlled trials were included. The meta-analysis found that PBL had a positive effect on gaining higher theoretical (SMD=0.88, 95% CI [0.46, 1.31], p<0.0001) and practical scores (SMD=1.48, 95% CI [0.95, 2.00], p<0.0001). However, the pooled result did not show any positive effect on gaining higher pass rates (RR=1.06, 95% CI [0.97, 1.16], p=0.21). This meta-analysis suggests that the PBL pedagogy is considered superior to the traditional lecture-based teaching in this setting. PBL methods could be an optional supplementary method of dental teaching models in China. However, Chinese dental schools should devise PBL curricula according to their own conditions. The effectiveness of PBL should be optimized maximally with all these limitations. Dr. Beilei Huang is a Ph.D. student, Department of Prosthodontics, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, PR China; Dr. Liwei Zheng is a Ph.D. student, Department of Dental Technology, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, PR China; Dr. Chunjie Li is a Ph.D. student, Department of Oral and Maxillofacial Surgery, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, PR China; Dr. Li Li is a Ph.D. student, Department of Endodontics, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, PR China; and Dr. Haiyang Yu is Professor, Department of Dental Technology, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, PR China. Direct correspondence and requests for reprints to Dr. Haiyang Yu, Department of Dental Technology, West China College of Stomatology, SCU, State Key Laboratory of Oral Diseases, West China Hospital of Stomatology, Sichuan University, No. 14, 3 rd section, Renmin South Road, Chengdu , PR China; yhyang6812@scu.edu.cn. Keywords: dental education, problem-based learning, China Submitted for publication 6/9/11; accepted 11/11/11 Problem-based learning (PBL), as it is known today, dates from the 1950s and 1960s. It was developed from educational innovation to resolve dissatisfaction with the conventional medical education practices. 1-3 PBL pedagogy was not applied to dental education until the Malmö Dental School in Sweden became the first one to introduce it to undergraduate courses in It has been applied in various modes (full implementation, hybrid with medicine, course-based hybrid, PBL/traditional hybrid) in dental schools in Europe (Sweden, The Netherlands, Norway, and the United Kingdom), Asia (Hong Kong, Singapore, and Thailand), Australia and New Zealand, the United States, and Canada. 4 In China, however, PBL application in dental education is not a routine pedagogy for multiple reasons. First, the tuition of dental school is less than 10,000RMB (around US$1,300). The fees of dental equipments and materials used by the students are totally paid by the government. There is deficiency in providing materials for the application of PBL. Second, the level and standard of dental education dramatically vary among different institutions in China; standard criteria for dental education across the nation do not exist. Third, the traditional educational methods have been used for many years and almost constitute the whole educational experience of them. Also, the traditional education follows a teacher-oriented pattern, which places great emphasis on disseminating factual knowledge, which students passively receive. Fourth, the curricular arrangements in China are totally different from that in other countries, such as the United States. Dental students complete all the dental courses (nearly fifty credit units) in 1 to 1.5 years, before the beginning of dental clinical training. Time limitations make dental schools employ more convenient ways of teaching, rather than a revolutionized one. Furthermore, humility and courtesy are deeply embedded in Chinese communication. According to research based on the Myers-Briggs Type Indicator (MBTI), Chinese dental school applicants showed much March 2013 Journal of Dental Education 377

2 higher ratio of introversion to extroversion mental attitude compared to that of the English applicants. 5 The introversion attitude is not typically encouraged in an open discussion because introverts are usually not enthusiastic to express their opinions, especially those opinions that may be different from others. For all these reasons, the development of PBL in dental education is behind that in other countries such as the United States. In consideration of the great demand of high-level dentists in China, the application of PBL pedagogy will be meaningful in curriculum revision. Recently, various dental institutions have taken exploratory steps in PBL pedagogy in China. West China School of Stomatology, Sichuan University (WCSS) adopted a PBL curriculum hybrid in teaching medicine in an eightyear program. The School of Stomatology at Wuhan University (WHUSS) established a multidisciplinary PBL curriculum for clinical education in a seven-year program. 6 Reports and reviews have been published on the effectiveness of PBL methods in dental education in China with certain positive outcomes. This systematic review aims to summarize the trials done in China to evaluate the overall effectiveness of PBL pedagogy compared to the traditional teaching methods. The objective of this meta-analysis of the PBL pedagogy studies was to assess the effectiveness of PBL, student-centered programs compared with conventional teaching method in Chinese dental education. Methods We selected studies that met the following criteria: 1) participants the sample included undergraduate and graduate students of dental institutions in China; 2) intervention PBL pedagogy was the evaluated method in the experiment; application of PBL pedagogy in all dental disciplines can be included; 3) comparison the study should compare the effectiveness of the PBL pedagogy group and the control group (traditional teaching methods), and neither group should be exposed to supplementary teaching methods that could have an impact on the results; 4) outcome the outcomes can be presented as data or description; and 5) study design the research should be designed as a randomized controlled trial (RCT). The literature search was done in November 2010; we updated the search on March 25, 2012, but no new studies were included. Chinese National Knowledge Infrastructure (CNKI), VIP (Chinese database), and Chinese Biomedical documental database (CBM) were searched. English computerized databases including PubMed, EMBASE, and Cochrane Database were also searched. Next, the snowball method was used, and the reference list of selected articles was reviewed for additional related reports. Study inclusion was done by two reviewers independently; any discrepancies were resolved by discussion. The studies were selected based on reviewing the title and abstract first. All the studies that could possibly meet the inclusion criteria were recorded, and their full text versions were retrieved and examined further to confirm study inclusion. Two investigators independently extracted the data, including the first author, year of publication, number of students in each group, mean scores of both groups, and the involved dental disciplines. Statistical analysis was carried out using Revman 5.1. Statistical heterogeneity was tested by the test for heterogeneity (I2 test). I2 test was used on the level of α=0.10. If I2>50%, p 0.10, a random-effects model was adopted; otherwise, a fixed-effects model was used. The combined results for dichotomous data were expressed as risk ratios (RRs) and 95 percent confidence intervals (CIs) and, for continuous data, mean difference (MD) and 95 percent CI were analyzed; but when the assessment tools were different, we used standardized mean difference (SMD) and 95 percent CI. The statistical significance for the hypothesis test was set at p<0.05 (two-tailed z tests). Sensitive analysis was conducted to test stability of the results. If we were not able to pool the data, we used descriptive statistics only. Results The literature search identified 130 articles in CNKI and no articles in CBM and VIP. None was extracted from PubMed. The inclusion process is shown in Figure 1. Eleven 7-17 articles were included in the review: two for prosthodontics, two for oral and maxillofacial surgery, two for orthodontics, one for operative dentistry, one for periodontics, one for oral medicine, one for pedodontics, and one for dental imaging. All of the articles were published from 2006 through Table 1 shows the study characteristics of the eleven included studies. The effects of PBL methods were evaluated both by data and description. This meta-analysis reports the effects based on the synthesis of theoretical, practical scores and pass rate, which were more 378 Journal of Dental Education Volume 77, Number 3

3 Figure 1. Inclusion and exclusion of randomized controlled trials of PBL effects in dental education in China mainland objective than other indices. The effects of PBL on theoretical scores were reported in eight studies included in this analysis. Six of them were pooled. Heterogeneity analysis showed that a significant heterogeneity among studies (p=0.010; I 2 =67 percent) was identified, and the meta-analysis showed that PBL had significant effects on improving students theoretical scores (SMD=0.88, 95 percent CI [0.46 to 1.31], p<0.0001). Five studies reported practical scores. Three studies could be pooled. There was a significant heterogeneity among studies (p=0.08; I 2 =60 percent), and the meta-analysis showed PBL effect was significant (SMD=1.48, 95 percent CI [0.95 to 2.00], p<0.0001). Only two studies reported pass rate. The result showed there was a lack of heterogeneity among the studies (p=0.46; I 2 =0 percent), and the pooled PBL effect was not significant (RR=1.06, 95 percent CI [0.97 to 1.16], p=0.21). These results show statistical difference in the effects between PBL methods and conventional methods on theoretical and practical scores. The results indicate that PBL has an effect on improving students academic achievement. Although the superior effectiveness of PBL on practical scores was demonstrated, we could not conclude that the students clinical performance, skills, and problemsolving ability were improved. Of the three synthesized studies, only one study (HU Sheng, 2007) tested clinical performance and skills, and the other two studies (WANG Quan, 2010; XU Chunjiao, 2007) tested the students analytical ability. We merged the three studies to evaluate the effects on practical scores. Therefore, the outcome was not conclusive when evaluating the effects on some clinical issues, e.g., the clinical skills, case analytic ability, clinical reasoning, integration of clinical and March 2013 Journal of Dental Education 379

4 Table 1. Characteristics of the eleven included randomized controlled studies Number Number Dental of Students of Students Disciplines PBL/Traditional Male/Female Outcome Measure WANG Quan (2010) Oral maxillofacial surgery 63/63 Not given Measured by both theoretical and practical scores LAN Peng (2010) Oral maxillofacial surgery Not given 21/58 Questionnaire survey JI Yali (2010) Pedodontics 20/20 Not given Measured by theoretical scores and questionnaire survey ZHOU Jianping (2009) Orthodontics 20/20 18/22 Measured by theoretical scores YANG Di (2008) Operative dentistry 18/18 Not given Measured by theoretical scores and questionnaire survey HU Sheng (2007) Prosthodontics 32/23 Not given Measured by both theoretical and practical scores DONG Hong (2008) Oral medicine 20/21 Not given Measured by theoretical scores and questionnaire survey GU Xiangsheng (2007) Dental imaging 15/15 Not given Measured by both theoretical and practical scores SHAO Ping (2007) Orthodontics 15/15 14/16 Descriptive XU Chunjiao (2007) Periodontics 15/15 Not given Measured by both theoretical and practical scores and questionnaire survey SUN Xumei (2007) Prosthodontics 35/35 38/32 Measured by practical scores and questionnaire survey basic science knowledge, and independent study of clinical problems. The analysis did not show statistically significant difference on pass rate. Pass rate is the ratio of the number of students who passed the exams (score 60) to the total number of students. The result indicates that the PBL pedagogy did not increase the pass rates. According to the questionnaire results, most students at the PBL pilot institutions show more enthusiasm for PBL methods rather than conventional teaching methods. The questionnaire results also show that the PBL methods are superior to the conventional teaching methods in improving the students abilities such as critical thinking skills, team spirit, resource researching ability, independent thinking skills, interpersonal skills, ability of oral expression, etc. Discussion Of the eleven included studies, none described random allocation or blinding methods. Therefore, methodology quality of included articles is low. Table 2 shows the risk of bias assessment of the eleven included studies. Also, differences of age, gender, and scores on college entrance exam between PBL and controlled groups have not been mentioned in those studies, so sampling error is not eliminated. In many studies, the students in these RCTs were divided into two groups that were even in number. We confirmed these numbers with the researchers of each original study. In China, the PBL method is still at the elementary stage. Some of the dental schools have taken the exploratory steps in employing PBL methods in dental education. PBL, as an efficient way to prepare dental students for their career in dentistry, should make greater inroads more than just exploratory implementation in China. However, dental schools in China may not duplicate other countries successful experiences in PBL teaching methods since Chinese dental education has its own special qualities. Under these conditions, PBL pedagogy could be applied when the students are in the internship. PBL should not be limited to a particular curriculum or a particular specialty, but should be aimed to cultivate general dentists who can provide the most optimized treatment schedules for the patients. Regarding the significant heterogeneity among the six pooled studies of the theoretical scores and three pooled studies of the practical scores, the reasons lie in the diversities with respect to the following. First, the level of the students included and level of the dental schools where the study was taken 380 Journal of Dental Education Volume 77, Number 3

5 Table 2. Risk of bias assessment of the eleven included randomized controlled studies Blinding (a) participants blinded (b) operator blinded Incomplete Selective Allocation (c) accessor blinded Data Data Other Randomization Concealment (d) statistician blinded Report Report Bias WANG Quan (2010) Unclear Unclear (a) Unclear None None None LAN Peng (2010) Unclear Unclear (a) Unclear None None None JI Yali (2010) Unclear Unclear (a) Unclear None None None ZHOU Jianping (2009) Unclear Unclear (a) Unclear None None None YANG Di (2008) Unclear Unclear (a) Unclear None None None HU Sheng (2007) Unclear Unclear (a) Unclear None None None DONG Hong (2008) Unclear Unclear (a) Unclear None None None GU Xiangsheng (2007) Unclear Unclear (a) Unclear None None None SHAO Ping (2007) Unclear Unclear (a) Unclear None None None XU Chunjiao (2007) Unclear Unclear (a) Unclear None None None SUN Xumei (2007) Unclear Unclear (a) Unclear None None None place are different. In the Chinese dental education system, dental schools are set up by various levels of colleges, both universities and academies. The criteria for admission of dental students are not based on a standard examination such as the Dental Admission Test in the United States. For example, of the six pooled studies in the evaluation of theoretical scores, one study (WANG Quan, 2010) was taken in an academy, while others were taken in universities of higher level. This is one fundamental cause for the heterogeneity between those studies. Second, the pooled studies are of different dental disciplines. Different dental disciplines may demonstrate different effectiveness when the PBL pedagogy applied. In this review, we synthesized the studies of different dental disciplines to evaluate the total effectiveness. This March 2013 Journal of Dental Education 381

6 could also contribute to the heterogeneity. Third is the way the examinations were taken. Six studies adopted different examinations. There is no criterion for the evaluation of the effectiveness of PBL pedagogy on the accumulation of theoretical knowledge. Fourth is the way that PBL pedagogy was applied. Because the dental institutions did not adopt a uniform PBL pedagogy, different teaching methods and aims undoubtedly lead to different outcomes. Limitations and Future Studies Methodology qualities of included articles are low. Included studies are research in the field of medical education, so it is impossible for the researchers to implement allocation concealment and blind. Although studies we included are designed as randomized controlled trials and two investigators examined the studies and extracted the data independently, selection bias and performance bias are unavoidable. Furthermore, the way the measurements were taken was not uniform. That is to say, there is no standard criterion for evaluating the effectiveness of the PBL pedagogy. Therefore, measurement bias also exists in this review. However, there is no evidence of attrition bias in this review. Finally, methodology qualities of studies in the field of medical education could not be comparable to that of a medical science research for many reasons. However, this review has its own merits. It gives us some instruction to implementing the PBL pedagogy and reminds us to adopt a more standardized method in the evaluation of a teaching method. In this meta-analysis, the eleven included articles did not adopt a uniform outcome measurement. There was no standard examination to test the theoretical and practical outcomes, which are aimed to reflect precisely the effect of the PBL pedagogy. Before a standard evaluation system of PBL pedagogy is established, any existing standard examination, such as the National Board Dental Examination (NBDE) in the United States or corresponding examination for any country, could be used as an evaluation test. In 1993, Vernon and Blake did a meta-analysis on PBL in medical education using the National Board Medical Examination (NBME) as a theoretical test. 18 Also, Fincham and Shuler reported in 2001 that dental students in the PBL curriculum at the University of Southern California had significantly superior performance on the NBDE Part I to those students educated with traditional teaching methods. 4 In the case of China, the standard examination such as the Chinese dental license examination could be adopted as the standard examination for PBL pedagogy evaluation, and high-standard randomization is required to diminish bias. None of the eleven included studies stated random allocation methods with detail. During the design of a PBL curriculum, there should be definite educational objectives of PBL pedagogy. According to Barrows, a leader in the field of medical PBL education, the critical educational objectives of authentic PBL are as follows: 1) the acquisition of a rich body of deeply understood knowledge that is integrated from a wide variety of disciplines, structured in ways that will facilitate recall and application to other problems, and enmeshed with the problem-solving required to analyze and solve patient problems; 2) the development of effective clinical problem-solving, self-directed learning, and team and interpersonal skills; and 3) the development of an insatiable curiosity and a desire to continually learn. 4 Also, according to Susarla et al., the impact of PBL on postdoctoral plans and extracurricular activities, especially research, should be assessed in future studies. 19 Conclusions Based on the findings from the meta-analysis of PBL pedagogy effects, the data suggest statistically significant results based on the theoretical and practical scores, despite the fact that the pass rates did not show any difference between PBL pedagogy and traditional teaching methods. This however needs to take into account the heterogeneity of the studies and the relatively low n of subjects in the selected studies. Most of the questionnaire surveys also showed the positive effects on students critical thinking skills, team spirit, resource researching ability, independent thinking skills, interpersonal skills, ability of oral expression, etc. In these respects, PBL pedagogy is considered superior to the traditional lecturebased teaching. PBL methods could be an optional supplementary method for dental teaching models in mainland China. However, Chinese dental schools should devise PBL curricula according to their own conditions. The effectiveness of PBL should be optimized maximally with all these limitations. 382 Journal of Dental Education Volume 77, Number 3

7 REFERENCES 1. Newman M. A pilot systematic review and meta-analysis on the effectiveness of problem-based learning: special report 2. Middlesex University, School of Lifelong Learning and Education and School of Health and Social Sciences. 2. Huang C, Bian Z, Tai B, Fan M, Kwan CY. Dental education in Wuhan, China: challenges and changes. J Dent Educ 2007;71(2): Dochy F, Segers M, Van der Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instruction 2003;13: Fincham AG, Shuler CF. The changing face of dental education: the impact of PBL. J Dent Educ 2001;65(5): Wu SJ, Miao DM, Zhu X, Luo ZX, Liu XF. Personality types of Chinese dental school applicants. J Dent Educ 2007;71(12): Wang G, Tai BJ, Huang C, Bian Z, Shang ZJ, Wang Q, Song GT. Establishing a multidisciplinary PBL curriculum in the School of Stomatology at Wuhan University. J Dent Educ 2008;72(5): Wang Q, Li ZL, Liu Y. The oral and maxillofacial surgery PBL teaching practice and research. Jilin Med J 2010;31(24): Yang D, Qiu LH, Li ZM, Wang XM, Bao MR. Application and thinking of PBL method in the teaching of endodontics [in Chinese]. China Higher Med Educ 2008;(12): Hu S, Chen MC, Yang J. Tentative application of PBL method in the teaching of dental practice [in Chinese]. Stomatol 2008;28(8): Ji YL, Wang ZG. Application of PBL method in the teaching of pedodontics. Res Med Educ 2010;9(3): Dong H, Chai H. Application of problem-associated teaching method in the teaching of oral medicine [in Chinese]. Sci Technol Inform 2008;19:175, Xu CJ, Wu YF, Peng JY, Que GY, Li HL, Mi DL, et al. Experiment research of application of PBL method in the teaching of seven-year program periodontics [in Chinese]. China Med Engineering 2007;15(1): Lan P. Application of PBL method in the teaching of oral anaesthetics practice [in Chinese]. Chinese Med Modern Distance Educ China 2010;14: Zhou JP, Feng G, Ren YS, Dai HW. Application of problem-based learning (PBL) in x-ray cephalometric experimental teaching. Res Med Educ 2009;8(12): Gu XS, Cui MY, Zeng DL. Application of PBL method in the teaching of dental imaging [in Chinese]. Sun Yatsen Univ Forum 2007;27(10): Shao P, Wang T, An JT, Zhang MM, Zhang H, Zhao HY. Application of problem-based learning in orthodontics for clinical trainees. Northwest Med Educ 2007;15(5): Sun XM, Yang JY. Application of PBL method in the teaching of prosthodontics practice [in Chinese]. J Community Med 2006;4(4): Vernon D, Blake R. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med 1993;68(7): Susarla SM, Medina-Martinez N, Howell TH, Karimbux NY. Problem-based learning: effects on standard outcomes. J Dent Educ 2003;67(9): March 2013 Journal of Dental Education 383