Annual BEAP Report February 24, th Annual BPD Conference Cincinnati, Ohio

Transcription

1 Annual BEAP Report February 24, th Annual BPD Conference Cincinnati, Ohio

2 BEAP Team Vicky Buchan Colorado State University Brian Christenson Lewis-Clark State College nson Ruth Gerritsen-McKane University of Utah Tobi DeLong Hamilton University of Nebraska at Kearney Roy (Butch) Rodenhiser Boise State University edu Marshall Smith Erin Gerdes Graduate Student: Colorado State University

3 To Access BEAP Or Resource Center BEAP Resource Center Accreditation Resources

4 Foundation Curriculum Assessment Instrument

5 Purpose of the FCAI 1. Provides Pre/Post test in seven major curricular areas of a BSW program 2. Provides a direct measure to assist programs with evaluation of their curriculum 3. Assists with identification of curricular areas that may need attention 4. Provides national comparative data

6 Curricular Components Curriculum Area Number of Questions Practice 13 Human Behavior & Social Environment 10 Policy 9 Research 9 Ethics and values 8 Diversity 8 Social and Economic Justice 7

7 Sample HBSE Question The concept person-in-environment includes which of the following: a. Clients are influenced by their environment b. Clients influence their environment c. Behavior is understood in the context of one s environment d. All of the above

8 Sample Practice Question Determining progress toward goal achievement is one facet of the stage. a. Engagement b. Evaluation c. Assessment d. Planning

9 Testing History Version 3 1 Test 65?s n= 305 Version 4 1 test55?s n= 381 Version 5 1 test82?s n= 286 Version 6 1 test74?s n= 36 Version 7 1 test72?s n= 318 Version 8 1 test 64?s n=2231 TOTAL =3557

10 Reliability Testing Version 8 Tested in two junior practice classes Students tested twice, 2 weeks apart Pearson s correlation coefficient r =.86 a

11 Item difficulty index Overall difficulty or average should be around.5 (Cohen & Swerdlik, 2005) FCAI =.523 (n=415) This is a very good difficulty level for the test. Not likely to misrepresent the knowledge level of test takers.

12 Expansion beyond BSW Based upon CSWE assertions related to educational levels in social work education, we expanded testing to three additional groups: MSW foundation students: entering exiting Advanced standing students: entering

13 Mean Scores by Educational Program level Level Version 8 N Mean Score Std Dev Min Max BSW entering BSW exiting MSW F entering MSW F exiting Advanced Standing entering

14 Pre/post test for difference BSW & MSW Foundation Groups by pre/post scores Mean Pre test Score Mean Post test score SD t- valu e Sig. BSW ENTERING/BSW EXITING MSW F ENTERING/MSW F EXITING

15 Tests of Difference by Educational Level Groups by Educational level Mean MSW Exit Score Mean BSW Exit Score SD t-value Sig/Pvalue

16 Tests of Difference by Educational Level Group s by Educ. Level BSW exiting Advanced Standing S D t- value Sig

17 Domain Comparisons- Version 8 Overall BSW Entering BSW Exiting t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

18 Domain Comparisons- Version 8 Overall MSW Entering N MSW Exiting t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

19 Domain Comparisons-Version 8 Overall BSW Exiting Advanced Standing N t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

20 Domain Comparisons- Version 8 Overall BSW Exiting MSW Exiting t-test p-value N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

21 Foundation Curriculum Assessment Instrument (FCAI) Program Name: Students are: Entering BSW Program Exiting BSW Program Month, Year, Semester I. Program Cumulative Scores Compared with all Student Scores N = Score Range SD t-test Value p Program National

22 II. Program: Section Scores Compared with all FCAI Section Curricular Area Practice Program M Program SD National M Program SD t-test value p HBSE Policy Research Ethics & Values Diversity Social & Econ Justice

23 III. Program: BSW Student Scores by Individual Curricular Area Curricular Area Question Suggested Competency Cumulative Correct Mean (% correct for question) Practice Question C Question A Question A, 2.1.9A, A, D

24 Field Practicum/Placement Assessment Instrument (FPPAI)

25 Need for Field Evaluation In response to evaluate social work program outcomes related to the 2008 EPAS competencies and practice behaviors The need for a field/practicum assessment that measures competencies related to practice behaviors

26 FPPAI Piloting Phases Initial Piloting was conducted May 2008 Second Piloting was conducted in Fall 2008 /Spring 2009 Third Piloting phase was conducted Fall 2009 Full implementation: Fall 2010

27 Assessment Methodology 55 Likert Scale questions measuring practice behaviors linked to the EPAS 2008 competencies. Qualitative feedback form for each domain available for program use. Will be available online and in print format. Individual program outcomes report with national comparisons. Individual program outcomes report with national comparisons for EPAS 2008 Competencies & Practice Behaviors Can be used as a final field assessment and midtest/post test design. MSW pilot in the planning stages.

28 FPPAI Scale

29 BEAP & EPAS 2008 Educational Policy Identify as a professional social worker and conduct oneself accordingly Practice Behavior use supervision and consultation Measures BEAP Exit BEAP FPPAI BEAP Employer BEAP Graduate

30 FPPAI Reporting and EPAS 2008 Overview: Click on the screen to see the video.

31 Call for Foundation Year MSW Participation If your program is interested in being a part of the national MSW Foundation Online pilot, please access the FPPAI website: Dr. Brian Christenson: blchristenson@lcsc.edu Dr. Ruth Gerritsen-McKane: Ruth.Gerritsen-McKane@socwk.utah.edu

32

33 What BEAP can do for your program Student demographics Assessment on Knowledge, Skills, & Values Employment information Advising feedback Professional development of students & alumni/ae Peer comparison by region, program type, auspice & nationally and more

34 BEAP Instruments & EPAS Competencies/Practice Behaviors The Matrix

35 BEAP & EPAS Engage in researchinformed practice and practiceinformed research. Graduate Exit Employer FCAI FPPAI A.Use practice experience to inform scientific inquiry C19, 20, 21 D ,36,37,38 B. Use research evidence to inform practice E4 D19 Q14 23,36,37,38

36 BEAP Instruments National Overview of Data 2010

37 Sample vs. Population Program Type Entrance 2008 Entrance 2009 Entrance 2010 Entrance Overall Program Type N % N % N % N % BEAP % BSW Only Combined Programs Total

38 Sample Profile Overview of all Respondents* Overall Respondents Respondents Respondents Respondents Respondents Entrance 28,296 2, ,374 32,481 Exit 16,850 2, ,391 Alumni 4, ,546 Employer 1, ,003 Totals 51,856 4,899 1,528 2,138 60,421 *Note: SWVI no longer reported

39 Sample Profile Response Rates Instrument Average % 2008 Average % 2009 Average % 2010 Average % Overall Average % Entrance Exit Alumni Employer

40 Auspices of College or University (2010 Entrance) Auspice Program Percentage Student Percentage Public (#schools) Private Denominational (#schools) Private Non- Denominational (#schools) 85.2% 9.1% 5.7%

41 Ethnicity at Entrance Ethnicity % N= 28, % N= 2, % N= % N= 1,374 Overall % N= 32,481 European/ Caucasian African/Black Native American Other Other Hispanic Asian/Pacific Islander Chicano/Mexican Puerto Rican Note: Percentages equal more than 100% due to multiple self reported ethnicities.

42 Financial Aid at Entrance Aid Source % N= 28, % N= 2, % N= 524 Overall % N= 31,107 Self Work/Savings Loan* Grant* Family Assistance *Includes Federal and State only.

43 2010 Financial Aid at Entrance Aid Source 2010% N= 1,374 College or University Funds 55.2 Loan* 76.1 Grant* 82.6 Private Organization Funds 30.0 *Includes Federal and State only.

44 Plan to be Employed During BSW Education (Reported at Entrance) % 2008 % 2009 % 2010 % Overall % N= 28,296 N= 2,287 N= 524 N= 1,374 N= 32,481 Plan to be Employed

45 Future Educational Plans-Exit Educational % 2008 % 2009 % 2010 % Overall % Plans N= 16,850 N= 2,157 N= 761 N= 623 N= 20,391 Have Future Educational Plans MSW Other MA Other BA Note: Percentages equal more than 100% due to multiple self reported future educational plans.

46 KSV- Exit Mean Score * Overall Knowledge Skills Values *2010 Likert scale changed from 0-10 to 1-9 (Scale of 0 10)

47 Exit Advising Scores N= 16, N= 2, N= * N= 623 Overall Average N= 20,391 Course selection/ curriculum planning Professional advising Career planning *2010 Likert scale changed from 0-10 to 1-9 (Scale of 0 10)

48 2010 Advising Scored by Auspice- Exit Course Selection and Curriculum Planning Planning for Further Education Professional Advising Private Denominational College or University (# Schools) Private-Non Denominational College or University (# Schools) Public College or University (# Schools) (Scale of 1 9)

49 2010 Advising Scores by Program Type-Exit Course Selection and Curriculum Planning* Planning for Further Education* Professional Advising* BSW Only (# Programs) Combined Programs (# Programs) N= 568 t test *p=.000 (Scale of 1 9)

50 Current Employment Information Primary Function- Exit Primary Function % n= % n= % n= % n=563 Overall % n=5857 Generalist Practice Direct Practice with Individuals, Families, or Groups Training Administration Other Functions n= number reporting

51 Current Employment Information Primary Functions-Exit Primary Function % n=6, % n= 623 Administration/Management Direct Practice with Individuals, Families, or Groups Practice with Communities Practice with Organizations Policy Analysis/Practice Research & Evaluation Supervision Teaching/Training

52 Current Employment Information Primary Role- Exit Primary Function 2010 % n= 623 Case Manager 9.8 Advocate 8.5 Counselor 6.0 Teacher/Educator/Coach 5.6 Enabler/Facilitator 5.3 Outreach 3.7 Coordinator 2.2 Convener/Mediator 2.1 Planner 1.7 n= number reporting

53 Client Systems- Alumni/ae % 2008 % 2009 % 2010 % Overall % N= 4,916 N= 337 N= 192 N= 101 N= 5,546 Individual Clients Family Clients Group Clients Organization Client Community Clients Other Client systems n/a n/a 2.8* * scores

54 Income Level Most Represented by Clients- Alumni/ae Income Level % N= 4, % N= % N= % N= 101 Overall % N= 5,546 Poverty or Below No Typical Income Above Poverty and Below Middle Income Middle Income or Above

55 Professional Development During the Last Year (Reported at Alumni/ae) Overall N= 4,916 N= 337 N= 192 N= 101 N= 5,546 Mean # of times used evaluation of client progress * Mean # of times used program evaluation Mean # of times used other research techniques * Mean # of professional workshops attended Mean # of professional conferences attended * results only

56 Current Employment Information- Field of Practice ( ) Overall Exit n= 7,577 Corrections/Criminal Justice 23.1% Child Welfare/Child Protection 14.6% Youth Services 8.4% Mental/Behavioral/CMH 7.1% Aging and Gerontological Social Work 5.9% Overall Alumni/ae n= 4,083 Child Welfare/Child Protection 19.4% Mental/Behavioral/CMH 15.4% Aging and Gerontological Social Work 10.9% Health/Medical 7.6% Family Service 7.5% n= number of responses

57 Current Employment Information 2010 Exit-Field of Practice 2010 Exit n= 623 Overall Exit ( ) n= 7,577 Crisis Intervention/Information & Referral (5.9%) Mental/Behavioral Health or Community Mental Health (5.3%) Family Service (5.0%) Child Welfare (4.5%) Corrections/Criminal Justice 23.1% Child Welfare/Child Protection 14.6% Youth Services 8.4% Mental/Behavioral/CMH 7.1% Health/Medical Care (3.9%) Aging and Gerontological Social Work 5.9% n= number reporting

58 Current Employment Information 2010 Alumni/ae- Field of Practice 2010 Alumni/ae n= 101 Child Welfare (12.9%) Overall Alumni/ae ( ) n= 4,083 Child Welfare/Child Protection 19.4% Aging/Gerontological Social Work (8.9%) Mental/Behavioral/CMH 15.4% Mental/Behavioral Health (6.9%) Aging and Gerontological Social Work 10.9% Health/Medical Care (5.9%) Health/Medical 7.6% Developmental Disabilities (3.0%) Family Service 7.5% n= number reporting

59 Mean* Knowledge Scores at Exit (2010) BSW only BSW & MSW Combined Test** Public Private- Denominational Private- Non Denominational * Mean score on 9 point scale ** t test, N= 394

60 Mean* Skill Scores at Exit BSW only (2010) BSW & MSW Combined Test** Public Private- Denominational Private- Non Denominational * Mean score on 9 point scale ** t test, N= 398

61 Mean* Value Scores at Exit (2010) BSW only BSW & MSW Combined Test** Public Private- Denominational Private- Non Denominational * Mean score on 9 point scale ** t test, N= 409

62 Region I Southeast* Knowledge, Skills and Value Scores by Deciles *Alabama, Arkansas, Mississippi, Georgia, Florida, South Carolina, Louisiana, Tennessee

63 Region 1 Public BSW Only Programs Percentile Group Knowledge Skills Values Mean Mean Mean

64 Region 1 Public Combined Programs Percentile Group Knowledge Skills Values Mean Mean Mean

65 Region 1 Private Denominational BSW Only Knowledge Programs Skills Values Percentile Group Mean Mean Mean

66 Region 1 Private Non-Denominational BSW Only Programs Percentile Group Knowledge Skills Values Mean Mean Mean

67 Region VI Northeast* Knowledge, Skills and Value Scores by Deciles *New York, New Jersey, Connecticut, Rhode Island, Massachusetts, New Hampshire, Maine

68 Region 6 Public BSW Only Programs Percentile Group Knowledge Skills Values Mean Mean Mean

69 Region 6 Public Combined Programs Percentile Group Knowledge Skills Values Mean Mean Mean

70 Region 6 Private Denominational BSW Only Programs Percentile Group Knowledge Skills Values Mean Mean Mean

71 Region 6 Private Non Denominational BSW Only Programs Percentile Group Knowledge Skills Values Mean Mean Mean

72 2009/2010 Version 8 Participating Universities, Entering BSW Delta State University Dominican College Ferris State University Fort Hays State University Jacksonville State University LaSalle University Lewis and Clark State College Lock Haven University Luther College Middle Tennessee State University Mississippi State University Mount Mercy College Colorado State University Anderson University

73 Participating Universities, Entering BSW Auburn University Azusa Pacific University Bridgewater State College California State University- Long Beach Cedar Crest College Philadelphia Biblical University Plymouth State University Ramapo College Sacred Heart University Saint Mary s College Salem State College Slippery Rock University University of Nebraska-Kearney University of Northern Alabama University of Texas- Arlington University of Wisconsin- Oshkosh Weber State University

74 Participating Universities, Entering BSW Cedarville University Cleveland State University University of Central Missouri Wichita State University

75 Mean Raw Scores by Grade Level Grade Level BSW Entering BSW Exiting N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

76 Mean Raw Scores by Grade Level Grade Level MSW Entering MSW Exiting Advanced Standing N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

77 Domain Comparisons-2010 BSW Entering BSW Exiting t-test p-value N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

78 Domain Comparisons-2010 MSW Entering MSW Exiting t-test p-value N Total Score (64) Practice (13) Domain Comparisons HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

79 Domain Comparisons-2010 BSW Exiting MSW Exiting t-test p-value N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

80 Domain Comparisons-2010 BSW Exiting Advanced Standing t-test p-value N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

81 Mean Raw Scores by Grade- Overall Grade Level BSW Entering BSW Exiting N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

82 Mean Raw Scores by Grade- Overall Grade Level MSW Entering MSW Exiting Advanced Standing N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

83 Domain Comparisons- Version 8 BSW Entering Overall BSW Exiting t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

84 Domain Comparisons- Version 8 MSW Entering Overall N MSW Exiting t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

85 Domain Comparisons- Version 8 Overall BSW Exiting MSW Exiting t-test p-value N Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)

86 Domain Comparisons-Version 8 BSW Exiting Overall Advanced Standing N t-test p-value Total Score (64) Practice (13) HBSE (10) Policy (9) Research (9) Ethics (8) Diversity (8) Social & Econ Justice (7)