The Development of Assessment Tools for CRM210 Final Report Rationale for requesting the Assessment Grant: The department of Criminology and Criminal Justice s Introduction to the American Criminal Justice (CRM210) course was accepted in the General Education curriculum in the spring 2017 semester. Before the start of the fall 2017 semester, the department needed to develop a pretest/posttest, a rubric for the journal assignments, and a self-report survey. The creation of these instruments was needed in order to assess the student learning outcomes associated with General Goals 1, 2, and 12. Using the funds associated with the assessment grant, a one day retreat was held to discuss the creation of these assessment instruments. Meeting Attendees: Ethan Amidon, Julie Baldwin, John Eassey, Brett Garland, Diane Leamy, Caryn Saxon, and Ivy Yarckow-Brown Meeting Date: 7/5/2017 Meeting Minutes: The meeting began with a discussion of the topics that should be covered in the pretest/posttest. Based on the original General Education proposal, the department indicated that the pretest/posttest would be used to assess student learning outcomes (SLO) 1.1, 1.2, 1.4, 2.2, 12.1, and 12.2. In order to identify which subjects to cover in this assessment instrument, we created a list of all of the subjects that were contained within the descriptions associated with the previously mentioned SLOs. After all of the subjects were outlined, we identified a number of topics within each subject to potentially examine in the pretest/posttest. During this discussion, we reevaluated the use of the pretest/posttest to assess SLO 2.2 and 12.2, and we concluded that the assessment of these two SLOs could be achieved solely through the use of journal assignments. Appendix A contains the list of the subjects within the remaining SLO descriptions and the list of the topics that were identified within each of these subjects. The next topic of discussion was the use of journal assignments to assess the SLOs identified in the original General Education proposal (i.e., SLO 2.1, 2.2, and 12.4). During this discussion, it was determined that we could strengthen our assessment of the subjects outlined in our SLO descriptions by expanding the number of SLOs covered by the journal assignments. Overall, the group decided to expand the use of the journal assignments to cover SLO 1.1, 1.2, and 1.4 as well. Appendix B contains the list of journal assignment criteria that were established based on the descriptions for SLO 1.1, 1.2, 1.4, 2.1, 2.2 and 12.4. Finally, we discussed the use of a self-report survey to assess SLO 1.1, 1.2, 1.4, 2.1, and 12.1, which were identified in the original General Education proposal. However, based on the expanded reliance on journal assignments to assess the SLOs, the group questioned whether the use of a self-report survey in the course would be necessary. Based on this discussion, the group decided that the combination of the pretest/posttest and the journal assignments would allow us to fully assess all of the SLOs. Therefore, we voted to eliminate the use of a self-report survey in the course. 1
Post Meeting Actions: Following the meeting, the group created a number of pretest/posttest questions that were designed to capture the subjects and the topics identified in Appendix A. This process initially yielded a 60 question pretest/posttest. After assessing the length of the initial instrument, it was decided that it would be best to reduce the number of questions in the pretest/posttest. We hoped that by reducing the number of questions in this instrument that we could potentially encourage students to take the completion of the entire pretest/posttest seriously. However, with the potential reduction in the number of questions, we realized that we would not be able to address all of the topics that were identified for each subject in Appendix A. After weighing the pros and cons associated with reducing the number of questions, we believed that potentially enhancing the quality of the data gathered was more important than covering every topic we had identified during our meeting. The process of reducing the number of questions in the pretest/posttest resulted in a 30 question version of the instrument, which is contained in Appendix C. We have included identifiers in Appendix A and C that indicate how the questions contained in the pretest/posttest align with the subjects and topics that were discussed during the meeting. The final version of the pretest/posttest was approved during a departmental meeting on August 17 th, and this instrument was deployed in all of the CRM210 sections at the beginning of the Fall 2017 semester. The journal criteria are also being used by all instructors to assess the required journal assignments. 2
Appendix A: Pretest/Posttest SLO Subjects & Topics SLO 1.1: Identify and follow through on personally and socially relevant problems and reasonable solutions to those problems. Description: Students often harbor preconceived notions about criminal offenders and the justice system, which are often based on false or misleading information disseminated through popular culture. During the course students will receive extensive exposure to information on offenders, victims, and criminal justice policies and practices. Particular emphasis will be placed on pressing problems within the field, such as the implications of mass incarceration and reentry, police use of force, plea bargaining, crime rate trajectories, offender recidivism, and victim roles in the justice process. Students will be exposed to and reflect upon the crime problem and its impact on society and current and past strategies for addressing critical issues in criminology and criminal justice. Potential Subjects & Topics: Implications of mass incarceration (1.1A) o Citizen safety o Use of private prisons o Costs and quality Prisoner reentry (1.1B) o Collateral consequences o Different forms of reentry o Barriers to prisoner reentry Police use of force (1.1C) o Use of force continuum o Qualified immunity Plea bargaining (1.1D) o The degree to which plea bargaining is used o The role of attorneys o The use of coercion in the process Crime rate trajectories (1.1E) o Recent changes in crime rate trends Victim roles in the justice process (1.1F) o Presentence investigation report o Pressing charges o Restorative justice 3
SLO 1.2: Identify relevant information sources, make reasoned choices among those sources, and open-mindedly follow where those sources lead. Description: Students will explore a wide variety of methods involved with collecting and disseminating information on criminal activity, victimization, and criminal justice processes. The strengths and limitations associated with official records, victimization studies, and self-report data will be examined. Potential Subjects & Topics: Methods involved with the collection and dissemination of statistics on criminal activity, victimization studies, and self-report data (1.2) o UCR o NCVS o Dark figure of crime o Media and public representation of crime problem Strengths and limitations of these data sources (1.2) SLO 1.4: Analyze evidence, statements, alternative viewpoints, graphics, and other forms of information. Description: Students will be challenged to consider and analyze their beliefs and perspectives on the factors that influence criminal activity and how the criminal justice system operates. As part of the course, students will analyze competing explanations behind the purposes of criminal law, philosophical justifications underlying offender punishment and reform, and interpretations of criminal justice organizational goals, roles, and functions. Students will examine the strengths and weaknesses of differing viewpoints on criminal justice issues. Potential Subjects & Topics: Examine explanations behind the purposes of criminal law, philosophical justifications underlying offender punishment and reform (1.4A) o Rehabilitation o Deterrence o Incapacitation o Retribution o Utilitarian o Conflict Interpretations of criminal justice organizational goals, roles, and functions (1.4B) o Role of prosecutor and defense attorney o State, federal, and local o FBI o Diversion o Technical violations o Revocation o Intermediate sanctions 4
SLO 12.1: Identify the rights and responsibilities they have in their own communities and the broader society. Description: The course will inform students about citizen rights in relation to criminal justice processes. Students will also learn about discrimination issues in policing, the courts, and correctional systems. Exploring these issues will help students identify their personal and social responsibilities for ensuring equality and fairness in the distribution of justice. Potential Subjects & Topics: Provide information on citizen s rights (12.1A) o Amendments o Constitutional rights o Responsibilities Explore discrimination issues in policing, the courts, and correctional systems (12.1B) o Race/ethnicity o Gender o Religion o Sentencing disparities o Racial profiling 5
Appendix B: Journal Assessment Criteria Criterion 1: Identify and explain the nature of critical problems facing the modern criminal justice system (SLO: 1.1) Criterion 2: Explore merits and shortcomings of contemporary solutions to pressing criminal justice issues (SLO: 1.1; 1.2; 1.4; 2.2) Criterion 3: Develop recommendations designed to address crime-related problems and how to improve criminal justice policy and practice (SLO: 1.1; 2.1) Criterion 4: Explore how social factors can impair communities and foster criminal activity (SLO: 1.2; 12.4) Criterion 5: Examine how communities can be strengthened to aid crime prevention (SLO: 12.4) Relevant SLOs: SLO 1.1: Identify and follow through on personally and socially relevant problems and reasonable solutions to those problems. SLO 1.2: Identify relevant information sources, make reasoned choices among those sources, and open-mindedly follow where those sources lead. SLO 1.4: Analyze evidence, statements, alternative viewpoints, graphics, and other forms of information. SLO 2.1: Develop creative and novel solutions to personally and socially relevant problems. SLO 2.2: Take account of novel, alternative, contradictory, and even radical viewpoints in creating new ideas, products, or solutions appropriate to the domain or subject matter. SLO 12.4: Recognize the needs of the communities to which they belong and understand how to address those needs. All of the journal assessment criteria will be evaluated in terms of whether the students Exceed expectations, Meet expectations, or Fails to meet expectations. 6
Appendix C: Final Pretest/Posttest SLO 1.1: Identify and follow through on personally and socially relevant problems and reasonable solutions to those problems. 1. Which of the following was the most significant contributor to the growth in prison and jail populations in the 1980s and 1990s? (1.1A) A. War on drugs B. War on poverty C. Computer crimes D. Terrorism 2. Any activity or program dedicated to preparing and integrating parolees into the community as law-abiding citizens using a collaborative approach with parole officers, treatment providers, and the community is referred to as. (1.1B) A. Surveillance B. Prisoner reentry C. Supervision D. Intensive-supervision probation 3. What pattern does research reveal regarding officers involvement in a use-of-force incident? (1.1C) A. Fewer than two percent of police-citizen contacts include the use of force by the police B. A small portion of officers accounts for a sizable portion of the abuses C. Problem officers tend to be young and inexperienced members of the force D. All of the above are correct 4. Approximately what percentage of criminal cases goes to trial? (1.1D) A. Less than 10% B. 25% C. 50% D. 75% 5. From the mid-1990s to the present, violent crime rates in the United States have primarily: (1.1E) A. Increased B. Decreased C. Remained stable D. Fluctuated 7
6. Victim services are dedicated activities conducted to help: (1.1F) A. Facilitate victim recovery B. Ensure offender prosecution C. Force victim cooperation with prosecution D. Determine which victims are eligible for restitution SLO 1.2: Identify relevant information sources, make reasoned choices among those sources, and open-mindedly follow where those sources lead. 7. What is considered to be the most widely cited source of criminal statistics? (1.2) A. Uniform Crime Report B. National Crime Victimization Surveys C. National Indicators of Drug Abuse D. Bureau of Justice Statistics 8. Which of the following is one of the problems associated with the validity of the Uniform Crime Report? (1.2) A. No federal crimes are reported B. If an offender commits multiple crimes, only the most serious crime is recorded C. Only a small percentage of jurisdictions report their crimes to the FBI D. Crime rates are computed per 50,000 individuals 9. Which of the following would not be considered a violent crime? (1.2) A. Robbery B. Prostitution C. Rape D. Homicide SLO 1.4: Analyze evidence, statements, alternative viewpoints, graphics, and other forms of information 10. According to this justice perspective, the justice system serves as a mechanism of caring for and treating people. It assumes people are at the mercy of social, economic, and interpersonal conditions. (1.4A) A. Crime control B. Rehabilitation C. Due process D. Equal justice 8
11. Deterrence theory has played a significant role in the development of societal law and the purposes behind the use of penal punishment. What is the main proposition within deterrence theory? (1.4A) A. Humans are rational actors who determine their course of action by weighing the expected pleasures and pains associated with the action B. Human actions are driven primarily by their biological makeup C. Human actions are shaped primarily by their environment D. Human actions are shaped by both their biological makeup and their environment 12. What is the primary purpose behind the use of incapacitation? (1.4A) A. To punish offenders for their crimes against society B. To protect society through the long-term incarceration of dangerous offenders C. To rehabilitate offenders through intensive institutional treatment D. To provide substance abuse offenders with therapeutic treatments while incarcerated 13. What is the duty of a defense lawyer to their client and legal system? (1.4B) A. To participate in the courtroom workgroup B. To fully and competently defend their client C. To engage in the pursuit of justice D. To represent the state 14. Which of the following is an advantage of diversion programs? (1.4B) A. It allows the offender to postpone prison/jail time until diversion completion B. Rehabilitation services can be accessed while in the community C. The cost is higher but so is the success rate D. Diversion allows the inmate out of their cell for longer lengths of time 15. Community oriented policing links police effectiveness to: (1.4B) A. Productive interaction with the community being served B. Interrelationships between cooperative police agencies C. Efficient utilization of existing personnel D. Optimized usage of advanced technology 16. The contemporary criminal justice system can be divided into three main components: (1.4B) A. Law enforcement, the courts, and the legislature B. The courts, the correctional system, and the legislature C. Law enforcement, the courts, and the correctional system D. Law enforcement, the correctional system, and the legislature 9
17. Which of the following best describes the federal court system? (1.4B) A. There are three tiers: district courts, federal appellate courts and the United States Supreme Court B. There are four tiers: trial courts, district courts, appellate courts and the United States Supreme Court C. There are two tiers: district court and the United States Supreme Court D. There is one tier: the United States Supreme Court that decides all important cases 18. What is the function of a Grand Jury? (1.4B) A. Grand Juries hear evidence and determine whether a defendant is guilty beyond a reasonable doubt B. Grand Juries hear evidence and determine whether a defendant is guilty by a preponderance of the evidence C. Grand juries function in appellate courts and determine whether a previous verdict/decision should be affirmed or reversed D. Grand juries function to determine if there is enough evidence against a particular defendant such that the criminal process should continue 19. The process of determining the appropriateness of jurors to sit on the jury is known as: (1.4B) A. Voir dire B. Venire C. The jury array D. The process of rebuttal 20. The U.S. Supreme Court is comprised of members that are. (1.4B) A. 8; appointed for 9 year terms B. 9; appointed for lifetime terms C. 9; elected by the voters during the presidential election D. 10; recommended by congress 21. Under this sentencing model, parole boards determine when offenders are rehabilitated and ready for release on parole. (1.4B) A. Mandatory sentencing B. Indeterminate sentencing C. Presumptive sentencing D. Determinate sentencing 10
22. In the 1970s, there was the movement away from individualism, rehabilitation, and sentence indeterminacy towards giving the offender a sentence based on the offense. This type of sentencing practice is: (1.4B) A. Indeterminate sentencing B. Determinate sentencing C. Split sentencing D. Rehabilitative sentencing 23. The is conducted primarily to gain information for judicial sentences. (1.4B) A. Intake B. Presentence Investigation Report C. Diagnosis D. Adjudication 24. During the classification process, what procedure provides a measure of both the offender s degree of dangerousness to the public and the offender s propensity to engage in future criminal activity? (1.4B) A. A review of the conditions B. Risk assessment C. Needs analysis D. Principles of effective treatment SLO 12.1: Identify the rights and responsibilities they have in their own communities and the broader society. 25. Which amendment refers to freedom of religion and speech and often requires police intervention to protect protesters or those protested against? (12.1A) A. 1 st B. 2 nd C. 3 rd D. 4 th 26. The Amendment guarantees the defendant the right to a jury trial. (12.1A) A. 4 th B. 5 th C. 6 th D. 14 th 11
27. The Amendment to the Constitution prohibits excessive bail. (12.1A) A. 6 th B. 8 th C. 14 th D. 16 th 28. Which of the following groups has the highest incarceration rate in the U.S.? (12.1B) A. African Americans B. Native Americans C. Hispanics D. Caucasians 29. Many studies have concluded that drivers are disproportionately stopped by the police. (12.1B) A. Women B. Elderly C. Caucasian D. African American 30. Which of the following is most likely to affect a community s relationship with police departments? (12.1B) A. Forensic science B. Crime mapping C. Racial profiling D. Crime rate 12