KIRKPATRICK TRAINING EVALUATION DOESN T HAVE TO BE AS FORMAL AS YOU THINK

KIRKPATRICK TRAINING EVALUATION DOESN T HAVE TO BE AS FORMAL AS YOU THINK Wendy Kirkpatrick and Jim Kirkpatrick, PhD Thursday, January 11, 2018 Training Magazine Network

Trademark and Copyright Warning Copyright 2010-2018, Kirkpatrick Partners, LLC. All rights reserved. This presentation remains the exclusive property of Kirkpatrick Partners, LLC. Distribution or duplication of these slides and content, in whole or in part, in any way is strictly prohibited. These are for the personal use of program attendees only. The following marks are the property of Kirkpatrick Partners, LLC as related to educational / training materials, programs and books: Kirkpatrick i k BrightLight Kirkpatrick Four Levels Blended Evaluation Plan The One and Only Kirkpatrick Kirkpatrick Training Evaluation Doesn t Have to Be as Formal as You Think Wendy Kirkpatrick and Jim Kirkpatrick Training Magazine Network January 11, 2018 1

Keys to Powerful Yet Simple Training Evaluation 1. Build in training evaluation at the beginning 2. Use simple, non-technical evaluation methods 3. Focus your efforts on Kirkpatrick Levels 3 and 4 The Kirkpatrick Model Level 4: Results The degree to which targeted outcomes occur as a result of the training and the support and accountability package 4

The Kirkpatrick Model Level 3: Behavior The degree to which participants apply what they learned during training when they are back on the job The Kirkpatrick Model Level 2: Learning The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training 5

The Kirkpatrick Model Level 1: Reaction The degree to which participants find the training favorable, engaging and relevant to their jobs Participant Poll What is the highest level to which you have ever evaluated any program? A. Level 1 Reaction B. Level 2 Learning C. Level 3 Behavior D. Level 4 Results Feel free to add a comment in the chat area. 6

Why Evaluate? Improve the Program Improve Job Performance Maximize Organizational Results Effective Training Training Effectiveness BUILD IN TRAINING EVALUATION AT THE BEGINNING 7

Four Levels Application Guidelines Determine first what information you need Usefulness Information used to make decisions related to the program and progress afterwards (effective training) Credibility Information showing your stakeholders that performance improved and organizational results were positively impacted (training effectiveness) 2010-2018 Kirkpatrick Partners. All rights reserved. Level 1: Reaction Level 4: Results Level 2: Learning Level 3: Behavior Level 3: Behavior Level 4: Results Level 2: Learning Level 1: Reaction 8

Expected Outcomes Level of Effort Critical Behaviors Coordinated Approach Learning Participant Interaction In the chat below, share at least one way that t you will build in evaluation at the beginning of your programs (or go back and add it now for existing programs). 9

Participant Interaction If you have any questions as this point, please ask them in the chat area. We will respond to as many as we can in the time available. USE SIMPLE, NON-TECHNICAL EVALUATION METHODS 10

Blended Evaluation Plan A methodology in which data are collected from multiple sources using multiple methods, in a blended fashion that considers all four Kirkpatrick levels. 11

Formative Evaluation The collection of data or information during a learning event or phase 2010-2015 Kirkpatrick Partners, LLC. All rights reserved. Level 1: Reaction Formative Methods Instructor observation Pulse check Dedicated observer 2010-2018 Kirkpatrick Partners. All rights reserved. 13

Level 2: Learning Formative Methods Knowledge test/check Discussion Individual / group activity Role play 2010-2018 Kirkpatrick Partners. All rights reserved. Participant Interaction In the chat below, share at least one formative evaluation method you use today, or plan to use in the future. 14

Brinkerhoff Study Training Application Group 1 - Telling Training s Story, Rob Brinkerhoff, 2006 2010-2018 Kirkpatrick Partners. All rights reserved. Brinkerhoff Study Resources Employed Group 2 - Telling Training s Story, Rob Brinkerhoff, 2006 2010-2018 Kirkpatrick Partners. All rights reserved. 16

Brinkerhoff Study Training Application Group 2 - Telling Training s Story, Rob Brinkerhoff, 2006 2010-2018 Kirkpatrick Partners. All rights reserved. 2010-2015 Kirkpatrick Partners, LLC. All rights reserved. 17

Required Drivers Processes and systems that reinforce, monitor, encourage and reward performance of critical behaviors on the job Level 3: Behavior Methods, Tools and Techniques (Required Drivers) Reinforce Follow-up modules Work review checklist On-the-job training (OJT) Self-directed learning Refreshers Job aids Reminders Executive modeling SUPPORT Encourage Coaching Mentoring ACCOUNTABILITY Monitor Action learning Interviews Observations Self-monitoring KPI s (key performance indicators) Reward Recognition Pay for performance 2010-2018 Kirkpatrick Partners. All rights reserved. Action plan monitoring Dashboard Work review Survey Touchbases / meetings 18

Participant Interaction In the chat area, please pose any questions you have at this time. We will answer as many as time allows. FREE Resources Register for free at kirkpatrickpartners.com to receive: Instant access to Resource Library with 100+ items Subscription to weekly e-newsletter You will also receive a follow-up email next week if you registered prior to the live event 19

More In-Depth Information Kirkpatrick Four Levels Evaluation Certification Program Build an effective program implementation and evaluation plan Upcoming programs: Online begins on February 7 (Conducted d live) March 14-15 in Dulles, VA March 21-22 in Atlanta, GA Kirkpatrickpartners.com Training & Events tab More In-Depth Information Kirkpatrick s Four Levels of Training Evaluation Now available! Expert guidance in how to apply the model Templates and tools Case examples tinyurl.com/k4lte-book 20

Participant Interaction I have at least one idea I can implement in my work. A. Yes B. No I would recommend this program to others in this field. A. Yes B. No In chat or the Insights widget, please list one specific action you plan to take as a result of your participation in this program. 21

Participant Interaction Would you like to talk with us? Do you have more in-depth questions? Contact us at inquiries@kirkpatrickpartners.com or (770) 302-3500 Thank you! Customer service: information@kirkpatrickpartners.com Sales: inquiries@kirkpatrickpartners.com (770) 302-3500 kirkpatrickpartners.com 22

KIRKPATRICK BLENDED EVALUATION PLAN FORM SAMPLES You may pull items from the following sample evaluation forms to build an evaluation form for your own program. Participant Survey Instructions: Thinking about the course you just completed, please indicate to what degree you agree with each statement using this rating scale: 0 1 2 3 4 5 6 7 8 9 10 Strongly Disagree Strongly Agree Please provide comments along with your rating to help us to improve this course in the future. Strongly Disagree Strongly Agree The class environment helped me to learn. 0 1 2 3 4 5 6 7 8 9 10 I was engaged with what was going on during the program. 0 1 2 3 4 5 6 7 8 9 10 The activities and exercises aided in my learning. 0 1 2 3 4 5 6 7 8 9 10 I was given adequate opportunity to practice what I was learning. 0 1 2 3 4 5 6 7 8 9 10 I will be able to immediately use what I learned. 0 1 2 3 4 5 6 7 8 9 10 The program material will contribute to my future success. 0 1 2 3 4 5 6 7 8 9 10 I would recommend this program to my co-workers. 0 1 2 3 4 5 6 7 8 9 10 Comments: From what you learned, what will you be able to apply on your job?

What assistance or resources will you need to successfully apply what you learned on the job? How confident are you that you will be able to apply what you have learned back on the job? (Circle one rating) Not at all confident 0..1..2..3..4..5..6..7..8..9..10 Extremely confident Comments: How committed are you to applying what you learned to your work? (Circle one rating) 0..1..2..3..4..5..6..7..8..9..10 Not at all committed Extremely committed Comments: What outcomes are you hoping to achieve as a result of your efforts? What other feedback would you like to share?

Delayed post-training survey for (provide course name) Instructions: Thinking about the course you completed 3 months ago, please indicate to what degree you agree with each statement using this rating scale: 1 = Strongly Disagree 2 = Disagree 3 = Agree 4 = Strongly Agree N/A = Not Applicable Please use Comments to provide a brief explanation or further feedback. The course itself: 1. I was clear about the purpose of the course before I attended 1 2 3 4 N/A 2. I was clear about what was expected of me after taking the course 1 2 3 4 N/A Comments: Practical application 3. I am successfully applying what I learned in the course 1 2 3 4 N/A 4. If you answered Agree or Strongly Agree to Question 3, what are the most significant reasons? (check all that apply) Comments: My past experience The course itself Extra help from course instructors Help from my co-workers Help from my immediate supervisor A good system of accountability Formal or informal recognition for my efforts My own efforts and discipline to apply what I learned Referring back to the course materials Additional training

5. If you answered Disagree or Strongly Disagree to Question 4, what are the main reasons? (check all that are true) Comments: What I learned is not useful for my job I have been told not to use it I don t remember what I learned I have too many other things to do I got stuck and did not know how to find help It is too difficult to apply I have not been encouraged to apply it There are no incentives for me to apply it Overall 6. The course was a worthwhile use of my time 1 2 3 4 N/A 7. I am already seeing positive results from this course 1 2 3 4 N/A 8. I am expecting positive results from this course in the future 1 2 3 4 N/A Comments: 9. What suggestions do you have that would make you better able to apply what you learned? 10. Please provide a specific example of how the course has helped you achieve positive results in your area. 4

We would like to conduct a short interview with a number of you to get more details. If you are willing to help us, please provide your email address so we can contact you to make convenient arrangements. Thanks. Name: Email:

This type of form can be used at the end of all but the last day of a multi-day program in order to identify any issues that require attention or correction on the remaining days of the program, or in subsequent program updates. Kirkpatrick Day One Evaluation Instructions: For questions 1 3, please use the following rating scale: 0 = strongly disagree 10 = strongly agree Please circle the appropriate rating to indicate the degree to which you agree with each statement. Please provide comments to explain your ratings. If your session had two facilitators, please fill in the key below and score each individually in question 3. Facilitator A: Facilitator B: Date and Location: strongly disagree Rating strongly agree 0 1 2 3 4 5 6 7 8 9 10 1.) I took responsibility for being involved in today s session. Comments: 0 1 2 3 4 5 6 7 8 9 10 2.) The information in today s session is applicable to my work. Comments: A: 0 1 2 3 4 5 6 7 8 9 10 B: 0 1 2 3 4 5 6 7 8 9 10 3.) The presentation style of the facilitator contributed to my learning experience. Comments: 4.) Please provide any suggestions for change / improvement you may have for tomorrow and for future sessions of this program.

Supervisors can use this type of checklist while listening to customer service calls in order to rate the performance of customer service representatives (CSRs) after they have completed training. Rating Scale 1 = Effective use of targeted behavior 2 = Moderately effective use of targeted behavior 3 = Ineffective use of targeted behavior Coaching comments may include specific observations that support the rating, and coaching notes to help the associate to be more effective. Target Behavior Rating Coaching Comments CSR made good initial connection with the callers needs to create a rational exchange. CSR used relevant, open-ended questions to gather initial information about the callers needs. CSR asked follow-up questions to gain further clarification of the callers needs. CSR presented relevant possible solutions to the callers concerns. CSR offered ongoing support to address the callers needs while considering the best interests of the company. CSR closed the meeting with a clear course of problem resolution.

Dr. Jim Kirkpatrick is the Senior Consultant for Kirkpatrick Partners. He is a thought leader in training evaluation and the creator of the New World Kirkpatrick Model. Using his 15 years of corporate experience, Jim trains and consults for organizations around the world. He is passionate about assisting learning professionals in redefining themselves as strategic business partners. Jim delivers lively keynote addresses and conducts workshops on topics including maximizing business results, creating powerful training and evaluation strategies, building and leveraging business partnerships and increasing the transfer of learning to on-the-job behaviors. Jim co-authored three books with his father, Don Kirkpatrick, the creator of the Kirkpatrick Model, and four books with Wendy Kirkpatrick, including Kirkpatrick s Four Levels of Training Evaluation. Jim and Wendy also served as the subject matter experts for the United States Office of Personnel Management s Training Evaluation Field Guide. Wendy Kirkpatrick is the President and Founder of Kirkpatrick Partners. As a global driving force of the use and implementation of the Kirkpatrick Model, Wendy leads companies to measurable success through training and evaluation. Wendy s results orientation stems from over two decades of business experience in retailing, marketing and training. She has held positions as a buyer, product manager, process manager and training manager, which leveraged her ability to organize multifaceted projects and yield rapid results. Wendy and Jim Kirkpatrick have written four books, including Kirkpatrick s Four Levels of Training Evaluation. They also served as the subject matter experts for the United States Office of Personnel Management s Training Evaluation Field Guide. Wendy is a recipient of the 2013 Emerging Training Leaders Award from Training Magazine. About Kirkpatrick Partners Kirkpatrick Partners is proud to be the One and Only Kirkpatrick company, and the only provider of authentic Kirkpatrick products and programs. The company carries on the work of Don Kirkpatrick, Ph.D., and the Kirkpatrick Model of training evaluation, also referred to as the four levels of evaluation. Kirkpatrick Partners offers training, consulting, impact studies, books and other written resources on the Kirkpatrick Model and related concepts. Read more about Kirkpatrick Partners at kirkpatrickpartners.com. Kirkpatrick Partners (770) 302-3500 information@kirkpatrickpartners.com kirkpatrickpartners.com

MAXIMIZING RESULTS WITH KIRKPATRICK Early part of the journey - Ensuring effective training Register for FREE resources at kirkpatrickpartners.com Watch Kirkpatrick webinars Discuss effective training versus training effectiveness in your organization Participate in Getting to Kirkpatrick Levels 3 & 4 or Igniting the Inner Fire workshops Join the Kirkpatrick Evaluation discussion group in LinkedIn Participate in the Kirkpatrick Four Levels Evaluation Certification Program - Bronze Level Farther down the road - Building training effectiveness Connect Levels 3 and 4 through required drivers, critical behaviors and leading indicators Schedule a Kirkpatrick Strategic Evaluation Planning Certificate Program Implement a systematic evaluation decision-making process Obtain Kirkpatrick silver level certification Nearing the final destination - Maximizing organizational ROE (return on expectations) Consider a Kirkpatrick Business Partnership Analysis Schedule a Kirkpatrick Impact Study Present your compelling chain of evidence to your corporate jury Obtain Kirkpatrick gold level certification 04/2017 (770) 302-3500 information@kirkpatrickpartners.com www.kirkpatrickpartners.com

FREE RESOURCES & INFORMATION JOIN THE KIRKPATRICK COMMUNITY Kirkpatrick Partners strives to provide the information you need, when and how you want it. REGISTER WITH KIRKPATRICK PARTNERS Register online at www.kirkpatrickpartners.com to receive access to the online Kirkpatrick Resource Library and be automatically enrolled in our weekly newsletter. Receive all of the following: Access to online Kirkpatrick Resource Library containing 100+ free items on demand Articles, white papers, videos, podcasts, PowerPoint slides, diagrams and tools One efficient weekly email containing: Weekly Kirkpatrick Quick Tip Latest articles News, specials and event schedules Ability to comment on features in The Official Kirkpatrick Blog SELECT YOUR FAVORITE SOCIAL MEDIA FOR TIMELY UPDATES Do you prefer to get your news as it happens? Do you like your Facebook page better than your Outlook inbox? If so, this step is for you! Follow Kirkpatrick Partners on your favorite social media network. Join the discussion and receive the latest news by liking us on Facebook, following our Twitter account, @TheKirkpatricks, connecting with us on LinkedIn, following our boards on Pinterest, or subscribing to The Official Kirkpatrick Blog on our website. ADD CUSTOMIZED SUBSCRIPTIONS FOR MORE INFORMATION Do you want everything from Kirkpatrick Partners? If so, follow Jim, @Jim_Kirkpatrick, and Wendy, @WkKirkpatrick, on Twitter, join the Kirkpatrick Evaluation Discussion Group on LinkedIn and subscribe to the Kirkpatrick Partners YouTube channel. (770) 302-3500 information@kirkpatrickpartners.com www.kirkpatrickpartners.com