Getting Kirkpatrick Evaluation Right Case Examples and Tips from Practitioners

Transcription

1 Trademark and Copyright Warning Copyright , Kirkpatrick Partners, LLC. All rights reserved. This presentation remains the exclusive property of Kirkpatrick Partners, LLC. Distribution or duplication of these slides and content, in whole or in part, in any way is strictly prohibited. These are for the personal use of program attendees only. The following marks are the property of Kirkpatrick Partners, LLC as related to educational / training materials, programs and books: Kirkpatrick Kirkpatrick Four Levels The One and Only Kirkpatrick Bright Light Blended Evaluation Plan Getting Kirkpatrick Evaluation Right Case Examples and Tips from Practitioners Dr. Jim Kirkpatrick and Wendy Kirkpatrick Guest presenters October 12, 2016 Copyright , Kirkpatrick Partners, LLC. All rights reserved. 1

2 Participant Interaction In the general chat window, please introduce yourself with your name, location and a brief statement of what you do. Dr. Don Kirkpatrick Copyright , Kirkpatrick Partners, LLC. All rights reserved. 2

3 The Kirkpatrick Model Level 4: Results To what degree targeted outcomes occur as a result of the learning event(s) and subsequent reinforcement The Kirkpatrick Model Level 3: Behavior To what degree participants apply what they learned during training when they are back on the job Copyright , Kirkpatrick Partners, LLC. All rights reserved. 3

4 The Kirkpatrick Model Level 2: Learning To what degree participants acquire the intended knowledge, skills and attitudes based on their participation in the learning event The Kirkpatrick Model Level 1: Reaction To what degree participants react favorably to the learning event Copyright , Kirkpatrick Partners, LLC. All rights reserved. 4

5 Participant Poll What is the highest level to which you have ever evaluated ANY program? A. Level 1 Reaction B. Level 2 Learning C. Level 3 Behavior D. Level 4 Results Feel free to add a comment in the chat area Kirkpatrick Partners, LLC. All rights reserved. Why Evaluate? Improve the Program Improve Job Performance Maximize Organizational Results Effective Training Training Effectiveness Demonstrate relative contribution of each success factor Copyright , Kirkpatrick Partners, LLC. All rights reserved. 5

6 Level 1: Reaction Level 2: Learning Level 3: Behavior Level 4: Results Level 4: Results Level 3: Behavior Level 2: Learning Level 1: Reaction Copyright , Kirkpatrick Partners, LLC. All rights reserved. 6

7 Training Business Copyright , Kirkpatrick Partners, LLC. All rights reserved. 7

8 Jim and Wendy Kirkpatrick Cynthia DeWitte FBI Gerald Jones Booz Allen Hamilton Linda Hainlen Sedona Learning Solutions Joyce Donohoe Emirates Airlines A Blended Evaluation Approach Cynthia DeWitte October 12, 2016 Copyright , Kirkpatrick Partners, LLC. All rights reserved. 8

9 New Curriculum New Approach New mission required a new training approach Collaboration is essential to our success integration emphasized Curriculum Evaluation Input Instructor Trainee Development Team Copyright , Kirkpatrick Partners, LLC. All rights reserved. 9

10 Trainee Input Instructor Trainee Surveys Focus Groups Class feedback sessions Development Team Development Team Input Instructor Development Team Trainee Observations help us identify: Clarity in content and/or instruction Effectiveness of sequencing Level of trainee interaction Too much/too little content provided for time available Opportunities for trainees to practice or apply course content Relevancy of test questions Copyright , Kirkpatrick Partners, LLC. All rights reserved. 10

11 Instructor Input Instructor Development Team Trainee Lesson debrief sessions provide qualitative data Weekly summary provided to Executive Management Benefits of Blended Approach Instructor Trainee Holistic view of the curriculum Quick, yet effective changes to curriculum Development Team Copyright , Kirkpatrick Partners, LLC. All rights reserved. 11

12 Lifecycle Data from blended approach analyzed with Level 2 and Level 3 data Participant Interaction In the general chat window, please ask any questions you may have of Cynthia. Feel free to make comments as well. Copyright , Kirkpatrick Partners, LLC. All rights reserved. 12

13 Booz Allen Hamilton Gerald Jones January 2014 Bronze Replaced Level 1 & Level 2 surveys New data helped client ID operational shortfalls Increased trust & access to workers/leadership Commander s brain trust: 4 new projects under $800,000 contract extension Copyright , Kirkpatrick Partners, LLC. All rights reserved. 13

14 Silver Bronze gives the What and the Why Silver gives the How GPS Navigation to ROE Silver July 2015 Applications in Critical Thinking Evaluation Evaluation integrated from planning through execution Built Leading Indicators Chain of Evidence in only 3 months Presented preliminary results, demonstrated ROE Pool of students expanded to globally stationed Airmen Program expanded to three-part training series Contributed to multi-year client engagement extension Copyright , Kirkpatrick Partners, LLC. All rights reserved. 14

15 Participant Interaction In the general chat window, please ask any questions you may have of Gerald. Feel free to make comments as well. Meds Administration Project Linda Hainlen Copyright , Kirkpatrick Partners, LLC. All rights reserved. 15

16 Kirkpatrick Aha Moment Project: Meds Administration Goal: 90% compliance Outcome: 63% Failure is only permanent if you quit! Monitor and Adjust Outcome: 96% Copyright , Kirkpatrick Partners, LLC. All rights reserved. 16

17 Participant Interaction In the general chat window, please ask any questions you may have of Linda. Feel free to make comments as well. Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT Corporate Induction for Outstations SOAR (Service Over and Above the Rest) Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT Award-winning solutions Copyright , Kirkpatrick Partners, LLC. All rights reserved. 17

18 The Business Challenge Brand Reputation Emirates airline is loved and admired for its innovative products and excellent customer service on board their 235 aircraft fleet. This positive brand reputation did not extend to their 6 contact centres. Service Standard Customers complained that the service provided was inconsistent, impersonal and overly complicated. Rapid Growth Strategy Over the next 10 years, Emirates will be adding 277 more aircraft to their fleet, with a 20,000 seat capacity, year-on-year into the market, it was essential that Emirates Group Learning and Development team fixed problems in the global contact centres. Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest What Emirates Group Learning & Development did Root and branch learning needs analysis Presentation of findings and preferred solution Strategic Commercial and Service Initiatives Design and delivery of two bespoke programmes globally: Coach for Performance / SOAR. GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest Formalised onjob-support (OJS) Pre/post training conversations for every agent. Two weeks additional OJS for new agents. Aligned performance package Monthly 1:1 coaching sessions. Monthly call audits. Calibration sessions. Shout-Out Programme and Agent of the Month recognition schemes Copyright , Kirkpatrick Partners, LLC. All rights reserved. 18

19 Evidence of meeting business objectives The business objective of achieving 80% scoring on call quality audit was met As a result, the business achieved more than a 31.85% increase in sales at enterprise level, 11.85% percentage points higher than the target Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest Here s what customers had to say Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest Copyright , Kirkpatrick Partners, LLC. All rights reserved. 19

20 Thank you! Strategic Commercial and Service Initiatives GROUP LEARNING & DEVELOPMENT SOAR Service Over and Above the Rest Participant Interaction In the general chat window, please ask any questions you may have of Joyce. Feel free to make comments as well. Copyright , Kirkpatrick Partners, LLC. All rights reserved. 20

21 Participant Interaction I have at least one idea I can implement in my work. A. Yes B. No I would recommend this program to others in this field. A. Yes B. No In the chat area or Insights widget, please list one specific action you plan to take as a result of your participation in this program Kirkpatrick Partners, LLC. All rights reserved. Participant Interaction Would you like to chat with us? Do you have more in-depth questions? Contact us at inquiries@kirkpatrickpartners.com or (770) Kirkpatrick Partners, LLC. All rights reserved. Copyright , Kirkpatrick Partners, LLC. All rights reserved. 21

22 FREE Resources Register for free at kirkpatrickpartners.com to receive: Instant access to Resource Library with 60+ items Subscription to weekly e-newsletter New content every week News, specials and events You will also receive a follow-up next week if you registered prior to the live event Kirkpatrick Partners, LLC. All rights reserved. More In-Depth Information Kirkpatrick Four Levels Evaluation Certification Program Build an effective program implementation and evaluation plan 13 programs remaining this year Online begins on Nov 3 Oct and Nov in Dulles, VA Kirkpatrickpartners.com Training & Events tab Kirkpatrick Partners, LLC. All rights reserved. Copyright , Kirkpatrick Partners, LLC. All rights reserved. 22

23 More In-Depth Information Kirkpatrick s Four Levels of Training Evaluation Now available! Expert guidance in how to apply the model Templates and tools Case examples tinyurl.com/k4lte-book Kirkpatrick Partners, LLC. All rights reserved. Please Contact Us Customer service: information@kirkpatrickpartners.com Sales: inquiries@kirkpatrickpartners.com (770) kirkpatrickpartners.com Copyright , Kirkpatrick Partners, LLC. All rights reserved. 23

24 JOIN THE KIRKPATRICK COMMUNITY FOR FREE RESOURCES & INFORMATION Kirkpatrick Partners strives to provide the information you need, when and how you want it. STEP 1: REGISTER WITH KIRKPATRICK PARTNERS Register online at to receive access to the online Kirkpatrick Resource Library and be automatically enrolled in our weekly newsletter. Receive all of the following: Access to online Kirkpatrick Resource Library containing 60+ free items on demand Articles, white papers, videos, podcasts, PowerPoint slides, diagrams and tools One efficient weekly containing: Weekly Kirkpatrick Quick Tip Latest articles News, specials and event schedules Ability to comment on features in The Official Kirkpatrick Blog STEP 2: SELECT YOUR FAVORITE SOCIAL MEDIA FOR TIMELY UPDATES Do you prefer to get your news as it happens? Do you like your Facebook page better than your Outlook inbox? If so, this step is for you! Follow Kirkpatrick Partners on your favorite social media network. Join the discussion and receive the latest news by liking us on Facebook, following our Twitter connecting with us on LinkedIn, following our boards on Pinterest, or subscribing to The Official Kirkpatrick Blog on our website. STEP 3: ADD CUSTOMIZED SUBSCRIPTIONS FOR MORE INFORMATION Do you want everything from Kirkpatrick Partners? If so, follow and on Twitter, join the Kirkpatrick Evaluation Discussion Group on LinkedIn and subscribe to the Kirkpatrick Partners YouTube channel. The One and Only Kirkpatrick (770)

25 MAXIMIZING RESULTS WITH KIRKPATRICK Early part of the journey - Ensuring effective training Register for FREE resources at kirkpatrickpartners.com Watch Kirkpatrick webinars Discuss effective training versus training effectiveness in your organization Participate in Getting to Kirkpatrick Levels 3 & 4 or Igniting the Inner Fire workshops Join the Kirkpatrick Evaluation discussion group in LinkedIn Participate in the Kirkpatrick Four Levels Evaluation Certification Program - Bronze Level Farther down the road - Building training effectiveness Connect Levels 3 and 4 through required drivers, critical behaviors and leading indicators Schedule a Kirkpatrick Strategic Evaluation Planning Certificate Program Implement a systematic evaluation decision-making process Obtain Kirkpatrick silver level certification Nearing the final destination - Maximizing organizational ROE (return on expectations) Consider a Kirkpatrick Business Partnership Analysis Schedule a Kirkpatrick Impact Study Present your compelling chain of evidence to your corporate jury Obtain Kirkpatrick gold level certification 06/2016 (770) information@kirkpatrickpartners.com