FY 2014 ANNUAL MANAGEMENT REPORT

Transcription

1 THE DEFENSE INTELLIGENCE AGENCY LEADER EVALUATION of DEVELOPMENT and INVESTMENT DIALED IN FY 2014 ANNUAL MANAGEMENT REPORT A report designed to aid in making informed evidence-based decisions when recommending improvements to the Leader Development Program. Academy for Defense Intelligence DIA Leader Advantage Webinar Results.

2 DIA LEADER DEVELOPMENT PROGRAMS... 1 SUMMARY...1 RECOMMENDATIONS...1 LEADER ADVANTAGE WEBINAR SURVEY OVERVIEW and OBSERVATIONS... 2 OVERVIEW OF LEADER ADVANTAGE WEBINAR SURVEY TOOL...2 QUICK-LOOK SURVEY QUESTIONS... 2 WEBINAR AVERAGE BASED ON PARTICIPANT FEEDBACK, USING THE SCALE:... 2 OFFERED... 2 ATTENDED... 2 REGISTERED... 2 NO SHOWS... 2 CANCELLATION... 2 SURVEYS... 2 MOMENT of LEARNING NEED... 2 WEBINARS by TOPIC...3 WEBINARS by ITERATION...4 WEBINAR VALUE as DELIVERED by FACILITATOR / PRODUCER...5 1

3 DIA LEADER DEVELOPMENT PROGRAMS SUMMARY The collective survey data from participants in DIA s Leader Development Webinars events agreed or strongly agreed they found value and quality in DIA s Leader Development curriculum. As rated, the Primary Indicators of Value (PIV) and Primary Indicators of Quality (PIQ) suggest shared success across the development continuum. The FY2014 Defense Intelligence Agency Leader Evaluation of Development and Investment (DIALED IN) Management Report displays consolidated survey data drawn from Questionmark Perception (QMP). Managers can use this report to help with: Evidence-based decisions, review of individual and collective course survey results, and comparisons of iterative data against the averages for focused improvement. This FY2014 report shows survey results for Leader Advantage Webinars, using Quick-Look Micro-Metrics (QL2M) to measure Primary Indicators of Value (PIV) in the leader competencies delivered within the semiformal learning venue of the collaborative virtual environment. The QL2M survey helps evaluate each Webinar categorized by both topic and iteration in support of the DIA Leader Advantage Program. The survey also includes questions on the learner s moment of learning need. QL2M measures Webinar PIV in: Value, relevancy, and benefit. RECOMMENDATIONS Leader Advantage Manager: As Webinar NO SHOW rate continue to trend upward, attempt to determine if the same employees register yet do not attend. Though currently not an issue, NO SHOWS will become a challenge with the data showing a steady increase in registration numbers. This trend will require registration quotas in the future. 1

4 LEADER ADVANTAGE WEBINAR SURVEY OVERVIEW and OBSERVATIONS OCTOBER 1, SEPTEMBER 30, 2014 OVERVIEW OF LEADER ADVANTAGE WEBINAR SURVEY TOOL Participants in Leader Development Webinars were surveyed using Quick-Look Micro-Metrics (QL2M) to measure Primary Indicators of Value (PIV) in leader competency delivered within the semi-formal learning venue of the collaborative virtual environment. The metrics and measures used in the survey: QUICK-LOOK SURVEY QUESTIONS 1. The Webinar helped me see more clearly how this topic is relevant to my job. 2. The Webinar provided me with some practical suggestions and/or best practices. 3. The Webinar will help me discuss this topic with others in the workplace. 4. The knowledge I gained through interactions with others during the Webinar will help me improve my performance as a leader. WEBINAR AVERAGE BASED ON PARTICIPANT FEEDBACK, USING THE SCALE: 1 - STRONGLY DISAGREE 2 - DISAGREE 3 - NEITHER AGREE NOR DISAGREE 4 - AGREE 5 - STRONGLY AGREE OFFERED Number of times the Webinar was delivered. ATTENDED Total participants who attended the topic; average attended per topic. REGISTERED Total employees who registered to attend; average registration per topic. NO SHOWS Total expected on day of event that did not attend; rate of no shows per topic [NO SHOWs divided by (REGISTERED minus CANCELLATION)] CANCELLATION Total participants who cancelled; rate of cancellation per total registrants. SURVEYS Surveys completed; survey rate completed per attendees. MOMENT of LEARNING NEED Shown in % of Participants that attended the Webinar to: 1. Learn about this subject for the first time. 2. Learn more about this subject. 3. Refresh my memory to apply this subject. 4. Find a solution to a specific problem related to this subject. 5. Change my approach to the subject because things have changed. 2

5 WEBINARS by TOPIC OBSERVATION: The 462 surveys collected from 530 participants represents a highly acceptable 87% survey completion rate. These 462 data points provided the consolidated value-based survey data from 56 Webinar iterations within 24 Webinar topics offered between 1 October 2013 and 30 September Participants Agreed or Strongly Agreed they found value, relevancy, and benefit in the Webinar topics, rating the Overall Value (OV) in topic average at 4.38 on a scale of 5. With a rating of 4.41 and 4.40, participants indicate they found the most value in 1) practical suggestions and/or best practices provided in the Webinar topics and 2) the relevancy of the topic to their job. The Moment of Learning Need shows 57% of participants attended Webinars to learn more about a particular topic, 17% attended as a refresher, and 18% attended Webinars to learn about the subjects for the first time. FY2014 LEADER ADVANTAGE WEBINARS by TOPIC # OFFERED SURVEY QUESTION TOPIC AVERAGE Attended OV AVG Total per Total AVG per Total Rate Total Rate Total Rate Accountability at Work % 5 25% 9 100% 13% 78% 10% 0% 0% Building Trust Managing Rapid Change % 0 0% 7 78% 0% 46% 42% 13% 0% Business Execution Culture % 9 20% 24 89% 42% 43% 11% 0% 3% Communication Effectiveness % 0 0% 20 83% 15% 57% 28% 0% 0% Considering Conflict % 8 12% 34 85% 21% 46% 31% 0% 3% Controlling Projects % 7 27% 8 80% 13% 75% 13% 0% 0% Developing and Motivating People % 9 26% 13 93% 23% 77% 0% 0% 0% Diversity Expectations % 6 29% 9 82% 10% 43% 25% 10% 13% Finance and Your Role % 1 7% 9 100% 33% 44% 25% 0% 0% Giving and Receiving Feedback % 6 18% 16 89% 52% 31% 17% 0% 0% Influence and Persuasion % 5 14% 13 93% 0% 62% 38% 0% 0% Leading Effective Meetings % 33 31% 46 96% 12% 55% 15% 9% 9% Leading Successful Teams % 19 15% 74 82% 16% 48% 26% 4% 7% Listening to the Unspoken % 3 8% 18 86% 21% 56% 5% 12% 7% Motivational Leader % 2 7% 15 83% 8% 44% 19% 13% 15% Need for Perseverance and Flexibility % 3 12% % 8% 67% 17% 8% 0% Negotiating Technique % 4 22% % 18% 55% 27% 0% 0% Planning to Communicate Vision % 2 7% 14 88% 22% 53% 25% 0% 0% Problem Solving Process % 5 11% 21 81% 4% 66% 14% 7% 11% Removing Barriers to Change % 1 3% 25 76% 16% 46% 23% 10% 8% Removing Obstacles to Innovation % 5 22% 5 83% 0% 100% 0% 0% 0% Strategic Thinking % 2 10% 11 85% 45% 55% 0% 0% 0% Successfully Managing Up % 16 22% 35 92% 11% 51% 16% 9% 11% What to Delegate % 5 17% % 8% 65% 17% 5% 5% TOTALS, AVERAGES, RATES, and PERCENTAGES % % % 17% 57% 18% 4% 4% Registered No Shows Cancellations PARTICIPANTS Surveys MOMENT of LEARNING NEED 3

6 WEBINARS by ITERATION OBSERVATION: Participants Agreed or Strongly Agreed they found value in each Webinar, rating the Overall Value (OV) by iterative average at The annual average attendance of 9 participants per Webinar steadily increased from an average of 6 attendees per session in Q1, to 8 in Q2, to 11 in Q3, to 12 in Q4. The average 'No Show' rate of 32% shows a steady increase from 26% in Q1 to a 38% in Q4. An indicator of effective advertising and word-of-mouth marketing is shown in the steady increase of registration average per Webinar, up 300% from 8 in Q1 to 24 in Q4. FY2014 OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEPT WEBINAR ITERATION TIME DELIVERE PARTICIPANTS QUESTIONS OV No Shows Cancellations Surveys MOMENT of LEARNING NEED Attended Registered Total Rate Total Rate Total Rate Building Trust Managing Rapid Change % 3 75% 0% 67% 33% 0% 0% Removing Barriers to Change % 5 56% 0% 60% 20% 20% 0% Motivational Leader % 4 100% 25% 50% 0% 25% 0% Leading Successful Teams % 5 100% 20% 40% 40% 0% 0% Building Trust Managing Rapid Change % NOT 4 80% 0% 25% 50% 25% 0% Leading Successful Teams % COLLECTED 9 90% 11% 67% 11% 11% 0% Communication Effectiveness % FY14Q1 6 75% 17% 50% 33% 0% 0% Listening to the Unspoken % 11 92% 27% 55% 9% 9% 0% What to Delegate % 2 100% 0% 50% 50% 0% 0% Communication Effectiveness % 3 100% 0% 67% 33% 0% 0% Motivational Leader % 3 50% 0% 33% 33% 0% 33% Communication Effectiveness % 0 0% 11 85% 27% 55% 18% 0% 0% What to Delegate % 0 0% 4 100% 25% 75% 0% 0% 0% Planning to Communicate Vision % 1 9% 6 86% 17% 83% 0% 0% 0% Removing Barriers to Change % 0 0% 9 100% 22% 33% 33% 0% 11% Accountability at Work % 1 17% 5 100% 0% 80% 20% 0% 0% Finance and Your Role % 1 7% 9 100% 33% 44% 25% 0% 0% Considering Confllict % 4 31% 4 100% 0% 25% 75% 0% 0% Planning to Communicate Vision % 0 0% 4 80% 25% 50% 25% 0% 0% Removing Barriers to Change % 1 11% 6 86% 33% 50% 17% 0% 0% Removing Barriers to Change % 0 0% 5 63% 10% 40% 20% 20% 20% Leading Successful Teams % 0 0% 15 83% 7% 40% 47% 7% 0% Planning to Communicate Vision % 1 13% 4 100% 25% 25% 50% 0% 0% Strategic Thinking % 2 10% 11 85% 45% 55% 0% 0% 0% Leading Effective Meetings % 8 35% % 10% 70% 10% 10% 0% Motivational Leader % 2 14% 8 100% 0% 50% 25% 13% 13% Considering Confllict % 2 12% 4 40% 0% 67% 33% 0% 0% Leading Successful Teams % 4 18% 16 94% 18% 38% 19% 0% 25% Business Execution Culture % 7 30% 11 85% 20% 70% 0% 0% 10% Leading Effective Meetings % 17 47% % 31% 38% 15% 15% 0% Leading Successful Teams % 8 30% 11 79% 18% 46% 27% 0% 9% Leading Effective Meetings % 8 31% 11 92% 0% 55% 27% 9% 9% Business Execution Culture % 2 11% 10 91% 40% 60% 0% 0% 0% Giving and Receiving Feedback % 5 26% 9 82% 33% 33% 33% 0% 0% Leading Effective Meetings % 0 0% 12 92% 8% 58% 8% 0% 25% Diversity Expectations % 3 33% 4 100% 0% 25% 50% 0% 25% Considering Confllict % 2 11% % 20% 60% 10% 0% 10% Problem Solving Process % 3 12% 13 93% 8% 69% 15% 0% 8% Business Execution Culture % 0 0% 3 100% 67% 0% 33% 0% 0% Accountability at Work % 4 29% 4 100% 25% 75% 0% 0% 0% Considering Confllict % 0 0% % 63% 31% 6% 0% 0% Successfully Managing Up % 8 23% 16 89% 19% 38% 6% 13% 25% Diversity Expectations % 3 25% 5 71% 20% 60% 0% 20% 0% Leading Successful Teams % 7 17% 18 69% 22% 56% 11% 6% 6% Developing and Motivating People % 9 26% 13 93% 23% 77% 0% 0% 0% Listening to the Unspoken % 3 13% 7 78% 14% 57% 0% 14% 14% What to Delegate % 5 28% 7 100% 0% 71% 0% 14% 14% Negotiating Technique % 4 22% % 18% 55% 27% 0% 0% Successfully Managing Up % 7 39% 5 100% 0% 80% 20% 0% 0% Controlling Projects % 7 27% 8 80% 13% 75% 13% 0% 0% Giving and Receiving Feedback % 1 7% 7 100% 71% 29% 0% 0% 0% Influence and Persuasion % 5 14% 13 93% 0% 62% 38% 0% 0% Problem Solving Process % 2 11% 8 67% 0% 63% 13% 13% 13% Successfully Managing Up % 1 5% 14 93% 14% 46% 21% 14% 7% Need for Perseverance and Flexibility % 3 12% % 8% 67% 17% 8% 0% Removing Obstacles to Innovation % 5 22% 5 83% 0% 100% 0% 0% 0% TOTALS, AVERAGES, RATES, and PERCENTAGES % % % 17% 54% 20% 5% 5% 4

7 WEBINAR VALUE as DELIVERED by FACILITATOR / PRODUCER OBSERVATION: Shows facilitator/producer Webinar iteration ratings against their own average. The Overall Value (OV) rating per iteration was transposed to each facilitator/producer that delivered the session. This table does not intend to, nor should it be used to compare ratings between individual facilitators/producers. Evidence exists of a positive correlation between the OV rating of Webinar topics and the facilitator/producer involved in delivering it OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEPT WEBINAR NAME TIME FACILITATOR PRODUCER Building Trust Managing Rapid Change Removing Barriers to Change Motivational Leader Leading Successful Teams Building Trust Managing Rapid Change Leading Successful Teams Communication Effectiveness Listening to the Unspoken What to Delegate Communication Effectiveness Motivational Leader Communications Effectiveness What to Delegate Planning to Communicate Vision Removing Barriers to Change Accountability at Work Finance and Your Role Considering Confllict Planning to Communicate Vision Removing Barriers to Change Removing Barriers to Change Leading Successful Teams Planning to Communicate Vision Strategic Thinking Leading Effective Meetings Motivational Leader Considering Confllict Developing Talent (FPM 4/15) Leading Successful Teams Business Execution Culture Leading Effective Meetings Leading Successful Teams Leading Effective Meetings Business Execution Culture Giving and Receiving Feedback Leading Effective Meetings Diversity Expectations Considering Confllict Problem Solving Process Business Execution Culture Accountability at Work Considering Confllict Successfully Managing Up Diversity Expectations Leading Successful Teams Developing and Motivating People Listening to the Unspoken What to Delegate Negotiating Technique Successfully Managing Up Controlling Projects Giving and Receiving Feedback Managing Performance (FPM 9/4) Influence and Persuasion Problem Solving Process Successfully Managing Up Need for Perseverance and Flexibility Removing Obstacles to Innovation WEBINAR AVERAGE WHEN FACILITATOR BY WEBINARS FACILATATED/ PRODUCED