Finding the Truth: Interview and Interrogation Training Simulations. Ron Punako Senior Software Engineer

Transcription

1 Finding the Truth: Interview and Interrogation Training Simulations Ron Punako Senior Software Engineer 1

2 CTC Overview 501(c)(3) nonprofit established in 1987 Staff of 1,400+ professionals More than 50 locations 900,000 sq. ft., including labs & demonstration space Top 100 Government Contractor Quality/EH&S Management System comprised of industry-best models: ISO 9001 (Quality) and (Environmental), AS9100 (Aerospace), and CMMI- SE/SW (Systems/Software Engineering) Nationally recognized security capabilities with 300,000+ sq. ft. of Top Secret/Sensitive Compartmented Information Facility Space, JWICS, SIPRNet, and NIPRNet access 2

3 Interview System Goals Subject Environment Simulation of Body Language Simulation of Expression Proxemics Attention to Child s Needs 3

4 Interview ILS Goals Develop a proof-of-concept simulation to train investigators to conduct interviews of child abuse victims. Develop a training scenario to benefit interagency organizations for both military and civilian applications. Provide a virtual interactive training module avatar system, with all necessary motion and appearance to project the behavioral indicators of abuse. Enable investigators to practice interviewing child abuse victims and to receive feedback after the training has reached completion. 4

5 Subject: Cynthia Baker Physical profile 6 years old Petite Brown hair Brown eyes 5

6 Environment: Child s Interview Room Playroom environment Sofa, adult chairs, child chairs Toys Posters One way glass 6

7 Simulation of Body Language Challenge: Communicate emotional state using body language Designed 20 animations to communicate emotional state Body animations designed to be blended across 2 layers including idle and emotive Shrugging off Coloring Casual dialogue Closing off/hiding 7

8 Simulation of Facial Expression Challenge: Attempt to accurately portray facial expression Designed for sadness, fear, anger, indignant, disgust, happiness, confusion and shame across 4 layers including: blink, idle, lip-sync and emotive Selected expressions modeled after Facial Action Coding System action unit combinations. Happy Angry 8

9 Proxemics First question deals with seating arrangements relative to Cynthia Experimented with the Xbox 360 Kinect controller to provide Proxemics 9

10 Attention to Cynthia s Needs Challenge: Simulating the special needs of a child interviewee Talking at the child s level Sitting at the child s level Paying attention to the child's special needs (requests for mother, playing and coloring) Playing under the table 10

11 Interrogation System Goals Subject Environment Simulation of Body Language Simulation of Voluntary Facial Expression Simulation of Micro Expression Detection of Micro Expression Continuation of Disposition/Personality 11

12 Interrogation ILS Goals Develop a proof-of-concept simulation to train investigators to conduct interrogation of criminals suspected of sexual assault. Develop scenarios that can be manipulated to provide challenging interrogation exercises that are real and relevant to the current threat of perpetrators of serious crimes against persons. 12

13 Subject: Sergeant Mike Hagan Physical profile 30 years old 71 inches 156 lbs Brown hair Brown eyes Criminal record First time offender 13

14 Environment: Interrogation Room Minimalist design One table Two chairs Security Camera 14

15 Simulation of Body Language Challenge: Communicate emotional state using body language Designed 35 animations to communicate emotional state Body animations designed to be blended across 2 layers including idle and emotive Shrugging off Stop pressuring me Defensive rebuke Closing off/hiding 15

16 Simulation of Voluntary Facial Expression Challenge: Attempt to accurately portray facial expression with a fidelity model. Designed for sadness, fear, anger, contempt, disgust, happiness and surprise across 4 layers including: blink, idle, lip-sync and emotive Selected expressions modeled after Facial Action Coding System action unit combinations. Happy Angry Worried Disgusted 16

17 Simulation of Micro Expressions Challenge: Can the architecture handle extremely fast animation transitions and blending? Designed for sadness, fear, anger, contempt, disgust, happiness and surprise Achieved required 1/25 second micro expression duration. Worried micro expression 17

18 Detection of Micro Expressions Challenge: How to integrate a micro expression mini-game into the simulation? A possible micro expression occurs while a detect button is shown Not a test of dexterity False-positives possible. Worried micro expression 18

19 Continuation of Disposition/Personality Disposition Modifiers (First time offender) Red = +1 Yellow = +0 Green = -1 Disposition Threshold Disposition 5+ Angry/ Uncooperative Visual Output 2-4 Upset Disposition Modifiers (Repeat offender) Red = -1 Yellow = +0 Green = Idle/ Cooperative 19

20 Common Design Elements: Decision Sequencing Linear - sequencing of events with no opportunity for deviation from sequence trunk. Branching - sequencing of events that allows deviation from the sequence trunk. Recursive revisit previous events. Any order events presented in a discreet group in any order. 20

21 Common Design Elements: Decision Making Select a response which would help assess the child s episodic memory level. A. You were telling me about your Christmas (Green = Best) B. I ll bet Christmas day was fun, right? (Yellow = Average) C. Specifically, where were you and what did you do Christmas day (Red = Poor) 21

22 Common Design Elements: Performance Scoring Quantitative 1. Choice A 2. Choice B 3. Choice C Objective 1: 50% Objective 2: 100% Final: 75% Learner makes a choice Qualitative 1. Choice A 2. Choice B 3. Choice C Learner makes a choice Score is averaged by objective and recoded Objective 1: Good Job! Feedback is recorded by objective. Final score is averaged across objective scores Good Job! You chose choice A. Choice A was best because Feedback is provided to learner 22

23 Common Design Elements: Confession or Disclosure Challenge: How to simulate a meaningful confession/disclosure? Culminating point in final act Reached through majority green and yellow path decisions Must avoid terminal red path decisions 23

24 Common Design Elements: Notebook Challenge: How to provide the learner with an effective record of kinesics exhibited? Running log of all kinesics, dialogue, choices, decisions and mentor feedback Designed for learner reflection, performance review and memory/analysis aid 24

25 Common Design Elements: After Action Review Summary Panel Rollup of objectives Scored or un-scored against objective thresholds Details Panel For each decision displays screen shot, decision response, related objective and mentor feedback 25

26 Technical and Integration Details Work with internal and client SME to develop a script that details what content (objectives, voice over, feedback, remediation, interactivity, others) to integrate Staff works to produce content Custom development pipeline built on the Unity Game Development Tool supports integration of content. Simulations currently deployed to Web, PC, Mac, ipad 26

27 Next Steps System under evaluation for a Certificate of Networthiness Pursuing evaluation by learners at Fort Leonard Wood, MO Advanced feature integration Full implementation of continuation of disposition/personality Implementation of motion capture for Proxemics Photo-realistic body language and facial expression Improve development pipeline Evaluate potential for application in broader industry. 27

28 Questions? Demo? 28

29 Technical Point of Contact: Ron Punako Senior Software Engineer Business Development Point of Contact: David A. Kingston, P. E. Director, Learning and Human Performance Solutions