CogScreen Fundamentals

Transcription

1 CogScreen Fundamentals 16 September 2016 Gary G. Kay, PhD President, CogScreen, LLC St Petersburg, FL President, Cognitive Research Corporation St Petersburg, FL DISCLOSURE: I AM THE AUTHOR AND PUBLISHER OF COGSCREEN. I RECEIVE ROYALTIES FROM THE SALES OF MY TEXT: AEROMEDICAL PSYCHOLOGY

2 Aviation Neuropsychology Aviation neuropsychology is the field of clinical neuropsychology as it is applied in the context of civil and military aviation. Purpose of Aviation Neuropsychology Examinations Determine suitability for medical certification Assessing neurocognitive status of aviators with neurological trauma or illness Assessing fitness of aviators recovering from alcohol/substance abuse or dependence Evaluating aviators and controllers with problems that arise in training, upgrading, transitioning aircraft, or in proficiency testing.

3 CogScreen CogScreen is a computer-administered cognitive test battery initially developed in the late 1980 s for the U.S. Federal Aviation Administration for the purpose of medical certification testing of aviators. The test battery is used worldwide in aviation medicine and selection testing. CogScreen The test battery is used worldwide in aviation medicine and selection testing. Countries, in addition to U.S., where CogScreen is used: U.A.E. Colombia Norway Canada Sweden Chile U.K. Germany Mexico Australia Russia Japan India Kazakhstan Singapore Israel Hungary South Africa

4 CogScreen: Claim CogScreen-AE is primarily intended for use as a screening test. It was not designed for, nor is it capable of substituting for a comprehensive neuropsychological examination. 7 Cognitive Domains CogScreen Test battery includes measures of: Attention Memory Visual perceptual functions Sequencing functions Logical problem solving Psychomotor Speed & Coordination Simultaneous information processing abilities Executive functions

5 Steps in Test Development Reviewed task analyses of aviation human performance requirements i.e., evaluated cognitive ability demands at different phases of flight Identified performance tests and screening tests potentially sensitive to the presence of brain dysfunction. Implemented the selected tests in to a selfadministered computer-based test battery. Determined the predictive validity of the test battery for detection of brain dysfunction (ref: FAA Phase B). Developed norms for commercial aviators (ref: FAA Phase C). 9 Key Features System validation; must pass system test (timing and graphics test) prior to test being administered Screen calibration; guarantees that the size and distance between visual stimuli is standardized, regardless of monitor used Incorporates use of specialized software timers to provide superior timing resolution Timing resolution has been independently verified (1 msec) Standardized, intuitive, heads-up human-computer interaction using touchscreen for response input 10

6 Features Consistent format is used throughout battery: Simple and Brief Instructions Demonstration Practice (performed to criteria before subtest can be started) Subtest administration Examiner alerted and testing suspended when needed e.g., detection of timing violations; and detection of excess practice trials Designed for repeated testing (alternate forms; more than 25 have been used in a clinical trial) Validated scoring and reporting algorithms 11 Features Reporting Features Base-rate analysis Results presented in terms of Speed, Accuracy, Thruput, and Process Variables, as well as Multitasking Scores System maintains an encrypted database of all prior exams Summary results in XLS and CSV output 12

7 Aviation Applications of CogScreen Medical re-certification of aviators with potentially disqualifying medical, neurological and psychiatric conditions Head Injury, Alcoholism, HIV, etc. Initial medical certification of student aviators with a history of medical, neurological and psychiatric conditions Proficiency Problems (Busted PC) Transition Training (727 -> 767) Selection Testing Baseline Testing FAA Protocols Requiring CogScreen General neuropsychological evaluations e.g., head trauma, multiple sclerosis Special Issuance for pilots taking SSRI antidepressant medication HIV + pilot screening Evaluation of pilots recovering from alcoholism/substance abuse (HIMS program) Pending requirement: ADHD

8 Why administer a screening test (CogScreen) if you re administering an entire battery? 1) Relationship between conventional measures in the test battery and aviation are unknown. CogScreen is a valid predictor of flight performance and flight training. 2) Conventional measures are weak with respect to assessment of multitasking ability. CogScreen provides measures of multitasking ability that are not obtained with conventional testing. 3) CogScreen is designed for repeated measures testing. The majority of conventional measures are not designed for repeated measures testing. 4) Extensive normative data is available on pilots for CogScreen, whereas only limited normative data is available for pilots on conventional measures. Non-Aviation Applications

9 PHARMACEUTICAL RESEARCH CONDUCTED WITH CogScreen ANTIHISTAMINES ADHD TREATMENTS ATYPICAL ANTIPSYCHOTICS ANTIHYPERTENSIVES NUTRASWEET MS AND ALZHEIMER S TREATMENTS CHOLESTEROL LOWERING MEDICATION ORAL DIABETES MEDICATION HORMONE REPLACEMENT THERAPY QUINOLONE ANTIBIOTICS ANTIMUSCARINICS (GI & UROLOGY) Clients sponsoring these trials include: Accera Alexza Allergan Avanir Bayer Bristol-Myers Squibb CeNeRx Cephalon (Teva) Coca-Cola Dart NeuroScience Eli Lilly and Company Factor Nutrition Forest Laboratories GlaxoSmith-Kline Johnson and Johnson Kraft Foods Merck Mylan Novartis Otsuka Pharmaceuticals Pfizer Schering-Plough Shire Sunovion VIVUS Watson

10 20

11 Immediate vs Sustained Release Caffeine AN AWKWARD MOMENT The new clinical version of CogScreen, CogScreen-AE2, will be introduced and premiered at this meeting, but is not yet ready for release. For this workshop the focus will be on the current version, CogScreen-AE. In this presentation limited references may be made to CogScreen-AE2. A more thorough presentation on CogScreen-AE2 will be made in a subsequent workshop. Note: the interpretive strategies provided in this workshop apply to both versions of the test.

12 PRELIMINARIES: PROPER SETUP AND ADMINISTRATION FOR COGSCREEN-AE CogScreen-AE Hardware Designed for PC running Windows-XP Operating System PC Desktop or Laptop ELO (Accutouch) LCD Touchscreen Monitor Keyboard Gel Wrist Pad Plastic Stylus Speaker or Headphones

13 INSTALLATION Straightforward for PC with Windows XP Operating System; accept defaults For Windows 7/8/10 must select a different destination for the database file: C:\Program Files\ NOT Default which is C:\Program Files (x86) We don t warrantee that CogScreen will properly function on machines using an operating system other than Windows XP But we will try to help you Critical: Only install the ELO Touchscreen driver recommended by CogScreen! The MultiTouch Driver is NOT compatible ELO Setup and Calibration Setup Install proper ELO Touchscreen driver (before plugging USB cable from ELO Touchscreen into PC) When you plug the ELO s USB cable into the PC installation should complete automatically MARK the USB port on the PC and always use this port for the ELO Set ELO Properties Click on the ELO application; likely in task bar Turn off the Beep on Touch feature Turn on Enhanced alignment Turn on Mouse emulation 26

14 ELO Calibration FIRST: If using a Laptop and an ELO (or a secondary monitor): DIRECT ALL VIDEO TO THE ELO the other screen should be dark! Open the ELO application Click on Align Tap the Bullseyes CogScreen Setup Click on the CogScreen icon on the desktop Logon as Professional May use default user name: admin password: cogscreen 28

15 ADMIN SETUP Follow instructions in Operations Manual You need to setup 2 different Credentials. EXAMINER for administering the test PROFESSIONAL for generating reports, ordering tests, changing settings You can use the same password for both. Our convention: EXAMINER USER NAME: First Name, e.g., Gary PROFESSIONAL USER NAME: Last Name, e.g., Kay ADMIN SETUP: Licensing and Test Uses Activate license and install test uses: Click on Tools->Order Tests Call CogScreen Provide us with your 18 character code to receive License Key which you will enter on this screen

16 CALIBRATION: CogScreen Screen Calibration While holding the half sheet of paper up to the screen be careful not to touch the screen (remember, it s a touchscreen) or you will have to start over. Use arrow keys (initially right arrow and down arrow) to make rectangle on screen the same size as the half sheet of paper. Contrast with CogScreen-AE2 Download online from Requires Windows 10 Operating System Runs on Microsoft Surface Pro 3 / 4 or PC running Windows 10 + ELO (Accutouch) LCD Touchscreen Monitor Requires only 1 Username and Password Many configurable settings or leave the defaults.

17 TEST ADMINISTRATION Standardized Administration MANDATORY: Follow written administration instructions in Operator s Guide Read test instructions verbatim Demonstrate use of Stylus Use Gel Wrist rest if available Remove keyboard! Readjust position of Monitor if necessary Instruct examinee to NOT MOVE the keyboard. Tell them that you will position the keyboard when they reach the subtest that refers to the keyboard. Demonstrate tracking test! 34

18 Standardized Administration: Demo for Tracking Alone Demonstrate the Tracking Test (using non-dominant hand; Tap arrow keys) 35 CONTACT THE TEST ADMINISTRATOR The message shown above appears when the examinee either Fails Practice 2 times Requests to Repeat Instructions 2 times Type Alt-F1 and follow prompts on menu Generally choose to repeat instructions and provide examinee with assistance as needed After 3 rd failed attempt at Practice, tell subject to try their best and start the subtest. 36

19 Standardized Administration: TESTING COMPLETED Completing the test session Completed session: type m a i n (Enter) Examiner s Observations screen will appear Not necessary to make entries Must proceed to Exit CogScreen to close database To Abort Session (between subtests) Type Alt-F1 and then Esc Proceed with Completing the Session instructions above Indicate if subject will return to complete testing You will not be able to generate a report If you have indicated that subject is NOT returning You will be able to generate a report 37 Generating Reports Log on as Professional Click examinee s name on list on left side of screen Select Test Session Unfilled Box testing not completed no report available Filled Box testing completed report available Click on Reports Tab Select Aeromedical Report Select comparison group For non-commercial pilots (e.g., general aviation pilots) recommend using Regional Pilot Norms CogScreen prints to Default Printer 38

20 CogScreen-Aeromedical Report 39 40

21 Troubleshooting: Common Problems & Solutions ELO not responding to taps ELO or CogScreen needs calibration; calibrate ELO not recognized due to change of USB port; uninstall and reinstall (correct) driver Make sure your PC is only showing video on ELO Unable to restart session; Must have at least 1 new session on license to re-start a prior session Subtest Crashes; most likely due to other program, process, or window trying to open Close all programs Take system off-line COGSCREEN SUBTESTS Show video demo: Cog XP Subtest Demo

22 TEST INTERPRETATION CogScreen Scores & Nomenclature Speed (RTC) Presented in seconds Median response time for correct responses Inverted J distribution skewed Accuracy (ACC) Presented as percent correct Accuracy scores tend to be high (90-100%) except on SAT Discovery subtest J distribution 44

23 CogScreen Scores & Nomenclature Thruput (PUT) Presented as number of items correct per minute; a measure of efficiency Most normally distributed of CogScreen measures 45 CogScreen Nomenclature Process refers to the group of measures that didn t neatly fit under the headings of Speed or Accuracy Coordination scores Tracking scores (e.g., boundary hits) Rules completed Failure to maintain set Premature responses (i.e., response inhibition)

24 CogScreen Nomenclature Subtest Abbreviations (Prefixes) ASC BDS DAT DTT MAN MATH MTS PF Auditory Sequence Comparison Backward Digit Span Divided Attention Test Dual Task Test Manikin Math Matching to Sample Pathfinder CogScreen Nomenclature Subtest Abbreviations (Prefixes) SAT SDC VSC Shifting Attention Test Symbol Digit Coding Visual Sequence Comparison

25 Construct validity supporting test interpretation TEST INTERPRETATION Subtest x Cognitive Domain Backward Digit Span (BDS) Attention; Working Memory Visual Sequence Comparison (VSC) Attention; Focused, Working Memory Visual Perceptual Speed Information Processing Speed Language; letter sequencing Math (MATH) Language; reading comprehension Attention; Working Memory Executive Functioning; Logical Problem Solving Math Achievement; Math knowledge; Math speed/fluency 50

26 Subtest x Cognitive Domain Symbol Digit Coding (SDC) Attention; Focused, Working Memory Information Processing Speed Symbol Digit Coding Immediate Recall (SDCIR) Memory Paired associate memory Short delay recall Matching to Sample (MTS) Attention; Focused, Working Memory Visual Perceptual Speed Information Processing Speed Manikin (MAN) Mental Spatial Rotation 51 Subtest x Cognitive Domain Pathfinder Number (PFN) Attention; Focused Visual perceptual speed, visual scanning Number sequencing Psychomotor coordination Pathfinder Letter (PFL) Attention; Focused Visual perceptual speed, visual scanning Language; Letter sequencing Psychomotor coordination Pathfinder Combined (PFC) Attention; Focused Visual perceptual speed, visual scanning Executive Function Adaptive sequencing; applying a novel sequencing rule Psychomotor coordination 52

27 Subtest x Cognitive Domain Divided Attention Part I (Visual Monitoring) (DAT) Choice reaction time Visual perceptual speed Executive Function; inhibition of response Divided Attention Part II (Multi-tasking) (DAT) Attention; Divided *measures same abilities as Part I and adds multitasking challenge Auditory Sequence Comparison (ASC) Attention; Working Memory Auditory perception; Sequencing Symbol Digit Coding Delayed Recall (SDCDR) Memory Paired Associate Recall; Delayed 53 Subtest x Cognitive Domain Shifting Attention Test Arrow Direction Choice reaction time Shifting Attention Test Arrow Color Executive Function; ability to focus attention / and adapt to change in response rule; mental flexibility Shifting Attention Test Instruction Condition Attention; Working Memory; Shifting Attention Language; ability to rapidly read and follow written instruction Shifting Attention Test Discovery Condition Executive Function; Deductive (Logical) reasoning, Perseveration 54

28 Subtest x Cognitive Domain Dual Task Test Tracking Alone (DTT I) Psychomotor Dual Task Test Previous Number Alone (DTTPNA) Attention; Working Memory Dual Task Test Dual Condition (DTT D; DTTPND) Attention; Working Memory Psychomotor Multitasking ability 55 That s a lot of scores That s why we perform Base Rate Analysis 56

29 Heaton et al. Comprehensive Norms: Expected Frequency of Impaired Test Scores for a battery of 25 measures 57 58

30 59 What is Impaired? FAA s Concern is: Aeromedically Significant What level of performance reflects impairment? Scoring below the 5 th percentile for aviators? Classic neuropsychology cutoff: 5 th %ile Scoring below the 15 th percentile for the general population? More modern neuropsychology uses 15 th %ile when correcting for age & education According to what norms? Based on your normative age group? Based on others performing the same activity? 60

31 Criterion validity supporting test interpretation TEST INTERPRETATION FAA Phase B Validation Trial Determined ability of CogScreen to detect mild brain dysfunction. Compared CogScreen to conventional neuropsychological screening tests. 41 pilots (Class I-III) 42 age/education matched non-pilots with Head injury Cerebral tumor Recovering alcoholics (min 10 wk sobriety) Mild cognitive impairment Conventional Tests: WAIS-R, PASAT, Trails 62

32 FAA Phase B Validation Trial The Full Scale IQ score accounted for < 5% of the total variance on any CogScreen measure. Normal subjects (pilots and non-pilots) performed significantly better than patients on 17 of the 22 CogScreen measures. There were no significant differences between pilots and non-pilots (matched for age and education). Patients differed significantly from normal subjects on: Measure of multitasking; sequence comparison accuracy under single task (VSC) and multi-task (DATSC) conditions Difference between Pathfinder Combined and Pathfinder Numeric 63 FAA Phase B Validation Trial Origin of our Base Rate Approach 82.5% of patients had at least one score at or below the 5 th percentile versus 34% of controls. Among control subjects, fewer than than 10% had 2 or more scores at or below the 5 th percentile. Classification rule: 5 th percentile on 2 or more of the 7 indicator scores SENSITIVITY = 73% SPECIFICITY = 90% For a prevalence rate of 50%, the Positive Predictive Value for a positive finding using this classification rule is

33 FAA Phase C Validation Trial Base rate analysis (set at 95% specificity) revealed the following frequency of impaired performance (i.e., -1.5 SD) Alcohol referrals 55% Aviation performance referrals 82% Psychiatric referrals 50% Confirmed neurologic patients 80% Suspected neurologic patients 68% 65 Military Head Injury Studies US Navy Study (Moore & Kay, 1995) N=24 mtbi naval aviators, average of 8 months postinjury Performance was normal on conventional NP testing Significant relationship was found between PTA (r=0.56) and CogScreen LRPV score (particularly for LRPV speed measures) LRPV was also significantly correlated with history of LOC (r=0.50) In this sample LRPV was not significantly correlated with age, IQ, GCS, or months since injury LRPV was correlated with the PASAT (r=0.57) 66

34 Military Head Injury Studies Soviet Army Aviator Study (Yakimovich et al, 1992) 13 mild to moderately head-injured Russian military aviators, tested at time of return-to-duty evaluation Performance was compared to Russian (commercial) aviator norms (n=250). Utilized base-rate analysis: 3 of the 13 injured aviators performed 5 th percentile on two or more measures of accuracy and thruput when compared to agebased norms 1 of the 13 injured aviators performed 5 th percentile on measures of speed, accuracy and thruput when compared to age-based norms 5 of the 13 showed no scores 5 th percentile 67 Classification Analysis: Origin of the CogScreen LRPV Score Neurologic patients with mild brain dysfunction (N=40) including individuals with mild-to-moderate TBI, and patients post-surgical for treatment of epilepsy or tumors. Healthy normal individuals (N=60) matched for age and years of education Used stepwise likelihood-ratio logistic regression to predict probability of brain dysfunction (calculated LRPV value) Model resulted in 82.9% correct classification of brain dysfunction group, and 95% correct classification of normals; overall 90.1% classification accuracy. 68

35 Distribution of LRPV Scores 69 70

36 LRPV A logistic regression probability value Higher scores (i.e. closer to 1.0) indicate increased likelihood of brain dysfunction Based on 9 subtest scores: SPEED MEASURES: DATIRTC MTSRTC SATINRTC VSCRTC ACCURACY MEASURES: ASCACC MATHACC SATDIACC PROCESS MEASURES: DATIPRE DTTAHIT Poor performance on any one of these measures will elevate the LRPV score. Frequent elevations due to Math, Discovery, or combo of Instruction + VSC 71 Distribution of LRPV Scores Impact of Age 72

37 Distribution of LRPV Scores Impact of Age: GA Pilots 2014 PROBABILITY OF LRPV 0.80 Age Range n Percent % % % % % % % % 73 Screening for Brain Dysfunction 74

38 US FAA EXPERIENCE WITH AVIATORS SEEKING SPECIAL ISSUANCE FOR TREATMENT OF DEPRESSION Gary G. Kay, PhD CogScreen, LLC St Petersburg, FL James R. DeVoll, M.D. Office of Aerospace Medicine Federal Aviation Administration Washington, DC Federal Aviation Administration Introduction April 2010: FAA announced SSRI Policy allowing FAA airman medical certification for selected psychiatric disorders and use of SSRIs. However there was recognition that: Psychiatric disorders manifested by depression or other affective symptoms may result in impaired cognitive functioning Selective Serotonin-Reuptake Inhibitor (SSRI) medications can potentially impair cognitive function US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 76

39 REQUIREMENTS OF THE FAA SSRI POLICY Acceptable medications: Fluoxetine (Prozac) Escitalopram (Lexapro) Sertraline (Zoloft) Citalopram (Celexa) Limited to: depression (excluding Bipolar Disorder) Treatment: 12 months documented stability on medication And: no evidence of aeromedically significant neurocognitive deficiencies US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 77 US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 78

40 Original SSRI Testing Protocol CogScreen-Aeromedical Edition AND Full Neuropsychological Battery (NP-battery) The Wechsler Adult Intelligence Scales Trail Making Test, Parts A and B Executive function tests Category Test or Wisconsin Card Sorting Test; and Stroop Color-Word Test Paced Auditory Serial Addition Test (PASAT) A continuous performance test Test of Variables of Attention [TOVA], Conners Continuous Performance Test [CCPT], or Integrated Visual and Auditory Continuous Performance Test [IVA] Test of verbal memory WMS-IV subtests, Rey Auditory Verbal Learning Test, or California Verbal Learning Test-II Test of visual memory WMS-IV subtests, Brief Visuospatial Memory Test-Revised, or Rey Complex Figure Test Tests of Language Boston Naming Test and testing for verbal fluency (i.e., the COWAT and a semantic fluency task) Psychomotor testing (Finger Tapping,and either Grooved Pegboard or Purdue Pegboard) Personality testing to include Minnesota Multiphasic Personality Inventory (MMPI-2) US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 79 Experience of the first 24 months of the SSRI Program: Concerns: 1. Process was complex for airmen and HIMS AMEs 2. Process was also complex for FAA personnel processing the requests for special issuance 3. Costs were high for the airman and with respect to the hours required for the to FAA process requests 4. There were ethical and privacy concerns over the release of testing results and data to the FAA US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 80

41 FAA Internal Study Objectives Evaluate CogScreen-AE vs. Full NP-battery for assessing eligibility for medical certification under the SSRI protocol Determine if there are CogScreen-AE indices or patterns of scores that might be used to further refine prediction of performance on the Full NP-battery. US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 81 Methods Case Identification: Queried the Medical Appeals Branch database for all cases evaluated under the SSRI protocol from 4/2010 through 6/2012 Sources of information: Medical Appeals Branch Database: Case status and final certification determination FAA electronic medical records (DIWS) and hardcopy records: results of CogScreen-AE and NP-battery testing and other supplementary information US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 82

42 Methods Selection Criteria: Reviewed most recent record (if >1 record on file) Both CogScreen-AE and NP-battery results needed to be available for review A final determination had been made regarding issuance of a medical certificate Analysis Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 83 Results For the period April 2010 June 2012: Total individual airmen: 128 Total meeting selection criteria: 98 Issued: 79 Denied: 19 Neurocognitive deficiencies 11 Other disqualification* 8 * Other medical condition; (a) did not meet SSRI policy requirements due to type of psychiatric diagnosis, (b) suicide history, (c) inadequate treatment duration or stability, or (d) unacceptable medication usage (multiple medications, use of unapproved SSRI, etc.) US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 84

43 Results Cognitive Deficiency on Full NP Test Battery Yes No Total CogScreen-AE: Weak/Deficient Performance Yes No Total NP Determinations 11 (Disqualified) 87 (Qualified) 98 CogScreen-AE identified 25.2 % as potentially impaired (100% Sensitivity for CogScreen) NP-battery: Impairment confirmed in 12.6 % (87.4% Specificity for CogScreen) Overall: 12.6% disqualified for neurocognitive deficits US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 85 Results CogScreen-AE performance: Sensitivity 100.0% Specificity 87.4% PPV 50.0% NPV 100.0% for LRPV score >0.80 O.R US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 86

44 Discussion CogScreen-AE performed well as a screening test by identifying 100% of airmen with significant cognitive deficiencies evidenced by testing with the full neuropsychological test battery. The positive predictive value (PPV) of 50%, indicates that aviators performing poorly on CogScreen have a 50/50 chance of passing on the comprehensive conventional neuropsychological test battery. US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 87 Discussion Based on the findings of this study, 77.6% of airmen tested by CogScreen-AE may be spared full NP battery testing. These results were the basis for revising the FAA SSRI Program. US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 88

45 Programmatic Impact FAA Modified the SSRI Process: 2013 The requirement for the full NP test battery was eliminated as a requirement for initial special issuance consideration, unless deficiencies are suggested by CogScreen-AE. Subsequent annual testing requires only CogScreen-AE, except where clinically indicated on a case-by-case basis. US FAA Experience with Aviators Seeking Special Issuance for Treatment of Depression, ICASM, October 2013, Jerusalem 89 CogScreen and Flight Performance Studies Documenting Relationship Between CogScreen and Flight Performance Hoffmann CC et al The Role that cognitive ability plays in CRM. Paper presented at NATO Symposium, Human Factors & Medicine Panel on collaborative Crew Performance in Complex Operational Systems, Edinburgh, Scotland, April Taylor et al Relationship of CogScreen-AE to Flight Simulator Performance and Pilot Age. Aviation, Space, and Environmental Medicine, 71(4) ,

46 CogScreen and Flight Performance Yakimovich et al. Russian Flight Data Recorder (FDR) Study Method: Analyzed logs of flight parameter violations obtained from FDR. Violation frequency was compared to CogScreen test performance. Violations were obtained from a computer program which analyzes flight data recorder information. The list of violations were obtained for 75 Captains over a period of 3 years in two commercial jet aircraft (IL-86 and TU-154). 91 CogScreen and Flight Performance RESULTS Flight performance was found to be significantly correlated (p <.01) with 11 CogScreen variables. Yakimovich et al. Russian Flight Data Recorder (FDR) Study 92

47 AEROFLOT TU-154 REGRESSION ANALYSIS CogScreen prediction of flight performance violations (obtained from Cockpit Data Recorder): R 2 =0.32 Subtests: Divided Attention Test Dual Tasking Test Shifting Attention Test AEROFLOT IL-86 REGRESSION ANALYSIS CogScreen prediction of flight performance violations (obtained from Cockpit Data Recorder): R 2 =0.46 Subtests: Dual Tasking Test Pathfinder Backward Digit Span

48 Pilot Selection: Commercial Aviation Research Line Match Score Differentiates Commercial Line Pilots from General Aviation Pilots Training Success Score Training Problems Score Overall Aviator Prediction Score 95 CogScreen and Flight Performance Taylor et al Relationship between CogScreen AE factor scores and flight simulator performance in aircraft pilots aged METHOD 100 licensed pilots, 58 +/- 3 years Frasca model 141 flight simulator 96

49 CogScreen and Flight Performance Flight parameters Staying on course Dialing in communication frequencies Avoiding conflicting traffic Monitoring cockpit instruments Executing approach aylor et al, Relationship of CogScreen-AE to Flight Simulator Performance and Pilot Age 97 CogScreen and Flight Performance RESULTS CogScreen Speed/Working Memory had the highest correlation with the flight summary score (r=0.57) Four CogScreen variables could account for 45% of the variance in flight summary scores Speed/Working Memory Visual Associative Memory Motor Coordination Tracking Taylor et al, Relationship of CogScreen-AE to Flight Simulator Performance and Pilot Age 98

50 CogScreen and Flight Performance Compared to 5 Composite CogScreen Scores the Factors Speed/Working Memory (WM) Visual Association Memory Motor Coordination Tracking Attribute Identification Taylor et al, Relationship of CogScreen-AE to Flight Simulator Performance and Pilot Age 99

51 5-STEP INTERPRETIVE STRATEGY TEST INTERPRETATION Interpretive Strategy Step 1 Review key demographics Age, education, aviation hx, reason for referral Check the LRPV score 58 y.o. 16 yr educ Airline First Officer 23,000 hours Alcoholism (pre HIMS test) Meds: None Any Taylor Factors 40? Visual Learning and Recall

52 Interpretive Strategy Step 2 Review Base Rate Scores Speed 5 5 th %ile (T<33) 5 15 th %ile (T=42) Accuracy 2 5 th %ile (T=40) 2 15 th %ile (T=51) Process 1 5 th %ile (T=46) 1 15 th %ile (T=58) Interpretive Strategy Step 3 Review Individual Response Speed Scores Review Individual Response Accuracy Scores Review Individual Process Scores Speed 5 th %ile Pathfinder Combined Pathfinder Letter Pathfinder Number SAT Arrow Color SAT Arrow Direction Accuracy 5 th %ile SDC Immediate Recall SDC Delayed Recall Process 5 th %ile DTT Boundary Hits Dual

53 Interpretive Strategy Step 4 Identify scores contributing to an elevated LRPV score Multitasking analysis: Compare DAT alone vs dual Compare DTT alone vs dual LRPV - N/A for this case DAT Visual Monitoring Alone: 90 th pctile Dual: 50 th pctile DAT Sequence Comparison -ok DAT Premature Responses - ok DTT Tracking Error - ok DTT Prev Number Accuracy - ok DTT Boundary Hits Alone 1 hit (55 th pctile) Dual 12 hits (2.5 pctile) Interpretive Strategy Step 5 Interpretation: Slow on measures of visual scanning and tracking. Slow on measures of choice reaction time. Appears that the airman trades accuracy for speed. On a prior test he had 3 more scores at or below the 15 th percentile (simple reaction time and multitasking). Accuracy is good except on memory testing. Recalled only 2 of the 6 symbol-digit pairs. Poor performance on psychomotor tracking (2.5 percentile) under multitasking conditions. Recommendation: neuropsychological assessment with focus on memory and multitasking ability (repeat CogScreen-AE, Trail Making Test, and PASAT).

54 Interpretive Strategy Step 1 Review key demographics Age, education, aviation hx, reason for referral LRPV Taylor Factors y.o. 16 yr educ Major Airline Captain 13,000 hours Performance problems questionable & erratic Meds: None Pilot spent 135 minutes taking CogScreen Deductive Reasoning T=40.4 Visual Memory T=27 Interpretive Strategy Step 2 Review Base Rate Scores Speed 1 5 th %ile (T=48) 5 15 th %ile (T=42) Accuracy 1 5 th %ile (T=50) 3 15 th %ile (T=46) Process 1 5 th %ile (T=46) 5 15 th %ile (T=30)

55 Interpretive Strategy Step 3 Review Response Speed Scores Review Response Accuracy Scores Speed 5 th %ile Manikin 15 th %ile Tracking (alone & dual) Math *Visual Sequence Comp. Accuracy 5 th %ile SDC Immediate Recall 15 th %ile DTT Previous # Dual SAT Instruction Interpretive Strategy Step 3 (contd.) Review Process Scores Process 5 th %ile Perseverative Errors 15 th %ile DAT Premature Hit Dual DTT Boundary Hit Dual Coordination Combined & Letter

56 Interpretive Strategy Step 4 Identify scores contributing to elevated LRPV Multitasking analysis: Compare DAT alone vs dual Compare DTT alone vs dual LRPV Visual Sequence Comparison - probably ok for age DAT Visual Monitoring Alone: 90 th pctile Dual: 40 th pctile DAT Sequence Comparison Speed ok Accuracy - poor DAT Premature Responses - poor DTT Tracking Error both poor DTT Prev Number Accuracy 42 nd to 12 th pctile - poor DTT Boundary Hits 97 th to 7 th poor Interpretive Strategy Step 5 Interpretation: At 20 th percentile on measures of speed. Took more than 2 hours to complete test. Average response accuracy. Deficient on memory testing. Recalled only 1 of the 6 symbol-digit pairs. Poor performance on measures of multitasking ability and coordination under high task demand. Recommendation: neuropsychological assessment with focus on memory, working memory, deductive reasoning and multitasking ability. Post Script: (4 week re-test; fitness-for-duty) Very little improvement on CogScreen. Continued difficulty with reading speed and speed of mental rotation. Showed age-related declines in memory and processing speed. On conventional memory testing also showed declines relative to age norms (verbal and spatial). Reading speed found to be low normal. Good performance on working memory, spatial problem solving, and abstract reasoning.

57 Summary of Strategy for Test Interpretation Base Rates any T-scores below 40? 5 th percentile criteria? 15 th percentile criteria? LRPV score below 0.70 very reassuring Aviation Factor Scores - any T-scores below 40? Individual test score look at scores at or below T=40: SPEED ACCURACY (THRUPUT) PROCESS Mark scores that contribute to LRPV Multitasking issues? 113 Case Study 114

58 Case 5 67 yo, seeking 2 nd class special issuance for TBI Prior SI for Type II diabetes, coronary artery disease, & hypertension Fell on ice. Concussion with 5 min LOC. CT showed left frontal hematoma. GCS=15. Mild post-concussive symptoms (slowed cognition, balance difficulty, headaches, vertigo and fatigue. Underwent surgical evacuation of hematomas. PCS symptoms fully resolved by 7 mos. Case 5 EEG at 24 months showed no abnormalities 29 months s/p CHI saw a psychologist who stated that poor test performance was due to anxiety and age. Consultant, Dr. Moore, compared CogScreen scores to 145 older aviators using these norms still deficient in memory and deductive reasoning. Performance on CogScreen was 5 th percentile for older pilot norms on 10 of 16 thruput scores. Deficiencies noted on WAIS Working Memory and CogScreen multitasking.

59 Case 5 At 31 months s/p CHI received notice that his application was declined due to neurocognitive deficits. Airman informed by FAA that evaluation needed to be conducted by a neuropsychologist. Background: Dropped out of HS in 10 th grade to begin working. Became a truck driver and then fleet supervisor for major trucking company. Also runs successful commercial aviation company. Began flying at age 17. Licensed pilot for 40 years logged flight hours. IFR and multiengine rated. Check the LRPV Score

60 Check the Base Rates Check the Taylor Factor Scores

61 Review Individual Scores with T 40 Run GA Pilot T-Score Calculator Enter Raw scores for variables where examinee has scored at or below the 15 th percentile on Airline Pilot Norm.

62 Case 5 Test Scores and CogScreen Results to be Reviewed SUMMARY Difficulty evident on Matrix Reasoning and Category Test Processing speed Average to Low Average Strong performance on perceptual reasoning and organization Under speeded conditions accuracy suffers Average Arithmetic performance on WAIS Weak on Taylor scores for Memory, Deductive Reasoning, and (to a lesser extent) processing speed Normal performance on CVLT, BVMT-R Good performance on executive function measures: WCST, SAT Discovery, Stroop, Verbal Fluency Poor Trails B due to weak letter sequencing

63 Case 5 Recommended granting airman a special issuance Class II FAA Medical Certificate It appears that the only residual deficit from his head injury is a mild deficit in letter sequencing. Case 2: HIMS (Alcoholism) 35 yo Hispanic regional airline Captain w 7800 logged flight hours DWI arrest. Began drinking around noon. By 7 pm (when arrested) he had consumed pint of vodka and 8-12 beers. Had been drinking heavily 5-7 days per week. Reported a couple of blackouts per month in last year of drinking. Evaluation conducted 8 months post sobriety date. Completed 28-day inpatient program. 126

64 Case 2 Enrolled in aftercare program; attending AA meetings 4 days per week. Meeting w peer pilot counselor. Monitoring by employer. Seen by a psychologist and psychiatrist. Notified by FAA that results indicated the presence of cognitive deficiencies and that he would require further evaluation. Background: Father was an alcoholic. Attended a college offering an aviation training program. Hired as instructor for 3 yrs after graduating. 127 Case 2 Hired by cargo company for one year before being hired by regional airline. Flies regional jet. Reported that employer is very supportive. No prior mental health history. Medical history is unremarkable Prior testing revealed moderate impairments in visuospatial reasoning, visuospatial memory, and finger tapping speed. Mild deficits were found in nonverbal problem solving, working memory, and sensory-motor integration. Psychologist had recommended granting a 128 special issuance medical certificate.

65 Check the LRPV Score Check the Base Rates

66 Check the Taylor Factor Scores Review Individual Scores with T 40

67 Case 2 Review CogScreen Results Marked improvement compared to first evaluation Conventional Test Findings Review spreadsheet Poor arithmetic for an aviator Poor visual memory on RCFT Okay visual memory on CogScreen Okay memory on TPT Recommended that pilot be granted a special issuance first-class FAA medical certificate 133 Submission of Test Data to FAA

68 Submission of Test Data to FAA Submission of Test Data to FAA

69 Submission of Test Data to FAA For Additional Information contact me via 138