Diana Hardmeier Moritz Jaeger Rebekka Schibli Adrian Schwaninger Center for Adaptive Security Research and Applications (CASRA) Zurich Switzerland

Transcription

1 EFFECTIVENESS OF COMPUTER-BASED TRAINING FOR IMPROVING DETECTION OF IMPROVISED EXPLOSIVE DEVICES BY SCREENERS IS HIGHLY DEPENDENT ON THE TYPES OF IEDS USED DURING TRAINING Diana Hardmeier Moritz Jaeger Rebekka Schibli Adrian Schwaninger Center for Adaptive Security Research and Applications (CASRA) Zurich Switzerland Center for Adaptive Security Research and Applications (CASRA) Zurich Switzerland Center for Adaptive Research and Applications (CASRA) Zurich Switzerland Center for Adaptive Security Research and Applications (CASRA) Zurich Switzerland Abstract - Several previous studies have shown that threat detection performance of X ray operators can be increased substantially if computer based training (CBT) is used. This applies particularly to objects that are never or rarely encountered at a security checkpoint as for example Improvised Explosive Devices (IEDs). However, little is known so far about the importance of using different types of IEDs in CBT such as conventional, unconventional and inert IEDs. Conventional IEDs are made with real explosive and conventional detonators with a primary and secondary charge. Unconventional IEDs are made with real explosive and unconventional detonators that do not contain a primary charge. Inert IEDs are made with explosive stimulants and fake detonators. Two studies conducted with 420 and 433 airport security screeners are reported. All of them had already used an individually adaptive CBT program for improving x ray image interpretation competency during several years however containing only conventional IEDs. In study 1, screeners were tested with a computer based test containing conventional and inert IEDs before and after CBT for several months including both types of IEDs. In study 2 X ray operators were tested with a computer based test containing conventional, unconventional and inert IEDs before and after CBT for several months including all three types of IEDs. The following results were revealed in both studies: 1) Detection performance in the first test was high for those types of IEDs that X ray operators knew from previous CBT and 2) IEDs that were not detected well were effectively trained using CBT that contains these types of IEDs. In summary, CBT can be a very effective tool for increasing X ray image interpretation competency of screeners if it contains conventional, unconventional and inert IEDs. Index Terms - aviation security, computer based training, detection performance, human factors, improvised explosive devices, x ray screening. I. INTRODUCTION Several terror attacks in aviation evidence the importance of a reliable security control at checkpoints. Even though state-ofthe-art machines with automatic detection of explosive material, liquid detection and other functionalities help in detecting threat items, the human factor is still essential. X-ray screening of passenger bags is one important measure to prevent passengers bringing in threat items into the security restricted area and on board an aircraft. So far several studies showed that the detection of threat items in X-ray images depends highly on training as they often look quite different than in reality [1, 2, 3, 4, 5]. Thus, screeners not only need to know which threat items are prohibited but also what they look like in X-ray images of passenger bags. Studies including threat items of the four categories guns, knives, Improvised Explosive Devices (IEDs) and other 1 were able to show that especially the detection of the two categories IEDs and other is rather difficult without training. Threat objects of both categories vary enormously in shape and look quite different in X-ray images than in reality. Additionally, IEDs are normally neither seen in reality nor at a checkpoint. Despite the variety of IEDs, a study by [6, 3] revealed not only large training effects, but also large transfer effects for this category. That is, detection performance of IEDs is much higher after several months of training and IEDs that are visually similar to the learned ones can be detected even if they were never seen before. However, this transfer effect was only revealed for one IED type, namely conventional IEDs. Whether the training effect depends on the IED type learned or whether a transfer effect could as well be shown for different types of IEDs was investigated in this study. Therefore, three different types of IEDs: conventional, unconventional and inert IEDs were used. All screeners who participated in this study already used an individual adaptive training program for several years including conventional IEDs only. To test the performance for all three IED types before 1 The category other includes threat items like electric shock devices, chemicals, etc. 1

2 and after training a computer based test was used before and after Computer Based Training (CBT). Furthermore, the test design of the study not only allowed measuring the training and transfer effect but as well the influence of the bag complexity level on the detection performance of all types of IEDs used. Last, improvement in the detection for screeners who trained most and screeners who trained least was evaluated. II. STUDY 1 In the first study the two types conventional IEDs and inert IEDs were compared before and after CBT using a computer based X-ray screening test. A. Methods 1) Participants and Procedure: The sample consisted of 420 airport security screeners working at one European airport. All screeners had already several years of computer based training with the training system X-Ray Tutor (XRT) including conventional IEDs only. To measure the training effect in X-ray image interpretation, all screeners conducted an X-ray test before and after a 4 months period of recurrent computer-based training with a special IED version of XRT. During the training period, screeners trained on average 5.64 hours per week (SD = 5.02). To avoid test repetition as the reason for performance increase, two tests were developed and screeners randomly distributed into two groups: group 1 started with test A and finished with test B whereas group 2 conducted the tests in the reversed order. The data of both groups are combined for the analysis. 2) Material: a) X-ray screening test: The two X-ray image interpretation tests (test A and test B) were developed to measure how well airport security screeners are able to detect IEDs in X-ray images of passenger bags. The two parallel tests consisted of 192 X-ray images of passenger bags using images of Smiths-Heimann Hi-Scan 6040i machines. Whereas half of the bags where harmless bags (no threat item included), the other half of the bags contained an IED that had been virtually inserted into the bag by aviation security experts. The tests were integrated into the training system XRT and took about 1-2 sessions of 20 minutes to complete. Images were shown for a maximum of 15 seconds on the screen. The task was to visually inspect the images and judge whether a bag was OK (contains no IED) or NOT OK (contains an IED). X-ray image interpretation competency was measured by the Hit Rate (% of threat images correctly identified), the False Alarm Rate (% of harmless bags falsely identified as a threat), and A which is a detection performance measure that takes into account both the Hit and the False Alarm Rate [7]. Three groups of IEDs were used: trained conventional IEDs, conventional IEDs, and inert IEDs. IEDs of the first two groups belonged to the same IED type and thus showed both the most common IED characteristics: organic explosives of a quadratic shape and detonators with primary and secondary explosive charge. Trained conventional IEDs were already trained by the screeners before the study started whereas new conventional IEDs were not trained before. Inert IEDs were non-functioning IEDs visually different from conventional IEDs amongst others in color and in the constitution of the detonator. They were not trained by the screeners before this study started. In each group, 16 IEDs were used and each of the IED was inserted into both a bag with high and low complexity. Thus, both tests were composed of 192 images: 3 (IED groups) * 16 (objects) * 2 (bag complexity levels) * 2 (harmless images vs. threat images). b) X-ray screening training: Between the two tests, screeners conducted recurrent computer-based X-ray training using X-Ray Tutor, a widely-used training tool that effectively improves X-ray image interpretation competency [3, 8]. Using XRT, screeners have to inspect X-ray images of passenger bags created at the point of use by an adaptive algorithm that merges threat items and passenger bags with increasingly difficult combinations of viewpoint, superposition, and bag complexity. The images have to be judged as OK (contains no threat item) or NOT OK (contains a threat item). XRT has the same appearance as the test except there is immediate feedback and screeners have to click on the IED to show that they correctly identified it. For details on XRT and its algorithm please refer to [1, 9]. For this study, an XRT edition specialized on IEDs was created with IEDs belonging to the same groups as those used in the test: trained conventional IEDs, conventional IEDs, and inert IEDs. In order to measure the transfer effect, only half of the IEDs (Set A) used in the test were also integrated in the training. The other IEDs (Set B) were only included in the test version. B. Results and Discussion In both studies the detection performance measure A was calculated in order to compare the results in the computer based test before and after computer-based training. A takes the Hit Rate (H) and False Alarm Rate (F) into account. The Hit Rate shows how many times an IED was correctly identified in percent and the False Alarm Rate shows how many times a harmless bag was wrongly judged as NOT OK in percent. A is calculated by the following formula: 2

3 A = [(H F)(1+H-F)]/[4H(1-F)], if H > F A = [(F H)(1+F-H)]/[4F(1-H)], if H < F For more information about detection performance measures and A please refer to [10] or [11]. 1) Reliability: To examine whether the two tests which were used to measure detection performance are reliable and may be used for evaluation purposes an item analysis was conducted. Table 1 shows Cronbach Alpha and Guttman split-half reliabilities for each test version (test A and test B) before and after training. Reliabilities are calculated based on hit and correct rejection rates. In order to calculate Guttman split-half reliabilities the images were ordered according to the level of difficulty of the bags and then assigned into two halves. Cronbach Alpha values above.825 indicate good reliability [12]. As well all split-half coefficients >.864 are clearly above the limit of acceptable reliability [13]. Thus, these results affirm a reliable measurement of detection performance before and after training using our computer based test. TABLE I RELIABILITIES N = 420 1A 1B 2A 2B Cronbach Alpha Guttman split-half Cronbach Alpha and split-half reliabilities in the first study. Note. 1A and 1B are the two test versions before CBT, and 2A and 2B are the ones after CBT. The same persons passed test 1A/2B respectively 1B/2A. 2) Overall Detection Performance: As illustrated in Fig. 1, the objects with the highest detection performance are the trained conventional IEDs, followed by new conventional IEDs and inert IEDs. Independent of the IED group, a higher A value results from computer-based training with XRT. For all IED groups the difference between the first and the second measurement was significant with t(838) = -4.70, p <.001 for trained conventional IEDs, t(838) = -3.82, p <.001 for conventional IEDs and t(838) = -5.65, p <.001 for inert IEDs indicating training effectiveness. Figure 1. Detection performance A for each IED type before and after training. The bars represent standard deviations. Please note that due to security reasons detection performance values are not specified. In addition to the independent sample t-test a two-way analysis of covariance (ANCOVA) with the two withinparticipant factors IED group and measurement and the covariate training hours was conducted with the following results: There were main effects of IED group F(2,836) = , p <.001 and measurement F(1,418) = 25.70, p <.001 with an effect size of η 2 =.58 and η 2 =.06, respectively. The two-way interaction of IED group and measurement was also significant, F(2,836) = 6.00, p <.01 with an effect size of η 2 =.01. These results reveal that the improvement in detection performance is different for the different groups of IEDs. Thus, the detection performance of IEDs which have not been trained before, but are very similar to the trained ones (belonging to the same IED type) is higher than for another IED type which is visually completely different. Furthermore, the significantly better improvement for inert IEDs shows that screeners are able to learn new types of IEDs very quickly. 3) Detection Performance of Images in Training and Test (Set A) and in Test Only (set B): In order to evaluate whether IEDs within the same IED group - which were not included in the training session (set B) - have a lower detection rate, three ANCOVAs were conducted with the two withinparticipant factors measurement and set plus the covariate training hours (see Fig. 2). 3

4 To conclude the IEDs presented in the test only were overall easier than IEDs presented in both, the test and training session. However, the non-significant interactions between measurement and set indicate that the transfer effect for each IED group is quite strong. 4) Comparison of Detection Performance in Easy and Complex Bags: A two-way ANCOVA for each IED group was conducted in order to investigate for each group whether the detection differs in easy and complex bags. Figure 2: Detection performance A, with standard deviation bars, of images in tests only and in tests and training sessions. The results of the ANCOVAs for each IED group were as follows: Trained conventional IEDs: significant main effects of measurement with η 2 =.07, F(1, 418) = 29.50, p <.001 and set with η 2 =.56, F(1,418) = , p <.001, non-significant interaction with η 2 =.00, F(1, 418) =.87, p =.35 showing that the detection performance of images which were in the test and of those which were in both, the test and training session were significantly different. Also the difference in detection performance before and after CBT was significant. However, the interaction between measurement and set was not significant indicating that screeners improvement in the detection performance was not higher for images also shown in the training. Conventional IEDs: significant main effects of measurement with η 2 =.03, F(1, 418) = 11.77, p <.01, and set with η 2 =.32, F (1, 418) = , p <.001, non-significant interaction with η 2 =.00, F (1, 418) =.91, p =.34. These results are very similar to those obtained from trained conventional IEDs, revealing that there is again no significant interaction. Inert IEDs: significant main effects of measurement with η 2 =.05, F(1, 418) = 20.65, p <.001 and set with η 2 =.05, F (1, 418) = , p <.001, non-significant interaction with η 2 =.00, F (1, 418) =.48, p =.49 indicating that also for inert IEDs the improvement in detection performance did not differ for images which were only in the tests and those which were both in the test and training sessions. Figure 3: Comparison of detection performance A of easy and difficult bags. The bars again represent the standard deviations. As illustrated in Fig. 3, the detection performance of IEDs in easy bags was higher for all three IED groups. For trained conventional IEDs there were significant main effects of measurement η 2 =.03, F(1,418) = 13.58, p <.05 and complexity η 2 =.22, F(1, 418) = , p <.001, but no significant interaction η 2 =.00, F(1, 418) =.33, p =.57. The results for conventional and inert IEDs were similar. There were significant main effects of measurement η 2 =.02, F (1, 418) = 7.12, p <.01 respectively η 2 =.03, F(1,418) = 12.80, p <.001 and complexity η 2 =.15, F(1,418) = 75.56, p <.001 respectively η 2 =.01, F(1,418) = 5.73, the interactions were again not significant with η 2 =.00, F(1, 418) = 1.84, p =.18 respectively η 2 =.00, F(1,418) =.63, p =.43. These results reveal that the improvement in detecting IEDs in complex and easy bags between the first and second measurement were comparable for all three IED groups even though inert IEDs 4

5 were expected to be detected worse in complex bags because of their missing features. 5) Effect of Training Hours: In this part it was investigated whether frequent training affects the detection performance. To generate two groups with a considerable difference regarding the amount of training hours, the 24 airport security screeners with most training hours were compared against the 24 screeners with least training hours. For the analysis one outlier with more than 60 hours of training was deleted. Figure 4: Detection performance A with standard deviations bars for the 24 persons that trained most and the 24 persons that trained least. As shown in Fig. 4, the 24 screeners that trained most have a higher improvement in detection performance for all three IED groups, and they improved the most for inert IEDs. However, the analysis of variance (ANOVA) with the two withinparticipant factors measurement and training hours indicates no significant interactions of all three types of IEDs with η 2 =.04, F (1, 23) = 1.03, p =.32 for trained conventional IEDs, η 2 =.04, F (1, 23) = 1.02, p =.32 for conventional IEDs, and η 2 =.15, F (1, 23) = 3.89, p =.06 for inert IEDs. However, the p- value for inert IEDs which is just above the significance level of.05 indicates that by trend airport security screeners who trained more also show a higher increase in their detection performance. III. STUDY 2 In the second study the detection performance measure A in a computer based test for the three types of IEDs, namely conventional, unconventional and inert IEDs was calculated again before and after CBT. A. Methods 1) Participants and Procedure: The sample consisted of 433 airport security screeners working at the same European airport. Again,screeners conducted an X-ray test before and after a 2.5 months period of recurrent computer-based X-ray training with XRT. Equally as in study 1 two tests were created and the screeners were randomly distributed into two groups with a reversed test order. The screeners trained on average 2.37 hours per week (sd=2.03). Again results of both groups are analyzed together. 2) Material: a) X-ray screening test: Two new X-ray image interpretation tests were developed to measure how well airport security screeners can detect conventional, unconventional and inert IEDs in X-ray images of passenger bags. Both tests consisted of 132 X-ray images of passenger bags using images of Smiths-Heimann Hi-Scan 6046i machines with 50% of the bags containing an IED. Three IED types were chosen. Conventional IEDs (30 objects) and inert IEDs (6 objects) corresponded to the two IED types used in study 1. The objects of conventional IEDs were new and therefore not trained in study 1. The inert IED objects were the same as in study 1 and therefore already trained. The new type unconventional IEDs (30 exemplars) featured IEDs with the following characteristics: inorganic and metal explosives, detonators without primary and secondary explosive charge and various non-quadratic shapes of explosives. b) X-ray screening training: Again, an XRT edition specialized on IEDs was created with IEDs belonging to the same IED types as those used in the test. This edition was adjusted as follows: images were now shown in Smiths- Heimann Hi-Scan 6046i quality and the amount of implemented IEDs was largely increased. The IEDs in the training belonged to the same types as in the tests: Conventional IEDs, unconventional IEDs, and inert IEDs. In contrast to study 1, all IEDs in the tests were also used in training thus no measurement of the transfer effect within the same category was conducted in this second study. 5

6 B. Results Once more the psychophysical detection performance measure A was used for analysis. 1) Reliability: Cronbach Alpha and split-half reliability coefficients are listed in table 2, again separately for both test versions. All reliability coefficients show reliable measurements with Cronbach Alpha coefficients >.805 and split-half coefficients >.803. TABLE II RELIABLITIES N = 433 1A 1B 2A 2B Cronbach Alpha Guttman split-half Cronbach Alpha and split-half reliabilities of the second study. Note. 1A and 1B are the two tests before CBT, and 2A and 2B are the ones after CBT. 2) Overall Detection Performance: As in study 1, airport security screeners showed a better result after several months of training. This improvement in detection performance was significant for all three IED types with t(864) = -4.92, p <.001 for conventional, t(864) = , p <.001 for unconventional and t(864) = -2.47, p <.05 for inert (see Fig. 5). Figure 5: Detection Performance A for each IED type before and after training, n = 433. All three types showed significant effects (p <.005). The ANCOVA with the two within-participant factors IED type and measurement plus the covariate training hours gave the following results: there were significant main effects of IED type η 2 =.36, F(2,862) = , p <.001 and measurement η 2 =.05, F(1.431) = 23.56, p <.001 and a significant interaction of IED type and measurement η 2 =.04, F(2,862) = 16.46, p <.001. These results are also illustrated in Figure 5, which visualizes that the influence of training is different for unconventional IEDs than for the other two IED types. Because inert IEDs had now been trained for several months, the detection performance of this IED type was much higher compared to the detection performance of unconventional IEDs. It was also much higher in comparison to the detection performance of this type revealed in the first study. 6

7 3) Effect of Training Hours: Again, the results are similar as in study 1 (see Fig. 6). Figure 6: Detection performance A (with standard deviation bars) for the 24 persons that trained most and the 24 persons that trained least. Although the interactions of the two-way ANOVA were not significant η 2 =.11, F (1, 23) = 2.71, p =.11 for conventional IEDs, η 2 =.07, F (1, 23) = 1.67, p =.21 for unconventional IEDs and for inert IEDs η 2 =.00, F (1, 23) =.02, p =.90 Figure 6 indicates that screeners who trained the most have a slightly higher improvement in the detection performance than those who trained the least. Furthermore, the biggest improvement was found for the types of IEDs which never had been trained before. IV. SUMMARY AND CONCLUSIONS The aim of this study was to investigate the importance of having different types of IEDs in a computer based x-ray screening training. This was tested using three types of IEDs, namely conventional, unconventional and inert IEDs. In both studies screeners conducted a test before and after several months of computer based IED training with XRT. The results of both studies indicate that the detection performance of the three types of IEDs differs significantly. Conventional IEDs which were already trained for several years are detected the best, whereas the detection performance of new IED types was much lower in the beginning. However, this detection performance was increased substantially after training with XRT. Especially results for inert, but also for unconventional IEDs indicate that screeners are able to improve their visual knowledge within short time. These results are also consistent with former studies showing that training increases detection performance for all kind of threat items within short time [8, 3, 6, 5], Further, a transfer effect for similar looking IEDs within the same IED type was found. Conventional IEDs which had been trained for several years by the screeners were detected best. Whereas new conventional IEDs similar in shape to the trained ones had a slightly lower A value. This transfer effect was also found within each IED type as there was no significant interaction between measurement and set. That is, IEDs which were presented in both the test and training condition were not detected better after training than IEDs only presented in the test condition. These results are also consistent with previous studies which showed that screeners are able to detect visually similar items also without training of these specific items [3, 6]. However, unconventional and inert IEDs, two visually completely different types of IEDs have a significantly lower detection performance before training. Thus, learning how conventional IEDs look like does not help to recognize other IED types such as unconventional and inert IEDs in X-ray images. Furthermore, the improvement in the detection performance of IEDs in bags with different complexity levels reveals that with training all types of IEDs can be detected quite well independent of the complexity level of the bag, as well as inert IEDs which have no concise features. Although there is no significant effect between screeners who trained the most and screeners who trained the least, there was a clear trend that more training between the two test sessions leads to better detection performance especially for new IED types, namely unconventional and inert IEDs. This affirms that the most marked improvement in detection performance is for items which have not been trained before and that the training libraries need not only to be very broad, but also need to be constantly updated and adapted with new threat items. In short, results from this study evidence the importance of a variety of different IED types in X-ray screening training systems. For this study three types of IEDs were defined: conventional, unconventional and inert IEDs. Further studies should investigate whether other types and how many types of IEDs in total should be included in the training session in order to guarantee both, effectiveness and efficiency of the training system. 7

8 V. ACKNOWLEDGEMENTS We are thankful to Zurich Airport, Switzerland and the security companies at Zurich Airport supporting this research study. VI. REFERENCES [1] A. Schwaninger, Increasing efficiency in airport security screening, in Proceedings of AVSEC World 2004, November 3-5, Vancouver, B.C., Canada, [2] A. Schwaninge, D. Hardmeier and F. Hofer, Airport security screeners visual abilities & visual knowledge measurement, IEEE Aerospace and Electronic Systems, vol. 20(6), pp 29-35, [3] S. M. Koller, D. Hardmeier, S. Michel, and A. Schwaninge,r, Investigating training, transfer, and viewpoint effects resulting from recurrent CBT of x-ray image interpretation, Journal of Transportation Security, vol 1(2), pp , [4] J. D. Smith, J. S. Redford, D. A. Washburn, L. A. Taglialatela, Specific-Token Effects in Screening Tasks: Possible Implications for Aviation Security, Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 31(6), pp , [5] J. S. McCarley, A. F. Kramer, C. D. Wickens, E. D. Vidoni and W. R. Boot, Visual skills in airport screening, Psychol Science, vol. 15(5), pp , [6] A. Schwaninger and F. Hofer,. Evaluation of CBT for increasing threat detection performance in X-ray screening, in: K. Morgan and M. J. Spector, The Internet Society 2004, Advances in Learning, Commerce and Security, pp , Wessex: WIT Press, [7] I. Pollack, D. A. Norman, A non-parametric analysis of recognition experiments, Psychonomic Science, vol 1, pp , [8] A. Schwaninger and A. W. J. Wales, One year later: how screener performance improves in X-ray luggage search with computer-based training, in Proceedings of the Ergonomics Society Annual Conference, 2009, pp [9] A. Schwaninger, S. Michel, A. Bolfing, A statistical approach for image difficulty estimation in x-ray screening using image measurements, in Proceedings of the 4th Symposium on Applied Perception in Graphics and Visualization, ACM Press, New York, USA, 2007, pp [10] J. B. Grier, Nonparametric indexes for sensitivity and bias: Computing formulas, Psychological Bulletin, vol.75 (6), pp , [11] D. Aaronson and B. Watt, Extensions of Grier s computational formulas for A and B to below-chance performance, Psychological Bulletin, vol. 102 (3), pp , [12] D. George and B. Watt, SPSS for Windows step by step: A simple guide and reference, Boston: Allyn and Bacon, [13] L. Crocker and J. Algina, Introduction to classical and modern test theory, Orlando: Holt, Rinehard and Winston, VII. VITA Dr. Diana Hardmeier is the General Manager of CASRA since Prior to that she worked nearly three years at Zurich State Police Airport Division and was responsible for the development, deployment and supervision of quality control measures for airport security control. This allowed her combining both the operational and the theoretical perspectives which she acquired at the University of Zurich. During her employment at the University of Zurich, Dr. Hardmeier was project manager of testing and certification projects in Switzerland, Belgium and Germany. She received her doctoral degree in Moritz Jäger is in his master studies in the Psychological Department of the University of Zurich. He has practical experience in aviation security x-ray screening and in the field of computer-based training and testing of x-ray screeners. Rebekka Schibli graduated from Zurich University in 2009 with a Bachelor of Natural Science degree. She works for CASRA since Prof. Dr. Adrian Schwaninger lectures at the University of Zurich since 1999 and at the University of Applied Sciences Northwestern Switzerland since He is the head of the Center for Adaptive Security Research and Applications ( in Zurich and the head of the Institute Humans in complex Systems (MikS) at the School of Applied Psychology, University of Applied Sciences Northwestern Switzerland ( His areas of expertise are aviation security, human factors, scientifically based software development, applied cognitive psychology, and humanmachine interaction. Prof. Schwaninger is a member of the ECAC Training Task Force, the ECAC Technical Task Force, the ICAO Working Group on Training, and he leads the ECAC Technical Task Force TIP Study Group. Prof. Schwaninger is recognized as a leading authority of aviation security. He has more than 70 publications and more than 150 invited presentations. In 1999 he received the Young Researcher Award in Psychology. In 2003 he received the ASI International Award of Excellence in Aviation Security: Enhancement of Human Factors. 8