ADAPTIVE COMPUTER-BASED TRAINING INCREASES ON THE JOB PERFORMANCE OF X-RAY SCREENERS

Transcription

1 ADAPTIVE COMPUTER-BASED TRAINING INCREASES ON THE JOB PERFORMANCE OF X-RAY SCREENERS Dr. Adran Schwaninger University of Zurich Department of Psychology Visual Cognition Research Group BinzmOhtestrasse 14/22 CH-85 Zurich and Max Planck Insfitute for Biological Cybernetics Dept. BOlthoff Spemannstrasse 38 D-7276 TObingen GERMANY Dr. Franziska Hofer Zunch State Police Airport Division, Security Control Quality Management P.O. Box CH-858 Zurch-Flughafen and University of Zurich Department of Psychology Visual Cognition Research Group Binzmi)hlestrasse 14/22 CH-85 Zurich Olive E. Wetter University of Zurich Department of Psychology Visual Cognition Research Group BinzmQhlestrasse 14/22 CH-85 Zurich and Zurich State Police Airport Division, Security Control Quality Management P.O. Box CH-858 Z)rich-Flughafen Abstract - Due to severe terrorist attacks in recent years, aviation security issues have moved into the focus of politicians as well as the general public. Effective screening of passenger bags using state-of-the-art x-ray screening systems is essential to prevent terrorist attacks. The performance of the screening process depends critically on the security personnel, because they decide whether bags are OK or whether they might contain a prohibited item. Screening x-ray Images of passenger bags for dangerous and prohibited items effectively and efficiently is a demanding object recognition task. Effectiveness of computer-based training (CBT) on x-ray detection performance was assessed using computer-based tests and on the job performance measures using threat image projection (TIP). It was found that adaptive CBT is a powerful tool to increase detection performance and efficiency of screeners in x-ray image interpretation. Moreover, the results of training could be generalized to the real life situation as shown in the increased detection performance in TIP not only for trained items, but also for new (untrained) items. These results Illustrate that CBT is a very useful tool to increase airport security from a human factors perspective. Index Terms - Aviation security, x-ray screening, computer-based training (CBT), threat image projection (TIP), x-ray image interpretation. 1. INTRODUCTION Airport security screening has become an essential component in the prevention of terrorist attacks. In order to ensure securty for passengers, reliable detection of threat items is necessary in aviation security procedures. Although state-of-the-art x-ray machines provide automatic explosive detection algorithms, they still have a high false alarm rate. The last decision is always taken by a human operator (screener) who often has only a few seconds to decide whether a bag contains a prohibited item or not. Therefore, as long as humans are the last entity in the securty process at an airport, research should focus on the question how humans can be trained appropriately, in order to assure a high quality of their work. Schwaninger, Hardmeier and Hofer [1] pointed out that threat object recognition in x-ray screening depends on visual abilities and visual knowledge based on training and experience on the job. Airport security screeners need to know which items are prohibited and what they look like in x-ray images of a passenger bag These knowledge-based factors depend mainly on training and experience on the job. Schwaninger et al. [1] have also identified three image-based factors: Objects are more difficult to recognize if they are depicted from an unusual viewpoint (effect of view difficulty), when they are in a complex bag (effect of bag complexity) and when they are superimposed by other objects (effect of superposition), In order to measure how well humans can cope with these image-based factors, the X-Ray Object Recognition Test (X- Ray ORT) has been developed. As shown by [1-31, there are indeed large differences between people regarding their visual abilities needed to cope with these image based factors. Most object recognition models agree that the recognition process involves matching an intemal representation of the stimulus to a representation stored in visual memory (for an overview see [4, 5]). Hence, if a certain type of forbidden object has never been seen before, no representation in visual memory could be formed and the object becomes very difficult to recognize unless it is similar to stored views of another object. Therefore, the silver bullet solution for x-ray screeners is to train and familiarize with as many different forbidden items (e.g. guns, knives, IEDs and others) as needed to represent all different types of threat items. In agreement with that, recent research on human factors in airport security emphasizes the importance of training in general. Different studies, which have investigated the detection of threats in x-ray images of passenger bags, could /7/$ IEEE 117

2 show that sensitivity and response times improve significantly as a result of practice (e.g. [6-9]). These studies analyzed data from experimental studies [8] or from data stemming from an individually adaptive computer-based training system [6, 7, 9]. Training is only as effective as it can improve x-ray performance of screeners in the real life situation. Therefore, the main objective of this study was to investigate the effectiveness of individually adaptive computer-based training (CBT) in the real life situaton "on the job" at the airport. To this end, data from threat image projection (TIP) has been used to measure human performance increases resulting from recurrent CBT. TIP is a software function available on state-of-the-art x-ray machines. Using image processing algorithms, fictional threat items (FTIs) are projected into x- ray images of real passenger bags at the checkpoint. The FTI cannot be distinguished from a real threat item, so that TIP technology provides a valid tool to measure training effects in the real life situation. Whenever a TIP-image has been shown to the screener, an immediate feedback message is provided in order to inform screeners that the threat was fictional. In addition, a message prompts the screener to follow the appropriate security procedures, e.g. search the bag for real threats, after the TIP image disappears from the screen. In hold baggage screening (HBS) ucombined threat images" (CTIs) are used, i.e. whole bags with threat items are shown to screeners who work on a remote screening system. This method provides the possibility to project also harmless bags (combined non threat images, CNTIs) in order to measure the false alarm rate properly, i.e. the probability of a screener to rate a harmless bag as being dangerous. In this study, a TIP library with 512 CTI's was used, consisting both of x-ray images of IED's, which were not part of the CBT, and threat items, which were part of CBT. Using both kinds of IED's allows investigating whether a training effect can also be observed for non-trained IEDs. If a CBT ensures that the training effect is prominent for new threat items belonging to the same category (e.g. IEDs), training would indeed be very useful to improve airport security. For example, if a CBT contains several exemplars of guns, is this training also beneficial for the recognition of other exemplars of guns, which are not part of the CBT? In contrast, if this training effect is limited to trained exemplars only, serous concerns about the usefulness of CBT would have to be made. Evidence for the former comes from Koller, Hardmeier, Michel and Schwaninger [1], who show a clear transfer effect of training, i.e. similar looking but untrained threat objects are recognized better after the use of CBT compared to a control group. Whether such transfer effects also exist in the real working situation remains to be shown, because Koller et al. analyzed data from CBT only. This is exactly were this study ties up: Using IEDs in the TIP system, which are not part of the training, allows not only to investigate the effectiveness of CBT in the real life setting, but also to investigate the question, whether transfer effects can also be observed at the aviation security checkpoint. Smith, Redford, Washburn and Taglialatela [11 ] conducted several experiments in which they investigated whether new (untrained) exemplars of a trained category are better recognized after repeated presentation of the training exemplars by adapting the well-known 'dotdistortion paradigm' [12, 13]. Detecting prohibited items in x-ray images of passenger bags can be understood as a categorization process, whereas a potentially dangerous item has to be categorized to the corresponding threat category (e.g guns, knives, IEDs, others). In one experiment, Smith et al. used x- ray material of passenger bags [1 1]. The main finding of the mentioned study was that when new (untrained) targets were introduced after repeated presentation of training targets, performance fell significanty and dramatically compared to trained targets. In other words: they found only very small or no transfer effects for untrained exemplars belonging to the same category. The authors argued that the lack of a transfer effect could be due to the fact that participants used mainly 'category-specific' tokens and hardly 'category-general knowledge'. They observed a steady performance increase while targets repeated during testing, showing the feasibility of using familiarity and specific-token information. But when novel, unfamiliar targets were introduced, performance dropped sharply, although these unfamiliar targets were members of the trained category. If this applies to airport security screeners, the value of CBT for increasing human detection performance in x-ray screening is highly questionable. In contrast, if we can show that CBT not only improves detection of trained prohibited items, but also of new exemplars belonging to the same category, the use of CBT for increasing detection performance of screeners is recommended. 11. METHOD X-Ray Tutor (XRT) is an individually adaptive CBT based on visual cognition research [6, 7]. This system aims at training object recognition by increasing number and strength of view-based representations of prohibited items in visual memory. XRT is driven by software algorithms that monitor the trainee's performance, and adjust images presented to provide threat views, level of superposition and bag difficulties adapted to each individual trainee. Progress is based on the trainee's individual increase in performance. For a description of XRT in more detail, see [6, 7]. In order to assess visual abilities that play an important role in the recognition process relatively independent of the visual knowledge of a screener, we applied the X-Ray Object Recognition Test (ORT), which features high validity and excellent reliability [1-3]. A. Participants Originally, the study was planned with 72 HBS screeners. Data of fifty-two were complete regarding both training and TIP data and were therefore used for the analysis. Of these fifty-two screeners, thirty-seven were female. Age varied between 23.9 and 63.3 years (M = 47.9 years, SD = 8.9 years). None of the screeners had received CBT before. The original sample of 72 screeners had been divided into four groups matched for detection performance. To this end, 16 IEDs rated in the pilot study (see next section) were used, which were not part of the training library. Each x-ray image on an was combined with an x-ray image of a bag of medium and high difficulty (difficulty levels 9 and 15, estimated in the pilot study described in the next section). The whole preselection test consisted of 64 trials: 16 IEDs * 2 difficulty levels * 2 tral types (threat images vs. harmless bags). The order of image presentation was counterbalanced across screeners. The screeners' task was to decide whether the presented x-ray image contained an IED or not. Two univariate analyses of covariance (ANCOVAs) suggest that the four screener groups are equivalent in terms of detection 118

3 performance. We examined if, there is an effect of the independent variable "group" (A, B, C, D) on the dependent variable "detection performance" (d' or Am). D' and Am are parametric sensitivity measures derived from signal detection theory (e.g. [14, 151). Both measures take into account hit and false alarm rates, which is a prerequisite for a valid measurement of detection performance in terms of sensitivity. The main difference between the two estimates is that for the calculation of Am, less statistical assumptions have to be fulfilled compared to the calculation of d' (see [14, 15, 17] for details). The results suggest that there are no differences in performance between group Al (N= 12), group B (N=14), group C (N- 14) and group D (N= 12) in d' (F(3, 45) =.49; p.7) and Am (F(3, 45).6; p.98). In these analyses, age and days on job served as covariates. IIl. A. Pilot study PROCEDURE AND MATERIALS As stimuli, threat images were created by combining x-ray images of IEDs with x-ray images of passenger bags using a customized version of image blending algorithms provided by the x-ray manufacturer. 125 IEDs were used as prohibited items. The aim of the pilot study was to create sixteen CTIs (bags containing an IED) for each IED, differing in difficulty. To this end, 4 x-ray images were used, i.e. 2 harmless bag images (CNTIs) and 2 CTIs (threat images using 125 IEDs * 16 bags per IED). Subsequently, image difficulty was rated by eight expert screeners using a slider control (rating scale -1). Inter-rater reliability was estimated by calculating Cronbach's Alpha among raters. Alpha for IEDs averaged across the 16 different bags per IED was.96. Alpha for x-ray images without averaging was.82. Images were ordered by the average rated difficulty so that 16 difficulty levels were obtained per IED. B. Training library In XRT, 64 of the 125 IEDs were used. Thus, the training library consisted of 124 x-ray images containing a bag with an IED (64 IEDs * 16 bag difficulty levels) and 124 harmless x-ray images showing the same bags without IEDs. C. TIP library In TIP, 64 of the 125 IEDs rated in the pilot study were used again. Half of the IEDs were the same as in the training (Sets A and B, see section "Threat Image Projection on the job (TIP)"), whereas the other half of the IEDs were new bombs (Sets C and D). The mean difficulty of all four sets of IEDs was equivalent as shown in a pilot study. For the TIP library, eight different CTIs were constructed wth each IED. The CTI difficulties have been rated in the pilot study so that eight difficulty levels for the training library could be defined subsequently. Of each bag, an image containing an IED as well as an image without led was used. Therefore, the TIP library consisted of 124 images (64 IEDs * 8 bags per IED = 1 One participant of group A did not participate in this preselection test because she was not available during the period of testing which lasted 32 days. 512 CTIs and the corresponding 512 harmless bags, i.e. CNTIs). D. D'as measure of detection performance For a valid measure of detection performance in terms of sensitivity it is important to use a score which takes into account the hit rate (correctly judging a threat bag as NOT OK) as well as the false alarm rate (wrongly judging a harmless bag as NOT OK). Due to this reason, we chose d' as a measure of detection performance [16]. It equals z(h)- z(fa) whereas H represents the hit rate, FA the false alarm rate, and z the z-transform [14, 15]. For more information on d' and other measures of detection performance in x-ray screening see also [1 71. E. Computer Based Training (CBT) using XRT The 64 IEDs used for training were distributed among four blocks (16 IEDs each) so that all blocks were of comparable mean difficulty according to the difficulty ratings of the pilot study. Standardized measures of difficulty ratings were subjected to a one-way repeated measures analysis of variance (ANOVA) with training block as within-participant factor. This analysis confirmed that the four training blocks were of equal difficulty. There was no effect of training block, F(1.72, 12.4) =.47; p =.94 and pairwise comparisons between the training blocks showed no significant differences for any of the comparisons (all p-values >.25). One training block consisted of 512 images, i.e. 16 IEDs * 16 bags (difficulty levels) * 2 trial types (threat images vs. harmless bag images). During training, each led was first presented in its easiest difficulty level. The order of IEDs was randomized within a block across participants. The difficulty level was increased successively for each screener based on his or her achievements in training (for more information on XRT see [6, 7]). Each training session was automatically terminated after 2 min. The order of training blocks was counterbalanced across the four groups of trainees using a Latin Square design. Group A: E JE X N = 12 GroupB:: N = 14 1, Group C: I N =14:9i Group D: N = 12 ~J3j 16 bns 1o Decfian FeblMar Mar/Aprt AprilMay Tests XRT * - - n Thme TIP Ft_L Fig. 1: Illustration of the Latin Square design used in this study. A-D: Training blocks. Before each training block, a detection test (Test XRT) containing the 16 IEDs of the following block was used. TIP1-5: Illustration of the five TIP measurement dates. 119

4 Between each training block, the detection performance in XRT was measured in four testing blocks (see section 'Testing Blocks in X-Ray Tutor"). The whole study lasted about six months. On average, the 52 screeners took 31 training sessions dunng the six months period (SD = 8 sessions). During training, each x-ray image was presented for a maximum of 8 sec. Trainees had to decide whether the bag contains an IED or not by clicking one of two buttons, to which they received immediate feedback. For x-ray images containing an [ED, feedback messages were either 'Threat detected" (hit) or 'Threat missed" (miss). For harmless x-ray images, feedback messages were "False alarm" or ubag OK" (correct identification of a harmless bag). In addition, an information window showing a labeled x-ray image and a photograph of the ied could be displayed. F Testing blocks in XRT Before each of the four training blocks, participants were tested using IEDs they had never seen before. This design allows the investigation of transfer effects within the CBT. This was achieved by showing the IEDs from the next training block (see Fig. 1). In the test, each IED was presented for 4 and 8 sec in bags of the two highest image difficulty levels (15 and 16). As during training, each bag was presented without led as well in order to obtain the false alarm rate needed for the calculation of d'. Participants had to answer these images in the same way as in the training blocks described above. All four tests consisted of 128 trials: 16 IEDs * 2 display durations (4 and 8 seconds) * 2 difficulty levels (levels 15 and 16)* 2 trial types (threat images vs. harmless bags). The order of presentation was randomized. In contrast to the training blocks, no feedback and no additional information about the IEDs was available during tests. G. Threat image projection on thejob (TIP) TIP data were recorded at five different measurement dates during the study (see Fig. 1): prior to the first training block (TIPI), between each training block (TIP2, 3, and 4), and after the last training block (TIP5). The first and last TIP measurements lasted 23 days, whereas the other three TIP measurements lasted 1 days. In order to find out whether there is an effect of repetition in TIP, the 64 TIP images were divided into four different sets (set A, B, C and D) of equal difficulty. Therefore, each set contained 16 different IEDs. The IEDs of set A and B were shown during training as well: set C and D contained other IEDs, which were not part of the training. Set A and C were used in all 5 measurements, Set B and D in measurements 1 and 5 only. Using these four sets of IEDs, it is possible to investigate whether there is an effect of repetition in TIP, i.e. if repeated presentation of IEDs leads to a better TIP performance. Whenever a TIP image was shown on the workstation, the screener had to respond by pressing a button. Subsequently, there was always a feedback message presented to the screener indicating whether the answer had been correct or not. A. Training data IV. RESULTS There was a large increase in detection performance d' as shown in Fig. 2(a). In order to assess training effectiveness, we calculated percent increase values as compared to baseline measurement (first test results). Relative detection performance d' was increased by 91.54% in the 4 sec condition and 77.9a/o in the 8 sec condition (Fig. 2(b)) t: a sec --8sec _ Dec/Jan Feb/Mar Mar/Apr Apr/May C : Ca c O -.-4 sec, -*8 sec Dec/Jan Feb/Mar Mar/Apr Apr/May Fig. 2: (a) Absolute detection performance (means and mean standard errors) and (b) relative increase of detection performance (means) averaged across all 52 screeners in XRT. Display durations were 4 and 8 sec. Number of training sessions (mean) are indicated. Only p-values indicated due to security reasons. Inferential statistics confirm the effectiveness of CBT. A two-way ANOVA on d' scores with the within-participant factors test date and display duration showed significant effects of test date, F(3, 153) , p <.1, with 2 =.7 and display duration, F(1, 51) 33.95, p <.1, with r =.4. According to the classification by Cohen [181, these effect sizes are large. The interaction between test date and display duration was not significant (F(3, 153) =.11, p =.95). All Bonferroni-corrected pairwise comparisons between different test dates were significant confirming training effectiveness for the whole period of six months (all ps <.1, except the comparison between test dates 3 (Mar/Apr) and 4 (Apr/May) with p <.1). 12

5 Training, has also an effect on reaction times. Fig. 3(a) illustrates the reduced reaction times (RTs) for correct trials (hits) after training for threat bag images. This is supported by the two-way ANOVA on RTs with the within-participant factors test date and display duration. There was a main effect of test date, F(1.78, 91.4) = 44.95, p <.1, with i2. 47 and display duration, F1, 51) = 67.73, p <.1, with 12 =.57. There was also a signfficant interaction, F1.84, 93.91) = 4.95, p <.5, q =.9. Figure 3(b) illustrates the RTs for correct answers of harmless bags (correct rejections). As can be seen, no improvement for reaction times of correct rejections is observable. In contrast, RTs increased for images presented for 8 sec after the first training block. Inferential statistics show that the main effects of test date, F2.14, 19.2) = 5.5, p <.1, 12 =.1 and display duration, F(1, 51) , p <.1, 2= =.74, as well as the interaction are statistically significant F(2 62, ) = 18.52, p <.1,12=.27. co._ a: E c = 35 - Reaction Times for Hits 4 sec 8 sec Reaction Times for Correct Rejections 4~~~~~~~~~~ _ 4 sec - 8 sec a: Dec/Jan Feb/Mar Mar/April April/May Fig. 3: Reaction times (msec) for hits (a) and correct rejections (b) averaged across all 52 screeners in XRT. Display durations were 4 and 8 sec. Error bars represent standard errors. B. Effects of adaptive CBTon TIP performance We have found an effect of CBT not only in the testing blocks implemented in XRT as described above, but also in the TIP data recorded while participants were working at the screening system. There is a significant correlation between the number of images seen in XRT (representing the amount of training) and the increase in d' between TIPI and TIP5 after having partialled out the potentially confounding variables "number of images seen in TIP", participant's age, and days on job. I TABLE I PARTIAL CORRELATIONS Measure 1 Measure 2 r r,w d'tlp-d'tilp # XRT Images seen.399**.59 d'tlp5 # XRT Images seen' 52***.631 d'tip1 # XRT Images seen dtipb # TIP-images seen Partial correlations controlling for # TIP-images seen' I # XRT- 2 images seen, as well as age and days on job before and after correctng for reliability. Note. ** p <.1, *** p <.1 Controlling for the same variables, we further found a significant relationship between the amount of training and participants' performance at TIP5, whereas there was no significant relationship at TIPI. After taking into account that TIP tests are not perfectly reliable and thus applying a correction factor according to Murphy & Davidshofer [19], the significant partial correlations between amount of training and detection performance reach up to.631. As can be seen, no significant correlation could be observed between detection performance in TIP5 and the number of images seen in TIP, when partialling out participant's age and days on job. Moreover, there is no significant correlation between amount of training and performance in the ORT (r =.288, p -.52). This result indicates that visual abilities which are measured by the ORT remained relatively unaffected by CBT at least for the duration of this study. This is consistent with [1] who claimed that visual abilities which are needed to cope with view difficulty, superposition by other bags and bag complexity are less dependent on training than knowledge about which items are prohibited and what they look like in passenger bags. The same effects were found using univariate ANCOVAs on the data displayed in Fig. 4. A first ANCOVA with the between-participants factor "amount of training" (4 quartiles calculated by # of XRT images seen) revealed a large significant main effect, F(3, 45) = 4.87, p <.1, I =.25 on d'tip5. As covarates served # TIP images seen, age and days on job. Note that none of the covariates proved significant (all ps >.27). a1p5 -.-ORT :b3 2 I hu l WIQZZ z quartile 1 (little quartile 2 quartile 3 quartile 4 (much training) training) Fig. 4: Mean detection performance and mean standard errors in TIP5 and ORT. Because of security reasons, y-axis values have been multiplied with an arbitrary constant. An ANCOVA with the same factor and covarates but using detection performance before training (d'tip1) showed no significant main effect, F(3, 45) = 2.15, p =.17. None of the covariates were significant (all ps >.252). 121

6 In a third ANCOVA, we investigated the effect of "amount of training" on performance in the ORT. Covariates were the same as in the above analyses. We found no significant effect of the factor "amount of training", F(3, 42) = 1.19, p =.33, on d'ort. This analysis confirms that the increased detection performance in TIP5 after a larger amount of training is not due to differences in visual abilities. Interestingly, the covariate "age" tumed out to have an effect with F(1. 42) = 8.66, p <.1, r 2 =.17. This is consistent with a recent study by Riegelnig and Schwaninger [2] how showed that visual abilities needed to cope with View difficulty, superposition and bag complexity decrease with age. Two more ANOVAs examined the effect of the factor "test date" (TIPI, TIP2, TIP3, TIP4, TIP5) on hit rate and false alarm rate, respectively. We found a large effect (1n =.5) of "test date" on hit rate in terms of an increase (p <.1), but no effect on false alarm rate (p =.935). Detection performance of all bomb sets increased remarkably during the study. -4-Set A (trained bonbs, hgh repetition in TIP) --;Set C (untrained bombs, high repetition in TIP) USet B (trained bombs, tow repetiion intip) --Set D (untrained bombs, low repetiion in TIP) 9) I- c: a.2 9) 91. a BI Reaction Time for Hits -4--Set A (trained bombs, high repetiton in TIP) -U-Set B (trained bombs, low repetition in TIP) Set C (untrained bornbs, high repetiion in TIP) -e- Set D (urtrained bombs, low repetition in TIP) TIP I TIP 2 TIP3 TIP4 TIP5 Reaction Time for Correct Rejections -+ Set A (trained bombs, high repetition in TIP) -U-Set B (trained bombs, low repetition in TIP) -- S, -- Set C (untrained bombs, high repetition in TIP) -- Set D (untraied bombs, low repetition in TIP) 1 c E. U. c V1 T- 8 {~ r , TIP 1 TIP 2 TIP 3 TIP 4 TIP 5 Fig. 6: Reaction Times in sec. for hits (a) and correct rejections (b) for the different IED sets A, B, C and D. (means and mean standard errors) in TIP (N = 39). IT-- --For the reaction times of correct rejections (Fig. 6(b)), no I- TIPI TIP2 TIP3 TIP4 TIP 5 reduction can be observed. This is consistent with inferential Fig. 5: Absolute detection performance (meanisand mean statistics, which only reveal a significant interaction between standard errors) at the five TIP measurement dattes (N = 39). "test date" and "repetition", F(1, 38) = 7.74, p <.1 with Because of security reasons, y-axis values have been I 2_ =-.17. multiplied with an arbitrary constant. V. DISCUSSION AND CONCLUSION This can also be shown by inferential statistics. Apart from The main objective of this study was to investigate whether the effect of training, we further wanted to exannine if there computer-based training (CBT) with X-Ray Tutor (XRT) is an are transfer effects from trained IEDs to new iones and if efficient and effective training method to improve detection repetition of IEDs in TIP itself leads to bett Dr detection performance of airport security screeners. We could show, performance. To this end, we conducted a repeated that detection of improvised explosive devices (IEDs) which measures ANOVA with three factors. They weree "IED type" were never seen before increased remarkably as a result of (trained vs. untrained IEDs), "test date" (d'tip1 vs d't,p5) and training. After only 31 training sessions of 2 minutes during "repetition" (IEDs with repeated presentation in 1riP vs. IEDs six months, screeners' detection performance in terms of d' without repeated presentation in TIP). We fotmnd a large almost doubled. Using threat image projection (TIP) data we effect of "test date", F(1, 38) = 74.74, p <.1 wilth q=-.66 on found that training resulted in large detection performance detection performance and a significant interactiion between increases on the job as well. When analyzing response time "test date" and "repetition", F(1, 38) 8.3, p 112_,1 = <.1 with (RT) we found that trained screeners became faster in 7. The other two factors as well as all other interactions detecting IEDs. This is in agreement with results of McCarley were not significant (all ps >.27). [8]. They found improvements in sensitivity and response Training did not only improve detection perfoirmance, but times as a result of practice. Using eye movement analysis, also improved RTs for hits (Fig. 6(a)). For hits, Ithere was a they could also show that improvements in sensitivity were significant effect of "test date" F(1, 38) = 22.89, p <.1 with due to improved object recognition ability and not due to 12=.38. No other factor or interaction was significrant (all ps > more effective visual scanning..15). Besides the large effect of training, we found a small but reliable effect of display duration (4 sec vs. 8 sec). Performance was slightly better when images were displayed for 8 vs. 4 seconds. Note however that trained screeners achieved a much better detection performance even when 122

7 images were displayed only for 4 seconds compared to untrained screeners when images were displayed for 8 seconds. Thus, display duration is much less important than amount of training. Large effects of CBT with X-Ray Tutor were also apparent in threat image projection (TIP) data. There is a highly significant partial correlation between the amount of training and the increase in detection performance on the job over time as well as the detection performance after six months in TIP data. An ANCOVA using detection performance d' after six months training (TIP5) as a dependent variable further confirms this influence of the amount of training on detection performance on the job. The already mentioned effect of "test date" appeared also in a repeated measures ANOVA with TIP data. This analysis further revealed a transfer effect: After training, there is no effect of "old" and "new" IEDs. Detection performance for IEDs previously encountered in training is not better than detection performance for new IEDs. This implies that the acquired detection skills can easily be generalized and used for detecting new IEDs on the job. At first glance these transfer effects, which could be observed in CBT and TIP, are contradicting to the results of Smith et al. [11], who showed that improvement in screening performance is mainly attributable to specific-token familiarity and not to a category generalization. However, it might be possible that the new (untrained) IEDs in our TIP library were so similar to the trained IEDs of the CBT that a specific-token strategy could have been successful to evoke the observed transfer effect. The interaction between the factors test date and repetition of IED found in TIP suggests that repeated presentation of IEDs per se does also slightly improve detection performance in TIP. However, this effects is rather small (n2=.17) compared to the influence of CBT (nl2-66). The results of reaction times (RTs) show that CBT also proves to be a very useful tool to improve reaction times for hits in TIP. As in training, no effect of RTs could be observed for correction rejections. These results suggest that training results in faster and more reliable access to the stored representation, i. e. CBT most probably strengthens the memory representation and does not improve visual search strategies per se (otherwse reduced reaction times should be observed). Apart from knowledge-based factors that can be trained using XRT, visual abilitles (i.e. the ability to cope well with image-based factors) affect detection performance as well (e.g. [1, 3]). These can be measured by the X-Ray Object Recognition Test (ORT) and are thought to be relatively constant over time and relatively independent of training [1]. Consistent with this assumption, no relationship between amount of training and performance in ORT was observed, confirming that the increased detection performance on the job is not due to an increase visual abilities needed to cope with image-based factors such as rotation of objects (view difficulty), superposition by other objects, and bag complexity. However, it is an open question whether visual abilities measured with the ORT can be increased through more recurrent training, e.g. over two years. We are currently investigating this and preliminary results indicate that indeed it is possible to increase visual abilities when XRT is used for weekly recurrent training over years. Participants' age and working experience have turned out to be of no influence concerning the increase in detection performance achieved by six months of training with X-Ray Tutor. However, participants' age contributed significantly to performance in the ORT in the way that with increasing age, performance decreases slighty. This is consistent with a recent finding reported by Riegelnig and Schwaninger [21 who found that visual abilities measured by the ORT decrease slightly with age. However, it should also be mentioned that Ghylin, Drury and Schwaninger [91 found that effects of age are about 5 times smaller than the effets of computer-based training on screener detection performance. Thus, while there is a small decrease in visual abilities with age, this decrease can be fully compensated by slightly more computer-based training. In summary, this study clearly showed that weekly computer-based training with X-Ray Tutor results in large increases of screener detection performance in x-ray screening. In addition, screeners can detect faster when there is a threat in an x-ray image of a passenger bag. Vl NOTE All d' values have been multiplied by an arbitrary constant for security reasons. VIl. ACKNOWLEDGEMENT This research was financially supported by the European Commission Leonardo da Vinci Programme (VIA Project, DE/6/C/F/TH-843). VUIL REFERENCES [1] Schwaninger, A., Hardmeier, D., & Hofer, F. (25). Aviation Security Screeners Visual Abilities & Visual Knowledge Management. IEEE A&E Systems Magazine, June 25, [2] Hardmeier, D., Hofer, F., & Schwaninger, A. (25). The x-ray object recognition test (x-ray ort) - a reliable and valid instrument for measuring visual abilities needed in x-ray screening. IEEE ICCST Proceedings, 39, [31 Hardmeier, D., Hofer, F., & Schwaninger, A. (26a). Increased detection performance in airport security screening using the x-ray ort as pre-employment assessment tool. Proceedings of the 2nd Intemational Conference on Research in Air Transportation, ICRAT 26, Belgrade, Serbia and Montenegro, June 24-28, 26, [41 Graf, M., Schwaninger, A., Wallraven, C., & Bulthoff, H.H. (22). Psychophysical results from experiments on recognition & categorisation. Information Society Technologies (1ST) programme, Cognitive Vision Systems - CogVis; IST [51 Schwaninger, A. (25a). Object recognition and signal detection. In B. Kersten and M.T. Groner (Eds.), Praxisfelder der Wahmehmungspsychologie (pp ). Bem, Switzerland: Huber. [6] Schwaninger, A. (23c). Training of Airport Security Screeners. Airport, 5, [71 Schwaninger, A. (24b). Computer based training: a powerful tool to the enhancement of human factors. Aviation Security Intemational, FEB/24, [8] McCarley, J.S., Kramer, A.F., Wickens, C.D., Vidoni, E.D., & Boot, W.R. (24).Visual Skills in Airport- Security Screening. Psychological Science, 15,

8 [9] Ghylin, K.M., Drury, C.G., & Schwaninger, A. (26). Two-component model of security inspection: application and findings. le^ World Congress of Ergonomics, lea 26, Maastricht, The Netherlands, July, 1-14, 26. [1] Koller, S.M., Hardmeier, D., Michel, S., & Schwaninger, A. (27). Investigating training and transfer effects resulting from recurrent CBT of x-ray image interpretation. Proceedings of the 29th Meeting of the Cognitive Science Society, [111 Smith, J.D., Redford, J.S., Washburn, D.A:, & Taglialatela, L.A. (25). Specific-Token Effects in Screening Tasks: Possible Implications for Aviation Security. Joumal of Expermental Psychology: Learning, Memory, and Cognition, 31, [12] Posner, M. I., Goldsmith, R., & Welton, K. E. (1967). Perceived distance and the classification of distorted patterns. Joumal of Experimental Psychology, 73, [13] Posner, M. I., & Keele, S. W. (1968). On the genesis of of abstract ideas. Journal of Experimental Psychology, 77, [14] Green, D.M., & Swets, J.A. (1966). Signal detection theory andpsychophysics. New York, USA: Wiley. [15] MacMillan, N.A., & Creelman, C.D. (1991). Detection theory: A user's guide. Cambridge, England: Cambridge University Press. (161 Schwaninger, A. (23a). Reliable Measurements of Threat Detection. Airport, 1, [171 Hofer, F. & Schwaninger, A. (24). Reliable and valid measures of threat detection performance in X-ray screening. IEEE ICCST Proceedings, 38, [18] Cohen, J. (1988). Statistical power analysis for the behavioural sciences. New York: Erlbaum, Hillsdale. [191 Murphy, K.R., & Davidshofer, C.O. (21). Psychological Testing: principles and applications. Upper Saddle River, USA: Prentice Hall. [21 Riegelnig, J., & Schwaninger, A. (26). The influence of age and gender on detection performance and the criterion in x-ray screening. Proceedings of the 2nd International Conference on Research in Air Transportation, ICRAT 26, Belgrade, Serbia and Montenegro, June 24-28, 26,