By Steve Wolff
There seems to be trouble in Aviation Security Paradise and it has been coming out into the open (or at least into the press) over the past few months. It started with a couple of shots aimed our way but was most recently put front and centre by the terrible attacks in Paris on Friday 13th November, which will have large ramifications for aviation as well as other security modalities.
It has seemed to me – and to others – that the past few years of relative calm has led to a dialling back of our commitment to enhance security in favour of improving passenger facilitation. As has so often happened in our industry, the pendulum seems once again be starting to swing the other way. The opening salvo was the recent US Department of Homeland Security Inspector General Red Team report, which, if you’re not familiar with it, shared the unclassified results of running 70 covert tests through different US checkpoints to see how well the system was able to find them. The answer: very poorly. While modelling the end-to-end detection of various checkpoint configurations during the original IATA Checkpoint of the Future (the initiator of Risk Based Screening and PreCheck) development, we predicted that the checkpoint process would be able to find somewhere between 17% and 31% of the broad threat range out there (guns, knives and explosives). It turns out that we were wildly optimistic, at least with respect to the US. The recent IG report and subsequent testimony by the latest TSA leadership to the US Congress shows that there is still much work to do to achieve not just a cost- and operationally effective security system but also some measure of basic effectiveness for the billions of dollars spent.
The second salvo aimed at aviation was the destruction of MetroJet 9268 in Sharm el-Sheikh, which is increasingly looking like an act of sabotage. While the investigations are still ongoing as I write this, it appears that MetroJet 9268 is another illustration that technology alone, without well-established staffing, training, operations and auditing programs will not address the challenges we face. Interestingly, another article appeared as I started writing this piece: namely that the Egyptian military had ‘reverse engineered’ the ADE 651, the divining rod of bomb detection that has been the subject of fraud trials in the UK for the past two years, and was using it to ‘screen’ checked bags at hotels.
As I was absorbing all this and discussing the situation with another consultant, I ended up going through an interesting, and worrying thought analysis. I thought I’d share it and use it to illustrate a point, even though it’s likely to be controversial. So here goes. The US DHS Inspector General determined that the current TSA checkpoint had a missed detection rate of 95%. The ADE 651 has a missed detection rate of 100% of course; from a detection perspective it’s worthless. However, if one were to combine the ADE with a behaviour detection officer aimed at observing the reaction of the passenger being screened and sending any passengers that behaved strangely to additional search, the overall process’s detection would improve (at least it couldn’t be worse) even though the ADE on its own is worthless. In the discussion, it prompted the question: could such a combined system actually be as effective as the TSA’s measures from an end-to-end perspective? At least based on the Red Team results, it wouldn’t have too far to go in that regard; a 5% end-to-end detection rate (combining behaviour detection with the ADE) would, mathematically at least, put it on par with all that technology and people deployed across the US (and potentially beyond). While this analysis is perhaps farfetched, it illustrates a point that the Checkpoint of the Future development team realised was important back in 2010-11 while working with IATA: the importance of both predicting and measuring the total system-wide performance of the security process, which is of course, what the Red Team (or any similar covert testing) does.
However, if you look through the various requirements for technology solutions coming out of TSA, DHS and the European Union, they are still focused on individual boxes – both from a development and from a testing perspective. The industry even has parlance for the maturity of these boxes: the Technology Readiness Level or TRL. TRL levels run from 1 (a bit more than a twinkle in someone’s eye) to 9 (the manufacturers’ Holy Grail: products ready to sell). However, as we’ve repeatedly seen over the years, there remains a disconnect between what the lab tests say the systems should be capable of detecting and what the covert testers actually find the process is capable of detecting. The recent IG report is nothing new; in my opinion, it does not reflect on the current crop of government regulators, managers, system developers or users as much as the underlying system we have in place, which falls short when it comes to developing and implementing truly effective security measures. Essentially we run the marathon but then stop a few hundred meters before the finishing line.