The law of large numbers

It is shocking, apparently, to hear that the Transportation Security Administration has had more than 25,000 security breaches since 2001, or roughly 2,500 breaches every year.  Representative Jason Chaffetz (R-UT) thinks its a scandal that proves that airport security is “a mess.”  (And, now, apparently, Congressman Chaffetz and TSA are in a tussle about whether or not he was authorized to release these numbers).

But what’s really a mess is how our Representatives (and, sometimes, the press) report these sorts of numbers.  They are always portrayed as absolute values and in that abstract context they seem immense.  Who, after all, could approve of 2,500 mistakes per year?

But the abstract context is just that — abstract.  Numbers have meaning only in a concrete context.  So how about this for context:  Domestically, there are approximately 2 million enplanements (passengers boarding aircraft) every day.   That’s roughly 700 million passengers a year, or 7 billion passengers in the 10 years for which the security breach data are reported (and bear in mind that this is every security breach however minor).  That’s an error rate of less than 0.0001%.  In what human endeavor is that considered a poor performance?

So … absolute numbers mislead.  What we really need to know is what the types of breaches are, how systematic the vulnerabilities are and what, if anything, TSA is doing to fix the issues identified?  Those are interesting questions.  Talking about numbers out of context is not.


Comments are closed.