In the past 25 years data mining has grown from a mere gleam in the eye of some scientists to an everyday occurrence, in the commercial sphere and by governments around the globe. Dozens of U.S. government programs abound. This project has identified those programs with a nexus to national security issues, but the same techniques are used daily by other agencies of the government (for example, by the IRS to detect patterns of fraudulent tax filings). The growth of data mining is likely to increase exponentially in the coming years.
So what does the future hold? What sorts of programs might we see in the next few years? Here is a sampling:
Geo-location Techniques – Programs that collect data about an individual’s physical location and use that data to identify patterns in his or her behavior.
Video Analysis – What about programs that link facial recognition technology to pattern analysis systems that can discern facial patterns of anxiety, or recognize the same face in different places?
DataSphere/Knowledge Discovery and Dissemination/Catalyst – Programs that allow intelligence analysts to sift through large volumes of data to find patterns of terrorist activity that were previously not recognized.
Babel – We are quickly developing programs that do automated translation of multiple languages. Deploying this technology will help solve one of the hardest problems in data mining – the correlation of disparate data bases and different data forms.
Crowdsourcing – One of the more innovative ways of doing data analysis is to harness the power of the web through crowdsourcing. The Navy has already begun using crowdsourcing to fight piracy. Imagine a world in which average citizens helped the government analyze threat data – or data about political opponents.