The United States government uses many data mining programs (defining that term broadly to include link analysis). Most of them have nothing to do with national security or homeland security – there is, for example, a program that sifts through health data to spot outbreaks of influenza.
Those programs are interesting, of course, but the more interesting (and, to some, troubling) uses of data mining and link analysis occur in the field of national security. Ever since the Total Information Awareness program was first disclosed in the press, the public has given increased scrutiny to data mining programs that collect personally identifiable information and use it to thwart terrorists and criminals.
In this section of the project, we outline some of the other national/homeland security data mining programs being operated by the Federal government. Many of these are under-reported (some have never been publicly examined in the press) and they are potentially worthy of investigative reporting.
Most such programs today can be found in the Department of Homeland Security, the Department of Defense, the Department of Justice, the Department of the Treasury, and in the Intelligence Community (acting through the Office of the Director of National Intelligence). Additionally, the independent Government Accountability Office has published a number of reports on data mining conducted by federal agencies.
Department of Defense
An information sharing application run by NCIS. It “provides participating law enforcement partner agencies with secure access to regional crime and incident data and the tools needed to process it, enabling investigators to search across jurisdictional boundaries to help solve crimes and resolve suspicious events.” There are more than 760 NCIS LInX partner agencies with approximately 30,000 trained users.
The Department of Defense (DoD) is collaborating with NCIS to introduce the Defense Law Enforcement Data Exchange (DDEX) project to expand upon the success of this program to include thirteen DoD agencies. Similar to LInX, the DDEX will be populated using data already collected by the defense criminal investigative organizations in fulfilling their functions, such as incident, offense, and case reports.
This rich piece of descriptive obfuscation comes from the Privacy Impact Assessment (PIA): “KNET provides integrated analytical and knowledge management capabilities across multiple internal and external sources of criminal investigative, counterintelligence, and counterterrorism. More specifically, KNET consists of tools that allow users to search unclassified disparate data sources from one common interface, perform entity extraction of key objects and concepts contained in message traffic and databases, visualize relationships between objects, collaborate, and report.”
Documents “intelligence, counterintelligence, counterterrorism and counternarcotic operations relating to the protection of national security, DoD personnel, facilities and equipment, to include information systems.” It includes individuals “involved in, or of interest to, DoD intelligence, counterintelligence, counterterrorism and counternarcotic operations or analytical projects as well as individuals involved in foreign intelligence and/or training activities.”
Department of Homeland Security
Designed to support the sharing of information through the Information Sharing Environment about suspicious activities which are defined as “official documentation of observed behavior reasonably indicative of pre-operational planning related to terrorism or other criminal activity [related to terrorism].”
Aggregates data from other sources and uses classic logarithmic analysis to look for “non-obvious relationships”; also uses commercial data to fill in information gaps.
A repository for reports generated to record and track suspicious activity that may implicate terrorism-related or criminal activity.
Establishes guidelines for how publicly available info on social media websites can be used to provide “situational awareness,” defined as “information gathered from a variety of sources that, when communicated to emergency managers and decision makers, can form the basis for incident management decision-making.”
Allows users to search across multiple TSA databases in support of its mission to assess risks and threats to the nation’s transportation infrastructure.
Department of Justice
A repository for tracking bomb/explosive device related incidents. Aggregates data on these incidents from law enforcement agencies across the country.
DEA aggregates data on criminals, suspected criminals and other “criminal law enforcement information.”
Aggregates Suspicious Activity Reports (SARs) received by the FBI, allows for the dissemination of this information to state and local law enforcement. “eGuardian is at its very essence, simply a platform to standardize the disparate SAR systems currently utilized by agencies to collect information, which will enhance communication among law enforcement entities as well as situational awareness.”
The Investigative Data Warehouse holds as many as 6 billion records, from phone and hotel records to “pocket litter” found in suspects’ pockets during detainment situations. Along with a Foreign Terrorist Tracking Task Force database, the two programs have been integrated to comprise what is termed the National Security Branch Analysis Center’s “data exploitation system.”
Creates a database allowing citizens to submit “tips” on terrorism-related or criminal activity. DHS also has a tip line, which forwards all information received to the FBI.
Another information-sharing database for federal, state and local law enforcement. The PIA money line: “The N-DEx system will make apparent linkages between already existing law enforcement information that were previously not apparent.”
Office of the Director of National Intelligence
Given the nature of the ODNI programs, little is known about them and most of what we know suggests that they are “future” programs rather than current efforts:
“Will enhance data fusion and entity resolution, as well as discovery of unknown relationships. DataSphere enables analysis of the activities of terrorists such as their communication networks and travel. The goal of DataSphere is to provide analysts with a tool to aid in the discovery of unknown terrorism relationships and the identification of previously undetected terrorists and terrorism information.
In its end-state, Catalyst will enable data fusion/analytic programs to share disparate repositories with each other, to disambiguate and cross-correlate the different agencies’ holdings, and to discover and visualize relationship/network links, geospatial patterns, temporal patterns and related correlations.”
The focus of the KDD program is to develop novel approaches that will enable the intelligence analyst to effectively derive actionable intelligence from multiple, large, disparate sources of information, to include newly available data sets previously unknown to the analyst.
Will attempt to figure out how to employ pattern analysis to video data. The full synopsis from the link below is worth a look; we’re basically talking about YouTube spying here.
