U.S. Government Accountability Office Reports

The U.S. Government Accountability Office (GAO) is an independent, nonpartisan congressional agency that serves a watchdog and investigative function. Over the years, the GAO has issued various reports on government data mining in pursuit of national security objectives, the attendant privacy concerns and related issues. Below, we’ve included some of the pertinent highlights of the agency’s work.

U.S. Agencies Should Systematically Assess Terrorists’ Use of Alternative Financing Mechanisms (November 2003) — An analysis of the varied “alternative” means by which terrorists acquire, transfer and store money to finance their activities. The report offers a glimpse into the types of financial data that is likely tracked by government data mining programs in efforts to detect or deter terrorists and their supporting organizations – a capability greatly enhanced following the attacks of Sept. 11 by the USA PATRIOT Act.

Federal Efforts Cover a Wide Range of Uses (May 2004) — Presents the first major assessment of government data mining initiatives produced by the GAO. Their research uncovered 199 data mining efforts across 52 federal departments and agencies, of which 29 programs were specifically intended to serve crime fighting or counterterrorism purposes. The GAO undertook the investigation at the request of Sen. Daniel Akaka, who asked that the oversight body “identify and describe operational and planned data mining systems and activities in federal agencies.”

Radio Frequency Identification Technology in the Federal Government (May 2005) — As noted in this report, radio frequency identification (RFID) is used or contemplated in the execution of various governmental functions. The technology allows electronic readers to track objects using radio frequency waves via embedded “tags.” For example, all U.S. passports issued beginning in 2006 have included RFID tags, enabling the tracking of citizens’ international travel. As data storage capabilities increase alongside decreasing computing costs, RFID is an increasingly economical means of tracking shipments, facilitating travel and potentially, gathering intelligence to be data mined.

Agencies Have Taken Key Steps to Protect Privacy in Selected Efforts, but Significant Compliance Issues Remain (August 2005) — A follow-up to the GAO’s broad assessment of government data mining efforts in May 2004, focusing on five specific programs. Among the agencies that hosted the programs, GAO found varying degrees of compliance to Privacy Act, Federal Information Security Management Act and E-Government Act requirements. The report concludes that “The agencies responsible for these data mining efforts took many of the key steps required to protect the privacy and security of the personal information they used. However, none of the agencies followed all the key privacy and security provisions we reviewed.”

Agency and Reseller Adherence to Key Privacy Principles (April 2006) — Examines the extent to which government agencies and the data resellers that they occasionally contract with adhere to Fair Information Practices, standards that agencies are expected to observe in collecting the personal information of U.S. citizens. In 2005, 91 percent of data purchased by the federal government went toward law enforcement or counterterrorism purposes. The report found that adherence by resellers was often “largely at odds with the nature of the information reseller business,” and found uneven compliance by government agencies using purchased data as well.

Treasury’s Roles and Responsibilities Relating to Selected Provisions of the USA PATRIOT Act (May 2006) — Assesses the role of the Department of Treasury in combatting terrorism financing, specifically through the department’s Financial Crimes Enforcement Network (FinCEN) program and by assisting foreign partners in establishing similar financial intelligence units.

The Federal Government Needs to Establish Policies and Processes for Sharing Terrorism-Related and Sensitive but Unclassified Information (May 2006) — A lack of information sharing across departments and agencies was considered one of the greatest failures in the lead up to the Sept. 11 attacks. In the decade since, agencies have increasingly sought to remedy the intelligence sharing deficit. As more information becomes accessible to more agencies, an increase in data mining efforts across disparate and diverse data sets seems inevitable.

Opportunities Exist to Enhance Collaboration at 24/7 Operations Centers Staffed by Multiple DHS Agencies (October 2006) — Looks at the inter-agency collaborative process at four 24/7 DHS National Operations Centers charged with broad security responsibilities, all of which include representatives from multiple DHS components and agencies outside the department. The four centers have distinct but similar national security roles, from monitoring entrants into the United States to coordinating activities with state and local entities. One of several GAO recommendations advises the centers to develop a more compatible network framework for sharing data across agencies.

Early Attention to Privacy in Developing a Key DHS Program Could Reduce Risks (February 2007) — Examines the DHS program known as Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement (ADVISE). The program in many ways picked up where Total Information Awareness left off. GAO was tasked with identifying the program’s features, goals and attendant privacy concerns. The GAO cited several privacy concerns in the report and made recommendations aimed at mitigating potential privacy abuses, but after investing more than $40 million in its development, DHS abandoned ADVISE later that year.

Progress Made but Challenges Remain in Notifying and Reporting to the Public (April 2007) — Evaluates the DHS Privacy Office’s efforts to comply with privacy laws pertaining to the use of U.S. citizen data. The GAO noted “substantial progress” made in some areas, but also found room for improvement, including the need to update the System of Records notices for databases containing citizens’ personal information, and in ensuring the timely issuance of statutorily mandated reports.

Alternatives Exist for Enhancing Protection of Personally Identifiable Information (May 2008) — Addresses the federal government’s efforts and ability to effectively safeguard citizens’ personal information in light of “the increasingly sophisticated ways in which personally identifiable informationis obtained and used.” The report urged Congress to consider revisiting the existing privacy law architecture and recommended specific areas where protections could be improved.

Agencies Should Ensure That Designated Senior Officials Have Oversight of Key Function (May 2008) — Evaluates the adequacy of the roles and responsibilities of 12 federal agencies’ designated privacy officers. The report found varied structures and personnel responsibilities in addressing privacy concerns among the agencies, and recommended that at least half the assessed agencies grant greater oversight of privacy issues to the senior agency official who holds the privacy officer title.

Department of Defense

An information sharing application run by NCIS. It “provides participating law enforcement partner agencies with secure access to regional crime and incident data and the tools needed to process it, enabling investigators to search across jurisdictional boundaries to help solve crimes and resolve suspicious events.” There are more than 760 NCIS LInX partner agencies with approximately 30,000 trained users.

The Department of Defense (DoD) is collaborating with NCIS to introduce the Defense Law Enforcement Data Exchange (DDEX) project to expand upon the success of this program to include thirteen DoD agencies. Similar to LInX, the DDEX will be populated using data already collected by the defense criminal investigative organizations in fulfilling their functions, such as incident, offense, and case reports.

This rich piece of descriptive obfuscation comes from the PIA: “KNET provides integrated analytical and knowledge management capabilities across multiple internal and external sources of criminal investigative, counterintelligence, and counterterrorism. More specifically, KNET consists of tools that allow users to search unclassified disparate data sources from one common interface, perform entity extraction of key objects and concepts contained in message traffic and databases, visualize relationships between objects, collaborate, and report.”

Documents “intelligence, counterintelligence, counterterrorism and counternarcotic operations relating to the protection of national security, DoD personnel, facilities and equipment, to include information systems.” It includes individuals “involved in, or of interest to, DoD intelligence, counterintelligence, counterterrorism and counternarcotic operations or analytical projects as well as individuals involved in foreign intelligence and/or training activities.”

Department of Homeland Security

Designed to support the sharing of information through the Information Sharing Environment about suspicious activities which are defined as “official documentation of observed behavior reasonably indicative of pre-operational planning related to terrorism or other criminal activity [related to terrorism].”

Aggregates data from other sources and uses classic logarithmic analysis to look for “non-obvious relationships”; also uses commercial data to fill in information gaps.

A repository for reports generated to record and track suspicious activity that may implicate terrorism-related or criminal activity.

Establishes guidelines for how publicly available info on social media websites can be used to provide “situational awareness,” defined as “information gathered from a variety of sources that, when communicated to emergency managers and decision makers, can form the basis for incident management decision-making.”

Allows users to search across multiple TSA databases in support of its mission to assess risks and threats to the nation’s transportation infrastructure.

Department of Justice

A repository for tracking bomb/explosive device related incidents. Aggregates data on these incidents from law enforcement agencies across the country.

DEA Aggregates data on criminals, suspected criminals and other “criminal law enforcement information.”

Aggregates SARs received by the FBI, allows for the dissemination of this information to state and local law enforcement. “eGuardian is at its very essence, simply a platform to standardize the disparate SAR systems currently utilized by agencies to collect information, which will enhance communication among law enforcement entities as well as situational awareness.”

The Investigative Data Warehouse holds as many as 6 billion records, from phone and hotel records to “pocket litter” found in suspects’ pockets during detainment situations. Along with a Foreign Terrorist Tracking Task Force database, the two programs have been integrated to comprise what is termed the National Security Branch Analysis Center’s “data exploitation system.”

Creates a database allowing citizens to submit “tips” on terrorism-related or criminal activity. DHS also has a tip line, which forwards all information received to the FBI.

Another information-sharing database for federal, state and local law enforcement. The PIA money line: “The N-DEx system will make apparent linkages between already existing law enforcement information that were previously not apparent.”

Office of the Director of National Intelligence


Given the nature of the ODNI programs, little is known about them and most of what we know suggests that they are “future” programs rather than current efforts:

“Will enhance data fusion and entity resolution, as well as discovery of unknown relationships. DataSphere, enables analysis of the activities of terrorists such as their communication networks and travel. The goal of DataSphere is to provide analysts with a tool to aid in the discovery of unknown terrorism relationships and the identification of previously undetected terrorist and terrorism information.

In its end-state, Catalyst will enable data fusion/analytic programs to share disparate repositories with each other, to disambiguate and cross-correlate the different agencies’ holdings, and to discover and visualize relationship/network links, geospatial patterns, temporal patterns and related correlations.”

The focus of the KDD program is to develop novel approaches that will enable the intelligence analyst to effectively derive actionable intelligence from multiple, large, disparate sources of information, to include newly available data sets previously unknown to the analyst.

Will attempt to figure out how to employ pattern analysis to video data. The full synopsis from the link below is worth a look; we’re basically talking about YouTube spying here.

Comments are closed.