The deciders: How Washington evaluates vulnerabilities

WASHINGTON — In August, the National Security Agency found itself scrambling to figure out how a previously unknown group or individual, dubbed the Shadow Brokers, dumped what appeared to be several of the agency’s hacking tools online, exposing its tactics to foreign adversaries.

But it also revealed that the agency relied on several unknown vulnerabilities – known as zero-days – found in the commercial software from corporations such as Cisco and Fortinet, potentially putting at risk those companies as well as its customers. Suddenly, the unpatched flaws in their systems have been laid bare for criminal hackers and foreign spies to take advantage of for their own purposes.

While it’s unknown if the NSA may have used these tools for surveillance operations, it’s clear that the agency kept the flaws from the software vendors, depriving them from a chance to patch their systems. This conflict between the US intelligence community and the tech sector is something the government has grappled with for more than a decade. In April of 2014, White House Cybersecurity Coordinator Michael Daniel published a blog post detailing the general guidelines the government uses to decide whether to disclose or withhold a flaw in a process known as the Vulnerabilities Equities Process (VEP).

“Disclosing a vulnerability can mean that we forego an opportunity to collect crucial intelligence that could thwart a terrorist attack,” he wrote. But even Mr. Daniel recognized the potential problem of hoarding too many of these flaws, saying that “building up a huge stockpile of undisclosed vulnerabilities while leaving the internet vulnerable and the American people unprotected would not be in our national security interest.”

Daniel listed nine criteria that the agencies – which may include representatives from the NSA, CIA, FBI and Homeland Security – involved with the VEP take into account when deciding whether or not to disclose a vulnerability. The blog post says the agency that finds a vulnerability considers “how much the vulnerable system (is) used in the core internet infrastructure … in the US economy, and/or in national security systems.” The agencies also consider if the vulnerability imposes a significant risk if it is left unpatched and how much harm an adversary could do if it had information of the vulnerability.

It’s unknown how many zero-day vulnerabilities the NSA keeps for its digital espionage operations, but in 2015 it said it discloses 91 percent of the serious flaws it finds. But Jason Healey, a senior research scholar at Columbia University’s School for International and Public Affairs, says that while it’s unclear if the agency means 91 percent of 10 or 10,000, his research indicates that at most only a few dozen zero-days are in the government’s possession.

“It didn’t really seem reasonable that NSA is keeping like 5,000,” Healey said. “That means that they would be keeping so many, and we would only be discovering a tiny, tiny, tiny, tiny fraction of them.”

But despite observations that the VEP is likely “a nicely balanced system,” even Healey isn’t ready to call the numbers he uses “data.” He says that there’s not enough information to use such a label, and other experts aren’t ready to commit to the numbers given the VEP’s lack of transparency, particularly on the timeline of release: there’s no indication of whether the NSA waits two hours or two years to disclose a vulnerability.

“Nobody has any idea,” said Bruce Schneier, a noted cybersecurity researcher and cryptographer. “Well, some people do — they won’t tell you because it’s classified. So anybody who tells you that they have an idea, doesn’t know…I wish we did, but we don’t.”

Ari Schwartz, formerly one of President Obama’s top cybersecurity advisers, said that most documents relating to the VEP are classified because they are tied to national security issues. Mr. Schwartz, currently the managing director of cybersecurity services at the law firm Venable, said the exact groups involved in the VEP process can’t be disclosed because the government doesn’t want adversaries to “game the system.” But, he said, NSA heads the process and reviews the zero-days that other government agencies may uncover. But the review process isn’t restricted to the intelligence community.

“We emphasize the importance of having non-intelligence agencies as part of the process, such as the Commerce Department, the State Department and the US Trade Representative,” said Peter Swire, a professor of law and ethics at Georgia Tech University Professor, who helped craft the VEP process. “And the Commerce [Department] and Trade Representative are important because there are clearly commercial implications [of the VEP].”  

Though the NSA has stated it discloses the vast majority of the vulnerabilities it finds, the opacity of the VEP process has many privacy and security experts skeptical about its logistics and whether it adequately protects the interests of tech companies.

While tech companies have been the main opponents of the government stowing away vulnerabilities, the American public is also affected by the government’s decisions when it comes to revealing zero-days, say critics.

“We all use the same technology,” said Chris Soghoian, the principal technologist at the American Civil Liberties Union. “We all use the same laptops, we all use the same web browsers, we all use the same word processing programs.”  

Soghoian’s argument mirrors that of Apple in its feud with the government following the 2015 San Bernadino terrorist attacks. The FBI, hoping to access the shooter’s iPhone but unable to break in without a passcode, took the tech company to court for not complying with the agency’s request for a backdoor into the device. Apple CEO Tim Cook called the request “chilling” and refused to create what he called “a master key, capable of opening hundreds of millions of locks.”

In the end, Apple didn’t have to comply — the FBI purchased a zero-day to access the shooter’s iPhone from Israeli hackers, which could technically also work on any iPhone. Neither the hackers nor the FBI have told Apple what the vulnerability was, and it is unclear whether or not Apple has been able to patch the flaw.

But the civil liberties principles on which Apple based their argument against the government remain a big part of the conversation for critics of the system who think that the government is sacrificing the interests of the people for greater offensive capabilities.

“The parts of the government that are most capable of channeling the needs and interests of the American public are not even invited into the room,” said Soghoian, suggesting the FTC play a part in the VEP process. “You’re really sitting a bunch of wolves around the table asking them how you want to design the hen house.”

Even Schwartz, formerly of the Obama administration, said everyone would “be better off if there were just an unclassified version of the process.”

Schwartz recommended this past summer that the Obama administration to issue an executive order on what the process does and why, but he doesn’t expect it to happen. He said the blog post Daniel published provides a basic summary of the process, but the government needs to issue a more in-depth explanation.

“Government policy,” Schwartz said. “Especially national security policy, through a blog post isn’t the greatest practice.”