(Updated 11/17/13) Google’s somewhat-delayed update to its semi-annual transparency report was released this morning and we’ve just started crunching the numbers, but thought we’d share a handy visual overview that Google Blog posted in the interim, along with a few of our own graphics.
Quick summary: The government requests for user accounts info and user data just keep on coming, particularly in the U.S. They have doubled since 2010 — 25,900 requests in the first half of 2013 compared to 13,400 in the same period of 2010. And the US now makes up 42% of all requests, compared to 32% in 2010.
On the bright side, the percentage of U.S. requests that eventually led to some data being released has been steadily declining. It was 83% in the first half of this year and 94% at the end of 2010. The type of request that had the highest chance of leading to data being released: A wiretap order (100% in 7 cases). The least likely: A court order that wasn’t a subpoena or search warrant, with 69% of cases ending with data release.
And here’s a chart we at OTB put together to show how the authorities come knocking.
And now that Google has weighed in, here is a summary of all the major companies that are now issuing transparency reports.
Very nice interactive infographic/backgrounder on the NSA controversy and issues, put together by the Guardian. Click on image below to begin.
Freedom of Information Act requests filed with the National Security Agency have boomed since Edward Snowden began leaking top-secret documents in June.
The requests have leveled off somewhat from earlier in the summer when the first media leaks appeared, although they continue to be much higher than normal. For perspective, for all of FY12 we received only 1809 requests,” the NSA said in its memo.
U.S. government agencies asked Microsoft to share customer information or content nearly 8,000 times in the first half of 2013. Those requests involved 22,300 accounts, new data released by the Seattle-based firm on Friday said.
Based on so-called “transparency” reports released so far this year from Microsoft, Yahoo, Twitter and Facebook, US agencies between January and the end of June made, on average, up to 183 requests per day across the four companies, for a total of 33,338 requests affecting 84,597 accounts.
Google, which has been releasing law enforcement request data longer than any major competitor, has yet to release its data for the first half of the year. Microsoft first released some data early last summer after former NSA employee Edward Snowden began leaking top-secret documents about government surveillance programs involving the major internet companies.
Turkey, Germany, the United Kingdom and France followed the U.S. in the No. 1 spot for requests from Microsoft. Together, the five made up 3 in 4 of the global 37,196 requests affecting 66,539 accounts.
Just over 1 in 10 U.S. requests led to user content being released. In 2 out of 3 cases, at least some user account information such as name, gender and Zip Code was turned over. Only about 1 in 100 was rejected for not meeting legal requirements. And in just under 1 in 20 cases — 17% — no customer data was found.
Yahoo on Friday reported that U.S. authorities asked for user data 12,444 times in the first six months of this year — covering 40,322 accounts. That is 69 requests a day, on average.
Most of those cases resulted in the government getting at least some data, including e-mails, photos and uploaded files.
Friday’s was Yahoo’s first-ever transparency report and it says it will continue every six months. Twitter and Microsoft released their reports earlier this summer; Google, which has been releasing reports longer than the other major players, has yet to for the first half of 2013. (Related: The Washington Post reported this morning that Google has stepped-up its efforts to encrypt that data that moves between its servers in an attempt to thwart spying).
“Democracy demands accountability, and accountability requires transparency,” Yahoo General Counsel Ron Bell wrote in a blog post on Friday. “We hope our report encourages governments around the world to more openly share information about the requests they make for users’ information.”
Yahoo reported that in about 8% of cases, either no data was found or Yahoo rejected the request. So in just over 9 in 10 cases, at least some data was turned over.
About half the time — 55% — that was “non-content data,” which Yahoo describes as “basic subscriber information including the information captured at the time of registration such as an alternate e-mail address, name, location, and IP address, login details, billing information, and other transactional information (e.g., “to,” “from,” and “date” fields from email headers).”
In nearly 2 in 5 cases, other content was turned over. Yahoo’s description: “Data that our users create, communicate, and store on or through our services. This could include words in a communication (e.g., Mail or Messenger), photos on Flickr, files uploaded, Yahoo Address Book entries, Yahoo Calendar event details, thoughts recorded in Yahoo Notepad or comments or posts on Yahoo Answers or any other Yahoo property.”
Yahoo reported about the same number of government requests as Facebook, but affecting substantially more user accounts — 40,300 vs. up to 21,000. Twitter reported 902 requests affecting 1,319 accounts for the first half of the year.
Germany, Italy, Taiwan and France filled out the Top 5 in number of requests after the U.S. Outside the U.S., requests totaled 17,026, involving 22,453 accounts.
The government requests usually involve criminal investigations and came come by way of warrant or subpoena. Yahoo says it only complies “in response to valid, compulsory legal process from a government agency with proper jurisdiction and authority.”
National security authorities also make the requests. All companies are restricted about how much they can say — even specific numbers — about requests under the Foreign Intelligence Surveillance Act. Yahoo and others have been pressing the government to allow them more freedom to divulge those details.
The report covers the first half of 2013 and says the U.S. made between 11,000 and 12,000 requests involving between 20,000 and 21,000 users or accounts. Facebook complied with 79% of those requests. The range for the U.S. vs. a specific number for other countries is believed to be because of U.S. requirements that requests involving national security can only be released in ranges. Facebook and several of its rivals are urging the government to allow specific numbers be released instead of ranges.
“We continue to push the United States government to allow more transparency regarding these requests, including specific numbers and types of national security-related requests. We will publish updated information for the United States as soon as we obtain legal authorization to do so,” Facebook General Counsel Colin Stretch said in releasing the data.
Facebook’s data seems to indicate a significant drop-off in the number of users or accounts from the second half of 2012. In first-ever data it released after the PRISM program details were first leaked by Edward Snowden earlier this year, it said it had received 9,000-10,000 requests involving 18,000-19,00 users or accounts between July 1 and Dec. 31, 2012. (See a table that summarizes data reports from a variety of companies).
Globally, Facebook said it has up to 26,607 requests involving 38,954 users in about 70 countries. The average compliance rate was 33%. India (3,245 requests and 4,144 users/accounts) and the United Kingdom (3,245/4,144) were No. 2 and No. 3 behind the U.S.
“The vast majority of these requests relate to criminal cases, such as robberies or kidnappings,” Facebook’s Stretch said. “In many of these cases, these government requests seek basic subscriber information, such as name and length of service. Other requests may also seek IP address logs or actual account content. We have strict guidelines in place to deal with all government data requests.”
Facebook’s compliance rate is higher than Twitter’s (it reported 67% a few weeks ago) and is about the same as Microsoft, which reported about 80% for 2012, and is lower than Google’s most recently reported 88%, which itself was down from 94%.
→ Earlier stories on transparency reports.
ON THE JUMP: A sortable chart with all the Facebook data.