Thursday, May 05, 2011

Twitter Prohibits Research on Osama Bin Laden Tweets...

Here's a quick news story that should be tossed into the "ridiculous" category.

Academic researchers are constantly data mining social media websites to collect information. This can be extremely useful in analyzing trends and other metrics.

So after the news broke on Sunday night that Osama Bin Laden had been killed, some researchers thought it might be valuable to analyze the thousands of tweets referring to the story. I was personally emailed a link to an archive of such tweets in XML format to be used in conjuction with DiscoverText software.

The datafiles were samples taken from live feed Twitter imports starting shortly after the announcement that Osama bin Laden’s death.

  • Twitter searches for "bin laden" (647,585 documents, 505 MB)
  • Twitter searches for "osama" (586,665 documents, 451 MB)

This was all for research purposes, however Twitter quickly shut down the project citing their Terms of Service (TOS) Agreement.

I was notified of the shut-down in a follow-up email that reiterated...

To be clear: we were giving the data away, not selling it. Also, it was not scraped of Twitter. Rather, it was gathered using a Twitter-authorized account and an API that lets us fetch 1500 items at a time. It is a shame that the now 2 million tweets cannot, for example, be sampled and coded using a crowd source model.

Stuart Shulman is exactly right - this is prime historical data and there is no conceivable reason why Twitter would need to prohibit the aggregation of such data for non-commercial research purposes.

Someone over there needs to implement a little common sense.

Fascinated by policy and law? Learn more at law degrees online.


At 9:53 AM, Anonymous Anonymous said...

You should really donate your collection to talk to Jason Scott at (on twitter @textfiles). If you check Internet Archive for items by ArchiveTeam, you can see it would fit in very nicely there.

At 4:29 PM, Anonymous Rickesha Whyte said...

I remembered when the announcement was made of the capture of Bin Laden by U.S militia. This was a very interested and excited night because most of my family members and friends all had something to say about what was said and how all of this unfold perfectly right before elections. Usually I stay out of politics and I keep my opinion for myself because most of the time, people will voice their own opinion on politics and stick with it, so most time I think its a waste of time to even bother debating.

However, on the night that President Obama announced the capture of Osama, my opinion kept on flying in the air and I would ask everyone I came across what did they think about this story. Let me start of my stating that I do not believe Bin Laden is caught and then dumped into the ocean. No way am I going to accept that story. Secondly after 10years of threats and pointless leads, now the U.S finally had intel of where Bin Laden was located. After 10 years of searching for a terrorist, the U.S military caught him, blew off his head and then dumped his body into the ocean and to be absolutely sure it was Bin Laden, they compared his DNA to his long last sister's, who died from cancer.

My thought is that all of this information perfectly ties into a political stragergy for people to vote in the upcoming election and the governmental officials regaining influence on public opinion. The notion that Twitter had to make clear their objection toward the research/observation link and then shut down their link just goes to show the influence of the government on telecommunications.


Post a Comment

<< Home