This article is reproduced with permission from the Connectivist. The article was first published on December 18, 2013.
In times past, counting up scholarly citations– i.e. how often other academics were using your work for their own research – was one of the only ways to know how widely read and appreciated a piece of research had been.
But today, a number of journalspublish more modern alternative metrics, or altmetrics, such as how many times articles have been tweeted, shared on Facebook, downloaded, or written up in news reports. And institutions and scientists can track responses to their work using services offered by new nonprofits and companies.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
But what does it mean to have a paper go viral on social media? And what’s more important: tweet-ability orthe traditional citation from the scientific community?
A study analyzing Twitter links to biomedical articles, which was published last month in the Journal of the American Society for Information Science and Technology, indicates that popularity on Twitter is probably not a reliable measure of scholarly esteem.Twitter mentions show a low correlation with citations, so a tweetable paper isn’t necessarily a well respected one.
“It’s one of the largest studies that looked at the connection between scientific articles and their [mentions] on social media,”said Emily Darling, a David H Smith Conservation Research Fellow at the University of North Carolina who studies both coral reefs and how scholars use Twitter. “It was interesting that there’s no connection to scholarly impact.”
“I was surprised that the correlation was so low,” said Euan Adie, cofounder of Altmetric, a London-based company that tracks Internet response to scholarly works.The study reinforces a point Adie and his Altmetrics colleagues often make: high levels of online engagement do not necessarily say anything about an article’s scholarly quality.
Lead author Stefanie Haustein, a researcher at the University of Montreal, and colleagues used data from Altmetric to track tweets of links to papers published between 2010 and 2012.The study encompassed 1.4 million studies indexed on PubMed, a site that catalogs biomedical articles, and Web of Science, a site that tracks scholarly citations.Haustein’s team found that less than 10 percent of these articles were ever tweeted at all. They used only papers published in 2011 to check correlations between tweets and citations.
“Social media was never meant to replace traditional statistics like journal impact factors or article citations,” Darling writes in an article published last week on The Conversation that responds to Haustein’s study. Rather, “Twitter gives you connections beyond the ivory tower that you don’t normally have,” she said. As a conservationist, for instance, she hopes to spread her research to policymakers and resource managers, so they can take actions based on it. And she wants to inform the public.
But Haustein says that more work is needed to distinguish between the various reasons an article is tweeted—whether to spread important information or for fun. For instance, Altmetric last week released a list of the top 100 papers that received the most online attention in 2013, including tweets, Facebook posts, news stories, blog posts, and more.
Some papers were clearly shared because they were of vital interest to the public. A study related to the Fukushima nuclear disaster in Japan topped the list.It was an open-access paper that chronicled the levels of radioactive cesium found in freshwater fish around Japan, and it was tweeted largely by members of the public, with the highest representation in Japan.
Data Source: Altmetric.com/top100
Other papers on the Altmetric list were more likely there because they were funny.Number 19 on the list determined the composition of chicken nuggets from two major fast-food brands. “Chicken nuggets are mostly fat, and their name is a misnomer,” the study concluded. Shocking.
In order to better classify tweets, Altmetric now collects demographic data about tweeters, analyzing Twitter bios and networks to determine whether tweeters are doctors, scientists, members of the public, or science communicators — in other words, whether their tweets are likely to be informed.
Haustein hopes to go a step further and analyze the text of tweets to determine their intent, be it to amuse, to inform members of the public about an important issue, or foster technical discussion. “If you know that someone is a scientist, it does not mean that he tweets every scholarly paper in a scientific way,” she said.
And Jason Priem, founder of ImpactStory, a nonprofit that helps scientists understand and present altmetrics, speculates that with a sophisticated enough analysis tweets even could be used someday to predict citations. “My mom’s tweets would not be indicative of my article’s success but someone else in my field who was selective might be better.” He says that a machine learning algorithm could learn which people’s tweets had predictive power.
For now at least, much of the power of altmetrics lies in exploring specific stories of how information travels. “Altmetrics is a way of trying to find those invisible threads of how science is being shared online,” said Darling. Monitoring social media mentions and media coverage allows her to think about how to best present findings so that they will be catchy and reach the right audience. I.e. finding the perfect recipe to make science spread.
“[Twitter] lets me take the pulse of science from my actual peers who I follow in the field,” Darling said. “I don’t know how I did science without Twitter before.”
Haustein enjoys using Twitter to promote her work, to see when others mention her work, and to communicate at conferences. Just because Twitter activity isn’t correlated with citations “doesn’t mean that scientists shouldn’t be on Twitter,” Haustein said. “It’s a great tool to reach a larger audience than you usually do.”
The Connectivist is an online magazine created in partnership with the National Cable & Telecommunications Association (NCTA) to connect the dots between technology, innovation, web culture and TV.