Not this kind of predator. CC-BY Image from Flickr user Dimitry B

But the term “predatory” can be misleading.  It implies evil intent, and Beall ascribes evil intent to publishers who charge high author processing fees, do little work with regards to peer review or copyediting, and make lots of money.  In this case, the authors are the prey and are being duped into forking over large amounts of money to be published. I worry that Beall is making a stronger connection than necessary between predatory practices and open access.

Let’s look at another, more traditional scenario.  Let’s say a publisher wants to make money.  The publisher asks authors to submit their content for free, and even makes the authors sign away all rights to the work prior to publication.  The publisher may provide some services like coordinating peer review and type setting a final document (although the quality of these can vary widely). They then turn around and sell that publication for exorbitant fees, making healthy (or more than healthy) profits.  In this case, I might argue that authors AND subscribers are the “prey” as the publisher exploits the system of academic tenure and promotion to make a profit.

The big difference between “honest” commercial publishers (whose goal is to make money) and the “predatory” open access publishers on Beall’s list (whose goal is also to make money) is the quality of the product they are producing.

I can’t deny that some of the publishers on Beall’s list are seeking out author contributions (and author fees) by being purposefully vague about their journal quality.  But each time my library gets a bill from one of the big publishing companies with a 6% price increase, I feel a bit like a gazelle being eyed by a hungry lion and I have nowhere to run.

It can be very hard to determine if a journal is predatory or not: that term is dependent on the publishers motives. Sometimes journals just aren’t very good, but they are trying. Sometimes journal quality is great, but they are squeezing the life out of the organizations that subscribe to their journals (just ask most any librarian what has happened to their book budget). Evaluating overall quality (independent of publisher motives) can be much easier and more relevant for authors looking for a publication venue.

If you are considering publishing your work in a journal you’ve never heard of before, here are just a few of the many things consider:

• Have any of your colleagues read, reviewed, or published in that journal?
• Where is the journal indexed? Can you find it via the databases you usually use to find information?
• Is it associated with a scholarly society you (or your colleagues) have heard of?
• Are there any reliable metrics associated with the journal (traditional or alternative)?
• Who is the editor? Who is on the editorial board?  Have you heard of them?  Can you find out more about them?
• Does the journal come with the regular trappings of a serial publication? ISSNs, DOIs, etc.

You may also check out Beall’s (somewhat controversial) list of predatory publishers, and his criteria for determining which publishers are predatory.

Naturally – and I know you’ve already thought of this – these are great questions to ask your local librarian, especially since you have data to analyze or an experiment to set up or papers to grade.  It will be a welcome change from the scores of reference questions from first year students about the proper placement of commas in APA style citations.

Works Cited

Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA : the journal of the American Medical Association, 295(1), 90–3. doi:10.1001/jama.295.1.90

Larsen, P. O., & von Ins, M. (2010). The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84(3), 575–603. doi:10.1007/s11192-010-0202-z

This time they wanted to see whether these changes apply specifically to top-tier journals rather than being just a general trend. They looked at seven “elite” journals as well as at a group of six “emerging” journals. The “elite” journals in the sample had to be considered as “prestigious” to have had a high Impact Factor in 2011, not be review journals, publish a high number of articles every year and exist for at least several decades. They chose three multidisciplinary journals: Science, Nature, PNAS (included for its prestige rather than its IF) and four biomedical journals: Cell, Lancet, New England Journal of Medicine (NEJM), and the Journal of the American Medical Association (JAMA). The “emerging” group journals were chosen because they have had the highest growth in their proportion of top cited papers over the past 40 years. The group includes PLoS One, Nano Letters, Advanced Materials, Nature Materials, Journal of Clinical Oncology and Chemical Reviews. Of these journals PLoS One, Nature Materials and Nano Letters have been founded in the 21^st century, Chemical Reviews is older by several decades (1925) and the Journal of Clinical Oncology and Advanced Materials have been published since the 1980s.

But, before we get into the decline of the journals, let me mention that the “Highly cited” threshold is going up. When the authors ranked the top articles for each year since 1970, they found that to be included in the 1% and 5% best-cited, articles have to receive about twice as many citations as they did 30-40 years ago.

Figure 1: Citation threshold for inclusion in the top 5% and 1% most cited papers, 1970-2010

Back to the journals; In comparison with 20-25 years ago, all the elite journals studied publish now a smaller proportion of highly cited articles. Also, they all publish a larger proportion of the highly cited 1% articles than of the highly cited 5% articles. PNAS, for example, went from publishing almost 9% of the 1% highly cited and over 4% of the 5% highly cited in the mid-1980s to 2.7% and 2.2% respectively in 2010. Nature and Science used to publish 20-25 years ago about 7% and 6% of the top 1% articles, but now publish only 4% and 3% respectively. The NEJM’s share of articles in the top 1% has been decreasing since the 1980s, while the Lancet managed to keep it relatively stable. However, they both went from having about 1% of the 5% top articles in the 1970s to about 0.5% in 2010. JAMA increased its proportion of articles in the 5% until the late 1990s and its proportion of articles in the 1% until the early 2000s, but both proportions went down again.

Since the number of articles in most journals has increased since the 1970s, the authors wanted to make sure the changes in proportions weren’t merely a by-product. They calculated a yearly normalized top 1% index for each journal. It measures the number of the top 1% articles published in a journal in a specific year in comparison with the number of top 1% articles it was supposed to publish if journals published top articles at random. As the authors put it “for a given year, it is the relative number of top papers in a given journal (top papers / total papers) divided by the proportion of top papers published in all journals.” If the journal publishes the exact number of top articles we expect it to publish by chance, the coefficient will be 1. Naturally, the coefficient of high-impact journals is a lot higher than one – that is, they publish way more top articles than can be expected by chance. However, even by the normalized index, high-impact journals publish less top articles than they used to (Figure 2).

Figure 2: Normalized 1% Index for elite journals, 1970-2010

The emerging journal group also publishes more top 1% articles than can be expected by chance (Figure 3), and Chemical Reviews and Nature Materials are now in the 40-50 coefficient range. PLoS One fans would have a bit of a disappointment for here, since its coefficient is in the “low single digits” as the authors put it (Figure 3) and that is probably because of the large number of articles it publishes per year.

The digital age means it’s easier to create and publish new journals, which are bound to decrease the market share of the old ones. Some older publishers try to deal with the problem by creating their own new specialized journals. For example, the Nature Publishing Group now publishes almost 40 journals under the “Nature something” title, which all enjoy a bit of the “Nature” prestige. The proportion of the top 5% articles of all the Nature journals combined went up from 3% in the 1970s to about 5% today. The authors point out that while the Nature Publishing Group is for-profit, the non-profit organization American Association for the Advancement of Science (AAAS), which publishes Science, only owns three peer-reviewed journals.

Are top-tier journals in decline? My suggestion is not to cancel the Nature subscription yet. The top journals still publish many highly cited articles. Journals, in general, still matter – even though this paper was first published in ArXiv, it’ll be published in a journal as well, to receive its “stamp of approval.” However, the decline in the share of top articles published by older elite journals means that first, articles can be judged more independently than they used to and second, that the hierarchy of journals might be changing.

Vincent Lariviere, George A. Lozano, & Yves Gingras (2013). Are elite journals declining? arXiv: 1304.6460v1

George A. Lozano, Vincent Lariviere, & Yves Gingras (2012). The weakening relationship between the Impact Factor and papers’
citations in the digital age arXiv: 1205.4328v1

CC-BY-SA image courtesy of Flickr user Takeshi Life Goes On

Here is the problem:

• A student searches for high quality content in a database that the library pays a lot of money for.
• Finding a great article, they copy and paste the URL from their browser into a Word document they are using to take notes (now isn’t the time to go into all the ways they could/should do this better.)
• Later that evening, when they go back to their research, they find that the URL they copied and pasted doesn’t take them to the entry they need, but to an error page or to the database homepage.
• Frustrated, the student finds that they have to do their searching all over again – what a waste of time!

There are a couple of reasons for this.  The first reason is that sometimes there are on-campus/off-campus access issues.  I may talk about that another time. But the second reason is that some databases don’t provide persistent URLs for their content.  Or you have to click on a special “persistent URL” link or icon to get a link that can be re-used by another person or by yourself later on.

I want this to be easier – the URL indicated at the top of the browser should get you back to where you want to go.

I know that the DOI can be used as a persistent identifier.  But but my students don’t (not until I tell them about it, anyway).  And in this case, I don’t need the long term stability a DOI provides, just the ability to get back to an article or database entry within a several week time-span.

I don’t think I am asking too much. Some database creators get it right, why can’t everyone? When a researcher copies and pastes the URL from their browser they should be able to use that link to back to the same page – whether they are on a search results page, an index page for an article or something else.

I worry because this seems to make life unnecessarily complex for my students, and I can’t give them a good reason why this exists.  I’m sure that there are technical reasons why this happens.  But I don’t care.  I just want databases to work in a way that students (and other researchers) expect them to.  We expect to be able to copy and paste a URL and use it later on.

So, if you are a database creator and your database has this problem: fix it (I’m looking at you, LexisNexis).  If you use a database and notice this problem: tell the database creator to fix it.

Okay, rant over.

According to the LR, the university with the highest impact is MIT. The overall ranking is ruled by American universities in the first 20 places (other than Ecole Polytech Fédérale Lausanne at the 13th place). The next non-American institutes on the list are the Weizmann Institute of Science (23^rd) and the University of Cambridge (24^th). The ranking focuses on reviews, research articles and letters, though letters have less of a weight than the other two in the calculations.  It’s important to note that the LR doesn’t deal with Arts and Humanities, because Web of Science doesn’t cover these disciplines well, and the LR people acknowledge its limitations. If your university has a large, excellent philosophy department, it won’t help its ranking one bit. Leiden works with several main impact indicators:

Mean Citation Score (MCS) – the average number of citations to the university’s publications.

Mean Normalized Citation Score (MNCS) – the average number of citations to the university’s publications, but with taking into account publication years, document types (reviews receive, on average, more citations than articles and letters).

Proportion top 10% publications (PP_{(top 10%)}) – publications from the same year, field and document type are compared and the indicator is calculated according to the proportion of the university’s publications which belong to the 10% most frequently cited.

That SHELX publication

In my first Impact Factor post I wrote about the case of “A short history of SHELX”. SHELX is, according to its website, “a set of programs for the determination of small (SM) and macromolecular (MM) crystal structures by single crystal X-ray and neutron diffraction.” That article sky-rocketed the journal Acta Crystallographica Section A’s Impact Factor from about 2 to over 50. In the previous LR (2011/2012) it has been skewing the University of Göttingen’s MNCS. Normally, the PP_{(top 10%)}) and the MNCS are strongly correlated, but in Göttingen’s case, it was ranked 2^nd according to its MNCS and 238^th according to its PP_{(top 10%)}). It single-handedly increased Göttingen’s MNCS from 1.09 to 2.04. The PP_{(top 10%)}), on the other hand, was not influenced, because it’s not sensitive to the exact number of citations. A paper which barely made it to the top 10% and the SHELX paper are the same to it.

Full and fractional counting

The LR counts publications in two ways: in the full counting method, all of the university’s publications (except for letters) have equal weight. In the fractional method, publications which are collaborations receive less weight than those which weren’t written in collaboration with other institutes. If an article has five authors and two of them are from the same university, the fraction given to that university is 0.4. The LR people prefer the fractional method, because this way they don’t count collaborative publications multiple times. The differences between the two methods are field-dependent. In clinical medicine, the differences can be substantial; in chemistry, engineering and mathematics they are rather weak (I assume that is because there are more collaborations in clinical medicine). Almost all universities have a higher PP_{(top 10%)}) in the full counting method than in the fractional counting method (high-impact publications are often products of collaborations). If we look at Mount Sinai School of Medicine, for example, we’ll find its goes from PP_{(top 10%)}) 19.2% in full counting to 15.4% in fractional counting (all the numbers based on the 2011/12 ranking)

The language bias

About 2% of the publications LR is based on are not written in English. They are mostly written in German, Chinese and French. Publications in languages other than English normally have low impact, a result of a smaller reader pool. Though most universities can take or leave the impact of non-English publications, as it doesn’t affect their ranking, some French and German universities benefit significantly from excluding non-English publications. The Paris Descartes University goes from PP_{(top 10%)}) of 9.9% to 11.9%, and the German University of Ulm goes from 9.9% to 11.1% (again, numbers are from the 2011/12 ranking).

The Leiden Ranking is exactly what it says on the tin: a bibliometrical ranking. It’s not interested in the quality of teaching in an institute or how many Nobel Laureates it has among its professors, like the Shanghai Ranking. It isn’t partly based on a survey like the Times Higher Education Ranking (THE). It’s also not much help for prospective undergraduate students. It’s about publications, citations, and co-authorships, and should be treated as such.

ETA: I was just corrected: there are two other bibliometric-based rankings. One is the “University Ranking by Academic Performance” and the other is the “Performance Ranking of Scientific Papers for World Universities” (Hat tip: Isidro Aguillo, @isidroaguillo).

Ludo Waltman, Clara Calero-Medina, Joost Kosten, Ed C. M. Noyons, Robert J. W. Tijssen, Nees Jan van Eck, Thed N. van Leeuwen, Anthony F. J. van Raan, Martijn S. Visser, & Paul Wouters (2012). The Leiden Ranking 2011/2012: Data collection, indicators, and
interpretation ArXiv arXiv: 1202.3941v1

Sheldon Cooper, The Big Bang Theory

Between the recent ACUMEN (academic careers understood through measurement and norms) workshop and my searches for a post-doc, it seemed like an excellent time to look at one of the most important land marks in an academic’s career: the tenure. Once you have tenure, you cannot be dismissed without a cause, and it better be a good one. Harvard has never dismissed a tenured professor, and that includes John W. Webster.

Tenure has been created for two purposes; one of them is to ensure academic freedom.  Tenure allows a professor to criticize power structures, even those which pay their salary. Professor Neve Gordon called for an academic boycott on Israel, but still holds his position at a government-funded Israeli university. Tenure lets a professor have unpopular opinions without being punished.

An interesting case is the one of Ward Churchill, formerly a tenured professor of ethnic studies at the University of Colorado. After 9/11 he wrote an essay using the term “little Eichmanns” regarding the World Trade Center victims. Naturally, his words caused a major public uproar, and many called for his dismissal. At first, University of Colorado ruled that his writing was legally protected. However, Churchill was eventually fired in 2007 for the scholarly misconducts of plagiarism and misrepresenting his colleagues’ work. Churchill denied the university’s allegations and claimed the investigation has been retribution for his political views. He sued the University of Colorado several times in different courts, until April 1^st, 2013 when the U.S. Supreme Court declined to hear his appeal.

The second purpose tenure fulfills is job security. That can cause problems; some claim tenure can make faculty members into “deadwoods,” who rarely publish, win grants, or put in any kinds of effort. I’ve never seen one of those – all the tenured professors I’ve met seemed even more driven, if possible, than their students – but they exist, and they are hard to fire. Another tenure problem is that tenure committees emphasize research over teaching ability in the pre-tenure period, creating faculty members who are way more research-oriented than student-oriented.

Getting tenure has never been easy. A 1958 murder-mystery novel by Isaac Asimov deals with the death of a Chemistry PhD student, but only as an excuse to go deep into the world of academic intrigue, where the student’s advisor is a 42-year-old assistant professor whose hope for tenures diminishes by the year. However, the general chances for tenure back then were better than they are now. The percentage of part-time faculty positions has increased by 300% between 1975 and 2011, while full-time tenure and tenure-track positions only increased by 26%. Tenure and tenure-track faculty made about 55% of the faculty in 1970, but 41% in 2003. According to Clawson (2009) tenured faculty members might have contributed to this rise of part-time faculty by delegating to them the jobs they found least appealing.

American Trends in Instructional Staff Employment Status, 1975–2011

Source: US Department of Education, IPEDS Fall Staff Survey (in the Annual Report on the Economic Status of the Profession, 2012-13 of the American Association of University Professors).

In the U.S, between the years 1993-2008, the percentage of those who have earned a doctorate in science, engineering, and health during the previous three years and now have tenure-track or tenured positions has been between 16% and 19%. As the National Science Board report puts it:

“At the doctorate level, the decline in the availability of tenure track positions, which used to be an incentive for students who decided to pursue a doctorate, may result in many doctoral recipients looking for careers outside academia.”

Metrics and tenure

A Nature poll from 2010 showed that almost 70% of the responding researchers thought metrics (e.g. h-index, impact factor of journals) have an effect on tenure decisions. On the other hand, when Nature interviewed provosts, department heads and so forth, they claimed metrics didn’t play such a big role in hiring, promotion and tenure. What mattered most, according to the interviewees, were letters of recommendations from outside experts in the field. However, they did admit that sometimes reviewers of applications use metrics. Richard Zare, former department chair at Stanford University’s Chemistry Department wrote that the department collects 10-15 recommendation letters from outside experts prior to the tenure decision. As for bibliometricians, I can say we are careful not to define metric criteria (“h-index of X gets you tenure in field Y”), since we acknowledge there’s more to a person than her h-index. Unfortunately, we can’t prevent other people from seeing metrics as the beginning and end of things.

A few words of advice: if and when you reach the promised land of tenure, try not to embarrass your institute too much. Consider this charming disclaimer by Lehigh University regarding Associate Professor Michael Behe:

While we respect Prof. Behe’s right to express his views, they are his alone and are in no way endorsed by the department. It is our collective position that intelligent design has no basis in science, has not been tested experimentally and should not be regarded as scientific.

Can you read the wistful “If only he didn’t have tenure” between the lines?

References

Abbott, A., Cyranoski, D., Jones, N., Maher, B., Schiermeier, Q., & Van Noorden, R. (2010). Metrics: Do metrics matter? Nature, 465 (7300), 860-862 DOI: 10.1038/465860a

Clawson, D. (2009). Tenure and the Future of the University Science, 324 (5931), 1147-1148 DOI: 10.1126/science.1172995

Fishman, J. (2005). Tenure: Endangered or Evolutionary Species. Akron L. Rev., 38, 771.

Fishman, J. (2009). Tenure and Its Discontents: The Worst Form of Employment Relationship Save All of the Others.

National Science Board. (2012). Science and Engineering Indicators. Washington DC: National Science Foundation; available at http://www.nsf.gov/statistics/seind12/.

Tierney, W. G., & Lechuga, V. M. (2010). The social significance of academic freedom. Cultural Studies↔ Critical Methodologies, 10(2), 118-133.

Zare, R. N. (2012). Editorial: Assessing Academic Researchers. Angewandte Chemie International Edition, 51(30), 7338-7339.

Mendeley is useful to academics and researchers on several fronts. The desktop application helps folks organize all the PDFs of journal articles that live on their computer. It is kind of like an iTunes for journal article PDFs, helping users create folders, rename files and automatically download the metadata (author, title, journal, etc.) that comes with the articles. Mendeley helps users collaborate with others by sharing reading lists and files and helping folks discover new research based on their interests. And Mendeley helps users create in-text citations and works cited sections in their manuscripts.

All of that tends to fit with what Elsevier does: publishing scholarly journals and databases.

But there are some real differences in the reputations of the two companies, and some real differences in how the two companies have approached the various “open” movements related to academic publishing (open access, open science, open data etc.). Some of these differences can be seen in Jason Hoyt’s (now of PeerJ, formerly of Mendeley) reaction to the news.

Mendeley demonstrates its openness and flexibility by applying Creative Commons licenses to images it posts Flickr. CC Image courtesy of Mendeley on Flickr.

Mendeley has appeared to embrace “openness” in several ways. It uses a “freemium” model, where the basic product is free and users can pay for additional storage space or more collaboration features. Mendeley embraced alternative metrics, a hallmark of open access publications like PLOS ONE. Mendeley released an incredibly useful Open API that has allowed a lot of really creative people to create a lot of really creative tools. Although they don’t publish anything themselves, they have associated themselves with open access to scholarly research.

Elsevier, on the other hand, has not embraced “openness” in any meaningful way. They charge a lot of money for their products, leading some researchers to boycott the company. They have been reluctant to automatically grant researchers the right to use text mining techniques with their products, even when researchers have already paid for access to those journals via library subscriptions. And they appear to have an overall corporate philosophy that goes against the principles of openness. A 2012 report aimed at Elsevier stockholders (and potential stockholders) suggested that the company has an “obsession with maintaining a huge degree of control on both content and its access” and suggested that this wasn’t a good long-term strategy: “We continue to be baffled by Elsevier’s perception that controlling everything (for example by severely restricting text- and data mining applications) is essential to protect its economics.”

So this acquisition of Mendeley by Elsevier has some folks shaking their head. Some folks are feeling betrayed, vowing to discontinue using the product. Despite reassurances, some users are concerned that Mendeley will cease to be available for free. Others feel this sale (for about $70-$100 million, according to TechCrunch) is a vote of confidence demonstrating just how good Mendeley is.

The companies are trying to mitigate concerns. They started by giving things away: every Mendeley user will now see their online storage capacity doubled (from 1GB to 2GB for users at the free level). In the Q and A posted on their website, Mendeley seeks to reassure users that nothing bad will happen. The problem is that the power relationship has changed. As Mike Taylor points out, Mendeley is no longer in charge and would have to abide by decisions made by Elsevier.

Admittedly, the sale makes me rethink my views on Mendeley as a company. I recently reviewed their related product, Mendeley Institutional Edition (MIE), for Library Journal, and my colleagues and I spoke with a representative about it just yesterday. MIE is interesting but expensive, and as we talked, the representative discussed their flexible pricing options. Will this flexibility remain now that they are owned by Elsevier (who is not known for flexible pricing models)? My library already sends a lot of money in Elsevier’s direction for journals and databases, and I’ll admit I’m reluctant to send any more their way.

It will take some time to see how this all plays out: there are philosophical issues as well as user experience issues to look at. William Gunn, Mendeley liaison to the academic community, suggests that this represents an honest move by Elsevier to rehabilitate the reputation I’ve described above, and the official Elsevier announcement of the acquisition suggests this a bit.

In the meantime, I will continue to use Mendeley and recommend it to those I work with. But I’ll be paying close attention to policy changes and updates, and I’ll be backing up my data.

The mobile apps for accessing library resources are always a bit weird.  Because libraries and institutions pay lots of money for access to databases from Proquest, Ebsco, Elsevier, etc., their mobile apps and websites need to have a way of figuring out that the mobile user is associated with one of those institutions.  They do this in a variety of ways, some of which are annoying all the time and some of which are annoying just once in a while.

Some of my iPhone apps

Some of these apps or websites allow you to search for and find out that an article exists, and perhaps learn a bit about the article (an abstract): The PubMed mobile website is like this.  Other apps allow you to access the full text content of a particular article, but aren’t very good at searching: The ACS Mobile app allows you full text access to current research.

I’ve seen various methods of authenticating users.  Some require you to sign up for an account via your desktop computer while on your institution’s network, then sign into the same account on the mobile platform.  Hopefully they don’t make you sign in each time.  Others require you to be on the campus network with your smart phone, or to use a mobile VPN to access content.  I’m seeing less of this lately, which is great because this is a really annoying way to use these apps.  I really liked the way that Ebsco handled authentication: From the desktop app, you click on a link and Ebsco sends an email with an authentication link.  You open the email on your mobile device and get access for 9 months to the app – no need to sign in over and over, and you don’t even need a user account if you don’t want one.

Of course, doing big time research on these apps isn’t the main goal – it would be slow and cumbersome to do your research primarily on a tiny handheld device.  But for quick look ups in meetings and while out and about, these apps and websites can be incredibly useful.

Here is a small list of apps and mobile websites for accessing scholarly information on your mobile device.  All of the apps listed are free to download, but may require a user to be affiliated with an institutional subscriber in order to access content. (Note that this list tends to be iOS-centric because I don’t have an Android device. Sorry.)

• Ebscohost
  • Platforms: iOS, Android, Web
  • Description:  Access full text articles and article abstracts from a large number of library databases that use the Ebsco platform (GeoRef, Business Source Complete, Greenfile, etc.).  To get access, visit one of your library’s Ebsco databases (like Academic Search Complete) and click the “EBSCOhost iPhone and Android Applications” link at the bottom of the page.  You’ll get an email that you need to open up on your phone/device and tap the authentication link. The app will work on campus and off campus, no matter which network you are on.

• Sciverse Scopus Alerts
  • Platforms: iOS, Web
  • Description: Search the Scopus database for article abstracts and citations.  Get search and citation alerts. Users must register first at Scopus, then log in on the app with the Scopus username and password.

• arXiv
  • Platforms: iOS
  • Description: Free full text access to the pre-prints available at the arXiv.org website in Physics, Mathematics, Nonlinear Sciences, Computer Science, Quantitative Biology, Quantitative Finance, and Statistics.

• iResearch
  • Platform: iOS
  • Description: Full text access to American Institute of Physics Journals.  Requires an individual subscription or an institutional subscription to access full text.

• PubMed
  • Platform: Mobile website, iOS apps (PubMed On Tap, PubMed4Hh)
  • Description: PubMed is a free index to the biomedical literature, with links to publishers websites for (paid) full text and links to PubMedCentral for free full text (where available). Several apps can be used to access PubMed and create lists or do more complex searching.

• SciFinder
  • Platform: Mobile website
  • Description: For those affiliated with an institution with access, you’ll need your SciFinder username and password to look up journal literature or look for chemical information by substance name or CAS registry number.  Chemical structure searching isn’t available on the mobile site.

• ACS Mobile
  • Platform: iOS
  • Description: Access recent articles from your favorite ACS publications. Requires the user to be on the institutional network or VPN, or requires an individual subscription for full text access. Non-subscribers can still browse article abstracts.  Good for discovering recent work, not great for searching the archive of ACS publications.

• GeoScienceWorld
  • Platform: Mobile website
  • Description: Users from institutions with a subscription can browse and read the geoscience literature.  Read the HTML version of an article (much easier on a mobile device) or download (or email) the PDF.  Authentication requires users to get an authentication key while on an institutional network before using the site off-network, which is a bit confusing.

• Google Scholar
  • Platform: Mobile website
  • Description: Any mobile user can search Google Scholar for free, and often get access to freely available documents.  If you are using your mobile device at your institution, and if your institution has configured their Google Scholar library links, you will also see links connecting you to library resources.  You can set up these links for yourself under “Settings” (for the mobile and desktop versions).  This website is great for discovering content, but when you click on an article title you enter the publishers website that may or may not be mobile friendly.

Obviously, I’ve left a lot out – apps and mobile websites for individual publications, apps for collecting and analyzing scientific data, etc.  Do you use your mobile device to read or search the scientific literature? What apps or mobile websites do you use ?  Leave your suggestions and recommendations in the comments below.

In my Brownie uniform, circa 1984

I was a Girl Scout for twelve years.  By the time I graduated from high school, there were four girls in our troop:

• Dr. Airani Sathananthan is an endocrinologist and an assistant professor at the Western University of Health Sciences.
• Dr. Alice Suroviec is a chemist and an associate professor at Berry College.
• Tina Steger is an industrial engineer at General Motors.
• I started out in geology but shifted gears and am now the Science Librarian at SUNY Geneseo.

I can’t say that we would all be English teachers or marketing managers without scouting, but we had incredible role models, STEM related activities to broaden our minds and the opportunity to develop life skills that have served us well in scientific fields.

While there were many wonderful women who led our troop over the years, our most consistent leader, and the one who stuck with us until graduation, was Jean Harper. Mrs. H. was a lecturer in mathematics at the local university and tops my list of “women I admire.”  Her work in math and computer science ensured that we never had an opportunity to think “girls can’t do math” but she also made sure we were exposed to a wide variety of experiences.  She taught us how to count in binary when we were 9 and how to do folk dances from around the world in middle school.  (I remember the binary but was never fond of the folk dancing.  Sorry, Mrs. H.)  In a high school that lacked an AP math class, Mrs. H taught us calculus after school during our senior year. She is a remarkable story teller, and while her recent retirement is great news for her, I will miss the descriptive emails she sent of her latest classroom adventures.

In addition to the outstanding role models, my experiences in scouting led directly to my career choices. I spent most of my summers at Camp Timbercrest.  The camp occupies an amphitheater-shaped valley in Western New York and a lake was formed by damming one end of the valley. By hiking and boating and camping in that valley I got see glacial features like glacial erratics and lacustrine clay from pro-glacial lakes.  While I didn’t know what those things were as a teenager, those experiences had me asking “Why does this valley look the way it does?” and “Why is all of this clay here?”  My desire to learn more about geology was born and the camp eventually became the setting for my first research experiences examining the glacial border  region for my bachelor’s thesis.

Keyser Lake at Camp Timbercrest. Image copyright Jennifer Schlick and used with permission.

My good friend Dr. Alice Suroviec also has clear memories of scouting experiences that influenced her career decisions.  She remembers learning BASIC computer code during troop meetings and doing experiments to determine if salt water freezes faster than fresh water.  But she also had larger formative experiences during Girl Scout sponsored summer trips.  Dr. Suroviec had an opportunity to  learn about polymers by doing experiments at Akron University and to see how basic science is translated into patents by exploring the National Inventors Hall of Fame.

Of course, the things we learned in Girl Scouts go beyond specific pieces of scientific knowledge and inspiration. Through guest speakers, field trips and research for badges, we had an opportunity to think beyond our small town. As Dr. Suroviec said recently,

“I think that being a Girl Scout let me dream big about what career options were out there and not just the typical careers (teacher, doctor, lawyer).”

Girl Scouts also provided us with a wide variety of opportunities to improve our leadership skills.  We took turns leading projects and setting the agenda for our weekly meetings.  We worked with younger scouts leadings craft activities and games.  And as I got older, Girl Scout camp provided me with a wide range of opportunities to lead scouts, fellow staff members and occasionally community members.

As I think back on my time in Girl Scouts, I am struck by the similarity of our weekly troop meetings to my work related meetings today.  We had to work collaboratively to set goals and accomplish tasks, and we had to recognize the strengths and weaknesses of our colleagues in order to get things done. Tents didn’t get pitched, meals remained uncooked, and workshops got out of control if we didn’t work together.

We learned how to communicate with each other. For some, Girl Scouts gave them the opportunity to speak up.  I sometimes struggled to let others talk, and I slowly  started to learn when to be quiet and let other folks talk. I also started to learn a bit of tact (it took a while, and some may argue that I have some work left to do).

So in honor (belatedly) of Girl Scout Week and the 101st anniversary of the founding of the Girl Scouts of the USA, I would like to say thank you to the Girl Scout organization and to Mrs. H., Alice, Airani and Tina in particular.  The experiences  I had and the relationships I formed had a profoundly positive influence on my life and my career in STEM.  But I’m still not fond of folk dancing.

“But wait a sec,” you might say, “Were scientists keeping those results to themselves?”

Sharing is what makes science work. CC image from Flicker user creativecommoners.

Not exactly, but the results of these scientific studies aren’t always available to just anyone. The results of scientific research are published in scholarly journals that aren’t sold in your local bookstore. While some of these journals share their content with anyone online, most of these journals aren’t available for free.

Accessing articles in these “closed access” journals can be pretty difficult for the average american taxpayer who is not affiliated with a research university. It might require a (sometimes quite costly) subscription. If you don’t have a subscription, you might be able to purchase an electronic copy of the 10-page article for $20-$40 (typically more than the cost of a hardcover book). You could also try to find a local academic institution that already has a subscription:

1. Make a lot of phone calls and check a lot of library websites looking for one that has a subscription,
2. Fill up your car with gas (how much is that now?),
3. Drive to the library,
4. Attempt to find the university’s visitor parking (why is visitor parking so far away from everything you want to visit?),
5. Sign up for a guest borrower card (do they have those?),
6. Hope that library staff can log you in to a computer,
7. Hope that the license agreement between the library and the publisher allows non-affiliated users to access the content you need.

Of course, this assumes that you live within driving distance to an institution that has the journal you want.

But the new White House directive asks scientists to make sure their work is shared more broadly. Lots of other initiatives from universities and scientific societies have encouraged this as well.

The White House asked each federal agency that spends more than $100 million per year on research and development to develop a policy requiring scientists who receive federal funding to make sure that the resulting journal articles are available to the public within 12 months of their original publication. The National Institute of Health already has a policy like this, and anyone can read these articles on PubMed Central for free. This new directive will ask the National Science Foundation, The Department of Energy, and other agencies to ask the scientists they fund to do something similar.

The practice of making scholarly journal articles available at no cost to readers is called Open Access, and I believe it is the natural extension of the sense of community and advancing knowledge that drives science.

Big thanks to the Obama administration and all of the open access advocates who have worked to educate researchers and policy makers about these issues!

For example, is an h-index of 9 good, bad or indifferent?

• It can depend on discipline.  Citation patterns vary: mathematicians cite fewer papers than Earth Scientists, who cite fewer papers than those in biomedicine (see Podlubny, 2005).  Co-authorship traditions vary: publications in high energy physics or genetics tend to have more authors than those in paleontology.  All this makes it difficult to compare h-indexes across disciplines.
• It can depend on expectations.  At a primarily undergraduate institution like mine, expectations for research output are lower than at major research universities (we are also less likely to rely on metrics like the h-index).
• It can depend the stage of the researcher’s career.  An assistant professor just two years into their first permanent position can’t be expected to have a higher index than the researcher who was just promoted to full-professor.

Likewise, it is hard to evaluate impact factors across disciplines (along with many other problems with the IF).  The flagship journal of the American Chemical Society (JACS) has an impact factor of 9.707.  The flagship journal of the Geological Society of America (GSA Bulletin) has an impact factor of 3.787.  We can’t compare the two like this, and we certainly can’t use these numbers to compare researchers from the two disciplines.

In a recent article in EMBO Reports, Bornmann and Marx (2013) argue for the greater use of percentiles in evaluating researchers, institutions and publications.  You remember percentiles, right?  When you took the SAT or the GRE, your results came back with a score and a percentile: if you were at the 85 percentile, you scored higher than 85% of your peers.

Percentiles can provide important context in one easy to read number.  You know that the top is 100 and the bottom is 0.  The median is 50, and this helps us makes sense of things even if the data set is skewed.  However, it becomes incredibly important to select the right group to compare against.  Typically, subject and year can create reasonable groups.

The folks who look at research metrics outside of citations also have the challenge of providing context. Altmetrics examine the different ways that folks interact with journal articles (in addition to citations).  Are they talking about the article on Twitter? Saving the article to a bookmarking cite like CiteULike or Mendeley? Is the public citing the article on Wikipedia?

Once again, context is vital.  An article was tweeted about twice.  Is this good, bad or indifferent?  Sixty folks on Mendeley have added it to their libraries.  But what does that mean?  The premier tool for easily showing and displaying altmetrics, Impact Story, can provide a bit of context for these numbers by calculating percentiles based on a comparison group of randomly selected items from the same publication year.  Right now, it doesn’t appear that Impact Story is taking advantage of subject categories (which is more difficult).  As a result, articles in some disciplines would automatically have lower percentiles as an artifact of lower average citations in that discipline.

Using any metric to evaluate scientific research is tricky – you are trying to boil down the intellectually complicated act of advancing human knowledge into a single number.  But these metrics are being used more and more by tenure and promotion committees, institutional advisory boards, grant review committees and more.  If folks choose to use metrics (like percentiles) that can provide reasonable and reliable context, we can avoid at least a couple of the standard pitfalls.

See also:

• Bornmann, L., & Marx, W. (2013). How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons. EMBO reports. doi:10.1038/embor.2013.9
• Podlubny, I. (2005). Comparison of scientific impact expressed by the number of citations in different fields of science. Scientometrics, 64(1), 95–99. doi:10.1007/s11192-005-0240-0
• Elsewhere on this blog, my excellent co-blogger Hadas Shema has talked about some of the key concepts that underlie citation analysis and bibliometrics. See her posts about the impact factor, problems with citation analysis and negative citations, for example.

claimtoken-511bdd808802f

• Criticizing previous work
• Disclaiming work or ideas of others (negative claims)
• Disputing priority claims of others (negative homage)

It’s not that negative references don’t exist; it’s just that they go “undercover”. MacRoberts and MacRoberts (1984) called this practice “The art of dissembling.” They described three methods of avoiding or disguising criticism:

1. Praising – calling the work one wants to criticize ‘important’, ‘pioneering’, ‘classical’ and so forth. Of course, there are times in which these words are used to actually praise a work, but sometimes they are used to disguise criticism.

However, Harwood (2008) claims it’s possible that the authors praise works’ strengths as well as pointing out their weaknesses. The praises are genuine, but they tell only part of the story. He says it might be that MacRoberts and MacRoberts have been “overly cynical” regarding those praises. Having read several of their papers, I tend to agree with Harwood about the cynicism. If we take into account that most references serve more than one purpose (Brooks, 1986) we can assume the praises genuinely compliment the referenced work while dissembling the criticism against it at the same time.

2. I’m just going to leave this reference here… – That is when one mentions that work has been done in the field by X, Y and Z, but doesn’t say she thinks X’s work is a waste of funding.

3. I didn’t mean you, Professor X! – In this case, the authors don’t directly challenge the theories of influential people in their field. Instead, they use quotes from outsiders who hold similar theories. Even better: they attribute theories and views to people who passed away. Dead people don’t attend conferences and can’t punch anyone.

The use of negative references may also change across disciplines. Authors at the art and humanities disciplines tend to cite critically more than the hard science disciplines. Linguist Ken Hyland examined verbs used in citations and found that while the popular verbs in sociology were “argue”, “suggest”, “describe”, “note”, “analyse” and “describe”, the verbs used most in physics were “develop”, “report” and “study.” The verb “Argue” was used only by social sciences and humanities authors while “report” was used by science and engineering authors in 82% of the cases Hyland studied.

Why do authors disguise or avoid criticism? One reason is to avoid confrontation with colleagues who might be friends, mentors or influential in the field. Another reason is the peer review process. If the authors know the journal often sends articles for review to those being criticized in the manuscript, they might tone down their criticism to increase their chances of getting published. If they don’t tone down the criticism by their own accord, the editors might ask them to do that, in order to avoid arguments between the referees and the authors. A different case altogether is when authors don’t even bother referring to works they believe are incorrect, simply because they had no use for them.

To quote MacRoberts and MacRoberts:

“…not only is the form of the scientific paper highly ritualized and artificial, but so is published ‘criticism’, for if criticism were aired in the journals as it actually occurs in the lab, its frequency and nature, and hence that of negational citations, would be quite different from what ultimately appears in print.”

In short, what you hear in the lab is not what you’re going to read in the journal. Science, unfortunately, is not as impartial or impersonal as we would like to believe.

Brooks, T. A. (1986). Evidence of Complex Citer Motivations. JASIS

Hyland, K. (1999). Academic attribution: citation and the construction of disciplinary knowledge Applied Linguistics, 20 (3), 341-367 DOI: 10.1093/applin/20.3.341

MacRoberts, M., & MacRoberts, B. (1984). The Negational Reference: or the Art of Dissembling Social Studies of Science, 14 (1), 91-94 DOI: 10.1177/030631284014001006

Garfield, E. (1962). Can citation indexing be automated? Essays of an Information Scientist, 1, 84-90

But there are also multiple versions of folks with each of these names. There are several Bonnie Swogers or Bonnie Mullers and thousands of Reds and Bonnies in the US alone.

In order to distinguish me from other folks with the same name, you need to look at other factors: location, occupation, physical appearance, etc.

But there is one way the immediately identifies me and keeps me distinct from everyone else with my name: my Social Security number. I have many names, but only one social security number. Conversely, my social security number applies only to me and not to anyone else.

When we start looking at information sources, there are many similar types of identifiers that help us pinpoint the exact item we are talking about while helping to distinguish that item from other similar items.

ISBNs and barcodes of the books on my desk right now.

In libraries, a common identifier is the ISBN, or international standard book number. Go ahead and pick up the book closest to you. Most likely, if you look at the bottom right hand corner of the back cover, you will see a bar code and a series of numbers on top of the bar code preceded by “ISBN:” Each ISBN identifies a particular version of a book. I have three hardcover versions of my favorite book Pride and Prejudice (insert isbn’s here), one paperback version (isbn) and one Kindle version (ISBN). So while the ISBN is extremely useful for identifying a particular version of a book, it isn’t very good at identifying all versions of that book. ISBN were introduced in the 1970s and have revolutionized the way that publishers, booksellers and libraries keep track of their inventories, but they aren’t necessarily helpful when you don’t care which version of Pride and Prejudice you want.

Another identifier used extensively in the scientific literature is the Digital Object Identifier (DOI). A DOI is just like a social security number for a digital item (journal article, data file, presentation file, etc.). Each journal article has a DOI, and no two articles have the same DOI. A DOI normally consists of numbers, letters and other punctuation. It will look like this:

10.1016/j.acthis.2007.10.006

10.1186/1475-2875-9-284

The DOI provides a way to permanently find a particular item. Publishers and scholarly societies change their websites all the time. Recently, a major publisher completely re-did their website, messing up all links into their site. I was quite annoyed. But the DOI could still link you to an article in a way that a URL couldn’t.

The DOI is a great way to identify a particular journal article, but identifiers for journals have been around for some time. The ISSN is an 8 digit number applied to serial publications (magazines, journals, etc.), and can be especially helpful in distinguishing journals with similar names or commonly misspelled names. The ISSN suffers from some of the same challenges with respect to format as the ISBN. Typically, a publication could have two different ISSN numbers: one for a print version and one for an electronic version.

While ISBNs and DOIs apply to information sources across the disciplines, certain disciplines have specific information needs. In chemistry, keeping track of molecular compounds can be quite difficult. Historic names can be regional, obscure or too similar sounding to other terms. Compounds can be made of the same components but built differently. Compounds with similar structures can contain different elements or isotopes. In the mid-20th century, Chemical Abstracts Services, a division of the American Chemical Society, started assigning unique identifying numbers to each compound described in the chemical literature. With a few exceptions, each distinct chemical compound gets a unique CASRN. This help chemists purchase, research and use the exact chemicals they need for their work.

As the amount of information available to us increases exponentially and occasionally feels overwhelming, identifiers become more and more important to help us identify the information we need while filtering out the stuff we don’t. What identifiers do you use or come across regularly?

One of the most fun sciencey things I’ve seen lately is the #overlyhonestmethods meme on twitter. This was a cathartic exercise for scientists to say out loud the things that influence how they do science that often don’t make it into the clinical jargon-filled primary research articles they write.  There were lots of comments about the length of an experiment having to do with working hours or meeting times, some amusing comments about picking field locations due to weather, road access and good food and/or drink, and a few confessions regarding analytical equipment held together with duct tape.

But there was a whole section of confessions about how scientists work with the scientific literature.

One that caught my eye (and many others) was this comment by twitter user @droenn:

This tweet reminded me of a very clear example where this must have happened many, many times.

A student approached the reference desk looking for this article:

Tan, D. X.; Chen, L. D.; Poeggeler, B.; Manchester, L. C.; Reiter, R. J. (1993). “Melatonin: a potent, endogenous hydroxyl radical scavenger”. Endocrine J 1: 57–60.

The student had found the citation via the Wikipedia entry for Melatonin (the citation has since been removed). We started with the usual process: look up the journal, find the right volume and go from there. Except when you look up Endocrine Journal, you find that the volume number doesn’t match the year, nor are there any articles with a similar title in the publication. Author searches in the same journal also yield nothing.

Since the citation came from Wikipedia, it’s seemed reasonable that there was an error. Searches on Google and Google Scholar failed to find a copy of the article, but Google Scholar indicated that the article has been cited over 1300 times! The student found another article by some of the same authors on the topic and was content, but as a librarian, I wanted the answer.

Theoretically, the article had to exist, since it has been cited so many times. So I tried other databases: PubMed didn’t list the article at all, and Scopus indicated that over 1100 folks had cited it, but still didn’t have any information other than the citation.

So I started looking for similarly named publications. The journal Endocrine Journal is published by the Japan Endocrine Society and the years don’t match up, so perhaps the abbreviation referred to something different? I located a journal called simply Endocrine (try finding that one in a Google search!) published by Springer. This started to look promising because the first volume of of Endocrine was published in 1993, which was just want I needed. But volume 1 isn’t available on the publisher’s website, so I couldn’t confirm my suspicions.

If Endocrine is the journal we want, why can’t I find it indexed in a database? I checked indexing information. PubMed only started indexing it in 1997. Scopus started indexing it in 1993, but only with the fifth issue, and we need issue 1. And Google Scholar won’t have it (other than the citation) because it isn’t on the Springer website or in PubMed.

I started to think that the citation really refers to an article in Endocrine, not Endocrine Journal. But Scopus has over 1000 folks citing Endocrine Journal. It seems unlikely that so many people would make the same error.

Then I checked Ulrich’s guide to periodicals. We have it in print here, and the brief entry illustrates the missing piece of our puzzle.

The entry in Ulrich's clearly indicated this journal's former title, a fact that is missing from the publisher's website.

From 1993 to 1994 there were two Endocrine Journals!

For a brief period of time (<2 years), Endocrine called itself Endocrine Journal.  Perhaps they discovered the Japan Endocrine Society’s Endocrine Journal as the internet was making international collaboration easier.

Since I found the original ISSN (0969-711X, a number used to identify a periodical, no two have the same number), I submitted an ILL request to confirm my thoughts.  Sure enough, here’s the article masthead, but with Macmillan Press as the publisher, not Springer.  The early issues available on the Springer website have Stockton Press as the publisher in 1995.  It seems to have changed publisher several times.

Screen shot of the PDF file I received via ILL. Note the publisher at the top.

What’s the moral of this story?

1. Journals really need to select unique names. (Do new journals think about Google-ability of their names?)
2. Given my difficulty tracking this down, I have to ask: How many of the 1100-1300 folks that cited this article actually tracked it down? I bet there are some who never laid eyes on it.

And citing something without looking at it can cause problems. Let’s say that Jane Doe looked at the original article in 1994 and described its conclusions in a few sentences in her paper.  When John Doe (no relation) couldn’t find the original 1993 article, he used some similar sentences to Jane’s paper and cited the 1993 article.  If this happens a lot, that original brief description can become corrupted, and the original article could end up being cited to support conclusions that it doesn’t, in fact, support.

So, I suppose there is one more moral to this story:

• If you can’t find a copy of the article, don’t just cite it, ask a librarian to find it for you.

Other than your papers not being cited enough, what’s wrong with measuring scientific influence based on citation count? Citation analysis-based decisions concerning grants, promotions, etc. have become popular because, among other things, they’re considered “unbiased.” After all, such analysis gives numbers even non-professionals can understand, helping them make the best and most accurate decisions.

The written above is polite fiction. Why? First of all, citation analysis can only work with written, actual citations, but being influenced by something doesn’t mean you’re automatically going to refer to it. One of the basic assumptions behind citation analysis is that all, or at least most, of influences are cited in articles. It doesn’t work that way. MacRoberts and MacRoberts (2010) define influence as “When it is evident in the text that an author makes use of another’s work either directly or through secondary sources he or she has been influenced by that work.” According to a series of studies they conducted, only about 30% of influences are cited.

Secondary sources – Goodbye, citations. Once your article has been covered in a review or two, your findings will often be credited to the review article rather than your own. I’m citing only two MacRoberts & MacRoberts’ articles, one of them a review, because A. those are the ones I’ve read and B. I’m too lazy to read and cite all the research they refer to. That’s okay for informal scientific literature. However, if this was a peer-reviewed article, all the authors and articles not individually cited would have lost a citation. There’s a reason review articles are cited so often.

No informal citations. Those important conversations you had with your dissertation advisor or in a conference over lunch are forever gone, even though you might have gotten some of your best ideas from them. The paper you’ve been impressed with but couldn’t find a place to cite suffers the same fate. To quote MacRoberts and MacRoberts (1996) again:

“If one wants to know what  influence has  gone into  a particular bit of research,  there is  only  one  way  to  proceed:  head  for  the  lab  bench,  stick  close  to  the  scientist  as  he works  and  interacts  with  colleagues,  examine  his  lab  notebooks,  pay  close  attention  to what  he  reads,  and  consider  carefully his  cultural  milieu.”

They’re right, but their suggestion is hardly practical. That is why, in the last few years, bibliometricians have been trying to come up with metrics of academic social media cites. As the Altmetrics manifesto (2010) says “…that dog-eared (but uncited) article that used to live on a shelf now lives in Mendeley, CiteULike, or Zotero–where we can see and count it.” Unfortunately, Altmetrics indices are still far from accurate (not that citation indices are, but we’re stuck with them). If we’re to add new metrics to the mix, they better be good.

Limited databases. I mentioned it before in this blog, but it’s worth repeating: citation databases are painfully limited to a fraction of scientific publications, most of the covered ones being peer-reviewed journals. I have six Google Scholar citations for my blogs characterization article, but only two in Scopus. That’s one of the reasons your GS indices are usually higher than your Web of Science and Scopus ones. My dissertation advisor, Prof. Mike Thelwall, has an h-index of 47 in GS, 31 in Scopus, and 25 in WoS. All are correct, all are wrong. It depends on the coverage and the speed of update.

The Matthew Effect – or “the rich get richer.” People tend to cite already well-cited material by well-known researchers, either because that’s what they’ve read, because they’re appealing to the authority of the better known, or both.

Multiple motives -  as this helpful comics shows, there are multiple motives for citations, many of them have less in common with “giving credit where credit is due” than we would like to think.

Even if one is trying to be as honest and accurate as possible in her citations, she can only cite what she’s familiar with, and the number of articles one can read is limited (again, a factor in the popularity of review articles). She’s going to cite her professors, her co-authors, the people she heard in conferences and the big names in her field, but she is bound to miss some relevant material no matter what.

As much as citation analysis seems attractive, it’s not as accurate as we would like to believe. It represents only a part of the scientific world, and should not be taken as gospel.

ETA: I’m afraid there was a mistake in the original post – the Web of Science h-index for Mike Thelwall is 25 and not 15.

MacRoberts, M., & MacRoberts, B. (1996). Problems of citation analysis Scientometrics, 36 (3), 435-444 DOI: 10.1007/BF02129604

MacRoberts, M., & MacRoberts, B. (2010). Problems of citation analysis: A study of uncited and seldom-cited influences Journal of the American Society for Information Science and Technology, 61 (1), 1-12 DOI: 10.1002/asi.21228

Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). altmetrics: a manifesto http://altmetrics.org/manifesto/

Broadly speaking, metadata is simply a structured description of something else. The most popular example of metadata comes from the library catalog. Each book has a title, author, call number, publisher, ISBN etc. listed in the online catalog. These elements comprise the book’s metadata, and there are rules to make sure that things are standardized.

Without metadata, discovery and reuse of digital information would be much harder. This is why discussion about metadata has increased greatly since the second half of the twentieth century.

The Google Ngram of "metadata." The Google Ngram Viewer analyzes the full text of items in Google books and graphs the occurrence of the term entered over time.

The best way to understand metadata is to look at a few examples of metadata at work.

Here is part of a digital data table:

Who knew Santa uses Google Docs?

If you stumbled across this list on the web you might be able to guess what it was, but you couldn’t be sure. It would also be difficult to find this list again if you were looking for it.  The list creator might find this pretty useful, but if he or she shared it with others, we would want some added information to help the new user understand what he or she was looking at: this is metadata.

Metadata for this data file:

• Who created the data: Santa Claus, North Pole.  An email address would be nice. This way we have some contact information in case we need clarification.
• Title: “My List” isn’t a title that is conducive to finding the file again. While it might be tempting to just call this “Santa’s list” that won’t help other folks who see this file. The title should be descriptive of what the data file contains, and “Santa’s List” could be many things: Santa’s list of Reindeer?  Santa’s list of toys that need to be made?  A more descriptive title might be “Santa’s list of naughty and nice children.”
• Date created: We don’t want to confuse this year’s list (2012) with last year’s list (2011).  This could lead to all sorts of unfortunate events where nice kids get coal, naughty kids get presents, or infants (who weren’t around in 2011) get nothing at all.
• Who created the data file: Perhaps Santa created the data, but then used an elf to input the data into a computer file. Many computer programs automatically record this information, although you may not realize this.
• How the list was created: Behavioral scans? Parental surveys? Elf on the Shelf reports? All of the above?  In order to reuse this data in future research projects, we need to know how it was collected, including collection instruments and methodologies.
• Definitions of terms used: What is “naughty” what is “nice”?  How did Santa place a child into one category or another?
• File type: What kind of file is it? The data here are pretty simple, but Santa has lots of different file formats to choose from: excel, .csv, xml, etc. Knowing the file type helps end users determine if they can use the data

Naturally, a different kind of item might have a completely different set of metadata.

This is my mom’s favorite Christmas picture of me:

I wasn't amused.

My mom remembers the details of where, when and how this picture was taken, but if she isn’t around to tell the story, metadata can help:

Metadata for this photo:

• Date the photo was taken: December, 1981.  The digital version was created on 12/13/2012
• Who took the photo: A mall employee.  This can have implications for who owns the rights to use and distribute the image.  The photographer?  The folks who paid to have the photo taken?
• Camera used to take the photo: I have no idea what camera was used for this picture.  Luckily, modern digital cameras often automatically record this information as a part of the .jpg file.  Digital cameras can also record all the detailed camera settings (for those who understand these things).
• Location where the photo was taken: Arnot Mall, Horseheads, NY. Some digital cameras can automatically capture this information too, using built in GPS.
• Picture format: .jpg
• Picture size: Original size of the photo is 3.5 x 5.5 (I think).  The original scanned image is 852 x 1116 pixels.
• Description of the photo: Currently, the primary way of searching for an image is for a computer to search for the associated text. Good file names and good descriptions can be key to finding the image again.  Bonnie J M Swoger, age 3, sitting on Santa’s lap. Her grandpa brought her to the mall to visit Santa. While not enthusiastic about it, she loved her grandpa and obliged him by sitting on Santa’s lap.
• Copyright information: I don’t think the mall Santa folks were thinking about copyright in 1981 because there wasn’t an easy way to copy the photo. These days, it is important to state explicitly what rights other folks have to use the picture.  Creative Commons licenses are great for being explicit about what users can do with your content.

Depending on the type of data, there may be many more metadata elements.  Geospatial data, chemical data, astronomical data, etc. each have specific descriptive elements that are used.  Many organizations have developed standards describing what kinds of metadata should be included and how the metadata should be formatted.  This helps data creators add metadata that can be read by computers and reused by other interested folks.

Once you have well established metadata formats, you can start analyzing the metadata.  Common metrics used to evaluate scholarly publication (impact factor, alt metrics, etc.) all rely on high quality metadata.

I think we can agree that Santa would use sound data management practices, including the creation and use of proper metadata, to keep track of his gift giving and logistical data.  He would want the rest of us to use good metadata so we can always locate that 30 year old picture of him, too.

Be like Santa and make sure your data is findable and re-useable: use good metadata!

For a more robust (yet clear and understandable) definition of metadata, see NISO’s Understanding Metadata (PDF).

The tools I use to manage my own personal information can roughly be divided into two main categories: tools to manage information that I create, and tools to manage my acquisition and consumption of information created by others.

Managing the information I create:

• TaskPaper – simple to-do lists.  This is a mac app that works with drop box to create simple text-based to-do lists.  Lists can be separated into projects and sub-projects, tags can be added and (best of all) a line appears through a task when you check it off.  It’s purely psychological, but I like seeing that I’ve accomplished something, rather than having the task disappear.
• Google Calendar – If it isn’t on my calendar, it doesn’t exist.  Lately, I’ve been trying to schedule time to just work on various projects so they don’t fall off my radar.
• Evernote – Great for taking notes, composing blog posts, attaching pics and files.  Great for emailing notes and sharing links for notes. I use it for meeting notes, lesson plans (I can attach presentations or worksheets to my lesson outline), conference sessions and Christmas list planning.
• Paper Notebooks – Sometimes I still need to put pen to paper, and I’m picky about my pens and my paper.  I use a Notabilia notebook from Levenger for everyday notes, and a small pocket notebook to create short lists of what I need to accomplish today. And I can get a little irate if a colleague tries to walk off with one of my pens.

Managing information from others that I want to read:

• Google Reader – I have way too many RSS feeds for journal TOC, blogs I like, fun comics, citation alerts, etc.  I only consume a small portion of all this on a daily basis, and I’m probably overdue to delete some of my feeds.
• Diigo – For bookmarking websites.  Diigo has lots of collaboration and highlighting and note taking tools, but I mainly use it to bookmark websites I will never want to go back and read.
• Mendeley – The absolutely necessary tool for managing my collection of journal articles.  It’s like iTunes for journal articles.  Plus it helps with in-text citations and bibliographies.  I highly recommend it.
• SoundGecko – My new favorite tool.  I have a long commute by car, and while NPR, Wait Wait Don’t Tell Me and The Now Show fill some of the time, I like to use some of it productively.  SoundGecko takes all of those webpages that I didn’t have time to read and converts them to audio files I can listen to on my way home from work.  I’m using the free version now, which limits the number of articles I can get via audio, but subscription services are available.

With the exception of my paper notebooks, all of these tools are available to me on my computer and on my iPhone.

Sometimes I find that it is easy to get caught up in playing with the wide variety of tools available to manage information – the “Productivity” category of any app store is fascinating.  Unfortunately, it’s easy to miss opportunities to actually read or digest articles, stories and commentaries.

And sometimes it isn’t about the tools at all, but about focus, how you work and how you prioritize specific pieces or types of information.

What tools and strategies do you use to manage the “Information overload” that many of us face on a daily basis?

Mr. Rob Walsh

Obviously, the current academic publishing system has many flaws…Can you tell us which ones Scholastica is aiming to correct?

The first main problem is inefficiency of the entire review process. Scholastica is designed to decrease both the time between submitting an article and getting a decision, as well as decreasing the time from deciding to publish an article and actually making it available to scholars. Scholastica makes it easier to manage reviewers, and we include one-click Open Access online publishing to help scholarship be available as soon as possible.

The second main problem is the need for easy-to-use and low-cost infrastructure. We’ve heard from journal editors that there are two main drawbacks across most of the software out there to run a journal: first, it is clunky to use so editors waste time doing routine actions; and second, it costs too much either in terms of dollars or time. We’ve heard about this last point from many editors who are frustrated about having to spend time managing software installations, maintenance, training, troubleshooting, etc.

A third problem Scholastica fixes is the problem of technology getting in the way of experimentation. Many editors have told us about great ideas they want to implement to improve scholarly publishing – but get frustrated when they find how much time they waste trying to make technology do what they want. Scholastica eliminates the hassle of managing technology by being a hosted solution, and we pave the way for journals to experiment through one-click features and constantly partnering with journals to add new functionality that ALL journals can utilize. We remove the silo between journals so they can learn from each other and focus on using their time for what is most important: reviewing and publishing good scholarship.

How was your system received by editors and authors? Were you met with enthusiasm, skepticism, or perhaps both?

Everyone acknowledges that academic publishing exists somewhere on the spectrum from inefficient to archaic – so scholars have been really supportive of Scholastica because it is makes a clear contribution by making it easier and more efficient to run a scholarly journal.

We speak to editors using our system every week, which has helped us develop new features (and refine old features) that editors need to make their jobs easier. Since Scholastica is a centrally hosted solution, every time Scholastica gets a new feature it’s available for everyone. Editors at smaller journals have been especially pleased to have the same quality software that a larger journal enjoys, but with pricing scaled based on their submission volume.

Authors are also happy with Scholastica, especially since journals have the ability for authors to be notified as their submission moves through the process. This means that authors are alerted when such occurrences happen as reviewers being invited to the manuscript, when reviewers accept the invitations, when reviewers are submitted, all the way to a decision being made on the manuscript.

Looking through the site, I noticed that most of the journals using Scholastica are Law Reviews journals. Did you have law journals in mind when you designed the system?

Actually, we designed Scholastica with sociology and literature and history in mind (because the founders’ graduate work was in those areas), but our goal with Scholastica has always been to provide journals of all kind the infrastructure they require to be successful. We’re excited that some of the most well known journals using Scholastica are law reviews, but Scholastica is by no means specific to law reviews: journals in disciplines ranging from the social sciences to physical sciences to humanities use Scholastica.

We have added specific functionality that was designed with law reviews in mind, but through conversations with editors of non-law reviews we have been able to adapt these features to benefit all journals. And that’s really our goal, to build functionality that helps specific academic disciplines but then make that functionality available to journals of ALL disciplines in order to promote innovation and experimentation across disciplines.

The Scholastica submission fee is pretty modest (5$  per submission for Law Review journals, 10$ for others). How do you intend to sustain the project?

It’s been widely covered in the tech and business press how the costs to run a technology company today (hosting, storage, etc.) are a fraction of what they were five or ten years ago – and Scholastica is living proof of that. Our business model is pretty simple: there is a very modest fee for each article submitted to a journal on Scholastica. That’s it.

We’re very proud of the journal diversity we have in terms of submission volume: we have journals using Scholastica that receive 20 submissions a year and journals that receive 2,000 articles submissions a year. This is possible because we have pricing that scales with submission volume, and we give journals flexibility on who pays the article submission fee (the journal, the author, the home school, the host association, etc.).

Scholastica is currently integrated with ArXiv. What made you decide to create an ArXiv plug-in, and do you intend to create more for other repositories?

That’s a good question! When Timothy Gowers mentioned Scholastica in one of his blog posts after the Cost of Knowledge campaign kicked off, we started speaking to a lot of mathematicians. We learned that they frequently have their papers on the ArXiv pre-print server and that it would be really helpful to be able to submit papers directly from the ArXiv – so we built that functionality. We also open sourced a Ruby Gem to make it easier for others to work with ArXiv – you can find the code on our GitHub.

As I said before, we’re passionate about giving scholars the infrastructure that they need to be successful in managing and disseminating new knowledge. Whenever we’re approached with functionality that benefits scholars or journals as a whole we don’t hesitate to add it to our queue.

Is there anything else you’d like to add?

Scholastica is only as successful as the scholars and journals that help us make it great. It’s also a living, breathing product; we’re constantly refining it and building new features in concert with the dialogue we have with our editors and authors. We’ve got lots of exciting benefits in the pipe that we can’t wait to share with the scholarly community. If any of the readers have any feedback for us or new ideas about where they think digital scholarship needs to head, please do contact us – we love talking to scholars about how to improve scholarly publishing!

Scholastica's founders Rob Walsh and Cory Schires

I’d like to thank Mr. Walsh for the interview. Photos courtesy of Scholastica.

Scholastica’s blog

One of the recurring themes of this conference, no matter which hat I was wearing, was the need to make certain skills and concepts that are implicit to one group of folks much more concrete and explicit to another group of folks.

The Nature of Science. Dr. Lynn Marquez from Millersville University gave a great talk about teaching geoscience students about the nature of science. Scientists tend to have an innate sense of what is (and what isn’t) science. Some of them are good at explaining this to others, some aren’t. We understand what makes the scientific method work, and we believe that the best way to learn about science is to do science. But what about those students sitting in our large introductory lecture classes? In a class of 100 or more, often without a lab, can these students really learn about the nature of science by simply learning how plate tectonics works or by memorizing the different types of silicate minerals? Students may learn about what science has discovered, but do they really come away with a better knowledge of how the process works? Dr. Marquez argued convincingly that we need to make the nature of science more explicit in our introductory courses – we can’t assume that students will simply pick it up over the course of a semester. She argued that this can be done without making drastic changes to our current lesson plans, too. We just need to talk about the process of science a bit more.

Interestingly, my fellow SciAm blogger Doc Free Ride recently discussed the challenge of teaching students about the nature of science in a philosophy of science course that also (like our large lecture classes) doesn’t have students actually doing science.

Unfortunately, we can't learn from osmosis. Students don't learn about the nature of science by simply being in the same room as a scientist. CC-BY-SA image by Flickr user Sage Ross.

Data Management. I learned about several new geoscience data resources at GSA, and this underscores the increasing importance of high quality data management practices. Data that is made available on the web can be useless if it doesn’t come with good metadata – that is, the stuff describing what the data actually is and how it was collected. Conversely, well described data can disappear forever if it only exists in print lab notebooks on a PIs bookshelf. Data management is one of those skills that is often not explicitly taught to our novice researchers (sometimes because their advisors aren’t very good at it themselves). Funding agencies are starting to recognize the importance of good data management practices by requiring grant applicants to submit data management plans.

Information literacy. Part of being a good scientist is the ability to effectively communicate your research to others. Another part is knowing what other folks have already learned about your research area. Part of my job is to make this a bit more explicit to students. Taking the time to introduce students to different types of scientific literature, strategies for pulling together information, and ethical use of that information will make those students better prepared to communicate their science in the future. Importantly, they will be better prepared to think critically about the information they find on the web, because they will know what types of stuff they should be looking for.

Of course, taking class time (even just a bit) to talk about these things means less class time is spent on the concepts and facts that make up a specific subject.  Less subject content may be “covered.” I think the trade off is reasonable, largely because textbooks and online resources make it easier than ever for students to find out about scientific facts and discoveries on their own. Faculty can also collaborate to ensure that students aren’t getting the exact same nature of science, data management or information literacy lessons in each of their courses.  This isn’t just about educating the next generation of scientists, but about educating the next generation of world citizens who need to understand how science, data and information works in order to solve global problems.

Dr. Victor Henning

You co-founded (with  Jan Reichelt and Paul Föckler) Mendeley in 2007. What was your vision for Mendeley then, and has it changed since?

When we started Mendeley, we primarily wanted to solve our own problems: We were Ph.D. students and each had hundreds of research papers in PDF format that we needed to manage. We wondered why there wasn’t an easier way to manage those PDFs and keep them linked to bibliographic references, to enable you to create citations in the papers you’re writing. That was the first key idea behind Mendeley: Let’s develop a desktop app that can automatically turn my loose collection of PDFs into a structured research paper database that I can read, annotate, cite, and share with others.

The second key idea was: If we can get hundreds of thousands of researchers to use this desktop app to automatically extract information from research papers, why not crowdsource all of this data into an anonymized, open database? We realized that if the data was streaming in from researchers all over the globe in real time, we’d be able to analyze research trends across academic disciplines, connect researchers with similar interests, show readerships statistics for individual research papers, and generate collaborative filtering recommendations similar to Amazon’s, i.e. “people who have read research paper A have also read research paper B”.

That has been our vision from the start, and it hasn’t changed significantly since then. The biggest change was probably that we were initially quite focused on creating visualizations of research connections – e.g., you could visualize the citation network contained in your research paper collection as a 3D map within the Mendeley Desktop app. Later on, we dropped this because it was a magnificent task in itself, and we realized that we could let our community build visualization tools themselves if they wanted to. So our vision shifted to enabling third-party developers to tap into our crowdsourced data – for free, under a Creative Commons “CC-by” license – to build their own apps. Since the release of our Open API (http://dev.mendeley.com), more than 260 third-party research apps have been built –visualization tools, collaboration apps, semantic annotation tools, raw genome data mashups, expert finder services, Kindle and Android Apps, search interfaces, you name it. Our vision is now focused on turning Mendeley into a platform to let this ecosystem flourish.

The new Mendeley report shows that the biggest group of Mendeley users (31%) comes from the biology and medicine disciplines. Is it because you targeted those disciplines, because of their dominance in the global market of science in general, or are there other reasons?

We discussed whether to target specific disciplines in the beginning, but decided against it because we felt that Mendeley Desktop would be useful for anyone trying to manage academic knowledge – in the sciences as well as the humanities. I believe that the demographics of the Mendeley user base are possibly quite representative of the overall demographics of research: There are simply a lot more biologists and doctors than there are linguists and philosophers.

We did, however, target specific geographic regions first. Because all three of us Mendeley founders are German, our personal academic network was predominantly German, too. We wanted to avoid giving the impression to US and UK academics that Mendeley was German network, so we treaded carefully with inviting our friends first. Instead, when we launched, I did a speaking tour along the US East Coast and presented Mendeley at places like Princeton, NYU, Yale, Brown, MIT, Harvard, and Dartmouth – and it really took off from there.

Mendeley now has an institutional edition. What are its main functions, and what is its value for academic institutes?

In a bit more detail: Institutions that subscribe to the Mendeley Institutional Edition (MIE) can upgrade all of their students and faculty to a Mendeley premium account, meaning they receive more cloud storage/sync space, personalized research recommendations, enhanced group functionality, and premium support. The data dashboard tracks which journals are being read the most by the institution’s faculty and students, allowing their library to optimize their subscriptions – meaning, they can cancel journals that are not getting used, and subscribe to journals which are popular, but not yet provided by the library. This enables librarians squeeze more value out of ever-scarcer resources and provide a better service to their researchers.

Moreover, the MIE data dashboard tracks the faculty’s publications: In which journals are they publishing, and how much impact – measured in global readership – do these publications have? This helps institutions gather information for research excellence assessments and other reports, as well as highlighting to them who their current and future star researchers are.

Lastly, MIE contains a “social” tab which shows the librarian the public groups  in which their faculty are most active. Subject librarians can then potentially join in the discussion, re-establish connections with their constituency, and help them discover relevant research content. They can also set the OpenURL library resolver for all researchers at their institution to ensure that, once they discover interesting citations on Mendeley, receive full-text access to the content their library has paid for.

Your impact index (in the institutional edition) is a new way of measuring articles’ influence. Have you heard of any cases where it has been used in institutional decision-making? What do you think the index reflects?

MIE is still young – we only started rolling it out across campuses this fall semester – so I’ve only heard anecdotal evidence of it influencing institutional decision-making. All of our customers are excited about the prospect of making better subscription decisions while cutting subscription costs, and one librarian at a North American institution has told me specifically that he plans to use the data as negotiation leverage when discussing journal subscription renewals. It’ll be interesting how this plays out over the next few months, as we’re gradually expanding coverage to more campuses.

As for the meaning of Mendeley’s real-time readership index: I believe it is simply a very good measure of research impact, and a leading indicator of citation metrics. Logically, you should have read a piece of research before you cite it – though there are studies which suggest that, far too often, research gets cited without having been read[1]. Yet, we’ve never seen it as our job at Mendeley to interpret the data: We want to be a neutral data provider, and leave it up to each field of research to determine which metrics are important to them and how to interpret them.

Interestingly, there have been four bibliometric studies in the past year alone which have found a significant correlation of Mendeley’s readership data with Thomson Reuters’ Impact Factor, the Scopus Citation Index, and Google Scholar citations [2],[3],[4], [5] as well pointing out the remarkable Mendeley’s breadth of coverage.

Have there been any attempts to manipulate Mendeley’s numbers?

Not to our knowledge, and it would be very difficult to do so. Because of the scale of Mendeley – we now track the data of more than two million users – and its distributed nature, you would have to create thousands of fake “puppet” accounts from different computers, and have all of those puppet accounts read papers by specific authors. We do have measures in place to prevent bots from creating accounts, and we’d certainly notice odd usage patterns like the one I just described. So, it’s much harder to manipulate our data than it is to manipulate citations, which – as I’m sure you know – is a fairly common problem. Earlier this year, Thomson Reuters kicked a few journals out of their citation index because their editors had gamed their citations metrics.

The Mendeley Team

I’d like to thank Dr. Henning for the interview. Photos courtesy of Mendeley.

Disclaimer: even though most of the articles cited were written by my thesis advisors, they were cited by Dr. Henning – we’re simply working a lot about alt-metrics at the moment.

GSIS is a very small scholarly society made up of librarians who work with geoscientists, mostly at academic institutions.

GSIS is responsible for two major publications: the society newsletter and the more scholarly GSIS Proceedings.  The Proceedings stem from presentations given at our annual meeting, held in conjunction with the Geological Society of America Annual Meeting.  This year’s meeting is happening right now in Charlotte, NC.

Until now, the Proceedings has been a print publication and a benefit of membership.  We are long past the point at which we need to move online, and the society has been discussing the best way to do this.

Like many small organizations, we have several options, including open access and toll-access options.  As the society works through our options, here are some things we are thinking about:

Organizations considering open access have philosophical and practical concerns

First, many of us (myself included) are committed to the open access philosophy of scholarly publication.  As authors, we aren’t making money off of the publication, so we want the results to be easily and freely accessible to others.

Second, we have some concrete fiscal considerations.  The editorial process of producing the Proceedings is done by society volunteers and costs us nothing.  We do have some print costs to consider, and whichever option we select cannot be too much of a money looser for the society.  If we elect to make the Proceedings open access, we will need a publishing partner who can do this for a very low cost.  Thankfully, many libraries are now able to take on the publication of small journals by using open source publishing platforms like Open Journal Systems.  Even a library as small as mine can help scholarly societies make their information accessible – we are working with a New York education society to help them publish their journal Educational Change.

Third, receiving a copy of the proceedings is included in membership to the society.  If the Proceedings becomes an open access publication, does this change the benefits of membership?  Getting the proceedings volume was not the reason I joined the organization, but others might feel differently.

Fourth, as librarians and information professionals, we are deeply concerned about the long term stability of whichever platform/location we choose to store the virtual copies of the Proceedings.  If we select an institutional or commercial partner, we need to know that they will be reliable in the long term and committed to keeping our content accessible. Can they help us provide DOIs or stable URLs for our content? Long term accessibility is important.

The GSIS is not alone in thinking about moving to open access, and we are not alone in our concerns about how to best share our information with others.  Has your organization recently considered a move to open access?  What concerns did your membership have?  How did you resolve the issue?

In the end, this isn’t really about open access, but the fundamental nature of scholarly communication:  if we don’t share our scholarship, it will be like it never happened.  So our goal is to find the best way to share this with the audience that might be interested.

I did an email interview with Dr. Price about peer review, the future of academic publishing, and how Academia.edu fits into it.

Dr. Richard Price

First of all, it was a surprise to find out you have a doctorate in Philosophy, because bibiliometrically speaking, philosophers are “lone wolves” who prefer to publish alone or with very few coauthors. How did you end up building an academic social network?

I was doing a PhD at Oxford in philosophy, and, while I was doing that, I noticed the inefficiency in academic publishing. I remember there was a three-year time-lag between submitting my first paper to a journal and the journal publishing it. The disparity between the dynamism of the web as a whole and the system of academic communication and publishing was very apparent to me. Just imagine it taking three years for Twitter to publish a tweet; it’s unfathomable.

Philosophy papers tend to be more single-authored than papers in fields like biology, but the process that leads to a paper being written is still very collaborative: chatting about the ideas with colleagues, and so on. Furthermore, once one has published a paper, one is keen for it to be distributed as quickly as possible to the community at large, so others can criticize it, or build on it.

A lot of the world’s innovation in medicine and technology depends on progress in science, and so I thought this was a worthwhile problem to work on. When I finished my PhD, I raised $600,000 from some London-based angel investors and venture capitalists and moved to San Francisco to build the company. I wanted to be in San Francisco because the Bay Area is the main hub for technology companies in the world.

Have you noticed discipline-dependent differences in using the site?

Historically, we have focused on building functionality that serves the whole of the research world. Some common themes that we have focused on are the sharing of papers and analytics around the consumption of those papers.

Moving forward, we expect some aspects of the site to be more popular with some scientists than others, depending on the type of media they typically use to share their research. For instance, a biologist typically works with large data-sets, but a pure mathematician typically doesn’t.

I understand you’ve raised about seven million dollars from investors so far, but I suppose that won’t last forever (and that they’re expecting a return for their investment…). What is the company’s business model?

The goal is to provide trending research data to R&D institutions that can improve the quality of their decisions by 10-20%. The kind of algorithm that R&D companies are looking for is a ‘trending papers’ algorithm, analogous to Twitter’s trending topics algorithm. A trending papers algorithm would tell an R&D company which are the most impactful papers in a given research area in the last 24 hours, 7 days, 30 days, or any time period. Historically it’s been very difficult to get this kind of data. Scientists have printed papers out, and read them in their labs in un-trackable ways. As scientific activity is moving online, it’s becoming easier to track which papers are getting more attention from the top scientists.

There is also an opportunity to make a large economic impact. Around $1 trillion a year is spent on R&D globally: about $200 billion in the academic sector, and about $800 billion in the private sector (pharmaceutical companies, and other R&D companies).

Today, publishing in a top journal is considered a “stamp of approval”. In your opinion, what will be the future Web equivalent of publishing in a high-impact journal?

There will be a family of credibility metrics that reflect the impact of a piece of research on the scientific community. Ultimately, a credibility metric is trying to reflect the sentiment of the scientific community toward a particular piece of content. The historical peer review process ends up taking the opinions of two peer reviewers as a proxy for the opinion of the scientific community. As noted above, 2 people is not a large enough sample size.

One feature of the future of credibility metrics in science is that they will be based on much larger sample sizes. It is going to be possible to see what hundreds of scientists think of a paper, and not just what two people think.

Another feature is that there is going to be a family of signals about the quality of any given paper. Historically, there has been one signal of quality for an academic paper, and that has been the title of the journal that the paper is published in. In the last 5-7 years, citation counts have also emerged as a valid credibility metric, mainly because Google Scholar started making them available for any given paper. Academia.edu’s Analytics Dashboard is helping scientists see usage metrics associated with their work: page view counts, download counts, and related metrics.

Resources are scarce in science, and this means that there is significant competition for any given grant or job. When you are up for a job or a grant, there are typically 200 other people applying who have a similar number of peer-reviewed publications as you. You are incentivized to try to make your application stand out. That competitive spirit has driven the adoption of new credibility metrics in science: the citation counts and the page view metrics that Academia.edu offers. Many Academia.edu users take screenshots of their Analytics Dashboards and include them with their applications for tenure track jobs or grants. These credibility metrics demonstrate across a variety of dimensions the impact of the researcher’s work.

There will be a growing number of credibility metrics in science, each of which reflects a different kind of sentiment. This mirrors the diversity of credibility metrics on the web more broadly. There are credibility metrics as part of Twitter (followers, retweets), Github (repos, followers), YouTube (views), StackOverflow (reputation), Facebook posts (likes, comments). The way a person or a community thinks about a particular piece of content is complex and multidimensional, and increasingly credibility metrics will reflect that multi-dimensionality.

What will drive the adoption of these credibility metrics is the competitive spirit in the scientific community. A scientist is incentivized to add as many strings to his/her bow as possible when applying for a grant or a job. The way the credibility metrics will be introduced will be in a grassroots way, with scientists saying ‘This metric presents my work in a good light, so I am going to use it.’

It’s worth mentioning that any credibility metric in any domain is going to be gamed. The journal publishing system is subject to this as much as anything else. For instance, there is the practice of defensive citation: a scientist is incentivized to cite anyone who might conceivably peer review their paper. That is a way of gaming the journal system.

Looking at the broader web, people try to game Google. Google has a certain amount of built-in resistance to spam because its algorithm is recursive: Google looks not just at the number of inbound links to a website, but the quality of the linking site as well. Nonetheless, link farms exist to try to game PageRank. Any site that runs a credibility metric has to stay one step ahead of people trying to game or spam the system. This is a solvable problem, as many sites have shown. But it is an issue that you have to be prepared for.

Tell us a bit about your new analytical dashboard. How can it benefit its academic users?

The Analytics Dashboard fits into the general trend of scientists wanting a direct relationship with their audience and wanting to track analytics around that relationship. One reason they want to see those analytics is personal: in the researcher’s mind, the analytics validate that the research they are doing is having an impact.

But the more important role that the analytics have is professional: being able to establish to the world at large, and especially grant and hiring committees, that your work is having an impact. There  is a need to stand out from the crowd when applying for a grant or a job. The Analytics Dashboard on Academia.edu helps an academic do that.

Academia.edu offices

Should every scientist think about her or his self as a Web brand?

In the past, the journal would sit in between the scientist and his/her audience and mediate that relationship. We are moving toward a world where the personal brands of scientists are starting to eclipse those of journals. This is reflective of a broader trend occurring on the web, where sites like Twitter, Facebook, YouTube, Github, and others have enabled content creators to have direct relationships with their audiences.

We are moving toward a world where the key node in the network of scientific communication is the individual rather than the journal. The individual is increasingly going to be the person who drives the distribution of their own work and also the work of other people they admire.

I’d like to thank Dr. Price for the interview and Paige Schoknecht for arranging it. Photos courtesy of Academia.edu.

Other links:

The future of science, guest post by Richard Price in TechCrunch

Academia.edu blog

The primary goal is to “pay it forward” by talking with undergraduates. This year we also paid tribute to Dr. J. Mark Erickson, a long time St. Lawrence professor who recently retired.

There are three main parts to the conference:

• Opportunities for networking (mostly revolving around food and beverages)
• Research and professional talks. I know more about how the oil industry uses biostratigraphers (thanks, Art!) and more about the structural geology of the Adirondacks (thanks Graham!)
• Panel discussions about grad school and careers

I was able to attend one of the first alumni conferences when I was an undergrad, and I am happy to return and help out the current students in any way I can.

In the “pay it forward” portion of the weekend, I ended up a member of the career panel.  I was joined by folks who work in mineral exploration, environmental consulting, oil and gas exploration, and other careers that geology undergraduate students are actually interested in. I’m a realistic person, and I knew going in that no one in the audience had ever thought about librarianship as a career. And that’s okay.

I shared a bit about what science librarians do (hint: I don’t check books in and out all day), and the path I took to get here. I wanted to let the undergraduates know that it is okay if they don’t know what they want to be when they grow up yet.  They have time to figure it out (it certainly took me a while).

One of the interesting aspects of this conference is that as the sole librarian in attendance, I answer a lot of information related questions for the undergraduates and the alumni. Why is it so hard to locate a paper when I have the author and year? (Answer: often it is problems with author disambiguation). How can I see titles and abstracts of journal articles so I know which ones to ask my company to buy? (Answer: I’ll send you a list of resources). I really like these conversations, and hopefully my answers are helpful.

Near the end of the conference, I presented a talk about the contributions of my former professor, Dr. J. Mark Erickson, to the geological literature. Dr. Erickson, has been a champion of undergraduate research for 40 years, long before it was fashionable, and his publication record supports this. Dr. Erickson co-authored with his students on most of his publications, and had a remarkably consistent publishing career. In a brief biography written for the conference program, Dr. E mentioned his motto, which seems to sum things up rather well,

Know your self, know your stuff, know your students.

I’ll keep working on that.  Thanks, Dr. E.

The American Chemical Society is well known in library circles for having aggressive year-to-year price increases. Last year, my library cancelled its subscription to the “all ACS journals” package in favor of a new, smaller, package of 16 ACS journals to avoid an effective 11% price jump on the “all journals” package. The year before our cost for the ACS archive (pre-1995 journals) doubled as the ACS moved to a new pricing model. While prices for the smaller journal package held steady for us this year, I keep a list of things that we might need to cancel when (not if) prices increase faster than the library budget. I’m concerned that we will have to cancel this smaller journal package in favor of just a few ACS subscriptions sometime in the next few years. After several years of declining or steady library budgets, my library has made all of the “easy” cuts we can in order to afford scholarly content from the ACS and other publishers: the book budget has been slashed, we’ve cancelled many magazines and newspapers, the student worker budget has been cut, we aren’t binding print journals anymore, etc. Other libraries are in a similar position where the only thing left to cut are journal subscriptions.

Many folks have responded to Potsdam’s move on library and chemistry blogs and other news sites (John Dupuis has a good list of posts and commentary). Some applaud their actions as standing up to the “Goliath” of publishers. Others lament that their students will lose access to high quality research published by ACS. Few ask about the non-chemistry students who would lose access to their own discipline’s high quality research in order for Potsdam to afford the ACS subscriptions. Every time journal subscription costs go up faster than library budgets, something has to be cut.

For many folks, subscriptions to these journals were once considered un-cancelable. Perhaps we’ve been pushed to the point now where that is no longer the case.

In a recent article in the Chronicle of Higher Education the author talked to Jenica, a chemistry faculty member, and reached out to the ACS about the issue, expanding on some of the issues raised in Jenica’s original blog post. The ACS seemed to say that they aren’t going to talk to us about this in the places where we talk about this:

“We find little constructive dialogue can be had on blogs and other listservs where logic, balance, and common courtesy are not practiced and observed,” Glenn S. Ruskin, the group’s director of public affairs, said in an e-mail message. “As a matter of practice, ACS finds that direct engagement via telephone or face-to-face with individuals expressing concern over pricing or other related matters is the most productive means to finding common ground and resolution.”

This statement infuriated many mild mannered librarians. Librarians use blogs, listservs, and other new-fangled online communication tools to discuss library issues, to help each other out and to engage with publishers on issues of mutual interest.

After several chemistry librarians politely expressed their outrage and disappointment in the ACS on the chemical information societies listerv, Mr. Ruskin emailed a response, stating that a final sentence had been left off his quote:

“Therefore, we will not be offering any response to this blog posting or the conversation that has ensued.”

His explanation suggested that the ACS was not going to respond to Jenica’s blog post and this particular issue. I’m not sure that Mr. Ruskin’s clarification seems to help the matter. Whatever your feelings about the tone of Jenica’s blog post (or her previous posts about the ACS in which she has described herself as “feeling pointy”), Mr. Ruskin is still saying that they aren’t going to talk about this issue in the places where librarians talk about issues like this.

Personally, I don’t blame Jenica for using blunt language and the occasional curse word when talking about ACS with friends and colleagues. I became a librarian because I love information, and I’m passionate about teaching students to access, understand and use the scientific literature. When publishers make it more difficult for students to access it by charging fees that we can no longer afford, I feel “pointy” too. Just ask the librarians whose cubicles are next to mine.

Those of us struggling to provide our students and faculty with high quality research are frustrated that we don’t have partners on the other side of the table willing to engage with us in honest conversation about journal prices. We are left with few options: renew at the prices ACS is charging or cancel something. It isn’t a nice position to be in.

Talk to us, ACS. And I don’t mean by calling me privately. Engage with librarians and chemists about this issue on listservs and blogs. Open a dialog on what a reasonable pricing model would include. We know that you have good content, and we’re not expecting to access it for free. But when we can’t afford it anymore we are left with few options, and almost everyone loses. I would love to see a greater variety of journal package options (a package of 8 or 12 journals, for example) at a lower cost. I would like to see some honest figures about why my college’s cost per download is about 10 times the cost per download of our nearby university. I would also love to hear about how the aggressive price increases and higher-than-other-scholarly-societies subscription costs mesh with the mission statement of the ACS “to advance the broader chemistry enterprise and its practitioners for the benefit of Earth and its people.”

Don’t shut down the public conversation.

“Why are journals so expensive?”

We had spend the past 45 minutes talking about the scientific literature: what is peer review, what is a primary research article, and what happens after an article is published. I took two minutes to discuss finding journal articles and I gave my standard spiel about why students should use inter-library loan instead of paying for journal articles we don’t have: hundreds of thousands of your tuition dollars already pay databases and journal subscriptions. If we don’t already have a copy of the article you need in another database, we can borrow it from another library more cheaply than you can buy it online – free to you, low cost to us. Besides, the journal publishers don’t need more of your money. Journal subscriptions are much more expensive than magazine subscriptions – hundreds or thousands of dollars a year for just one journal.

The students’ eyes went wide at the last statement about the cost of journal subscriptions. You can get a year of People for just $100, or a year of Scientific American for only $25. So why does a library subscription to the Journal of Co-ordination Chemistry (24 issues per year) cost $11,367 per year?

I believe this graph of journal costs over the years is mandatory for any blog post, presentation or article that discusses journal prices. Here it is. The yellow dotted line is the consumer price index. The top red line is journal costs. From ARL.

Since I had about 90 seconds to provide some kind of answer to this question, my mind quickly raced through the details of the “serials crisis,” distinctions between journal prices for STM and humanities journals, the rise of the for-profit publisher after World War II, open access mandates and everything else.

In the end, I told the student there were two main reasons why publishers charge so much for journal subscriptions:

1. The subscription isn’t just for one person, it is so that everyone at the university could (theoretically) read that content
2. Because they can.

Journals have content (articles, reviews) that scholars and students want. Due to the nature of academic publishing, that exact same content (the results from a particular study or experiment) can’t be found in another journal. These mini-monopolies put power in the hands of publishers as scientists and scholars need access to particular content.

It’s kind of like the way that HBO can control its subscription price. If you want to watch Game of Thrones, you have to subscribe to HBO. You may get it via Time Warner or Direct TV, but HBO can still set the price. The major difference here is that unlike TV shows for entertainment purposes, some scholarly content can be considered vital to the educational and research mission of a college, making it difficult to say NO to.

This may be changing. There have been several high profile cases of libraries saying NO to high priced journal content (more on that next time), and researchers are more aware than ever of the repercussions of publishing in expensive journals. Hopefully this will lead to a greater balance of power between scholarly publishers and the institutions that purchase their content.

Trends in single and multiple authorship in computer science articles (Solomon, 2009)

The impact of team authorship

A 2007 Science article (Wuchty et al.) analyzed 19.9 million research articles taken from the Web of Science (WoS) database, as well as in 2.1 million patent records. Highly cited work was considered as one which received “more than the mean number of citations for a given field and year“. I have a bit of a problem with this definition, since it gives the same weight to a work which received 8 citations in a field where the mean is 3 and to a work in the same field which received 50.

They defined a measurement of relative team impact (RTI) for different periods and fields. RTI is “the mean number of citations received by team-authored work divided by the mean number of citations received by solo-authored work.” The RTI was bigger than one for all periods and all large research areas, and grew over time. In science and engineering, citations for team articles increased from an average of 1.7 times those of single-authored articles in 1955 to 2.1 times in 2000.

To make sure the teams’ advantage didn’t come from self-citations, they ran the analysis again without them. Removing self-citations reduced the RTI by 5%-10% (depends on the field), but the relative advantage of teams remained more or less the same. Not only that, but team-authored articles also have a bigger chance of being top-cited than single-authored articles. A team-authored article in science and engineering is 6.3 times more likely to receive more than 1,000 citations.

Aksnes (2003) looked at highly cited papers from the National Science Indicators (NSI) and the National Science Report (NCR) for Norway between 1981 and 1996. His definition for a “highly cited paper” was one that was cited “at least 17 times the mean NSI citation rate in the particular subfield/year.” The average number of authors per article in Aksnes’ sample was 8.9, but he mentions that the results are skewed due to 4 articles with more than 200(!) authors.  The average for the rest of the publication set was 3.7 authors per article. It’s worth noting that, unlike Wuchty et al., Aksnes chose to remove the art and humanities category from his analysis because bibliometrical databases mostly cover journals, and publishing in those disciplines is usually done through books and monographs. The increase in authorship Wuchty et al. showed in art and humanities (not too big anyway) could have been, in theory, cancelled out if they were able to include those kinds of publications.

A shows the change in fraction of articles and patents written by teams over time, while B shows the changes in average team size. Source: Wuchty et al., 2007

Credit Issues

Increased number of authors can mean dilution of credit. In a survey of medical schools’ promotion committees, a majority of respondents agreed that in a three-author byline, the first did most of the work, while the last deserved most of the credit for the idea and supervision.  The last author’s perceived credit didn’t change in a five-author byline, but the first author’s part of the idea credit decreased from 37% to 29% (Wren et al., 2007). I don’t want to know what happens in an article of 300 authors. Another study (Costas and Bordons, 2011) found what everyone already knows: junior researchers tend to be first authors; senior, older researchers tend to be last.

The up side of co-authoring with a senior, famous scientist is that there’s a better chance somebody’s actually going to read the article. The down side is that your name will pale in comparison, because people give more credit to researchers who are already well-known. That is what renowned sociologist Robert K. Merton (1968) called “The Matthew Effect.” The name comes from the Gospel of St. Matthew: “For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath.” Essentially, it’s the academic version of “The rich get richer.”

Team work is the present and the future of science. However, it might be exploited for less-than-ethical purposes. In part II of this post we’ll talk more about the size of teams, why has team work become so popular, and the ethics of authorship.

Costas, R., & Bordons, M. (2011). Do age and professional rank influence the order of authorship in scientific publications? Some evidence a micro-level perspective. Scientometrics, 88, 145-161

Cronin, B. (2001). Hyperauthorship: A postmodern perversion or evidence of a
structural shift in scholarly communication practices?
JASIST, 52 (7), 558-569 DOI: 10.1002/asi.1097

Merton, R.K. (1968). The Matthew effect in science Science (159), 56-63

Wren, J.D., & et al. (2007). The write position. A survey of perceived contributions to papers based on byline position and number of authors. EMBO Rep, 8 (11), 988-991 DOI: 10.1038/sj.embor.7401095

Solomon J (2009). Programmers, professors, and parasites: credit and co-authorship in computer science. Science and engineering ethics, 15 (4), 467-89 PMID: 19247811

Wuchty S, Jones BF, & Uzzi B (2007). The increasing dominance of teams in production of knowledge. Science (New York, N.Y.), 316 (5827), 1036-9 PMID: 17431139