Last year, we launched a new premium feature called Mentions, which notifies you whenever papers uploaded to Academia.edu mention your name. You can also make these mentions public to demonstrate the impact your work is having on scholarship and science.

We built Mentions, in part, because citation data — increasingly an important part of tenure and promotion decisions — misses many forms of impact. An academic might acknowledge you for providing input on a draft, or a colleague might refer to your forthcoming work, but without a full citation; others may cite your published work in a thesis, draft, foreign language, syllabus or bibliography. Many citation counts, such as those generated by Google Scholar, miss these alternative forms of impact.

We wanted to know what percentage of Academia Mentions are missed by Google Scholar. So we randomly selected 77 Academia users who have made between 5–20 of their mentions public, and cross-checked them with Google Scholar. We found that about 43% of Academia Mentions are not known to Google Scholar.

Impact is multidimensional, and impact statistics should reflect this multidimensionality.

Academia’s mission is to make all research available for free to everyone in the world. Upgrading to premium is a way to support our mission and help make all scholarship and science easily and freely accessible to everyone, not just those affiliated with well-endowed institutions.

If you don’t see Mentions in your Academia.edu navigation bar, that’s probably because we haven’t rolled out this premium feature to you yet. We expect a full site-wide rollout within a couple of months. If you’d like early access, let us know at premium-support@academia.edu! If you have any other questions about Academia.edu, or you’d like to write a guest blog post for us, let us know at support@academia.edu.

Does Google Scholar know about your Academia Mentions? was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

“Someone just searched for you on Google and found your page on Academia.edu.” Ever received this email and wondered how you were found? Google stopped providing us most of these terms in 2011, so for now, keep on wondering.

But, if you do wonder, then check out our newest premium feature, “Academia.edu Searches”! In the keywords tab of your analytics, you’ll see what search terms others use on Academia.edu to find your papers. This can help you understand what specifically about your work is attracting attention, and perhaps even help direct your next project.

You can view the terms for every Academia.edu search where we suggest your paper, and the rank of your paper on the results page. (The search term is premium, the rank is free!). As always, the external keywords sent to us by Google, Yahoo and Bing are still free, and still in your keywords tab of analytics!

Academia’s mission is to make all research available for free to everyone in the world. Upgrading to premium is a way to support this mission and help make all scholarship and science easily and freely accessible to everyone, not just those affiliated with well-endowed institutions.

Academia Premium also includes features such as Mentions, Readers, Advanced Search and Extra Analytics, in addition to Academia.edu Searches. If you don’t see these features, we probably haven’t rolled them out to you yet. We expect a full site-wide rollout within a couple of months. If you’d like early access, let us know at premium-support@academia.edu!

How do people find your papers? was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

Academia hosts a global community of faculty members, students, independent researchers, and other academics. Over 45 million people have joined the site, and more than 7 million of them have listed a university or other organization that they’re affiliated with. In all, more than 130,000 universities in over 80 countries are represented on Academia.

Given the diversity of places where Academia’s users come from, a natural question is, which universities have the largest presence on the site? Searching for a university’s name brings up its current user count, but there’s no easy way to compare universities on the site itself. So we dug into our data to find out what the top universities are on Academia.

The university rankings look very different depending on which users we include, so we’ll answer three versions of the question above:

• Which universities are the most common among all Academia users?
• What are the top universities among people who joined the site recently?
• Which universities do the most active users come from?

The top universities among all Academia users

The graph below shows the top 20 universities among all Academia users (as of November 1) who have listed affiliations, broken down by position. The university with the largest presence on the site is the National Autonomous University of Mexico (UNAM) with about 36,000 Academia users, half of whom are alumni of the university. For comparison, UNAM currently has about 350,000 students and an academic staff of almost 40,000. Next on the list are Anna University (India) and two Indonesian universities, the University of Indonesia and Gadjah Mada University, each with about 24,000 to 26,000 users. Several universities with 10,000 to 20,000 users each round out the list, including 5 more in Southeast Asia and India, 4 in the UK, 3 in the US and Canada, 3 in South America, and 1 in Europe.

Most of these universities have many Academia users who are students (both undergraduate and graduate), faculty members, or other department members. The number of alumni from each university varies widely, and in general there are few postdocs, adjuncts, or emeritus professors.

The fastest growing universities on Academia

Which universities are currently growing the fastest on Academia? Among all users who joined in October 2016 (about 1.5 million, including 51,000 with affiliations), the top 20 universities are shown in the graph below. This list of universities is similar to the first one in many ways: UNAM is still at the top of the list, and all of the top 8 for October are among the top 20 all time. The rest of the list shows rapid growth from several universities in Asia, Europe, and Africa that don’t already have large populations of users on Academia. Compared with all Academia users, more of the users who signed up in October are students and fewer are faculty members or other department members, although there are some exceptions: for Umm Al-Qura University (Saudi Arabia) and the University of Dhaka (Bangladesh), the majority of new users are faculty members. We’re only looking at each user’s primary affiliation here, so it’s possible that some of the users who signed up several years ago were students when they joined but are now faculty members or hold another more advanced position.

The top universities among active users

Finally, let’s look at which universities have the most people who are currently active on Academia. There are many possible ways to define activity on the site; here we’ll include anyone who clicked on at least one link on Academia on 15 or more separate days in October 2016 and had already joined the site before the start of the month. This gives us about 22,000 people, over 19,000 of whom have listed university affiliations and positions. In the graph below of the top 20 universities among these active users, we see some of the same universities that appeared in the previous lists. However, there are also many European universities (and a couple in the US) with large shares of active users despite ranking lower in total user counts. The typical positions of active users at these universities are different, too, with many more faculty members, department members, and postdocs, and far fewer undergraduates.

Splitting the Academia community by university affiliation and position is just one way to look at who’s using the site, but there are plenty of others that we plan to explore and write about in the future. If there’s another question about Academia’s network of users and papers that you’d like to know the answer to, let us know in the comments.

Which universities have the most Academia users? was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

Academia’s mission is to accelerate the pace of discovery. We’ve added a new premium feature called Mentions to help you demonstrate the impact of your work. You’ll receive notifications whenever others upload papers that mention your name.

Think of Mentions as lightweight citations. You might discover that a professor acknowledged you for providing input on an earlier draft, or a colleague referred to your forthcoming work, but without a full citation, or another cited your published work in a thesis, draft, foreign language, syllabus or bibliography, and so Google Scholar ignored it. Mentions helps you find these alternative forms of impact often missed by citation counts.

Starting today, you’ll also be able to make your Mentions public to anyone visiting your profile page. Just go to your mentions page and select “make public.” Once you make at least one mention public, you’ll see a link on your profile called Public Mentions.

The idea for Mentions came from conversations we had with academics about the need to demonstrate the global impact of their research. As the legal historian Paul J. du Plessis explained:

“Academia’s Mentions feature found papers citing me that Google Scholar had missed. I am adding this to my tenure case.”

Dennis Klass, who specializes in the psychology of religion, added “I have never known as much about who is using my ideas as I do now.” According to Klass, Academia’s Mentions feature found dozens of new references to his work not known to Google Scholar.

Mentions also makes it easier to connect with other researchers in the field. Claudia Boscolo, for example, a scholar of Italian Studies, said that “discovering that I was mentioned or fully cited in articles and books — which I previously had no idea they had been published — helped me to establish new relationships and connections.”

If you do not see Mentions anywhere on Academia.edu, that’s probably because we have not rolled out the feature to you yet. We expect a full site-wide rollout within a few months. If you’d like early access, let us know at premium-support@academia.edu!

What’s a ‘Mention’ on Academia? was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

Readers have the option to tell authors what sparked their interest in the paper. These will be sent to authors as messages, who will now be able to learn much more about the impact their work is having in the world. Here are a few examples of ‘reasons for downloading’ left by readers:

A farmer in the sahara posted about water conservation:

“I farm in the Sahara desert so conserving water is important to me, and I want to find out more about how to re use our limited water to feed myself and my animals.”

A teacher left a reason for downloading a paper on longhand vs. laptop note taking:

“My kids are in a K-8 school. The school says pencils will be banned in 2017. I am looking for research to change that decision”

A bachelor left a reason downloading a paper on Masbatenyo grammar:

“I just met the girl I want to marry and I want to know about her by learning her native language”

A computer scientist left a reason for downloading a paper on ethnobotanical quantification:

“I am working on digitizing Strychnos Ligustrina for traditional medicine to cure Malaria”

A farmer left a reason for downloading a paper on acid soils:

“I want to increase production of corn in the jungle of Peru. So I need more knowledge about NPK. It is hard to find research for jungle soil and new hybrid seed. Usually the hard yellow corn has low t/ha but they don’t need any care. On the other hand, hybrid corns need fertilizer but there is not much information available to us.”

We would also love to hear your feedback about this new feature. Please feel free to contact us at hello@academia.edu.

Academia Introduces ‘Reasons For Downloading’ was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

We announced the results of the study on our home page, and it was covered by Fortune Magazine. More importantly, we put all of our data and code online. Anyone could — and still can — download our data and code, and easily replicate or modify any part of our study.

The study generated some discussion. A week after our announcement, Phil Davis published a blog post raising some questions about our data sample. He pointed out several “non-research” articles in our sample and asked whether the presence of these in the data might explain the result.

In response to that question, we have spent the last several weeks classifying the nearly 45,000 articles in our dataset, and identifying such “non-research” articles.

Today, we’re pleased to announce our revised study, which answers that question. Excluding any articles that we did not identify with high confidence to be original research or scholarship, we find a 73% citation increase associated with articles posted to Academia.edu. This is a little less in relative terms than the 83% we found in our original data, but amounts to approximately the same in absolute terms — about one extra citation every year for the median paper. We find a 64% citation increase to articles posted to Academia.edu compared with being posted on other open access venues, such as an open access journal, or a personal homepage (down from 75% in the original data).

The revised paper is available for download here.

Just as with the original version of the study, all of our data and code are available online at the paper’s Github repository. We encourage those interested to download these materials and do their own review of our work. Promoting open science is one our core values. We believe that opening up research accelerates, improves, and democratizes scholarship and scientific discovery.

In the rest of this post, we’ll describe some of the background leading to our revised paper, and the methodology we used to refine our result.

Our Original Paper

Our data consists of a sample of journal articles published and posted to Academia.edu from 2009 to 2012, along with a sample of articles published in the same journals, but not posted to the site. You can find more detail about our data collection in the paper.

Controlling for the age of the articles, the impact factors of their journals, their academic disciplines, and whether they were freely available online (besides on Academia.edu), we compared the citations to articles posted to the site with those to similar articles not posted. We found that a typical article with full-text available online on a non-Academia.edu site received about 20% more citations than a similar article with only paid-access. We also found that an article posted only to Academia.edu received about 83% more citations than a similar paid-access article.

A Review from The Scholarly Kitchen

A few days after we publicized our results, we were contacted by Phil Davis, a researcher specializing in academic readership and citation. We corresponded with Dr. Davis for a few days, answering questions he had about our data and methodology, pointed him to the public data, and encouraged him to review it.

After that review, Dr. Davis raised a question about “non-research” articles in our dataset. Specifically that such articles, which often receive few citations, could be driving down the citation estimates for the non-Academia.edu sample and biasing our results. Dr. Davis subsequently wrote up this question in a blog post on Scholarly Kitchen, “Citation Boost Or Bad Data: Academia.edu Research Under Scrutiny”.

How could “non-research” articles affect our study?

Academic journals publish more than just articles with original research. They also publish content such as book reviews, errata, editorials, letters responding to recent articles, and even obituaries. These “non-research” articles often receive few or no citations.

In his blog post, Dr. Davis claimed that we were comparing articles posted to Academia.edu that were mostly research articles to articles not posted to the site that contained there other types of articles — comparing citable apples to uncitable oranges.

For example, consider an on-Academia sample with eight research articles, one book review, and one erratum. The research articles all receive 10 citations, while the book review and the errata each receive only one. The average citation in this group is 82 citations divided by ten articles, or 8.2.

Next, consider an off-Academia sample with five research articles, three book reviews, one editorial, and one erratum. The research articles all receive 10 citations, while the others each receive only one. The average in this sample is 55 citations divided by ten articles, or 5.5.

Therefore, even though research articles in both the on- and off-Academia sample received the same number of citations (10), if we didn’t account for the “non-research” articles, we would estimate a 50% citation difference (8.2 over 5.5).

So it’s a good question — does the presence of “non-research” articles in the data set explain any or all of the citation difference? For the presence of “non-research” articles to cause a big effect: (1) there have to be a lot of them, and (2) they have to be more prevalent in the off-Academia sample. If (1) isn’t true, then they’ll have little effect on the average citations in each sample. If (2) isn’t true, then they’ll have the same effect in both groups, which will get canceled out in the comparison.

Could this explain our result?

Looking at our data, we saw that in order to conclude that “non-research” articles explained the entire result, we would have to make some extreme assumptions:

1. There were no “non-research” articles in the on-Academia sample.
2. All “non-research” articles in the off-Academia sample receive no citations.
3. 25% of all papers in the off-Academia set are “non-research.”

It was easy to confirm that (1) and (2) were not true. This means that the share of non-research articles in the off-Academia sample would have to be even higher — something like 1 in 3.

Even without a thorough review of all the articles in the data, it was unlikely that a third of all the articles in our off-Academia sample were non-research articles. But, in order to answer the question definitively, we decided to conduct a thorough review of all the articles.

As we’ll see below, our review of the data found that the share of non-research articles was closer to 1 in 10, and that their presence in our data accounts for very little of our result.

Classifying the articles in our sample

One way to address this problem is to identify “non-research” articles, and remove them from our data.

To identify “non-research” articles, we used Amazon’s Mechanical Turk. Mechanical Turk (or MTurk) is a marketplace for crowd-sourcing surveys or large data processing tasks. It is commonly used by academics to, for example, perform online experiments, collect survey data, or train and validate machine learning algorithms.

We provided links to the journal site for each article in our sample to over 300 MTurk workers, and asked them to answer some simple questions about the abstract or full text they found there (a sample instruction page is available in an appendix to the paper. Primarily, we asked them to classify the article as one of the following types:

1. A summary of a meeting or conference
2. An Editorial or Commentary
3. A response to a recent article in the same journal;
4. An article with original research, analysis or scholarship, or a broad survey of research on a topic
5. A Book Review, Software Review, or review of some other recent work or performance
6. An Erratum, Correction, or Retraction of an earlier article
7. Something else

Sometimes, a worker might fail to categorize an article, giving one of these reasons: the link was broken, there was no abstract or text available on the site, the article was in a foreign language, or they just couldn’t tell.

We had each article reviewed by three different workers. In the final version of our analysis, we only included articles that three workers all agreed were “Original Research” (option 4 above). This left us with about 35,000 articles from the original 45,000. The other 10,000 were excluded because either the workers could not agree on a classification, or they unanimously agreed on a “non-research” classfication. Many of the excluded 10,000 are surely original research articles also, but to have the highest confidence we excluded non-research articles, we only included the unanimous results. (If we went by majority rule, or two out of three workers’ classifications, then we would have classified about 40,000 (90%) of 45,000 as original research articles.)

We checked the quality of the workers in two ways. First, we removed any workers with suspicious results, such as giving all their articles the same classification, or completing tasks unreasonably fast. Second, we compared their results to a “Gold Standard” set of 100 articles that were classified by Academia.edu staff. We included in this set a number of articles we considered tricky to classify. Based on a majority-rule classification, the workers agreed with us on whether the articles were original research or not over 85% of the time.

Updated Results

This is an extract of table 13 in the original version of our paper. It shows the citations predicted over time for articles (1) not freely available online, (2) available online but not on Academia.edu, (3) available only on Academia.edu, and (4) available on Academia.edu as well as elsewhere online. These figures are based on articles published in the median-impact-factor journal in our sample, and are estimated using the original sample of 45,000 articles.

After 5 years, we predicted that an article only available behind a paywall in the median journal will receive 7 citations. An article posted to Academia.edu, on the other hand would have received 12.9 citations, a difference of almost 6 citations, or 83%.

Below is the same table, updated to only include the 35,000 articles that were unanimously classified as original research. Notice that the predicted citations are higher across the board — in part a result of excluding low-citation “non-research” articles. Here a paid-access-only article is predicted to receive 8.1 citations, while an article posted to Academia.edu is predicted to receive 14, a difference of, again, almost 6 citations, but only 73% (because of the higher baseline).

More detailed results are in the paper.

There’s no perfect data

Data on academic publications and their citations is known to be complicated and imperfect. Even large, widely used data sources, such as Thomson’s Web of Science, have been found to have inaccuracies.

Categorizing academic journal content is especially complicated. There’s no single agreed-upon way to classify content, and the wide variety of types of content found in different journals across different disciplines makes is difficult to make comprehensive rules for doing so. For example, Scopus and Web of Science use very different categories for document types, and often disagree. Even the authors of our study sometimes debated with each other the proper classifications of certain articles in our sample.

It is possible in reviewing the raw data, someone will find some classifications they disagree with or believe are inaccurate. But this is also the case with nearly every study on academic citations. We believe that our classification process, while it has some unavoidable imperfections, is sufficiently accurate for us to conclude that “non-research” articles do not explain a significant amount of the citation advantage we find for articles posted to Academia.edu.

Conclusions

We provide the data and code for our work because we want to engage in an open scientific discussion of our research. Having data and code available means that any researcher with a competing hypothesis can evaluate it in a complete and rigorous way. Dr. Davis’s critical feedback was useful and much appreciated. We think the work resulting from it has strengthened our analysis.

The outcome is a small modification of the original result: a paper in the median impact factor journal receives 73% more citations over five years if uploaded to Academia.edu (rather than 83%). Dr. Davis’s hypothesis that the original result may have been “entirely explainable by bad data” is not borne out.

— Carl Vogel, Yuri Niyazov, Ben Lund, Richard Price

Academia.edu, Citations, and Open Science in Action was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

Academia.edu just passed 10 million users. I remember when the site started, and a hundred or two hundred people were joining each day. Now around thirty to forty thousand people are joining each day.

The ten million milestone reflects the growing interest in open science. A few years ago open science was a niche movement. It’s now starting to be mainstream to want to share papers openly.

We need to get to a world where every science PDF ever written is on the internet, accessible for free. Why is this important? The main reason is that spreading knowledge is a wonderful thing that can lead to all kinds of unexpected benefits. It can lead to a more informed population that can make better decisions about things like stem cell research or climate change.

We need to get to a world where every science PDF ever written is on the internet, accessible for free.

Another benefit is that outsiders may come to science with fresh perspectives and say “I have a crazy idea” There are a few examples in science of outsiders coming to science with a beginner’s mind and moving the field forward. A car mechanic in Argentina, Jorge Odon, came up with the first invention in assisted child birth in 150 years after watching a YouTube video about getting a cork out of a bottle.

It would be good if science was able to harness crazy ideas from wherever they can be found. I wrote about some more examples of outsiders bringing fresh thinking to science in a recent guest article in Times Higher Education.

It would be a great thing if we could get every science PDF ever written on the internet, available for free. There is a lot of work to do before we make that vision a reality, but this 10 million user milestone is a good start.

by Richard Price, Founder of Academia.edu

Academia.edu passes 10 million users was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

In research, we stand on the shoulders of giants past. We gather together with our peers to clarify, develop, and validate new insights.

To reflect the collaborative nature of research, Academia.edu now supports adding co-authors to papers. That simple act is an act of sharing — sharing credit for work done, and sharing your research with a wider audience. Here are some ways adding co-authors will improve your experience:

Increased Exposure: Adding co-authors increases the ways that a paper can be discovered.

Saved Time: The papers of tagged co-authors appear across many profiles, enabling all authors to take credit for their contributions, and saving busy researchers from repeated uploads.

Sharing Analytics: When your co-author uploads a paper and tags you, you immediately gain access to the analytics for that paper. Co-authors share the views, downloads, and bookmarks of works they’ve been added to.

Building connections between researchers is at the center of Academia.edu. Adding co-authors to your papers is a great way to strengthen your existing connections and accelerate the dissemination of important research in the process.

Co-authors can be added to your existing papers here: https://www.academia.edu/coauthors_wizard

Introducing co-authored papers. was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.

There were many reasons for changing the design and backend architecture of the feed. Namely, we wanted to load more stories, load them faster, and create more emphasis for each item as a separate event. Each story is now contained within a separate white module so that they are more clearly distinguishable. We’ve also added abstracts if they are available to provide more context without having to leave your feed.

We hypothesized that all of these changes would result in more researchers interacting with their feed, which can be measured in views of papers, bookmarks, and downloads.

It’s only been a week since this update rolled out, so we’ll continue monitoring the data and listening to your feedback to determine how to make it better. So far, it looks like we’ve roughly doubled the number of downloads and bookmarks of papers per day.

We think this is great news. Researchers should too. Providing the easiest way to gain more exposure for research is what drives our product development. Take the newsfeed as an example: the new design has resulted in a huge increase in downloads and bookmarks which means more papers are being discovered and recirculated since the change.

The platform that makes research the most discoverable is best for everyone.

However, the updates won’t stop here. In an effort to continue to make Academia.edu the best way to share and discover new research, similar changes and experiments will be coming to other features. We’re adding roughly 70,000 researchers to our community every day. Because of this, we have to experiment with alternative solutions to ensure that we are creating the best experience for both new researchers as well as existing ones. Sometimes this means making imperfect compromises but we believe providing the platform that makes research the most discoverable is best for everyone.

We’ll be experimenting with some adjustments and watching the metrics to make sure we’re building both a functional and enjoyable experience. If you’d like to share your thoughts on the newsfeed redesign, or other changes, let us know. We look forward to hearing from you.

by Conway Anderson, VP of Product at Academia.edu

Your feed just got better. was originally published in Academia on Medium, where people are continuing the conversation by highlighting and responding to this story.