Friday links: what Altmetric scores really measure, and more

Also this week: how to email your professor, advice on when to start a family, Greenpeace vs. Ray Hilborn, The R Objects That Shall Remain Nameless (all of them, apparently), and more.

From Jeremy:

David Wardle on what Altmetric scores really measure: whether you tweet your paper, how many times you tweet it, and whether you have many co-authors who also tweet it. In light of that, I agree with David that’s there’s no reason for you or anyone else to care about your Altmetric scores, or any reason for you to put them on your cv.

As an aside, David’s paper reveals that the proportion of highly-cited ecologists who use Twitter matches the proportion of all academics, or all adults, who do so: about 19%. A reminder, if one were needed, that the academics you meet on Twitter are only a sample of all academics.

Speaking of Twitter, hope you’re not too attached to it.

How to email your professor. Probably futile to share this, as the students who need this advice are unlikely to see it and follow it. Especially the “read the syllabus first” part. But I live in hope.

Margaret Kosmala’s concrete advice on when to start a family as an academic.

Andrew Hendry on Greenpeace’s attack on Ray Hilborn over purported conflicts of interest.

Sociologist and longtime blogger and Twitter user Kieran Healy on social media and academic sociology. Some of it is specific to sociology, but some of it generalizes. A couple of choice quotes, to give you the flavor and encourage you to click through:

To the degree that thinking, talking, and arguing about research in one’s spare time and in public is a feature your field, it is a sign that your discipline is confident about what it does.
[S]uccessfully engaging with the public means doing it somewhat unsuccessfully very regularly. This fact is closely connected to the value of doing your everyday work somewhat publicly. You cannot drop a lump of text onto the Internet and expect anyone to pay attention if you have not been engaging with them in some ongoing way. You cannot put your work up on your website, or “do a blog”, or manufacture interest in your research like that. There is a demand side as well as a supply side to “content”, and most of the time the demand side does not care about what you have to say. This is why, in my view, one’s public work ought to be be continuous with the intellectual work you are intrinsically motivated to do. It is a mistake to think that there is a research phase and a publicity phase.

Wait, objects in R don’t have names?

8 thoughts on “Friday links: what Altmetric scores really measure, and more

  1. Re: Altmetrics – all metrics can be gamed in one way or another and all need to be looked at in a more nuanced way than simply saying “tweeting papers increases your Altmetric score therefore Altmetrics should never be used”. The Altmetric algorithm down-weights Twitter mentions relative to (arguably more important) components (e.g. news items, policy documents):

    https://help.altmetric.com/support/multilingual/solutions/articles/6000060969-how-is-the-altmetric-score-calculated-

    I’m struck by how low those average Altmetric scores are for the papers, probably because the papers themselves are only about a year old and have not had time to accumulate mentions in policy documents, etc.

    Not saying that Altmetrics should appear on CVs necessarily, but they do give an indication of how the wider world is reacting to a piece of work, viz Kieran Healy’s points.

    • As I understand David’s piece, his concern is not, or not just, the weighting of tweets relative to other mentions. It’s that the number of tweets a paper gets is apparently so heavily sensitive to whether and how often the authors tweet it themselves. A metric that purports to tell me something about the paper (or the wider world’s reaction to the paper) is actually telling me something about the authors (specifically, their Twitter use).

      • i’ll second jeffs view here. (although i have to admit I still have to read the paper)
        but, fwiw: everyone can easily click on the altmetric score of any individual article where one finds a visual display how the share is affected by different mentions see here for an example http://www.psc.isr.umich.edu/newsletter/wp-content/uploads/images_spring2014/Science_shot.jpg
        most of the really high scoring articles have particularly a lot of news and blog coverage while for them tweets and facebook shares usually make up for less than half of the total score. given that news coverage is usually seen as a good thing today and people display it prominently on their cvs and pub lists, I think Davids point can be easily addressed by including altmetric info graphics instead of pure scores into your website/cv or whatever.

      • @Gregor:

        Fair enough, but that seems like a different way of making my point. As you say, people already list news coverage on their cv’s, and I think that’s fine and I presume David Wardle does as well. So if the point of including your Altmetric graph is “highlight independent news and blog coverage of my work”, well, why not just highlight independent news and blog coverage of your work rather than showing the Altmetric graph?

    • I had a moment of confusion also because it was “Altmetric” and not “altmetric.” I would suggest that “Altmetric” scores on a CV probably aren’t useful, but “altmetric” numbers (individual citation counts, etc.) are a bit more robust at least after a few years from publication. But then it also depends on the position…if a job description wants evidence of extensive public engagement (quite common in the natural history museum world), an Altmetric score is probably pretty handy.

      • “if a job description wants evidence of extensive public engagement (quite common in the natural history museum world), an Altmetric score is probably pretty handy.”

        Sure, although that gets to a different question–“is the *applicant* publicly engaged” vs. “what’s the broad level of interest in this *paper*”?

        And if the question is about whether the applicant is publicly engaged, I personally probably wouldn’t consider the Altmetric scores of their papers the *most* informative metric. I’d probably care more about, say, how many Twitter followers the applicant has (and many other lines of evidence of public engagement, many of which have nothing to do with social media).

    • I don’t tweet my papers, so I can’t speak to David’s point.

      But I can say that within the scope of my papers, Altmetric say 1 month after it goes online is a pretty good predictor of how it will be cited say 2 years down the road.

      It may be that Altmetric is more useful for comparing papers within an author than across authors (due to varying social media behaviors biasing things). Or it may be that Altmetric is good at sorting out top 1% papers vs top 10% vs top 50% (by citations), but very prone to twitter habits in scoring bottom 50% papers (to be clear I have papers in all these categories and Altmetric did sort them pretty well, but my papers in the bottom 50% just 2 or 3 more tweets would have made a noticeable difference to its Altmetric, but not so much the ones in the top)..

  2. One reason to care about Altmetrics I can think of: NGO Funding bodies seem to increasingly care about Altmetrics. Just that alone is good enough reason for me (a poor student with few funds) to also care since I am asking for their coin.

    I also understand why they care- they want to see the work they fund (and their name) seen by as many people as possible. They seem to think Altmetrics is a fairly good way for them to gauge that. To them its also largely irrelevant how people see the paper, that is whether it is the author or someone else that brings it to peoples attention, as long as people see it.

    Altmetrics is also quicker than trying to manually keep track of every media output so it is useful on that front as well. As an aside, it is probably easier to ‘game’ your social media followers so using that as a measure (even a partial one) is also problematic.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.