Also this week: Get Out The Science, RIP Kenneth Arrow, should we all move to Switzerland?, blogs > Twitter, and more.
From Meg:
I definitely have this:
Get Out The Science is calling on scientists to spend the week after the March for Science (that is, April 24-31) sharing, inspiring, and engaging related to science. Some ideas (more info at the website):
From Jeremy:
Kenneth Arrow has died. He was one of the world’s greatest living economists, the youngest ever to win the Nobel Prize. He’s perhaps best known for founding an entire subdiscipline, social choice theory. Social choice theory started with Arrow’s famous “impossibility theorem” proving that, when voters have three or more options, no rank order voting system can convert those options into a community-wide ranking that obeys certain intuitively-plausible desiderata. In other words, those desiderata are not all compatible, implying that no single ranked ballot system is always and everywhere superior to the others (so all those arguments you read on the internet about the “best” voting system are wrong) Arrow also made foundational contributions to environmental economics and government innovation policy, to name two areas of some interest to readers of this blog. But to focus on any one of Arrow’s contributions is to fail to do him justice. To wit:
Stephen Heard expertly trolls the open access evangelists by asking why they aren’t also arguing for open access groceries. As with all the best trolls, Stephen has more of a point than you would think.
How Planet Earth II filmed the “snake island” chase sequence.
Axios Review is closing. I was an editor for it and I’m sad to see it go. I think it was a tremendously useful, narrowly-targeted publishing innovation. But not enough authors agreed–there wasn’t just enough uptake for it to break even.
The US National Academy of Sciences runs a scientific consulting hotline for filmmakers. The link includes a short video on the role of scientific consultants in Arrival. Related: my review of Lab Coats In Hollywood, a fascinating and fun book about Hollywood science consultants, by a scientist who became one.
I’m visiting Zurich next week, where I’ll see a number of US, British, and Canadian colleagues who’ve been hired by ETH Zurich or other Zurich institutes. Like Justin Fox (no relation), I’m curious whether we’ll see more US academics and grad students heading to Zurich in the next few years.
One of the nice things about being a blogger who is not really on Twitter is that I never have to experience predictable Twitter arguments. I can just come along later and read any useful blog posts they happen to spawn. For instance, apparently there was some sort of huge Twitter kerfuffle recently about how many hours/week you have to work to be successful as a scientist. Or something? Anyway, here’s my advice: whatever it was, don’t read it. No, not even if somebody storified it. Just read Terry McGlynn’s and Mike Kaspari’s blog posts about the topic, and Meg’s post from a couple of years ago. (Very minor aside: I would quibble with Mike’s suggestion that you also read E. O. Wilson’s Advice To A Young Scientist. It includes some very, very bad advice. Wilson illustrates that it’s not just people on Twitter who overgeneralize from their own experiences.)
Dynamic Ecology 
“As with all the best trolls, Stephen has more of a point than you would think.”
This is quite possibly the most fun thing anyone has ever said about me 🙂
Clearly you need more fun in your life. 🙂
“But not enough authors agreed–there wasn’t just enough uptake for it to break even.”
From my personal experience, I won’t blame authors alone. I used Axios once and I am quite happy on the Axios part, but then the targeted journal axios and myself agreed upon sending the paper ignored axios reviewers (which I had already answered and were happy) and start the process again from scratch (new reviewers, etc…). At the end for me was double effort as I ended up answering two sets of reviewers anyway. Clearly that journal didn’t get the point. This is the main reason I didn’t use Axios a second time.
Yes, this was an issue for me too the last time I used Axios myself. Selective journals don’t always treat the Axios reviews as on a par with reviews they obtain themselves. This wasn’t a problem for all users of Axios–a substantial fraction of papers that went through Axios Review got accepted without further review by the journal. But it was definitely an issue for more than a few Axios users.
I think that’s a shame. If I were an editor at a journal, and I got a ms that had been referred by Axios, I’d just make a decision (which of course might be “revision”) based on the Axios reviews, unless I had some very specific reason to want an additional review from a reviewer of my own choosing. But apparently that attitude puts me in the minority. Perhaps if Axios had been able to hang on longer attitudes at journal editorial boards would have changed if Axios had managed to stick around longer and been more widely used.
I had a similar bad experience with Axios (not Axios’ fault, and it was the second time I used the service) as I wrote on Twitter. Reviews from Axios were not considered by the journal (actually, I was explicitly told to leave them out, change the formatting etc.), it underwent full review again for a total of 7 months and 6 reviewers and at the end it was rejected (that would be another funny story, and I encourage people with long-term anger problems to change their name when making comments in Adobe. One week later I was asked to review one of his/her papers, but I took the Everest road and declined…).
I was an early adopter, I like the idea very much, the service was smooth and super-professional, but there was no point in going through Axios if what I was receiving were basically “friendly” reviews and journal recommendation among a group of 3 journals I suggested, even at no cost.
On the other hand, there is little incentive for journals to outsource reviews, reviews are provided for free and the process supervised for free by scientists. And we all know how journals like to present themselves as selective and fair and process-based etc. I also think that an explicit “reviews were provided by Axios” would have bothered many people, as in “oh, so to be published there I need to pay for reviews”, which of course is wrong, but I saw enough skeptical looks that made me think that would have been the general reaction in the case Axios had gained some traction.
In synthesis, a more efficient system was not accepted by journals (the bottleneck), since they have little interest in marginal improvements in efficiency. Good case study and there are many parallels with failed attempts (or nobody making an attempt, except Econ) to make the recruitment process in academia more efficient (as in not sending 60 applications, 180 recommendation letters, where journals, in this case, are committees). In other worlds, think sports (agents, it does not happen that professionals send their CV/tapes to teams), private companies (mix of recruiting and direct application), more efficient systems have been welcomed.
“On the other hand, there is little incentive for journals to outsource reviews, reviews are provided for free and the process supervised for free by scientists. ”
Well, there’s some incentive. Journals have an increasingly hard time getting high-quality reviews in a timely fashion. They also waste a lot of time desk-rejecting lots of mss for lack of fit. Which gives them an incentive to be receptive to pre-reviewed papers that an independent, knowledgeable, experienced expert sees as a good fit. But it clearly wasn’t a strong enough incentive for enough journals.
My conclusion is that, as much as editors complain about how hard it is to get reviews these days, they’re not *that* concerned about it. They have ways to deal with it (increasing rates of rejection without review, sending more invitations to review, etc.). And those ways are currently working well enough from their perspective.
My other conclusion from the fate of Axios is that other mechanisms of review sharing, like peer review cascades, aren’t likely to be a panacea for the increasing burdens on the peer review system. If editors don’t treat reviews that were arranged by someone else on a par with reviews they arranged themselves, then there isn’t much point to review sharing. I’d be curious to hear from editors of jouranls that are part of formal review cascades (as Ecosphere and Ecology & Evolution are) how often they ask for additional reviews they arrange themselves. Any answer substantially higher than “never” means that review cascades aren’t doing all that much to reduce the burden on the peer review system.
Which suggests one potential solution: having two journals, one selective and one not, run by the same editorial board. Every submission is considered simultaneously for both journals and only gets reviewed once. If the editors think it’s sufficiently novel/interesting/important/etc., it’s accepted by and published in the selective journal. if the editors think it’s non-novel/boring/unimportant/etc. but technically sound, it’s accepted by and published in the selective journal.
I agree and I add something. If journals had to pay $100 for each review and Axios provided them for free (to journals), there would have been a strong incentive. With no money involved, there is almost no incentive.
As for time, we all know that journals uses various tricks to respect self-imposed deadlines and at the end if someone has to wait 3 months instead of 2 for reviews, it does not make a big difference.
In addition, if I remember well, my two submissions to Axios were desk-rejected by the first journal. A posteriori, success for Axios was very unlikely for how the journal system is set up, but Tim had an excellent idea.
Maybe it is just me, but I do not like selective, non-selective journals (apart from the blurry lines dividing the two), especially the way they are presented. Every time I receive an email with “was not considered suitable for this journal, but we invite you to pay $$ to publish in this other journals”, I have this bad vibe of “not good for free, but if you pay…”. I do not know it others feel the same way. It would be different if both were free for the authors (leaving aside for now the can of worms of open access, paywalls, Elsevier etc), but as they put, it never felt like an attractive option for me. As far as I know, there are only free (selective), pay (non-selective) options.
“As far as I know, there are only free (selective), pay (non-selective) options.”
Well, many selective journals give authors the option of paying a fee to make their paper open access, but IIRC (and I may not) those fees often are larger than those charged by purely open access journals.
Hi Jeremy,
Thanks for the link.
I would re-quibble that any short, readable book of advice by a groundbreaking scientist is worth the few hours on a Saturday afternoon *precisely* because it has generated so much discussion and disagreement. The best advice books are like libraries of modular code that you can plug into your own program. In all cases, your mileage may vary, but it only takes one really good idea to change your life. And the book has lots of them.
MK
Good point. Indeed, better than mine.
Stephen Heard will appreciate this tweetstorm about Ken Arrow. It takes a squirrel to appreciate another squirrel:
Fox, hedgehog, squirrel… the mammalian metaphors are multiplying 🙂
I am indeed jealous of Arrow’s career. He was clearly a much, much better and smarter squirrel than I will ever be.
Just in time for March Mammal Madness!
I currently have a paper in review with Axios. Do you know if they’re going to wrap up handling those manuscripts they have currently in review? Or have they already shut their doors and I should send my ms out to a journal?
Email Tim Vines at Axios and ask. I think they’re going to continue handling mss already in the system. Not 100% sure though.
It’s so sad to see Axios now is closed.
As a former Managing Editor of scientific journals, I do support platforms like Axios, which is one of the pioneers emerged in response to the urgent need of improvement in the conventional publishing process.
According to my knowledge, platforms like Axios can help all the parties involved in the publishing process if journals accept Axios reviews.
In current model, reviewers mostly are selected by journals using keyword matching. A voluntary peer review process would take one week to one month, sometimes, even months, because: editors need time to invite enough reviewers who agree to review the manuscript; after that there are risks that some reviewers could not return on time, some would quit the review, etc. To avoid such situation, editors either have to try new reviewers and start a new round still with risk, or even at the beginning, invite more reviewers than need. As a result, when collect enough review reports, they either cancel those ongoing reviewers or waste time on waiting.
If the papers finally are rejected, authors have to undergo such a round again in another journal, and the former reviews are wasted. To save their own time, some authors would submit their manuscripts simultaneously to multiple journals (though it is defined to be unethical, but it exists). When one journal accepted the paper, they withdraw it from other journals. It is a waste of publishers’ resource.
There are so many to be improved and solved, which was hopefully doing by platforms like Axios. If journals accept its service, many time and resources can be saved:
1. Authors can have their papers reviewed only once and in a timely fashion, then match a journal that is willing to accept their papers;
2. Reviewers’ effort on one paper will not repeat, not waste but be paid;
3. Publishers can receive a submission that have been reviewed and revised, only need to make a decision as to accept or reject it, or further revise it according to some of the review comments. This will significantly save their own resource and accelerate publication as they only need to focus on examine the review reports, authors’ response and make a decision. In my personal opinion, as Axios provides reviewer’s identity to the journals on request, there is no more risk on review quality than their own reviews.
Independent service is usually considered less bias, more transparency in many industries. It should be a trend in publishing as well. What a pity that many pioneers usually became victims first before being recognized.
It always needs time, usually long, for new trends to be broadly accepted, just like Open Access journals do. Let us wait for the journals to awake.
I myself give my respect to all the scientists who have tried the new platforms.