Meg was swamped this week, so it was Jeremy’s job to read ALL the things and pick the best ones. Also this week: ASN award nominations open, data on what it takes to get hired as an evolutionary biologist, the root “problem” with peer review is not actually a problem, new R package vs. sloppy results sections, online supplements vs. reviewers, the government of Oklahoma vs. ESA memberships, plagiarists are losing the technological arms race, and MOAR. Lots of good stuff this week!
From Jeremy:
Applications and nominations for the 2016 ASN Jasper-Loftus Hills Young Investigator Awards (and other ASN awards) are due Jan. 1. Details here. You have to be in your final year of grad school or less than 3 years post-PhD on Jan. 1. The award comes with $500 cash, travel expenses for the ASN annual meeting in Austin, and a presentation slot in the Young Investigators symposium at the meeting. Sweet! Plus, it’s a nice feather in your cap that will look great on your cv. Four awards are available. You don’t have to be an ASN member to apply. Note that I said you should “apply” for the award. Don’t think of applying as some unseemly form of self-promotion. Think of it as you would applying for a grant. And don’t assume that you won’t be competitive and let that stop you from applying! I’m on the J-LH Young Investigator awards committee this year and I’m really looking forward to it. But we can only choose from among the applicants we get. I can tell you from past experience on other awards committees (including the ESA’s Buell and Braun awards) that it is unfortunately very common for people to not apply for awards for which they would be very competitive. Looking forward to seeing your applications!
Stephen Heard on lectures as active learning. Another reminder that lecturing is not inherently bad teaching. It’s just one teaching tool among others, which can be done well or done badly. So enough with calling it unethical and arguing for it to be banned already.
A bit late to this, but better late than never: postdoc Jeremy Yoder on what’s a postdoc, again? He has a very reasonable, healthy attitude towards his postdoc, and to the academic job market.
Via Jeremy’s piece, some data on how many papers it’s taken to get hired as an evolutionary biologist at the NCSR in France from 2005-2014. The NCSR has a stable, formulaic hiring process, making cross-year comparisons straightforward. We’re only talking about one institution and 55 hires, so the sample is small and possibly non-representative of other places. But FWIW, from 2005-2014 the paper productivity of new hires almost doubled, from a mean of 12.5 papers to a mean of 22. That’s largely but not entirely because the post-PhD experience level of new hires jumped dramatically during that time, from 3.25 to 8 years on average. One small question I have is how the proportion of first-authored and many-authored papers on new hires’ CVs has changed over that period. (Aside: when it comes to post-PhD experience, the NCSR data are in some ways not representative of what’s happening elsewhere, at least in ecology. Previously, we’ve linked to survey data from ASLO showing that the typical post-PhD experience level of new faculty hires in ecology has dropped over time, from 4-7 years in the 1980s to 3 years from the 1990s til today. But note that there is lots of variation around what’s “typical”.)
Thinkpiece on the decline of Twitter. Sometimes unclear and way too jargony, but still worth a read. tl;dr: Twitter’s problem is that it’s written communication with some–crucially, only some–of the features of oral communication. I was mostly interested in it because of its implications for blogging. As bloggers, we occasionally run into a bit of the same problem as Twitter has, especially when a post goes viral and gets read by lots of strangers who’ve never read us before and never will again. But only a bit, because we’re writing thoughtfully and at some length. So Dynamic Ecology is mostly written communication.
Very interesting preprint simulating various pre-publication peer review systems. The take-home message is one I and others have been banging on about: the root “problem” with peer review is not a problem at all, but just a fact of life: we all disagree with one another. So peer review cannot be “fixed”, including by any form of post-publication review. Well, unless you think it’s both possible and desirable to force all scientists to agree with one another about everything…A couple of other minor comments on the simulations. First, the assumption that reviewer decisions on whether or not to review mss are independent of ms quality is false, though I’m not sure if its falsehood much affects the author’s results. I and most reviewers I know are more likely to accept invitations to review if we think the ms will be good, based on the abstract, the journal, and (if provided) the authors’ names. Not that our guesses are infallible–far from it!–but I bet they’re better than coin flips. Second, I was pleased to see that the editorial decision-making method that worked best in the simulations is the one I think all good editors use: the editor uses the reviews to inform his or her own judgment, rather than just counting votes.
Over half of the psychology papers that reported at least one null hypothesis test from 1985-2013 reported at least one p-value “grossly inconsistent” with the reported test statistic and degrees of freedom. And in a depressing non-surprise, such gross inconsistencies were more likely for p-values reported as significant. At least the prevalence of such sloppiness isn’t increasing over time. The work was done using a new R package called statcheck, about which I am now very curious. Note that I haven’t read the paper and am relying on the abstract, which may be unwise, so you should definitely click through and read it for yourself if you have more than a passing interest. (ht Retraction Watch)
Writing in Nautilus, James O’Dwyer with a popular account of why ecologists ought to seek universal laws. Cites an old blog post of mine.
Matt Ridley with a provocative editorial arguing that basic scientific research doesn’t lead to useful technological innovation, not even via serendipity. Puzzling and difficult to follow in many places, at least for me. It’s a summary of a forthcoming book of his, so perhaps the longer treatment is clearer and more coherent. (ht Retraction Watch)
In immunology, only 2.4% of citations are negative, and only 7.1% of papers attract any negative citations, even under a broad definition of “negative”. And negative citations have little or no association with how often papers are subsequently cited. None of this is very surprising, I don’t think, but it’s nice to have it quantified. I leave it to you to decide what, if any, implications these data have for arguments that the scientific literature needs more criticism. (ht Retraction Watch)
Wait, has the governor of Oklahoma really just ordered every employee of public universities in the state to give the state written notification every time they give an invited seminar out of state, every time they renew a scholarly society membership, and every time they buy a piece of equipment worth >$10,000?! Any Oklahoma readers who can enlighten us in the comments as to what’s going on?
Attention stupid/lazy/desperate/dishonest students and academics: you know those “article spinning” sites that promise to rephrase plagiarized work so that it won’t be caught by text-matching software? They suck, in hilarious ways that will instantly give you away, and risk making you the butt of jokes on the internet. Sorry, you’ll have to go back to disguising your plagiarism by hand. Which will either fail, or be so much work that you might as well just use your own words. Unfortunately, the free technical solution to your stupidity/laziness/desperation/dishonesty has yet to be invented.
BAHFest is coming to London in 2016, with a two-day event on “evolution” and “big science”. Click the link to propose a talk on a logical, well-argued hypothesis that is also totally, hilariously wrong. 🙂
When I complained recently about how online supplements are getting out of hand, I didn’t know the half of it:
Protip: do not try to publish your thesis as a single 10-page paper with 300 pages of supplementary material. 🙂 So, what’s your best guess as to what would possess anyone to do this? Mine is “student took supervisor’s advice not to ‘salami slice’ waaaay too seriously.” 🙂 (Ok, I admit this an extreme, atypical example, but I do think it illustrates a growing problem.)
Is it wasteful to discard the leftover pumpkin seeds from your jack o’ lantern rather than toasting them? A trickier question than you might think…
And finally, this week in People With Too Much Time On Their Hands: someone made an emoji-based cellular automata model of forest fires.
Dynamic Ecology 
Another fun fact about those French numbers is that there is now a law in France (across all job sectors) that caps the time that one can be on temporary contracts at 6 years. There’s more details to it, but what it basically means is that you can’t legally be a postdoc in France for > 6 years. And 6 < 8. The law is relatively recent and it's not clear to me how this will play out in the long-term for the research community here.
I really liked Stephen’s post too (as well as The Little Professor post he linked to).
Stephen’s identification of study skills and practices as a central issue is a good one. And I agree with Stephen that I am NOT in favor of letting up the expectations here. And a good lecture is definitely NOT passive learning.
I’m also reminded of research in other fields. In psychotherapy (a career my wife has spent time in), there was a big study on which modality of therapy worked the best (Freudian, cognitive, behavioral, narrative, etc). The answer that came back: the effect size of the individual therapist swamped which mode worked best. And in primary education, a professor in New Zealand (Hattie) has essentially pulled off a meta-analysis of meta-analyses of all research education examining 138 different teaching practices. He then broadly divides these practices into 4 categories: a) effect size greater than effect size of teacher, b) effect size of same magnitude of effect size of teacher,c) effect size positive but smaller than effect size of teacher (also =null model of don’t do any teaching for a year and come back when they’re a year older), and d) effect size negative. Only a small handful of practices are in category (a) and many of them are super obvious and fundamental like specialized teaching for students with learning disabilities or accelerated classrooms for gifted students or keep the classroom under control or staff development (=train the teachers).
I strongly suspect that a good lecture is still a lot better than a bad active learning class. And for what its worth the Freeman et al 2014 analysis of Cohen d=0.47 puts active learning squarely in category (b) from above (assuming effect sizes in primary schools carry over to university education) so it clearly is worth taking note of. But it does not swamp teacher vs teacher variation. So its consistent with the idea that a good lecture is better than a bad active learning class. And its not like active learning is going to cause people 20 years from now to go “woah – college students are so much better trained now” (of course that is not what its proponents claim either)
It would be really nice if there was a silver bullet to make all classes scintillating with maximum student achievement. Which is what the “it is unethical …” quote suggests. But if we really want to be serious about what research shows, then what it really shows is the best way to get that perfect classroom would be hire for teaching ability, invest a lot of time in training professors to teach on an ongoing basis, and reduce class sizes. And that has a lot of implications nobody wants to talk about. Especially at a research university.
That said, its not an excuse for faculty to laze around and throw up the same powerpoints they used 10 years ago and ignore all possibility of new techniques that are beneficial.
I also hope that future research really takes the Hawthorne effect seriously (brought up in the comments on the The Little Professor post. In industrial engineering it is fairly notorious – any change produces a positive effect for a while. Make lights brighter and productivity goes up. 2 years later make lights dimmer and productivity goes up. For myself this is my main pedagogical direction – keep it new.
Hawthorne effect–never heard of that! Will need to look into it.
Free Wikipedia research at your service (actually a pretty good discussion): https://en.wikipedia.org/wiki/Hawthorne_effect
Thanks, Brian! Glad you liked that post. Interesting perspective on effect sizes. I’d be interested in a citation for the Hattie study (granted, kindergarten not equal to university, or at least only on days I’m feeling really cynical…) Thanks for your perspective here – if you’re willing to cut-and-paste it into the comments section for the piece, that would be great. –Steve
Hattie’s work is in a book called “Visible Learning”.
Good points Brian.
I also just wanted to say that your comment connects today’s linkfest post to my post on “what’s a small effect?” from earlier in the week. Dynamic Ecology: where everything is connected. 🙂
“I also hope that future research really takes the Hawthorne effect seriously (brought up in the comments on the The Little Professor post. In industrial engineering it is fairly notorious – any change produces a positive effect for a while. Make lights brighter and productivity goes up. 2 years later make lights dimmer and productivity goes up. For myself this is my main pedagogical direction – keep it new.”
– Does it mean that you try to change things through a course? Active learning 2 weeks, lectures 2 weeks and so on?
From the debate/science/personal experience I have the strong resulting impression that students’ learning depend on 1) students, 2) teacher (personality, attitude, energy, choice of topics), 3) teaching modality.
As a side note, I often point out to my colleagues that I remember next to nothing of what I studied in college and I guess it is the same for them. They usually strongly disagree, but upon further reflection, they tend to agree. It is something I keep in mind when teaching.
“Does it mean that you try to change things through a course? Active learning 2 weeks, lectures 2 weeks and so on?”
I wouldn’t think so. Haven’t yet followed up Brian’s links on the Hawthorne effect, but I’m guessing it has something to do with people getting into a rut when things have been a certain way for a long time. If you only do active learning for two weeks, then switch to another teaching technique, there’s no time for the students to get used to it before you switch. I don’t think the Hawthorne technique is an argument for constant chaos and confusion as the optimal state of affairs. 🙂
” I often point out to my colleagues that I remember next to nothing of what I studied in college and I guess it is the same for them. ”
I do retain some information I learned in college. But mostly information that I’ve made use of later. It’s a bit like a second language–use it or lose it. The most important thing I learned, and retained, isn’t any particular bit of information though. It’s how to think my way through problems. How to use what I know to think my way through stuff I don’t yet understand.
I asked that question because variety works if the same people are carrying out the same activity for a long time (i.e multiple years). It is frequently observed for instance in sports, in which it is common to have (and/or expect) a better performance (if not result) immediately after the coach is fired. Then, regression to the mean is very common as quality of players/students is much less plastic.
In terms of retaining information and the best method for teaching, context is of the utmost importance. The vast majority of students (myself included, and I was a very good student) retain next to nothing and it is good to be aware of it. I do these tests or ask these questions very frequently. For instance, I remember next to nothing of 160 hrs of lecture of organic chemistry, geology, isotope ecology. This should help us put all the pedagogy discourse in perspective, especially when going beyond alphabetization.
First, we are spending an enormous amount of time assimilating information we have no chance to retain. Yes, we may say we “learn to learn”, but is true beyond the single case (e.g. woman in academia/intellectual work)? Should we spend that time more “productively”?
Second, as far as I know, the effectiveness of the pedagogical choice is measured during/immediately after teaching; but, what happens 1, 2, 5 years down the road? I guess results would be interesting, and I expect a strong regression to the mean (of student).
My hypothesis is that, apart from the left tail (terrible teacher, disorganized approach etc.), after a few months/years we will observe a very small effect size for any of the common pedagogical approaches.
For me it means changing across years. I figure the 2nd or 3rd year after I teach something is probably my best year (the 1st having too many mistakes and last minute items to be my best) and then I start just locking in and not being creative and adaptive. So It means really big changeups every 4 or 5 years.
But I imagine your change within a single course would also probably be a good idea.
I think your observation that we don’t really recall that much from classes is true. I think the question is what to do with that. The active movement approach is predicated on moving up Bloom’s taxonomy and that will be remembered for a long time better than facts and figures. I think that is true. But I think an equally true perspective is that what a student really gets out of say intro biology is learning how to learn. I think that is a valuable learning outcome too. Just because learning how to really absorb a set of facts is not thrilling and they will be forgotten, doesn’t mean universities should stop teaching how to really absorb a set of facts. And that skill of course lasts a long time as well and is probably ultimately more useful to most students late in life than moving up Bloom’s taxonomy from memorizing, say phylo to a higher taxonomy understanding of the haplo-diploid life cycle. Of course once we’re explicit that a goal is to “learn how to learn” it has implications for our teaching including probably spending more time talking about study skills (Stephen talked about that in his post and it certainly rang true for me – a lightbulb went off when I was teaching BIO 100 when I realized it was as much about teaching study skills as teaching biology). It probably also means lots of rapid feedback (e.g. the clickers or mini pre-quizzes that Meg has talked about in her flipped bio class) rather than just a midterm and a final.
Brian, so variety works directly for you and indirectly for students. I agree with you.
I think that asking tough question is necessary. If most of the info is not to be retained, why not “learning to learn” while learning information/subject that is more likely to be retained? It is very inefficient “learning to learn” while learning something that is not to be retained, or very unlikely to be. We are talking about thousands and thousands of hours.
For instance, and this is a pet-peeve of mine, there is no “learning to learn” course or memorization class (we do not derive things from first principles every time, and memorizing/accessing information is very important, but it is never taught, at least as far as I know).
The usual state of affairs, and I am talking in general terms here, is throwing a bunch of facts or info or ways of solving stuff to students and then say to ourselves “well, maybe they will retain nothing of this biology class, but at least they learned how to learn”. First, while the first is very often true (very little is retained), is the second true? Is there any test or study on the “learning to learn”?
I am thinking a lot about these issues. Another example is foreign language classes. They do not work, the vast majority of students learn absolutely nothing, but we are still using the same teaching methods, thinking that the 2 or 3 hours a week for 5 years will magically start to work after 50 years of major failures.
I might be too cynical, but I am quite disappointed by the fact that I spent hours and hours learning to recognize rocks and now, after ten years, I would not be able to tell the difference between quartz and clay.
All good points and we’re definitely discussing the right questions.
I agree that there should be pedagogy of how to learn and we don’t do it well.
I’m not sure I agree with you that it represents bad teaching that I don’t remember things I learned 10 years ago. I’m more in Jeremy’s camp of most things are use it or lose it. Just for example calculating an integral. I knew it well in high school and college. I retained it about 5 years after college then just in time I started graduate school and used it a lot. But since graduate school I have not used it (now >10 years) and I am now not very good at anything more than the basics. Given that I retained it for almost 20 years I can hardly blame the teachers.
Gold prize for lectures!
Education researchers don’t seem to get that students actually have agency – they can learn new ways to learn and they can adjust their level of effort.
Also, the lecture that anti-lecture people refer to is a straw man. In real live, a lecture isn’t just prof speaking. It includes pics, diagrams, questions, class participation, demonstrations and other so called “modalities”
IMO, a lecture driven course is the most efficient form of teaching an learning for the best students.
BTW I retain a GREAT DEAL of what I learned in school and use it regularly and effectively in every day life