Note from Jeremy: This is a guest post from Peter Adler. Thanks very much to Peter for taking the time to write another thought-provoking post for us. Enjoy!
********************************
Here is an observation that may both horrify and empower graduate students: it doesn’t get easier. Just over a decade ago when I finished my PhD, I figured that I was still on the steep part of the learning curve, and that mastering a few more tricks of the trade would have me churning out Ecology papers in my sleep. I’ve learned those tricks, and they have made me faster and helped me to avoid some dead-ends, but getting a first-authored paper written and accepted in a good journal, or putting together a winning proposal, still requires tremendous time, effort, and determination.
Perhaps I’m an outlier? It’s possible that I peaked early or that my development stalled, but I don’t think that I’m alone. Over the last few months I’ve been asking other researchers in my cohort about their experience and they generally agree with me. Sure, with a little seniority comes more collaborations and more publications, which look nice on a CV, but doing really good science is just as hard for us now as it was when we were graduate students. I have three explanations for why it doesn’t get any easier.
1. Making anything really good is hard
I remember hearing Ira Glass on This American Life say that raising anything above mediocrity requires “a tremendous force of will.” This applies to any field, art, literature, radio, sports, or science. Mediocrity is a powerful attractor: no one wants to do bad work, and often elevating bad to decent isn’t so hard. But the motivation to turn decent into excellent is more elusive and the challenge much more difficult. Doing something that no one else has done before requires creativity, persistence, and often some luck. There are no short cuts.
2. Fragmented time
If wisdom accumulates with experience, time dissipates. Some of the demands on my time are important and well-justified, like teaching and mentoring. Other demands aren’t so welcome (administrative minutia, committee meetings, annual reports). The net effect is a decrease in time available for research, and fragmentation of the time that remains. I think that I have gotten more efficient, but those gains are offset by the loss of those big chunks of time essential for creative thinking. And for reading, which brings me to the most important explanation.
3. All that matters is good ideas
I’ve written enough papers now to be amazed at how a few seem to write themselves while many feel like beating my head against a wall. The difference is simple: the rare papers that wrote themselves were about good ideas, the ones that were a struggle were about not-so-good ideas. The good ideas are novel and simple, easy to explain and to test and, ultimately, (relatively) easy to publish. The not-so-good ideas lead to constant re-framing of the original research question, re-analysis of the data, and re-submission and revision of the manuscript. By the time one of these papers makes it through the peer review mauling and into print it has chunks of rotting flesh falling off. So where do the good ideas come from and how can we make more of them? If only we really knew. But in my experience, and from my casual tracking of research on creativity, reading is the key. It’s as simple as that. I certainly don’t read nearly as much as I used to, despite the folder of reprints I carry with me to waiting rooms, airports, and family vacations. I still enjoy reading ecology, I just don’t have the time or the drive to do it in the same volume. So the steady trickle of good ideas slows to a drip. Hopefully a steady drip, into a super-efficient production system.
I’m interested to hear your point of view. If you have had your PhD for a decade or more, has doing good research gotten easier? If you are a student or recently finished your PhD, does this message strike you as empowering or depressing? Have I missed important explanations for why this game doesn’t get easier?
Dynamic Ecology 
Too right.
I still feel like it takes me forever to write a lead-authored paper. Often with the day broken up by teaching, admin, etc., it is easier to use those gaps in the schedule to deal with turning around student papers (they’re the ones who need a job when they finish, after all), and wait for rare large blocks of time for my own stuff.
Another likely driver of among-researcher variance in slowness is in how much you worry about whether your results are robust. Do you re-analyze your data numerous different roughly equally valid ways to make sure the result doesn’t suddenly go away, even though you know you won’t have space to put all of it in the paper? When you have a paper rejected, do you address the reviewer criticisms that you think are valid before resubmitting elsewhere? So in this respect, I’m not sure I have the same experience as Peter that the time it takes to write a paper is connected mainly to whether it’s a good or not-so-good idea. That’s definitely a factor, but a bigger contributor for me is how complicated and time consuming it is to convince myself that my results are not an artefact of some arbitrary decision I made about how to approach the problem.
Sean,
I also find myself doing the same thing (and driving members of my lab to do so as well).
Sean,
Good point about the reanalyses, though I think this axis of variation runs independent of good-bad ideas. I definitely do more modeling than I did as a graduate student, and because there are so many decisions and choices to make, I often go round and round many times before believing my results. So this may contribute to my sense that “it” hasn’t gotten easier.
A digression: I used to think modeling papers were faster/easier than field-based papers, but now I see field papers as faster. This reversal may simply reflect the fact that as a grad student I collected all my own data, but now I typically have technicians in the field. As a grad student, 3 years of field work is daunting, whereas you can start generating results on that modeling paper right away. As a PI, once the results of a field experiment land on my desk it’s pretty easy to run a few ANOVAs and write a paper, whereas the modeling paper will involve hundreds of hours of my time. The model that this paper (http://onlinelibrary.wiley.com/doi/10.1111/j.1461-0248.2010.01496.x/abstract) is based on took me years and years (and ultimately Steve Ellner’s help) to build.
Peter,
I have had a pretty similar experience (~10 years out of PhD) to yours and your post struck a chord with me. The one aspect which I do feel has become somewhat easier is fitting everything into “the big picture” when writing manuscripts. While I can not be certain as to why, I think that has as much to do with reading and editing manuscripts (both from folks in my lab, collaborators and as a reviewer/editor).
I agree!
I’m in the (late?) middle of a PhD. I had this same illusion — that things would soon get easier. some of our postdocs are so productive that I can’t help but imagine that they have learned some “tricks”. So in a way this was disheartening.
But also empowering? I’d like to think that the ‘quality’ of science one produces tends to increase, even if perceived effort increases — suggesting that the return-on-effort is increasing? At the very least, it is encouraging to know that this is uphill for the professionals, too.
One thing surprised me: your statement that “reading is the key” to good ideas. I’ve often felt that interesting & novel ideas come from spending hundreds of hours observing your system, not reading other scientists’ papers. Not that literature is unimportant, of course — but as an early-career scientist I find it less inspirational and more distracting.
How do you know what to observe? Or where?
Luck!
Just kidding. Of course it’s essential to inform your curiosity with the literature! But I think lots of really good ideas come from intimate familiarity with a study system.
You don’t; that is the point and the magic of discovery.
I’d like to pick up on something you mentioned in your post, though it isn’t really the post’s focus. In your “worthwhile” activity, you mention mentoring. Yet in your list of “unwelcome” activities, you list committee meetings. Ideally, committee meetings should be a key part of the mentoring process. However, in my experience, the sentiment you express seems to characterize how most students and professors feel about committee meetings. It’s too bad that committee meetings have come to this point: an administrative obstacle rather than being integral to the grad student experience. Anyway, I know this isn’t the main point of your post but I’m wondering how many out there feel committee meetings are superfluous or, conversely, whether anyone out there has been in an environment where committee meetings are highly valued.
Graduate committees are fun, it’s the “University IT Committee” or Faculty Senate or even Dept. meetings that are the real snoozers. Apparently, the University of California system even has a Committee on Committees, which sounds like a Monty Python bit.
Hi Peter,
Great article, thanks for sharing!
I’m also in the midst of my PhD and am hearing the same line everywhere: “it doesn’t get any easier!” Discouraging? Some days, sure. Other days, the pros outweigh the cons. I suppose I shoot for the latter kind of days, or else I wouldn’t still be here!
As my advisor is fond of quoting (his advisor): “If it was easy, someone would have done it already.” I’m always struck when revisiting the classic works in marine community ecology just how straightforward it was. Go down the tidepool, pick out all the snails, see what happens. Now experimental design and statistical analyses are much more complex, and not just because someone else has already done the foundational work. The questions are bigger and more intricate–and more relevant–but so are the insights. Its just with so much already out there, and trying to wrap my brain around complex systems, its difficult to converge on those “good ideas!”
Cheers, Jon
UC faculty performance is assessed every 3 years at UCD, perhaps at different short intervals at other UCs. The Committees on Committees at UC campuses is exceedingly important, especially to young and mid career faculty. CoC chooses who is to be on such committees as CAPs, Committees on Academic Personnel (plurals for different UC campuses, among which names vary). If no ecologists or even whole organism biologists on key committee at your campus, then how is the work of your colleagues (you) assessed? Comment seen from reviewers on such committees: “Why no NIH support?” Senior faculty have to step up and agree to serve on these committees. At UCD, CAP is a 1/2 time commitment, which means no teaching or other admin work. Members see hundreds of folders from the gamut of disciplines: English, History, Womens’ Studies….
Don, I didn’t mean to suggest that we shouldn’t serve on those committees, or take them seriously. I am a good Boy Scout and do my duty. And I appreciate being reminded of why the service is important. But I don’t enjoy it like I do science, and I doubt I ever will.
Peter – good post. I think of this as kind of a career conservation of mass analog: it does get a bit easier going forward to do the same tasks….but you have less time to do them! So in the end, not really any easier. Am now 19 years post-PhD, and with every career step, I seem to look back at the prior one and wonder what the hell I did with all my time back then? But a key and what I think should be heartening thing for the early career folks: as you move along, yes your time for any one thing gets compressed and the need to do annoying work that doesn’t contribute to your science increases, but the positive is that you are increasingly in a position to be a part of more good science. Sometimes as the lead, more often as a team player, and rarely with the level of focus possible in grad school or postdoc time, but either way the portfolio expands beyond what was possible in grad school and postdoc days. And with care, you get to define the scope of that portfolio. If you tend to the generalist side of things, you can dabble in all kinds of areas. If you like drilling deep into a topic, you can do so more fully as a collective. So while it’s true that you don’t get as much time to just read and think and collect data on a given topic, you do get to play in a bunch of different games. I find that quite rewarding.
Alan put it really well. Doing individual tasks, and knowing which ones matter, might get easier, but you have less time to do those tasks. I think the really effective people beyond grad school are the ones who can use time well and manage the time of others effectively.
As you get more experienced, then there’s the trap of doing what you know will work but not be thrilling, and not taking a chance on something that might be exceptional. In grad school, it’s totally normal to spend days or longer collecting data or working up a preliminary idea to see where it evolves. As a mid-career faculty member, it’s a lot harder to take that risk, and a much huger investment of your time because available time is more limited.
“In grad school, it’s totally normal to spend days or longer collecting data or working up a preliminary idea to see where it evolves. As a mid-career faculty member, it’s a lot harder to take that risk, and a much huger investment of your time because available time is more limited.”
Which is paradoxical in a way because the whole point of tenure is that it’s supposed to give you the security to take intellectual risks…
As a non-tenure-track faculty, I have found things to get easier. The key thing for me is that I now have the freedom to work on the projects I want to work on, and the ‘juice’ to get collaborations going when I need them. The Ph.D. workflow is something like: Come up with a research question. It doesn’t fit in with what’s going on with your lab, so change it until it does. (The new question is much more boring). Do the work yourself. Come up with an elaborate excuse for not doing experiments that can’t analytically be done in your lab or one of your advisor’s buddies’ labs. Write it up, without the benefit of knowing what’s really new and exciting in the field (because that conversation happens at conferences, in clusters between sessions that you can’t take part in).
Now, I’m much more plugged in to what the community is excited about. If I think an experiment is worth doing but requires a new instrument, I can write a grant for it (and it might actually get funded!) Or I can reach out to colleagues I don’t know personally who could do the analysis, and they’re much more likely to listen to me. When it is time to write, I have a much easier time because I have a much better sense of what is interesting.
The excitement of being 100% soft money adds a new dimension to research, but overall I’d never go back to my Ph.D. days.
It DOES get easier, but one’s standards also rise. The fact that practice matters is the major cause of the productivity-quality correlation, in my opinion.
Coincidentally, a New Yorker blog covered part of this topic agree, and the writer largely agrees with you:
http://www.newyorker.com/online/blogs/sportingscene/2013/08/psychology-ten-thousand-hour-rule-complexity.html
Yeah, but the 10K hour rule is about the same activity. Often, doing something awesome requires doing activities at which you aren’t expert.
“The good ideas are novel and simple, easy to explain and to test and, ultimately, (relatively) easy to publish.”
A good idea does not necessarily translate into a good paper, though. I’ve had plenty of good ideas where the concept was good, the experiment was straightforward, but the results were ambiguous or hard to interpret. This is what T.H. Huxley called, “The great tragedy of Science — the slaying of a beautiful hypothesis by an ugly fact.”
Great post and important to get the idea out there. When I was at McGill I had a senior faculty mentor who regularly reminded me the non-science load only gets heavier. I could have gotten depressed, but I actually saw it as a rather important fact to get my head around. Specifically it meant thinking along the lines of “I’ll do that in a few years when I have more time” was bad thinking. It was actually rather freeing to just dive in and do what I thought was important and fun.
Although I agree with Peter and Alan’s general theme of constancy of scientific productivity over a career through a trade-off between time and efficiency, I think the postdoc might be a bit of an exception to the rule. It is a rare few year window where the skills are in place and the time pressures are not on. I know I never produced research before or after like I did as postdoc (and 1st year tenure track). Recognizing this, I think cranking during the postdoc is really important. Also, when 1st year faculty ask me for advice, I usually tell them to duck, not raise their hand and act like a postdoc until they are dragged out of it in year 2 or 3.
And another theme a few places on this page is time management. I teach all my students and postdocs that learning time management skills is essential to their career success, right up there with stats and experimental design. It is something you can read about, ask successful people about, and get better at.
I agree about the post-doc years, and about time management. As a grad student, I once complained to my advisor, Bill Lauenroth, about all those little slices of time that are hard to use effectively. His eyes almost popped out of his head, which was unusual for a guy who almost never reacts at all, and he waved a finger at me and said, “You will never have more time than you have right now!” He then gave me a simple piece of advice I still (try) to follow: If you have a 15 minute window, start a 15 minute task, not a 3 hr task, and if you have a 3 hr window, don’t waste it on a bunch of little tasks, tackle a big one.
Great advice… “If you have a 15 minute window, start a 15 minute task, not a 3 hr task, and if you have a 3 hr window, don’t waste it on a bunch of little tasks, tackle a big one.”
I already do this but never realized it consciously so now I will try to follow this advice more rigorously. To help do so, what I so is keep a really long task list on my desktop (computer desktop and real desktop) so I can pick short or long tasks depending on my time available. From “organize new PDFs” to “write X an email” to “analyze Y data set” to “think about design for new study on topic Z”
Great conversation-thank you Peter.
I got my PhD in May 2000.
I think whether your perception is that it gets harder must be in part dependent on your responsibilities as a student and your teaching load etc, now. As a student, especially an MS student, I TAed a ton, took a lot of classes, did a lot of crap for my advisors, etc and as a Prof at UNC my teaching load is relatively light; so all-in-all I doubt my “non-science” work time has changed much, especially now that Ive got teaching dialed in. Grad school friends that were on NSF fellowships and now work at teaching institutions have experienced something entirely different.
Also, my sense is that doing ecology has gotten A LOT HARDER. I think it is much tougher to publish in the top ecological journals (the expected level of novelty at times seems preposterous), in the US funding has gotten radically harder to obtain, the cost of doing science has increased 2-3X over the last decade or two, the need for reviewers has grown, etc. Plus, there are so many more distractions now (email! the internet! blogs! twitter! facebook!); even in the late 1990s, we PhD students lived in little academe-eco cocoons, with little connection to the broader world. (nobody was wasting hours on the web) Ahhh, we had it so good compared to students today:)
I like your point that doing ecology has, objectively, gotten harder. That seems clearly to be the case for funding, as the plummeting funding rates show. Do we have equally good data on publishing? Are journals rejecting a much higher fraction of submissions than they were a decade ago? Has this been true for every generation of ecologists?
Don Strong would know what the stats are on rejection; but for two reasons, that might not get at whether expected “quality” has gone up: 1) there could be more submissions, increasing rejection with constant “quality”, 2) the addition of so many new journals could negate this and thus, combined with author selection could lead to change in acceptance at Ecology, with an increase in “quality”. Suffice it say, you can no longer simply exclude an enemy and expect to publish the work in Ecology. That said, I don’t think it has become harder to publish somewhere-there is always another journal to go to nowadays; it is just bloody hard to publish with the big dogs. (And thank god for PLOS One)
My perception is that the intellectual content of articles published in our discipline is increasing due to a sort of Flynn Effect (Google it). We know more to begin with, our education and intellectual environment provide us with more and better tools and more information—huge volumes of information and rally cool ways to process it. We seize upon this to produce the articles that you read today. I think back to grad school and post doc times. It was those of us with passion to do it all of the time, over and over, that are still doing it today. I do recall some of those who took a path out of science did just fine and are quite pleased with their lives. Some are quite wealthy. One in particular, tends the grapes in his vineyard; his passion was not academics but radiology and business. Those who succeed today in ecology are driven by passion. I could list the students around here (but I won’t). Passion is almost insufficient to describe how driven they are to do science. Examples abound.
Some ecology journals publish their rejection rates, along with other data (like number of submissions) in an editorial in January. Ecology is one of these. I just looked up the data from Ecology over the past few years:
2012: 1355 submissions, 21% acceptance rate
2011: ~1500, 20%
2010: 1510, 15%
2009: 1477, 24%
2008: 1564, 25%
2007: 1452, 25%
2006: 1417, 27%
2005: ~1250, 22%
2004: editorial states that acceptance rate was between 25-30%, as it had been for the past few years.
I didn’t look further back, as I don’t know that the data are there (going much further back takes you back to the days before online ms handling systems).
I suppose there might be a weak downward trend in acceptance rates at Ecology over the last 8 years. But even if there is there’s almost as much year-to-year variation, presumably due to various idiosyncratic factors. Fluctuations in submission and rejection rates happen for all sorts of reasons, including reasons independent of the distribution of submission quality. EiCs may ask the handling editors to be more selective in order to clear a backlog. The journal may decide to publish more articles. Etc.
FWIW, I share the subjective impression that it’s getting harder to publish in leading journals. But I hesitate to put much stock in that subjective impression without looking at more data.
The EICs do not ask subject matter editors to be more selective or less so. The 2010 dip was a result of high reject following editorial review (REFR) due to the need to stem the backlog of unpublished mss. Ecosphere has cured the backlog problem.
@Don:
Sorry, didn’t mean to imply that the EiCs at Ecology ask subject matter editors to be more or less selective. I meant that remark to apply more generally–sometimes, journal EiCs do ask subject matter editors to vary their selectivity, though Ecology hasn’t done so. When I was at Oikos, the subject editors were asked by the EiC a couple of times to be more selective in order to clear a backlog.
To echo John Bruno:
4. The competition is getting more fierce. More people applying for stagnant (or shrinking) pots of money. Maybe tougher publishing competition (or maybe not).
My inference is that austerity stemming from right wing forces has cut the rate of investment in science and led to less money for research. Young scientists are in a real pickle owing to this horrid force in US politics.
I agree Margaret, especially competition for scarce jobs.
Pingback: INTECOL 2013 Wednesday Thursday Wrapup – #INT13 | Dynamic Ecology
Pingback: What we’re reading: Rams’ horns, beetles’ testes, the rules of CC-BY reuse, and “Gatcaatgaggtgga …” | The Molecular Ecologist
Pingback: Expiscor (26 August 2013) | Arthropod Ecology
I think this paper by John Platt from the 1960s gets to the core of “dong good science”. In addition to reading widely (which he also recommends), Platt emphasizes thinking- sitting down, writing (or drawing) out all the possible outcomes of the experiment and what you would conclude “if X happens.” I found the paper really insightful, and also very blunt.
Click to access science64_strong_inference.pdf
Interesting. Platt’s paper is famous of course, but mostly for his recommendations on how to rigorously narrow down the space of alternative hypotheses via experiments that distinguish them. I last read the paper years ago, and hadn’t recalled that he had as much to say about how to identify the alternative hypotheses, or on how to decide what questions to ask in the first place (that last one being the issue with which Peter’s post is most concerned, I think). Will have to go back and have another look.
Maybe I should be embarrassed, but I didn’t know of this paper. Loved it, thank you, it’s chock full of great quotes. Perhaps my favorite: “We praise the “lifetime of study,” but in dozens of cases, in every field, what was needed was not a lifetime, but rather a few short months or weeks of analytical inductive inference..” Your comment also made me realize I never really defined what I meant by “doing good science.” I lazily implied it equates to publishing in high impact journals or getting grants funded. But I am not proud of all my (relatively) high impact papers, and some of the work I am proudest of came out in middle tier journals. The Platt paper captures what I had in mind–it really boils down to “figuring stuff out.” Did you really learn something? Are you sure of it? Is the interpretation of the result crystal clear?
Pingback: Friday links: significant excess of (barely) significant results, sympathy for the dean, and more | Dynamic Ecology
Pingback: Communicating my research (but not with dance) | Urban Phosphorus Ballet
Pingback: The art of delegation: Perspectives from Academia | Arthropod Ecology
Pingback: Happy second birthday to us! | Dynamic Ecology
Pingback: Is fundamental research a young ecologist’s game? | Dynamic Ecology