When I review papers, I often read the introduction and methods, and then skip to the figures to see what I take away from them before reading the results. This can also be done the opposite way: read the results and imagine what they would look like in figure-form, then go look at the figures. I find this really useful when reviewing for making me get out of the passive reading of a manuscript and for encouraging me to think critically about the results. Sometimes, there’s a great match. Sometimes there isn’t and I realize I misunderstood something (which sometimes is just me messing up, but sometimes suggests something that is unclear in the paper). And sometimes I can’t figure out the reason for the discrepancy, which ends up being something I bring up in my review.
I was originally thinking about this as a tip for reviewing – as I said, it helps me think more deeply and critically about a paper. But, over time, I’ve realized it relates to a bigger issue: the accessibility of a paper. If you have a figure that clearly summarizes your results, your paper will be much more accessible to everyone from specialists in your area (the people who review your manuscript!) to non-specialists (including people who serve on search committees and award committees) and perhaps even to the general public.
As an instructor, I am always looking for interesting examples to use in class. Sometimes, there’s a figure that beautifully shows the results and that is accessible. I just read this paper and saw this figure and immediately thought that I need – need! – to add it to my lecture on food webs:
But, much more often, I see an interesting paper on a topic I teach about, but there’s no accessible figure that summarizes the results.
In 2001, Charles Krebs had a piece in the ESA Bulletin entitled “Why are my brilliant research findings not utilized in ecology textbooks?” In it, he suggests the following exercise:
Read a paper in Ecological Monographs (for example) that is not directly in your field of expertise, and try to extract a 1-2 sentence summary of findings reported in this paper, along with one figure to illustrate key results. You will find you cannot do this for most papers because the authors have not provided a succinct abstract or summary diagram to illustrate their findings. Now go back and look at your key papers and see if you have done the same thing.
I suspect that most people would not be able to do this for most of my papers, which suggests this is something I need to work on! And that’s even with having received this advice as a grad student. Back then, someone who read a draft of a manuscript I was working on said something along the lines of: “This could end up being a textbook example. Make the figure one that could go in a textbook.” As a graduate student, that was something I hadn’t considered, but it was good advice and made me think really hard about how the figure should look. Even if your work doesn’t end up in a textbook (and, as far as I know, mine hasn’t), it never hurts to have a clear, accessible figure!
So why is it so hard to find papers that do a good job of meeting Krebs’ target? In some cases, it might be unavoidable that there isn’t one key figure that tells the paper’s story – some results are more nuanced. But, even in those cases where there isn’t one key, broadly accessible figure in the paper, it should be possible to create a graphic that tells your story clearly. As one example, my postdoc Nina Wale recently had a paper come out based on her thesis work, and worked with the Penn State press office to create this visual synthesis of her work:
(source)
Making a synthetic figure like this takes time, but it also leads to more people reading your work. One journal found that adding visual abstracts to tweets led to 2.7 times more people clicking the link to read the paper. I find them useful for teaching, too – for example, I use this graphical abstract in my class:
as a way of setting up the experiment before showing them some data from it.
(Source: Goodrich et al. 2014)
So, I think I need to set myself a new goal for manuscripts: when making the figures for them, I should think harder about whether one of the figures can synthesize my story. And, if there isn’t one figure that I can point to, I should consider making a synthetic figure that can be used as a visual abstract. Krebs noted:
Graphical summaries or flow diagrams are particularly economical ways of communicating research findings, yet very few papers use them to encapsulate the discussion and synthesis of results.
This is a great excuse to use some of my #readinghour time this semester to read Edward Tufte’s Beautiful Evidence (which I’ve been wanting to read, but haven’t gotten to yet)!
Do you think your papers would meet Krebs’ target? When preparing a manuscript, do you think about making your figures textbook-ready? Have you used visual abstracts or created a graphical summary? I’d love to hear from readers about their experiences and tips!
Dynamic Ecology 
Along these lines, Elsevier (booo, I know) recommends submitting graphical abstracts of papers
https://www.elsevier.com/authors/journal-authors/graphical-abstract
Via Twitter, a quibble:
And the follow-up:
Very important point you make.
As the literature explodes and people increasingly skim read, I bet the fraction of readers who download a paper and who only read the abstract and look at the figures is over 50%. Make them count!
As an EiC screening which papers to send out to review, figures come in 2nd or 3rd after abstract and maybe cover letter on what we look at first and influences us most in deciding whether to send it to review.
Figures also are a great way to help one figure out the point of their paper or even design the experiments (e.g. Terry’s post on a nut figure https://smallpondscience.com/2017/11/13/starting-experiments-with-a-nut-fig/)
Add in your points about reviewers and the general public and as you note, great figures are a win all the way around! The time spent on the figures is probably more valuable than on any other part of a paper (with the abstract coming in 2nd I would argue).
Via Twitter:
Totally agree that it’s worth our time to invest more in figures that tell a clear, almost stand-alone story. I’ve had the exact same experience in teaching and book-writing of “knowing” a result is out there (memory says it’s been observed multiple times), but then not able to easily find a nice graphical illustration. Most often a simple x-causes-y scatterplot or bar chart tells the clearest story.
A related issue I’ve come across increasingly in recent years is plots of raw data vs. plots of estimated values (y-axis in particular). Especially with more sophisticated models, the effect of x1 on y (the story we’re interested in) is often estimated after accounting for effects of bunch of other fixed and random factors, so the raw data doesn’t reflect the stats, and might not even clearly show the relationship revealed in the statistically “correct” model. So we see more and more plots of model estimates in place of raw data, which (as editor) I’ve seen cause miscommunication between authors and reviewers.
Should the story-summarizing figure always be raw data?
“A related issue I’ve come across increasingly in recent years is plots of raw data vs. plots of estimated values (y-axis in particular). Especially with more sophisticated models, the effect of x1 on y (the story we’re interested in) is often estimated after accounting for effects of bunch of other fixed and random factors, so the raw data doesn’t reflect the stats, and might not even clearly show the relationship revealed in the statistically “correct” model. So we see more and more plots of model estimates in place of raw data, which (as editor) I’ve seen cause miscommunication between authors and reviewers. ”
THIS. So much this. But what to do about it? I have no idea.
At minimum, simply stating clearly what’s in a graph seems like it should be obligatory (and easy).
For graphical relationships that are central to conclusions, one could imagine making it routine to also show (in an appendix) the relationship using raw data. Seems easy enough, although there are cases where the distinction isn’t even that clear (e.g. an average community-wide trait for which values for some species were themselves estimated using data not from that plot).
More stats-savvy people presumably have more informed opinions than mine.
For GLMs I’m thinking of plots of least square means and partial regression plots. But those don’t have analogues for many other sorts of statistical models (do they?)
At the risk of causing this comment thread to go viral, I’ll note that this issue gets back to Brian’s old post on whether the statistics in ecology papers are becoming too hard to understand.
I feel like Andrew Gelman must have some posts on this. He’s big into graphical presentation of data, but also big into fitting Bayesian hierarchical models with lots of terms.
This is where the legacy of history in stats is biting people. LS means are just a type of prediction from the model at certain values of covariates; you can generate the same kinds of predictions for any model like this, not just one that assumes the responses are conditionally distributed iid Gaussian. Using GLMs (the generalized version; for the general linear model we should just call them LMs) there is the extra step of applying the inverse of the link function to get back on to the natural response scale, with GLMMs there’s an additional aspect of whether you want population-level effects or estimates for lower levels in the hierarchical model. We don’t need a special name for all variants of these things; the original method got it’s name because of the time in which it was invented/proposed. Knowing what we do now, we wouldn’t special-case the Gaussian or least squares-based approaches.
It’s all the same recipe, just clouded by confusing terminology. Richard McElreath continously emphasises this & related points in his book and video lectures.
Yes, I agree that this relates to Brian’s old post (and had originally thought I should link this to that but then forgot!) I agree with Mark that a first step is specifying exactly what is being plotted (sometimes it’s impossible to figure this out!) I also am in favor of plotting the raw data in an appendix if the figures in the paper are based on the estimates from analyses.
We were just discussing this with a student’s paper. She made one version of a plot which shows the effect sizes, which makes a lot of sense for one point we want to make, but the raw data helps with another. So, we’ve concluded we should include both; the main remaining decision now is which to put in the main text and which in the supplement.
Mark and Jeremy – I have been working on this since Thanksgiving day
https://middleprofessor.shinyapps.io/Hplot/
Which is the beginning of the answer to how I’ve been thinking about these questions. This is also related to how we teach students statistics and providing proper tools for this. In short, Most papers plot means and SEs or CIs of the mean but often what we get out of a model are effects and it is these that interpret. But, we want to also see the raw data. So I combine these into a 2 part plot, with a dot plot (raw data) and boxplot (summary distribution) in the lower part and effects with CI in the upper part. The framework is explicitly modeling (with some concession by supplying anova tables after everything else).
There were two motivators 1) researchers who only plot bar plots with 1SE need an easy tool for better plots because this isn’t in excel (many researchers in physiology and cell biology cannot code in R so supply R code isn’t going to help) and 2) training undergrads starting in intro biology with the plots that “tell the story” which is very indirectly told by a bar plot with SE error bars.
I think you have some of this backwards, Mark; things like random effects are there to account for variation in the response that is due to some clustering or other source of random variation unrelated to the thing of interest. You have data from lots of sites but there are intrinsic differences between sites that show up in the raw data as extra variance. This can mask effects when estimating them in a model, especially if unaccounted for.
The authors here averaged data at the transect level and then used ranefs at the plot/site level. They averaged away the variation within plots/sites but tried to account for inherent variation at the plot/site level whilst estimating the treatment effect.
I do agree with your last point however; we need both kinds of plots. A plot of the estimates from the model and their uncertainties, and a plot of the effects superimposed on data. The first makes it easy to quantify for the read the effect and estimated differences between treatments. The latter gives the reader a chance to evaluate the magnitudes of the effects on a meaningful scale. Both plots serve readers, perhaps different readers, well.
Then, however, we have a tension between page limits and presenting the full story…
What an excellent point and a good post for teaching. I do have a question that arises from your motivating plot (the bar graph, Fig. 3 in the paper) for researchers that work with count data with lots of zeros. Specifically, the plot captures the story in spirit but misrepresents the detail of the data. The authors kindly supplied the data and code so it is easy to replot it. Focussing on the sunflower star in 3A (which is driving the before/after events), if instead of a barplot, one plots a jittered dot plot (or simply histograms of the distribution), it is clear that most of the counts (or densities since its per m^2) are zero both before and after. While the means are far apart, the medians are zero both before and after. So the plotted SE for the bars is pretty meaningless to me and certainly doesn’t represent the analysis (similar to Mark Vellend’s point) – note that two SEs for some of the bars in Figure 3 would imply negative counts.
My question is, how do people with these kinds of data plot the data to compare the distributions or tell the story? It is hard to plot count on a raw scale and using a log10 axis removes all the data with count =0. So I can fake it by recoding the zeros to 0.1 (for the purpose of plotting not analyzing) and create a pseudo-log y axis that works pretty well. It’s even better to draw lines between the before/after points from the same transect (counts of all but 3 transects drop or stay zero).
I didn’t mean “It is hard to plot count on a raw scale” but plotting the count on a raw scale (with a histogram or jittered dots) doesn’t reveal much of a difference between before/after since most counts are zero or very small (and so masked on a raw scale).
This data is clearly shouting out for a GLMM, with a Poisson (or some other count) distribution plus an offset to account for the areas or random effects at the transect level. That way the raw count and the model would coincide nicely.
I see this approach (the one used by the authors) all the time, where a simpler model would have been obtained than the one the authors ended up with all because they worked with data on an odd scale.
[Having now looked more closely at their data, the issue is more about averaging transects.]
At the basic level, the authors collected counts of things — integers. It is only their desire to do some processing of the data that leads them down the path to choosing to model data with strong mean-variance relationships with a transformed response shoved into a linear mixed model. There data didn’t look count-like once they’d averaged over transects at the plot level but it doesn’t meet the assumptions of their modelling technique (LMM) so they log(y + 1) transform it.
As the authors have provided the data (good on them!), I’m going to have a go at a different analysis that addresses these issues.
I had a play with the transect data and prepared a few plots: https://gist.github.com/gavinsimpson/bf11b821be44fc24e30ff8546ad8ca9d
I’m assuming the Poisson GLMM fits the data OK – a more careful analysis would check that the data were conditionally distributed Poisson, plus do some other checks etc.
I generated similar figs but your figure with the sqrt axis is what I was looking for.
IMO figures clearly illustrating the story are desirable but using them as a stand-alone summary is fraught with the potential for over-simplification and overlooking of counter arguments or parts of the question that aren’t clearly answered by the data. The probs with a particular analysis should be as much a part of the story as the questions it appears to answer clearly, no?
I guess I would argue it is one-directional. There should be good figures that clearly demonstrate the claim. But that doesn’t mean the claim should be accepted just from the figure – a full dive in methods is needed for that.
Yeah, I don’t know about that. I’m all for great figures and clear commumication, but if we’re making it “clear” by taking out something important – the parts of the results that don’t quite fit the hypothesis – that worries me. Part of “the claim” is that there is a justification for overlooking the results that don’t support the claim. It seems to me that such a rational is a key point.
This relates to my previous complaints that online supplements are changing how we write papers, in some cases for the worse (I would argue). Indeed, I don’t worry so much about individual figures being presented in such a way as to tell an oversimplified story. I do worry about online supplements being used to structure the entire ms in such a way as to oversimplify the story. If the main text is basically just an extended abstract for a huge, poorly-written, poorly-organized online supplement, in what sense has the paper actually “told” the story, or told the story well?
https://dynamicecology.wordpress.com/2015/10/26/online-supplements-have-ruined-nature-and-science-papers/
https://dynamicecology.wordpress.com/2014/09/24/what-belongs-in-the-appendices-vs-the-main-text-in-scientific-papers/
https://dynamicecology.wordpress.com/2015/05/04/scientific-papers-recut/
An email correspondent remarks that figures that tell a story are equally important for theory papers. Many of the most influential theoretical papers in ecology are influential at least in part because they have memorable figures illustrating the model’s mathematics. Think of Tilman’s resource ratio model of competition, or the marginal value theorem for habitat selection.
Of course, the problem is that pictures are memorable, period. So for instance, Connell’s figure illustrating the prediction of the intermediate disturbance hypothesis is very memorable–even though it’s not illustrating a mathematical model, or even a prediction derived from a mathematical model. That’s not a criticism of Connell–it’s fine to have the germ of an idea and to illustrate it! It’s just to say the IDH became a zombie idea (https://dynamicecology.wordpress.com/2012/09/13/zombie-ideas-in-ecology-the-tree-paper-is-now-online/) because the original germ of the idea kind of got picked up by empiricists, disseminated, and reified before it really grew beyond the germ of an idea. Which happened because the original germ of an idea was illustrated with a memorable figure.
My correspondent reminds me of another, even better example: MacArthur & Wilson’s famous picture of the island biogeography model. Which is a picture of underlying math, but also can be used to generalize the math.
Same for the Rosenzweig-MacArthur model of predator-prey dynamics. The picture illustrates what the math is assuming, and helps you understand why the model predicts what it does. But it also provides an informal-yet-quite-reliable way to generalize from the explicit mathematical model to a broader but not-explicitly-specified class of mathematical models.
MacArthur’s graphical approach isn’t one you see used much any more in the theoretical literature (do you?) I wonder why that is? I’m tempted to say it’s just because theorists now focus on models and phenomena that aren’t amenable to graphical analysis (e.g., models with many state variables rather than just one or two; models with stochasticity, etc.). But maybe that temptation is wrong–maybe one just needs different kinds of pictures to illustrate and generalize the sorts of models ecological theoreticians study these days? Or maybe the temptation is right, but for a different reason–maybe theoreticians these days are too often studying complicated special cases, rather than also (or instead) focusing on simple cases that are amenable to graphical illustration and that “capture the essence” of the more complicated cases?
The second example in this post is an excellent example of a figure telling the story. And Meghan is apparently too modest to link back to it herself, so I’ll do it. 🙂
https://dynamicecology.wordpress.com/2012/11/19/do-bird-papers-have-the-best-figures/
My PI once told me that people will only remember two things from your talk: the first page and the last page. What do you want them to remember? Now that I work in industry, I find that the only thing that keeps people interested in my presentations is a good visualization — make a compelling image, and they will listen!
Pingback: When writing, tell us your biological results! | Dynamic Ecology