In the past year, I’ve been working on several projects that used Likert-scale data (e.g., 1 = strongly disagree, 5 = strongly agree). And, in several instances, there were questions that it made sense to pair. As one example (which I blogged about in more detail earlier this month), for Morgan Rondinelli’s undergraduate thesis project on student mental health, we asked students whether they would think less of someone who sought mental health care and also whether they thought others would think less of someone who sought mental health care? In that case, I was curious not just about the aggregate percentages in the different categories, but also how individual views compared. So, being a good evolutionary ecologist raised on reaction norms (where genotypes are plotted in different environments, with the points for each environment connected by a line), I made a paired line plot:
This figure shows me that no students viewed themselves as more judgmental than the average: none of the lines go up. That’s not information that I could get from other ways of plotting the data (shown in my earlier post).
A different example comes from a project studying student views on climate change, which I’m working on with Susan Cheng and JW Hammond. We asked students the same questions at the beginning and end of the semester. To focus on one question, we asked students “Do you think climate change is happening” at the beginning of the semester and again at the end of the semester. The overall results were promising:
My take on these results is: nearly all students come in to the course accepting that climate change is happening (good news!) and they become more sure of that over the course of the semester (also good!)
But, again, I was curious about how individual students changed. So, again, I went to a paired line/reaction norm-style plot:
Some students became more skeptical/less sure about climate change over the course of the semester, but the vast majority either stayed the same or became more confident/more sure. That’s good news, too!
I showed these plots to several people – collaborators on the projects, folks who do pedagogy research, colleagues at Michigan’s Office of Academic Innovation. Their initial reaction was generally a furrowed brow, tilted head, or something along those lines. That was my clue that this is perhaps not a typical way of plotting Likert scale data! However, after a minute or so, they generally seemed to think it was an interesting way of viewing the data.
This led me to wonder if I had accidentally stumbled onto a useful, not widespread way of plotting Likert scale data, by virtue of my training in evolutionary ecology. I’m not sure – I definitely haven’t read enough literature in fields that typically use Likert-scale data to know for sure. But my google image search only found two papers that seems to have paired line plots for Likert scale data. (I probably spent more time on this than I should have, but it was far from exhaustive and definitely sensitive to keywords; it’s entirely possible this is common and I just don’t realize it!)
So, just in case this is useful for others working with Likert scale data, I figured I’d put the idea out there that paired line charts can be a useful way of getting a sense of how individual views compare for paired questions. In the future, I’m planning on comparing these with heat maps, but R and I were not getting along well on Friday. Hopefully there will be a future post with that comparison!
Sure, these are great plots for repeated measures data (which these data are) that you often see as exploratory plots of the raw data — it doesn’t matter if the measure is continuous or ordinal. Maybe people furl their brows because they aren’t bars with standard errors and don’t have asterisks (and they don’t explore their data?). My problem with standard summary plots (mean and error) of repeated measures data is that they hide this correlation between points that your plot shows. So maybe just add the modeled mean and modeled CI (or bootstrap CI) and you have the best of both worlds.
I learned the importance of plotting the raw data as a grad student. When I finished collecting data for a big, labor intensive project, I entered the results and was super excited to see what had happened. I originally plotted the means & standard errors and thought the experiment had yielded super boring results. I sent a disappointed email with the graph to my advisor, who noticed that something seemed to be going on with the error bars (why were they so much bigger at one time point?), and reminded me that I should always look at the plot of the raw data. He was definitely right!…Fast forward 15-20 years, and now I regularly find myself reminding people in my lab to plot the raw data!
I saw this style of plot in an education talk at ESA this year and have been wanting to use them for my plethora of bio ed data (seriously, #drowningindata), but I haven’t gotten around to figuring out how to make them in R. Could you suggest a package or tutorial?
Look up “parallel coordinates” and “parallel sets”. There are a variety of R packages that implement variations on this theme.
Here is a skeletal ggplot script to make a plot like these https://rdoodles.rbind.io/2019/01/paired-line-plots/
I used paired line plots to report feeding trials with deer. Deer had two choice of food, and on one side of the line, I plotted consumption of food A, on the other side of the line, consumption of food B (my goal was not to promote my article here, but in case someone is curious: https://doi.org/10.1139/cjz-2017-0063).
I agree that it’s a great way to show individual differences! And I think those graph are not used often enough!
I wish I’d known about (or thought of) these plots before I wrote this post: https://dynamicecology.wordpress.com/2018/12/03/poll-results-which-classic-topics-do-ecologists-care-about-and-which-ones-do-they-think-all-ecologists-should-care-about/
“I showed these plots to several people – collaborators on the projects, folks who do pedagogy research, colleagues at Michigan’s Office of Academic Innovation. Their initial reaction was generally a furrowed brow, tilted head, or something along those lines. That was my clue that this is perhaps not a typical way of plotting Likert scale data! However, after a minute or so, they generally seemed to think it was an interesting way of viewing the data.”
Meghan, do you see this as an example of something Andrew Gelman talks about–how a figure that’s hard to grasp at first glance can actually be a *good* thing? Because the challenge of understanding it draws the viewer in and gets the viewer thinking actively about what the figure means.
Obviously, one can have too much of a good thing here. A figure can just be plain ol’ confusing or over-complicated. But are there cases in which the optimal figure prompts a *bit* of brow furrowing and head tilting on the part of the viewer?
The simple answer is that it’s inarguably true that there are optimal figures that induce brow furrowing: Many varieties of data are sufficiently complex that it’s impossible to show all of the relevant detail in a single figure.
Context-switching between figures however is a significant cost – viewers do not remember what they’ve seen, they remember what they’ve told themselves about what they’ve seen (see “change blindness”, “inattentional blindness”, etc). As a consequence, there’s quite often a tradeoff to be made between using a more complex representational paradigm to cram a bit more information into one figure, and splitting the representation into multiple figures.
Since the context-switch cost is so large, optimality usually lies just this side of sufficient brow-furrowing that the viewer gives up on reading the figure at all.
From my point of view, a more interesting phenomenon occurs when viewers prefer, presumably through habit, representations that are actually harder to understand, and furrow their brows about representations that are significantly easier and faster to parse. One of the simplest examples of this is the pie chart, which is the optimal representation for just about nothing, but which many viewers reflexively prefer, even if they have a hard time extracting any useful information.
A more interesting one I run into a lot in my work, is the bizarre fixation of molecular biologists on representing motion as a series of poses captured as though by a stroboscope. Despite the fact that every human on the planet, if asked “how something moves”, waves their hands as though drawing a time-lapse image in the air – show a molecular biologist a time-lapse image of molecular motion, and much brow-furrowing ensues. Show them a batch of Muybridge-horse-galloping poses overlaid on top of each other and they’ll nod their heads sagely – and then not be able to answer a single question about any aspect of the motion…
This makes me think of how limnologists always plot things upside down. Depth is on the y-axis, with zero at the top. It makes sense if you’re used to thinking about lakes, but it makes everyone else’s heads explode.
To get to Jeremy’s original point: I remember being taught explicitly in grad school that a complex figure that you need to really think about to understand is not necessarily a bad thing. But I’ve wondered if that advice is less true now than it was before. People don’t spend as much time on individual papers now, and I definitely have changed my writing to try to factor that in (e.g., putting the key message for a figure right up front in the figure legend, for someone who is only glancing at the figures).
Despite being an outsider to ecology, I love the breadth of topics that this blog manages to cover and that’s good enough to keep me coming back, but this time, against all odds, you’ve wandered directly into my lab!
Plots like this are actually a special case of “parallel coordinates” ( https://en.wikipedia.org/wiki/Parallel_coordinates) plots, popularized by Inselberg. Inselberg focussed primarily on continuous data rather than ordinal or categorical data, but the visualization (if not geometric interpretation) generally applies to binned data as well ( https://eagereyes.org/parallel-sets ).
Our contribution to the party was the realization that what you’re really visualizing is a contingency table, and in the most obvious version, the edges between the categories on the parallel axes are the cell populations. However, for some analyses, what’s interesting is whether the cell populations are different from the expectation based on the marginals, so what you’d rather see are the residuals.
Unfortunately our server is offline at the moment, so I can only point you to a horribly dated paper ( https://www.ncbi.nlm.nih.gov/pubmed/15215351 ), but: If you take a version of the parallel-sets idea of using a scaled region on the axis to represent how much of the population falls in a given category (the marginals), and then use an equally scaled “link-bar” between the marginals to indicate the joint (cell) population, you get a decent visual version of a contingency table.
For some analyses, this turns out to have wasted ink: If 100% of your population lands in category A on the first axis, and 100% lands in category B on the 2nd axis, it’s hardly meaningful to show the viewer that the joint population is also 100%. So, in some cases, one gets more insight by subtracting the expected (based on the marginals and assumed independence) cell population from the observed cell population and plotting the link-bar for the residual instead.
I wish our server was up (I really should poke someone in the lab about that), as we have some nice software that lets you plot your data in both our circular (our data typically uses a lot more than 2 axes, and we’re looking for complex network features) format, and planar format, as well as as provides a lot of options for exactly what data is presented for the links between features.
I agree, this is novel for me too and we use rating scales a lot in my area of research. It is nice how it preserves all individuals and highlights anomalies when there are large numbers of respondents.
If one is interested in change, then normally I think we would calculate post-value minus pre-value. That could be graphed as a frequency plot.
While the change may be meaningful, it is also possible that it is due to a changed interpretation of the scale, or a misunderstanding of the scale (i.e., random error).
Just out of curiosity, for anyone, what book do you give to students to introduce them to R?
Getting Started With R: https://global.oup.com/academic/product/getting-started-with-r-9780198787846?cc=ca&lang=en&
Review here: https://dynamicecology.wordpress.com/2012/09/30/book-review-petchey-and-beckermans-getting-started-with-r/
But your mileage may vary. I like this book in part because the topic coverage overlaps a lot with the tasks that my beginning undergrads need to do in R. If your students have different needs than mine, you’ll want a different book.
Great Jeremy, thanks. For me, actually. Skill set update.
Pingback: Comparison of ways of visualizing individual-level Likert data: line plots and heat maps and mosaic plots, oh my! | Dynamic Ecology