What it takes to do policy-relevant science

I just spent last week at SESYNC which some ecologists might still know better as “the new NCEAS”. It is however different from NCEAS in at least two ways. First the first “S” in SESYNC is “Socio” – not that NCEAS didn’t do plenty of applied ecology with a human dimension, but it is right up front and part of every single project at SESYNC. The second is that the focus is on what SESYNC calls “actionable” science. As some of you know, I have a joint appointment in the Sustainability Solutions Initiative (hereafter SSI) at the University of Maine which has a very similar orientation. Of if you want another hook, the NSF SEES granting program has a very similar orientation. Loyalty demands that I point out that University of Maine’s SSI was the first of these 3 programs!

I should warn you this is a long post. This represents 3+ years of active thinking. Feel free to skim or skip if this is not a topic of interest to you.

I don’t say you need to do the kind of science these three institutions/programs are aiming for. Nor do I say that all science should be of this kind. I still have a healthy component of basic research in my own work and think this needs to keep going. But many, many scientists, especially I find earlier career scientists, want to deliver solutions. And I think I could (if I weren’t lazy) build a case using data from NSF that funding for this kind of science is on the upswing. There is an increasing discomfort with the “loading dock” model of science (do the science, put it on the loading dock, and wait for the adoring users to back up their trucks and do all the work to load it up and use it)

The rest of this post reflect my musings, hard won experience, and education by my colleagues on what it takes to deliver policy-relevant (or actionable science or solutions) .A good part of what I am writing here was learned from my colleagues in SSI*.

So without further justification, let me present my version of the 7 P’s of policy-relevant science (yes I am a sucker for alliterative memory gimmicks). . Also just to be clear I am trained and experienced as an ecologist, so that is obviously my bias, but in this piece I am saying things I believe to apply to science generically (and broadly to include all empirical quantitative researchers and maybe even all academic researchers, certainly not just biophysical science, but remember I can only really claim expertise in ecology).

1. Presentation – one common claim is that science would be used for policy decisions more if we only presented it better. This is the idea that scientists are terrible communicators. It is epitomized in Randy Olson’s movie Sizzle. I might even go so far as to say this has been the biggest effort by NSF to make work more useful – they have been conducting extensive media training under the title “Becoming the messenger” conducting workshops all over the country. Personally, I think this one is a big cop out  Are there some terribly poor science communicators? Yes. Would all scientists benefit from additional training in communication? Yes. Will the world change if all science were well communicated? NO! There are already plenty of great scientist communicators. There are also plenty of NGO organizations functioning on the boundary with professional communicators repackaging the work of scientists. Improving presentation is relatively cheap and requires little effort and change on the part of scientists – it would be nice if it were the solution. But its not.  You can put me down for this being 1% of the problem.
2. People – scientists should talk to people who are going to be affected by or care about the problems they addressing. This more commonly goes under the label “stakeholder engagement” and has been a movement building since the social activism of the 1970s and become a US government mandated part of land management under Clinton in the 1990s under the label Community Based Management or CBM. I 100% think policy-relevant science requires engagement with stake-holders (lots of engagement in most cases). However, I don’t think there is much training for scientists in how to do this. And even more importantly, I don’t think there has been much thought in how this process should occur in a way that leaves the scientist doing research instead of just turning into another activist voice at the table. Many social scientists have written research arguing that scientists are just another voice at the table. And of course many politicians are also framing scientists in this just-another-voice box to pursue their own agendas. Thus, while I think stakeholder engagement is critical, I think scientists need to take some real ownership and leadership in figuring out what this looks like and in training our peers. Iin my experience most stakeholders (and most politicians who aren’t pushing an anti-science agenda and many social scientists like Cash) also think science should not just be another voice at the table, but they don’t yet have a well-formed idea of what science-stakeholder engagement should look like either.  In my opinion David Cash has written some of the most thoughtful work on this topic (e.g. this piece which coined the idea of “loading dock” research). So to repeat myself, scientists need to be thought leaders in what stakeholder-engaged research looks like. The next 5 P’s are some of my thoughts of what this should look like.
3. Problem co-defined – this is probably the most radical but most important of the 7 P’s. The vast majority of scientific questions that get asked come from the scientists themselves. The proportion is a bit lower in natural resource departments  but even there a high proportion of the questions are scientist driven. If one steps back though, it is blindingly obvious that if one wants to deliver useful, policy-relevant science, one ought to ask potential stakeholder constituencies and policy makers what science might be useful to them! This is no different than a business finding out what their customers want. This is not to say that scientists are then obligated to do whatever the stakeholders ask. Often they ask impossible questions, questions more expensive than what they or society are willing to pay for, questions outside the expertise of the scientists talking to them, and yes, questions uninteresting to the scientists talking to them. Question definition is best done as a joint negotiation between scientists and stakeholders, and this has been called in the literature as “problem co-definition”. Despite the obviousness of the necessity for this approach, there is an equally obvious reason why it is rarely used – it involves loss of control for the scientists! it is frightening. It is a radical break from existing practice. The experience of SSI with over 20 separate projects is that every time you start a project thinking you know what the stakeholders need but then go ask them before starting, you are usually only somewhere between 0-50% right. Every single SSI project has changed their research questions in fundamental ways in response to stakeholder engagement. This approach fully embraced will radically change the kind of science that is done. That said, in SSI every scientist is still happy with the questions they co-defined and even happier with the fact they are sure the work will be useful. This might be radical, but it might not be as painful as people think at first glance. It might even be good for science for us to be pushed to pursue some questions we avoid because they’re hard!
4. Place-based – One common difference between what the researchers want and the stakeholders want is that the stakeholders want much more specific research that is useful to them. In contrast, most scientists are trained to generalize, generalize. It can seem a come down to be asked “what is the population size of deer over the last 5 years in Orono, Maine, USA?” Asking what are the general drivers of deer populations is a more scientific seeming question. And again, natural resource department researchers are already much more used to this kind of research (for which biology/EEB departments sneer at their colleagues for not doing general research and for which the natural research department folks sneer back at their colleagues for building castles in the sky). This is a real issue and research really does fall on a spectrum from highly general to place-based (and organism based and time-specific). To some degree people wanting to do policy-relevant science might just need to sacrifice a bit and move past their comfort zone to do more place-based research. But I would like to argue this dichotomy is made out to be bigger than it really is. How often does a researcher aiming for general results really span more than a handful of sites in a small geographic area at a limited point in time? This is research that is place-based even if it might also be designed to be generalizable. Many scientists can do policy-relevant place-based research funded by policy making agencies (e.g. USDA, state wildlife departments) while still writing very general science papers from the data in addition to the detailed grey-literature place-based reports they deliver.
5. Poly-disciplinary – OK – I made this word up to fit my P fetish. Interdisciplinary would be the more common word. Or the hip word would be transdisciplinary (so fully merged the individual disciplines aren’t recognizable). But I think most ecologists recognize that if you want to make the world more sustainability the biggest challenge is humanity. So you better study humanity. The NSF program Coupled-Natural Human (CNH) systems gets at this, although I think it is fair to say that not all policy-relevant science need use a CNH approach (often times the human system completely dominates the natural system and the idea of delicately balanced back-and-forth feedbacks isn’t too useful) nor do all CNH studies lead to policy-relevant science. Instead it is often important to understand the psychology of what motivates people to change, the economics and policy of creating appropriate incentives, human dynamics of population and consumption growth, technology change, etc.
6. Post-research-engagement – There is some talk in the literature about co-definition of the problems (see #3 above) and some talk about improving mass-communication (See #1 above), but I think truly successful policy-relevant science requires more – namely stakeholder engagement during research to some degree, but especially after the research is done. As science transitions into policy, there is a very important period of interpreting research. Again scientists like to pretend we don’t do interpretation, but there is no denying it happens in a policy arena. Being actively engaged with stakeholders during this stage is critical. In part it is because stakeholders can help with the communication and help us strip out jargon and avoid obvious pitfalls (like communicating climate change in degrees Celsius instead of Fahrenheit to the American public). More subtle issues of presentation are often important too – for example there have been interesting studies of “boundary objects” – things that sit on the fence between science and policy like maps and what makes them successful or not. But things beyond communication like pulling out what is important to policy, the implications for policy etc also must occur. To my mind this aspect of stake-holder engagement in interpreting research after the research is done (or at least a round of research is done) is the most overlooked and ignored part of doing policy-relevant science. This is why I prefer the phrase co-production of science with its focus on engagement with stakeholders from start-to-end over the more common phrase co-definition of problems or the other common-phrase of stakeholder-engagement that doesn’t given an explicit relationship of stakeholders to research (opening the door to the scientists are just another voice at the table paradigm)
7. Personal relationships – stakeholder-engagement has already been discussed above. But it is important to note that successful stake-holder engagement will depend directly on personal relationships built up over time and through informal contacts over meals and what not as well as formal meetings. In a boundary-spanning class I co-taught (more about this below) we brought in 20 people with experience in spanning boundaries between science and policy and literally all 20 people said that in the end personal relationships were the most important thing. This is pretty different from science with its efforts (although never fully successful) to focus on objective progress independent of personal links. But if a legislator is going to take a vote based on science, you can bet they want to know the person who did the science and trust them.

So to do policy relevant science, all you have to do is spend a lot of time  with stakeholders, let them have 50% of the job of deciding what research to do, study complex social behaviors in addition biophysical sciences, become more place-based, and open up the research interpretation process to other people and invest time in building relationships with them. No problem, knock it off with a few extra hours of work, right? Of course not! Policy-relevant science involves a fundamental change to the way science is done (again assuming policy relevance is your goal – not all science should have this goal). I have argued above that some aspects like the place-based research and the stakeholder co-defined research need not be as frightening as they seem. But in total, there is no denying doing policy-relevant science is a lot of work! And a big change for many scientists.

So for those scientists who are making this change, or for those educators and administrators trying to facilitate this change, how should the change happen? As always, I am ready with a silly mnemonic. The four T-s of transition.

• Training – This should be obvious to professors, but in truth we have a blind spot – we have gotten so good at teaching ourselves things related to our discipline that we think we can teach ourselves anything. Or that science is the hard part, policy is easy to pick up by osmosis (this is obviously false). If we want scientists to do policy-relevant science we need to train them in skills related to the 7 P’s and this includes soft skills like facilitation  To my mind this can be best summarized as the task of boundary-spanning. A transformative event for me (and I think the 3 colleagues I co-taught the course with – David Hart, Laura Lindenfeld and Kathleen Bell) was teaching a 1 week intensive course in boundary spanning for graduate students. In addition to reading some of the literature and theory of boundary spanning (probably a whole other post in there), we brought in panels of people who successfully span boundaries and had them speak about what they saw as key success factors. But whatever you call it, there are a lot of skills that boundary-spanning scientists doing policy-relevant science need. Media training (P #1) is just the tip of the iceberg.
• Teams – the biggest error in thinking is that one person can or should do all 7 P’s. It is a rather laughable idea actually when you really stop to think about it. This means that policy-relevant science almost always happens in teams. In these teams everybody needs to have the rudiments of the 7P’s but different individuals with different strengths and expertise can complement each other to actually fulfill the full range of the 7 P’s.
• Tag teams- policy relevant science can also happen by one person doing one piece and making it available in a public manner and then somebody else picking it up and doing another piece, not unlike children’s tag-teams where when you tag somebody you go out and they step in to do something (or WWF wrestling if you prefer). People can do this without ever thinking of themselves as a team or ever even meeting. There is obviously a continuum from strong teams through weak coalitions to tag-team processes. And tag-teams are dangerously close to the loading dock model that was derided above and in the piece linked to above by Cash. Thus I expect the majority of successful policy-making will occur closer to the team than the tag end of the spectrum. But some good policy-relevant science happens through tag if people do it in a thoughtful way. And no doubt this approach can be used to channel more basic research in useful ways. As an extreme example you don’t need stakeholder engagement to know that cold-fusion or understanding the causes of extinction are important research topics. Just don’t use this as a cop-out to avoid teams.
• Thinking – I’m talking about big changes. Scientists need to apply their academic approach of discussion, analysis and data to the 7 P’s and success factors in doing policy-relevant science. Social scientists have been the ones studying and writing about this problem most to date. Many of them are doing a good job. A few of them in my opinion don’t get biophysical research at all. But it is kind of ironic that if the ideas which emerge are things like teams and inter-disciplinarity that the topic of how to policy-relevant science is not team-based and inter-disciplinarity. A full understanding of how best to do policy-relevant science needs to include the biophysical scientist’s expertise and perspectives too. This has not been happening enough.

You can call everything I have talked about boundary spanning or co-production of science (as Cash that I cited above does). I think both of these phrases are useful. But whatever you call it, it is a lot of work and the world needs more of it. And as a scientist trying to learn this material, I found very little published material (journal articles or otherwise  to help me learn. That needs to change.

I am very curious to hear what others think. Again I don’t want this to turn into a debate about whether policy-relevant science should happen and scientists should be part of it. They should but not everybody should be forced to do so – basic science is important to. For those who would describe themselves as already engaged in policy relevant science, what did I get right or wrong? For those just starting out, was this helpful? what are your impressions? For those who have never done policy relevant science, does this scare you away or appeal to you?

* Including amongst many others David Hart, Tim Waring, Kathleen Bell, Mac Hunter, Aram Calhoun, Laura Lindenfeld and Shaleen Jain