What I learned about scientific misconduct from reading the NSF OIG’s semiannual reports

The National Science Foundation’s Office of the Inspector General (NSF OIG) investigates misuse of NSF funds by individuals and organizations that receive awards from or conduct business with NSF. That includes, but isn’t limited to, researchers who commit scientific misconduct in the course of NSF-funded research. The OIG reports on its activities to Congress every 6 months. The reports are public, you can read them online. As part of my ongoing efforts to educate myself about scientific misconduct, I skimmed the five most recent OIG reports. Here’s what I learned.

• Misspending and fraud detected by NSF OIG amounts to something like ~0.1% of the NSF budget. The headline number of each semiannual report is the amount of money OIG saved taxpayers. It includes recoveries and penalties paid by individuals and institutions found to have misused funds, and “questioned costs”. In a typical recent year, NSF OIG reports something like $8-10 million in savings to taxpayers. Sometimes more, sometimes less, but that’s the ballpark figure. That’s as compared to the NSF annual budget (all of it, not just grants) of a bit more than $8 billion in recent years.
• Most of the misspending and fraud detected by NSF OIG does not involve scientific misconduct. That is, most of the taxpayer savings NSF OIG reports don’t come from cases of NSF-funded researchers fabricating data, falsifying results, or committing plagiarism. Rather, the bulk of the taxpayer savings seem to come from OIG busting misspending and embezzlement, mostly by institutions and private businesses but some by academic researchers. For instance, a university misspending research grant funds to pay teaching assistants for many years. A small businessman defrauding an NSF technology transfer program to pay his wife a bunch of money. A company defrauding NSF by billing twice for the same work. Etc.
• The large majority of allegations of scientific misconduct to OIG involve plagiarism. This semi-annual report includes helpful tables (Tables 3-5) breaking down all allegations of scientific misconduct received, and investigated, by OIG from 2010-2019, and how those investigations were resolved. Almost 80% of allegations of scientific misconduct received by OIG involve plagiarism; only ~20% involve fabrication or falsification of data. The same is true for investigated allegations: OIG investigates about 80% of allegations of scientific misconduct it receives, and ~80% of those investigated allegations involve plagiarism.
• Scientific misconduct allegations to OIG have dropped a lot since 2013. The same tables linked to in the previous bullet indicate that, from 2010-2013, OIG received over 100 allegations of scientific misconduct annually, maxing out at 117. Allegations are way down since then–below 100 every year, and below 60 every year but one. In 2019 there were only 33 allegations. Not really sure what to make of that. Maybe allegations of fraud were higher in the wake of the ARRA stimulus money NSF had to spend quickly during the Great Recession, and have since fallen back to “normal” levels? Any ideas?
• OIG finds scientific misconduct in ~24% of the allegations it investigates. That number is calculated from Tables 3-5 in the report linked to above. I am surprised it is that low. I would be interested to see a breakdown of the reasons why misconduct isn’t found, because there could be various reasons. Maybe the alleged misconduct doesn’t qualify as “misconduct” under NSF’s definition. Maybe evidence was lost or destroyed. Maybe the allegation was found to be baseless or malicious. Etc.
• As allegations of scientific misconduct to OIG have dropped, a greater fraction of allegations have been substantiated. Sticking with Tables 3-5 in the report linked to above: from 2010-13, OIG found scientific misconduct in 14-20% of its investigations (the exact percentage varied a bit from year to year). Since then, it’s only dropped below 30% once, and has been as high as 54%. No idea what to make of that. Maybe whistleblowers who have suspicions, but lack cut-and-dried evidence, are more hesitant to go to OIG than they used to be, perhaps because they’ve come to doubt that going to OIG would help? Maybe, with fewer allegations of scientific misconduct, and thus fewer investigations, OIG can devote more resources to each investigation? Maybe scientific misconduct is taken more seriously these days than was the case 10 years ago, so the institutional investigations that often inform OIG’s own decision-making are improving in thoroughness? I dunno, just spitballin’.
• There’s some terrible scientific misconduct out there, and some bizarre excuses for it (even if it doesn’t cost NSF much money in the grand scheme of things). The upshot of the first two bullets is that scientific misconduct detected by OIG is a drop in the bucket for NSF’s budget. Indeed, it’s such a small drop that even if (say) 90% of scientific misconduct in NSF-funded research goes undetected, it’s still a drop in the bucket for NSF’s budget.* But that doesn’t make the cases of scientific misconduct detected by OIG any less terrible. You read the summaries of some of these cases, and you’re just appalled, both by the misconduct and by the bizarre excuses for it. What is wrong with some people? An NSF-funded postdoc sabotaged a grad student’s cell cultures with decontaminant, then admitted it but tried to rationalize his actions?! An NSF Program Officer plagiarized a rejected grant proposal?! A PI plagiarized a grant proposal and blamed the plagiarism on a postdoc who didn’t actually exist?! A PI claimed his fake data were given to him by a colleague who doesn’t actually exist, at a conference he didn’t actually attend?! After reading some of these, it was almost a relief to read the one about a department chair who used his university purchasing card at strip clubs. I felt like, thank goodness that the dumbest, most cliched form of embezzlement is still a thing. It restored my faith that the world is comprehensible and that I have a basic grasp of its workings.**

I don’t draw any grand conclusions from any of this. OIG reports address only a limited, non-random sample of scientific misconduct writ large. And I don’t know nearly enough to have any opinions about how good OIG is at its job, or whether it should operate differently, or anything like that. I just found it interesting to learn a bit about what sorts of allegations get brought to OIG, and what becomes of them.

*The amount of money involved sounds like a lot from the perspective of an individual researcher, because it is. When you hear that somebody faked the data on, say, a million dollar grant, your natural reaction is something like, “A million dollars?! What a colossal disaster! Think of all the good science you could buy with a million dollars!” But the perspective of an individual researcher is the wrong perspective from which to think about the damage done by scientific fraud. Yes, you personally could do a lot more good science if you had an extra million dollars–but you’re just one scientist among many, and your science is a drop in the bucket in the context of science as a whole. Instead, think of it this way: if Congress cut NSF’s $8+ billion dollar budget by one million dollars, would you think that was a colossal disaster for science? No, you wouldn’t. (At least, I assume and hope you wouldn’t.) Which isn’t to minimize the badness of scientific misconduct. Scientific misconduct is bad! It’s fine to be outraged when someone commits scientific misconduct! But you should be outraged because scientific misconduct is so contrary to the core principles of science (and because of the damage it does to the collaborators and trainees of those who commit scientific misconduct). Not because misconduct does appreciable damage to “science as a whole”, or wastes some appreciable fraction of all science funding. Here are some further thoughts on this in a slightly different context.

**Do not take this as your cue to tell me about old cases of scientists treating Sabotage and They Might Be Giants songs as instruction manuals. I do not want to know that scientific fraudsters back in the day were just as pathetically bad as some of today’s fraudsters, and their excuses just as laughable. I want to keep my comfortable illusion that, back in the day, scientific fraudsters all showed some creativity and ambition.