What I learned about scientific misconduct from reading the NSF OIG’s semiannual reports

The National Science Foundation’s Office of the Inspector General (NSF OIG) investigates misuse of NSF funds by individuals and organizations that receive awards from or conduct business with NSF. That includes, but isn’t limited to, researchers who commit scientific misconduct in the course of NSF-funded research. The OIG reports on its activities to Congress every 6 months. The reports are public, you can read them online. As part of my ongoing efforts to educate myself about scientific misconduct, I skimmed the five most recent OIG reports. Here’s what I learned.

  • Misspending and fraud detected by NSF OIG amounts to something like ~0.1% of the NSF budget. The headline number of each semiannual report is the amount of money OIG saved taxpayers. It includes recoveries and penalties paid by individuals and institutions found to have misused funds, and “questioned costs”. In a typical recent year, NSF OIG reports something like $8-10 million in savings to taxpayers. Sometimes more, sometimes less, but that’s the ballpark figure. That’s as compared to the NSF annual budget (all of it, not just grants) of a bit more than $8 billion in recent years.
  • Most of the misspending and fraud detected by NSF OIG does not involve scientific misconduct. That is, most of the taxpayer savings NSF OIG reports don’t come from cases of NSF-funded researchers fabricating data, falsifying results, or committing plagiarism. Rather, the bulk of the taxpayer savings seem to come from OIG busting misspending and embezzlement, mostly by institutions and private businesses but some by academic researchers. For instance, a university misspending research grant funds to pay teaching assistants for many years. A small businessman defrauding an NSF technology transfer program to pay his wife a bunch of money. A company defrauding NSF by billing twice for the same work. Etc.
  • The large majority of allegations of scientific misconduct to OIG involve plagiarism. This semi-annual report includes helpful tables (Tables 3-5) breaking down all allegations of scientific misconduct received, and investigated, by OIG from 2010-2019, and how those investigations were resolved. Almost 80% of allegations of scientific misconduct received by OIG involve plagiarism; only ~20% involve fabrication or falsification of data. The same is true for investigated allegations: OIG investigates about 80% of allegations of scientific misconduct it receives, and ~80% of those investigated allegations involve plagiarism.
  • Scientific misconduct allegations to OIG have dropped a lot since 2013. The same tables linked to in the previous bullet indicate that, from 2010-2013, OIG received over 100 allegations of scientific misconduct annually, maxing out at 117. Allegations are way down since then–below 100 every year, and below 60 every year but one. In 2019 there were only 33 allegations. Not really sure what to make of that. Maybe allegations of fraud were higher in the wake of the ARRA stimulus money NSF had to spend quickly during the Great Recession, and have since fallen back to “normal” levels? Any ideas?
  • OIG finds scientific misconduct in ~24% of the allegations it investigates. That number is calculated from Tables 3-5 in the report linked to above. I am surprised it is that low. I would be interested to see a breakdown of the reasons why misconduct isn’t found, because there could be various reasons. Maybe the alleged misconduct doesn’t qualify as “misconduct” under NSF’s definition. Maybe evidence was lost or destroyed. Maybe the allegation was found to be baseless or malicious. Etc.
  • As allegations of scientific misconduct to OIG have dropped, a greater fraction of allegations have been substantiated. Sticking with Tables 3-5 in the report linked to above: from 2010-13, OIG found scientific misconduct in 14-20% of its investigations (the exact percentage varied a bit from year to year). Since then, it’s only dropped below 30% once, and has been as high as 54%. No idea what to make of that. Maybe whistleblowers who have suspicions, but lack cut-and-dried evidence, are more hesitant to go to OIG than they used to be, perhaps because they’ve come to doubt that going to OIG would help? Maybe, with fewer allegations of scientific misconduct, and thus fewer investigations, OIG can devote more resources to each investigation? Maybe scientific misconduct is taken more seriously these days than was the case 10 years ago, so the institutional investigations that often inform OIG’s own decision-making are improving in thoroughness? I dunno, just spitballin’.
  • There’s some terrible scientific misconduct out there, and some bizarre excuses for it (even if it doesn’t cost NSF much money in the grand scheme of things). The upshot of the first two bullets is that scientific misconduct detected by OIG is a drop in the bucket for NSF’s budget. Indeed, it’s such a small drop that even if (say) 90% of scientific misconduct in NSF-funded research goes undetected, it’s still a drop in the bucket for NSF’s budget.* But that doesn’t make the cases of scientific misconduct detected by OIG any less terrible. You read the summaries of some of these cases, and you’re just appalled, both by the misconduct and by the bizarre excuses for it. What is wrong with some people? An NSF-funded postdoc sabotaged a grad student’s cell cultures with decontaminant, then admitted it but tried to rationalize his actions?! An NSF Program Officer plagiarized a rejected grant proposal?! A PI plagiarized a grant proposal and blamed the plagiarism on a postdoc who didn’t actually exist?! A PI claimed his fake data were given to him by a colleague who doesn’t actually exist, at a conference he didn’t actually attend?! After reading some of these, it was almost a relief to read the one about a department chair who used his university purchasing card at strip clubs. I felt like, thank goodness that the dumbest, most cliched form of embezzlement is still a thing. It restored my faith that the world is comprehensible and that I have a basic grasp of its workings.**

I don’t draw any grand conclusions from any of this. OIG reports address only a limited, non-random sample of scientific misconduct writ large. And I don’t know nearly enough to have any opinions about how good OIG is at its job, or whether it should operate differently, or anything like that. I just found it interesting to learn a bit about what sorts of allegations get brought to OIG, and what becomes of them.

*The amount of money involved sounds like a lot from the perspective of an individual researcher, because it is. When you hear that somebody faked the data on, say, a million dollar grant, your natural reaction is something like, “A million dollars?! What a colossal disaster! Think of all the good science you could buy with a million dollars!” But the perspective of an individual researcher is the wrong perspective from which to think about the damage done by scientific fraud. Yes, you personally could do a lot more good science if you had an extra million dollars–but you’re just one scientist among many, and your science is a drop in the bucket in the context of science as a whole. Instead, think of it this way: if Congress cut NSF’s $8+ billion dollar budget by one million dollars, would you think that was a colossal disaster for science? No, you wouldn’t. (At least, I assume and hope you wouldn’t.) Which isn’t to minimize the badness of scientific misconduct. Scientific misconduct is bad! It’s fine to be outraged when someone commits scientific misconduct! But you should be outraged because scientific misconduct is so contrary to the core principles of science (and because of the damage it does to the collaborators and trainees of those who commit scientific misconduct). Not because misconduct does appreciable damage to “science as a whole”, or wastes some appreciable fraction of all science funding. Here are some further thoughts on this in a slightly different context.

**Do not take this as your cue to tell me about old cases of scientists treating Sabotage and They Might Be Giants songs as instruction manuals. I do not want to know that scientific fraudsters back in the day were just as pathetically bad as some of today’s fraudsters, and their excuses just as laughable. I want to keep my comfortable illusion that, back in the day, scientific fraudsters all showed some creativity and ambition.

11 thoughts on “What I learned about scientific misconduct from reading the NSF OIG’s semiannual reports

  1. I do think the concern about damage to “science as a whole” has some merit — the most brazen and long-running cases of misconduct tend to get more media attention (and occasionally, attention from the political sphere), doing actual damage to the public perception of the craft. How much damage? No idea. I’d place it closer to the “contrary to core principles” end of concerns than “some money has been wasted” end, though.

    • Hmm. I doubt that even the highest profile cases of scientific fraud do much damage to the public perception of science or scientists.

      My understanding of public polling data is that scientists are regarded fairly well, and that there aren’t any dips associated with high profile cases of scientific misconduct.

      Which makes sense, I think. Even the most widely publicized scientific fraudsters aren’t all *that* famous to the general public. I mean, if you polled people asking them who (say) Diederik Stapel is, basically no one would know. Even someone like Elizabeth Holmes, who’s *way* more famous than Stapel (and who’s arguably not really a “scientific” fraudster), probably isn’t known to the majority of US adults. And her example makes people think badly of *her*, not of science or scientists (or Silicon Valley) in general.

  2. We all have favorites, sometimes because they are close to home. Over 40 yrs ago NSF (dont recall the panel) sent me a proposal to review. It was to study some aspect of predation/foraging; frankly it was a mediocre proposal. BUT, surprizing to me one 10 page section was lifted straight from my phd thesis. They did not even retype it, just made a photocopy. It was not cited as being from the thesis. It was my in-depth overview of the literature in foraging theory ca 1973. It was put forth as being THEIR overview. Imagine my further surprize when I phoned the program officer, and was told that I should ignore the plagiarism because, heck, it was not really a publication, and just judge the proposal on the rest!

    • What did I say about not wanting to know that scientific misconduct was just as pathetically bad back in the day? 🙂

      In seriousness, that is indeed quite the story!

      It’s also a small bit of evidence that times have changed. Today, I don’t think the NSF program officer would react that way. As evidenced by the fact that the OIG has sanctioned a number of researchers for plagiarism in research proposals recently.

  3. The misconduct cases summarized in the post illustrate why I’ve given up on understanding what goes on in the heads of some people who commit scientific misconduct. If you are *inventing non-existent people* in order to blame them for your misconduct, you are not doing a rational calculation of the risks and rewards of misconduct. Nor are you just a lazy person. Nor are you just someone who’s over-attached to his own scientific ideas. Nor are you just someone who’s feeling a lot of pressure to get grants and publish papers. Whatever’s motivating you, I can’t grok it.

    • Hip Hip Hoorah, Jeremy! I could not have said it better myself! Once, many moons ago, I had entire paragraphs copied from one of my published manuscripts. When I subtly brought it to the offender’s attention, I was told I had “plagiarized myself”… When I brought it to the attention of the offender’s institution, I was told it was “an honest mistake”…

      I agree with you that this goes beyond being “just a lazy person”. In fact, I believe it results from a combination of a deeply flawed character and a heavy dose of denial. That somewhere along the line involving their childhood development, there was a role model who suffered from the same flaws. But who can say, really.

      I found the whole ordeal to be chock full of the off the wall insanity, as the offender simply churned out one useless excuse after another for their ethical deficiencies.

  4. It would be easier to read your text if all of it were written in the standard dark black color – the black of your title and topic headings are what I would appreciate having. On the grayscale, I put the body of your text at about 7 or 8 steps off of this value. Faded gray does not work…(for me).

  5. One number that’d be interesting for comparison with the 8-10 millions/year is the yearly budget of the OIG within NSF. There is of course also moral value in uprooting waste and malpractice, but if the two numbers would be vastly different (one way or the other) that might be an argument for NSF to adjust how much money it spends on its OIG.

    • I don’t know how much NSF spends on OIG. I’d be very surprised if it was as much as, say, $100M annually. Even if OIG costs more than $8-10M annually, I suspect it’s still worth it even on purely financial grounds. Because OIG likely has some deterrent value. If OIG didn’t exist, fraud against NSF would probably increase.

      OIG has four divisions. Even if each division has just a few employees, you’d probably talking about something like at least $1M annually in salary and benefits alone. So I doubt that the annual cost of OIG is an order of magnitude less than its typical annual recovery for taxpayers. And even if OIG does cost less than its recoveries, would that still be the case if you gave it a lot more resources? I don’t know. Just offhand, I kinda doubt it, but that’s a wild guess based on not much.

  6. Pingback: Friday links: the perfect scientific crime, pollinator action movie, and more | Dynamic Ecology

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.