March 13th, 2007

Disappearing R01s


Geoff Davis

Success rates for NIH R01 grants have been falling fast over the last few years, to the great dismay of life scientists. Rates peaked at 32% in 1999, then fell to 23% in 2005.

In an article in Science, Elias Zerhouni blamed increasing numbers of grant applications: the number of applications for R01s increased by 39% between 1997 and 2005.

The demand-side story is only half the picture, though. While poring over some NIH data in preparation for my talk at NBER this week, I noticed an interesting thing: the number of R01s being awarded has been dropping rapidly since the doubling. Take a look:

Number of R01 awards

The number of new new R01s in 2007 is down from by more than 25% from the peak in 2003. (Note that all figures before 2007 come from NIH; the 2007 number of applications has been extrapolated, and the number of awards comes from CRISP. For this discussion, I’m going to restrict my attention to new awards; something similar has happened with renewals)

The story out of NIH is that there are problems because of flat budgets. That’s true, but the overall decrease in the budget in real terms has been relatively modest: the overall budget is down 6.6% in real terms. Moreover, as of this year, NIH has stopped giving automatic inflationary increases for multi-year grants. If all grant amounts stay the same, a 0% change in the budget should have no effect on the numbers. The overall budget is down from its 2003-2004 peak, but it’s still substantially greater than it was in 2002. Basically, a little less than half of the increase in the last year of the doubling has been eroded. Why has such a modest budget decrease resulted in such a dramatic decrease in the number of awards?

Scientists are complaining that funds have been diverted from R01s into other projects. This is also true: R01s received 60% of the overall grant budget in 1997; in 2005 the R01 share was down to 52.5%. This decrease took place slowly over the course of the doubling, however, and at least as of 2005 had not resulted in any decrease in the total amount of funding for R01s. So conceivably, the number of R01s could have increased even more, but again, there is no reason for the number of awards to have decreased.

What happened, I think, is more interesting. I believe that the real problem is independent of these essentially external events: the problem is built in to the system, and a contraction would have occurred regardless of the tax and allocation issues.

Here’s the story: The doubling consisted of a series of ~15% budget increases followed by basically flat budgets. (Let’s forget about inflation for now.)

In year 1, NIH gets a huge new influx of money into the R01 program. R01s last 3-5 years, mostly closer to 5 (average = 4.3 years), so they can only modify the 20% or so of grants that roll over each year. R01 budgets did not increase dramatically, so there was money left over that could be used to expand the number of awards. Similar things happen in years 2 and 3.

Toward the middle of the doubling, the costs of the higher-budget awards created in the first couple of years start to eat up more and more of the budget surplus, so the number of awards can’t be increased, and numbers start to level out. The amounts of the awards keep increasing, though.

Finally, at the end of the doubling, things max out.

New R01 amounts

Here’s the key observation: if one does not let new R01 budgets decrease from year to year, then increasing the size of R01s in one year locks one into budget increases for the next 4 years as older, less expensive R01s roll over and are replaced with newer, more expensive R01s. Thus, an increase in the average new R01 budget in 2003 locks the NIH into budget increases through 2007. Budgets since 2003 have been flat or declining, so propagating the 2003 (and 2002 and 2001, etc) increase through successive cohorts has forced numbers of grants to be cut.

In 1997, before the doubling, there were ~5 cohorts of R01s with an average of 3244 awards in each cohort and an average budget of $208,000 per year.

In 2003, at the end of the doubling, there were 4569 new R01s with an average budget of $351,000 per year.

If the 2003 numbers were to have been sustained, we would have seen the number of awards increase by 41% and their size increase by 69%. The trouble is that the budget increased only by a factor of 2, and 1.41 * 1.69 = 2.4 > 2. Basically, the gradual expansion of the R01 budgets appears to have led to more grants being created than could be sustained.

Ideally one would adopt a pay-as-you-go kind of system, where R01 numbers and budgets are not increased in any year without enough surplus in the budget (or guaranteed budget increases) to apply a similar increase to all cohorts in future years. The budget increases in the early years of the doubling appear to have been more than the NIH needed to expand the R01 program at a sustainable level, so it grew too big. The NIH can’t save money from year to year, so it wasn’t unreasonable for them to have spent the surplus growing the program. Perhaps it would have been better to have put the extra funds into a pool of shorter, transitional grants that were clearly marked for termination after a few years?

(5) Comments

  1. Victoria McGovern - March 14, 07 at 9:58 am

    On the other hand, buying stuff gets more expensive. The “real” budget drops in your pay-as-you-go model. Not so bad if you buy pencils and maybe a new pencil sharpener every twenty years, but a disaster if you buy animals and fresh chow and other things that are heavily impacted by the rising costs of freight.

    The real value of the “if you do good work, you can usually stay funded” approach is that a researcher can focus on the long view. If there were a pool of short, openly transient grants, I think they’d have to be earmarked for high risk projects that really have the potential to change the long view. But the human tendency, I think– from both the researcher side and from the peer review side– would be to support “sure bets” so after most grants one can say “See? I accomplished something!” rather than to use this kind of grant to let smart people take a flyer.

    So in a very unsure system, how do you stimulate people to take risks?

  2. Geoff Davis - March 14, 07 at 10:53 am

    A slight tweak to pay-as-you-go is to only increase the amount of an R01 when you have enough in the budget to increase subsequent cohorts by the same amount plus some inflation factor. We don’t know future inflation, but NIH has been using 3% as a rough guesstimate, and that’s probably a reasonable guess for now. So that’s not a big deal.

    It’s not clear what to do with the surplus in a pay-as-you-go scheme. R21s are a possible model for short-term stuff. Or you could try to pay for infrastructure or training or things that don’t necessarily create future needs.

    During my talk, I learned that public universities have been very creative in creating financial vehicles that allow them to carry money forward from year to year. Conceivably one could hand chunks of cash to universities with some stipulation that they spread it out using an appropriate schedule. In any case, surpluses like that would only a problem during times of sustained increase, and I don’t foresee anything like that any time soon.

  3. JS - March 15, 07 at 9:54 pm

    If your analysis is accurate, it reinforces the primary impression I had from the Zerhouni article — why are people with such a poor grasp of budget planning in charge of NIH? It’s one thing to fail to anticipate the unexpected, but can’t they anticipate the obvious?

  4. Bob - March 21, 07 at 1:06 am

    Here is a related article. Note the “sky-is-falling” presentation to Congress yesterday. There is an interesting contrast between reality, and the version Congress hears.


    Looming crisis’ from NIH budget

    Four years of flat funding causing major shifts in US biomedical research, university officials and senior scientists warn Congress

    Ted Agres

    [Published 20th March 2007 02:53 PM GMT]

    The “stagnated” budget for the National Institutes of Health (NIH), now entering its 4th straight year of flat-funding, is creating a “looming crisis” that is forcing scientists to downsize labs and abandon innovative work, and alienating the next generation of young researchers, a panel of university officials and senior researchers told Congress yesterday (March 19).

    “Promising research is now being slowed or halted,” said Edward Miller, dean of Johns Hopkins Medicine. “We are seeing veteran scientists spending time not in labs but on the fundraising circuit. We are seeing young researchers quitting academic research in frustration, having concluded that their chances of having innovative research funded by NIH are slim to none,” Miller told a Capitol Hill news conference yesterday.

    The scientists released a report prepared by 20 leading researchers from a consortium of nine academic institutions and universities, that outlines the benefits of increased NIH funding on biomedical innovations, and warns of the negative implications should the present budget be left unaddressed. The report cited threats from unexpected new diseases, such as SARS and pandemic influenza, as well as obesity, HIV, and bioterrorism.

    While Congress and the White House doubled NIH’s budget from 1998 to 2003, funding has failed to keep pace with inflation. NIH’s budget has hovered at around $28 billion, but once inflation is factored in, its purchasing power has fallen 13% over the past four years. According to the report, an average of eight out of ten NIH grant applications currently go unfunded, while at the National Cancer Institute, only 11 percent of grants are funded. “This is a recipe for disaster,” Miller said. “The number of termination letters at Johns Hopkins is up three-fold.”

    The report was released following a hearing of the Senate Appropriations Subcommittee on Labor, Health & Human Services, and Education, during which NIH Director Elias A. Zerhouni acknowledged the budget difficulties. “To support our vision and initiatives in the current budget environment we have made difficult but strategic decisions,” Zerhouni said. These include holding the average cost for competing grants to Fiscal 2007 levels and not providing inflationary cost increases for non-competing grants. (Zerhouni also sharply diverged from Administration policy yesterday when questioned about expanding Federal funding for human embryonic stem cells beyond the currently authorized 21 cell lines, saying US scientists and citizens would benefit from access to more lines.)

    Stephen M. Strittmatter, a professor of neurology and neurobiology at Yale University’s School of Medicine, told legislators that his laboratory’s discovery of the NogoReceptor molecule occurred during NIH’s budget-doubling period when he and other researchers were more willing to take risks. Today, he said, “researchers shy away from real discoveries. They’ve become worriers, not explorers.”

    Robert Siliciano, an infectious disease expert at Johns Hopkins University School of Medicine, told the Senate panel the reduction in NIH grants has forced him to scale back on promising research into optimizing antiretroviral therapies. “Typically, in the past, I would spend about 30 percent of my time applying for grants; now about 60 percent of my time is spent preparing applications,” he said.

    Postdocs and even undergraduates are observing the frustration felt by many lab directors, said John Carethers, chief of the gastroenterology division at the University of California San Diego. “I try to be a face out there for young researchers, but they see worry lines on my face,” Carethers told The Scientist, “If I falter, how are they going to continue?”

    Ted Agres

    Links within this article

    T. Agres, “Flat NIH funding again in ’08″, The Scientist, February 6, 2007

    Edward Miller

    Within Our Grasp – Or Slipping Away? Assuring a New Era of
    Scientific and Medical Progress

  5. Geoff Davis - March 21, 07 at 4:52 am

    Nicely spotted – thanks, Bob!