Engineering Science - NIH Crisis

Fixing the NIH grant-making process

2007-12-07T18:03:00Z

There's a piece in this week's Chronicle about some possible changes in the NIH's grant-making process. About 6 months ago, the NIH solicited suggestions from the general public for ways to improve the process, and an advisory committee has been sifting through the thousands of ideas they received.

The article describes a few broad classes of ideas:

Streamlining the application process by reducing the length of grants from 25 pages to 15
Limiting the number of proposals a person can submit
Basing funding decisions more on an individual's than on specifics of their proposal
Providing more affirmative action for younger scientists

These ideas aren't yet official recommendations - those won't be out until later this week - but they are likely indicative of the kinds of things the NIH will actually do. Many of these ideas are good ones; I'm just not convinced they will have the hoped-for effects.

Reducing the amount of effort required to submit a proposal sounds great. People invest huge amounts of effort on their proposals; I'd much rather have them spending their time doing science than chasing money. The trouble is that making it easier to submit a grant, will probably mean that people will submit more grants, driving the success rate down even more. Going back to the lottery ticket/grant analogy: during the budget doubling, the NIH increased both the odds that a ticket would win and the amount of money paid out by a winning ticket. Not surprisingly, people bought a lot more tickets. Streamlining the proposal process, while a worthy goal, effectively cuts the price of a ticket, which again increases the net payoff. If this happens, I predict we'll see even lower success rates in the future.

Limiting the number of proposals someone can submit is a non-starter, I suspect, despite the AAMC's endorsement of the idea. The idea has some merits: it would probably reduce the number of proposals the NIH receives and force people to submit only their best ideas. However, I think that there are legitimate scientific reasons for some larger labs to be submitting multiple proposals per year. Zerhouni opposes the idea. A better alternative might be to impose a surcharge, like publication fees charged by journals, to cover review costs on proposals after the first. This would reduce the number of people submitting multiple proposals while still making it possible to do so.

Judging proposals on the reputation of the submitter rather than on their content is a recipe for all sorts of trouble. Sure, it would make life more convenient for the elites, but I suspect that the result would be complacency, not greater willingness to take risks - just throw something over the fence and you get your funding, so why make the effort? Younger scientists are a source of a lot of crazy new ideas, but they don't have much of a track record, so this kind of scheme could shut them out of funding even more than they already are.

There's an alternative approach that I think avoids most of these difficulties - more next time.

Universities and the money fix

2007-09-24T04:17:00Z

Nature has been running some good stories this past month on the mess at the NIH.

Universities and the money fix, by Brian C. Martinson, points out what I think is the central problem:

[L]argely because of the structure of the funding flows between the NIH and the universities, there are few checks in the system to keep competition for grant funding at a healthy level. Thus, calls for further increases in the NIH budget may only make matters worse. In my view, it is time to ask the biggest beneficiaries of NIH largesse — the universities and academic health centres — to find ways to balance supply and demand that better reflect their obligations to researchers and society....

(emphasis added) Exactly right. The current problems are structural in nature. Anything that fails to address the underlying issues can only serve at best as a temporary stopgap.

There are insufficient 'feedback loops' linking the production of biomedical researchers to the availability of resources to support them. Instead, the educational system is replete with incentives to generate ever more PhDs and medical doctors. In the short term these arrangements may benefit universities, but in the longer term, such extreme levels of competition for funding are unsustainable. And they may already be doing harm.

The harm Martinson sees is greater potential for ethical lapses, something also predicted by David Goodstein. Reader BioScientist has more to say about the potential for harm in a comment on the NSF's new postdoc mentoring requirement:

None of these proposals matter, nor will any changes or improvements to them. The bottom line is that as long as there are too many scientists present the competition to stay alive will be intense and therefore conditions will remain poor. NIH/NSF proposals like this attempt to legislate behavior without recognizing the realities on the ground.

The lack of attention to graduate/post-graduate training is borne from two areas: 1) PIs are pushed by the system to extract every bit of effort possible effort from employees. The penalty for not doing so is a loss of funding and the end of a career. 2) As long as there are multiple applicants for every scientist/PI position there's less need to insure development of most individuals. As an example, one can simply ignore the bottom sixty percent of CVs to no ill effect when considering tenure-track positions. Those that remain will no doubt be pretty impressive. In this respect, post-graduate training is less an education than it is a 'selection' process in the biological sense. Put another way: why bother training postdocs, when most of them will fall out of the "system" anyway? The best will claw their way to success on their own and the rest are irrelevant. (This is a sentiment I've heard multiple times from faculty at my current Tier-I, top-20 research institute.)

Until the supply of PhD scientists comes back into line with demand working conditions will remain poor, salaries low, and hours long. In absence of a solution to the supply/demand imbalance all attempts to solve the resultant phenotypes will fail.

While I agree with Martinson's diagnosis, he unfortunately doesn't offer much in the way of solutions:

So is the only solution to force long-time NIH grant getters into retirement? Perhaps not. Universities have benefited handsomely from the efforts of senior faculty members in securing NIH grants during their careers, perhaps those same universities could now return the favour by taking full responsibility for paying these faculty salaries in their later years. This would serve the dual purpose of getting them off the NIH dole, and encouraging them to share their knowledge with their younger colleagues through more teaching.

Getting enough senior scientists to give up research for teaching to make any kind of difference seems, um, implausible.

Employment Trends in Biomedical Sciences

2007-08-22T21:20:00Z

Ginny C just pointed me to a recent FASEB presentation that summarizes recent trends in the life sciences labor market. It's great that they have done this, since I suspect a lot of people don't know the big picture, and FASEB has a very broad reach. Give it a read.

There is a great deal of overlap with Paula Stephan's findings and a few other things that Peter and I have discussed here.

A few things that struck me in the slides:

I knew that numbers of women have been increasing rapidly in the life sciences, but the graphs in the presentation are still pretty striking. Ditto for the number of postdocs on temporary visas.
Success rates for NIH fellowship applications have been falling almost as fast as for R01s. They're down from ~45% in 2001 to ~27% in 2006.
NIH spending on students as a percent of the total budget is down from ~4.3% in 1985 to ~2.7% in 2006
Foreign PhD recipients are increasingly staying in the US
The fraction of all US biomedical PhDs who are tenured or in tenure-track positions is steadily decreasing. ~46% in 1981 to ~28% in 2006
Almost all the new positions created during the NIH doubling period were MDs in clinical departments
Hiring of PhDs by med schools has pretty much ground to a halt in the last couple of years.
Average GRE Quantitative scores are surprisingly low for life sciences folks: 529 for health sciences and 606 for biological sciences applicants (out of 990 total). I have always wondered if part of the reason the labor market for life scientists is so much worse than for physical scientists and engineers is that quantitative skills give the latter folks more options.

Survival of the "Fittest"?

2007-08-09T00:17:00Z

There's an intriguing article in yesterday's Times about a new theory about the factors that gave rise to the Industrial Revolution in England.

For centuries, England's citizens lived on the brink of starvation. Although innovations would periodically increase agricultural productivity, greater access to food invariably led to population increases, which in turn brought per capita food levels right back to where they started. It took the Industrial Revolution to finally bring the growth rate of the food supply above the growth rate of the population.

Historian Gregory Clark's study of wills from 1200-1800 found the following:

Given that the English economy operated under Malthusian constraints, might it not have responded in some way to the forces of natural selection that Darwin had divined would flourish in such conditions? Dr. Clark started to wonder whether natural selection had indeed changed the nature of the population in some way and, if so, whether this might be the missing explanation for the Industrial Revolution....

Generation after generation, the rich had more surviving children than the poor, his research showed. That meant there must have been constant downward social mobility as the poor failed to reproduce themselves and the progeny of the rich took over their occupations. “The modern population of the English is largely descended from the economic upper classes of the Middle Ages,” he concluded.

As the progeny of the rich pervaded all levels of society, Dr. Clark considered, the behaviors that made for wealth could have spread with them. He has documented that several aspects of what might now be called middle-class values changed significantly from the days of hunter gatherer societies to 1800. Work hours increased, literacy and numeracy rose, and the level of interpersonal violence dropped.

Clark speculates that there may be genetic and/or cultural components to these changes in behavior:

Dr. Clark says the middle-class values needed for productivity could have been transmitted either culturally or genetically. But in some passages, he seems to lean toward evolution as the explanation. “Through the long agrarian passage leading up to the Industrial Revolution, man was becoming biologically more adapted to the modern economic world,” he writes. And, “The triumph of capitalism in the modern world thus may lie as much in our genes as in ideology or rationality.”

What was being inherited, in his view, was not greater intelligence — being a hunter in a foraging society requires considerably greater skill than the repetitive actions of an agricultural laborer. Rather, it was “a repertoire of skills and dispositions that were very different from those of the pre-agrarian world.”

I don't know enough about behavioral genetics to have a sense of whether this is plausible; regardless, his application of Darwinian thinking in this particular case is intriguing.

Substitute funding for food, and it's clear that the current NIH mess is a Malthusian crisis. And as with England, this is only the most recent of a series. What are these selection pressures doing to the population of academic scientists?

When times are tight, it becomes a lot less pleasant to be an academic. Fewer proposals get funded, and even if you do get funded, a lot more work has to go into your proposals. Industry starts looking a lot more attractive by comparison. Increased numbers are forced out, and more interestingly when we start thinking like Darwin, increased numbers either leave by choice or never seek academic careers in the first place.

Economics tells us that the more attractive one's industry prospects relative to academic alternatives, the more likely one is to end up there. And if you work in industry, you don't "reproduce" by training students. Thus, academia's Malthusian crises may very well be selecting against those who are most capable of success outside of academia.

Funding levels in academia are driven by the prospect of economic returns to investments in research. Unfortunately for all concerned, unlike Clark's hypothesized England, academia appears to be selecting against some of those most capable of increasing the "food" supply.

What have they been up to with all that money?

2007-07-20T20:53:00Z

While the NIH budget doubling has created a crisis for life sciences grant applicants, what about its benefits? One of the most pronounced effects of the budget doubling was a huge building spree by medical schools. Presumably there has also been an increase in research output, right?

A new NSF report makes me wonder. According to the press release, "the number of U.S. science and engineering (S&E) articles in major peer-reviewed journals flattened in the 1990s, after more than two decades of growth.... Flattening occurred in nearly all U.S. research disciplines and types of institutions."

I haven't read the full report yet, so I'm still scratching my head. Here's the report, and here's a discussion on Slashdot.

Risky Business

2007-07-06T20:55:00Z

The Chronicle reports that the National Science Board (the overseers of the NSF) are urging the NSF to fund riskier projects with the potential for big breakthroughs rather than safer, more incremental projects. (Here's the NSB report)

Taking more risks is likely to be a good thing for NSF. Consider the financial analog: it's like holding nothing but T-bills in your portfolio. Adding stocks (a riskier asset) to the mix in the right way can dramatically increase returns without greatly increasing risk.

I hope the goal of increased risk-taking on "transformative" research finds its way into the NIH, because if anything, the NIH is even more risk-averse than NSF. How much more can you do to avoid risk than the NIH's policy of (a) requiring in essence that people demonstrate that their projects will work before funding them (requiring "preliminary data") and (b) placing heavy emphasis on an established track record? These two policies are particularly hard on young people, who research suggests are more likely to come up with creative new ideas: because they are young, they don't have much of a track record, and and without an existing lab and funding in place, it's quite difficult to generate preliminary data. The result: less and less funding for young researchers.

The NIH is undergoing an examination of its own grantmaking processes as well in the wake of plummeting rates of grant success. Perhaps the National Science Board's report will shape their thinking?

The Yankees and Research Funding

2007-06-21T04:58:00Z

As I read Geoff’s posts about the University of Kansas Medical Center, and the seeming mismatch between what they are clearly expecting with regard to NIH funding and what the national NIH funding picture looks like I made me wonder: how could an institution pull off a major increase in NIH funding?

The answer is simple: hire some major research talent. I imagine that one could do a search on all NIH R01 grants and develop a ranked list of which individual investigators are getting the most money. Hire those guys and gals. Or, hire some of those Howard Hughes Fellows – those guys and gals are loaded.

In science, just as in baseball, the data are available to rank order the players by whatever criteria you want. If it’s money you want, you could assemble the “dream team.” But, like George Steinbrenner, you’re going to have to have a lot of money to do it.

The coupling of money and scientific talent has been known for a long time. Harvard, Princeton, Stanford and a handful of other schools have huge endowments and a [halo effect] (http://en.wikipedia.org/wiki/Halo_effect) that attracts more money in the form of alumni donations, private contributions, etc. Government money is also drawn to such honey pots because:

These schools have attracted top talent (both faculty AND students)
There often have better facilities
The “halo effect”

This is a self-re-enforcing mechanism that helps the rich schools get richer while the “also-rans” get to scurry around for the crumbs. And when NIH funding gets tight I would speculate that the relative decline in funding is greater for the second-tier schools than for the top 10.

The halo effect also works in reciprocal. Young hot-shot professors cannot help but experience a shiver of glee when they get a call from the President of MIT or Stanford. But the University of Kansas?

This is where shrewd marketing and recruiting comes in. The Dean of the Medical School of the University of Kansas cannot hope to compete on the same terms as Harvard or Caltech. But maybe there are other terms that might sway a young hot-shot. How about a cushy job for her spouse? Or maybe the potential for building a huge research empire may appeal? Some of the top research schools are at a significant disadvantage in a number of quality of life areas. Good luck buying a house as a UCSF professor (unless your wife happens to be an investment banker). And Caltech is nice but if you have to live in Glendale? Forget it! But you don’t want to offer too cushy an environment for your prospective hotshot – you want her to remain hungry and monomaniacally focused on bringing in the big bucks.

Kansas’s attempt to leap over its peers in terms of NIH funding will require some really state-of-the-art buildings. But it will rely even more on attracting the most successful researchers (with success measured not in teaching quality, contribution to the campus or nice looks but solely MONEY). There have been examples where a top Ivy science department has gotten complacent and has been displaced by a hungrier rival. If the Kansas State Legislature or the Kansas Congressional delegation want to help – good for them.

But don’t forget the simple truth of baseball: what sells tickets is the players, not the stadium.

Did they miss the memo? Follow up

2007-06-20T12:32:00Z

I sent a note to the reporter who wrote the story on the KU Medical Center expansion; here is his reply:

Geoff,

Thank you for your note, and for passing along the Science magazine article. I have written on this topic before as I increasingly am hearing from researchers about the intensifying competition for federal grants. So far, Kansas researchers appear to be defying the national trends. NIH funding at the KU Medical center rose 22% last year, which means, a top medical center official said today, that Kansas researchers are taking federal money away from labs in other states.

The university's ability to maintain this success with a vastly expanded program will be a very important issue to consider as the community is asked to support the endeavor. I will strive to draw more attention to the issue.

Best regards, Jason

Jason Gertzen The Kansas City Star

I was impressed by the recent growth in the KU Medical Center's funding, so I decided to dig in a bit. While I was up at NBER earlier this spring, I wrote some code to parse all the NIH's data on awards. Here's what I found about the University of Kansas Medical Center (hope I have the right place - I didn't see a Kansas University Medical Center):

The medical center did have impressive growth from 2005 to 2006 (11%). However, in the years before that, it stayed the same (2004-2005), fell by 6% (2003-2004), rose by 2%, etc. There has been growth over time, but the rate has fluctuated considerably from year to year. It's quite possible that the 11% gain last year was just random variation. (I'm not sure of the reason for the discrepancy between the growth rate Jason cites and what the NIH numbers show - the NIH data sets I'm using may be incomplete)

Over the last 5 years, the medical center's NIH funding grew by 15% overall, from $35M to $40M. That sounds impressive until you consider that total NIH grants grew by 26% over the same time period. To double their funding in the next 5 years, the medical center's funding will have to grow by 15% per year, roughly 5 times the current rate. Part of their growth over the last 5 years came from the tail end of the NIH's budget doubling. This time around, that boost won't be there. I'm sure the additional people will help; no doubt the new hires will be under tremendous pressure to bring in money.

I wish them well.

Did they miss the memo?

2007-06-19T16:37:00Z

Kansas University announced an ambitious $800 million initiative today to add 900,000 square feet of research space to the medical school and to hire 244 new researchers.

The plan is part of a broader effort to roughly double the medical school’s research grants, which totaled $88 million in 2006, over five years and then to double them again in five more, reaching at least $340 million in grants by the end of 2016.

I wonder if the med school administrators have been reading Science lately? KU is a bit late to the party! The NIH budget doubling inspired similar moves by dozens of major research universities over the last 10 years; all told, schools invested $11 billion over the last decade "expecting to recoup their investments from the NIH grants investigators would haul in." That plan hasn't worked out so well: there are now so many new researchers applying for NIH grants (to pay for all those new facilities) that it's harder than ever before to obtain funding.

To make matters worse, the NIH funded programs at unsustainable levels during the doubling (“We didn’t model [annual budget] increases below 4% a year because the tradeoffs and sacrifices that would have been caused…were too difficult for us to deal with in the model.”) and is now having to scale the number of R01 grants way back. There are now fewer R01s than there were before the doubling started.

I wish them well, and I hope they do a little reading before they spend too many of their millions.

“Boom and Bust” – April 20, 2007 article in Science

2007-05-02T06:15:00Z

Since this blog started in the Fall we’ve had an active dialog on the subject of funding at NIH. Science Magazine’s April 20 issue has a long article by Jennifer Cousin and Greg Miller detailing the issue, and confirming many of the trends and factors we have discussed on this blog. (Note that I think you have to be a AAAS member to see the entire article – if you aren’t, e-mail me at peterfiske@yahoo.com and I can perhaps get you a copy) The authors spoke to “dozens of investigators… [and] six NIH institute directors and agency head Elias Zerhouni” and confirmed what a lot of the readers of this blog are saying. Maybe Jennifer and Greg have been reading this blog all along?

In any event, the article is good and thorough and I thought it would be useful to provide a link to a pdf version that you can read – and to discuss this article as a group.

One issue that Geoff has speculated about – and that these authors confirm – is the fact that the doubling of NIH funding prompted a lot of infrastructural investment at universities and national labs in the US. According to the authors of this article, schools invested $2.2B in new medical school construction from 1990-1997, $3.9B from 1998-2002 and a whopping $7.4B from 2002-2007! Schools hired new faculty to fill the offices and were “expecting to recoup their investments from the NIH grants investigators would haul in.” Most ominously, the article notes that growth at medical schools has lagged the doubling of funding: “many institutions are still expanding”, they report.

Institutions have been confronted with huge increases in demand for bridge funding for their investigators. Dana Farber’s CEO told the authors that his institution was setting aside $3M-$4M for this year – historically the amount of money set aside for this category was a small fraction of this amount.

We have explored the growth in NIH grant applications in our blog – this article confirms the trends Geoff and others have reported. As we have discussed – investigators have reacted to the paucity of funding by trying to up their odds by submitting more proposals.

The result: misery among the investigatorial classes. Several young and mid-career investigators are interviewed and their lives over the past few years sound miserable. Ironically, NIH has tried to protect some of the youngest investigators by giving them a slight edge in funding for their first R01s – but this simply robs resources from the mid-career folks who are often struggling to get R01’s renewed. What a perfect time to get hosed by the NIH: just when you’re up for tenure review.

The authors talk a bit about the effect on early career scientists and graduate students. No data were presented but the authors describe the situation at Brown University: 49 faculty members were polled regarding how many were planning on taking a new graduate student next year: only 25% said they were – down from 90%. But Harold Varmus is quoted by the authors as saying “I don’t think we’re losing young people outright yet.”

My experience with Science magazine is that the journalism staff is first rate, but they tend to stick to the establishment line of thought. They shy away from perspectives that might be too critical of the powers-that-be, and often choose a “silver-lining” to close out the discussion. None of the anger that has been so eloquently expressed in this blog is evident in any of the quotes in the article. One might think that the catastrophic failure of the management of NIH, and of the science policy community as a whole, to prepare its community for this “day of reckoning” would elicit a bit more ‘what the F!@#!! were you thinking???’ comments from folks. According to the authors of this article, eight senior scientists and policy makers published a commentary in Science in 2002 presenting different budget models for NIH – the most pessimistic of which modeled increases of 4%. One of those involved is quoted: “We didn’t model increases below 4% a year because the tradeoffs and sacrifices that would have been caused…were too difficult for us to deal with in the model.” [!]. Director Zerhouni attributes the dive in funding rate to below inflation levels on 9/11, the wars and Katrina. No doubt they might have marginally affected the schedule of declines – but certainly not the inevitability: for any agency to assume long-term growth above the rate of inflation is… well, unsound at the very least…

Which gets us back to this blog. We started this discussion because, despite its abundant successes, “science” in the United States is far from perfect. The current crisis in funding at NIH is only the latest in a long history of similar crises in different disciplines since the advent of the “modern” science funding model proposed by Vannevar Bush. Even today, policy decisions on science funding almost entirely disregard issues related to the scientific workforce – and this makes young scientists particularly vulnerable.

"A Crisis is a Terrible Thing to Waste"

2007-04-06T16:37:00Z

Reader Eric has some strong words about the current situation at the NIH:

"As a scientist, it hurts me to say that I don't want academic scientists to get out of their recent NIH funding quagmire. Let them suffer painful contraction...they deserve it. Let's throw in a sudden cut-off of their cheap foreign labor pool. Add in a little backlash from junior scientists and declining American enrollments. Maybe the 'profession' could use a perfect storm to clean up its mess."

I can certainly sympathize with the sense of frustration at the huge mess that has been made with all the NIH's new resources. I don't think there is much danger that things will get better soon, either -- I think we will start seeing a lot of lab downsizing as bridge funds start to run out, and the job market for life scientists will probably take a turn for the worse as all the surge in new PhDs from the doubling start to graduate.

I recently saw a great quote from economist Paul Romer: "A crisis is a terrible thing to waste." There's nothing like being in a giant mess to focus one's mind on fixing things. How might the current NIH mess be used productively?

Here are a few ideas:

Planning - It's pretty clear that the current NIH situation has arisen from poor planning on the parts of both universities and the NIH. Universities built way more lab space than they can afford, and the NIH increased the number and size of its R01 grants in a way that was unsustainable. I think that both of these things could have been foreseen from the outset and possibly prevented if there had been some investment in forecasting the behavior of the parties involved.

The good news is that I am hearing things that make me think that some of those kinds of investments in modeling and forecasting may be in the works. I think the NIH would have a much stronger case in Congress if they went in and said, "We have done some extensive self-study on the budget doubling. Here's what went wrong. Here's how we know it won't happen again. Here's a low-cost way to fix the worst of the problem. Please help us," rather than the current approach, which comes across to me more like, "You stopped increasing our budget when you said you would! Now everything has gone to hell, and it's all your fault! You hate science! Give us more money or your children will be speaking Chinese!"
Training - I hear postdocs complaining all the time about how hard a time they are having finding jobs. Then this week I see this in the paper: Shortage of scientists threatens biotech boom. Um... there are about 30,000 life sciences postdocs running around who would be very excited to work for Genentech or Gilead, I am sure. Now perhaps part of this is a cynical ploy to get the H-1B visa cap raised so biotechs can keep labor costs down. But I wonder how much of it is arising from a genuine mismatch between how life sciences PhDs are trained and the needs of biotech companies? I have seen some very smart friends try to get industry jobs, and they've had a pretty rough time of it. The sense I have is that once you're in, it's not that difficult to make a lateral move to another company, but getting in is very difficult. What's missing from the PhD that these postdocs seem unworthy of hiring?

I have a neighbor who is doing a professional science master's degree at NC State. The program is doing some really smart things: he takes a mixture of business courses and science classes, and every summer he does an internship at a biotech company in the Triangle. When he graduates, he'll have an MBA, a master's in science, and a foot in the door in 2 or 3 biotech companies. How hard would it be to incorporate some of these elements into a PhD program? Or into a postdoc? It's not like everyone would have to do these things; if even a modest fraction of postdocs went this route, we'd end up with an amazing technology transfer mechanism and a ton of scientifically talented people with enough know-how to potentially start new companies. You want science to create new jobs and grow the economy? Then train some scientists to take part in the process!
Immigration - This is a longer conversation that I'll try to start next week. A few big issues that the current crisis might be used to change: (1) there doesn't seem to be much discussion in the scientific community about what the longer term ramifications of globalization will be for science in the US, and (2) the H-1B program looks broken both operationally and strategically.

"Looming Crisis" at NIH

2007-03-20T19:51:00Z

A couple of interesting signs of trouble at NIH:

First, Bob notes this piece in The Scientist:

'Looming crisis' from NIH budget: Four years of flat funding causing major shifts in US biomedical research, university officials and senior scientists warn Congress

Looks like NIH has dispatched some senior folks to go to Congress with hats in hand. Don't get me wrong - I think there is a crisis in the works, but I'm not convinced that more money is the way to fix the problem, or at least not with the current strategy. After all, it was more money that got NIH into the current fix.

Will this approach work? Just for kicks, try the following experiment:

Google the word "crisis". I get 125,000,000 web pages containing the word.
Now google the for words "crisis" and "education". I get 84,200,000 pages containing both "crisis" and "education".

Think about that: more than two thirds of all the pages containing the word "crisis" also contain the word "education"! The education sector seems rather prone to crises, no? It sounds to me like this approach is pretty well-worn.

Put yourself in the shoes of a congressperson. The NIH has just talked you out of an enormous budget increase a few years ago. Now they are coming to you with tales of impending crisis because you have not continued to increase their budget beyond what you have already given them. The other 99.99% of your constituents want more money to insure the uninsured, provide better K-12 education, house the poor, care for disabled veterans, pay down the deficit, properly fund Social Security, pay for the Iraq war, and lower taxes. What's your call?

Second, I learned of a new NIH Bridge Fund program.

Basically NIH has set aside a pot of money to tide over people who submit grants and just miss the cutoff. It's a rather big pot: $91 million to support "vulnerable research programs"! Sounds like NIH anticipates a lot of people really struggling.

I think this is going to get pretty ugly.

Disappearing R01s

2007-03-13T22:08:00Z

Success rates for NIH R01 grants have been falling fast over the last few years, to the great dismay of life scientists. Rates peaked at 32% in 1999, then fell to 23% in 2005.

In an article in Science, Elias Zerhouni blamed increasing numbers of grant applications: the number of applications for R01s increased by 39% between 1997 and 2005.

The demand-side story is only half the picture, though. While poring over some NIH data in preparation for my talk at NBER this week, I noticed an interesting thing: the number of R01s being awarded has been dropping rapidly since the doubling. Take a look:

The number of new new R01s in 2007 is down from by more than 25% from the peak in 2003. (Note that all figures before 2007 come from NIH; the 2007 number of applications has been extrapolated, and the number of awards comes from CRISP. For this discussion, I'm going to restrict my attention to new awards; something similar has happened with renewals)

The story out of NIH is that there are problems because of flat budgets. That's true, but the overall decrease in the budget in real terms has been relatively modest: the overall budget is down 6.6% in real terms. Moreover, as of this year, NIH has stopped giving automatic inflationary increases for multi-year grants. If all grant amounts stay the same, a 0% change in the budget should have no effect on the numbers. The overall budget is down from its 2003-2004 peak, but it's still substantially greater than it was in 2002. Basically, a little less than half of the increase in the last year of the doubling has been eroded. Why has such a modest budget decrease resulted in such a dramatic decrease in the number of awards?

Scientists are complaining that funds have been diverted from R01s into other projects. This is also true: R01s received 60% of the overall grant budget in 1997; in 2005 the R01 share was down to 52.5%. This decrease took place slowly over the course of the doubling, however, and at least as of 2005 had not resulted in any decrease in the total amount of funding for R01s. So conceivably, the number of R01s could have increased even more, but again, there is no reason for the number of awards to have decreased.

What happened, I think, is more interesting. I believe that the real problem is independent of these essentially external events: the problem is built in to the system, and a contraction would have occurred regardless of the tax and allocation issues.

Here's the story: The doubling consisted of a series of ~15% budget increases followed by basically flat budgets. (Let's forget about inflation for now.)

In year 1, NIH gets a huge new influx of money into the R01 program. R01s last 3-5 years, mostly closer to 5 (average = 4.3 years), so they can only modify the 20% or so of grants that roll over each year. R01 budgets did not increase dramatically, so there was money left over that could be used to expand the number of awards. Similar things happen in years 2 and 3.

Toward the middle of the doubling, the costs of the higher-budget awards created in the first couple of years start to eat up more and more of the budget surplus, so the number of awards can't be increased, and numbers start to level out. The amounts of the awards keep increasing, though.

Finally, at the end of the doubling, things max out.

Here's the key observation: if one does not let new R01 budgets decrease from year to year, then increasing the size of R01s in one year locks one into budget increases for the next 4 years as older, less expensive R01s roll over and are replaced with newer, more expensive R01s. Thus, an increase in the average new R01 budget in 2003 locks the NIH into budget increases through 2007. Budgets since 2003 have been flat or declining, so propagating the 2003 (and 2002 and 2001, etc) increase through successive cohorts has forced numbers of grants to be cut.

In 1997, before the doubling, there were ~5 cohorts of R01s with an average of 3244 awards in each cohort and an average budget of $208,000 per year.

In 2003, at the end of the doubling, there were 4569 new R01s with an average budget of $351,000 per year.

If the 2003 numbers were to have been sustained, we would have seen the number of awards increase by 41% and their size increase by 69%. The trouble is that the budget increased only by a factor of 2, and 1.41 * 1.69 = 2.4 > 2. Basically, the gradual expansion of the R01 budgets appears to have led to more grants being created than could be sustained.

Ideally one would adopt a pay-as-you-go kind of system, where R01 numbers and budgets are not increased in any year without enough surplus in the budget (or guaranteed budget increases) to apply a similar increase to all cohorts in future years. The budget increases in the early years of the doubling appear to have been more than the NIH needed to expand the R01 program at a sustainable level, so it grew too big. The NIH can't save money from year to year, so it wasn't unreasonable for them to have spent the surplus growing the program. Perhaps it would have been better to have put the extra funds into a pool of shorter, transitional grants that were clearly marked for termination after a few years?

Troubling Doubling

2007-03-03T00:30:00Z

Paula Stephan gave a great talk on the NIH doubling this week here at Harvard. Here are her slides.

To recap, shortly after the NIH's annual budget doubled from $14 to $28 billion, the number of new applications for R01 grants (the mainstay of life sciences research) increased dramatically, and as a result, acceptance rates for grants plummeted. Lots of people are upset about it.

One of the big questions is, who are all these new applicants? Paula presented evidence for the following scenario:

When the NIH announced their budget increase, a lot of institutions saw an opportunity to expand their life sciences divisions.
Universities started building new facilities, hoping to recoup their costs with the overhead from increased NIH grant money. Most of these new buildings were associated with medical schools.
Lots of new research space started coming online just as the doubling was winding down.
There has been an increase in hiring to fill all this new space. Most of the new hires have been MDs, not PhDs, and most of the positions are non-tenure track, soft money positions.
Now that the buildings have been completed, they have to be paid for. Both new and existing faculty are under increased pressure to bring in outside support. A typical arrangement: 3 years from the start of one's appointment to bring in one's full salary. The expectation in some cases is three R01s per investigator.
The increased pressure to bring in money is spurring everyone to submit more often. The percentage of applications from new investigators (people who have never received and R01) has remained roughly constant (~25%) throughout the doubling, so the new applications are coming from people across the board.

Paula also has some interesting data from the Survey of Doctorate Recipients about how the doubling affected careers for younger scientists. In brief:

The length of time people are spending in postdoctoral positions decreased modestly.
Job growth occurred. Most of the new jobs were outside of academia and in non-tenure-track academic positions.
The probability that a biomedical PhD holder aged 35 or younger has a tenure track job was the same in 2003 as it was in 1993.
New investigators (those who have never been a PI on an NIH grant before) receive more awards than before the doubling, but all the increase is in small grants (R03s and R21s). The total number of R01s awarded to new investigators has remained roughly constant.

The bottom line: $14 billion new dollars per year made things a little better for young scientists. When you consider that for $2.5 billion, you could double the salaries of all 50,000 postdocs in the US, you can see how small a share of the doubling went to younger scientists.

Now that grant acceptance rates are falling, things are likely going to be a bit rough for all the new hires. The bar has been set high for them at a time when total funding for the NIH is stagnant (actually decreasing in real terms). We'll probably see signs of a shakeout over the next few years.

NIH to NRSA postdocs: Revise and Resubmit!

2007-02-13T16:21:00Z

I'm heading up to NBER next week to spend a month working with Richard Freeman. One of the things I will be looking into (assuming I can get some data sets) is what's going on at NIH with the falling approval rates for R01 grants. Expect lots of interesting things.

One symptom of the current troubles that I recently learned about (this PowerPoint presentation is quite informative) is that more and more R01 applicants are having to send in amended proposals. I got curious and decided to see if the same thing was happening for other types of grants.

It turns out the same thing is happening to postdocs. The NRSA applications that are accepted are increasingly likely to have been amended. Back in the good old days before the NIH budget doubling, about 10% to 15% of NRSA applications had to be revised before they were accepted. Now it's up to 34%. Is the review process getting harsher all around because of what's going on with R01s? Or is the number of NRSA applicants increasing in tandem with the R01 applicants?

Another weird thing: as best I can tell from CRISP, there were 800-850 new NRSAs per year back before the doubling. Now there are only about 650-700 NRSAs per year. So while the NIH budget doubled, the number of NRSAs went down by about 25%. Now there's commitment to training! The decrease in numbers might explain in part the increased scrutiny that applications appear to be receiving.

Why are these numbers going down at all? NRSA fellowships are one of NIH's premier forms of support for the training of new researchers. Postdocs are incredibly cheap, too -- maybe 10% of the cost of an R01, tops, so it's not like a lot of money is saved by cutting them.

Maybe the money was reallocated to Career Transition awards (K22 awards)? While K22s are nice, they're considerably more expensive than NRSAs. Maybe 200 NRSAs have been replaced with 50 K22s? If so, then what looked like an increased allocation of resources to postdocs was really just a relabeling of funds.