Engineering Science - Home

Manufacturing Innovation

2008-05-13T14:13:00Z

There's a fascinating New Yorker article on Nathan Myhrvold's current company, Intellectual Ventures. One of the company's core activities involves getting lots of smart people together in a room for a few days, brainstorming, and patenting the resulting ideas. They're kind of a patent troll outfit, but they do seem to be generating some genuinely interesting ideas and pushing at least some of them into commercialization.

Reading the piece brought to mind the various efforts by NSF and others to push interdisciplinary research. IV's efforts are explicitly interdisciplinary, but they have an extra ingredient that I think is absolutely crucial: they go and talk to people outside of academia who are trying to solve real problems. NSF has had a handful of initiatives that involve getting academics out of the ivory tower - industrial postdocs in math come to mind - but not very many. The real world is a source of a lot of great inspiration for research, and not just applied research. For example, some of my own work on wireless communications led me to some deep problems in number theory (distributions of digits in binary representations of elements of the Cantor set; I never got very far, so it's wide open). I think funding agencies could get a lot of scientific (not to mention political and economic) value from encouraging greater interaction between academics and people in industry.

The researchers involved in IV have rather interesting backgrounds, which the article's author credits in part for their ability to generate novel ideas. Their resumes aren't of the form, "got PhD from X, did postdoc at Y, became faculty at Z, wrote N papers;" rather, these are people who have walked across Texas on a whim / have leveraged careers in astrophysics into paleontology, etc. Two (Myhrvold and Jung) were former bosses of mine at Microsoft, so let's hope I'm on a good track.

Making the Grade

2008-05-07T00:35:00Z

Nature this week has an opinion piece about the continued mediocre ranking of the US in standardized tests of mathematics and science. The authors claim that the tests don't really matter that much because

1) it's the proportion of very high scorers that matters, not the mean, and 2) a lot of the countries that place ahead of the US are tiny.

Fair enough on the first count. As for the second, a number of the high-scoring but small countries are in Europe - I wonder how the US compares to the EU? Probably not quite well enough to be as sanguine as the authors appear to be.

They do sensibly recommend that "education policy for our highest-performing students needs to meet actual labour-market demand," and they cite the boom and bust market for scientists of the past few decades.

The article appears to be the highlights of a critique of the Gathering Storm report entitled In the Eye of the Storm: Assessing the Evidence on Science and Engineering Education, Quality, and Workforce Demand. Something to add to the to-read pile.

Privacy paranoia at NSF?

2008-05-03T16:48:00Z

Inside Higher Ed has an article with Orwellian-sounding overtones: Data on Minority Doctorates Suppressed. The gist of it is that NSF has tightened up its privacy rules and will no longer be reporting information on the ethnicity of doctorates when the cell size is 5 or smaller.

The trouble is that basic reporting on the demographics of small fields runs up against the new rules, particularly for groups such as Native Americans:

So while we know that in 2005, six black people earned doctorates in earth, atmospheric and marine sciences, the NSF won’t reveal how many earned the degrees in 2006 (covered by the most recent report). Information about the number of Latinos earning degrees in some engineering fields is gone, as are data about a number of categories for black Ph.D.’s. For Native Americans, where the base is smaller, the impact of the new policy is especially dramatic. The report was stripped of information on how many doctorates were awarded to all but 6 of the 35 subfields for which data were collected.

Commenters on the story are aghast, and some rail about Bush administration-style information suppression going on in the NSF. I don't think the motives are quite so sinister. But I am wondering if I might be partly responsible for the changes.

In late 2006 I requested a big batch of Survey of Earned Doctorates data from NSF for the Graduate School Guide. It was expensive and a pain to get, largely because the existing privacy rules blocked access to lots of interesting things, but eventually I managed to get the numbers. The one thing I had no trouble getting was information on PhD demographics. Sex, ethnicity, and citizenship were considered public information, and no cell size restrictions applied. (It's not clear from the article whether the tightening applies only to ethnicity or to sex and citizenship as well, but I'd assume everything.)

Apparently nobody had ever requested institution and discipline-level data before. Although my request was approved, I heard through the grapevine afterwards that the data set caused great consternation within NSF and provoked lots of meetings and arguments. I asked for some additional data later on, but never got an answer.

I suspect that at least part of what happened was that NSF decided after the fact that they were uncomfortable with some of the information they gave me, but that they had no rules in place at the time to prevent the request. I'm speculating that the tightening (at least in part) is to address the perceived problem. If so, I'd be curious to hear what else has been changed.

NSF has an important responsibility to protect the privacy of the people who participate in the Survey of Earned Doctorates. They are asking for data from individuals, and they want to make sure that those participating never have reason to be concerned about their information being released inappropriately. The NSF does a very good job of protecting the privacy of those participating - the rules for data access are quite draconian.

The trouble is that the NSF protects their information so zealously that, at least in my opinion, they compromise the larger mission of their organization. Suppressing things like the number of Native American geophysics PhDs granted is just silly. In what possible way does that reveal anything interesting about any individual? (The only thing I can think of is that you might be able to learn whether a particular individual participated. But you could learn the same thing for cell sizes larger than 5 when there is full participation by the subgroup.)

Computer security professionals have a maxim that the only truly secure computer is one that is off. Some of the new privacy changes sound like a step in the direction of pulling the plug to prevent software viruses.

A more productive approach would be to modify the terms of the survey's privacy policy to favor greater data access, not less. Sure, they might lose a handful of responses from the truly paranoid, but my guess is that people who are that concerned about their privacy aren't going to fill out the survey anyway. Greater access could allow some NSF to do some truly useful things with their data.

There is some good news for any of you who are interested in PhD demographics: there is a better place to get the data. The IPEDS data set contains complete information on the demographics of doctorates, with no data suppression. What's more, IPEDS also has data on master's students and on undergraduates, their data are not limited to science and engineering, they have a much more detailed taxonomy of disciplines than NSF, and they make their data available at the institution rather than the national level. IPEDS data are much more timely than NSF's. And, best of all, you can get the full data set without having to pay NORC lots of money. The NSF may have just done you a big favor.

PE for grad students, part 2

2008-04-24T02:46:00Z

This month's Wired has more on the exercise/cognitive function connection : aerobic exercise helps, weight lifting doesn't.

They also have a rundown on drugs that allegedly enhance cognitive functioning, handy for any who are thinking about emulating the 20% of scientists admitting to using brain-boosting drugs in a recent, informal survey in Nature.

The drugs have questionable efficacy, are expensive, and have nasty side effects (which Wired details - my favorite, for methamphetamines: "Prison"). Exercise, in contrast, works well, is free, and is beneficial pretty much all around. Forget the drugs - put on some sneakers and go run.

PE for grad students?

2008-04-22T14:17:00Z

Do the following strike you as things that graduate students or postdocs excel at?

make plans

keep track of time

keep track of more than one thing at once

meaningfully include past knowledge in discussions

engage in group dynamics

evaluate ideas

reflect on our work

change our minds and make mid-course and corrections while thinking, reading and writing

finish work on time

ask for help

wait to speak until we're called on

seek more information when we need it.

For many of these items, particularly the ones relating to planning and time management, my experience has been that these are things at which grad students and postdocs are particularly bad. And they're all important to career success. These kinds of activities are aspects of a broad measure of cognitive capability called "executive function" (the list was taken from here ).

I just saw a fascinating talk at Google by John Medina on his book, Brain Rules. The book is a summary of findings from neuroscience that are relevant to everyday life. In the first half of the talk Medina examines the relationship between exercise and cognitive function. Short summary: modest amounts of aerobic exercise (~20 minutes 3-4 times per week) have a big, quantifiable effect on executive function. It also helps prevent and treat depression, a condition to which grad students are especially prone.

One interesting implication is that exercise is an important component of education. Perhaps graduate programs should incorporate a mandatory PE requirement? At the very least, smart institutions should be making sure their grad students and postdocs have access to the gym, and they should encourage physical activity through things like intramural sports leagues for grads/postdocs. Students and their advisors shouldn't view exercise as time away from the bench, but rather as an investment in higher quality output.

Check out the talk.

"How Scientific Gains Abroad Pay Off in the U.S."

2008-04-20T01:40:00Z

An interesting piece in today's NY Times: http://www.nytimes.com/2008/04/20/technology/20ping.html

Quick summary: it's getting easier for US companies to farm out research tasks to low wage countries. America is becoming a "postscientific society": our future value-add will be in "product design, marketing and finance" not in scientific innovation.

In the short-term at least, higher spending on scientists by India and China could create a glut of them in these countries, driving wages down further and making the costs of acquiring science even lower.

Not to worry, though:

For the foreseeable future, United States companies will need their own highly paid scientists “to evaluate the purchase of foreign science and to make sense of it in their own labs,” says Daniel Sarewitz, director of the Consortium for Science, Policy and Outcomes at Arizona State University.

The implication is that a very different skill set will be needed by US scientists in the future. Prospects will continue to worsen in basic research as the cost of doing domestic research becomes prohibitively expensive relative to doing so elsewhere. The real opportunities will be in figuring out how to bring discoveries to market. We'll need some basic research to keep up some core skills and for teaching purposes, but increasing emphasis may be placed on people doing applied work, translational work, and so on.

Graduate Education at Stanford

2008-04-16T02:41:00Z

Mark Horowitz, Associate Vice Provost for Graduate Education at Stanford, gave a talk at Google a few weeks ago about some of the things Stanford is working on to enhance the quality of its graduate programs. After many years (a decade or more?) of having no senior leadership with responsibility for graduate education at the university level, Stanford has finally created an office of graduate education. Mark's talk was an overview of the initiatives coming out of the new office.

Stanford's new activity in graduate education is based on an internal report. The basic thrusts are

more interdisciplinary education,
greater diversity, and
leadership training.

These are all fairly standard ideas, but a few things struck me about Stanford's approach.

Stanford has a ridiculous amount of financial resources. It appears that the university is backing this initiative wholeheartedly. This is not just a few seminars cobbled together by an underfunded graduate or postdoc office - it looks like the real deal.
Stanford has amazing courses throughout the institution, but usually access to courses in one department is limited to students in that department. The new Office is doing the smart thing of trying to better leverage existing resources. To give a sense of how serious the university is about this, they are talking about things like moving the Law School from a semester to a quarter system (maybe the other way around? this is from memory) so that students outside the Law School can more easily take graduate courses and vice versa.
One problem people have with taking classes outside their department is that it's hard to find appropriate classes in other fields because there are so many classes out there. The catalog is paper based (!) and there is not much descriptive material on courses. Stanford has set up a web site called CourseRank, which is an exercise in using collaborative filtering to help students find relevant courses outside their departments. I suspect that CourseRank may be an exercise in trying to set up a social networking site for the sake of cool points, but the idea seems sound. One lower tech and much cheaper way to accomplish some of the same goals would be to do some basic mining of data from the registrar. Looking at past cross-disciplinary enrollments would be a simple way to provide students with suggestions for possible good courses to take outside their departments.
Stanford is setting up career development as formal classes during the summer and before fall quarter (the Stanford Graduate Summer Institute ). Students have to apply, and in some cases pay a fee. Having an application process is smart - it makes the course into a desirable thing that you have to compete for rather than a freebie extra thing that people have to be dragged to. The fee accomplishes the same purpose, plus provides resources to sustain the effort.
Career development workshops are being pitched as "leadership training". It's the same thing, but "leadership" is much sexier and probably easier to sell. I suspect that because of the word's additional usage as "leader in a field", "leadership" has positive connotations even for the most unreconstructed students-must-always-be-at-the-bench faculty.
Stanford is trying to set up mentors for students outside of the regular faculty advisors. The vision appears to be pairing students with area professionals, e.g. people at dot-coms, biotechs, etc. Definitely a good way to provide some guidance that many faculty are ill-equipped to do.

All in all a good effort. It's especially promising because places like Stanford tend to inspire places that want to be like Stanford. I'm looking forward to seeing the results and to seeing other universities try to follow in their footsteps.

Graduate School Guidance

2008-02-06T02:09:00Z

One key data set I used in building the Graduate School Guide was IPEDS, which is put out by the Department of Education's National Center for Education Statistics. IPEDS is incredibly useful for this kind of thing: the data set contains a near-complete list of all colleges and universities in the country together with detailed information about student enrollments and degrees granted. The data are well-documented, easy to access (you can download everything in CSV files or grab custom subsets of the data from an online tool), and relatively recent (2006).

NCES delivers tremendous value with IPEDS in several ways:

1) They provide a lot of summary statistics from the data to give an overview the current state of post-secondary education.

2) They make it easy for others to get ahold of the data, and they provide thorough documentation so that once you have the data, it's easy to work with. As a result, third parties such as The Chronicle of Higher Education delve more deeply into the data and deliver additional interesting insights.

3) They help participating institutions use the data set to compare themselves to peer institutions. In so doing, they implicitly give guidance on good questions for institutions to ask about themselves (e.g. how much financial aid do we give relative to our peers? how well do we retain students?)

4) They provide data to third parties who help students choose colleges. It's virtually certain that US News, Peterson's, Fiske, etc, all draw upon IPEDS data.

5) They run College Opportunities OnLine, a site that provides useful information directly to prospective college students.

6) They combine multiple data sets in ways that increase the value of all the components. For example, on the COOL web site they combine their core data on universities and graduations with data on libraries and on campus crime.

The NSF has similarly interesting sets of data in their Survey of Earned Doctorates, Survey of Doctorate Recipients, and the Survey of Graduate Students and Postdoctorates in Science and Engineering, but they have not been anywhere near as effective at extracting value from them.

Here's how NSF stacks up to NCES:

1) The NSF provides data reports in the form of very basic, annual (or bi-annual) summary statistics and in Science and Engineering Indicators.

2) NSF has a decent tool for extracting additional summary statistics in WebCaspar. Unfortunately, the data are aggregated at such a high level (nationally for most things), that you can't get at the most interesting information. Basically you can't get anything beyond a count of individuals at the level of a single institution.

It is possible to get ahold of more detailed data than what's in WebCaspar, but the process is difficult, expensive, and time-consuming. The data we used on our site cost $7,000 and took months for NORC to generate. There is a do-it-yourself alternative if you have a facility that satisfies NSF's security criteria (stringent), can handle NSF's audits, and have a license for SAS (expensive).

NSF has a responsibility to protect the privacy of survey participants, and to their credit, they take that responsibility very seriously. However, I think they tend to err so far on the side of caution that they detract from their own mission.

3) As far as I know, NSF doesn't make any effort to help universities use the data they generate in useful ways. I think this is an area in which NSF could take a real leadership role. For example, people have been expressing dismay for years about the length of time it takes to earn a PhD. The NSF has detailed data on exactly how long it has taken to earn pretty much every single PhD in the country. Providing institutions with stats on their own times to degree relative to their peers could be a powerful catalyst for inspiring improvements. They could provide similar motivation on placement rates, funding levels, and so on.

The value the NSF could provide here is not just in providing the data - it's in getting institutions to ask the right questions of themselves. I have talked to several deans who participated in the recent CGS study of attrition. One common thread I have heard is that they were able to make progress on reducing their attrition rates - the big problem they had was that nobody had ever looked at the issue before.

4) People love to hate US News's rankings. The chief complaint is the exclusive reliance on reputation. There has been sporadic talk over the years about institutions proposing alternatives, but the talk has never amounted to anything. As we have demonstrated here at phds.org, NSF's data can form part of a more balanced approach. Surely a set of US News rankings that incorporated outcome measures as well as reputational measures would be more helpful to students and less objectionable to faculty members than their current approach.

5) As far as I know, NSF does not use their data to provide any services targeted at prospective graduate students. I think that's a reasonable call on their part, since I don't think it's a great fit for the skill set that the organization possesses. However, if they were to make department level data sets available (aggregated over sufficient numbers of years to protect privacy) on a regular basis to third parties such as US News, Peterson's, phds.org, etc, they could ensure that would-be students would benefit from the NSF's hard work.

6) As graduate-school.phds.org demonstrates, the NSF data sets are much more interesting when combined. One frustration I had in working with the NSF's data sets is that they are not designed to work well with each other or with outside data such as IPEDS, so combining them took months of work.

Each of NSF's data sets uses a different, incompatible taxonomy of disciplines, all of which are different from the one used by IPEDS. The NSF uses an outdated list of institutions (FICE codes) that results in information for entire state university systems being lumped together and makes it difficult to combine with information from IPEDS, which uses an up-to-date list. Rethinking the NSF's field codes and updating their institution lists would make their data more valuable to end users and would simplify the lives of the people at the institutions who have to answer both NSF and NCES surveys.

So here's what I think NSF could do to remedy things:

Take a leadership role in defining metrics of interest, and provide institutions with summary statistics about their own performance relative to their peers.
Rework their various classification schemes so that their data is easy to combine and compatible with outside data.
Provide better quality control over department-supplied self-classifications and consider ways to handle multidisciplinary departments (e.g. a department of Mathematics and Statistics)
Build a list of departments and strive to provide department-level data when practical.
Release regular department-level data sets (aggregated over multiple years and suppressed as needed to protect participant privacy) for use by third parties.

I think the result would be universities receiving information they can use to make valuable improvements to their programs and prospective students getting better guidance on choosing suitable programs. That's a pretty big impact.

Pardon the dust

2008-01-24T18:19:00Z

I've spent the last 6 weeks of vacation time and weekends sequestered in a secure undisclosed location busily upgrading phds.org. The long-promised upgrade to our graduate school guide is now out, and the whole site has been moved over to a much faster (and hopefully more reliable) server. You should now be seeing a much more responsive site with much less down time. The server consolidation and upgrade is just the beginning of a series of improvements to the site, so stay tuned!

In the process of the move and upgrade of the back end, a few minor things have broken. I'm busily fixing things, and hope to have the main things nailed down in the next week or so.

Fixing the NIH grant-making process

2007-12-07T18:03:00Z

There's a piece in this week's Chronicle about some possible changes in the NIH's grant-making process. About 6 months ago, the NIH solicited suggestions from the general public for ways to improve the process, and an advisory committee has been sifting through the thousands of ideas they received.

The article describes a few broad classes of ideas:

Streamlining the application process by reducing the length of grants from 25 pages to 15
Limiting the number of proposals a person can submit
Basing funding decisions more on an individual's than on specifics of their proposal
Providing more affirmative action for younger scientists

These ideas aren't yet official recommendations - those won't be out until later this week - but they are likely indicative of the kinds of things the NIH will actually do. Many of these ideas are good ones; I'm just not convinced they will have the hoped-for effects.

Reducing the amount of effort required to submit a proposal sounds great. People invest huge amounts of effort on their proposals; I'd much rather have them spending their time doing science than chasing money. The trouble is that making it easier to submit a grant, will probably mean that people will submit more grants, driving the success rate down even more. Going back to the lottery ticket/grant analogy: during the budget doubling, the NIH increased both the odds that a ticket would win and the amount of money paid out by a winning ticket. Not surprisingly, people bought a lot more tickets. Streamlining the proposal process, while a worthy goal, effectively cuts the price of a ticket, which again increases the net payoff. If this happens, I predict we'll see even lower success rates in the future.

Limiting the number of proposals someone can submit is a non-starter, I suspect, despite the AAMC's endorsement of the idea. The idea has some merits: it would probably reduce the number of proposals the NIH receives and force people to submit only their best ideas. However, I think that there are legitimate scientific reasons for some larger labs to be submitting multiple proposals per year. Zerhouni opposes the idea. A better alternative might be to impose a surcharge, like publication fees charged by journals, to cover review costs on proposals after the first. This would reduce the number of people submitting multiple proposals while still making it possible to do so.

Judging proposals on the reputation of the submitter rather than on their content is a recipe for all sorts of trouble. Sure, it would make life more convenient for the elites, but I suspect that the result would be complacency, not greater willingness to take risks - just throw something over the fence and you get your funding, so why make the effort? Younger scientists are a source of a lot of crazy new ideas, but they don't have much of a track record, so this kind of scheme could shut them out of funding even more than they already are.

There's an alternative approach that I think avoids most of these difficulties - more next time.

White House Round Table Views

2007-11-27T06:18:00Z

There were two viewpoints that were in evidence at the White House Office of Science and Technology Policy meeting on graduate and postdoctoral education earlier this month.

The first view, which was most thoughtfully articulated by Michael Teitelbaum, Vice President of the Sloan Foundation, is that the national discussion on science funding has focused too much on the supply of scientists and not enough on demand. We need to do a better job of ensuring that the training we provide matches what employers (both academic and non-academic) need. We also need a better (quantitative) understanding of how science functions as a system, and we need to use that understanding to optimize the way that we invest in science. Michael's slides are available here This was more or less the majority view at the meeting, though some were skeptical that demand could be gauged in any useful fashion.

The second view, which was most forcefully advocated by David Skorton, President of Cornell, is that the real problem with science is that there are simply not enough federal resources being invested. Investments in science lead to economic growth, so demand-side problems will work themselves out. I must confess that his talk left me scratching my head, so my summary probably doesn't do Skorton full justice. Peter Lee, head of the CS department at Carnegie Mellon, summarizes Skorton's views and adds some of his own in his blog. Take a look, because similar arguments have been very influential in getting things like America COMPETES passed.

Having seen for myself in the mid-90's just how badly things go when the supply of PhDs outstrips the demand for them, I'm much more inclined to agree with Michael Teitelbaum's views than with Skorton's and Lee's. But I'll have more to say about both.

White House Roundtable

2007-11-15T06:02:00Z

Peter and I have just returned from "Evolving Demands in Graduate Education, Training and Career Development for Future STEM Professionals," a workshop hosted by the White House Office of Science and Technology Policy.

The meeting was in some ways a testament to the potential of young scientists and engineers to influence the community. Sharon Hays, the organizer, is a life scientist who earned her PhD during the last big downturn for scientists in the mid 1990's. As a result of her experiences in grad school and on the market during the 90's, she has had a longstanding interest in science careers, and she is now in a position to do something about them: she is the Deputy Director of OSTP.

Around the table were most of the key people involved in the support and training of graduate students in the United States along with a number of thought leaders from academia and industry. If ever there was an opportunity to explore this issue with all the stakeholders present – this was the time.

The meeting was also a reminder of the extreme inertia of the scientific world: 10 years ago, many of the same people (Sharon, Peter, Finley Austin, and myself) assembled at a similar meeting hosted by Congressman and physicist Vern Ehlers. As Sharon noted in her closing remarks last week, many of the slides shown at the meeting in 2007 could have been taken straight from the 1997 meeting.

Sharon opened the meeting with several questions:

Why are graduate students trained with a near singular focus on becoming academics when smaller and smaller numbers of these students actually go on to being academics?
Why does it take so long to earn a PhD? MD/PhD programs prove that the time can be made shorter: in 7 years, everyone earns both a PhD and an MD.
Can we better predict supply and demand for scientists and engineers?
How can we fix problems without undermining the science and engineering enterprise that is dependent upon low-cost graduate students and postdocs?

A few quick highlights of the meeting:

Are producing too many scientists? People are still asking this question, and there was considerable disagreement on the answer.
- Michael Teitelbaum made a compelling empirical argument for an oversupply - young scientists' wages are low and many people have considerable difficulties finding jobs - and raised concerns about the upcoming NSF budget doubling.
- Charlotte Kuh presented an interesting argument that overproduction was built in to the system as a result of incentives for the various actors in the system being incompatible with desired outcomes.
- All the industry people (Pfizer, Dow, Hoffman-La Roche) reported considerable difficulty in finding suitable hires.
- David Skorton, President of Cornell, argued vigorously that the problem is simply that not enough Federal money is being invested in science and engineering.
How are things in the trenches? We had a great panel of grad students and postdocs who related their own experiences. It's not clear how representative the 4 people were of grad students on the whole, but to the extent that they are, some positive things are going on. Some common themes that emerged from their stories:
- Most of the panelists had not gone straight through from undergrad to graduate studies and had pursued interesting paths in between. They went to grad school thinking that a PhD would lead to enjoyable work and give them control over their careers.
- All reported having received little to no career guidance along the way.
- Most were planning to pursue non-academic careers because of perceptions that life as a professor is undesirable - it's a career that leaves no time for family or anything else but work.
- Despite sounding happy about choosing not to pursue academic careers, most sounded a little defensive about their nonacademic goals.
- 2 of the 4 grad students were strongly influenced by NSF’s Research Experience for Undergraduates (REU) program. The REU gave them a window on research science that they found really enticing. Several wished they had had the opportunity to have explored something similar in industry.
- Several reported receiving some positive structured career guidance in the form of a disciplinary society workshop. All wished they had had such an experience earlier in their graduate careers and thought that it would have been most effective earlier on.
- All wished they had had more exposure to opportunities in industry early on in their educations.

My take away was that there are a few real opportunities for OSTP to improve things:

1) The recent NIH budget doubling is a useful negative case study. Some Federal resources invested in understanding what went wrong and in better resource planning could go a long way toward preventing similar problems with NSF's upcoming doubling and with future human resources planning in general. Funding an external agency like the Bureau of Labor Statistics to do the work would be particularly useful: BLS has relevant expertise that is simply not a core competency of NIH/NSF/DOE/NIST/etc; and the BLS budget would not be affected by their findings about national needs for S&E, so the conflicts of interest that affect NIH/NSF's existing resource planning (projections of future work force needs influence Congressional appropriations to NSF/NIH/etc) would go away.

2) There was broad consensus at the meeting that giving students more exposure to careers in industry would be very productive. One hurdle is that it is expensive and time consuming for universities to establish and maintain the kind of industrial ties that the envisioned internships / interactions would require. Having NSF and especially NIH (since NSF already does this kind of thing to a limited extent) allocate some substantial resources in this direction would be another way OSTP could have a big impact.

3) It would be useful to fund structured guidance (i.e. workshops / classes rather than occasional suggestions from an advisor). Disciplinary societies are filling some of the vacuum left by NSF's decision to stop analyzing its own data on S&E's, but the quality and timeliness is all over the map. Some funding to help disciplinary societies provide good data to students and faculty members would be useful as would funding for institution-level services at graduate student and postdoc offices.

Two more ideas from Peter:

4) Funding graduate students directly on research grants (as opposed to supporting them on fellowships) was discussed at several points. Directly coupling PhD production to research funding is nice for PIs because it gives them a ready and eager source of cheap labor and total control of that labor. However the direct coupling ties PhD production to R&D funding levels – which invariably creates a boom/bust cycle of PhD production. Michael Teitelbaum made a point of this with respect to NSF: for every one graduate student supported on an NSF fellowship, 12 students were supported directly on grants. If you double NSF funding over the next 10 years you are signing up for a substantial increase in PhD production. Teitelbaum argued that this NSF should lower this ratio substantially (presumably by funding more fellowships relative to grad student slots on R&D proposals). This idea of decoupling the funding for graduate students from the funding of research is not new – it was a central idea that came out of the LAST crisis. Back then, the idea was to empower graduate students to make their own choices as to who to train with and what projects to undertake. But if you look at the ratios of fellowships to support through research funds since 1990 the proportion on fellowship support hasn’t budged.

5) A mismatch between supply and demand. Despite the abundant data showing the indicators of overproduction of PhDs in some fields (this time – Life Sciences) the folks from industry insisted that attracting skilled scientists and engineers was a challenge. Several industry folks explained that it was not simply a question of supply of PhDs but rather PhDs with the suitable skills and INCLINATION to consider industry jobs. One of the industry reps highlighted the “Rodney Dangerfield” syndrome that many technical managers feel: they can’t get the respect of their peers in academia who train and counsel graduate students. Furthermore, the supply/demand issue is very field-specific. Presently in the Life Sciences we are producing too many academic-oriented PhDs: WAY too many for the available funding streams. However, in Computer Sciences, PhD grads (at least according to the Dean of one top-tier department) are beating away the recruiters with sticks.

One thing that has changed since the last PhD glut is the avenues for communication. Peter and I have invited the Roundtable participants to join this blog to share their observations and ideas. We will also post the presentations that are made available.

Breaking radio silence

2007-10-14T05:04:00Z

I've been pretty quiet these past few weeks - know that it's been for good reasons!

First, I've been head-down, in-the-trenches hacking away at some improvements to the graduate school guide. One limitation of the current site is that it is geared towards people who enroll full-time in doctoral programs. While that audience is important, the overwhelming majority of people who go to graduate school are in master's degree programs, and often (probably most of the time) they are looking for part-time programs near where they live and work. The new features are designed to help prospective master's students find appropriate programs.

Some good things are in store:

There is new information about master's degree programs, so there will be profiles of 22,000+ master's and doctoral programs rather than the current 6,000 or so doctoral programs.
I have expanded and refined the taxonomy of fields, so there will be new fields to choose from.
I have added a couple of new data sets - (1) more detailed information about degree recipients from IPEDS, and (2) information about grad student demographics and funding sources from the NSF's Survey of Graduate Students and Postdoctorates in Science and Engineering.

Second, I have just moved from North Carolina to San Francisco because I will be starting a job at Google on Monday. The job is kind of complicated to describe; the short version is that I'll be a hybrid statistician/developer working with the user interface research team. I've spent a lot of time these past years asking interesting questions of large piles of data. Google has some incredibly rich data sets to mine, and I'll be digging away.

In other news, in early November, Peter and I will be attending a round table discussion on graduate student and postdoc issues hosted by the White House Office of Science and Technology Policy. I'll be putting together some materials in advance to discuss at the meeting, and I'm planning to hash them out here - I'm hoping for some good feedback.

Universities and the money fix

2007-09-24T04:17:00Z

Nature has been running some good stories this past month on the mess at the NIH.

Universities and the money fix, by Brian C. Martinson, points out what I think is the central problem:

[L]argely because of the structure of the funding flows between the NIH and the universities, there are few checks in the system to keep competition for grant funding at a healthy level. Thus, calls for further increases in the NIH budget may only make matters worse. In my view, it is time to ask the biggest beneficiaries of NIH largesse — the universities and academic health centres — to find ways to balance supply and demand that better reflect their obligations to researchers and society....

(emphasis added) Exactly right. The current problems are structural in nature. Anything that fails to address the underlying issues can only serve at best as a temporary stopgap.

There are insufficient 'feedback loops' linking the production of biomedical researchers to the availability of resources to support them. Instead, the educational system is replete with incentives to generate ever more PhDs and medical doctors. In the short term these arrangements may benefit universities, but in the longer term, such extreme levels of competition for funding are unsustainable. And they may already be doing harm.

The harm Martinson sees is greater potential for ethical lapses, something also predicted by David Goodstein. Reader BioScientist has more to say about the potential for harm in a comment on the NSF's new postdoc mentoring requirement:

None of these proposals matter, nor will any changes or improvements to them. The bottom line is that as long as there are too many scientists present the competition to stay alive will be intense and therefore conditions will remain poor. NIH/NSF proposals like this attempt to legislate behavior without recognizing the realities on the ground.

The lack of attention to graduate/post-graduate training is borne from two areas: 1) PIs are pushed by the system to extract every bit of effort possible effort from employees. The penalty for not doing so is a loss of funding and the end of a career. 2) As long as there are multiple applicants for every scientist/PI position there's less need to insure development of most individuals. As an example, one can simply ignore the bottom sixty percent of CVs to no ill effect when considering tenure-track positions. Those that remain will no doubt be pretty impressive. In this respect, post-graduate training is less an education than it is a 'selection' process in the biological sense. Put another way: why bother training postdocs, when most of them will fall out of the "system" anyway? The best will claw their way to success on their own and the rest are irrelevant. (This is a sentiment I've heard multiple times from faculty at my current Tier-I, top-20 research institute.)

Until the supply of PhD scientists comes back into line with demand working conditions will remain poor, salaries low, and hours long. In absence of a solution to the supply/demand imbalance all attempts to solve the resultant phenotypes will fail.

While I agree with Martinson's diagnosis, he unfortunately doesn't offer much in the way of solutions:

So is the only solution to force long-time NIH grant getters into retirement? Perhaps not. Universities have benefited handsomely from the efforts of senior faculty members in securing NIH grants during their careers, perhaps those same universities could now return the favour by taking full responsibility for paying these faculty salaries in their later years. This would serve the dual purpose of getting them off the NIH dole, and encouraging them to share their knowledge with their younger colleagues through more teaching.

Getting enough senior scientists to give up research for teaching to make any kind of difference seems, um, implausible.

Bring Back the Office of Technology Assessment

2007-09-19T16:31:00Z

There's a great idea floating around science blogs: bring back the White House Office of Technology Assessment:

It used to be, for about 20 years (from 1974 to 1995), there was an office on the Hill, named the Office of Technology Assessment, which worked for the legislative branch and provided non-partisan scientific reports relevant to policy discussions. It was a critical office, one that through thorough and complete analysis of the scientific literature gave politicians common facts from which to decide policy debates. In 1994, with the new Republican congress, the office was eliminated for the sake of budget cuts, but the cost in terms of damage to the quality of scientific debate on policy has been incalculable. Chris Mooney described it as Congress engaging in "a stunning act of self-lobotomy" in his book the Republican War on Science (RWOS at Amazon).

The fact of the matter is that our government is currently operating without any real scientific analysis of policy. Any member can introduce whatever set of facts they want, by employing some crank think tank to cherry-pick the scientific literature to suit any ideological agenda. This is truly should be a non-partisan issue. Everybody should want the government to be operating from one set of facts, ideally facts investigated by an independent body within the congress that is fiercely non-partisan, to set the bounds of legitimate debate. Everybody should want policy and policy debates to be based upon sound scientific ground. Everybody should want evidence-based government.

Seems like a good way to help ensure that policies are made using the best available science (plus it will create a few good science jobs!)

More here