America: The Land of Freedom and Fairness Produces Great Scientific Advantage
Written by Lewis D. Eigen
The United States generates more scientific studies than any country of the world and more than the rest of the world put together. We also produce more of the critical breakthrough research than the rest of the world. However, over half of all the scientists working in America are foreign born. Many become Americans, but they did not start out that way.
At first glance, it might seem that the explanation is the American economy which pays scientists more than the rest of the world. Therefore, they come here to work for the same reason as the economic refugees do—greater economic opportunity. Whereas that is certainly true for scientists coming from India or China, it in NOT so for large numbers of scientist who come to America to do their research from Germany, Norway, France, Italy, Switzerland, England and other highly developed nations.
Another theory is that the large proportion of foreign scientists in America is the consequences of the inadequate American elementary and high school educational system. There are not enough qualified Americans to do the scientific work that is needed. To some extent this general overstatement does contribute to the problem but explains only a minor part.
Many Americans like to think that one major reason for so many foreign scientists coming to America is our freedom. While it is true that this has been a major factor spurring immigration throughout our history, it is not really true in modern science. The days of Galileo being persecuted by the government—then the Roman Catholic Church—of the day are gone from most of the world. Even in China, where there is extensive ideological control over just about every phase of life, scientists arequite free in their science work. The Chinese have learned a powerful lesson from their Cultural Revolution. Ironically, arguably the United States has had LESS freedom for scientists than the other industrial democracies because of the George W. Bush extreme politicization of science and the abortion wars that divide the nation. There was no other modern nation where research on stem cells was as restricted. However, in most areas of science there is freedom throughout the modern countries of the world. This is not what has attracted the scientists.
The reasons scientists come to America are twofold. One reason, understood by most of us, is that scientists, like professionals of other fields, prefer to be “where the action is at”—where there are peers to interact with and appreciate their work. This is what produces the Silicone Valley, incubators, and large university scientific communities. The other reason however is unrealized by most of us. It is FAIRNESS.
While scientists generally do not earn huge amounts of money like investment bankers, and are not as heavily motivated by personal wealth aspirations, many do require funds for their research–equipment, assistance, travel, etc. And these funds generally come from three sources the world over: Private wealth people or foundations, private companies, and government. The government factor is the largest, especially in the basic research arena. In America a scientist—regardless of where she might have been born, and regardless of where he might intend to work, can compete on a relatively equal footing with Americans. More important, young scientists can compete here with their more established and well-known scientists for research funds but this is not true ANYWHERE ELSE IN THE WORLD! Most of American basic scientific resources are allocated fairly—strangely, no other country does that.
Herein we explain why. To start with, we examine the UNIQUE aspect of American allocation of public research funds and learn how and why American research allocation fairness has so far exceeded the rest of the world to America’s great advantage.
The majority of the public research grants awarded by the American Federal Government are, despite the struggle for control among the politicians, the results of a peer review mechanism. Essentially, peer review is a process whereby the scientific merit of a proposed research study is assessed by scientists, not by government administrators and not by legislators. The politicians are “out of the loop” in the peer review process. As the Director of the Center for Scientific Review of the National Institutes of Health, Dr. Toni Scarpa (himself born and educated in Italy) put it, “We control the process, but the scientists control the science.” For most people, that seems to be a reasonable way of doing things, but peer review is not a common way of allocating a significant portion of public research funds in other countries of the world. As a matter of historical fact, scientific peer review of possible research projects was an American invention, and to this day is not used extensively anywhere else in the world. And this American invention did not come from the scientists or the politicians for that matter. It was originally an invention of the U. S. military, which ironically no longer uses it except in an advisory capacity. In 1879 Yellow Fever was rampant in the Mississippi Valley. The United States Army wanted a study done of the causes and transmission of the Yellow Fever. They announced a competition where American universities could submit proposals. However, the army realized that they did not have the level of scientists who could read the proposals and tell which ones are likely to be the best. So they put a panel of scientists together from outside the army and this group judged the proposals and recommended the best one. That university received a contract for $30,000. It was the first peer-reviewed application for research.
In 1940 the nation established the National Defense Research Committee which not only awarded research contracts but also identified 700 universities who were “qualified” to do future research. Most of the decisions at the time were the results of peer review. Federal research support grew and the Federal organizations jockeyed for responsibility and power. For health research, the Public Health Service has organized and reorganized, and for most of the medical research, the National Institutes of Health (NIH) were created. There are today 27 different components of NIH which are referred to as the individual Institutes. Some are: National Cancer Institute, National Eye Institute, National Library of Medicine, National Heart, Lung, and Blood Institute, National Institute on Deafness and Other Communication Disorders, National Institute of Nursing Research and the like.
Today, most of the medical and health-related research is either conducted by or funded by one of the NIH institutes. Some of the institutes operate their own research laboratories and hire their own scientists. However, the vast majority of the $29 Billion NIH Budget goes for funding of outside research. The NIH Center for Scientific Review administers the peer review system which reviews the majority of NIH grant applications. In 2008, the scope of its activities were:
- 50,000 applications were reviewed.
- It took 16,000 peer reviewers to do that.
- There were 240 NIH Scientific Review Officers.
- They organized 1,400 review meetings to peer review the proposals.
The numbers are much larger than most people expect. The logistics of the 1400 meetings for the 16,000 scientists alone makes NIH one of the largest users of airlines and hotels, although approximately 20 percent of CSR’s review meeting are now electronic meetings, involving online discussion boards or video conferences.
Essentially, this is what happens. Scientists who wish to compete for research funds, prepare a written proposal within a page limit that is set from time to time by NIH. For most investigator-initiated grant applications submitted to NIH there is a 12 page limit.. Applications also require resumes showing the people who would conduct the research, a budget showing not only the amount of funding but also the details of how the money shall be spent, and finally there is a section that specifies how human or animal subjects would be treated and what steps would be taken to obtain informed consent and meet all the ethical and legal requirements. This package of paper generally runs from 40 to 100 pages. Meanwhile NIH has 240 Scientific Review Officers. These are themselves trained, accomplished scientists who work full-time for the agency. However, they themselves no longer compete for the research money.
Each is responsible for a peer review committee. The committees tend to be organized around specialty areas. Each proposal is assigned to the committee that best matches the subject of the proposal. Each of the Scientific Review Officers is responsible for pulling together a peer review group of scientists who will read and assess the proposals. The committees tend to have about 20 members, and the scientific review officers will recruit and supplement the core with specialist scientists who are skilled and experienced in areas that match the proposals. The committee members meet several times a year—usually 1-2 days, in which they discuss the proposals that have been assigned to their committee and score and comment upon each. Perhaps, the most fascinating thing about the peer reviewers is the fact that they essentially are unpaid. Their travel expenses and hotel costs are covered for each of the meetings, and they currently receive a per diem to cover meals and other expenses and a $200 “honorarium.” . Almost all of the peer reviewers work for institutions which allow their staff to serve on such professional committees. These reviewers do not lose pay when the serve, but they contribute weeks of their limited free time to review their assigned applications. NIH Peer reviewers are among the scientific elite of the nation (there are a few specialists from Canada and some other foreign countries). About half are or have been full professors at universities and another 25 percent, associate professors. They are almost all published, accomplished scientists. The same is true of the National Science Foundation peer reviewers which is the other major scientific organziation that uses the peer review process.
Typically, peer reviewers receive a confidential CD from the Scientific Review Officer a few weeks before the meeting. This contains all the proposals that will be reviewed by the committee. Each proposal additionally is assigned one of the committee as the lead reviewer and two others as secondary reviewers. Each committee member reads the complete proposals to which he/she is assigned and assigns a preliminary impact/priority score as well as individual scores for each core review cretiron. At least two of the assigned reviewers produce written critiques. In addition most reviewers read many of the others as well. This process takes about 20-30 hours of time so this is a significant effort for each of the peer reviewers. Then they will additionally spend 1-2 full days at the review meeting itself and a day or more in travel time.
At the meeting, the proposals are discussed in order of their preliminary scores until about 50 percent of the applications are discussed in detail and the reviewers agree that none of the remaining applications need to be discussed. . The lead reviewer describes the proposed study to the committee as a whole and then shares his/her preliminary assessment. The other two (sometimes 3 or more) assigned reviewers will in turn give their views. Then individual members of the committee who wish will comment, ask questions and/or raise potential problems. The members of the committee discuss the proposal, and each member of the committee gives each proposal a final impact/priority score —the latest scoring system has a 1-9 score assigned from “Best” to “Worst.” The perceived strengths and weaknesses of the proposals are documented. In addition, the committee specifically comments on the budget, the human or animal subjects plan, and an assessment and score is made of the perceived impact of the study upon the public health and the scientific progress of the nation.
The Scientific Review Officer then, in the next weeks, informs the proposed principle investigator of the results. The average score is shared as are the strengths and weaknesses as well as the comments on other aspects such as the budget or for a human subjects potential problem that the committee might have observed.
This entire cycle is repeated several times a year. The peer review process is primarily administered by a unit of NIH which has no other responsibility other than managing the huge, critical, peer review process. However, neither the Scientific Review Officer or the peer review panel actually awards a research grant. The scores and comments are transmitted to the individual Institute (or sometimes two or more institutes) which has the responsibility for the general research area. This Institute—say the National Eye Institute—then makes the final funding decision after consulting with its advisory council. In some Institutes, they rank all the applications by peer review score and fund the highest scored studies first. They keep doing this until they have exhausted their research funds. Some other institutes might well rearrange the priority order in terms of their current priorities. A medium scoring study might have been determined by the Institute advisory committee to cover an area of the highest priority for that field of health. So the Institute sets the priorities, but they can only choose from those proposals that have received generally higher than average scores by the peer review committees.
Put negatively, under the existing peer review system, no Institute can simply unilaterally fund a study that some crony of the director’s might have submitted or a Congressperson might be trying to push, unless it passes the threshold of having a good score assigned by the external peer review committee. One of the great accomplishments of this system is that it has virtually eliminated cronyism, nepotism, the old boys’ network and political patronage from the research grant-giving process. Unfortunately, the same cannot be said of the research in the Department of Defense, Department of Energy and a number of other organizations. It is the National Institutes of Health and the National Science Foundation where the grant giving process has been essentially de-politicized. The standards of impartiality are so high that a member of the peer review panel who might be a faculty member at the University of Connecticut must recuse herself from reviewing and discussing any proposal where another faculty member from a different department of the school and whom he has never even met is even bid as a consultant. There can be no “appearance” of a conflict of interest.
In NIH, it is the individual institute that has the responsibility of overseeing the research project once funded. The institute is responsible for the oversight of fiscal and professional activity. The Peer Review only enters the picture at a single time and for a single purpose—to assess the scientific and technical merit of an application to inform the funding decisions made by the institute. There is a complete separation between the awarding of the studies and the oversight and administration.
The NIH external peer review process is not perfect by any means. However, the culture is a scientific one, and there is a constant introspection and review of all the procedures and practices. The agency makes alterations frequently, many of which are continued and some are rejected as experience shows that there is no marked improvement. There is a very high degree of satisfaction of the research community with the system and its openness to change. One complaint that occurs more frequently than anyone would like, occurs where a proposal is at the bleeding edge of a new field. The proposer concludes from the comments that there was no one on the peer review panel who understood the area well enough. Sometimes this is true, and occasionally the Scientific Review Officer will add a scientist to the panel who is at that cutting edge, but this is easier said than done. NIH has very strict and very good conflict of interest standards. There are some fields of science which are so new that the only 5 people in the world who know anything about it, all came out of the same university and were all students of one or two great professors. They are all friends, have co-authored papers, slept with each other, or employed each other. There is simply no way of getting a qualified reviewer who does not have a potential conflict of interest. So the other scientists do the best that they can without the depth of understanding of the elite 5 or so.
Probably the weakest elements of the NIH peer review system is that there are three kinds of studies that are not very likely to be funded. First is an idea that is so far towards the cutting edge, that no one, other than the proposer, understands it. The principle investigator is limited to only 12 pages for the entire technical proposal which leaves only about 5 or 6 pages to persuade the reviewers about the methodology. Not enough for anything that is really radically new and complex. However the price of allowing more pages is high as that means that 16,000 reviewers would have to read even more. Second, is an idea that comes from another field. A theoretical physicist, for example, who has never applied for a medical research application generally does not know how to write up her idea so that the health scientist will feel comfortable with it. The fact that mathematical equations prove something might be sufficient or strongly imply it is enough justification for the National Science Foundation to commit funds, but most medical scientists are used to seeing empirical research—not mathematical and theoretical.
The third kind of proposal that is unlikely to get funded is one which is proposed by an individual who is not affiliated with university or viable business–a “nobody”—a high school kid, a non-scientist, etc. There is no rule against such, but human nature being what it is, a high score would not often be given by most scientist reviewers to a principle investigator who has little or no scientific background and never conducted or assisted with a research study. An intelligent patient or nonscientist who, as a result of his experience, has an excellent medical idea would not likely be funded unless he found some medical scientists to collaborate with him
However these shortcomings of the system are known and there is constant effort to try and further improve the system. However, these same defects would certainly occur if members of Congress or administration officials were making the decision. So Peer Review does not solve all the possible problems, but far more than the other options.
Fortunately in America there are also private foundations some of which, like the MacArthur Genius Awards, are looking for those high risk, cutting edge ideas. But the NIH and National Science Foundation system is close to “as good as it gets.”
For the peer review system to work, there must by many volunteer scientists who commit their time. From the scientists’ point of view, they are faced with this reality: If they want peer review as opposed to administrative or legislative review, they have to participate. There is no free lunch, especially in science. (Only politicians seem to be able to get away with promising this.) Many millions of dollars worth of professional, scientific labor are donated to the process by scientists throughout the country.
The National Institutes of Health has one of the more sophisticated mechanisms for allocating public research funds in so large an operation. It is a model that is or has been studied by municipalities, states and even foreign governments who are in many cases beginning to adopt some American style peer review. The 4 key elements to the NIH peer review system are:
- The only possible, long term source for adequate support of our medical research is the taxing power of the federal government.
- The federal government and politicians must assure complete freedom for individual scientists in developing and conducting their research work.
- Reviews should be conducted by outside, objective scientific experts essentially without compensation.
- Program management and review functions should be separated
It should be noted that not all the NIH research is under this peer review system, although most is. There are some special type programs where the individual institute staffs make the decisions, They may or may not use outside scientists in reaching their decisions. Congress and the White House might or might not influence the decision. So even in NIH there are some research studies that avoid peer review. And there are some earmarks which almost always corrupt the scientific integrity of the study, but there are far, far fewer earmarks proportionately than are imposed by Congress upon other agencies. From the point of view of a member of Congress, the members are criticized for using their capricious and arbitrary judgment to determine who gets federal money. However, if Congress does not use its personal judgment, then the administration—the politicians of the Executive Branch—will use theirs. Of the two, Congress naturally prefers earmarks. But there is a third way—a better scientific way–a fairer way. Peer review tends to keep all the politicians out of the process. If politicians are to be the deciders of who should get research funds, then the research becomes part of the spoils system and one cannot blame the legislators for wanting to be the political deciders rather than the executive politicians.
As proven as the NIH and NSF peer review systems have been, we might think that that would be the model for almost all government research funding. But it is not. NIH is a component of the cabinet level Department of Health and Human Services, and there are some units within the DHHS family where there is executive capriciousness in the awarding of their grants and contracts. In one agency, the Substance Abuse and Mental Health Services Administration, the award giving was so capricious and arbitrary that in one year, the United States Federal Court of Claims had to issue two injunctions against the agency for failing to even follow their own administrative standards of fairness. What makes the NIH and NSF peer reviews so much more remarkable, is that there is always a constant pressure from both administrators and legislators to control more of the decisions.
Peer Review Contrasted to Other Nations
Most of the industrial democracies of Europe have traditionally used another method of allocating research funds. They fund positions and people rather than studies. For example, in England the Parliament and the Government make an appropriation to the Royal Astronomer for astronomical research. Whoever happens to be the Royal Astronomer at the time gets to be the decider of how the research funds would be allocated. So the government—the politicians—make the decisions to fund individuals and their institutions. And the actual studies and who principle investigators are to be are made by a small handful of what might be called Science Administrators. Or the government administrators may make the decision. There is a marked contrast with the American system where to a great extent studies are funded specifically—not people. A great scientist, who submits a proposal, that is mediocre or not perceived by the peers as highly significant, in America, might not get funded. In most countries, that scientist or his laboratory would be funded. Most other nations have an elite subgroup of scientists who the society bets on. The others either work for the elite or will get little or no national funds. It is a type of a scientific aristocracy. The American peer review system, in contrast, pays much less attention to what a scientist did yesterday, but demands to know what he intends to do tomorrow. In Europe or Japan for example, once a scientists reaches one of these prestigious positions, he/she is funded for life or until retirement. In the American peer review system, the scientist has to compete for his entire career. In one panel in which I was a peer reviewer, the committee discussed a proposal that had been submitted by an outstanding, experienced scientist, but the reviewers thought the methodology of this particular proposal was quite mediocre if not poor. Several on the panel mentioned the past great work of the scientist, but in the end, the proposal received a very low score and was never funded. If the scientist were to get more funding, he had to do better and compete with the younger and lesser known scientists. Being an experienced scientific principle investigator counts on the positive side, but does not by any means, assure funding in a peer review system.
In medical research for example, in the United States, about 80 percent is peer reviewed, and 20 percent not. The medical research that is not is usually funded by the non NIH components of the Department of Health and Human Services or the Defense Department. In the other industrialized nations only 5 to 10 percent, if any, of government funded research is peer reviewed. That is changing slowly with more interest in the American system being shown by other nations. The reasons are two-fold. First the U. S. biomedical and physical research has been incredibly productive. Most of the scientists and science policy makers credit the peer review system for a substantial part of the success. They observe that without a peer review system, other countries have, in effect, cut out a large proportion of their potential, scientific, research human resources and given fiscal resources to a small elite which may well be working on yesterday’s science or with yesterday’s methodology and way of thinking. There is less money available to fund the younger people who might be at the cutting edge of tomorrow’s science. In Europe, funds tend to go to the older scientists, when younger scientists are statistically more productive. Ironically, this elite scientist system has contributed to a type of brain drain of younger and up and coming scientists. An Italian scientists can get wonderful scientific training from leading scientists in Italy, but upon graduation with their degrees they then have to work for other scientists with the money and can only slowly get to do the research that they want. By emigrating to the United States, the Italian, English, German, Korean or other young scientist can be a lowly graduate student, instructor or assistant professor, and still write and submit his own proposals. If his/her scientific peers on the review committee think the study is meritorious, the foreign scientists will become a funded principle investigator—a virtual impossibility back in his/her country of origin and training. The fact that more than half of all scientists working in America were foreign born, in part, is due to the equal opportunity and fairness of the American peer review system.
The interesting political question is why the peer review system has not spread beyond the National Science Foundation and the National Institutes of Health. One cynical explanation of the status quo is that the politicians let the scientists have their peer review for NSF and NIH when the two agencies together did not have very large research budgets. But the politicians maintained control of the Defense Department and Department of Energy, NOAA, National Security Agency where the “big bucks” were and still are proportionally. Ironically, the other industrialized democracies may well adopt the peer review system sooner and to a greater extent than the other non-peer review agencies of the United States. This would be a shame. By expanding peer review into these other American agencies, we will not only have better and more productive scientific research but we will attract even more foreign scientists to American–the nation of scientific fairness.
 Toni Scarpa, Personal Interview, NIH, April 14, 2009
Entry filed under: Health & Medicine, Politics, Science, Solutions. Tags: brain drain, conflict of interest, earmarks, foreigners, grants, medicine, National Institutes of Health, National Science Foundation, NIH, NSF, peer review, Politics, redearch financing, research, Science, scientists.