ch8-1


FIRST AMONG EQUALS : SUMMARY AND ASSESSMENT


Summary

In the summer of 1958, while a new space agency was forming, the Space Science Board had solicited and evaluated proposals from scientists to participate in the nation's space science program. The Board sent its recommendations to the three agencies that it thought would be involved in space science-the National Science Foundation, the Advanced Research Projects Agency, and the new space agency, the National Aeronautics and Space Administration (NASA). In December 1958, shortly after it opened its doors, NASA established a policy that all scientists, whether in academia, industry or federal laboratories should have equal opportunity to participate in NASA's Space Science Program and that NASA Headquarters, not the Space Science Board, would decide which scientists would be selected.
By December 1959, however, NASA faced a serious crisis in the selection of space scientists. Many scientists were competing for the limited opportunities to fly their instruments on NASA's spacecraft; NASA had not yet established the procedures to implement its policy or produced a documented process for selecting space scientists; and the Space Science Board had continued to accept and evaluate proposals. By the end of 1959, scientists were confused as to who was selecting space scientists-the Space Science Board, scientists at NASA space flight centers, or scientists at NASA Headquarters.
As a result of this confusion and the lack of a formal selection process, scientists did not trust NASA to provide equitable access to, or make a fair selection from among the proposals of, the competing scientists. Academic scientists were convinced that the scientists at the two NASA space flight centers, the Goddard Space Flight Center and the Jet Propulsion Laboratory. had an advantage because they were heavily involved in planning scientific missions and in the selection of scientists for those missions. Conversely, the scientists at the two NASA centers were convinced that academic scientists had the advantage because the early selections of scientists for NASA missions had been made by the Space Science Board and the Space Science Board consisted only of senior academic scientists. In addition, these scientists at Goddard and JPL, particularly those at Goddard, felt that they did not have an equal opportunity to participate in the missions managed by the other center. They felt that the other center would favor proposals from their own scientists or from senior academic scientists.
Young scientists felt that they did not have a fair chance to compete because the selections were made by senior scientists on selection committees who often evaluated and selected their own proposals, a tradition that had been established in the early sounding rocket programs when there was only a handful of scientists interested in using rockets in their research program. The scientists who selected the scientists for the first satellite experiments had continued this tradition NASA Headquarters was under extreme pressure from Congress to fly some successful missions and obtain some exciting results in order to demonstrate to a worried public that the United States was once more the leader in space science and technology.
Early in 1960, NASA regained the confidence of the scientists by bringing the selection process into NASA Headquarter and placing it in the hands of scientists who had left active research to become administrators of scientific programs at NASA Headquarters and therefore should have no professional or personal interest in the outcome of the selection process. NASA created the Space Science Steering Committee to review the objectives of each scientific mission, the scientists who had been selected for that missions, and the process that had been used to select them. The Steering Committee created discipline subcommittees to evaluate the technical merits of their proposals and the scientific competence of the competing scientists and categorize each proposal as to its scientific merit and readiness for flight. Each of these subcommittees was chaired by a Headquarters scientist and its membership consisted of active space scientists from academia and the NASA space flight centers. Scientists were prohibited from evaluating their own proposals.
NASA evolved a "fairness" doctrine which combined scientific merit with equitable access to flight opportunities. NASA issued a formal announcement well in advance of the time it planned to select the scientists for a particular mission. This announcement contained the schedule and all necessary technical information. In a further effort to enable academic scientists to compete on an equitable basis with the scientists at the NASA centers, NASA Headquarters provided balloons and sounding rockets as well as funds for supporting research and development that enabled academic scientists to continue to analyze their data from previous missions, train graduate students, develop new instruments, and prepare proposals for the next competition.
NASA documented this process in TMI 37-1-1 and began to apply it rigorously to the selection of space scientists. The NASA process resolved the immediate crisis, established NASA's credibility, and provided sufficient confidence, encouragement, and support to enable over one thousand scientists to participate in the space science program. The basic process instituted in 1960 is still being used today.


Assessment

NASA's Unique Problems
NASA was the first federal agency to confront head-on the perplexing problem of how to select from a large group of highly competent and highly competitive scientists those few who would be allowed to use an expensive, highly visible, publicly furnished scientific facility.
The cost and risk of space science created many difficulties for NASA and its space scientists. By the end of 1959, it was apparent that each space science mission would take about as long to build and cost about as much as a major accelerator or telescope. Once built, however, a ground-based telescope or accelerator could continue to operate for a decade or more, whereas a spacecraft would operate for only a year or two. Scientists depended on ground-based facilities to work for ten or twenty years. Space scientists knew their spacecraft might fail after a year or two and they would receive no more data from the mission. Their space research program was over until they could place their instruments on another NASA mission or unless they could continue their research by using balloons and sounding rockets.
The fierce competition and the risks inherent in space flight, together with the lack of any assured continuity in a scientist's research program, placed extreme pressure on NASA for a commitment to fair competition among scientists.
NASA held a monopoly on U.S. space science. Scientists conducting ground-based research generally had access to several facilities and funding from two or three federal agencies, whereas a space scientist had no recourse but to work with NASA.
NASA operated under the glare of the TV camera. Ordinary people paid extraordinary attention to space activities. Sputnik shocked Americans and turned space research into a race with the Soviets. Exciting races, whether between horses or spacecraft, help sell news papers and TV time. In addition to the competitive aspect of space, people were full of questions and excited by the newness of space flight and space science, particularly in the areas of lunar and planetary exploration. Were there really canals on Mars? Did a civilization exist under the clouds of Venus? What causes the great red spot on Jupiter? The media understood this intrinsic interest and exploited it. Many scientists found themselves before television cameras explaining the "earth-shaking" significance of their data before they even had time to determine that their instruments were working properly. The media attention helped NASA justify scientific missions to the Bureau of the Budget and Congress, but it made the agency more vulnerable to complaints from dissatisfied scientists.

NASA Solutions to Its Problems
In order to solve its problems, NASA created a strong Headquarters organization with a scientific and technical staff to establish policy, formulate the research program, establish scientific missions, and select the scientists for those missions. NASA dealt with the intense competition among space scientists and all the congressional and media attention by establishing an elaborate procedure to select space scientists; the formal announcement of opportunity; the subcommittee's scientific and technical evaluation of proposals; the center's determination of compatibility of the instrument with the spacecraft; the preparation of a tentative payload by the program office; the Steering Committee's review of the selection process and the scientists selected-all before final approval by the associate administrator for the Office of Space Science. After approval, the associate administrator sent letters to all who had submitted proposals advising them of the outcome and listing the scientists that he had selected. This lengthy step-by-step process left ample time for dissatisfied scientists to complain and for those complaints to be dealt with prior to final approval. In addition, NASA rigorously followed its own procedures, resisted external pressures and kept scientists informed as to its plans and decisions.

Conflict of Interest
Were the subcommittees "old-boy" networks? Were conflict-of-interest principles violated? Were they fair? Did they do a good job? The subcommittees were certainly "old-boy" networks in the sense that the members were all competent, practicing, highly competitive, space scientists who knew one another well. Conflict-of-interest principles were sometimes violated. Subcommittee members evaluated proposals of other scientists who came from their own institutions, an ethical conflict of interest. Although subcommittee members did not evaluate their own proposals, they evaluated proposals that competed with theirs for space on the same spacecraft.
In practice, however, the subcommittee evaluations were fair. They had to be. The subcommittee members were selected on the basis of demonstrated competence. A subcommittee member knew his or her tenure on the committee was limited; a year or two later, the competitor whose proposal he or she was evaluating might very well be on the committee and evaluating a proposal of the current subcommittee member. Subcommittee members operated in the presence of ten to fifteen of their scientific colleagues; if anyone attempted to favor an obviously inferior proposal of a friend or colleague, that person's colleagues judged him or her stupid or dishonest, and either judgment was likely to damage a scientist's career. Van Allen points out how very difficult it is for scientists to put anything over on their colleagues on an evaluation committee, separately or collectively. ¹⁷⁵
NASA asked the most qualified and most knowledgeable scientists to serve on the subcommittees and many agreed to serve. The intense competition and the scrutiny of their peers forced them to examine competing proposals in great detail in order to understand their strengths and weaknesses and to be able to justify the subcommittees' recommendations. In one sense, the competition made their work easier: they generally had two or, three excellent proposals, each prepared by highly competent scientists, so that no matter who they recommended they could be sure they were recommending a first-class scientist with a first-class experiment.
Did NASA evolve a perfect selection process? No. There is no way to assemble a group of scientists with the requisite knowledge and experience to, evaluate a group of scientific proposals without, at the same time, assembling their particular prejudices, including their lack of time to carefully read all the proposals and their tendency to follow the herd in making decisions. Some will have to pontificate, thereby using up available time so that the final decisions will be made in haste in the closing hours of the meeting, with half the group gone and the other half eyeing the clock to be sure they catch their plane home. In convening such a group, a compromise will always have to be reached between using the most competent scientists and avoiding conflicts of interest.

Lessons learned
What lessons can be learned from the NASA experience in selecting space scientists? Which techniques can be applied to other large science programs? Several conclusions can be drawn from the NASA experience:
A scientific research program must be planned and administered by scientists, rather than engineers or professional administrators; only in this way can an agency hope to keep the scientific objectives uppermost in the program. Academic scientists and their graduate students must be involved; they provide new ideas and the new people essential for the long range health of the program. Academic participation helps ensure rapid dissemination of the scientific results into society.
Institutions that operate large research facilities must have a cadre of highly competent scientists to help plan and operate the facility. They are needed to ensure that the facility, whether a telescope or a spacecraft, is built to meet the scientific objectives and does not become merely an engineering marvel. These scientists at the facility must be able to conduct their own research projects. Knowledgeable scientists, who are not in competition for funds or the right to use the facility, should oversee the selection process in order to enable academic scientists and scientists operating the facility to compete on an equitable footing. Both groups of scientists are essential for a sound, creative, long-range research program.
Because of the nature of the space program, NASA needed a strong scientific organization at NASA Headquarters that worked with its own scientific advisory groups in order to mesh scientists and their experiments together with NASA's spacecraft, launch vehicles, and communication networks. Such strong headquarters organizations in Washington are usually not essential for the management of a large self-contained government-funded scientific facility and should be avoided.
It is particularly important to have a single principal investigator who is responsible for an experiment from conception to publication of results. This practice produces reliable, highly useful instruments because they are designed, built and tested by each scientist with his or her experimental objectives firmly in mind. In a highly competitive climate with innovation a criterion for selection, it will also rapidly advance the instrument technology.
Scientists and engineers must work together on major research projects. Both are required, both will be more productive and less anxious when they understand each other's objectives and motivations. Good communication between all the people working on a research program is essential. Working scientists need to understand the objectives and management philosophy of the agency that supports their work; the leaders of the agency need to understand the objectives of the scientists and help solve the problems that they face.