Can Big Science Be Too Big?

This article is from the Science section of the New York Times this last week. I had the “gut feeling” that this was true, but now we have an actual mathematically quantifiable study to demonstrate this conclusion.

A new study finds that small teams of researchers do more innovative work than large teams do.
Modern science is largely a team sport, and over the past few decades the makeup of those teams has shifted, from small groups of collaborators to ever larger consortiums, with rosters far longer than that of the New England Patriots. Answering big questions often requires scientists and institutions to pool resources and data, whether the research involves detecting gravitational waves in deep space, or sorting out the genetics of brain development.

But that shift has prompted scientists to examine the relative merits of small groups versus large ones. Is supersizing research projects the most efficient way to advance knowledge? What is gained and what, if anything, is lost?

Now they have at least the beginning of an answer. In the largest analysis of the issue thus far, investigators have found that the smaller the research team working on a problem, the more likely it was to generate innovative solutions. Large consortiums are still important drivers of progress, but they are best suited to confirming or consolidating novel findings, rather than generating them.

The new research, published on Wednesday in the journal Nature, is the latest contribution from an emerging branch of work known as the science of science — the study of how, when and through whom knowledge advances. The results could have wide-ranging implications for individual investigators, the academic centers that employ them and the government agencies that provide so much of the financing.

“There has been a huge amount of debate in the scientific community about the effects of moving to larger research teams,” said Albert Lazlo Barabasi, a professor of network science at Northeastern University, who was not involved in the research. “This new paper gives us a way to resolve the debate. It’s an enormous contribution.”

In the study, a trio of investigators led by James A. Evans, a sociologist at the University of Chicago, mined selections from three vast databases: the Web of Science, using more than 42 million articles published since 1950; the United States Patent and Trademark Office, with 5 million patents granted since 1976; and GitHub, with 16 million software projects posted since 2011.

By working backward, the research team, which included Dashun Wang of Northwestern University and Lingfei Wu of the University of Chicago, analyzed the pattern of citations generated by each paper or project. The content of the citations was the crucial indicator, not their number. Truly original contributions, such as Einstein’s general theory of relativity, published in 1915, are cited extensively by later papers. But, notably, those later papers do not cite the same research that the original did: that work marks a clean break from the past.

More incremental articles, by contrast, tend to be widely cited by later papers, but they are referenced along with a sizable chunk of their own citation list. These papers represent not a sharp bend in the river, but a steady, deepening stream.

Based on these differences in citation content, Dr. Evans and his team rated papers and projects on a measure of “disruption.” Nobel Prize-winning papers tended to cluster at the top of this disruption scale; they were also more likely to be mentioned by current leaders in each field of science, in phone surveys by the research team.

When the team correlated this disruption rating to the size of the group responsible for the project or paper, they found a clear pattern: smaller groups were more likely to produce novel findings than larger ones. Those novel contributions usually took a year or so to catch on, after which larger research teams did the work of consolidating the ideas and solidifying the evidence.

“You might ask what is large, and what is small,” said Dr. Evans. “Well, the answer is that this relationship holds no matter where you cut the number: between one person and two, between ten and twenty, between 25 and 26.”

It also holds within every field in science, whether physics, psychology, computer science, mathematics, or zoology, he added: “You see it within field, within topics. And two-thirds of the effect we found is within the individual. That means that if I’m writing a paper, and I partner with one other person, or two, the result is less disruptive with each person I add.”

Psychologists have found that people working in larger groups tend to generate fewer ideas than when they work in smaller groups, or when working alone, and become less receptive to ideas from outside. Why that would be — isn’t entirely clear, but it runs counter to intuition, said Suparna Rajaram, a professor of psychology at Stony Brook University.

“We find that the product of three individuals working separately is greater than if those three people collaborate as a group,” Dr. Rajaram said. “When brainstorming, people produce fewer ideas when working in groups than when working alone.”

There are upsides to working in groups, Dr. Rajaram said. Over time, group members learn a lot from each other, and incorporate that knowledge. “But overall, this new study provides findings on a large scale that are consistent with the underlying principles of our work,” she said.

It makes sense that science has shifted toward a large-team model. Large teams have clout; they typically include a number of prominent, influential figures at big-name institutions. They attract some of the best younger scientists, who gain a career boost by signing on. And these trends, in turn, lead to more published papers, promotions, grants and tenured positions.

The new study suggests that a different kind of funding approach may be needed, one that takes more risk and spends the time and money to support promising individuals and small groups, Dr. Evans said.

“Think of it like venture capitalists do,” he said. “They expect a 5 percent success rate, and they try to minimize the correlation between the businesses they fund. They have a portfolio, one that gives them a higher risk-tolerance level, and also higher payoffs.”

Benedict Carey has been a science reporter for The Times since 2004. He has also written three books, “How We Learn” about the cognitive science of learning; “Poison Most Vial” and “Island of the Unknowns,” science mysteries for middle schoolers.

This entry was posted in Center for Environmental Genetics. Bookmark the permalink.