It’s getting harder and harder to produce ideas

Innovation has become increasingly difficult. Not only are companies spending increasingly on research, but researchers are also less and less productive. These are the conclusions of an American research study published in 2020. They apply just as much to IT as to agriculture and medicine. However, LLMs (Large Language Models) offer new hope, which promises to revolutionize research. The productivity of researchers using these LLMs could be greatly increased.

Moore’s Law, the perfect example of declining productivity

Moore’s Law assumes that transistor density doubles every 2 years. This corresponds to a constant 35% annual increase in density. Between 1970 and 2020, Moore’s Law has been perfectly respected (see graph below). It has undoubtedly become an objective for manufacturers, who have made unprecedented efforts in terms of research to achieve it (this is in itself a virtuous application of Goodhart’s Law).

The 4 authors of the article show that the number of researchers needed to double the number of transistors is 18 times higher than in 1970. Meeting the targets, Moore’s Law sets implies a constant 35% annual increase in transistor density. Between 1970 and 2015, the researchers analyzed the research personnel expenses of all companies involved in chip development. The findings are clear from the graph below, taken from their article.

As the authors of the research write:

“Research effort has risen by a factor of 18 since 1971. This increase occurs while the growth rate of chip density is more or less stable: the constant exponential growth implied by Moore’s Law has been achieved only by a massive increase in the amount of resources devoted to pushing the frontier forward.”

This 18-fold increase in researchers ultimately corresponds to an average annual drop in productivity of 6.8%.

Why is this? Achieving the objectives of Moore’s Law means pushing back the technical and physical limits a little further (etching finesse, heat dispersion). Beneath this “drop in productivity” lies an exponential technological challenge as the years go by. We can thank Gordon Moore, whose prediction in 1965 undoubtedly turned into an unconscious industrial objective that helped to drive computer science forward.

Agricultural research not up to productivity challenges

Of course, research challenges are not limited to computing. Agriculture is another area where research is crucial to addressing the planet’s challenges. Taking all types of crops together, the authors show that the capacity to conduct yield increases (blue line) is declining, while at the same time, the research effort is increasing (green lines).

The graph is admittedly less easily read than the one concerning Moore’s Law. But the conclusions are the same. Simply put, it takes more researchers to conduct less research than before. Between 1970 and 2007, the decline in agricultural research productivity averaged 3.7% yearly.

The authors have analyzed research efforts in more detail according to crop type: corn, wheat, soybeans, and cotton. And, of course, the results are also consistent at this level (see image below). In the case of corn and soya, the number of researchers increased by 24 between 1970 and 2010.

Medicine gives patients less and less hope

If there’s one area where research efforts are visible to the general public, it’s medical research. Finding cures for the diseases that keep us from aging is a priority.

• the number of years of life “saved” for different types of disease (cancers, breast cancer, heart disease)
• The number of scientific publications on these different diseases as a proxy for the research effort.

They show that research effort increases linearly while research output decreases. Whatever the disease researched, the results are the same.

Conclusions

It is becoming increasingly more work to maintain the pace of innovation. However, you measure results (number of transistors, crop yields, years of life saved), and more researchers are working.

The “yield” of research is constantly decreasing because it’s simply increasingly complicated to “find.” We are pushing back the limits of matter, the limits of life, to try to improve what can be improved. But pushing these limits is becoming increasingly difficult. This explains why individual researchers are less and less efficient. So, they conduct increasingly to keep pace (Moore’s Law), and sometimes that’s not even enough (agriculture, medicine).

This is where LLMs (Large Language Models) bring hope. This disruptive technology may enable these same researchers to achieve productivity gains. Initial trials have been promising. A hackathon has identified various productivity gains in the search for new materials. LLMs may also simplify the discovery of new drugs. In short, beyond anecdotal applications such as content creation, LLMs open a field of possibilities that are likely to give a productivity boost to researchers in all fields.

By Pierre-Nicolas Schwab Dr. Pierre-Nicolas Schwab is the founder of IntoTheMinds. He specializes in e-commerce, retail and logistics. He is also a research fellow in the marketing department of the Free University of Brussels and acts as a coach for several startups and public organizations. He holds a PhD in Marketing, a MBA in Finance, and a MSc in Chemistry. He can be contacted by email, Linkedin or by phone (+32 486 42 79 42)