Scientafic!

Before I dive into the main topic of this post, I would just like to point out that when your car has an engine issue, you go to a mechanic to get the problem expertly diagnosed and fixed. Apparently, some people in the world would rather throw a few grenades at the car and be done with it. Academic science may have problems, but the solution is not to blow it up.

Science has an economics problem, which I define in three ways: 1) there is a shortage of funding for science; 2) scientific research has very real and significant resource costs beyond financial, including time, talent, environmental, and other opportunity costs; 3) there are no accurate measures of productivity in science. Here, I focus on the third problem because without accurate measures of productivity, the meritocratic distribution of scarce resources is impossible.

Can we develop an accurate measure of a scientist’s productivity? There are more qualified people who can answer this question, but as a practicing biomedical researcher, I see three main reasons why this is difficult:

1. We do not have a good way of measuring scientific output. We can and do judge a scientist’s contributions based on the number of papers they publish, the number of papers in high impact journals they have, the number of citations their papers receive, or their H index, but none of these are ideal indices for scientific output. Publications, even those in high impact journals, often do not change the course of science in any meaningful way and quickly become obsolete. Authorship on a particular publication is a poor indicator of how much someone contributed to the ideas and findings presented in that publication. Additionally, the number of citations a publication receives does not take into account the quality of those citations.
2. We do not have a good way of measuring opportunity cost. The amount of resources put into one study may produce a publication in a high impact journal, but if reallocated, the same resources may fund an even more impactful study. How can we know whether we are allocating resources efficiently? Also, what are these scarce resources, and how do we quantify them? Funding is probably the main one and is relatively straightforward to quantify. However, there are others that are more abstract, including institutional resources, social/networking opportunities, and the talent and hard work of trainees and other contributors.
3. We do not really know or agree on what productivity in science is, much less how to measure it. When it comes down to it, what really matters to scientific progress? A series of works each providing incremental progress can certainly lead to the development of novel medicines, but perhaps ultimately the only things that matter to science are paradigm shifts. How do we measure the impact of “incremental work” against “paradigm shifting work?” How do we define and identify breakthroughs?

In a series of posts under the category “Scientific Productivity Indices,” I will introduce a few of my more-or-less theoretical indices aimed at measuring true scientific productivity. Some of these are thoughtful; some may be rather whimsical. Subscribe to my blog now to follow this series!

This post is part of a series on scientific productivity indices. Check out the rest of the series here.