The Distinct Citations Index

In my post “Science Has an Economics Problem,” I noted how the number of citations a published scientific article receives is not a true indicator of the study’s impact and says nothing about the quality of those citations.

For instance, an article may accrue hundreds of citations shortly after publication because it deals with a hot, trendy topic. The problem is, by the very nature of being in a hot, trendy area, the article is less likely to contribute significantly to real, original scientific progress, as a lot of other scientists are working on the same or similar problem. An article in a more niche field may make or contribute to significant scientific advancements but go unrecognized immediately after publication. For instance, one of Katalin Karikó’s initial works investigating the use of mRNA for therapeutic protein delivery, published in 2001¹, which eventually paved the way for the 2023 Nobel Prize in Physiology or Medicine, only received one citation in its first eight years post publication (and only 17 total according to Scopus). Citations are fundamentally flawed.

Citation count by year of Karikó et al., 2001 (image taken from Scopus)

Furthermore, citations are often bundled, meaning that when an article is cited by another study, it is frequently cited alongside other articles that show a similar finding or make a similar point. As an example, this statement has three bundled citations²^,³^,⁴. Bundled citations, I would argue, are worth a lot less than standalone, or distinct, citations.

There are many ways to improve the measurement and use of citations as an index of scientific impact. Here, I will just introduce one: the Distinct Citations Index. This is an index that attempts to measure whether a scientific article is derivative, unoriginal, or redundant. An article is considered to be these things if all its citations are bundled with articles that were published before it. To calculate this index, let us define Distinct Citations (δC), which is given by the following equation:

δC = C – φ

where

C = the total number of citations

φ = the number of citations where the article is only bundle-cited with article(s) published before it

Distinct Citations Index (δCI) is then calculated as follows:

δCI = δC / C

An article where δCI = 0 is redundant or derivative, and δCI = 1 is considered highly original.

There are certainly many ways to improve this index. For instance, φ can be redefined as the number of citations where the article is only bundle-cited with article(s) published before it or within one year of its publication. This would account for articles that scooped their competitors by a short amount of time. If this article had never been published, the overall impact on science would likely be minimal.

Like this index? Please let me know your thoughts!

This post is part of a series on scientific productivity indices. Check out the rest of the series here.

References

Karikó et al. 2001. In vivo protein expression from mRNA delivered into adult rat brain. J Neurosci Methods 105(1):77-86. doi: 10.1016/s0165-0270(00)00355-1. ↩︎
Fake citation a ↩︎
Fake citation b ↩︎
Fake citation c ↩︎