Citation analysis is a common tool used to trace scholarly research, measure impact (trying to quantify the quality!), and justify academic tenures, promotions, and funding decisions (especially subjective decisions). As on date, several citation indices are available, like the Science Citation Index (SCI) from Thomson Reuters. Hence a natural question arises- of all citation indices, which one should be used to measure the quality of a journal?

In fact Thomson Reuters itself offers several citation indices; the SCI covering 3,769 journals, a larger version Science Citation Index Expanded covering 6,650 scientific and technical journals across 150 disciplines., Social Sciences Citation Index covering 2,474 social sciences journals across 50 disciplines, Arts & Humanities Citation Index covering 1,395 arts and humanities journals, and so on [1].

Even though SCI is widely used, it does not necessarily include several journals of repute, even in the fields of science and engineering itself. Since a university produces research papers in all fields, the basis for using SCI alone to judge the quality of research (for the entire university) is questionable.

Further, several studies have indicated the shortcomings of SCI, while trying to quantify the quality. One such study concludes that "contrary to the implications of a low impact factor, an analysis of journal citations in Advanced Cardiac Life Support publications over two decades suggests that emergency medicine has made significant contributions to a broad and important area of scientific inquiry" [2].

An insight into the relevant happenings in China is unavoidable here:

Rao Yi, a renowned neurobiologist and the Dean of Peking University's College of Life Sciences opines that [3] the "reliance on SCI paper numbers and the status of a journal in which a paper is published has negatively impacted the quality of work produced by China's scientists. Most of China's 5000 science and technology journals are not included in SCI".

The zealous pursuit of SCI papers has led Suning, editor-in-chief of China Medical Association journals, to suggest the acronym SCI is short for 'Stupid Chinese Index'. Yigong, Deputy Dean of the School of Medicine of Tsinghua University says that "the rampancy of SCI in China is just a result of the dominance of administrative officials in the evaluation of academic research. The officials do not have enough professional knowledge so they have to rely on SCI". Hongfei of the Chinese Academy of Sciences Institute of Chemistry suggests that "the wide use of objective evaluation methods such as SCI in China is mainly because the evaluators want to find a pretext to justify their decisions" [4].

The criticism of the SCI in China first appeared in early 2000s. In May 2003, several government organizations, including the Ministry of Science and Technology, the Ministry of Education, the National Natural Science Foundation, and the Chinese Academy of Sciences, issued a joint circular to oppose the sole use of SCI papers to evaluate scientists [4].

Alternative citation indices:

Scopus: a citation database of research literature and quality web sources covering nearly 18,000 titles from more than 5,000 publishers (of which 16,500 are peer-reviewed journals, including over 1,200 Open Access journals).

Google Scholar: a scholarly literature database that includes peer-reviewed papers, theses, books, preprints, abstracts and technical reports available from academic publishers, professional societies, preprint repositories and universities, as well as those available across the web. Search results are displayed according to relevance.

Other Services currently offering cited reference searching [5]: Chemical Abstracts / SciFinder / SciFinder Scholar, NASA Astrophysics Data System Abstract Service, Scitation / Spin Web, PROLA (Physical Review Online Archive), Optics InfoBase, CiteSeer, Science Direct (for Elsevier publications), PsycINFO, IEEE Xplore, Spires HEP, IOP (Institute of Physics), CrossRef.

Comparisons and Conclusions:

A few studies have been made to find the alternatives to SCI. In view of lower number of Indian journals listed on SCI compared to Scopus, Jain [6] concludes that "Scopus has wider scope than SCI for Indian scenario. The coverage of Indian journals is the highest ever in Scopus compared to any international indexing service".

Bauer and Bakkalabasi [7] have conducted a case study of comparing the citation counts provided by SCI, Scopus, and Google Scholar for articles from the Journal of the American Society for Information Science and Technology (JASIST) published in 1985 and in 2000. SCI provided the largest citation counts for the 1985 articles, although this could not be tested statistically. For JASIST articles published in 2000, Google Scholar provided statistically significant higher citation counts than either SCI or Scopus, while there was no significant difference between SCI and Scopus. They recommended that the researchers should consult Google Scholar in addition to SCI or Scopus, especially for a relatively recent article, author or subject area. Consulting Google Scholar may prove most useful for disciplines such as physics, where nontraditional forms of publishing are widely accepted.

The changes in scholarly communication, including preprint / postprint servers, technical reports available on the internet, and open access e-journals are developing rapidly, and traditional citation tracking using SCI may miss much of this new activity. Given the changes in scholarly communication induced by technology and the corresponding proliferation of resources that offer citation tracking, it is imperative that a rigorous study be undertaken to determine which sources perform best for particular subjects or time periods, instead of using SCI alone [7].

References:

1. http://thomsonreuters.com

2. Barnaby D.P. and Gallagher E.J., "Alternative to the Science Citation Index impact factor as an assessment of emergency medicine's scientific contributions", Annals of emergency medicine, Vol. 31, No. 1, pp. 78-82, 1998.

3. "Young scientist: Beware when SCI can harm you", http://www.sciencenet.cn.

4. Hepeng Jia, and Tao Sun, "Science Index a target for criticism in China", Chemistry World, Issue August 2009, http://www.rsc.org.

5. Roth D.L., "The emergence of competitors to the Science Citation Index and the Web of Science", Current Science, Vol. 89, No. 9, pp. 1531-1536, 2005.

6. Jain N.C., "Scopus has wider scope than Science Citation Index", Current Science, Vol. 88, No. 3, pp. 331, 2005.

7. Bauer K, and Bakkalbasai N., "An Examination of Citation Counts in a New Scholarly Communication Environment", D-Lib Magazine, Vol. 11, No. 9, 2005.