The electronic edition of scientific journals may influence the determination of their own Impact Factors?

Moreno Curti, Vanna Pistotti*, Gabriella Gabutti, Andrea Zeccato, Carmine Tinelli, Catherine Klersy
Direzione Scientifica - IRCCS Policlinico San Matteo, Pavia
*Gustavus A. Pfeiffer Memorial Library, Istituto di Ricerche Farmacologiche "Mario Negri", Milano (Italy)
curtim@smatteo.pv.it


Click here to see the ppt presentation


Background

Many journals have now a web site but the full text of articles are rarely available at no charge. The "typical" electronic version of a printed journal offers today only a table of contents and abstracts, usually with no restriction, while the view of full text is restricted to their subscribers, site licensees or to those using the pay-per-view system. A few of the most known journals publish only electronic version of an article and this one will never appear in the print version. Some have drawn up contracts with third parties. Packages of full-text journals are offered through an annual subscription, even when the library does not pay for a printed copy: the full text access is free, better say it appears free, to users on a campus or consortium network. Many database distributors like Ovid, SilverPlatter, Dialog have now linked their bibliographic records to full text articles which are immediately available when their users subscribe to that specific journal. A sort of "one-stop-shop" of the medical information.

Among biomedical journals those classified by the Institute for Scientific Information (ISI) in Philadelphia, are the most relevant. Their quality is related to the "impact factor", a term coined by Eugene Garfield, the inventor of the Science Citation Index, and Irving H. Sher in the early sixties. In 1955 Garfield suggested that reference counting could measure "impact" but the term "impact factor" was not used until the publication of the 1961 Science Citation Index (SCI) in 1963. This led to a byproduct, Journal Citation Reports (JCR), and a burgeoning literature using bibliometric measures. The current impact factor is given by the ratio of the number of citations of the journal over one year (e.g. in 1999) to the number of articles published by the same journal in the two years before (e.g. in 1997-1998). These considerations raise some questions: is the availability of the electronic version of a journal associated with the value of the impact of the same publication? Do changes in impact factor correspond to increased availability of the journal on the Internet?
 

Methods 

This was a retrospective longitudinal study. The availability of the journal on the Internet, in the different years considered (1995 to 2000), was assessed transversally between December 2000 and January 2001. The presence of table of contents (TOC), abstract, full text and free full text were evaluated. The journal considered were those ranked among the first 20 of each of the following subject categories in year 1998: cardiac and cardiovascular system; medicine, general & internal; medicine, research & experimental; multidisciplinary sciences; othorinolaryngology; surgery; hematology and transplantation. Changes in the impact factor over the years were modelled (at univariate and multivariate analysis) by means of a linear regression model, with calculation of Huber-White robust standard. Google.com was utilised as a search engine to identify journal web sites. Only sites with free access were included, where no subscription to journal or package of full text journals was required to enter the site.
 

Results

Overall 118/139 journals (85%) had the TOC available on the Internet, of these 107 (77%) had the abstract, 97 (70%) had the full text and 33 (24%) the free full text. Median impact factor for all the considered journals was 1.65 (0.84-2.80), 2.08 (1.00-3.51), 2.10 (1.20-3.44), 2.21 (1.63-3.65) and 2.35 (1.48-3.56) for years 1995 to 1999 respectively. Changes of impact factor in time appeared to be statistically significant, at regression analysis. Changes over years were different among subject categories. Regression models also showed the presence of TOC, abstract and full text on the Internet to be significantly associated with the impact factor, even when time and subject category were accounted for higher impact factors were observed when the journal had an electronic version.
 

Conclusions

The impact factor has been used for years to evaluate journals quality. In many research and academic settings, the impact factor is also used to fund research based on the evaluation of the quality of the several groups of researchers. Thus, it is particularly relevant to be aware of the appropriate use of the impact factor and to understand its limitations. Many factors contribute to the number of citations per paper, including the science; the nature of the research; the style of the communication (papers vary in the number of references they include, from none to over 100); editorial policy (several journals ration the number of references per article); circulation and readership (more readers equals more citations); citation bias (comparable to submission bias and publication bias); conformism (scientists often cite those papers that are currently cited); authors' tendency to cite their own work; and referees' self promotion (referees' tendency to recommend inclusion of references to their work). In addition to those already reported, a further limitation of this indicator appears here, where impact factor values would be related to the amount of circulation of the information through the Web. However this could be a temporary limitation, related to diffusion of journals on the Internet and Internet itself . 
 

Selected References
 

  • Journal Citation Reports Science Edition. Philadelphia: Institute for Scientific Information 1995 to 1999.
  • Garfield E. Citation indexes for science: a new dimension in documentation through association of ideas. Science 1955;122:108-11.
  • Amin M, Mabe M. Impact factors: use and abuse. Perspectives in Publishing 2000:1:1-6.
  • Curti M, Klersy C, Tinelli C, Visentin S. Le role d'une bibliothèque scientifique dans l'évaluation de la recherche dans le cadre d'un Institut Scientifique pour l'hospitalisation et les soins (IRCCS). In: Health Information - New Possibilities. London: Kluwer Academic Publishers, 1995:134-6.
  • Gallagher EJ, Barnaby DP. Evidence of methodologic bias in the derivation of the Science Citation Index impact factor. Ann Emerg Med 1998;31:83-6.
  • Garfield E. Fotnightly Review: How can impact factors be improved? BMJ 1996;313:411-3.
  • Garfield E. Citation analysis as a tool in journal evaluation. Science 1972;178:471-9.
  • Garfield E. Journal impact factor: a brief review. CMAJ 1999;161:979-80.
  • Joseph KS, Hoey J. CMAJ's impact factor: room for recalculation. CMAJ 1999;161:977-8.
  • Hecht F, Hecht BK, Sandberg AA. The journal "impact factor": a misnamed, misleading, misused measure. Cancer Genet Cytogenet 1998;104:77-81. 
  • Opthof T. Sense and nonsense about the impact factor. Cardiovasc Res 1997;33:1-7.
  • Tsay MY. The relationship between journal use in a medical library and citation use. Bull Med Libr Assoc 1998;86:31-9.