Criteria for quality assessment of medical information resources.

Marie Monik, Christine Wickman
Karolinska Institutet Library
Box 200 - 171 77 Stockholm, Sweden

Click here to see the ppt presentation

Today it's easy to find information, but can we find the "right" information? What about quality?

Looking at how endusers are performing their searches, we discovered a need for discussing quality criteria of different information resources. This topic became part of our mandatory graduate course. We are also running a course for librarians and information specialists, a course attended just for presentation and discussion of quality criteria and for help in preparing courses for endusers. Here we would like to focus on some of the criteria and questions we are considering.

Information can be provided in different ways: Oral information, printed information in books and journals, online information in databases and on the Web. Every time we are evaluating, we should have in mind the following five criteria: Accuracy, Authority, Objectivity, Currency and Coverage. Printed information is easy to transmit, there is no need for a personal contact as in oral information. Printed information is providing access to detailed information, it is retrievable later on, archiveable and citeable. The academic credit you get by publishing a scientific articleis not to be forgotten! The quality of printed information can be improved by following certain rules, "Instructions to authors". The specific structure of a scientific article makes it easy to orientate within the article. The peer-review system guarantees to find validity. To compile the reference list in a proper way is very important - just remind the citing-process. The accessiblity of information is part of the quality assessment process as well. To find references to the printed articles in international bibliographic databases helps to spread research results and to make them known worldwide. Publishers of medical journals are interested to get their journals indexed for Medline. NLM and its Literature Selection Technical Committee (LSTR) are selecting biomedical journals by using specific quality criteria. About 4 300 journals are covered by Medline, but there are about 15 000 biomedical journals alltogether. That means, a lot of information is not included in Medline! You have to search other databases as well!

Impact factors and the importance of publishing in the right journal, in journals with high impact factors has be give great attention. The database Journal Citation Reports provides a lot of information about the journals selected for Science Citation Index or Social Science Citation Index including publisher information and impact factors. The impact factor is calculated for all articles together not for a specific article published in the journal. Many researchers are reading review journals and biomedical journals publishing articles on molecular biology and genetics eg. They will cite the articles in their own research work, that means a lot of citations, that means high impact factors for the journals the articles are published in. Articles published in a specific clinical journal e g will not have that large number of readers and regarding to that the number of citations will be lower. But is the quality of an article published in those journals so much lower? Important to look up and compare impact factor within subject categories!

To find the right database, to find quality, to find the evidence is crucial today. The need of information has to be translated into search terms. By finding the correct terminology the search will be more cost-effective and the search result more relevant. That has a big impact on preclinical research, clinical practice and different decision making processes in which search results will be used. Searching with standardised terms like MeSH and Emtree means searching independently of the way the author/s described the topic. Just imagine to search for "human" by using free text! Using combinations with aspect words and publication types will increase the relevance of the result and the number of references found will be more reasonable too. According to our experience, the knowledge of standardised terms and their usefulness has to be improved. Endusers, both researchers and students, are using free text-searching mostly. One of the advantages with PubMed is just to type in search terms. The program will interpretate the terms and provide a result. Use the Details-button for information about how the search was performed! Very often a long list of references will be presented, often users will just read the first appearing references. If many of the important references were published earlier, they will not find them. We experienced that the use of MeSH-terms and different ways of limitation should have increased the relevance of the result a lot. To limit with "Randomized controlled trial" will find references to good clinical studies. Looking for a systematic review in Cochrane Library can help clinicians to find evidence and - together with other information - the best way to treat patients. By using Meta-analysis for their systematic reviews the power of studies, each to small to be reliable individually, can be increased. Cochrane Library can also be used as a bibliographic database for good quality clinical studies, something we often forget about.

The Web as a huge information resource is offering a lot of "good" information. For evaluation of the resources try to find out when the information was published and if it was updated. Who is the producer and whom is it aimed at? There are different ways of writing and structuring information resources aimed at different target groups. The first impression of a web site can result in just passing the site because it looks not serious, not professional done, despite the content of the page. The URL is of great help when looking for good quality. The suffixes com, gov and edu will mostly guarantee good information. Using link lists provided by established web sites helps to find quality information. The link lists of the Karolinska Institutet Library, including about 20 000 links to biomedical information, are used very much worlwide.

Finally we would like to focus on something to think of and to teach our users: To search PubMed only is not enough, use several information resources. Standardized terms and limitation possibilities will increase the quality of the search result. Evaluate the result and be critical. To improve quality we not only need to focus on different criteria and ways of finding good information resources but also to increase the production of high quality information resources. Database producers, database hosts, publishers and portal builders must take their responsibility. User-friendly effective search programs and good access probabilities have to be improved - not only for databases, but even for electronic journals, books and other fulltext-information. We as information specialists can help users to become better searchers. We should also take responsibility in collecting and structuring information within specific subject areas. Our competence is needed!