Claire Stansfield
Evidence for Policy and Practice Information and Coordinating (EPPI-) Centre
Social Science Research Unit, UCL Institute of Education, University College London
Corresponding author: Claire Stansfield, c.stansfield@ucl.ac.uk
Abstract
Background
Identifying relevant research evidence to support decision-making in the field of social care is not often straightforward. A programme of systematic reviews on a range of social care topics undertaken to inform UK NICE guidelines provided the opportunity to reflect on experiences in developing search strategies and analysing where relevant studies were found. This is informative for supporting future systematic reviews.
Objectives
To assess the utility of different bibliographic databases sources for identifying research for three topic areas related to the care and support of people within social care. To reflect on experiences in seeking social care literature, and on identifying literature from websites and specialist databases.
Methods
A retrospective analysis was undertaken to determine which sources were useful in identifying the studies included in the evidence reviews, in terms of the numbers studies and precision of searches. Comparisons were made for the search sources across the three reviews for different types of study (systematic reviews, controlled trials, qualitative studies, and other designs).
Results
Twenty-six systematic reviews; over 40 controlled trials and over 100 qualitative studies were located across a range of international medical and social science bibliographic databases. Data is presented for each type of search source for each review and collectively. The value of supplementary searching websites and specialised registers is also considered. We also reflect on the challenges of searching for research in the area of social care and social services.
Conclusions
This study highlights challenges and informs practice in identifying social care research.
Keywords: databases, bibliographic; social work; social welfare; systematic reviews; Review Literature as Topic; Information Storage and Retrieval;
Introduction
A programme of systematic reviews on a range of social care topics undertaken to inform three UK NICE guidelines provided the opportunity to reflect on experiences in developing search strategies and analyse where relevant studies were found. The reviews, undertaken 2013 to 2015, relate to the care and support of people across three separate topics:
- Home care for older people
- Transition between mental health hospital and community settings
- Transition between child and adult health and social care services
Each topic comprised of at least seven sets of systematic review questions that relate to what interventions work and what are people’s experiences of them, and barriers and facilitators to specific interventions. The literature search utilised a range of international and UK health, social science and social care and economic bibliographic databases, website searches and specialist registers and catalogues (mainly UK focussed), some citation searches and contributions from the guideline development team. The search strategies, and reports of the reviews are reported elsewhere [1,2,3].
Conceptualising the review questions into a search:
Each literature search was designed to cover all the review questions within a single topic as the questions related to common populations and settings. The three searches were conceptualised based on the population, settings and context.
Key challenges of designing search terms for these three topics:
- Population and settings: encompassed individuals to organisation-level
- Population and setting become merged (e.g. mental health patients and mental health units in a hospital)
- Identifying research about care and support, and reducing volume of less clinical studies
- Terminology is diverse, and controlled vocabulary sometimes very broad.
Retrospective analysis of bibliographic databases sources
1) Methods:
- Where the evidence searches identified studies
For two topics (child to adult transitions, and mental health setting transitions) the 152 citations included in the systematic reviews were checked against the results retrieved from the searches prior to duplicate checking to determine which sources they were obtained from. Precision was calculated as the proportion of studies collected from the database that were included in a systematic review.
- Where the studies exist in the databases
For the 225 citations from all the topics, title searches were undertaken across five database platforms (OVID, Proquest, EBSCO, Social Science Citation Index, British Nursing Index) in order to see where the literature could be found from:
- EBSCO: British Education Index, CINAHL Plus, Econlit, ERIC, MEDLINE
- OVID: AMED, Embase, Medline, Psycinfo, Health management information consortium (HMIC), Social Policy and Practice (SPP)
- HDAS (British Nursing Index)
- Web of Science (Social Sciences Citation Index)
- Proquest (ASSIA, ERIC, IBSS, LISA, PAIS, PILOTS, Sociological Abstracts, Social Services Abstracts, Worldwide Political Science Abstracts
EPPI-Reviewer 4.6 was used to assign codes for each citation which related to: the review question, study designs, which database the study exists in, where the searches were found. Frequency analysis, and cross-tabulation of codes, and searching within codes was undertaken to analyse the data. Determining the combinations of databases to find the citations was based upon selecting the highest ranking study, eliminating those citations, and assessing the highest rankings for the remainder.
2) Key findings:
Performance of the evidence review searches
Mental health setting transitions: Precision of the searches ranged from 0.6% to 3.4%, with a median of 1.1%. The ability of the searches to identify all the studies present ranged from 45% to 76%, with a median of 57%.
Child to adult transitions: Precision of the searches ranged from 0.6% to 3.4%, with a median of 1.1%. The ability of the searches to identify all the studies present ranged from 44% to 95%, with a median of 76%.
The proportion of studies sourced outside of databases varied across topic: 6% (mental health transitions), 14% (child to adult transitions) 23% (homecare).
Utility of sources
There is a large amount of overlap in the studies that are present across databases, however, this overlap is less pronounced in terms of the studies that were found from the evidence searches.
CINAHL Plus, EMBASE, HMIC, MEDLINE, Psycinfo, SPP and Social Sciences Citation Index, provided 96% (n=215/225) of studies. Of the remaining ten studies: nine were not present in any database, and one study related to school education re-entry present in the education database ERIC.
In terms of sourcing studies from our evidence searches for two topics, this combination located 130/152 (86%) of studies. However, if the British Nursing Index is also added to this combination, an additional 5 studies are located – with an overall yield of 89% (135/152). The remaining 17 items were found from other sources largely outside bibliographic databases (websites, trials registries, Zetoc, and one from ERIC).
Utility of sources – individual topics (items present only)
The individual databases that contained the most studies varied across topics. Homecare: SPP=75% of studies (n=55/73); Mental health setting transitions: Psycinfo =90% of studies (n=64/71); and for Child to adult transitions: SSCI, CINAHL, EMBASE each contained 52% of studies (70% collectively (n=57/81)).
Utility of sources – by study design (items present only)
The combination of databases that contain the most studies by study design, shown in table 1, varies according to study design across the three topics.
Table 1: Sources present within selected databases, by study type
| Study design (N=225) | Cost n=15 | Systematic Reviews n=26 | Views and Experiences n=123 | Effectiveness n=59 | Other n=13 |
|---|---|---|---|---|---|
| Total no. of citations retrieved by combination | CINAHL Plus, HMIC, Embase or Medline 15 (100%) | SSCI, Psycinfo, SPP 25 (96%) | CINAHL Plus, HMIC, SPP, SSCI, 113 (92%) | CINAHL Plus, Embase, SSCI, SPP 59 (100%) | Embase or Medline, and SPP 13 (100%) |
| Not present in a database | 1 | 8 |
Conclusions
This study highlights challenges of identifying social care research from database sources. There is wide variation in the ability of the search terms to capture the studies from individual databases, even with low precision searches. However, this is mitigated by searching a combination of databases and searching other resources and websites that are specific to individual topics. We identified a combination of eight databases that were important for finding literature. A matrix of database sources and study design is potentially useful in providing a picture of where certain types of studies are located, though some sample sizes are low.
Some limitations: Not all sources used in the reviews were analysed in this study. There were differing date limits across review questions, and the majority of views and experience studies were intentionally selected from the UK. Two studies were present in databases initially, but not during analysis.
REFERENCES
- Transition from children’s to adults’ services for young people using health or social care services: NICE guideline: February 2016. Available from: https://www.nice.org.uk/guidance/ng43/evidence/full-guideline-2360240173
- Home care: delivering personal care and practical support to older people living in their own homes NICE guideline: full version, September 2015. Available from: https://www.nice.org.uk/guidance/ng21/evidence/full-guideline-489149245
- Transition between inpatient mental health settings and community or care home settings. Draft for consultation: March 2016 Available from:
https://www.nice.org.uk/guidance/indevelopment/gid-scwave0711