Skip to Main Content

Systematic Reviews (2+ people)

This guide will look at resources for conducting a systematic review (SR) of literature on a topic

Grey Literature

There are many definitions of grey literature, but it is usually understood to mean literature that is not formally published in sources such as books or journal articles. Conference abstracts and other grey literature have been shown to be sources of approximately 10% of the studies referenced in Cochrane reviews (Mallett 2002). In a recently updated Cochrane methodology review, all five studies reviewed showed that published trials showed an overall greater treatment effect than grey literature trials (Hopewell 2007b). Thus, failure to identify trials reported in conference proceedings and other grey literature might affect the results of a systematic review.

(Higgins & Green, 2008)

 

“Grey literature stands for manifold document types produced on all levels of government, academics, business and industry in print and electronic formats that are protected by intellectual property rights, of sufficient quality to be collected and preserved by library holdings or institutional repositories, but not controlled by commercial publishers i.e., where publishing is not the primary activity of the producing body.” (Schöpfel, 2010)

 

 


Higgins, P.T. & Green, S. (2008) Cochrane Handbook for Systematic Reviews of Interventions. Chichester: Wiley.

Schöpfel, J. (2010) 'Towards a Prague Definition of Grey Literature', Twelfth International Conference on Grey Literature: Transparency in Grey Literature. Grey Tech Approaches to High Tech Issues. Prague, Czech Republic, 6-7 December. pp.11-26. Available at: https://archivesic.ccsd.cnrs.fr/sic_00581570/file/GL_12_Schopfel_v5.2.pdf Accessed: 05 April 2019.

Grey literature is generally not peer reviewed1, so you will need to evaluate the trustworthiness and likely accuracy of any grey literature you come across yourself.

One available tool for evaluating and critically appraising the quality of grey literature is the AACODS2 Checklist which is available from the Flinders University repository. There are two documents on the page and the second link is for the checklist.

Tyndall, J. (2009) ‘How low can you go? Towards a hierarchy of grey literature’. Available from: https://dspace.flinders.edu.au/xmlui/handle/2328/3326 Accessed on: 02 April 2019

Tyndall, J. (2009) ‘AACODS Checklist’ Available from: https://dspace.flinders.edu.au/xmlui/handle/2328/3326 Accessed on: 02 April 2019

 


1. Theses (PhD) could be considered the exception to this as they often go through a Viva process as well as being approved by more than one person who has knowledge and / or experience of that field of research or practice.

2. Authority --> Accuracy --> Coverage --> Objectivity --> Date --> Significance

 

  • Conference proceedings - this can include abstracts of posters, presentations, as well as the full papers themselves1
  • Clinical trials
  • Government reports (including Green and White papers)
  • Policy statements
  • Newsletters & leaflets
  • Fact sheets (from e.g. a G.P.s surgery / Library session or similar)
  • Research reports & their executive summaries
  • Theses (PhDs) and dissertations (postgraduate or undergraduate)
  • Maps (includes maps from geological / geophysical surveys)
  • Blog posts and other social media posts

Some material may be available electronically via the internet, some may only be obtainable directly from the source / author / organisation 

 


1 You may be able to find the abstracts for conference presentations and posters on the conference website although these are not guaranteed to be available indefinitely. There are a number of databases that index conference proceedings and, from our own collection, you can search: Scopus, the Web of Science, MEDLINE, PsycINFO. You may also find papers listed on Google Scholar (even if no full-text is available immediately) which could provide you with enough of a reference to request the item via the inter-library loan service.

Some conference proceedings may also be available on the digital repositories of the institution the author(s) of the paper work at, and a Scholar search may reveal those papers.

General

 

System for Information on Grey Literature in Europe

BASE - 'BASE is one of the world's most voluminous search engines especially for academic web resources. BASE provides more than 140 million documents from more than 6,000 sources. You can access the full texts of about 60% of the indexed documents for free (Open Access). BASE is operated by Bielefeld University Library.'

Conference proceedings

These can often be found in databases - whether discipline specific or general.  Proceedings may also be referred to as congresses or meeting papers. If you know that an organisation regularly holds conferences you can check their websites to see if they've put the proceedings or the abstracts online.

 

British Library Catalogue

WorldCat

Zetoc -  'over 35,200 journals and more than 58 million article citations and conference papers through the British Library’s electronic table of contents and over 875,000 OA article citations and conference papers from PubMed.'

Databases available through the University Library that may hold this information:

SCOPUS

Web of Science

MEDLINE

PsycINFO

CareKnowledge

 

Clinical trials

 

Cochrane

TRIP Medical Database (free version)

WHO International Clinical Trials Registry Platform

Medical / health related trials & research registers (international listings)

ISRCTN Clincal Trial Registry

ClinicalTrials.gov (NIH - U.S. National Library of Medicine)

EU Clinical Trials Register

* You may also find information on trials on industry websites e.g. pharmaceutical companies for trials related to their products. You can search for these by looking on Google for your key terms and then using the site: feature to limit your results to the company's domain name so you only get results from those web pages.

Practice Guidelines

Government reports (including Green and White papers)

 

UK Data Service - Provides data from national statistical authorities, UK government departments, intergovernmental organisations, research institutes. You will need to register for free to access this service. 

Government publications (Command papers)

Bills before Parliament

Select Committee Reports (House of Commons & House of Lords)

Acts of Parliament

 

Policy statements

 

NHS England

 

Theses (PhDs) and dissertations (postgraduate or undergraduate)

 

EThOS - British Library Electronic Theses Online

Open Access Theses and Dissertations

 

Maps (includes maps from geological / geophysical surveys)

Open Access

Some open access resources may also contain grey literature alongside resources that have gone through a formal publication route. You may also find that institutional repositories (i.e. a university's online, open access portal) contains details / content of grey literature produced by staff members at that institution. Not everything that comes up here will be grey literature as some of it will have been formally published. If you're not sure, please contact the Library.

arXiv - 'Open access to 1,547,252 e-prints in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics. Submissions to arXiv should conform to Cornell University academic standards. arXiv is owned and operated by Cornell University, a private not-for-profit educational institution. arXiv is funded by Cornell University, the Simons Foundation and by the member institutions.'

Open Science Framework Preprints

Earth ArXiv preprints - Earth Sciences preprints

ESSOAr - Earth and Space Science Open Archive

MarXiv - Ocean and marine-climate sciences repository

 

2