LibGuides: Literature Reviews: systematic searching at various levels: Screening &amp; critiquing

Being Critical

Screening may sound a little daunting, like a medical procedure, but it's something that you're likely to need to do because it's rare to do a search for a topic and only get a few results. It's more likely, especially when you're working on longer assignments, dissertations, and theses, that you'll still have hundreds (if not thousands) of results in your final list from the database.

The databases can only base your results on what you've asked them to look for and exclude (so your keywords that have to be present; date range; population groups; language; peer reviewed etc.) so it's not unusual to have an article in your results list that does contain your key terms but the overall context of the article is not relevant to your research - so you need to 'screen' out those articles.

Whatever level of study you're engaged in, the process is the same.

Step 1 - Look at the article title and the abstract

In those two locations you should have enough information to help you decide whether it's worth your while to read the article in full or not. The abstract should be able to confirm whether the article is based on primary research or not; what type of research was conducted and how; and what the conclusions of the research were.

The abstract is a distillation of the article's content; its purpose is to hit the high points to convince you to read it so it's a good guide as to whether it contains useful information or not. Sometimes there will be records that don't have an abstract so you will need to go through to the full text of the article in order to screen it out / in to your list of papers that you'll be discussing in your research, but most records will have an abstract either within the record itself or, sometimes in Library Plus, there'll be a link out to another database where the abstract is held.

You should be able to remove a lot of articles from your list of results in this title / abstract stage of the process.

Step 2 - Read the full-text of the articles you think are relevant

Don't be surprised when, as you read (or speed search) an article you sometimes decide it isn't relevant after all. It's okay to exclude an article at this point if the context isn't correct.

CASP Checklists
This set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
They are of use to many subject areas, not just health, as the critiquing is based on the design of the study.

These are free to download and can be used by anyone under the Creative Commons License.
Mixed Methods Appraisal Tool (MMAT) Version 2018
The MMAT is a critical appraisal tool that is designed for the appraisal stage of systematic mixed studies reviews, i.e., reviews that include qualitative, quantitative and mixed methods studies. It permits to appraise the methodological quality of five categories to studies: qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed methods studies
Critical Appraisal Tools from CEBM
This link will take you to a collection of critical appraisal tools from the Centre for Evidence-Based Medicine at the University of Oxford.
It includes worksheets in different languages; PICO critical appraisal sheets; and an Explanations & Examples sections about: Pre-test probability, SpPin and SnNout, Likelihood Ratios, and NNTs
Tools for Critical Appraisal
This is a collection of resources on critical appraisal from Temple University and includes checklists, information about grading the strength of the evidence, and also critical appraisal checklists by study design type
Critical Appraisal Notes & Checklists from SIGN
This webpage has a series of links brought together by Healthcare Improvement Scotland and is part of SIGN (the Scottish Intercollegiate Guidelines Network)
Critical Appraisal Checklists from Cardiff University
This guide provides links to a wide variety of appraisal checklists under the headings of: Systematic literature reviews of primary research studies; Intervention experimental studies; Observational studies; Qualitative views and opinions studies; Diagnostic accuracy studies; Economic evaluation studies; Assessing a body of research with different study designs; Establishing study type
Feedback or suggestion
EBM Toolkit
From the website: "This is a collection of tools for identifying, assessing and applying relevant evidence for better health care decision-making. The appraisal tools are adapted from the Users' Guides series prepared by the Evidence Based Medicine Working Group and originally published in JAMA"

This is from the University of Alberta.
It was last updated in 2013

A grey literature checklist:

Tyndall, J. (2009) ‘AACODS Checklist’ Available from: https://dspace.flinders.edu.au/xmlui/handle/2328/3326 Accessed on: 02 April 2019

Greenhalgh, T. (2019) How to read a paper: the basics of evidence-based medicine and healthcare. 6th ed. Newark: Wiley.

Haidich A. B. (2010) 'Meta-analysis in medical research', Hippokratia, 14(Suppl. 1), pp. 29–37.

Hartling, L., Bond, K., Harvey, K., Santaguida, P.L., Viswanathan, M. & Dryden, D.M. (2010) 'Appendix G, Round Two Algorithm and Glossary', in, 'Developing and Testing a Tool for the Classification of Study Designs in Systematic Reviews of Interventions and Exposures', Rockville (MD): Agency for Healthcare Research and Quality (US)

Young, J.M. & Solomon, M.J. (2009) 'How to critically appraise an article', Gastroenterology & Hepatology, 6(2), pp.82-91. doi:10.1038/ncpgasthep1331.