Systematic reviews and meta-analyses: a step-by-step guide | www.ccace.ed.ac.uk

Step 6

Critical appraisal of studies (quality assessment)

There is no consensus on the best way to assess study quality, but most methods encompass issues such as:

– Appropriateness of study design to the research objective

– Risk of bias

– Other issues related to study quality:

         – Choice of outcome measure

         – Statistical issues

         – Quality of reporting

         – Quality of the intervention

         – Generalisability

The consensus reporting guidelines for different study designs proposed by EQUATOR (http://www.equator-network.org/ ) are a useful starting point, but note these are guidelines for reporting of original studies, NOT for assessment of study quality. 

STROBE (http://www.strobe-statement.org)  also provides useful guidelines for good reporting of observational research, including checklists of items that should be included in this type of research.

Useful resources for assessing quality of different study designs can be found here, and some specific examples are QUADAS for studies of diagnostic accuracy.

This article by Sanderson, Tatt and Higgins (2007) provides a review of the wide range of tools used to assess study quality. It does not recommend any specific tool for general use, but lists the domains which should be included [1) appropriate selection of participants 2) appropriate measurement of variables and 3) appropriate control of confounding, as well as considering design specific biases]. You may need to develop your own quality assessment tool, but do seek advice on the best method of quality assessment for your review.

This articly by The Cochrane Collaboration describes a tool they developed for assessing risk of bias in random trials http://www.bmj.com/content/343/bmj.d5928.full.pdf

Move to Step 5         Move to Step 7                    Main Menu