Skip to Main Content

Health Research Reporting Guidelines, Study Execution Manuals, Critical Appraisal, Risk of Bias, & Non-reporting Biases

Note about the resources

All materials in this section are available for free. Check your institutional library for additional manuals and handbooks that are available in print or online.

Are you planning on conducting a systematic review?

Take a look at the HSLS Systematic Review LibGuide before you get too far into your project. Rather than relying on PROSPERO to create your protocol, take advantage of our PRISMA-P template to create it; you will have to use PRISMA to write your manuscript so this will save you a few steps!

Are you considering a systematic review?

Meeting the review family: exploring review types and associated information retrieval requirements

Article in:  Health Info Libr J. 2019 Sep;36(3):202-222. doi: 10.1111/hir.12276. PMID: 31541534.
Open access?: No, login required

"This study aims to characterise health related reviews by type and to provide recommendations on appropriate methods of information retrieval based on the available guidance."

Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks (PubMed link)

Article in: BMC Med Res Methodol. 2019 Nov 4;19(1):203. doi: 10.1186/s12874-019-0855-0
Open access?: Yes, freely available

"The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition."

Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches (PubMed link)

Article in: J Clin Epidemiol. doi: 10.1016/j.jclinepi.2018.03.003.
Year of publication: 2018
Open access?: No, institutional or personal subscription required

Objective: To compare and contrast different methods of qualitative evidence synthesis (QES) against criteria identified from the literature and to map their attributes to inform selection of the most appropriate QES method to answer research questions addressed by qualitative research.

Systematic Reviews

Cochrane Handbook for Systematic Reviews of Interventions

Produced by: The Cochrane Collaboration
Year of publication: Online edition updated regularly
Open access?: Yes, freely available

The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions. All authors should consult the Handbook for guidance on the methods used in Cochrane systematic reviews. The Handbook includes guidance on the standard methods applicable to every review (planning a review, searching and selecting studies, data collection, risk of bias assessment, statistical analysis, GRADE and interpreting results), as well as more specialised topics (non-randomized studies, adverse effects, complex interventions, equity, economics, patient-reported outcomes, individual patient data, prospective meta-analysis, and qualitative research).

  • Methodological Expectations of Cochrane Intervention Reviews (MECIR)
    • Standards for the conduct and reporting of new Cochrane Intervention Reviews, reporting of protocols and the planning, conduct and reporting of updates
    • Ensuring that Cochrane Reviews represent the highest possible quality is critical if they are to inform decision making in clinical practice and health policy. Methodological Expectations of Cochrane Intervention Reviews (MECIR) are Standards that should guide the conduct and reporting of Cochrane Intervention Reviews. They are drawn from the Cochrane Handbook for Systematic Reviews of Interventions (the ‘Handbook’). The development of the Standards has been a collaborative effort over several years, involving review authors, editors and methodologists from all corners of our community. In this document we present a complete set of Standards for intervention reviews.

Methods Guide for Effectiveness and Comparative Effectiveness Reviews

Produced/Managed by: Agency for Healthcare Research & Quality
Year of publication: 2017 plus updates
Open access?: Yes, freely available

"Effectiveness and Comparative Effectiveness Reviews are systematic reviews of existing research examining the effectiveness, comparative effectiveness, and comparative harms of different healthcare interventions. They are intended to provide relevant evidence to inform real-world healthcare decisions for patients, providers, healthcare systems, and policymakers, among others. In an effort to improve the transparency, consistency, and scientific rigor of the work of the Effective Health Care (EHC) Program, through a collaborative effort, the Agency for Healthcare Research and Quality (AHRQ), the Scientific Resource Center, and the Evidence-based Practice Centers (EPCs) have developed a Methods Guide for Comparative Effectiveness Reviews."

JBI Manual for Evidence Synthesis

Produced/Managed by: Joanna Briggs Institute (University of Adelaide, Australia)
Year of publication: Updated regularly
Open access?: Yes, freely available

The JBI Manual for Evidence Synthesis is designed to provide authors with a comprehensive guide to conducting JBI systematic reviews. It describes in detail the process of planning, undertaking and writing up a systematic review using JBI methods. The JBI Manual for Evidence Synthesis should be used in conjunction with the support and tutorials offered at the JBI SUMARI Knowledge Base.

The manual includes:

  • Chapter 1: JBI Systematic Reviews
  • Chapter 2: Systematic reviews of qualitative evidence
  • Chapter 3: Systematic reviews of effectiveness
  • Chapter 4: Systematic reviews of text and opinion
  • Chapter 5: Systematic reviews of prevalence and incidence
  • Chapter 6: Systematic reviews of economic evidence
  • Chapter 7: Systematic reviews of etiology and risk
  • Chapter 8: Mixed methods systematic reviews
  • Chapter 9: Diagnostic test accuracy systematic reviews
  • Chapter 10: Umbrella reviews
  • Chapter 11: Scoping reviews
  • Chapter 12: Systematic reviews of measurement properties

Handbook for Conducting Systematic Reviews of Health Effects Evaluations

Produced/Managed by: US National Toxicology Program Office of Health Assessment and Translation (OHAT)
Year of publication: 2019
Open access?: Yes, freely available
About: "Updates and clarifications to the handbook address two topics identified while conducting recent evidence evaluations: 1) the process for reaching hazard conclusions from human health data alone (i.e., in the absence of animal data or when there is low confidence in the available animal data), and 2) the process for developing confidence conclusions in the overall body of evidence across multiple outcomes, study types, or exposures."

Guidelines for Systematic Reviews of Health Promotion and Public Health Interventions (PDF)

Produced by: The Cochrane Collaboration Cochrane Health Promotion and Public Health Field
Date of Publication: July 2007
Open access?: Yes, freely available

The content has been prepared by health promotion and public health researchers, decision-makers and practitioners experienced with both the use and conduct of systematic reviews. While these guidelines were originally developed to support the conduct of systematic reviews, they are also important for the conduct of primary research and for more informal reviews of research evidence.

Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews

Produced by: National Health Service Centre for Reviews and Dissemination, University of York
Year of publication: 2009
Open access?: Yes, freely available

This guide covers: The methods and steps necessary to conduct a systematic review which are presented in a core chapter (Chapter 1). Additional issues specific to reviews in more specialised topic areas, such as clinical tests (diagnostic, screening and prognostic), and public health are addressed in separate, complementary chapters (Chapters 2-3). We also consider questions relating to harm (Chapter 4) costs (Chapter 5) and how and why interventions work (Chapter 6).

This guide focuses on the methods relating to use of aggregate study level data. Although discussed briefly in relevant sections, individual patient data (IPD) meta-analysis, which is a specific method of systematic review, is not described in detail. The basic principles are outlined in Appendix 1 and more detailed guidance can be found in the Cochrane Handbook11 and specialist texts.12, 13 Similarly, other forms of evidence synthesis including prospective meta-analysis, reviews of reviews, and scoping reviews are beyond the scope of this guidance but are described briefly in Appendix 1.

Finding What Works in Health Care: Standards for Systematic Reviews

Contributors: Institute of Medicine; Board on Health Care Services; Committee on Standards for Systematic Reviews of Comparative Effectiveness Research
Year of Publication: 2011
Open access?: Yes, freely available

In the Medicare Improvement for Patients and Providers Act of 2008, Congress directed the IOM to develop standards for conducting systematic reviews. In this report, the IOM recommends standards for systematic reviews of the comparative effectiveness of medical or surgical interventions (see the list of the standards). The standards are meant to assure objective, transparent, and scientifically valid systematic reviews.

Clinical Trials

Clinical Trial Policies, Guidelines, and Templates (US National Institute of Arthritis and Musculoskeletal and Skin Diseases)

Clinical trials must be conducted with a high standard of quality that assures the research question is answered in a reliable, valid, and unbiased manner, and that the rights and welfare of human subjects are protected. NIAMS has standardized procedures and provides templates to ensure compliance and the safety of human subjects.

Resources include:

  • Data and Safety Monitoring Guidelines and Policies
    • Guidelines for oversight and monitoring of clinical trials to ensure the safety of participants and the validity and integrity of the data;
  • Clinical Study Tools & Templates
    • Tools and templates for developing a manual of operating procedures to facilitate consistency across study participants and site;
  • NIH and Other Federal Guidelines/Policies for Clinical Research
    • Mandatory policies and guidance for studies involving human subjects.

PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary)

PRECIS-2 is "a tool to help trialists designing clinical trials consider where they would like their trial to be on the pragmatic/explanatory continuum". It is not a manual for execution per se, but rather "a tool to help trialists make design decisions consistent with the intended purpose of their trial". Read more about PRECIS-2 in this open access article:

  • Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015 May 8;350:h2147. doi: 10.1136/bmj.h2147. PMID: 25956159.

Studies on measurement properties of PROMS

COSMIN Checklists for Assessing Study Qualities

Stands for: COnsensus-based Standards for the selection of health Measurement INstruments
Additional information: About the COSMIN initiative

Observational Studies

Epidemiology for Practitioners (UK HealthKnowledge)

This is a basic course on epidemiology for practitioners. You will learn about basic concepts of measures of disease burden, of association and causation, and of bias and confounding, and will be introduced to epidemiological study designs, along with their application, strengths, and limitations.

Qualitative studies

Qualitative Research Methods: A Data Collector's Field Guide (USAID)

"The guide is divided into five modules covering the following topics:
Module 1 – Qualitative Research Methods Overview
Module 2 – Participant Observation
Module 3 – In-Depth Interviews
Module 4 – Focus Groups
Module 5 – Data Documentation and Management"

Taking Your Qualitative Research to the Next Level: A Guide for the Medical Educator

"In this overview article, we discuss reasons to undertake a qualitative study; introduce three research paradigms whose frameworks are commonly used by medical education researchers (grounded theory, ethnography, phenomenology); describe the four most commonly used data gathering techniques (interviews, focus groups, observation, document tracing); and discuss the nature of the sampling/data gathering process."

Animal Studies

SYstematic Review Center for Laboratory animal Experimentation

The manual from SYRCLE has 5 parts:

  1. Starting guide;
  2. Protocol;
  3. Search strategies;
  4. RoB tool; and 
  5. Meta-analysis guidance.