Advancing Systematic Review Workshop (December 2015)

The purpose of the workshop is to examine developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic) for use in chemical assessments.

On this page:

Workshop Details
Workshop Materials

EPA is developing and implementing a systematic review process within the IRIS Program to increase the transparency and clarity related to identifying and evaluating evidence and drawing conclusions in ongoing and future assessments. This workshop will serve as a follow-up to two previously held systematic review workshops held in August 2013 and October 2014. The August 2013 workshop examined the state-of-the-science of systematic review methodology, focusing on recent developments as the methodology has expanded outside the arena of evidence based medicine. The October 2014 workshop focused on recommendations in the May 2014 National Research Council report related to systematic review and evaluation of the risk of bias, integration of evidence streams, and combining quantitative results from multiple studies.

The December 2015 workshop examined developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic). Two sessions were held: (1) From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure, and (2) Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied? This workshop built on some of the sessions from the previous workshops and further addressed the practical application of systematic review methodologies to specific elements of IRIS assessments.

The December 2015 workshop includes presentations and discussions by scientific experts in areas pertaining to systematic review. The workshop was structured to include an opportunity for comments, questions, and engagement from stakeholders and members of the public.

To promote transparency, speakers were requested to disclose: (1) The nature of any financial relationships they may have with any organization(s) or entities having an interest in issues under discussion (e.g., consulting agreements, expert witness support, or research funding); and (2) The extent to which their comments were reviewed by an interested party.

Dates

The workshop was held on December 16 - 17, 2015.

Location

The meeting was held in the EPA Conference Center at 2777 South Crystal Drive, Arlington, Virginia 22202. The meeting was also available by webinar.

Background

The purpose of the workshop was to examine recent developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic). Two sessions were held: (1) From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure, and (2) Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied?

Session 1: From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure

Considerable efforts have been undertaken to develop and refine systematic review methods that address the scope, purpose, and designs of studies examining potential health effects or toxicities from exposure to environmental chemicals. Much of this work has built upon the methods and resources of the Cochrane Collaboration, such as recently-released methods designed specifically for the assessment of risk of bias in non-randomized studies of interventions (ACROBAT-NRSI). This session will use the ACROBAT-NRSI as a starting point to examine developments in methods used to evaluate studies. Discussion topics:

In what ways do the differences between evaluating “interventions” and evaluating “exposures” necessitate different approaches to assessing bias?
What insights can be drawn from the experience of various groups using or adapting ACROBAT-NRSI?
Does "risk of bias", as currently conceptualized and used, adequately encompass the breadth of issues needed for evaluation of studies of chemical exposures in observational epidemiology and in animal toxicology studies? If not, what approaches can be used to appropriately expand the focus?

Session 2: Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied?

Evaluations of the potential mechanisms underlying the toxic effects of chemicals can lend support to determinations of hazard in humans. Approaches to the comprehensive and systematic identification and evaluation of mechanistic data that ensure efficiency and transparency in this process are currently being considered. This is a particular challenge for in vitro studies, given their heterogeneity and, in many cases, the large number of studies/assays. This session will examine approaches to sorting, screening, and evaluating mechanistic evidence, drawing upon use of key characteristics of carcinogens as an organizing principle. Discussion topics:

What methods can be used to prioritize the relevant mechanistic studies that will most directly address questions on mechanistic events and pathways, i.e., mode of action?
What is the utility of evaluating in vitro studies for bias and what approaches are available?
Can the model of using key characteristics of carcinogens as an organizing principle be used for mechanistic evidence for other hazards (e.g., neurotoxicity, reproductive and developmental toxicity)?
Are there frameworks or approaches such as adverse outcome pathways (AOPs) that may be useful for prioritizing the most relevant mechanistic studies for addressing questions on mechanistic events and pathways, i.e., mode of action for cancer and noncancer effects?

Comments on the workshop may be submitted to EPA_Sys-Review@icfi.com.

Workshop Materials

Final Agenda

Advanced Systematic Review Workshop (Dec 2015) (PDF).