NJACTS-From Here to There: Beyond the Common Metrics

Invited speaker Cynthia L. Blitz, research professor and executive director of the Rutgers Center for Effective School Practices, presented 'From Here to There: Beyond the Common Metrics' in 2019 at the annual retreat of the New Jersey Alliance for Clinical and Translational Science (NJ ACTS). 

Topic areas Blitz presented include: 

• Program evaluation: a critical scientific approach that transforms how researchers understand and improve complex interventions in clinical and translational research. Researchers approach program evaluation through multiple lenses, each offering unique insights into intervention effectiveness. 

• Formative evaluation serves as an early warning system, assessing a program's feasibility before full-scale implementation. 

• Process/implementation evaluation then tracks the program's journey, identifying potential barriers and ensuring the intervention remains true to its original design. 

• Outcome/effectiveness evaluation takes a broader view, measuring the program's impact on knowledge and behavioral changes. 

• Economic (cost-effectiveness) evaluation provides a practical perspective by comparing resource investments against achieved results. 

• Impact evaluation assesses program effectiveness in achieving its ultimate goals. 

• Logic models: sophisticated frameworks that map out the hypothesized relationships between program components. These models illuminate the pathway from initial inputs to desired outcomes, accounting for assumptions and external factors that might influence program success. By creating these visual representations, researchers can better understand the complex dynamics of their interventions.

• Frameworks like the Consolidated Framework for Implementation Research (CFIR) offer structured methodologies for assessing implementation success. By examining intervention characteristics, individual factors, organizational settings, and implementation processes, researchers can develop more targeted and effective programs. 

NIH Clinical and Translational Science Awards (CTSA) is an initiative that needs quality evaluation to show the program is well implemented, efficiently managed, and effective. Common metrics provide a starting point, but truly comprehensive evaluation requires a more nuanced approach. Program evaluation is a collaborative journey. It demands clear goals, systematic investigation, and a commitment to understanding the complex mechanisms that drive successful interventions in clinical research.