I recently worked with Resilient Georgia (RG) to complete a National Landscape Scan of TIC/ACEs Evaluation Efforts. This chapter builds on RG’s first chapter of their National Scan: Trauma-informed Care and ACEs Prevention Efforts, conducted in 2019. The purpose of both scans was to learn from our peers and better understand the current landscape of TIC/ACEs efforts nationally. This chapter of the scan investigates evaluation efforts of the highest rated statewide initiatives that aim to deliver Trauma-Informed Care (TIC) and prevent or reduce Adverse Childhood Experiences (ACEs). The aim of the scan was to assess best practices of Collective Impact evaluation efforts of similar initiatives and to build upon evaluation methods to support Resilient Georgia’s efforts across the state of Georgia.
The scan revealed few relevant examples of statewide Collective Impact evaluation, particularly for community coalition-based programs. Similar coalitions' most common evaluation efforts tended to focus on specific programs rather than community-wide changes, perhaps because measures were more easily identifiable and controlled (i.e., how many people have been trained, how many organizations have adopted new policies, etc.). This finding mirrors the literature, indicating a gap in reliably measuring collective community ACEs initiatives at the community or state level.
One stand-out example the scan revealed was Washington State’s evaluation of their state-wide initiative (named Washington State Family Policy Council (1994-2012), ACEs Public-Private Partnership Initiative (2012-present)), which combined qualitative and quantitative methods and retrospective and prospective data to produce assessments of the effectiveness of comprehensive community change efforts. Although utilizing multiple evaluation methods, WA still concluded that it was difficult to know what outcomes could be attributed directly to their initiative at the community level (i.e. reduced ACEs by county).
Perhaps to no surprise to those that have worked with community-level interventions, understanding the complexity of community change can be quite difficult, let alone trying to determine direct outcomes of community-level interventions. It is not uncommon for interventions that focus on broader social inequity-related goals (such as preventing Adverse Childhood Experiences at the community level) to be unable to determine what changes result from one organization’s efforts. In this example, there may be multiple causes for reduced or increased ACEs; Covid-19 is a timely example.
While it may be difficult to ascertain the exact cause of ACEs reduction in the context of a community, we hope that efforts to support health equity, education, and other protective and compensatory childhood experiences (PACEs) may reduce the adverse effects of ACEs. However, the goal of evaluating an initiative that aims to prevent/reduce ACEs is to know whether or not an initiative's efforts are effectively addressing ACEs. By evaluating community change, coalition relationships, and programming, we can bring research and practice into alignment, increasing the potential impact an initiative can have on the community it serves.
Jessica Koreis is a dual Master of Public Health and Community Psychology PhD student at Georgia State University, her studies focus on childhood adversity and health outcomes, psychosocial impacts of trauma, and resilience informed prevention and intervention efforts across diverse populations.
Sources
Hargreaves, M. B., Verbitsky-Savitz, N., Coffee-Borden, B., Perreras, L., White, C. R., Pecora, P. J., ... & Adams, K. (2017). Advancing the measurement of collective community capacity to address adverse childhood experiences and resilience. Children and youth services review, 76, 142-153.
Hargreaves, M. B., Verbitsky-Savitz, N., Penoyer, S., Vine, M., Ruttner, L., & Davidoff-Gore, A. (2015). APPI cross-site evaluation: Interim report. Cambridge, MA: Mathematica Policy Research.
Kania, J., & Kramer, M. (2011). Collective impact (pp. 36-41). Beijing, China: FSG.
Stachowiak, S., Lynn, J., & Akey, T. (2020). Finding the impact: methods for assessing the contribution of collective impact to systems and population change in a Multi‐Site study. New Directions for Evaluation, 2020(165), 29-44.
Comments (2)