Evaluating family preservation services from a community well-being perspective: A time series analysis of Virginia's Comprehensive Services Act for At Risk Youth and Families

Abstract

Measuring family preservation outcomes has challenged child welfare professionals, policymakers, and researchers for over a decade. Most agree that a change from the traditional service delivery model was warranted, but there is little agreement on a model to replace the traditional model. Defining family preservation services, determining what outcome measures to use other than out of home placement prevention, and deciding how to measure effectiveness are at the center of the debate. This pilot study addresses all three of these issues. In 1993, Virginia was the first state in the US to uniformly legislate a family preservation service system of care. Using the flexible, community-specific, wraparound definition of family preservation services espoused by Virginia's Comprehensive Services Act for At Risk Youth and Families (SA), this study examines the impact of family preservation services on 23 rural Virginia communities. With the community as the unit of analysis and through the use of a hierarchical linear model analysis, four predictors of community well being (low birth weight, school drop out, births to girls, and poverty) were compared with five factors traditionally considered to be risks for children (early school failure, juvenile arrests, child maltreatment, violent teen deaths, and foster care placement). In a time series design, these comparisons were made at six points in time: two years prior to implementation of the CSA, the year the CSA was implemented, and three years after implementation of CSA legislation. The findings in this pilot study demonstrated that the community is an appropriate unit of analysis to study but that evaluation of individual success must also be considered. School drop out rate was found to be a statistically significant predictor of early school failure. Low birth weight and poverty were found to be statistically significant predictors of foster care placement. Measuring change through the use of the two-level hierarchical linear model appears to be a promising model compared to analyses more commonly used in time series designs. Low power due to small sample size and only three years under a new service paradigm signal the need for further study using larger samples and a longer period of observation.