How can we enhance our understanding of ACF programs?

Challenge

The Administration for Children and Families (ACF) administers over 60 programs to promote the economic and social well-being of communities across the US. Rigorous evaluation of these programs ensures they can efficiently and effectively serve the children, families, and individuals participating in them.

The Office of Planning, Research, and Evaluation (OPRE) researches these programs and policies, seeking to explore, understand, and advance analytic methods that enhance rigor in government research and evaluation. The advancement of research methods ensures high-quality, trustworthy, and robust findings that better inform our understanding of ACF programs and the communities they serve and improve program administration and effectiveness. People researching government programs need access to information and expertise on emerging or otherwise important methodologies.

Solution

Westat supports OPRE’s efforts for methodological advancement with extensive expertise in event coordination and policy analysis. Our team planned the annual OPRE Methods Meeting from 2015 to 2023, including identifying potential expert speakers, coordinating with logistical vendors, and managing subcontracts. This work brought together experts across disciplines and sectors to explore innovations in research design, analytic techniques, and data measurement.

Westat also developed communication products to disseminate findings and best practices generated from the Methods Meetings. Westat successfully completed this work through Methods Meetings and the facilitation of the ASPE Evidence Day Meeting held in 2021 to support the implementation of the Foundations for Evidence-Based Policymaking Act of 2018.

Westat developed technical assistance materials to support the evaluation capacity of human services programs. Our team of policy and evaluation experts updated OPRE’s Program Manager’s Guide to Evaluation by identifying and consulting with relevant partners about the guide’s strengths and opportunities for improvement. Westat developed additional technical assistance resources based on the opportunities for improvement, including briefs on engaging community representation in program evaluation.

Results

This project informed future research centered on human services programs. The work promoted innovative research and improved our understanding of rigorous analytic methods, resulting in higher quality data collection, analysis, and findings. The project encouraged evaluators to center community voice at all stages of research to enhance the research’s robustness and rigor and ensure research on human services programs is driven by the people most affected by them.

Resources

Westat developed resources for the 2017-2023 Methods Meetings. The following are resources on unit missingness in surveys for the 2023 Methods Meeting:

• Applying Mixed Methods and Qualitative Approaches to Social Policy Questions: Summary of 2022 OPRE Methods Meeting
• Digging Deeper into Nonresponse Reduction and Adjustment Techniques: Frequently Asked Questions
• Issue Brief: Probability and Nonprobability Samples in Surveys: Opportunities and Challenges
• 2023 Methods Meeting Session Videos