From 2003–2011, the William T. Grant Foundation awarded $20.9 million in grants and spent considerable staff time to understand and improve the quality of after-school programs. We are now assessing our efforts, looking at where we succeeded and what we will do differently going forward. This letter shares some early conclusions and relevant documents.
We primarily support research to improve the lives of young people, and 60 percent of our grantmaking on after-school during this time was for research. We wanted to understand how these programs affect young people, test ways to make them better, improve measures of their quality, and evaluate different program approaches. The remaining 40 percent of grant funds supported advocacy, capacity-building, and communication activities. We also strove to be an "honest broker" about what the research showed.
In this year’s Annual Report, President Robert Granger wrote an essay discussing our progress in improving the quality of after-school. We commissioned Pam Stevens to review this work, and her subsequent report features insights from her interviews with grantees and others who were involved. We also asked Sam Piha to examine California’s dramatic expansion of after-school programs during this period. Three main lessons emerged:
- Harnessing intermediaries, relationships, and networks is key to connecting research and practice. Most of us turn to our networks of peers and trusted advisors when we have questions. If researchers and research funders want to produce information relevant to policy and practice and get their work used, they need to understand and access these networks. Reliable findings and clear one-pagers are not enough. Funders can play a useful role in supporting ongoing collaboration between researchers and practitioners.
- Practitioner-friendly, evidence-based measures of program practices are powerful tools for improving practice. We see our support for the development, refinement, and use of various measures of program practices as the most productive example of our work on program quality. The second edition of a compendium reviewing such instruments has been downloaded over 10,000 times.
- Staff practice is critical. To measure quality, assess youth interactions in activities. To improve quality, change staff behavior to make these activities better managed, more supportive, and more engaging.
As we move forward with this work, we plan to focus on two activities. The first is more testing of approaches to continuous improvement. To improve after-school practice at scale, we think it is more useful to invest in understanding how to produce continuous improvement than in impact evaluations of small-scale programs. The best available evidence says that when promising programs are evaluated, in two-thirds of the cases, their outcomes are no better than those produced by other experiences available in the community. Even strong programs know that their quality varies both within and across locations. Second, we will focus more on practitioners’ questions. For research to be relevant to practice, it needs to address the questions of program administrators and managers. These are the people who adopt and implement programs, interpret regulations, allocate funds, set priorities, and so on. Their questions go beyond “what works” to issues such as cost, the practices that are common to effective programs, what systems need to do to support high-quality practice, and the organizational capacity needed to use an innovation successfully.
Download Referenced Documents:
- Robert C. Granger, "Our Work on the Quality of After-School Programs: 2003–2011"
- Pam Stevens, "The William T. Grant Foundation: Contributions to the After-School Field 2003–2011"
- Sam Piha, Joshua Julian, & Rozel Cruz, "Examining California’s Afterschool Movement Post Proposition 49"
- Nicole Yohalem, Alicia Wilson-Ahlstrom, Sean Fischer, & Marybeth Shinn, "Measuring Youth Program Quality, A Guide to Assessment Tools, Second Edition"
- Alicia Wilson-Ahlstrom, Nicole Yohalem, David DuBois, & Peter Ji, "From Soft Skills to Hard Data: Measuring Youth Program Outcomes"