The William T. Grant Foundation announces funding for seven new projects that advance our interests in understanding everyday youth settings and the uses of research in policy and practice that affect youth. The projects include support for research on the consequences of teacher and youth-worker practices, the measurement of effective teaching, the use of research in practice, and an examination of why intervention effects vary.
Further details on the seven projects follow.
Determinants and Impacts of Academic Grades: What Grading Strategies Work Best, for Whom, and Why (Phase I)
Harris Cooper, Ph.D.
January 2013–June 2013
How do grades and teachers’ grading practices affect student motivation and achievement? Teachers’ strategies for evaluating tests, assignments, and cumulative performance vary on a number of dimensions (e.g., grading on a curve, different types of feedback, letter grades, numeric averages). Thus, it is unclear to what extent grades are a good reflection of student achievement and if there are approaches to grading that are particularly effective in promoting achievement. This investigator plans to synthesize relevant studies to examine grading strategies and their impacts. In phase I, Cooper will determine whether there is sufficient research to conduct a synthesis on the impacts of grading strategies. While there is a diverse body of more than 3,000 relevant documents, it is unclear how many studies evaluate specific grading strategies and meet the inclusion criteria for methodological rigor. Cooper will conduct electronic literature searches and review references lists in relevant articles and books to assess the extent of the research base. The research base includes studies with experimental and quasi-experimental data. The studies must also document the characteristics of grading strategies and impacts of grading for a cohort of youth from kindergarten through college. Based on these searches, Cooper and colleagues will finalize their conceptual model and refine their coding frame. If he finds a sufficient number of high-quality, relevant studies, Cooper will proceed to phase II, examining relationships between grades and grading strategies on student motivation and performance and how characteristics of the school and student affect those relationships. The second phase involves coding studies for quality, content, and effect size estimates. Both quantitative and qualitative studies will be considered during the analysis and interpretation stages to better understand the experiences of students related to grading practices. Relevant qualitative studies with rigorous methods will also be included.
Who Builds the Village? Examining Youth-Adult Relationships Across Contexts and Time
Nancy Deutsch, Ph.D.
Valerie A. Futch, Ph.D.
University of Virginia
Parents play critical roles in the lives of youth. Cross-sectional studies suggest that other adults can serve as role models, sources of support, and guides for navigating transitions. Little is understood, however, about how these relationships develop to provide instrumental and emotional support to adolescents. Deutsch and Futch hypothesize that schools and youth programs are important settings for the formation of youth-adult relationships. They suggest that the quality, culture, and structure of these settings may facilitate or challenge the initiation and longevity of youth-adult relationships. A major goal of this study is to identify resources within youth, the youth-adult dyad, and settings that encourage and sustain youth-adult relationships. The investigators will map youth-adult relationships across settings and examine how the size, quality, and composition of youth’s networks of relationships with adults shift as they transition to new grades and schools. The investigators will also take an in-depth look at the ways adults provide instrumental and emotional support to youth. Finally, the investigators will probe for qualities of the organizational structure and setting that may facilitate or hinder YARs. Two cohorts of 20 youth will be sampled from several different after-school settings with different focal activities. The sample will also include youth who are not employed or engaged in formal after-school activities. One cohort will be followed starting in the 7th grade, the other starting in the 10th grade. Two non-parental adults per youth, each from a different setting, will also be included. The study will use a mixed-methods longitudinal design to follow both cohorts of youth for 24 months. Deutsch and Futch will interview the youth and the adults over the course of the study. Social network data will be used to create maps depicting the adolescents’ relationships with adults. Ethnographic observations of youth-adult relationships will occur four times across the setting.
Development of Self-Direction in Youth Programs and Family Interactions Systems: Latino and Non-Latino Adolescents
Reed Larson, Ph.D.
Marcela Raffaelli, Ph.D.
University of Illinois at Urbana-Champaign
This supplement will build on a 2010 grant investigating how project-based programs help youth—and Latino youth in particular—develop competencies such as strategic thinking, responsibility, emotional management, and teamwork. Findings suggest that program leaders, their objectives, and their skills in applying knowledge are critical for success. Initial findings also suggest that competency development involves cycles of learning in which youth are agents of their own development; parents appear to play a more nuanced role in supporting program involvement. Supplemental funds will extend consideration of connections between youth, parents, and the program. The investigators will finish collecting and analyzing data, double the sample size to allow for data collection with middle school program leaders, and add new observational measures. Collectively, the work will contribute to theory and practice on how to make youth programs more effective. The study will examine how youth develop responsibility, the strategies program leaders use to promote competencies, and the role of parental support in influencing youth development. Larson will pay specific attention to how cultural factors influence program and family interactions with Latino youth. The study includes 12 youth programs (4 each in Chicago, central Illinois, and Minneapolis-St. Paul) providing at least 120 contact hours to low-income youth ages 13 to 19. Programs are not highly selective nor do they target a single gender. The full sample will include all 240 youth in the programs, their parents, and 1–3 leaders from each program. Half of the youth sample is Latino. Subsamples of youth-parent pairs will allow for more refined group comparisons. Year 4 also adds a sample of 25 leaders from middle-school programs that are similar to the programs for older youth. Larson uses questionnaires, observations, and interviews to follow youth, parents, and program leaders at four points across the duration of the program (typically one academic year). Instrument development and a cross-sectional pilot study occurred in year 1. Data collection began in year 2 and will finish in years 3 and 4. Quantitative measures will be used to test hypotheses about interactions between youth, peers, and leaders. Qualitative approaches will provide details about how these relationships are experienced, with a focus on Latino youth. Observations of day-to-day activities and interactions between youth and leaders will be integrated with interview and qualitative data regarding emotional dynamics, learning, and leadership strategies.
The Motivational and Learning Benefits of Autonomy-Supportive Classroom Practices
Erika Patall, Ph.D.
Keenan Pituch, Ph.D.
University of Texas at Austin
Different teaching strategies may impact students’ engagement and experience of autonomy. In turn, adolescents’ perceptions of autonomy may foster greater motivation and healthier behaviors. This supplement supports the second phase of a study designed to identify specific teaching practices that promote engagement and achievement among high school science students within a class session and over time. Importantly, it also examines different ways to measure these relationships. Phase I assessed the feasibility and quality of the proposed methods and measurement tools. Patall conducted observations in two science classrooms. Within each class, she sampled five students and used experience sampling questionnaires to collect students’ reports of teaching practices in real time. Methods proved feasible and data suggest variability in instructional styles and links between teachers’ actions and students’ responses. Phase II will examine how different instructional processes are related to students’ experience of autonomy, motivation, and achievement. Using an innovative design, Patall will examine the robustness of these relationships across sources of information, levels of analysis, and time. The study will include 40 high school science teachers and 200 of their students at 6 high schools in the Austin Independent School District. The district is ethnically and socioeconomically diverse. Data collection will commence just prior to a six-week instructional unit, with students completing a background questionnaire about their psychological status, motivation, and prior performance. Teachers will also complete a questionnaire about their background, teaching experience, and motivational style. Throughout the instructional unit, students will complete experience-sampling questionnaires about teacher practices and their own motivation and engagement. Teachers will videotape classroom sessions and trained observers will code data on a parallel set of teaching practices and for engagement of the class as a whole. Achievement will be assessed using indicators of student engagement and performance obtained from student self-reports and teachers’ records of student quizzes, tests, and assignment grades.
A Grants Program for Early-Career Researchers to Conduct Secondary Data Analyses of the Measures of Effective Teaching Longitudinal Database (MET LDB)
Brian Rowan, Ph.D.
University of Michigan
The Measures of Effective Teaching (MET) study—which was supported by the Bill & Melinda Gates Foundation—examined how different evaluation methods might be combined to provide teachers with feedback about effective practices and help districts identify quality teaching. Quantitative and video data from the MET study has been archived at the University of Michigan’s Institute for Social Research (ISR). The program for early-career researchers will capitalize on this wealth of information and develop the capacity of young researchers to conduct meaningful research on teaching effectiveness. Brian Rowan will oversee the award, which will be administered jointly by the ISR and the National Academy of Education. The program will grant researchers access to the resources of the Inter-University Consortium for Political and Social Research through operation of a secure data enclave known as the Measures of Effective Teaching Longitudinal Database (MET LDB). The William T. Grant and Spencer Foundations will each support a set of five early-career researchers—within five years of an advanced degree—to conduct secondary data analysis of MET LDB. Rowan will lead the ISR and National Academy of Education in developing and distributing an RFP, guiding the proposal review process, and selecting grantees. Following a grants competition in fall 2012, grantees will be authorized to engage with the MET LDB between February 2013 and February 2014. Grantees will receive technical assistance, network with other grantees, and connect with experienced researchers within and outside of the MET community. The ISR will also provide grantees with opportunities for dissemination, including videoconferencing, symposium at conferences, and a possible edited volume.
Networks, Organizational Culture, and the Use of Research (Phase II)
Taryn Lindhurst, Ph.D.
Jerald Herting, Ph.D.
David Takeuchi, Ph.D.
University of Washington
Washington state policymakers are engaged in a large-scale effort to disseminate research findings and assessment tools related to adverse child experiences (ACE) to schools and community service organizations (CSOs). Research suggests that early exposure to trauma and abuse is associated with risky behaviors and physical problems later in life. The Washington State Department of Health and Human Services mobilized 42 Public Health and Safety Networks (PHSNs) that were created in the late 1980s to disseminate findings to schools and CSOs. In this two-part study, the research team first explored how state policy actors understood the ACE findings and tools, how they defined use of ACE, and what factors influenced their dissemination strategies. Interviews suggest that research and assessment tools have provided a common language for PHSNs to discuss community problems; however, there is considerable variability in the uptake of evidence among schools and CSOs. Phase II will build on these findings and investigate how relationships within and across organizations and organizational culture affect research acquisition and use. The team will examine how the PHSN leadership networks and the relationships between local organizations influence the diffusion of ACE research from the state to local communities, schools, and CSOs. Participants during phase II will include chairs and members from all 42 PHSNs. The study will also include 500 schools and 400 community-based organizations from within the PHSNs’ jurisdictions. Five to 10 non-instructional district personnel and CSO staff will be randomly selected within each school or organization. Document reviews and interviews will be conducted with key members of the PHSNs to assess dissemination activities. Randomly sampled members from a subset of 16 PHSNs with either high or low activity will then be surveyed to obtain social network data. Individuals from the schools and organizations will complete surveys assessing characteristics of the organization, relationships between organizations, and research use. Qualitative interviews will be conducted with a subset of 20 schools and CSOs to delve more deeply into how they identified and implemented ACE research as well as their social networks and organizational cultures.
Meta-analytic Exploration of Variability in the Effects of Youth Programs
Mark Lipsey, Ph.D.
Sandra Jo Wilson, Ph.D.
Joseph Durlak, Ph.D.
Loyola University, Chicago
Evaluations of interventions for children and youth typically show marked variability in their results. This project will explore that variability using the effect size estimates and associated descriptive variables from seven existing meta-analyses. These meta-analyses each include 58 to more than 500 evaluations of the effects of prevention and treatment programs on a range of youth outcomes. Lipsey and colleagues will further code community and organizational factors to shed light on other possible sources of variability not yet represented in these data sets. They will estimate how much of the variability in observed effects is due to the nature of the programs, their implementation, the settings, and the participants. They will also estimate how much of the variability is due to differences in the methods used to gather information, the study designs, the procedures used in analysis, and sampling error. Finally, the investigators will estimate how much of the observed differences are due to factors that cannot be accounted for by any of these sources. This three-step strategy will be applied to a broad range of settings, including after-school programs, school-based emotional and behavioral programs, interventions for juvenile offenders and youth at risk for delinquency, and family interventions. They will examine variation across participant samples, community contexts, organizational features, intervention components, doses, training, and methodologies. Understanding how these factors influence the variability of program effects will help researchers and practitioners identify the conditions under which program effects are particularly robust and the extent to which they can be generalized across different settings and applications.