Measuring Social Settings

Entire Site Grants Reports

Framework for Studying Settings

Measures of Social Settings

Design and Analysis of Group-Randomized Trials

Consultation Service and Optimal Design

Studying the Use of Research Evidence

Grantee Forms

Research-Practice Partnerships

We are interested in the development and improvement of measures of social settings, particularly in tools that are low-cost and useful to practitioners. The resources here focus on assessing the reliability and validity of such measures and choosing appropriate measures.

RECOMMENDED GENERAL READING

Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models (1998) by David Chan. In Journal of Applied Psychology 83 (2): 234-246.
Ecometrics: Toward a science of assessing ecological settings, with application to the systematic social observations of neighborhoods (1999) by Stephen W. Raudenbush and Robert J. Sampson. In Sociological Methodology 29: 1-41.
Cross-level research without cross-ups in community psychology (2000) by MaryBeth Shinn and Bruce D. Rapkin. In Handbook of Community Psychology, J. Rappaport and E. Seidman, eds. New York, NY: Kluwer Academic/Plenum Publishers, 669-695.

RELIABILITY AND VALIDITY OF SETTING-LEVEL MEASURES

Confidence Bounds and Power for the Reliability of Observational Measures on the Quality of a Social Setting (October 2011) by Yongyun Shin and Stephen W. Raudenbush
Studying the Reliability of Group-Level Measures with Implications for Statistical Power: A Six-Step Paradigm (March 2011 draft) by Stephen W. Raudenbush, Andres Martinez, Howard Bloom, Pei Zhu, and Fen Lin
Validity Studies Involving Measures of Classroom Quality (July 2009 draft) by Andres Martinez and Stephen W. Raudenbush

FOUNDATION GUIDELINES

Measurement

Measures of Effective Teaching Longitudinal Database

Researchers should consider the MET Database at the Institute for Social Research, which contains quantitative data such as standardized instrument scores, teacher and student surveys, and student assessments. The Database also houses more than 20,000 hours of video from active classrooms.

Choosing Measures for Youth Programs and Outcomes

Measuring Youth Program Quality: A Guide to Assessment Tools, 2nd Edition by Nicole Yohalem, Alicia Wilson-Ahlstrom, Sean Fischer, and Marybeth Shinn (January 2009). The Forum for Youth Investment updated this report, first issued in March 2007, which compares the purpose, history, structure, methodology, content, and technical properties of different program observation tools. (92 pages, 1.31mb PDF)

The Practitioner’s Guide to Conducting Classroom Observations: What the Research Tells Us about Choosing and Using Observational Systems (March 2009) was written by Megan W. Stuhlman, Bridget K. Hamre, Jason T. Downer & Robert C. Pianta in conjunction with the grant Empirical and Theoretical Issues in Classroom Observation: Creating Practical Tools for School-Based Researchers and Practitioners.

From Soft Skills to Hard Data: Measuring Youth Program Outcomes by Alicia Wilson-Ahlstrom, Nicole Yohalem, David DuBois, and Peter Ji (September 2011). The Forum for Youth Investment issued this guide, which reviews tools for measuring youth outcomes in social-emotional skill areas, such as communication, critical thinking, and self-direction. All measurement tools reviewed are appropriate for use in out-of-school time settings.