Designing educational evaluations with a population perspective.

This tool leads education researchers through:

Defining an inference population.

Researchers can define both broad and narrow inference populations of public schools using geographic, demographic, and administrative criteria from the Common Core of Data (CCD), American Community Survey (ACS) and the Integrated Postsecondary Education Data System (IPEDS).

Selecting important features of the population.

Researchers select demographic and administrative features that potentially explain variation in treatment impacts across schools.

Developing a recruitment plan for a future evaluation.

For researchers planning a study, The Generalizer creates a stratified recruitment plan based on the features selected. By recruiting some schools from each stratum, the final sample is compositionally similar to the inference population. Ranked lists are then provided with schools and contact information, as well as recruitment goals that can be used to guide the recruitment process.

image/svg+xml
image/svg+xml

Assessing the generalizability of a completed evaluation.

For researchers evaluating the findings from a study, The Generalizer compares the final sample of schools that took part in the evaluation with the relevant population of schools in the United States as well as each of the 50 states. Summary information is provided indicating the similarity between the sample and population, as well as guidelines regarding how this similarity affects the generalizability of findings.