What research questions are asked to evaluate Head Start?
What data are available to assess Head Start?
What is the research evidence on Head Start?
How should we interpret Head Start research evidence?
Equity considerations in Head Start’s research evidence
(What is research evidence?)
The third stage of the diversitydatakids.org policy equity assessment is research evidence. Traditional research evidence from rigorous studies shows that Head Start has a positive average impact on child school readiness at kindergarten — the program’s primary goal — and may have longer term impacts in adulthood, although this remains to be proven conclusively. Head Start also has positive impacts on school readiness among subgroups of especially vulnerable children, including black children and children who performed the lowest academically at program entry. Additionally, initial evidence from rigorous studies shows that using evidence-based curricula with Head Start children results in further improvements in child school readiness, and other similar studies are underway. Nevertheless, the program does not affect all important school readiness outcomes before children enter kindergarten; for example, there are few or no impacts in children’s early math skills after one year of Head Start. In addition, the magnitude of statistically significant impacts is often small, although this trend is in line with other programs of similar scale.
Importantly, while Head Start is one of the most studied social programs, additional equity-focused research evidence is still needed to:
Given the flexibility of local service design and evolution of Head Start goals, an analysis of only average program impacts may overlook the program’s design logic and resource capacity that are crucial to understanding whether, how, when, and for whom the program is effective. The diversitydatakids.org policy equity assessment examines not only the results of traditional effectiveness research but also multiple types of research evidence. This approach is consistent with the recommendation of the Advisory Committee on Head Start Research and Evaluation—which developed the blueprint for the Head Start Impact Study (HSIS)—that “the research and findings [of the HSIS] should be used in combination with the rest of the Head Start research effort to improve the effectiveness of Head Start programs for children and families.”
- Determine whether Head Start has positive school readiness effects for other important vulnerable subgroups of children enrolled in the program (e.g. children of immigrant parents and children who are not exposed to evidence-based curricula).
- Determine whether Head Start reduces school readiness gaps between the most and least vulnerable participants in the program.
- Determine whether Head Start reduces gaps on average at kindergarten entry between program participants and preschoolers not living in poverty.
- Better understand how variation in the implementation of local Head Start programs and the use of evidence-based practices may affect quality and lead to differential impacts in school readiness outcomes.
Bridging equity and effectiveness:
Does Head Start close gaps in child school readiness?
The diversitydatakids.org policy equity assessment of effectiveness highlights the equity-related successes and challenges of Head Start. Though equity goals are not explicit in the legislation and policy guidance, Head Start’s design logic
demonstrates the potential to improve equity in child school readiness by providing access for children who face significant cost and participation barriers to early education. For example, not only does Head Start serve low-income children, but within this population, the program also targets the enrollment of and tailors services to especially vulnerable children such as children with special needs, children of migrant and seasonal farmworkers and Dual Language Learners (DLLs).[67
Despite the promise in Head Start’s design, the program’s potential to significantly reduce inequities in school readiness is constrained by resource capacity
. Due to insufficient funding, Head Start only has the capacity to serve about half of low-income children, and access to the program varies by race, ethnicity, and geographic location.[68
] Moreover, the amount of exposure to services and the quality of these services varies by child race/ethnicity (e.g. on average, Hispanic and Asian students are exposed to lower intensity services, and a lower percent of centers attended by black students are high-quality than for other race groups).[69
] Moving forward, Head Start’s capacity could be bolstered by extending program operating schedules or further investing in evidence-based classroom curricula and training.
The research evidence
on Head Start’s effectiveness and equity impacts reflects both its promising design logic as well as its constrained capacity. This evidence suggests that Head Start has positive school readiness impacts for children at kindergarten entry, yet these impacts are not enough to eliminate school readiness gaps between Head Start participants and their non-low-income counterparts,[70
] or between more vulnerable subgroups (e.g. DLLs) and their less vulnerable counterparts (e.g. monolingual English-speaking children) within the program.[71
] However, some studies have found that although Head Start does not eliminate school readiness gaps, it does help narrow them, both overall[72
] and for specific vulnerable populations within the program.[73
] These findings suggest that improvements to the program could result in much greater equity impacts.
Although these findings are encouraging, numerous equity-related questions remain unanswered and more research evidence is needed to document the program’s potential to improve equity in school readiness. This research evidence could come from new equity analyses of existing data as well as from new studies designed with an equity focus. Future studies that include equity-specific research questions will require sample sizes large enough to conduct subgroup and gap analyses of intervention impacts (e.g., evidence-based curriculum enhancements). In addition, new studies should pay more attention to interventions and measures specific to more vulnerable subgroups (e.g. children with special needs and DLLs), such as the type and quality of instruction that can effectively prepare these children for successful learning.
Back to top.
What research questions are asked to evaluate Head Start?
Head Start is one of the oldest and most studied federal social programs targeting low-income children in the United States. Since the beginning of the program in 1965, researchers, policymakers and the public have wanted to know if and how it improves child outcomes. This concern is ongoing: in the past 15 years, there have been many rigorous studies conducted on Head Start, including the Head Start Impact Study (HSIS), mandated by Congress in an attempt to definitively answer whether Head Start ‘works’ and ‘for whom.’ Reviewing HSIS and other Head Start research evidence, we can determine what research questions have been asked, and should be asked, about the program to capture a complete picture of its overall and equity-focused effectiveness.
Below we summarize and give examples of the types of research questions that have been asked about Head Start (see Research Questions on Head Start Effectiveness
for a more complete list of questions with descriptions of the studies and data sources). Almost all Head Start studies were not designed at their inception to explicitly answer equity-focused research questions. Instead, research questions focus on children’s outcomes on average and in the best case scenario, examine subgroups of vulnerable children. Therefore, although important, many of these questions have a narrow focus with few equity considerations, and in some instances the evidence to answer them is difficult to interpret. To get an overall equity assessment of Head Start, we need to piece together evidence from different types of studies, and provide an integrative interpretation. Readers should note that although Head Start has important family engagement goals and may have long-term impacts on children, this Head Start policy equity assessment focuses primarily on child school readiness outcomes at kindergarten entry, which has been designated as the primary goal in Head Start’s legislation since 1998.
Selected Head Start research questions:
What are the demographics and skills of children enrolled in Head Start?
What do we know about Head Start program quality and how it is related to child outcomes?
What is the impact of Head Start on child outcomes for income-eligible children? Do impacts vary by subgroup?
- What is the racial/ethnic composition of preschool-age children enrolled in Head Start?
- What are the cognitive and social skills of Head Start children at the beginning and end of their first year in the program?
What is the impact of specialized/tailored curricula and services added to Head Start on child outcomes?
- At the start of Kindergarten, what is the direct impact of an offer of Head Start compared to no offer of Head Start (control group) on child school readiness?
- How did Head Start impacts differ by vulnerable subgroups? Subgroups are defined as: (1) children with special needs, (2) children with low pre-academic skills at baseline, (3) children with non-English home language, (4) caregiver’s minority race/ethnicity, (5) caregiver’s self-reported depressive symptoms, (6) household risk index, and (7) urbanicity.
- Are Head Start impacts influenced by community characteristics such as levels of neighborhood disadvantage, crime, and the availability of alternative child care options?
- Does the integration of a research-based enrichment curriculum and socio-emotional development components into Head Start enhance school readiness outcomes?
- Does the inclusion of a culturally proficient diet and exercise curriculum in Head Start change black and Hispanic children’s trajectories toward obesity?
While it is important to assess what questions have been asked about Head Start, it is just as critical to consider the questions that have not been asked, but should be. When questions are not asked, it may be of interest to dig deeper to see if these issues are systematically overlooked. Below are some examples of research questions that are not asked about Head Start.
Selected equity-related research questions missing from studies of Head Start effectiveness:
- Access: What is the size and demographic composition (e.g., child race/ethnicity) of Head Start programs’ waiting lists?
- Program performance: To what extent are programs achieving quality benchmarks? Can parents access this information to make decisions about which Head Starts centers to enroll their children? Is quality information accessible to parents of all racial/ethnic groups, and to parents of children with special needs?
- Program implementation: What does it take to attain full implementation of all Head Start components and high quality services? What factors account for the variation in Head Start service implementation, including levels of quality, among programs? Does variation in program implementation across sites explain differences in impacts? Is exposure to high-quality programs differential by child race/ethnicity?
- Children with special needs: What individualized services do children with special needs receive on the ground? How do these services influence their school readiness and achievement?
- Health services: What health services are delivered on the ground and how are they linked to a broad array of parent and child health outcomes measured by biomarkers and medical records, rather than self-reports? Do the program’s effects on health vary by child/parent race/ethnicity?
- School readiness at the population level: When Head Start children start Kindergarten, do they start behind their classmates and if so, by how much? What is the magnitude of the racial school readiness gap for children participating in Head Start compared to those that did not participate? How does the school readiness of Head Start children compare to the average U.S. child starting Kindergarten and to the average U.S. child living above the poverty line starting Kindergarten?
Back to top.
What data are available to assess Head Start?
What data and evidence are available to assess Head Start effectiveness?
Availability of research data can influence which aspects of a program are evaluated, the quality of research evidence and its interpretation. Therefore, the type and quantity of information sources that are available on a particular program can have a significant influence on how the program is perceived (click here
for a description of data sources for assessing program effectiveness). Compared to many social programs, Head Start has numerous sources of data and research evidence that can be used to evaluate its effectiveness. Below are a few examples; see the Methods and Measures in Head Start Research
table for a more complete list with detailed descriptions.
Selected Head Start studies and data sources:
- Program Information Report (PIR): Annual survey-based census of all Head Start programs. These administrative data may be used for program performance management and documentation of participant and staff characteristics.
- Family and Child Experiences Survey (FACES): Triennial, survey of a nationally representative sample of Head Start programs, centers, classrooms, teachers, children and families. This study follows children through the end of kindergarten and the data may be used for program performance management and outcome evaluations.
- Head Start Impact Study (HSIS): A nationally representative, multi-outcome measure impact evaluation of the Head Start program, following three- and four-year old children through the end of third grade. Data may be used to assess the impact of an offer of Head Start services (compared to no offer) on child school readiness and other outcomes.
- Studies of specific enhancements to Head Start. These enhancements include adding literacy and/or social-emotional curricula, health prevention, or parental employment services to regular Head Start services. For example, the Hip-Hop to Health Jr. program was an obesity prevention program for minority Head Start preschoolers that was integrated into Head Start programs. Other interventions have implemented dual-generation approaches in Head Start, such as adding parental employment assistance to usual Head Start services. Some federal agencies have sponsored research of specific evidence-based curricula. For example, a federal inter-agency working group funded a study known as the Research-Based, Developmentally Informed (REDI) Intervention, which was a mixed methods evaluation of a research-based literacy and social-emotional curriculum enrichment program within Head Start classrooms in Pennsylvania. Importantly, the REDI experiment was accompanied by an implementation study using qualitative methods and surveys describing in-depth Head Start classroom implementation of the REDI curriculum in terms of dosage, fidelity, generalization and child engagement.
Many of these studies and data sources collect information on race, ethnicity, parenting practices, disabilities, or household indicators such as unemployment or public assistance receipt. These data are essential and should be more systematically and uniformly collected across data systems, as well as further examined and analyzed to understand how program effects may vary across subgroups of children (What Works for Whom?).[79
] Unfortunately, only a limited number of studies have included implementation analysis, which is essential to understanding the conditions under which programs are effective (What Works Under What Conditions?).
What are limitations of Head Start data and research evidence?
Despite the wealth of information collected on Head Start, there are some important limitations in Head Start data and evidence. It is critical to understand these limitations as they affect the interpretation of this large body of evidence on Head Start effectiveness. Constraints in the study design or the analytical methods of an impact study can influence the quality of the findings. The lack of documentation of variation in program implementation can limit the interpretation of research evidence from impact evaluations. Below are a few examples of limitations inherent in some Head Start data and evidence sources; click here
for a more comprehensive list.
Examples of limitations in selected Head Start data sources and studies:
- Program Information Report (PIR): The PIR cannot be used to analyze data for individual children or to follow important trends over time.
- Family and Child Experiences Survey (FACES): FACES data cannot be used to draw causal relationships between program characteristics and child outcomes.
- Head Start Impact Study (HSIS): Interpretation of HSIS evidence is limited by a lack of detailed understanding of how the intervention was carried out by different programs, and a lack of easily digestible results on disparities in school readiness outcomes, such as gap analyses by subgroup.
- Head Start Research-Based, Developmentally Informed (REDI) Intervention: Interpretation of REDI evidence is limited by a lack of detailed understanding of the experiences of control group children.
When assessing the overall landscape of studies and data sources for population-specific Head Start programs, an additional major limitation is that American Indian/Alaska Native and Migrant & Seasonal Head Start programs and participants have been excluded from most major Head Start studies, including all FACES cohorts and HSIS. This dearth of research evidence limits evaluation of program effectiveness for these important subgroups, although there are new studies underway to help fill this gap.[80
Back to top.
What is the research evidence on Head Start?
We now turn to a review of the research evidence on Head Start effectiveness, keeping in mind the limitations of Head Start studies and data sources. Due to the wide array of research questions, designs and data used to assess Head Start’s success, it can be difficult to attain a clear sense of the program’s effectiveness. In short, Head Start research findings provide conclusive evidence that the program achieves its primary goal, improving child school readiness at kindergarten entry across multiple domains including some cognitive outcomes (such as language and literacy), social-emotional outcomes, health status and dental care.[81
] Importantly, however, these impacts are neither large in size nor consistent across all areas and stages of development, indicating that there is room for improvement within the program. It also appears that Head Start may be particularly beneficial for some minority children, such as blacks and English-speaking Latinos.
In addition, longer-term findings are also part of Head Start’s evidence-base, although long-term impacts are beyond the scope of the program’s legislation. Non-experimental longitudinal studies find long-term positive effects of Head Start participation in adulthood.[82
] However, short-term elementary school evidence from the Head Start Impact Study shows that cognitive scores converge for treatment and control children, and there is no clear pattern in socio-emotional or health outcomes. There are no current data to help understand why children who do not participate in Head Start catch up to their Head Start peers in elementary school. Given these findings, it remains to be proven conclusively whether there are any gains from Head Start that lasting beyond the preschool years.
Within Head Start, specific small-scale, experimental evaluations
of program enhancements targeting a variety of child outcomes such as socio-emotional skills and child overweight demonstrate significant short-term positive impacts in the one to two-year follow up period. These studies indicate that enhancements can be made in Head Start to further improve children’s school readiness and health. Importantly, however, the research does not show whether these enhancements have long-term effects over and above the usual Head Start services.
Below, the short-term overall findings and a few subgroup findings are presented for selected well-known studies of Head Start; see the Head Start Research Findings
table and Findings from Head Start Subgroup Analyses
table for more complete and detailed lists, including longer-term findings.
Examples of research evidence from selected Head Start studies:
Family and Child Experiences Survey (FACES) 2009
Head Start Impact Study (HSIS)
- Overall findings: Results from assessments of the 2009 FACES cohort demonstrate that Head Start involvement is associated with some school-readiness gains in the domains of cognitive development, social-emotional skills, and approaches to learning.
- Subgroup findings: Compared to children of all other races, black (non-Hispanic) and Hispanic/Latino children demonstrated the greatest gains in several measures of language or math cognitive development by the end of the first year in Head Start. Spanish-speaking children who were assessed in Spanish at baseline (as opposed to English) lagged significantly behind their English-speaking Head Start peers in all cognitive domains, and made progress in only letter-word knowledge across the first year.
Head Start Research-Based, Developmentally Informed (REDI) Intervention
- Overall findings: The impact study found that access to Head Start resulted in initial positive impacts during and at the end of preschool in all measured domains, including cognitive development, social-emotional skills, and health status and services.
- Subgroup findings: After one year of Head Start, positive cognitive impacts were particularly strong for Hispanic and black children. Head Start also had beneficial social emotional impacts for Spanish speaking children, Hispanic and white children, but mixed impacts for black children depending on the age cohort (3- or 4-year olds).
- Overall findings: Results suggest that exposure to the REDI program over a one-year period is positively associated with a number of school readiness measures of language development, emergent-literacy, and social-emotional skills.
- Subgroup findings: No analyses of subgroups based on child race/ethnicity or other demographic characteristics were conducted.
Back to top.
How should we interpret Head Start research evidence?
Although the empirical results of Head Start studies are, for the most part, not disputed, there is substantial debate about the interpretation of this research evidence and what it means for Head Start effectiveness. This debate is focused on the results of the Head Start Impact Study (HSIS), which was designed in an attempt to definitively answer the question of whether Head Start works.
The results of HSIS should be considered within the context of a long history of evidence on Head Start’s effectiveness. In 1997, the U.S. Government Accountability Office (GAO) released a report based on a comprehensive review of all existing Head Start research, which found that no conclusive statement could be made about Head Start effectiveness due to the small number of impact studies, an overemphasis on cognitive outcomes, weak research methods or designs, and lack of generalizable results.[83
] In the 1998 reauthorization of Head Start, HSIS was included as a direct response to this GAO report. Yet, to some degree, the HSIS study has not resolved the debate. Impact studies of other intensive localized preschool programs, such as the Abecedarian and Perry Preschool programs of the 1960s, have shown steady benefits well into adulthood. Some authors note that the shorter-term positive impacts on school readiness and the catching up of non-Head Start participants in elementary school found in HSIS seem negligible in comparison, and that Head Start is ineffective.[84
] However, other authors argue that when the entire body of Head Start research is examined (HSIS and non-experimental research), Head Start produces important short-term educational and health impacts related to the program's school readiness goals, as well as lasting effects in adult outcomes.[85
The debate is fueled by differing interpretations of HSIS research evidence. Traditional “What Works” impact assessments are important and can result in improvements and enhancements that make programs more effective. For example, the Coalition for Evidence-Based Policy suggested a reasonable conclusion based on its traditional impact assessment: Head Start needs improvements and should allocate funds to test more evidence-based practices (e.g. further curricula enhancements, professional development, mentoring and coaching for teachers), which may make the program more effective and produce long-term impacts. However, for a complex program like Head Start, relying on average impacts to determine effectiveness may be a narrow lens for interpreting evidence, and may result in the omission of important nuances, such as positive (differential) effects on subgroups of particularly vulnerable children. This could bar the program from the list of top tier evidence-based programs and have major implications, such as loss of public support or dollars for the program.
In contrast, an examination of the impact study results within the context of Head Start’s logic, capacity, and participant characteristics brings to light a more nuanced conclusion of program effectiveness. As of 1997, Head Start’s official goal
is to improve child school readiness; not school achievement at third grade or long-term adult outcomes. Moreover, the program’s capacity
is limited in significant ways as measured by funding, operations and quality indicators. For example, Head Start per-student funding is a fraction of that of the Abecedarian and Perry Preschool interventions,[86
] there is significant variation in child experiences within the program, and Head Start classroom ratings of instructional and social-emotional quality are average (as is the case for most preschool programs, a clear indication of the need for improved quality across the board). Within this context of program goals, constrained resources and high levels of variation, it is remarkable that studies were able to detect overall positive impacts on school readiness at kindergarten entry. Given the size of the effects, this average impact constitutes a modest success. Moreover, the later positive impacts in elementary school for particularly vulnerable subgroups, such as children from high risk households and black children, highlight important equity impacts and the role the program may have in closing school readiness gaps.
Perhaps the biggest take-away from HSIS is not a definitive conclusion about whether Head Start works, but rather a better understanding of how this program should be evaluated and improved. The flexibility inherent in the program’s design, and the substantial variation in how it is implemented in practice (operating schedules, languages, curricula, activities, etc.), make it difficult to capture a single Head Start experience. The variation may hide important impacts when assessing the overall “Head Start effect”, as in HSIS. It was with great foresight into the issues encountered by HSIS that the Advisory Panel for the Head Start Evaluation Design Project put forth research recommendations in 1990 stating that,
“An overall research strategy rather than a single large scale study is the appropriate framework for addressing critical Head Start research and evaluation questions. The panel recommends strongly against a single large-scale study of Head Start as the principal mechanism for seeking answers…Head Start is not, in any simple sense, a uniform “treatment.”[87
Indeed, the HSIS report evaluated an average of so many different experiences, that without an accompanying implementation analysis, the study likely overlooked critical nuances in what works for whom and why. On the other hand, smaller, local, and component-specific studies, such as the REDI intervention, yield cleaner, more conclusive results about the impacts of particular program components on specific child outcomes. While not as generalizable, these more focused studies may have more success detecting important impacts in Head Start due to the breadth and flexibility inherent in the program. Importantly, in all future evaluations of Head Start, large or small, implementation evaluation should be a key component because research suggests that positive results are more likely to be replicated when the fidelity and quality of implementation is high.[88
Mirroring the need to hone in on local variation in program evaluations, Head Start should also tap into local variation when making improvements. Along these lines, the Coalition for Evidence Based Policy and the Rand Corporation’s Promising Practices Network ratings of Head Start went beyond average impacts to note that within Head Start there are many different strategies and approaches to early education and care, and that the program should devote resources to expanding those strategies with proven effectiveness.[89
] Head Start should create strong incentives to adopt evidence-based curricula that integrate educational and socio-emotional components and practices that are proven to work for economically disadvantaged children, as well as for especially vulnerable subgroups, such as children with special needs and Dual Language Learners. Ultimately, identifying and expanding successful evidence-based practices within the heterogeneous Head Start population may be the key to improving school readiness for all children and narrowing school readiness gaps for the most vulnerable.
Back to top