Commentary on Watts, T. W., Jenkins, J. M., Dodge, K. A., et al. (2023). Understanding Heterogeneity in the Impact of PublicPreschool Programs. https://doi.org/10.1111/mono.12463

About the Author
Brenda Jones Harden
School of Social Work,
Columbia University
Brenda Jones Harden is the Ruth Ottman Professor of Child and Family Welfare at Columbia University School of Social Work. Her research examines the development and mental health of young children experiencing adversity, particularly maltreatment, foster care, or other forms of trauma. She focuses on preventing maladaptive developmental outcomes through early childhood programs, on which she has conducted numerous evaluations. The goal of her research is to improve the quality and effectiveness of child and family services and to inform child and family policy. She is currently the President of the Board at Zero to Three. She received a Ph.D. in developmental and clinical psychology from Yale University and a Master’s in Social Work from New York University.
Using Research to Advance Implementation of Public Pre-Kindergarten
In the middle of the last century, public schools in the United States (US) were contending with the integration of Kindergarten into traditional public education. In the first decades of the twenty-first century, we are at a similar crossroads at which Pre-Kindergarten (PreK) is rapidly becoming the first formal public educational experience for many young children. In fact, according to a recent report from the National Institute for Early Education Research (NIEER, 2023), almost all US states provide public PreK and almost half of US 4-year-olds were attending PreK or another publicly funded early learning program pre-COVID-pandemic, with attendance on the rebound (NIEER, 2023).
Although there is substantial evidence regarding the benefits that PreK confers upon young children, especially children from low-income backgrounds, there are many questions left to be answered. First, scholars are attempting to address the conundrum about the fade-out of the effects of PreK in early elementary school and beyond, sometimes referred to as the convergence of skills between control and intervention group children. An even more troubling finding emerged from the recent longitudinal evaluation of the Tennessee PreK program, which documented worse elementary school outcomes for children who had participated in this state-wide program.
Further, questions around the contexts of the PreK experience and their influence on PreK effects remain unanswered. For example, most studies rely on Regression Discontinuity or other designs that aggregate children across communities, schools, and racial/ethnic groups and do not collect data from children’s parents. Thus, there is limited evidence about the child and family factors that may moderate the impact of public PreK. Additionally, we know little about how other early childhood experiences may bolster or attenuate public PreK’s effects. Although PreK evaluations have been conducted in many parts of the US, most studies do not address the unique factors within and between states that may contribute to study findings.
Using rigorous and creative methods, the authors of this monograph have conducted a study of PreK in North Carolina (NC) that tackles these very complex issues. The study employed a unique approach that examined the effect of additional funding for NC PreK for each child in a county in a year, regardless of child’s program enrollment status, and had one of the largest sample sizes (over 1.2 million children) observed in the PreK literature. Undergirded by these methods, this study contributes to the literature that documents positive long-term effects (i.e., to fifth grade) of PreK on participant children, thus offering a counterargument to the fade-out hypothesis. Additionally, the study had a primary goal of examining the heterogeneity in a state PreK program as a potential explanation for some of the ambiguous findings of long-term effects of public PreK. In this vein, they examine variables that are often controlled or not collected in other preK studies, including child characteristics, and contextual factors related to early education, subsequent educational experiences, and the larger community.
Key aspects of this study are its grounding in theory, its aim to test three explicit hypotheses that may explain long-term outcomes, and its focus on interactions between PreK and other critical factors related to child outcomes. Building on an interdisciplinary approach that incorporates economic and developmental theories, the study tested whether PreK impacts reflected: 1) dynamic complementarity in that positive educational contexts bolstered PreK’s impact; 2) a compensatory effect such that PreK conferred more advantage to children from low-quality environments; or 3) an additive effect whereby PreK and other environmental supports in tandem positively affected children’s outcomes. This approach is cutting-edge in PreK research as often these studies are atheoretical and solely focus on the more simplistic question of whether PreK attendance differentiates the outcomes between the intervention group and the control/comparison group.
To address these theoretically-driven hypotheses, the authors focused on three compelling sets of interactions/moderators: child and family individual factors; educational environmental factors including those in the early childhood and later educational arenas; and economic features of subsequent community contexts. The authors approached their decision-making regarding data sources in a creative manner that arguably should be more prevalent in developmental research. Specifically, they capitalized on administrative records (e.g., birth and educational records, public school quality data, county level social-economic information) to derive information that is almost non-existent in PreK studies. It is important to acknowledge how challenging these data are to collect and how time-consuming it is to build relationships with the agencies that control these data so that they are willing to release and explain them to support studies of this type.
It is also important to state the obvious – that administrative data are limited. Thus, the authors were not able to obtain certain data that would enhance their data analyses and interpretation. For example, they were not able to obtain information on retention and special education placement prior to the third-grade year. This is an important period for understanding children’s trajectories in formal schooling (e.g., Ehrhardt et al., 2013), and may to some extent inform why they were unable to detect differences in the intervention and control groups regarding retention and special education in fifth grade (though clearly the methodological issues the authors present are critical). As the authors themselves acknowledge, there are many other developmental processes that are not available through administrative data that are important to consider. For example, executive functioning, self-regulation, and social-emotional functioning are outcomes that many PreK evaluations do not address but are linked to later academic functioning and well-being (Phillips, Lipsey et al., 2017; Yoshikawa et al., 2013).
Another disadvantage is that they had to rely on the administrative data for racial codes. Potential problems in this vein relate to the lack of clarity that accompanies the mixed-race code. Although there is emerging consensus on including the mixed-race category in research, there is some evidence that African Americans are more reluctant to wholly support such designation due to the historical, psychological, and sociological complexity regarding racial categories in this population (Brunsma, 2005; Qian, 2004; Wright, 2016). Additionally, the marital status code may obscure situations in which intimate but unmarried adult partners may be co-parenting and does not capture intergenerational households in which two related adults (e.g., grandmother and mother) may be supporting each other and the children with whom they reside. Both these family characteristics have been found to enhance children’s outcomes in early childhood and beyond (Bertera & Crewe, 2013; McHale & Jones, 2021).
The experience of African American children and children from other minoritized groups as a result of structural racism is garnering increased attention in the empirical literature. For example, Iruka and colleagues (Iruka, 2022; Iruka et al., 2022) have highlighted the importance of examining early childhood policy, practice, and research from an anti-racist lens, and argue for the disaggregation of African American students in early childhood studies. To the current authors’ credit, they disaggregated by race to examine the impacts of PreK on specific racial/ethnic groups. They found main effects of PreK for African American children, as well as larger effects when compared to White children. As Iruka (2022) states, the study summarized in this monograph, as well as many other current and historical studies, have documented the benefits of early care and education for African American children.
A compelling finding from the examination of African American students pertains to their achievement in the context of school performance. Overall, the findings supported the evidence that African American students are subjected to structural racism in the education system, as they were more likely to be in low-performing schools than their White counterparts (Merolla & Jackson, 2019). In regard to PreK, the effects were more pronounced for African American students in schools performing in the mid-range. However, the effects trended toward the negative for African American students in higher performing schools. Some studies have suggested that high-performing schools, given their lower proportions of students from minoritized backgrounds, the lack of “belongingness” perceived by African American students, and the potentially lower levels of academic preparation experienced by African American students when compared to white students, may affect their achievement in these settings (Bohrnstedt et al., 2015; Chapman & Bophal, 2019).
Main effects (as well as larger effects) were also found for Hispanic children, which is consistent with many studies of PreK (Gormley, 2008; Jones Harden et al., 2023; Laosa & Ainsworth, 2007). Notably, only 7% of the participants in this study were Hispanic, which reflected the proportion at the time the PreK service was delivered. However, given the growing number of Hispanic children in NC and the US in general, it is critical that future studies of NC PreK program address this population given its heterogeneity (e.g., urbanicity, immigrant status, country of origin, etc.). As the authors note, the NC study suggests that PreK may reduce early educational disparities between children from minoritized backgrounds and white children, particularly African American and Hispanic children.
Finally, main effects were found for children whose mothers had lower levels of education. This finding is consistent with many studies of PreK, which point to greater effects for children from more disadvantaged backgrounds (e.g., Weiland & Yoshikawa, 2013). Along with the greater impacts on African American and Hispanic children, the findings relative to lower education suggest that PreK funding may be better expended on children who are at risk for poorer outcomes. According to NIEER (2023), current PreK policy aligns with this research in that the majority of programs are not universal and therefore target children under a certain income threshold. These findings inform the debate around universal vs. targeted programs in the preventive arena (e.g., Dodge, 2020), suggesting that early care and education programs may be best delivered as a targeted intervention.
Beyond demographics, family risk is another factor to consider as a moderator of PreK effects. As Phillips and colleagues (Phillips, 2016; Phillips, Johnson et al., 2017) argue, it is critical to examine whether PreK is effective for young children from impoverished backgrounds who are exposed to high levels of adversity, such as “toxic stress.” In our work in an urban area with very high levels of criminal activity, we found that PreK was beneficial for participant children (Jones Harden et al., 2023). Although this type of examination would likely require more than just administrative data, the NC study could contribute substantially to our understanding of heterogeneity as a moderator of PreK outcomes had it been able to address family risk in some way.
Interestingly, although this was a study of PreK effects, the authors were able to capture more information on the quality of subsequent schooling (though limited in and of itself) than the quality of the PreK experience. It is critical to garner data on PreK quality in order to understand its effects and to situate findings in the context of other studies of PreK. The monograph authors acknowledge that NC has a mixed delivery system, which suggests wide variability in the quality of early childhood program provided. Although NC has a mandate that PreK programs must meet a specific threshold of structural and process quality, there is still heterogeneity that is not captured by these global measures. In our study of Baltimore City PreK, we found that PreK quality was variable based on the domains of the Classroom Assessment Scoring System (CLASS; Pianta et al., 2008). On average, classrooms in our study scored in the mid-range on Emotional Support and Classroom Organization, and in the low range of Instructional Support. Given evidence on the relation between early childhood classroom quality and child outcomes (Burchinal et al., 2009; Perlman et al., 2016), a major disadvantage of the current study is the lack of data on the quality of the PreK programs examined.
The study did not reveal any influence of the early childhood service delivery context on PreK impacts. Again, due to the limitation of administrative data, the authors were only able to examine factors such as access to alternative early childhood programs via county saturation of specific early childhood programs, such as Head Start, Smart Start, and Early Head Start. In our study of PreK (Jones Harden et al., 2023), we were able to ask parents about their children’s experiences in other early childhood programs, which revealed that approximately half of the children in both the intervention and control groups had other early childhood experiences. Despite this finding, we still observed positive impacts of PreK. It is important to note that saturation may have a different meaning in regard to programs for children under 3 years of age than those serving preschool children. For example, the national data suggest that Early Head Start only serves 11% of eligible children (Zero to Three, 2023), although there is state-specific variation. Further, because NC Smart Start is community-driven and addresses many different goals (Smart Start Annual Report, 2022), its relation to the goals of PreK may be very different depending on the county.
Although most of the findings from this study regarding environmental heterogeneity confirmed additive effects of most variables, PreK proved to be a protective factor against some environmental factors, such as subsequent school factors (e.g., low school achievement, low school-level federal funding, lower levels of teacher experience). In addition, compensatory effects were found for SNAP benefits, which was examined in one of the most innovative strategies employed by the research team (i.e., instrumental variable analysis of the community economic context). Although the authors define a larger level of SNAP receipt as a negative environmental factor, in some research expanded SNAP is perceived more positively (i.e., proxy for a jurisdiction more supportive of families from disadvantaged backgrounds). For example, research has shown that jurisdictions with expanded concrete resources for families (e.g., SNAP, Medicaid, TANF, etc.) have lower rates of infant mortality/morbidity (Prenatal-to-3 Policy Impact Center, 2022) and child maltreatment (Maguire-Jack et al., 2022). Another focus of the instrumental variable analysis was to examine increases in annual funding for PreK in individual counties. Similar to expanded SNAP, this variable may also reflect a broader and higher level of community and governmental support for children and other vulnerable populations.
Overall, this monograph reported on an innovative and rigorous study which addresses many thorny questions in the literature on PreK effects. It provides a timely set of additional findings documenting long-term benefits of PreK. Although there are clearly many “omitted” variables in this study (see Duncan et al., 2010), the authors’ examination of individual and environmental heterogeneity informs policy and practice regarding the factors that may bolster or attenuate PreK’s impact. Future research should continue to delve into these child, family, and contextual processes, as well as incorporate data on PreK quality. Moreover, policy research should address the question of the modest impacts of PreK in this and other studies. The scientific knowledge, garnered from this and other studies, should be used to advance the implementation of PreK programs as well as early childhood programs writ large in order to have a more substantial impact on young children’s development, especially those from low-income and minoritized backgrounds. An early childhood system of care (Goodman et al., 2019), which could include PreK and other services (e.g., home visiting, family support, child care, early childhood mental health consultation, economic supports), may allow us to reach that goal.
References
Bertera, E. M., & Crewe, S. E. (2013). Parenthood in the twenty-first century: African American grandparents as surrogate parents. Journal of Human Behavior in the Social Environment, 23(2), 178-192.
Bohrnstedt, G., Kitmitto, S., Ogut, B., Sherman, D., and Chan, D. (2015). School Composition and the Black–White Achievement Gap (NCES 2015-018). U.S. Department of Education, Washington, DC: National Center for Education Statistics. Retrieved [date] from http://nces.ed.gov/pubsearch.
Brunsma, D. L. (2005). Interracial families and the racial identification of mixed-race children: Evidence from the early childhood longitudinal study. Social Forces, 84(2), 1131-1157.
Burchinal, P., Kainz, K., Cai, K., Tout, K., Zaslow, M., Martinez-Beck, I., & Rathgeb, C. (2009). Early care and education quality and child outcomes. Research-to-Policy, Research-to-Practice Brief). Washington, DC: Child Trends and the Office of Policy, Research and Evaluation, Administration for Children and Families.
Chapman, T. K., & Bhopal, K. (2019). The perils of integration: exploring the experiences of African American and black Caribbean students in predominately white secondary schools. Ethnic and Racial Studies, 42(7), 1110-1129.
Dodge, K. A. (2020). Annual Research Review: Universal and targeted strategies for assigning interventions to achieve population impact. Journal of Child Psychology and Psychiatry, 61(3), 255-267.
Duncan, G. J., Ziol‐Guest, K. M., & Kalil, A. (2010). Early‐childhood poverty and adult attainment, behavior, and health. Child development, 81(1), 306-325.
Ehrhardt, J., Huntington, N., Molino, J., & Barbaresi, W. (2013). Special education and later academic achievement. Journal of Developmental & Behavioral Pediatrics, 34(2), 111-119.
Goodman, W. B., O’Donnell, K., Murphy, R. A., & Dodge, K. A. (2019). Moving beyond program to population impact: Toward a universal early childhood system of care. Journal of Family Theory & Review, 11(1), 112-126.
Iruka, I. U. (2022). Delivering on the promise of early childhood education for black children: An equity strategy. New Directions for Child and Adolescent Development, 2022(183-184), 27-45.
Iruka, I. U., Gardner-Neblett, N., Telfer, N. A., Ibekwe-Okafor, N., Curenton, S. M., Sims, J., … & Neblett, E. W. (2022). Effects of racism on child development: Advancing antiracist developmental science. Annual Review of Developmental Psychology, 4, 109-132.
Jones Harden, B., Brett, B. E., Gross, J. T., Weiland, C., Klein, E.L., & Tirrell-Corbin, C., & Berne, J. (2023). The impact of pre-kindergarten expansion on children in Baltimore, MD. Early Childhood Research Quarterly, 64, 1-12.
Laosa, L. M., & Ainsworth, P. (2007). Is public pre-K preparing Hispanic children to succeed in school. National Institute for Early Education Research Policy Brief, 13.
Maguire-Jack, K., Johnson-Motoyama, M., & Parmenter, S. (2022). A scoping review of economic supports for working parents: The relationship of TANF, child care subsidy, SNAP, and EITC to child maltreatment. Aggression and violent behavior, 65, 101639.
McHale, J. P., & Jones, S. C. (2021). Mothers, fathers, and coparenting others. In The Routledge Handbook of Family Communication (pp. 203-220). Routledge.
Merolla, D. M., & Jackson, O. (2019). Structural racism as the fundamental cause of the academic achievement gap. Sociology Compass, 13(6), e12696.
National Institute of Early Education Research (NIEER, 2023). The State of Preschool Report 2022. Brunswick, NJ: Rutgers University.
North Carolina Partnership for Children (2022). Smart Start Annual Report for Fiscal Year 2021-2022. Retrieved from https://www.smartstart.org/about-smart-start/annual-reports/
Perlman, M., Falenchuk, O., Fletcher, B., McMullen, E., Beyene, J., & Shah, P. S. (2016). A systematic review and meta-analysis of a measure of staff/child interaction quality (the classroom assessment scoring system) in early childhood education and care settings and child outcomes. PloS one, 11(12), e0167660.
Phillips, D. (2016). Integrating enriched learning and protection from toxic stress in early education settings. In S. Jones & N. Leseau (Eds.). Leading Edge in Early Childhood Education, (pp. 7-28). Cambridge, MA: Harvard University Press.
Phillips, D. A., Johnson, A. D., Weiland, C., & Hutchison, J. E. (2017). Public Preschool in a More Diverse America: Implications for Next-Generation Evaluation Research. Working Paper Series, No. 2-17. Poverty Solutions, University of Michigan.
Phillips, D. A., Lipsey, M. W., Dodge, K., Haskins, R., Bassok, D., Burchinal, M. R., … & Magnuson, K. & Weiland, C. (2017) Puzzling it out: the current state of scientific knowledge on prekindergarten effects. The Current State of Scientific Knowledge on Prekindergarten Effects. Washington, DC: Brookings Institution.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System™: Manual K-3. Paul H Brookes Publishing.
Prenatal-to-3 Policy Impact Center (2022). State options and actions to extend postpartum coverage in Medicaid (B.007.0222). Peabody College of Education and Human Development, Vanderbilt University. https://www.pn3policy.org/resources/state-options-and-actions-to-extendpostpartum-coverage-in-medicaid
Qian, Z. (2004). Options: Racial/ethnic identification of children of intermarried couples. Social Science Quarterly, 85(3), 746-766.
Watts, T., Jenkins, J. M., Dodge, K. A., et al. (2023). Understanding Heterogeneity in the Impact of Public Preschool Programs. Monographs of the Society for Research in Child Development, 88(1). https://doi.org/10.1111/mono.12463
Wright, L. (2016). One drop of blood. In Double Exposure (pp. 57-71). Routledge.
Yoshikawa, H., Weiland, C., Brooks-Gunn, J., Burchinal, M. R., Espinosa, L. M., Gormley, W. T., … & Zaslow, M. J. (2013). Investing in Our Future: The Evidence Base on Preschool Education. Society for Research in Child Development.
Zero to Three (2023). State of Babies Yearbook: 2022. Washington, DC: Zero to Three.
Citation:
Jones Harden, B. (2023). Using Research to Advance Implementation of Public Pre-Kindergarten. [Peer commentary on the article “Understanding Heterogeneity in the Impact of Public Preschool Programs” by T. W. Watts, J. M. Jenkins, K. A. Dodge et al. Monograph Matters. Retrieved from https://monographmatters.srcd.org/2023/06/13/commentary-jonesharden-88-1/