Commentary on Carroll, J. M., Yeager, D. S., Buontempo, J., et al. (2023). Mindset × Context: Schools, Classrooms, and the Unequal Translation of Expectations into Math Achievement. https://doi.org/10.1111/mono.12471
About the Author
Barbara Schneider
College of Education and the Department of Sociology,
Michigan State University
Barbara Schneider is the John A. Hannah Chair and University Distinguished Professor in the College of Education and the Department of Sociology at Michigan State University. Her research focuses on how the social contexts of schools and families influence the academic and social well-being of adolescents into adulthood. Schneider has also designed tools and research methods for measuring real-time everyday experiences that advance learning. A Fellow of the American Association for the Advancement of Science, National Academy of Education, American Educational Research Association, and Finnish Academy of Science and Letters, and holds an Honorary Doctor of Philosophy degree from the University of Helsinki. Professor Schneider has published over 150 articles and reports and written 20 books, three of which include: Trust in Schools: A Core Resource for Improvement; The Ambitious Generation: America’s Teenagers Motivated but Directionless; and Learning Science: The Value of Crafting Engagement in Science Environments.
About the Author
Lydia Bradford
College of Education,
Michigan State University
Lydia Bradford is an advanced graduate student in Measurement and Quantitative Methods at Michigan State University College of Education. She graduated from Duke University with a bachelor’s degree in Romance languages and Global Health with a minor in Economics in 2017. Before starting her doctorate, Lydia was a high school chemistry and economics teacher which has led her to an interest in science education; however, her main research interest lies in statistical modeling, analysis, research design, mediation and moderation effects, and causal inference within the context of educational research. In the fall of 2019, Lydia began working on two separate science curriculum interventions, Crafting Engaging Science Environments and Multiple Literacies in Project Based Learning. In these projects, she has participated in data construction and data management as well as various statistical analysis of both the main treatment effects and other exploratory analysis and applications of machine learning for scoring teacher observations.
Growing Growth-Mindset
Value of the Study
Growth mindset has unquestionably established itself in the canon of psychological scholarship, and its contribution for impacting learning is widely recognized. The recent study by Carroll, Yeager, and co-authors marks yet another opportunity to extend mindset’s theoretical and practical value by applying it specifically to expectations for success in high school mathematics. Our commentary briefly highlights the merits of this work, including its methodological considerations. We then turn to what design lessons we can learn from this large generalizable study of over 9,000 students in 56 high schools throughout the US.
Importance of the Study. Drawing upon the National Study of Learning Mindsets (NSLM) and its representative sample of diverse high schools and their ninth-grade students, the Mindset x Context study linked students’ experiences in their mathematics classrooms with the courses they eventually enrolled in as 10th graders. The title, Mindset x Context, immediately warns the reader that this is not a typical psychological study, but one that intentionally bridges psychological measures with sociological contexts of classrooms. Here, the context is where the teacher and peer group informal resources shape sustaining student expectations, which result in later mathematical course enrollments. One of the major contributions of this work is the deliberate, creative combining of fields to selectively choose factors recognized as influencing expectations, including: parent socio economic resources; gender, race, and cultural backgrounds; and classroom climate measures such as belonging, stereotyping, and challenging mathematical problem-solving activities.
The study’s use of tenth grade mathematics course enrollment as a dependent measure is particularly advantageous for estimating students’ schooling experiences and has been shown in other studies to lead to several positive outcomes, including college enrollment. The study’s review of the education and sociological literature explained how high school mathematics course taking operates in a hierarchical tracking system in which failure to move ahead each year from one more difficult mathematics sequence to another bemoans disaster, especially for those students who fail Algebra I at the end of ninth grade (Jiang et al., 2020). The students’ mathematics course sequences are fully described and detailed (see Figures 2-3 of the monograph).
Design and Measures. Using the well-constructed design of the NSLM study alleviated many of the worries of sample attrition (see Table 1 of the monograph), of the ability to measure reasonable heterogeneity among schools and participants, and of the reliability and validity of key measures. The study thoughtfully used growth mindset measures by modeling “fixed” mindset as a covariate. This helped to ensure that the relationship between expectations for success in math and course taking was not convoluted by students with a “fixed” mindset. Another important design consideration was the intensive examination of other psychological measures nested in school and classroom contexts. The inclusion of these contextual measures minimized the deterministic dispositions of personal beliefs by identifying how school, classrooms, and peer and teacher actions influence student mindset.
Analytic Procedures. There were several analytic procedures that strengthened this study by employing Bayesian techniques to estimate the heterogeneity of expectations by individual and contextual measures and undertaking an intersectional analysis of expectations of success by gender and race/ethnicity. The authors explained the benefits of using the Bayesian Causal Forest (BCF) model including: 1) accounting for confounding effects; 2) examining multiple complex interactions; and 3) taking full advantage of the Bayesian method, which reduces the concern of Type I errors. These Type 1 errors can often occur with multiple comparisons, which increases the chance of finding a significant result when there is not one. Multiple comparisons become a concern when investigating many different interactions and comparing each subgroup with one another. Although Bayesian methods are often used for addressing Type I and Type II errors, which the authors discussed, they are not commonly used in education, psychology, or sociology studies. However, such techniques could be valuable for answering many research questions. The second benefit, which the authors also explicitly noted is how the combination of Bayesian methods with machine learning allowed for multiple interactions, again without increasing the risk of Type I errors through the “regularization” process. This is an important analytic consideration as researchers are now focusing more on contextual moderating effects in observational and experimental studies.
Findings. When results complement the work of others, as shown in this study, it is a major accomplishment, especially when using a dataset of this breadth and complexity. The identification of low-SES families being the most vulnerable group and the one most likely to benefit from raising expectations is, from our perspective, a finding that has held across several major recent sociological and economic studies. We think it is important to underscore that it is boys in low-SES families that are most at risk here (recognizing the lack of race/ethnicity subgroups analysis within boys from low-SES families because of power issues). Another important point here is the value of expectations, which has been prominent in sociological studies from the Equality of Educational Opportunity Report (Coleman et al., 1966) through today (Saw et al., 2018). Educational expectations remain a strong motivator, here related to higher math courses rather the sociological factors, which relate more broadly to achievement in school, college enrollment, and adult success.
Future Considerations that Arise from this Work
This work establishes a strong foundation for connecting psychological and sociological research to understand how context affects the relationship between psychological measures and academic outcomes. In planning for future studies of this caliber, there are several design and measurement considerations that can be learned from this study.
Data Collection. The first consideration focuses on the design regarding when measures are collected. Most of the measures in this study are collected at the beginning of the school year, when the students are unaccustomed to high school, their math teacher, or their peers. When studying contextual factors that may change (such as using students’ perceived stereotype threat from their teachers), a single time point may be insufficient to measure the true contextual factors in a student’s day to day life. Specifically, for perceived stereotype threat, measuring this in the fall as opposed to the spring before the students have settled into their classrooms may skew the self-reporting of perceived stereotype threat. Additionally, although the authors establish expectations as being “situation-general,” psychological measures within-students can change over time. A more robust design may include measuring student expectations at multiple timepoints and allowing those to interact with contextual factors at the school and classroom levels. Overall, when considering contextual factors and psychological measures of students, a design that includes multiple data collection points may bring more insights into the interplay of students and their context.
Measures. The second consideration is the measurement of school informal resources and teacher and classroom stereotype threat. Carroll, Yeager, and colleagues defined informal resources as “peer challenge-seeking norms,” which they measured through having students choose math problems based on their challenge level and then calculate the school average number of high challenging questions that were chosen. Schools were then split into high challenge-seeking and low challenge-seeking settings. This way of measuring peer challenge-seeking norms takes challenge-seeking out of context (such as the subject, type of problems, peers within the classroom), whereas challenge-seeking is likely to be a contextual based measure. Additionally, this measure does not consider students’ interests or skill sets when looking at the problems provided by the researchers (Schneider, et al., 2022). High challenging-seeking norms is a hypothetical measure as opposed to directly measuring whether most students take on challenges and the variation of a challenge-seeking classrooms and school environment.
Additionally, Carroll, Yeager, and colleagues measured classroom stereotyping through a single question of whether the students worry that people will judge them by their gender in math. They collected this measure at the beginning of the year and averaged it across identity groups in the classroom. However, perceived stereotyping is not the same as actual stereotyping. For future considerations, it would be helpful to corroborate these findings with additional measures of stereotyping within the classroom, such as teacher bias and discriminatory practices (Papageorge, et al., 2020). As noted above, the collection of perceived stereotyping happens at the beginning of the year before students fully know their classmates and teachers; therefore, more data points of both students’ perceived stereotype threat and teacher bias would increase the credibility of the measure. Finally, an additional consideration is the use of expectations as the major variable of interest. Carroll, Yeager, and colleagues found that most students had high expectations. This raises the question of whether this is the variable that should be the major focus of research or are there other questions and areas that may influence math course enrollment more than expectations, such as supportive instruction, culturally responsive instruction, alignment of career goals with expectations, skills, and others.
Analysis. One of the issues in propensity score matching is how to account for unobserved heterogeneity. The authors have tackled this issue within their multilevel BCF model, which accounts for this heterogeneity with respect to both the intercept and the slope. Reporting of both estimates for the intercept and the slope is critical. In this case, the results of the random intercepts help us to understand the variation in 10th grade math enrollment by context after controlling for the predictors (e.g., prior achievement, expectations, SES, fixed mindset). The results of the random slopes help to understand additional unobserved variation from the schools and classrooms on the effect of expectations and its interaction with race/ethnicity and SES. Because of the importance of these estimates, lack of clarity in reporting of these random effects (as found in this monograph) can lead to an underrepresentation of the school and classroom level heterogeneity of the treatment (expectations) on the outcome.
The suggestions above are some ideas for executing additional studies of this scope and depth in the future. What this work does show is how creative and methodologically rigorous scholars can use a dataset and report compelling new results from that work. Datasets of this caliber allow researchers to further their knowledge and, as they construct future studies, additional considerations should be taken regarding the measures and possibilities for linking data to other sources to reduce the burden of data collection and strengthen contextual information. One of these are the racial structural effects found in high schools, where advanced level mathematics courses are not offered and racialized tracking practices persist (Francis & Darity, 2021). Carroll, Yaeger, and colleagues leave a key important message to psychology and broader research communities that “psychological measures are insufficient in addressing disparities… without also addressing the inequality-promoting forces within schools and classrooms.”
References
Carroll, J. M., Yeager, D. S., Buontempo, J., Hecht, C., Cimpian, A., Mhatre, P., Muller, C., & Crosnoe, R. (2023). Mindset × Context: Schools, Classrooms, and the Unequal Translation of Expectations into Math Achievement. Monographs of the Society for Research in Child Development, 88(2). http://doi.org/10.1111/mono.12471
Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., Weinfeld, F., & York, R. (1966). Equality of Educational Opportunity. US Department of Health, Education, and Welfare, Office of Education.
Francis, D. V., & Darity, W. A. (2021). Separate and unequal under one roof: How the legacy of racialized tracking perpetuates within-school segregation. RSF: The Russell Sage Foundation Journal of the Social Sciences, 7(1), 187-202.
Jiang, S., Simpkins, S. D., & Eccles, J. S. (2020). Individuals’ math and science motivation and their subsequent STEM choices and achievement in high school and college: A longitudinal study of gender and college generation status differences. Developmental Psychology, 56(11), 2137.
Papageorge, N. W., Gershenson, S., & Kang, K. M. (2020). Teacher expectations matter. Review of Economics and Statistics, 102(2), 234-251.
Saw, G., Chang, C. N., & Chan, H. Y. (2018). Cross-sectional and longitudinal disparities in STEM career aspirations at the intersection of gender, race/ethnicity, and socioeconomic status. Educational Researcher, 47(8), 525-531.
Schneider, B., Chen, I.-C., Bradford, L., & Bartz, K. (2022). Intervention initiatives to raise young people’s interest and participation in STEM. Frontiers in Psychology, 13. https://www.frontiersin.org/articles/10.3389/fpsyg.2022.960327
Citation:
Schneider, B. and Bradford, L. (2023). Growing Growth-Mindset. [Peer commentary on the article “Mindset × Context: Schools, Classrooms, and the Unequal Translation of Expectations into Math Achievement” by J. M. Carroll, D. S. Yeager, J. Buontempo, C. Hecht, A. Cimpian, P. Mhatre, C. Muller, and R. Crosnoe]. Monograph Matters. Retrieved from https://monographmatters.srcd.org/2023/08/14/commentary-schneiderbradford-88-2/