For the past two and half years, the RESTART Network has been tasked with conducting a systematic search for studies on district and school activities focusing in COVID-19 pandemic recovery and academic acceleration. We identified and reviewed academic recovery studies published between January 2020 and March 2024. We focused on critical topics identified by policymakers, school leaders and practitioners as imperative. These include: Literacy, Math, Student Wellbeing, Accelerated Learning, Tutoring, Expanded Learning Opportunities, and Educator Pipeline and Retention.
We discovered a wealth of information on what educators have been doing, but found a lack of rigorous studies, and consequently evidence, related to the effect these efforts have on helping students to recover from the effects of the pandemic. We realize that with the disruption and necessary focus on day-to-day needs and operations and then focus on recovery for students, staff, families and communities, research and evidence building were not a priority.
Why is there a Lack of Rigorous Evidence?
We really do not know whether the activities and strategies schools and districts used to support recovery in our schools worked or not. Few of the eligible studies we found used rigorous causal research methods to determine the effectiveness of these strategies in schools in the pandemic or post-pandemic eras. For example, out of 103 eligible educator pipeline and retention activities1 we reviewed, only nine included quantitative evidence of a causal impact2.
It’s no surprise there is a general lack of rigorous research. First, educators were in survival mode, which meant that priorities shifted starkly away from research participation and continuous improvement. Instead, they were focusing on public health, adapting to new technologies, reorganizing staffing, and supporting student well-being amid escalated challenges such as isolation and fear for safety. Despite these new interventions, the focus was not on evaluating their impact but on getting through this novel situation.
Second, rigorous research takes time. Not only were educators scrambling to find their way in the pandemic, but it also took the research community time to adjust, seek funding for new questions, and conduct the rigorous research. For those reasons, we are just beginning to see rigorous research released in the last year.
Rigorous Evidence More Important Now More Than Ever
Evidence-Based Pandemic Recovery Strategy: High Dosage Tutoring
The University of Chicago Education Lab found evidence that high dosage tutoring during school hours may improve math scores and reduce math course failures.
In addition to this rigorous evidence, Education Research for Action have translated the impact about high dosage tutoring into design principles for educators.
Georgia Policy Labs have examined the effects of virtual summer school on student achievement. They found that participation in summer school improved math scores but not reading.
Studies that provide rigorous evidence are more important now than ever. We continue to see significant and long-term disruptions to our schools sometimes caused by fires or storms. For instance, school has been disrupted in Asheville for flooding and in Los Angeles and surrounding areas for fire. We know that the post-COVID era is likely to be marked by disruptions to student learning for days, weeks, or months, due to these environmental or public health emergencies. In order to improve our responses to future school interruptions, we must learn from the pandemic. We need to improve our understanding of how to respond to these disruptions and how to best support our students and teachers.
The research community has a responsibility to build policymakers, leaders and practitioners toolboxes to be at the ready to respond to significant disruptions to the public education system. We need to improve our understanding of what works, when it works and for whom. For example, are there strategies that are uniquely suited for supporting students in recovery for a disruption? Or, do evidence-based strategies with evidence in pre-pandemic context need to be adapted during significant disruption? Are they still effective in the disruption context? And, do pre-pandemic evidence based practices still have the same effect in a post-pandemic context?
This is a call to the research community. We need more information. By all accounts our students and educators are still in need of recovery support. Some researchers have begun to create these tools such as the Act on Evidence Toolkit from Education Research for Action.
We need to continue our pursuit for evidence and for ways to systemically review and share effective strategies to policymakers, leaders and practitioners via trusted clearinghouses for example. Further, we need to be sure these resources are at the ready to be quickly adopted with clear insights into the contextual factors, skills, technology needed to implement a given strategy with success.
1 Our focus is on studies that address educator pipeline and retention strategies in the pandemic recovery context.
2 Six studies provided quantitative evidence at two or more timepoints, 4 studies provided quantitative evidence for two or more groups including 1 study provided both data about two or more groups at two or more timepoints.