Comprehensive Testing Program (CTP) norms are updated annually for each reporting cycle (fall and spring). These “rolling norms,” as they are called, are calculated after gathering three years of data. In the fourth year, we delete the earliest year and add data from the most recent administration.
CTP Norms During and After the Pandemic
During the pandemic, when many member schools suspended in-person testing, ERB “froze” the CTP norms from Fall 2020 to Spring 2022. Instead of updating them with limited data from that unusual moment in history, ERB provided members with the rolling norms established immediately prior to the pandemic.
Starting with the Spring 2022 reporting cycle and continuing into the Spring 2023 reporting cycle, ERB “unfroze” the rolling norms and updated them to be based on four years of data instead of three. This approach was used so that the norms were based on a mix of pre-pandemic and pandemic data. For the Spring 2024 reporting cycle, ERB is once again updating norms and continuing to base rolling norms on four consecutive years of data. This means the rolling norm for the Spring 2024 reporting cycle will not include data from before the pandemic (data prior to Spring 2020).
Overview of Norms for Spring 2024
Figure 1 shows the norms for the Spring 2024 reporting cycle are based on test-takers from 2020 through 2023, although test-taker volume was drastically lower in spring 2020.
Given that the volume of test-takers from Spring 2020 is extremely low, the norms for the Spring 2024 reporting cycle are largely driven by post-pandemic data from the last three years.
Figure 1. 2024 Spring CTP Norms Data and Test Volume by Year.
Spring 2024 Norms vs. Previous Reporting Cycles
When comparing the Spring 2024 rolling norms to the Spring 2023 rolling norms for independent school students, we found that the rolling mean continued to decrease for certain subtests, with larger decreases found at the upper-school level.
On average, the decrease is about 3 scale score points. When we further examined this phenomenon, we discovered that for certain subtests, scores in 2023 continue to rebound in many cases, especially for math-related subtests in Grades 3 to 8. For verbal subtests, students’ performance level has been relatively flat for the last three years. Nonetheless, the scores in 2023 were still consistently lower than the pre-pandemic scores in 2019. This resulted in an overall lower rolling norm in 2024 when compared to 2023 as we removed the pre-pandemic data and added the most recent year’s data for the rolling norms.
In Figure 2 below, average CTP scores for the four core subtests (Mathematics, Quantitative Reasoning, Reasoning Comprehension, and Verbal Reasoning) in Grades 3 through 10 are shown for the past four spring testing seasons. The sparklines in Figure 2 demonstrate visually the trend in performance over time.
Student performance in Reading Comprehension has remained mostly flat in the last few years and is not showing a rebound to the pre-pandemic level. Grade 9 and 10 performance in Mathematics, Quantitative Reasoning, and Verbal Reasoning continues to decrease post-pandemic.
Figure 2. Independent School Year-Over-Year Means for Four Core CTP Subtests.
What Does This Mean in Terms of Percentiles for Member Schools?
This means schools should expect to see a slightly higher percentile rank for a student with the same test score as compared to last year. For example, in the updated norms for Spring 2024, a Grade 8 student who has a scale score of 750 in Mathematics will have a percentile of 76. In the previous Spring 2023 norms, a Grade 8 student with the same scale score would have a percentile of 74.
These findings are consistent with the pandemic-related growth analysis that Rochon and Shuman (2021) conducted based on a cohort testing over two CTP fall windows. They found Quantitative Reasoning and Verbal Reasoning were two of the academic disciplines that were more impacted by the pandemic. In a subsequent study, using pre- and post-pandemic longitudinal data for two cohorts, Rochon and Shuman (2022) also found a restoration of pre-COVID levels of learning growth during 2021. The growth, however, is slower in verbal subtests than in quantitative subtests, which is also evidenced in the norms this spring.
These findings (although less severe) echo the results of a recent research study on the academic recovery of a national sample made up of mostly public school students. The authors of that study found that academic recovery has begun nationwide, but at the district level, students in many states are still behind in mathematics and reading comprehension as compared to 2019.
For more than a dozen states, the catch-up for public school students is equivalent to a third of a grade level or more in these two content areas. Given these results, it is even more crucial now to keep an eye on individual student growth to monitor progress over time.
Looking Ahead to Future Reporting Cycles
Moving forward, ERB will continue to use a four-year rolling norm. We also plan to roll out the student growth percentiles (SGP) in the upcoming school year to help educators better understand student progress over time.
As students recover academically from learning loss in the pandemic, some may need to grow more to catch up on what they have lost and stay on track with their goals.
Learn more about the Comprehensive Testing Program (CTP), ERB’s summative assessment for students in Grades 1-11 covering reading, listening, vocabulary, writing, mathematics, and science, or log into the 360 Access data reporting platform (for ERB members).
About the Author
Sihua Adrienne Hu, Ph.D. is ERB’s Senior Research Analyst. She supports test development and the research functions of ERB’s admission and achievement programs. Prior to joining ERB in 2020, Adrienne worked as a post-doctoral fellow at the School of Education and Social Policy at Northwestern University. As a researcher, she has published her works in the fields of mathematics education, education policy, and sociology of education. Adrienne earned her bachelor’s degree in mathematics and statistics from Sichuan University. She also holds a master’s degree in statistics and a doctorate in mathematics education from Michigan State University.
