What Works Clearinghouse
Curriculum-based interventions for increasing K-12 math achievement — middle school
Institute of Education Sciences
Intervention report Saxon Math updated December i , 2004
Intervention Saxon Math curricula and materials are available for grades K through 12, with the content and skills designed to meet National
Council of Teachers of Mathematics (NCTM) and various state standards. Each course lasts for one year, and students participate in
120-lesson courseware packages that last for about 60 minutes a day. Used in teacher-led lessons, Saxon Math is designed to teach
in increments, provide continual practice, and test cumulative learning every five lessons.
For Middle school students.
Findings One randomized controlled trial on Saxon Math found no significant difference in posttest scores between students using Saxon Math
and the comparison group using the University of Chicago Mathematics Project NCTM curriculum. One quasi-experimental design
study found that students using Saxon Math had higher gains in overall math, math computation, and math concepts compared with
other students. However, the study analyzed the data at the wrong level, making it impossible to accurately determine the significance
of the finding.
Evidence base 1 randomized controlled trial meets evidence standards.
y 1 quasi-experimental design study meets evidence standards with reservations.
G 4 studies do not meet evidence screens. ( see S y m £> 0 / key on page 7)
Evidence limits The evidence base is limited to two studies. The first is a randomized controlled trial of 8th-grade students in a rural-suburban
Nebraska junior high school. A second study is a quasi-experimental design study of 8th-grade students in Oklahoma City middle
schools. Quasi-experimental studies provide weaker evidence of effects because it is possible that unmeasured differences between
the groups affected the findings. Further, this study analyzed the data at the wrong level, which may bias the findings. The samples for
both studies were small (36-78 students). Four studies do not meet evidence standards.
Scope of use Saxon Math’s first textbook (Algebra I for 9th grade) was implemented in 1980, and Saxon Algebra 1/2 (8th grade) was implemented
in 1986. Information is not available about the number and demographics of students, schools, or districts using the intervention.
Developer and contact Saxon Publishers, www.saxonpublishers.com ; email: info@saxonDublishers.com : telephone: (800) 284-7019.
WWC Intervention Reports
Profile Saxon Math focuses on fundamental mathematics skills, targeting
children from kindergarten through grade 12. This report focuses
on middle school math, defined as grades 6 through 9. The 6th
grade curriculum covers simplifying expressions containing
parentheses, graphing functions, and understanding ratios and
proportions. The 7th grade curriculum covers pre-algebra topics
such as rate, powers, roots, and geometric proofs. The 8th grade
curriculum covers all topics usually taught in pre-algebra in
addition to topics from geometry and discrete mathematics. The
9th grade curriculum covers all topics usually taught in a first-year
algebra course (such as exponents, roots, and algebraic word
problems) as well as conceptual understanding, procedural
fluency, strategic competence, adaptive reasoning, and productive
disposition. As stated by the developers, Saxon Math covers all
five content and skill areas of NCTM standards and meets various
state standards.
Teaching
Each grade of Saxon Math consists of 1 20 daily lessons and 1 2
activity-based investigations. A daily lesson consists of warm-up
(10-15 minutes), introduction to the new concept (5-10 minutes),
practice focusing on new concept (5 minutes), and mixed practice
focusing on new and previously learned concepts (20-30 minutes).
Students are introduced to concepts incrementally, given
opportunities for continual review and practice, and assessed
cumulatively and frequently (every fifth lesson). An assessment
score of 80% or lower indicates a need for remediation, and a
provision for remediation is part of the program.
Lessons are designed to be one hour daily (this includes
practice and review time), and assessments occur every fifth
lesson, usually on Friday each week.
The teacher is responsible for facilitating and mediating the
warm-up session, introducing the new concept, and conducting
the practice sessions. Teachers introduce the daily concept
using manipulatives or representative models accompanied by
the procedures needed to solve the problem. Teachers are
instructed to conduct lessons in sequence, not skip lessons,
limit direct instruction to 10-15 minutes of group (or individual)
instruction, spend the majority of class time allowing children to
do mathematics problems in the problem sets, and assign all
problems in each set.
Supports are available for teachers of Saxon Math. Each state
has an educational representative. The curriculum developers have
a comprehensive Web site offering general information, resource
materials, and an email address for questions. Other supports
include customer service representatives, in-service training,
telephone teacher support, a helpline, teachers’ resource booklets,
in-service videos, and administrator’s guides to help principals and
administrators implement Saxon Math in their classrooms.
Scope of use
Saxon Math’s first textbook (Algebra I for 9th grade) was
implemented in 1980 and Saxon Algebra 1/2 (8th grade) was
implemented in 1986. Information about the number and
demographics of students, schools, or districts currently using
the intervention is not available from the Web site or the
customer service or educational representatives.
Cost
The student text costs approximately $50. Additional materials,
including the teacher’s manual, can total approximately
$380-$900 more.
WWC Intervention Reports
2
Study findings
Strength of the
evidence base
References
WWC Intervention Reports
Randomized controlled trial
The single randomized controlled trial on Saxon Math (Peters
1992) found no significant difference in posttest scores between
students in Saxon Math and the comparison group when
controlling on pretest. The intervention group scored slightly but
not significantly higher than the comparison group. There was no
evidence that the Saxon Math intervention was more or less
effective than the comparison curriculum, the University of
Chicago Mathematics Project NCTM curriculum.
Quasi-experimental design
The single quasi-experimental design study on Saxon Math
(Crawford & Raia 1986) found that students in the intervention
group made significant gains in overall math and math
computation scores but not on math concepts scores, compared
with the comparison group. Because of the limitations in the way
the analysis was conducted, it is not possible to determine
whether these findings are due to the curriculum or to chance.
The WWC collected more than 800 studies for the Middle School
Math Curriculum review. Six looked at the effects of Saxon Math.
One study, a small randomized controlled trial without serious
problems, met WWC evidence standards. A second study, a
small quasi-experimental design study without serious problems,
met WWC evidence standards with reservations. The remaining
four studies did not meet WWC evidence screens. In three of
these studies, there was only one intervention and one compari-
son unit, so the analysis could not separate the effects of the
intervention from other factors. The fourth study, a quasi-
experimental design study, does not account for pre-existing
differences between groups with matching or equating.
Studies were rated according to the strength of their causal
evidence. Studies that placed students into the intervention
and comparison groups randomly (randomized controlled trials)
without notable design or implementation flaws are classified
as meeting evidence standards (@ a ). Other studies that use
comparison groups (quasi-experimental designs) and
randomized control trials with notable flaws are classified as
meeting evidence standards with reservations ( ✓ a ).
Studies are further rated for intervention fidelity, outcome
measures, breadth of evidence, reporting on subgroups,
analysis, and statistical reporting. That information is provided
in study reports, but does not affect the overall rating.
In both studies, the interventions were well designed and
implemented — and both studies used nationally normed,
standardized tests. Neither study looked at all important groups
of students or settings. There were several issues with the
analysis. The studies were small — the randomized controlled trial
had 36 students and the quasi-experimental design study had 78
students. Further, some students in the RCT switched groups,
and findings from the QED study should be viewed with caution
because of problems with the analysis.
Tables A3-A4 describe the outcome studies conducted on
Saxon Math that meet WWC evidence standards and meet WWC
evidence standards with reservations. For a more detailed
description of the study, see the Detailed Study Reports or Brief
Study Reports . a See symbol key on page 7.
© Peters, K. G. (1992). Skill performance comparability of two
algebra programs on an eighth-grade population. Unpublished
doctoral dissertation, University of Nebraska, Lincoln.
✓ Crawford, J., & Raia, F. (1986, February). Analyses of eighth
grade math texts and achievement (evaluation report). Oklahoma
City: Planning, Research, and Evaluation Department, Oklahoma
City Public Schools.
Q Clay, D.W. (1 998). A study to determine the effects of a
non-traditional approach to algebra instruction on student
achievement. Master’s thesis, Salem-Teikyo University. (ERIC
Document Reproduction Service No. ED428963)
© Lafferty, J.F. (1996). The links among mathematics text,
students’ achievement, and students’ mathematics anxiety: A
comparison of the incremental development and traditional
References
(continued)
texts. Dissertation Abstracts International, 56 (08), 301 4A. (UMI
No. 9537085)
O Rentschler, R.V. (1994). The effects of Saxon’s incremental
review on computational skills and problem-solving achievement
of sixth-grade students. Dissertation Abstracts International, 56
(02), 484A. (UMI No. 9518017)
G Saxon, J. (1982). Incremental development: A breakthrough
in mathematics. Phi Delta Kappan, 63 (4), 482-484.
Table 1 The Saxon Math group seemed to score better than the difference between the two groups was somewhere between
Effects comparison group, but the difference is not significant. For the -0.54 (favoring the comparison group) and 0.76 (favoring the
randomized controlled trial, we are 95% confident that the Saxon Math group).
Study Measure
Favors Comparison Favors
comparison group group mean Saxon Math group
-2.00 -1.50 -1.00 -0.50 0 0.50 1.00 1.50 2.00
© Peters Orleans-Hanna Algebra
1992 Prognosis Test 3
(N=36 students)
0.11
/ Crawford CAT Overall Math b
& Raia (N=78 students 0 )
1986
0 . 41 d
CAT Math Concepts 13
(N=78 students 0 )
0 . 28 d
CAT Math
Computation 13
(N=78 students 0 )
0 . 44 d
Approximate percentile ranking 2
% 7°
h 16% 31% 50% 69% 84% 93% 98%
a A nationally normed, standardized test,
b California Achievement Test, a nationally normed, standardized test,
c Sample size reported is unit of analysis, not unit of assignment.
d When there is no solid line, the study did not provide data to correctly compute the confidence interval.
How to read this table: The wide, shaded bar indicates both the direction and estimated size of the effect of the intervention. The estimated effects reported here are
standardized differences in the mean values between the intervention and comparison groups. Bars extending to the right of zero denote estimated effects that favor the
intervention group and those extending to the left of zero denote estimated effects that favor the comparison group. The solid line through the shaded bar marks the
95% confidence interval of the estimated effect. When the line does not cross zero (and the bar is solid, not striped), the estimate is statistically significant. The bar is
striped if the effect is not significant or if significance could not be accurately computed. The scale at the bottom of the chart indicates the approximate percentile
distribution of students in the control group. The percentile ranking at the end of the shaded bar can be used to interpret the standardized mean difference in the
outcome. For example, an effect of .5 is roughly equivalent to an increase in the mean value from that of the average student in the comparison group (50th percentile)
to that of the average student at the 69th percentile.
What Works Clearinghouse
The What Works Clearinghouse (www.whatworks.ed.gov) was established in 2002 by the U.S. Department of Education’s Institute of Education
Sciences to provide educators, policymakers, researchers, and the public with a central and trusted source of scientific evidence of what works in
education. Please email all questions and comments to info@whatworks.ed.gov. The What Works Clearinghouse is administered by the U.S.
Department of Education through a contract to a joint venture of the American Institutes for Research and the Campbell Collaboration.
Appendix
Table ai Summary characteristics and findings from randomized controlled trials on Saxon Math
Study
Study
sample
Measure
Sample size
Mean outcome
Standard deviation 3
Estimated impact b
Intervention Comparison
group group
Total
Intervention
group
Comparison
group
Intervention
group
Comparison
group
Mean
difference
Standardized
mean difference
Peters
8th grade
[General]
19
17
36
95.6
95.1
4.53
4.09
0.5
0.11 (±0.65)
1992
“math-talented”
achievement
students
students
test c
Table A2 Summary characteristics and findings from quasi-experimental design studies on Saxon Math
Sample size
Mean outcome
Standard deviation 3
Estimated impact b
Study
Study
sample
Measure
Intervention Comparison Intervention
group group Total group
Comparison
group
Intervention
group
Comparison
group
Mean
difference
Standardized
mean difference
Crawford
& Raia
8th grade
students
CAT d Overall
Math
39
39
78 55.56
students 6
50.72
11.86
11.75
4.84
0.41 f
1986
8th grade
students
CAT d Math
Concepts
39
39
78 53.36
students 6
49.82
12.44
12.40
3.54
0.20 f
8th grade
students
CAT d Math
Computation
39
39
78 57.59
students 6
51.51
13.35
14.14
6.08
0.44 f
a Shows how dispersed the participants’ outcomes are. A small standard deviation would suggest that participants had similar outcomes,
b The WWC computed standardized effects, using statistics reported by the study author,
c A 60-item nationally normed, standardized test to predict student success in future algebra study,
d California Achievement Test.
e The sample size reported is unit of analysis, not unit of assignment.
f The unit of analysis did not match the unit of assignment, so accurate confidence intervals could not be computed.
WWC Intervention Reports
5
Table A3 Characteristics of interventions in reviewed studies on Saxon Math: Peters 1992
Evidence
base
rating 3
Characteristic
Description
0
Study citation
Peters, K. G. (1 992). Skill performance comparability of two algebra programs on an eighth-grade population.
Unpublished doctoral dissertation, University of Nebraska, Lincoln, NE.
Participants
36 8th-grade students. All the students were “math-talented” based on teacher recommendations, prior academic
achievement, and personal maturity.
Setting
Junior high school in a rural suburban district abutting Lincoln, Nebraska; students randomly assigned to one of two
classrooms (one intervention classroom and one comparison classroom). The same teacher taught both the
intervention and comparison groups.
Intervention
Participants in the intervention group were taught using the Saxon Math curriculum for 8th grade students (Algebra
1/2). Students in this group participated in 60-minute daily sessions for one year. In each session, the teacher
introduced a new concept incrementally, and students had opportunities to practice the new concept and past
concepts during each session. Students were assessed every fifth lesson. The intervention is designed to cover 1 20
lessons in one year.
Comparison
Participants in the comparison group were taught using an NCTM standards based curriculum called the University of
Chicago Mathematics Project designed to: build independent learners and thinkers, build understanding of math
vocabulary (such as mathematical signs), emphasize reviewing concepts within existing lessons, and increase
student comprehension.
Primary outcomes and
measurement
The primary outcome measure is the Orleans-Hanna Algebra Prognosis Test, a nationally normed, valid, and reliable
60-item test designed to predict student success in future algebra study.
Teacher training
Teacher training was not reported for this study, but teacher resources are available at the Saxon website, including
telephone and email access to customer service and educational representatives (in each state).
a See symbol key on page 7.
WWC Intervention Reports
6
Table A4 Characteristics of interventions in reviewed studies on Saxon Math: Crawford & Raia 1986
Evidence
base
rating
Characteristic
Description
•
Study citation
Crawford, J., & Raia, F. (1986, February). Analyses of eighth grade math texts and achievement (evaluation report).
Oklahoma City: Planning, Research, and Evaluation Department, Oklahoma City Public Schools.
Participants
78 8th grade students matched on pretest California Achievement Test (CAT) scores.
Setting
Four middle schools in the Oklahoma City Public Schools; four teachers taught both the intervention and the
comparison groups.
Intervention
Participants in the intervention group were taught using the Saxon Math curriculum for 8th-grade students (Algebra
1/2). Specific information about the level of implementation was not provided. The intervention is designed to cover
120 lessons across a one-year period with students participating in daily lessons, approximately 60 minutes a
lesson. Students participated during the 1984/85 academic year.
Comparison
Participants in the comparison group were taught using the Scott-Foresman Mathematics curriculum. Information
about this curriculum, including implementation, was not provided.
Primary outcomes and
measurement
The primary outcome measure is the California Achievement Test (CAT), including overall scores and scores for math
concepts and math computation. The CAT is a nationally normed, valid, and reliable test designed to measure
achievement in the basic skills taught in school.
Teacher training
Teacher training was not reported for this study, but teacher resources are available at the Saxon website, including
telephone and email access to customer service and educational representatives (in each state).
Symbol key
for evidence
base rating
0 Study meets evidence standards (randomized controlled trial without notable flaws).
V Study meets evidence standards with reservations (randomized controlled trial with notable flaws or quasi-experimental design
study without notable flaws).
©
Study does not meet evidence screens.
WWC Intervention Reports
