Skip to main content

Full text of "ERIC ED485398: Saxon Math. What Works Clearinghouse Intervention Report"

See other formats


What Works Clearinghouse 

Curriculum-based interventions for increasing K-12 math achievement — middle school 



Institute of Education Sciences 



Intervention report Saxon Math updated December i , 2004 



Intervention Saxon Math curricula and materials are available for grades K through 12, with the content and skills designed to meet National 

Council of Teachers of Mathematics (NCTM) and various state standards. Each course lasts for one year, and students participate in 
120-lesson courseware packages that last for about 60 minutes a day. Used in teacher-led lessons, Saxon Math is designed to teach 
in increments, provide continual practice, and test cumulative learning every five lessons. 



For Middle school students. 



Findings One randomized controlled trial on Saxon Math found no significant difference in posttest scores between students using Saxon Math 
and the comparison group using the University of Chicago Mathematics Project NCTM curriculum. One quasi-experimental design 
study found that students using Saxon Math had higher gains in overall math, math computation, and math concepts compared with 
other students. However, the study analyzed the data at the wrong level, making it impossible to accurately determine the significance 
of the finding. 

Evidence base 1 randomized controlled trial meets evidence standards. 

y 1 quasi-experimental design study meets evidence standards with reservations. 

G 4 studies do not meet evidence screens. ( see S y m £> 0 / key on page 7) 



Evidence limits The evidence base is limited to two studies. The first is a randomized controlled trial of 8th-grade students in a rural-suburban 

Nebraska junior high school. A second study is a quasi-experimental design study of 8th-grade students in Oklahoma City middle 
schools. Quasi-experimental studies provide weaker evidence of effects because it is possible that unmeasured differences between 
the groups affected the findings. Further, this study analyzed the data at the wrong level, which may bias the findings. The samples for 
both studies were small (36-78 students). Four studies do not meet evidence standards. 



Scope of use Saxon Math’s first textbook (Algebra I for 9th grade) was implemented in 1980, and Saxon Algebra 1/2 (8th grade) was implemented 
in 1986. Information is not available about the number and demographics of students, schools, or districts using the intervention. 



Developer and contact Saxon Publishers, www.saxonpublishers.com ; email: info@saxonDublishers.com : telephone: (800) 284-7019. 



WWC Intervention Reports 





Profile Saxon Math focuses on fundamental mathematics skills, targeting 
children from kindergarten through grade 12. This report focuses 
on middle school math, defined as grades 6 through 9. The 6th 
grade curriculum covers simplifying expressions containing 
parentheses, graphing functions, and understanding ratios and 
proportions. The 7th grade curriculum covers pre-algebra topics 
such as rate, powers, roots, and geometric proofs. The 8th grade 
curriculum covers all topics usually taught in pre-algebra in 
addition to topics from geometry and discrete mathematics. The 
9th grade curriculum covers all topics usually taught in a first-year 
algebra course (such as exponents, roots, and algebraic word 
problems) as well as conceptual understanding, procedural 
fluency, strategic competence, adaptive reasoning, and productive 
disposition. As stated by the developers, Saxon Math covers all 
five content and skill areas of NCTM standards and meets various 
state standards. 

Teaching 

Each grade of Saxon Math consists of 1 20 daily lessons and 1 2 
activity-based investigations. A daily lesson consists of warm-up 
(10-15 minutes), introduction to the new concept (5-10 minutes), 
practice focusing on new concept (5 minutes), and mixed practice 
focusing on new and previously learned concepts (20-30 minutes). 
Students are introduced to concepts incrementally, given 
opportunities for continual review and practice, and assessed 
cumulatively and frequently (every fifth lesson). An assessment 
score of 80% or lower indicates a need for remediation, and a 
provision for remediation is part of the program. 

Lessons are designed to be one hour daily (this includes 
practice and review time), and assessments occur every fifth 
lesson, usually on Friday each week. 



The teacher is responsible for facilitating and mediating the 
warm-up session, introducing the new concept, and conducting 
the practice sessions. Teachers introduce the daily concept 
using manipulatives or representative models accompanied by 
the procedures needed to solve the problem. Teachers are 
instructed to conduct lessons in sequence, not skip lessons, 
limit direct instruction to 10-15 minutes of group (or individual) 
instruction, spend the majority of class time allowing children to 
do mathematics problems in the problem sets, and assign all 
problems in each set. 

Supports are available for teachers of Saxon Math. Each state 
has an educational representative. The curriculum developers have 
a comprehensive Web site offering general information, resource 
materials, and an email address for questions. Other supports 
include customer service representatives, in-service training, 
telephone teacher support, a helpline, teachers’ resource booklets, 
in-service videos, and administrator’s guides to help principals and 
administrators implement Saxon Math in their classrooms. 

Scope of use 

Saxon Math’s first textbook (Algebra I for 9th grade) was 
implemented in 1980 and Saxon Algebra 1/2 (8th grade) was 
implemented in 1986. Information about the number and 
demographics of students, schools, or districts currently using 
the intervention is not available from the Web site or the 
customer service or educational representatives. 

Cost 

The student text costs approximately $50. Additional materials, 
including the teacher’s manual, can total approximately 
$380-$900 more. 



WWC Intervention Reports 



2 




Study findings 



Strength of the 
evidence base 



References 



WWC Intervention Reports 



Randomized controlled trial 

The single randomized controlled trial on Saxon Math (Peters 
1992) found no significant difference in posttest scores between 
students in Saxon Math and the comparison group when 
controlling on pretest. The intervention group scored slightly but 
not significantly higher than the comparison group. There was no 
evidence that the Saxon Math intervention was more or less 
effective than the comparison curriculum, the University of 
Chicago Mathematics Project NCTM curriculum. 



Quasi-experimental design 

The single quasi-experimental design study on Saxon Math 
(Crawford & Raia 1986) found that students in the intervention 
group made significant gains in overall math and math 
computation scores but not on math concepts scores, compared 
with the comparison group. Because of the limitations in the way 
the analysis was conducted, it is not possible to determine 
whether these findings are due to the curriculum or to chance. 



The WWC collected more than 800 studies for the Middle School 
Math Curriculum review. Six looked at the effects of Saxon Math. 
One study, a small randomized controlled trial without serious 
problems, met WWC evidence standards. A second study, a 
small quasi-experimental design study without serious problems, 
met WWC evidence standards with reservations. The remaining 
four studies did not meet WWC evidence screens. In three of 
these studies, there was only one intervention and one compari- 
son unit, so the analysis could not separate the effects of the 
intervention from other factors. The fourth study, a quasi- 
experimental design study, does not account for pre-existing 
differences between groups with matching or equating. 

Studies were rated according to the strength of their causal 
evidence. Studies that placed students into the intervention 
and comparison groups randomly (randomized controlled trials) 
without notable design or implementation flaws are classified 
as meeting evidence standards (@ a ). Other studies that use 
comparison groups (quasi-experimental designs) and 
randomized control trials with notable flaws are classified as 



meeting evidence standards with reservations ( ✓ a ). 

Studies are further rated for intervention fidelity, outcome 
measures, breadth of evidence, reporting on subgroups, 
analysis, and statistical reporting. That information is provided 
in study reports, but does not affect the overall rating. 

In both studies, the interventions were well designed and 
implemented — and both studies used nationally normed, 
standardized tests. Neither study looked at all important groups 
of students or settings. There were several issues with the 
analysis. The studies were small — the randomized controlled trial 
had 36 students and the quasi-experimental design study had 78 
students. Further, some students in the RCT switched groups, 
and findings from the QED study should be viewed with caution 
because of problems with the analysis. 

Tables A3-A4 describe the outcome studies conducted on 
Saxon Math that meet WWC evidence standards and meet WWC 
evidence standards with reservations. For a more detailed 
description of the study, see the Detailed Study Reports or Brief 
Study Reports . a See symbol key on page 7. 



© Peters, K. G. (1992). Skill performance comparability of two 
algebra programs on an eighth-grade population. Unpublished 
doctoral dissertation, University of Nebraska, Lincoln. 

✓ Crawford, J., & Raia, F. (1986, February). Analyses of eighth 
grade math texts and achievement (evaluation report). Oklahoma 
City: Planning, Research, and Evaluation Department, Oklahoma 
City Public Schools. 



Q Clay, D.W. (1 998). A study to determine the effects of a 
non-traditional approach to algebra instruction on student 
achievement. Master’s thesis, Salem-Teikyo University. (ERIC 
Document Reproduction Service No. ED428963) 

© Lafferty, J.F. (1996). The links among mathematics text, 
students’ achievement, and students’ mathematics anxiety: A 
comparison of the incremental development and traditional 




References 

(continued) 



texts. Dissertation Abstracts International, 56 (08), 301 4A. (UMI 
No. 9537085) 

O Rentschler, R.V. (1994). The effects of Saxon’s incremental 
review on computational skills and problem-solving achievement 



of sixth-grade students. Dissertation Abstracts International, 56 
(02), 484A. (UMI No. 9518017) 

G Saxon, J. (1982). Incremental development: A breakthrough 
in mathematics. Phi Delta Kappan, 63 (4), 482-484. 



Table 1 The Saxon Math group seemed to score better than the difference between the two groups was somewhere between 

Effects comparison group, but the difference is not significant. For the -0.54 (favoring the comparison group) and 0.76 (favoring the 

randomized controlled trial, we are 95% confident that the Saxon Math group). 



Study Measure 


Favors Comparison Favors 

comparison group group mean Saxon Math group 

-2.00 -1.50 -1.00 -0.50 0 0.50 1.00 1.50 2.00 


© Peters Orleans-Hanna Algebra 

1992 Prognosis Test 3 

(N=36 students) 












0.11 


















/ Crawford CAT Overall Math b 
& Raia (N=78 students 0 ) 

1986 












0 . 41 d 










CAT Math Concepts 13 
(N=78 students 0 ) 












0 . 28 d 










CAT Math 
Computation 13 
(N=78 students 0 ) 












0 . 44 d 










Approximate percentile ranking 2 


% 7° 


h 16% 31% 50% 69% 84% 93% 98% 



a A nationally normed, standardized test, 
b California Achievement Test, a nationally normed, standardized test, 
c Sample size reported is unit of analysis, not unit of assignment. 

d When there is no solid line, the study did not provide data to correctly compute the confidence interval. 

How to read this table: The wide, shaded bar indicates both the direction and estimated size of the effect of the intervention. The estimated effects reported here are 
standardized differences in the mean values between the intervention and comparison groups. Bars extending to the right of zero denote estimated effects that favor the 
intervention group and those extending to the left of zero denote estimated effects that favor the comparison group. The solid line through the shaded bar marks the 
95% confidence interval of the estimated effect. When the line does not cross zero (and the bar is solid, not striped), the estimate is statistically significant. The bar is 
striped if the effect is not significant or if significance could not be accurately computed. The scale at the bottom of the chart indicates the approximate percentile 
distribution of students in the control group. The percentile ranking at the end of the shaded bar can be used to interpret the standardized mean difference in the 
outcome. For example, an effect of .5 is roughly equivalent to an increase in the mean value from that of the average student in the comparison group (50th percentile) 
to that of the average student at the 69th percentile. 



What Works Clearinghouse 



The What Works Clearinghouse (www.whatworks.ed.gov) was established in 2002 by the U.S. Department of Education’s Institute of Education 
Sciences to provide educators, policymakers, researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info@whatworks.ed.gov. The What Works Clearinghouse is administered by the U.S. 



Department of Education through a contract to a joint venture of the American Institutes for Research and the Campbell Collaboration. 







Appendix 



Table ai Summary characteristics and findings from randomized controlled trials on Saxon Math 



Study 


Study 

sample 


Measure 




Sample size 




Mean outcome 


Standard deviation 3 


Estimated impact b 


Intervention Comparison 
group group 


Total 


Intervention 

group 


Comparison 

group 


Intervention 

group 


Comparison 

group 


Mean 

difference 


Standardized 
mean difference 


Peters 


8th grade 


[General] 


19 


17 


36 


95.6 


95.1 


4.53 


4.09 


0.5 


0.11 (±0.65) 


1992 


“math-talented” 


achievement 




students 
















students 


test c 





















Table A2 Summary characteristics and findings from quasi-experimental design studies on Saxon Math 











Sample size 


Mean outcome 


Standard deviation 3 


Estimated impact b 


Study 


Study 

sample 


Measure 


Intervention Comparison Intervention 

group group Total group 


Comparison 

group 


Intervention 

group 


Comparison 

group 


Mean 

difference 


Standardized 
mean difference 


Crawford 
& Raia 


8th grade 
students 


CAT d Overall 
Math 


39 


39 


78 55.56 

students 6 


50.72 


11.86 


11.75 


4.84 


0.41 f 


1986 


8th grade 
students 


CAT d Math 
Concepts 


39 


39 


78 53.36 

students 6 


49.82 


12.44 


12.40 


3.54 


0.20 f 




8th grade 
students 


CAT d Math 
Computation 


39 


39 


78 57.59 

students 6 


51.51 


13.35 


14.14 


6.08 


0.44 f 



a Shows how dispersed the participants’ outcomes are. A small standard deviation would suggest that participants had similar outcomes, 
b The WWC computed standardized effects, using statistics reported by the study author, 
c A 60-item nationally normed, standardized test to predict student success in future algebra study, 
d California Achievement Test. 

e The sample size reported is unit of analysis, not unit of assignment. 

f The unit of analysis did not match the unit of assignment, so accurate confidence intervals could not be computed. 



WWC Intervention Reports 



5 








Table A3 Characteristics of interventions in reviewed studies on Saxon Math: Peters 1992 



Evidence 

base 

rating 3 


Characteristic 


Description 


0 


Study citation 


Peters, K. G. (1 992). Skill performance comparability of two algebra programs on an eighth-grade population. 
Unpublished doctoral dissertation, University of Nebraska, Lincoln, NE. 




Participants 


36 8th-grade students. All the students were “math-talented” based on teacher recommendations, prior academic 
achievement, and personal maturity. 




Setting 


Junior high school in a rural suburban district abutting Lincoln, Nebraska; students randomly assigned to one of two 
classrooms (one intervention classroom and one comparison classroom). The same teacher taught both the 
intervention and comparison groups. 




Intervention 


Participants in the intervention group were taught using the Saxon Math curriculum for 8th grade students (Algebra 
1/2). Students in this group participated in 60-minute daily sessions for one year. In each session, the teacher 
introduced a new concept incrementally, and students had opportunities to practice the new concept and past 
concepts during each session. Students were assessed every fifth lesson. The intervention is designed to cover 1 20 
lessons in one year. 




Comparison 


Participants in the comparison group were taught using an NCTM standards based curriculum called the University of 
Chicago Mathematics Project designed to: build independent learners and thinkers, build understanding of math 
vocabulary (such as mathematical signs), emphasize reviewing concepts within existing lessons, and increase 
student comprehension. 




Primary outcomes and 
measurement 


The primary outcome measure is the Orleans-Hanna Algebra Prognosis Test, a nationally normed, valid, and reliable 
60-item test designed to predict student success in future algebra study. 




Teacher training 


Teacher training was not reported for this study, but teacher resources are available at the Saxon website, including 
telephone and email access to customer service and educational representatives (in each state). 



a See symbol key on page 7. 



WWC Intervention Reports 



6 





Table A4 Characteristics of interventions in reviewed studies on Saxon Math: Crawford & Raia 1986 



Evidence 

base 

rating 


Characteristic 


Description 


• 


Study citation 


Crawford, J., & Raia, F. (1986, February). Analyses of eighth grade math texts and achievement (evaluation report). 
Oklahoma City: Planning, Research, and Evaluation Department, Oklahoma City Public Schools. 




Participants 


78 8th grade students matched on pretest California Achievement Test (CAT) scores. 




Setting 


Four middle schools in the Oklahoma City Public Schools; four teachers taught both the intervention and the 
comparison groups. 




Intervention 


Participants in the intervention group were taught using the Saxon Math curriculum for 8th-grade students (Algebra 
1/2). Specific information about the level of implementation was not provided. The intervention is designed to cover 
120 lessons across a one-year period with students participating in daily lessons, approximately 60 minutes a 
lesson. Students participated during the 1984/85 academic year. 




Comparison 


Participants in the comparison group were taught using the Scott-Foresman Mathematics curriculum. Information 
about this curriculum, including implementation, was not provided. 




Primary outcomes and 
measurement 


The primary outcome measure is the California Achievement Test (CAT), including overall scores and scores for math 
concepts and math computation. The CAT is a nationally normed, valid, and reliable test designed to measure 
achievement in the basic skills taught in school. 




Teacher training 


Teacher training was not reported for this study, but teacher resources are available at the Saxon website, including 
telephone and email access to customer service and educational representatives (in each state). 



Symbol key 
for evidence 
base rating 



0 Study meets evidence standards (randomized controlled trial without notable flaws). 

V Study meets evidence standards with reservations (randomized controlled trial with notable flaws or quasi-experimental design 
study without notable flaws). 



© 



Study does not meet evidence screens. 



WWC Intervention Reports