Over the last few years, there has been evolution, although not a linear one, that has progressed from an emphasis on statistical significance to an emphasis on effect size to an emphasis on both of these concepts to what is believed to be a pragmatic emphasis on replicability. This paper presented two methods of estimating a study's replicability that researchers should consider reporting along with their statistically significant and effect size findings. One method of estimating the...
Topics: ERIC Archive, Effect Size, Research Methodology, Statistical Significance, Newman, Isadore, McNeil,...
This paper presents a discussion of various statistical concepts and techniques in light of two propositions. The first is that researchers need to select analytical techniques that prevent them from committing Type VI errors, which are inconsistencies between the research question and the statistical analysis. The second is that many statistical techniques are interrelated on a conceptual level. In addition, a list of resources is presented to assist researchers who want to pursue a more...
Topics: ERIC Archive, Multivariate Analysis, Research Design, Research Methodology, Statistical Analysis,...
Statistical significance and practical significance can be considered jointly through the use of non-nil null hypotheses that are based on values deemed to be practically significant. When examining differences between the means of two groups, researchers can use a randomization test or an independent t test. The issue addressed in this paper is whether Type I error rates produced by independent t tests of group means are impacted by the use of non-nil null hypotheses. The results of this Monte...
Topics: ERIC Archive, Groups, Hypothesis Testing, Monte Carlo Methods, Statistical Significance, Newman,...
Analyses were conducted to determine whether the two statistical tests of significance are identical. Such demonstrations regarding parallel correlational and analysis of variance procedures have brought further understanding to each of the domains in the past. When a researcher is investigating the relationship between two variables and thinks that the relationship may be different for two subpopulations under consideration, that question can be tested. Some texts present this test as the test...
Topics: ERIC Archive, Correlation, Interaction, Statistical Significance, Test Use, McNeil, Keith, Newman,...
This paper discusses a study that was conducted in order to add to the body of literature that investigates the manner in which feminist psychology is accepted among education graduate students. Graduate students (N=69) at a large public mid-western university were recruited and randomly assigned to one of four treatment groups. Participants were given the task of reading a paragraph that was critical of psychology's historical treatment of women. Each treatment group was told that the...
Topics: ERIC Archive, Authors, Feminism, Graduate Students, Higher Education, Perception, Psychology, Sex...
One may receive the most benefit from an evaluation of an educational program or the performance of a teacher if the evaluation process is approached from a Total Quality Management (TQM) point of view. Under the philosophy of TQM, the purpose of any evaluation process is to provide feedback for the continual improvement of the educational process that is being evaluated. In the process of obtaining this feedback, the evaluator must be cognizant of two concepts that are basic to the TQM...
Topics: ERIC Archive, Data Analysis, Educational Change, Feedback, Formative Evaluation, Longitudinal...
This paper reports the evaluation of a unique three-year program designed to support teachers new to a school district. The emphasis of the evaluation process was on the participation of the stakeholders in the evaluation process and the development of their abilities in evaluation. There is educational significance to this study on two levels. First, it is an example of using a highly participatory process for the formative evaluation of a teacher induction program. Second, the finding that...
Topics: ERIC Archive, Stakeholders, Beginning Teacher Induction, Formative Evaluation, Mentors, Mixed...
This paper provides examples of how one can use the research issue and the relationships between qualitative and quantitative research as a frame for instructing students and judging the quality of research. The emphasis is on validity estimates, also called legitimization techniques, with attention to the idea of a qualitative-quantitative research continuum. It is not the technique that makes something quantitative or qualitative, but rather the intention of its use. The question is posed...
Topics: ERIC Archive, Deduction, Hypothesis Testing, Induction, Qualitative Research, Research Methodology,...
The first part of this study replicates an earlier study (D. Waechter, I. Newman, and L. Rosenkoetter, 1998) that used a procedure for authentic assessment of students in a master's level research class, a class that is often feared and avoided by students, by means of a study on humor, since that content would be somewhat incompatible with anxiety. Students (n=88) completed a questionnaire on personal interpretation of humor, demographic information, and the Temperament Profile Survey....
Topics: ERIC Archive, Age Differences, Course Evaluation, Effect Size, Graduate Students, Graduate Study,...
In an earlier study, I. Newman and D. Waechter (1999) investigated graduate education students' perceptions and attitudes toward cheating in scientific research. A questionnaire was developed that was based on reactions and concerns related to cheating behavior seen in a public television video about faking scientific data. This study examined the factor structure of the developed Cheating Scale, which was administered to 167 graduate education students. Analysis identified a three-factor...
Topics: ERIC Archive, Cheating, Factor Analysis, Factor Structure, Graduate Students, Graduate Study,...
Educational researchers often use multiple statistical tests in their research studies and program evaluations. When multiple statistical tests are conducted, the chance that Type I errors may be committed increases. Thus, the researchers are faced with the task of adjusting the alpha levels for their individual statistical tests in order to keep the overall alpha value at a reasonable level. A three-step procedure is presented that can be used to adjust the alpha levels of the individual...
Topics: ERIC Archive, Decision Making, Educational Research, Error of Measurement, Program Evaluation,...
When investigating the impact of predictor variables on an outcome variable or measuring the effectiveness of an educational program, educational researchers and program evaluators cannot ignore the possible influences of interaction effects. The purpose of this paper is to present a procedure that educational researchers can follow in order to increase their understanding of the nature of the interaction effect between a treatment variable and a continuous independent variable. This technique...
Topics: ERIC Archive, Computer Software, Evaluation Methods, Interaction, Outcomes of Education, Predictor...
Teacher efficacy has been identified as a variable that can influence teacher effectiveness. The results of methods designed to change teacher efficacy, however, have been mixed. The purpose of this study was to evaluate the possible impact on teacher efficacy of the FOCUS (1992) instructional method, which is designed to create an environment in teacher-education classes that is receiver-oriented, using activities and strategies designed to match students' learning styles. Participants were 68...
Topics: ERIC Archive, Classroom Environment, Elementary School Teachers, Elementary Secondary Education,...
The purpose of this paper is to assist researchers, practitioners, and graduate students in identifying and addressing key questions related to the task of choosing among the analytic techniques designed to analyze a dichotomized dependent variable with a set of independent variables. The discussion is limited to (1) the analysis of data by the analytic procedures of ordinary least squares regression, discriminant analysis, or logistic regression; (2) the use of the Statistical Package for the...
Topics: ERIC Archive, Discriminant Analysis, Least Squares Statistics, Regression (Statistics), Research...
Counseling and educational professionals are encouraged to understand the research behind the theory and the development of a measure of personality in addition to the application of the measure. Personality assessment, especially in relation to the Enneagram classification system of personality, has been dramatically changing. Personality measures can be based in theory as well as utilize numerous developmental methods. Additionally, the scores from personality measures are validated in a...
Topics: ERIC Archive, Classification, Counseling Theories, Evaluation Methods, Personality Measures, Theory...
Multiple linear regression is used to model the effects of violating statistical assumptions on the likelihood of making a Type I error. This procedure is illustrated for the student's t-test (for independent groups) using data from previous Monte Carlo studies in which the actual alpha levels associated with violations of the normality assumption, homogeneity of variance, or unbalanced designs were determined. The observed Type I error rates were recorded, along with information coding the...
Topics: ERIC Archive, Estimation (Mathematics), Monte Carlo Methods, Multiple Regression Analysis,...
Based on the assumption that inferential statistics can make the operant conditioner more sensitive to possible significant relationships, regressions models were developed to test the statistical significance between slopes and Y intercepts of the experimental and control group subjects. These results were then compared to the traditional operant conditioning eyeball technique analysis. (Author)
Topics: ERIC Archive, Attendance Patterns, Behavior Change, Behavior Modification, Behavior Theories,...
This paper presents a testing procedure that incorporates three key elements. The first element is the use of non-nil null hypotheses. The second element is the determination of a practically significant level that is incorporated into the corresponding non-nil null hypothesis. The third element is the use of a randomization test to statistically test each non-nil null hypothesis. This procedure stresses two philosophical positions. First, the concepts of practical and statistical significance...
Topics: ERIC Archive, Cost Effectiveness, Statistical Significance, Test Construction, Fraas, John W.,...
The Michigan Alcoholism Screening Test (MAST) was designed to serve as a reliable and quantifiable measure of alcohol dependence (M. Selzer, 1971). Since its introduction, the psychometric properties of the MAST have been studied extensively, but there are several questions that have not been addressed or only partially answered. The purpose of this study was to evaluate critically the MAST's psychometric usefulness. Subjects were 94 continuous clients presenting for a chemical dependence...
Topics: ERIC Archive, Adults, Alcoholism, Patients, Psychometrics, Screening Tests, Test Use, Laux, John...
The block scheduling program used in a high school in a small Midwestern city was evaluated, considering the "hard" data of effects on grade point average (GPA) and attendance, but not information about student attitudes and perceptions. Data were available for approximately 500 students from the classes of 1997 (before the block scheduling), 2000, 2001, and 2001. The relationship between block scheduling and cumulative GPA was not significant, but there was a significant positive...
Topics: ERIC Archive, Attendance, Block Scheduling, Grade Point Average, High School Students, High...
Drawing on the work of Pruyser (1976) and Malony (1985, 1987), the Religious Status Interview (RSI), a measure of theological functioning, was created. The RSI was designed to help counselors judge the health of a parishioner's religious beliefs as they relate to the problem that brings the person to pastoral counseling. This study investigated the validity of the RSI using a sample of 45 Christians diagnosed with an addiction to food, drugs, or alcohol who were in a Twelve Step program....
Topics: ERIC Archive, Adults, Attitude Measures, Beliefs, Christianity, Counseling, Depression...
This study examined preservice teachers' views at the beginning and end of a Professional Development School (PDS) collaboration project in terms of their stages of concern about teaching, teacher efficacy, understanding and implementation of integrated curriculum, and implementation of technology in instruction. The study also compared traditionally prepared and PDS-prepared student teachers' levels of preparedness for the entry year. The project involved immersing 15 preservice teachers in...
Topics: ERIC Archive, College School Cooperation, Elementary Education, Higher Education, Preservice...
The authors describe five value-added methods (VAM) used in school assessment as the backdrop to their main thesis. Then they review the assumptions underlying measurement and evaluation, the foundation of all assessment systems, including value-added. They discuss the traditional criterion variable used in VAM: a standardized test score. Next, they challenge the univariate assumptions of VAMs, and argue that a multivariate paradigm of VAM is more advantageous for educators and stakeholders....
Topics: ERIC Archive, Value Added Models, Educational Assessment, Standardized Tests, Scores, Multivariate...
This paper describes a five-session course entitled "Program Evaluation," which was taught via interactive television in the summer of 2002 to 68 doctoral and master's students in 5 of 6 locations throughout New Mexico. Students received a 4-hour lecture and then participated in off-line activities directed by the instructor. Problems in distance education delivery and some recommended solutions are provided. These include technical delivery problems related to the interactive...
Topics: ERIC Archive, Course Descriptions, Delivery Systems, Distance Education, Higher Education,...
J. Frass and I. Newman (2000) proposed a hypothesis testing procedure that incorporated the following three key elements: (1) the establishment of a practical significance value; (2) the construction of a non-nil null hypothesis that incorporated the practical significance value; and (3) statistical testing of the non-nil null hypothesis with a randomization test. One of the difficulties researchers may encounter with this testing procedure is the implementation of the randomization test. This...
Topics: ERIC Archive, Computer Software, Hypothesis Testing, Statistical Significance, Fraas, John W.,...
The purpose of this paper is to introduce a systematic teaching approach known as the General Teaching Model for instructing college students in reading. The model consists of identifying appropriate objectives for the student, pre-assessment prior to beginning instruction, instructional procedures designed to help the learner achieve the objectives, and evaluation. Examples of instructional objectives for reading and an example of individualizing reading using Computer Assisted Instruction as...
Topics: ERIC Archive, Behavioral Objectives, College Students, Computer Assisted Instruction, Evaluation,...
This paper reports on an evaluation of a unique 3-year program designed to support new teachers in a school district. The emphasis in the evaluation was on the participation of the multiple stakeholders in the evaluation process, which was endorsed by the teachers union, school district administrators, and entry-year teachers. The people who are the intended focus of the Teacher Evaluation Program (TEP) are referred to as interns. Interns are assisted by consulting teachers who are experienced...
Topics: ERIC Archive, Beginning Teacher Induction, Beginning Teachers, Case Studies, Elementary Secondary...
