Measurement and Statistics for Teachers Written in a student-friendly style, this modestly priced text shows teachers how to use measurement and statistics wisely in their classes. Although there is some discussion of theory, emphasis is given to the practical, everyday uses of measurement and statistics such as how to develop and use effective classroom tests, how to carry out informal assessments, performance assessments, portfolio assessments, and how to use and inter- pret standardized tests. Part II provides a more complete coverage of basic descriptive statistics and their use in the classroom than any text now available. Malcolm Van Blerkom has been teaching various college courses in Psychology and Educational Psychology for the past 28 years, and is currently at the University of Pittsburgh at Johnstown.
He has also published research on topics as varied as cognitive styles, class attendance, and study strategies. Measurement and Statistics for Teachers Malcolm L. Van Blerkom University of Pittsburgh at Johnstown First published 2009 by Routledge 270 Madison Ave, New York, NY 10016 Simultaneously published in the UK by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN This edition published in the Taylor & Francis e-Library, 2008. “To purchase your own copy of this or any of Taylor & Francis or Routledge’s collection of thousands of eBooks please go to www.” Routledge is an imprint of the Taylor & Francis Group, an informa business © 2009 Taylor & Francis All rights reserved.
No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging in Publication Data Van Blerkom, Malcolm L. Measurement and statistics for teachers / Malcolm L.
Includes bibliographical references and index. Educational tests and measurements.26—dc22 2008020571 ISBN 0-203-88786-7 Master e-book ISBN ISBN10: 0–415–99565–5 (hbk) ISBN10: 0–805–86457–1 (pbk) ISBN10: 0–203–88786–7 (ebk) ISBN13: 978–0–415–99565–8 (hbk) ISBN13: 978–0–805–86457–1 (pbk) ISBN13: 978–0–203–88786–8 (ebk) Dedication For my wife, Diane, for all of her love, help, and encouragement v CONTENTS Preface xvii Acknowledgements xxi PART I MEASUREMENT 1 Section I Basic Issues in Measurement 3 Chapter 1 Introduction to Measurement 5 Introduction 5 The Role of Measurement 5 Measurement, Assessment, and Evaluation 6 Assessment 6 Measurement 6 Evaluation 7 Formal vs. Informal Assessment 8 Classroom Assessment 8 Preliminary or Placement Assessment 8 Diagnostic Assessment 9 Formative Assessment 9 Summative Assessment 10 Maximum vs. Typical Performance Measures 11 Uses of Measurement 11 Summary 12 Exercises 13 Spotlight on the Classroom 13 Study Tips: Setting Effective Academic Goals 14 vii viii • Contents Chapter 2 Frames of Reference: Interpreting Test Scores 16 Introduction 16 Four Frames of Reference 16 Ability-Referenced Interpretations 16 Growth-Referenced Interpretations 17 Norm-Referenced Interpretations 19 Criterion-Referenced Interpretations 20 A Comparison of Norm-Referenced and Criterion-Referenced Interpretations 21 Choosing a Frame of Reference for Assessment 22 Characteristics of Norm-Referenced and Criterion-Referenced Tests 23 Item Difficulty 24 Number of Items on the Test 24 Summary 25 Exercises 25 Spotlight on the Classroom 26 Study Tips: Time Management 26 Chapter 3 Developing Objectives 28 Introduction 28 Standards 28 Planning by Using Goals and Objectives 29 Goals vs.
Objectives 29 Benjamin Bloom’s Taxonomy of Objectives 31 Knowledge Level Objectives 32 Comprehension Level Objectives 32 Application Level Objectives 32 Analysis Level Objectives 32 Synthesis Level Objectives 32 Evaluation Level Objectives 33 Robert Mager’s Instructional Objectives 33 Norman Gronlund’s Instructional Objectives 34 Robert Gagné’s Learning Outcomes 35 Gagné’s Categories 35 Using Objectives 36 Summary 37 Exercises 38 Spotlight on the Classroom 39 Chapter 4 Reliability 40 Introduction 40 Contents • ix What is Reliability? 40 Theoretical Model of Reliability 41 Computing Reliability 45 Reliability and Validity 45 Estimating Reliability 46 Test–Retest Reliability 47 Alternate Form Reliability 48 Internal Consistency Reliability 48 Interpreting Reliabilities 51 Improving Test Reliability 52 Reducing Subject Effects 52 Reducing Test Effects 53 Reducing Environmental Effects 54 Some Final Comments about Reliability 55 Summary 55 Exercises 55 Spotlight on the Classroom 56 Chapter 5 Validity 57 Introduction 57 Perspectives on Validity 57 Content-Related Evidence of Validity 58 Criterion-Related Evidence of Validity 61 Construct-Related Evidence of Validity 62 Which Perspective is the Most Important? 63 Reliability and Validity 64 Summary 65 Exercises 65 Spotlight on the Classroom 66 Section II Classroom Testing 67 Chapter 6 Completion and Short-Answer Items 69 Introduction 69 Short-Answer Items 69 Advantages and Limitations of Short-Answer Items 71 Advantages 71 Limitations 71 Attributes Desired in Short-Answer Items 72 Evaluating Short-Answer Items 76 Summary 77 Exercises 77 Spotlight on the Classroom 77 x • Contents Chapter 7 Essay Items 79 Introduction 79 Advantages and Limitations of the Essay Format 79 Advantages 79 Limitations 80 Types of Essay Item 81 Scoring Essay Items 82 Holistic Scoring 82 Analytic Scoring 83 General Recommendations for Scoring Essay Answers 83 Attributes Desired in Essay Items 84 Evaluating Essay Items 86 Summary 87 Exercises 87 Spotlight on the Classroom 88 Chapter 8 Multiple-Choice Items 89 Introduction 89 Advantages and Limitations of Multiple-Choice Items 89 Advantages 89 Limitations 91 Attributes Desired in Multiple-Choice Items 92 Evaluating Multiple-Choice Items 98 Various Types of Multiple-Choice Item 100 Matching Items 100 Range-of-Value Items 100 Ranking Options 101 Interpretive Exercises 101 Number of Alternatives 102 Summary 103 Exercises 103 Spotlight on the Classroom 103 Chapter 9 True–false Items (and Variations) 105 Introduction 105 Advantages and Limitations of True–false Items 105 Advantages 105 Limitations 106 Attributes of Good True–false Items 107 Evaluating True–false Items 111 Variations in the True–false Format 112 True–false with Correction 112 Contents • xi Embedded True–false Items 112 Sequential True–false Items 113 Checklists 114 Summary 114 Exercises 114 Spotlight on the Classroom 115 Chapter 10 Producing and Administering Tests 116 Introduction 116 Designing a Test 116 Defining the Purpose of the Test 116 Choosing the Types of Items to Use 117 Choosing the Number of Items to be Used 117 Choosing the Difficulty Level of the Items 117 Assuring Sufficient Accuracy 118 Producing a Test 120 Preparing the Items 120 Ordering the Items 120 Formatting the Test 120 Preparing Instructions 121 Proofreading 121 Administering the Test 122 Setting Up an Appropriate Testing Environment 122 Summary 122 Exercises 123 Spotlight on the Classroom 124 Chapter 11 Analyzing Tests 125 Introduction 125 Test Analysis 125 Item Analysis 126 Item Difficulty 127 Item Discrimination 127 Distractor Analysis 129 Item Analysis Practice 130 The Stability of Item Analyses 132 Summary 133 Exercises 133 Spotlight on the Classroom 134 xii • Contents Section III Alternative Assessment Techniques 135 Chapter 12 Informal Assessment 137 Introduction 137 What is Informal Assessment? 137 Types of Informal Assessment 138 Informal Observations 138 Questions 138 Characteristics of Informal Assessment 139 Planning for Observations and Questions 142 Choosing Behaviors to Observe 142 The Validity Question 142 The Reliability Question 143 Techniques for Effective Informal Assessment 143 Planning Informal Assessment 143 Use Informal Assessment Frequently 144 Maintain Positive Interactions with your Students 144 Use the Results of Informal Assessment to Alter Instruction 144 Summary 145 Exercises 145 Spotlight on the Classroom 146 Chapter 13 Performance Assessments 147 Introduction 147 What are Performance Assessments? 147 Types of Performance Assessment 148 Process vs. Product 148 Simulated vs. Real Settings 148 Natural vs. Structured Settings 149 When Are Performance Assessments Appropriate To Use? 149 Advantages and Limitations of Performance Assessment 150 Advantages 150 Limitations 150 Planning and Developing Performance Assessments 151 Tie Assessment to Objectives 151 Measure Important Skills 151 Establish Precise Skills to Measure 152 Focus on Process or Product Only 152 Define the Tasks for the Students 152 Scoring Performance Assessments 152 Checklists 152 Contents • xiii Rating Scales 153 Rubrics 154 Summary 156 Exercises 157 Spotlight on the Classroom 157 Chapter 14 Portfolios 159 Introduction 159 What Makes Portfolios Distinctive? 159 Advantages and Limitations 160 Advantages 160 Limitations 160 Components of Portfolios 161 The List of Goals 161 Work Samples 162 Annotations 162 When is Portfolio Assessment the Most Effective? 162 Helping Students Develop Their Portfolios 163 Scoring Portfolios 163 The Future of Portfolio Assessment 164 Summary 164 Exercises 165 Spotlight on the Classroom 165 Section IV Additional Measurement Issues 167 Chapter 15 Teaching Students Test-Taking Skills 169 Introduction 169 General Test-Taking Strategies 170 Budgeting Time 170 Reading Directions 170 Reading Items Carefully 171 Checking Tests before Turning Them In 172 Test-Taking Strategies for Specific Test Formats 172 Strategies for Short-Answer Tests 172 Strategies for Essay Tests 173 Strategies for Multiple-Choice Tests 173 Strategies for True–false Tests 174 Familiarity with Testing Approaches 175 Approaches to Teaching Test-Taking Skills 175 Summary 176 Exercises 176 Spotlight on the Classroom 176 xiv • Contents Chapter 16 Standardized Tests 178 Introduction 178 General Characteristics of Standardized Tests 178 A Case Study in Developing a Standardized Test 179 Steps in Building a Standardized Test 180 Setting Interpretation Standards 180 Standardized Test Administration 181 Achievement Tests 181 Single-Subject-Area Achievement Tests 181 Survey Batteries 182 Diagnostic Tests 183 Reading Readiness Tests 184 Aptitude Tests 185 Individual Aptitude Tests 186 Group Aptitude Tests 187 Other Types of Standardized Test 188 Using Standardized Tests Effectively in the Schools 188 Selecting Standardized Tests 188 Making Predictions Based on Test Scores 189 Using Standardized Tests Appropriately 189 The Effects of No Child Left Behind and Other Federal Mandates 190 Summary 191 Exercises 191 Spotlight on the Classroom 192 Chapter 17 Alternative Ways to Report Test Scores 193 Introduction 193 Percentile Ranks 193 Standardized Scores 195 z-Scores 196 T-Scores 197 SAT Scores 198 Normalized Standard Scores 199 Normal Curve Equivalent Scores 200 Stanines 201 Grade Equivalent Scores 201 Building Confidence Intervals 202 Error Variance 202 Standard Error of Measurement 203 Using the SEM to Build Confidence Intervals 205 Factors Affecting the Width of Confidence Intervals 205 Contents • xv Summary 206 Exercises 207 Spotlight on the Classroom 208 PART II DESCRIPTIVE STATISTICS 209 Chapter 18 The Language and Logic of Statistics 211 Introduction 211 Basic Language and Logic 211 Constants and Variables 211 Populations and Samples 212 Parameters and Statistics 214 Measurement Scales 216 Categorical Data 216 Ranked Data 217 Numerical Data 218 Discrete Data vs.
Continuous Data 219 Summary 219 Exercises 220 Spotlight on the Classroom 221 Study Tips: How to Read a Measurement and Statistics Text 221 Chapter 19 Frequency Distributions and Graphs 223 Introduction 223 Frequency Distributions 223 Frequencies 223 Proportion and Percentages 225 Grouped Frequency Distributions 227 Graphing Frequency Distributions 229 Bar Charts 229 Pie Charts 230 Histograms 230 Frequency Polygons 231 Forms of Frequency Distribution 232 Cumulative Frequency Distributions 234 Summary 235 Exercises 235 Spotlight on the Classroom 237 Study Tips: How to Take Lecture Notes in a Measurement and Statistics Class 237 Chapter 20 Central Tendency: What is a Typical Score? 239 Introduction 239 xvi • Contents Measures of Central Tendency 239 Mode 239 Median 240 Mean 245 Deviation Scores 247 Characteristics of Central Tendency Measures 248 Stability of Central Tendency Measures 248 Uses of Central Tendency Measures 249 Central Tendency and Form 250 Summary 250 Exercises 250 Spotlight on the Classroom 251 Study Tips: Learning Technical Terminology 252 Chapter 21 Variability: How Spread Out Are the Scores? 254 Introduction 254 The Variability Question 254 Ranges 255 Variance and Standard Deviation 256 Summary 262 Exercises 262 Spotlight on the Classroom 264 Study Tips: How to Prepare for an Exam 264 Chapter 22 Correlation 266 Introduction 266 Bivariate Statistics 266 z-Scores 268 Pearson Product-Moment Correlation Coefficient 270 Computational Formula for the PPMC 274 Correlation and Prediction 275 Summary 276 Exercises 277 Spotlight on the Classroom 278 Study Tips: How to Learn from an Exam 278 References 281 Index 283 PREFACE This text is designed to give prospective and practicing teachers those skills required that will allow them to make intelligent decisions about testing and grading.