This document has been formatted for printing from your browser from the Web site of the Illinois Association of School Boards.
COPYRIGHT NOTICE -- This document is © copyrighted by the Illinois Association of School Boards. IASB hereby grants to school districts and other Internet users the right to download, print and reproduce this document provided that (a) the Illinois Association of School Boards is noted as publisher and copyright holder of the document and (b) any reproductions of this document are disseminated without charge and not used for any commercial purpose.
Illinois School Board Journal
May/June 2005
Hey, batter, batter...
Test score statistics miss bigger picture
by Alan Simon
Alan Simon is superintendent of schools for Arlington Heights SD 25 in suburban Cook County.
With baseball season now underway, batting averages tell us something about a player's performance. Baseball fans talk about a "complete player," weighing batting averages with other factors like fielding ability, speed and stolen bases, on-base percentage, slugging percentage, games played, walks, strikeouts and game winning hits.
Other intangibles involve team leadership, clubhouse demeanor, weather conditions, player health and home-field advantage. Thus, a single statistic is not the sole measure of an individual or a team — attitude, effort and results matter.
Testing tells us something about a student's performance. In today's data-driven world, performance is measured primarily by standardized tests. Test results, like batting averages, give us a measure of how students are performing.
In schools, we have teachers and principals as coaches and managers. We have local and state achievement tests, IQ tests and a plethora of jargon. Yet, we also need to qualify the statistics.
In the January 2001 Phi Delta Kappan, Scott Thompson and others argue that standards-based assessments should include more than test scores. However, tests and standards based measures are not the only way to define achievement and success. In the same issue Eliot Eisner argues that the really important variables in education are located outside the school. "It's what students do with what they learn when they can do what they want to do."
A successful student and school can demonstrate many achievements, rarely measured, but often recognized in areas like forensics, community service, art, athletics, musical and dramatic performance, musical composition, poetry composition, creative writing, computer programming, auto mechanics or design.
Research confirming effective teaching and administration often has been compartmentalized and applied only to improvement in achievement scores on standardized tests. In making judgments about students, schools and teachers, researchers and pundits have neglected to analyze, compare or weigh the wider range of student products and the teachers who facilitate these accomplishments.
Almost every school and district seeks to create life-long learners who will be productive members of society. The ultimate measure of the success of a teacher or a school district is not in the achievement tests but in the achievement itself.
Many desired outcomes can be quantified and measured, others cannot. If a subgroup does not meet an arbitrary standard and thus a school is "failing", is that criterion exclusive? Labeling schools as failures based exclusively on test scores is bad public policy. Is a school failing if no one thinks it is?
The testing culture is based on a premise that students must know a fixed world. Squeezing all children into a common learning mold ignores individual differences, and operating as if individual differences should not exist is unfair, according to Rex and Trudy Knowles. A fixed set of outcomes ignores the richness of educational experiences.
In an Educational Leadership conversation article, UCLA icon John Goodlad notes that what we're measuring does not correlate with our stated aims of education. "We mouth them at commencement addresses and forget all about them."
Research also indicates that attitudes about school and specific outcomes, as well as test scores, can increase by using techniques such as teaching note taking, using advanced organizers, providing meaningful feedback, setting goals, cooperative learning, using graphic organizers, direct instruction, memorization, concept attainment, mastery learning, advanced questioning and listening techniques, reinforcing effort with rewards, differentiating instruction (compacting, tiered assignments, interest centers), inquiry training, role playing and inductive thinking.
Instructional designs like team teaching, multi-age classes, flexible grouping and independent studies attempt to organize instruction to meet the needs of the learner as well as the requirements of testing. Thousands of case studies demonstrate the effectiveness of these methods and the specific accomplishments of students. But few, if any, statistics indicate how many teachers have been sufficiently trained to use these techniques effectively, and how often these teaching methods are employed.
The system tells the teacher to prepare the student for the test. The teacher and the student's parents know that the test of the system is to prepare the student for the future. The success of the system and the teacher depend upon skill acquisition in tested and non-tested areas. The challenge for educators in this era of data overload is to find a balance between the requirements of systems (skill testing) and the requirements of a successful life-long learner (achievements).
Reporting progress
The student and school accountability system should be flexible, wide in scope and recognize differences in students, schools and their achievements. Student, teacher and school products documenting specific accomplishments are needed to achieve this balance and communicate with parents and the community. Reports should be directed toward specific and general aims of education.
Arlington Heights School District 25 has developed a "State of the District Report." This biannual document starts with the district's strategic plan and reviews all aspects of the school system for the board and the public. The report includes sections on curriculum and instruction, business services, personnel and special education. The bulk of the report is devoted to what is taught and the support systems for curriculum development and implementation.
The executive summary notes that even though the report is lengthy and comprehensive, it is a snapshot. Used with other information, it can be a barometer of what the district is doing and how the district is doing.
General information about curriculum, programs, student achievements, facilities, assessments and finances is a first step in the communication process. The second step is to present specific results. Every August, District 25 produces a video, aired on cable television, documenting accomplishments from the previous year. In 2005, a compact disc will be distributed highlighting district programs and specific accomplishments.
The district also is producing a video about its technology program. This cable television presentation will summarize curriculum, standards and district expectations. Individual school videos highlighting particular achievements will follow.
The technology video demonstrates that results can be reported in different ways. A department Web page about the art program including curriculum, teacher methods, student expectations and a display of student projects and achievements is another way to document achievements. Parent and student surveys, course offerings and alumni tracking also paint a picture of school culture and achievement.
Currently, some teachers and students develop portfolios yearly. This needs to be institutionalized so that all students and staff have the opportunity to "show what they know and do." Individual student and school portfolios should include test results (standardized and local); and artistic products, compositions, projects, community service and other achievements, such as sports participation, teacher awards, trainings and implementations. Every student and teacher must be counted and have documented accomplishments. Methods should be recorded and displayed.
Documenting results beyond testing is what will balance the aims of education with the requirements of the system. This is time consuming and difficult to coordinate; however, it is worth the effort to publish more than a "batting average" test score. When you look at a baseball player's offensive capabilities, you look at the batting average as a skill indicator. It is up to the manager, coaches and the team to develop a well-rounded player who can get on base, score and help the team. The team succeeds if everyone contributes in many ways.
References
Eliot Eisner, "What Does It Mean to Say A School is Doing Well?" Phi Delta Kappan, January 2001
Scott Thompson, "The Authentic Standards Movement and Its Evil Twin," Phi Delta Kappan, January 2001
Rex and Trudy Knowles, "Accountability for What?" Phi Delta Kappan, January 2001
"Renewing the Profession of Teaching: A Conversation With John Goodlad," Educational Leadership, May 1999
Robert Marzano, Jane Pickering, and Jane E. Pollock, Classroom Instruction that Works: Research-based strategies for increasing student achievement, Alexandria, Virginia, Association for Supervision and Curriculum Development, 2001
Robert Marzano, What Works in Schools: Translating Research into Action, Alexandria, Virginia, Association for Supervision and Curriculum Development, 2003.
Bruce Joyce and Marsha Weil, Models of Teaching, 1996.