|
Article Excerpt
Because technology is more prevalent and accessible for use in assessment, this article highlights what counselors and educators need to know when considering the use of computers and the Internet for that purpose. The article concludes with some predictions on how technology might influence assessment and accountability in the future.
**********
We, as educators, are living in an age of data-based educational reform. Data are driving decisions about the degree to which state and district standards are being achieved, what is taught in the classroom, the performance of school personnel and students, and which educational and counseling programs seem to have been effective. Decisions on who passes a class, who graduates, who is promoted, what school a student attends, whether principals are retained, whether the superintendent has been effective, and what schools are rewarded with additional resources are increasingly based on data, especially test scores.
The use of test data to determine the quality of education offered in U.S. schools was given significant attention and visibility with the passage of the No Child Left Behind (NCLB) Act of 2001. This law mandates annual testing in reading, language arts, and mathematics in Grades 3 through 8 and at least once in Grades 10-12 by school year 2005-2006, followed later by science assessments. Key provisions of the law include (a) the need to specify state education standards and align the assessments to those standards; (b) the inclusion of all students, even special needs students and those with limited English proficiency along with reasonable accommodations for those students, in the state assessments; (c) reporting at the individual student level and summary results by school, district, and state, as well as results disaggregated by gender, major racial and ethnic groups, English proficiency, migrant status, disability, and economic status; (d) the timely delivery of test results so that they can be used for educational improvement at all levels; and (e) the demonstration of adequate yearly progress followed by the designation of schools and districts in need of improvement (Achieve, 2002; American Education Research Association [AERA], 2003; Education Commission of the States, 2003; U.S. Department of Education, Office of Elementary and Secondary Education, 2002).
Although school administrators are increasingly concerned with the use of test data for decision making, the vast majority, about 9 in 10, believe that accountability, testing, and standards are here to stay. Their major concerns include an overemphasis on tests for accountability, the perception that the federal government is too intrusive in local education, and the belief that the consequences are unfair. On the whole, school superintendents and principals, particularly those in large school districts, believe that the legislative requirements are generally useful, although there may be too much reliance on standardized tests (Farkas, Johnson, & Duffett, 2003).
In that survey (Farkas et al., 2003) of more than 1,000 school superintendents and more than 900 principals, fiscal concerns offered the most challenge to school administrators, with 70% of school superintendents and 58% of principals indicating that insufficient funding was their major concern. Compounding this concern was the intrusion of mandates that prompted this complaint by school administrators:
But just beneath the surface of their money concerns is one aspect that they find especially galling: the cost of obeying state and federal laws that require them to put very specific services or policies in place. According to school leaders there are far too many of these mandates. They come in regularly from federal, state, and local governments. Most don't come with sufficient funding. (Farkas et al., 2003, p. 11)
With insufficient funding constraining the resources available for education, states and districts are eager for mechanisms by which assessments can be more easily and cheaply created, administered, scored, interpreted, and analyzed for the purpose of improving instructional programs, meeting accountability expectations, and fulfilling the mandates of NCLB. There is a widespread understanding that technology can play a critical role in storing and analyzing data for decision making, supplying information for educational accountability, and providing data on individual students for instructional purposes (Dougherty, 2003; Valdez et al., 2003). Although many states and school districts continue to test in the paper-and-pencil mode, and probably will for some time to come (Education Week Technology Counts, 2003; Wainer, 2000), educational institutions are increasingly making the transition to computer and online assessments (Bennett, 2002; Education Week Technology Counts, 2003; Hansen, 2004; Harris, 2004).
Counselors, as well as educators, are involved in educational reform and assessment in schools. In a study of school counselor activities, 93% of those surveyed reported that their guidance activities included "testing and having tests interpreted for career planning purposes (for example, interest inventories, vocational aptitude tests)" (National Center for Education Statistics, 2003b, p. 36). In that same study, a majority (58%) indicated that required state assessment for high school graduation had a mostly positive effect on education. Furthermore, according to that study, approximately 20% of counselors reported spending more than 20% of their time on academic testing.
In the results of a survey of members of two counseling-related organizations that include school counselors, Elmore, Ekstrom, and Diamond (1993) found that 74% of those completing the survey indicated that tests were a very important or important part of their work. The American School Counselor Association's (2003) National Model for School Counseling Programs provides the direction that "school counselors should be proficient in the collection, analysis and interpretation of student achievement and related data" (p. 49). Counselors, as well as educators, share responsibility for education reform efforts, and it is important that they understand the various issues related to the use of technology-delivered assessments.
This article intends to delineate some of the areas in which technology can assist the education community by modifying, streamlining, supporting, and improving assessment procedures and practices. It outlines some of what is known about the use of technology in assessment and attempts to forecast what might be possible in the future with an ongoing research effort and some creativity on the part of assessment professionals who understand the needs of educators and learners.
Among the assessment areas that have been identified as noteworthy in the interaction of assessment with technology are test selection, test construction, scoring and feedback, cost efficiencies, time savings, the assessment of higher order skills, security and verification, and accessibility and reach (E. Baker, 1999; Bennett, 2001; Education Week Technology Counts, 2003; Kerrey & Isakson, 2000; National Center on Education Outcomes, 2003; Sampson, in press; Wall, 2004). This article provides an overview of these issues and what is generally known and accepted to date. Research in the area of technology and assessment is in its infancy, and future research on technology-delivered assessment certainly will unveil additional information, new directions, and refinements of the knowledge base about this issue.
TEST SELECTION
The Code of Fair Testing Practices in Education (Joint Committee on Testing Practices, 2004) stipulates that "test users should select tests that meet the intended purpose and that are appropriate for the intended test takers" (p. 3). This precept may be especially relevant when considering the use of tests that are delivered via computer or the Internet platform. There are certain factors that are important in test selection when changing from a paper-and-pencil format to a test delivered by computer or Internet. The issues deal primarily with comparability of the scores. Would the individual obtain the same score regardless of the administration mode? Test users need to be aware of potential differences in results and carefully examine available technical information to be sure that test takers are not favored or disadvantaged by the choice of delivery platform.
The Standards for Educational and Psychological Testing (AERA, American Psychological Association [APA], & National Council on Measurement in Education [NCME], 1999) specify that "a clear rationale and supporting evidence should be provided for any claim that scores earned on different forms of a test may be used interchangeably" (p. 57). This applies to tests that might be available in paper-and-pencil format, in computer-adaptive format, and/or by computer or Internet delivery. Initial research shows that carefully designed multiple-choice tests do not seem to be affected greatly by the difference in format (Hamilton, Klein, & Lorie, 2000; Russell & Haney, 1997). Similarly, scores for an interest assessment using selected response items were generally comparable among the paper-and-pencil, computer, and Internet versions (Lumsden, Sampson, Reardon, Lenz, & Peterson, 2004). Scores on speeded tests do seem to be influenced by format and administration modes (Mead & Drascow, 1993). In addition, transmission times over the Internet can vary and certainly will have an effect on speeded tests (McBride, Paddock, Wise, Strickland, & Waters, 2001) and probably on adaptive tests and tests rich in graphics...
|
