by Scott Methe
Schooling is so fundamental to social health and well being that Title II of the 2010 Affordable Care Act recognizes school achievement as an indicator of public health. To achieve is to become literate and as such, school achievement is the sine qua non for a thriving democracy founded upon an engaged public.Although a spoken language largely develops without instruction, mastering a written language requires expert teachers. Yet teaching is under attack and its most essential function, testing and assessment, is being taken away. Arthur C. Clarke predicted a future where “any teacher who could be replaced by a machine should be.”1 This dictum is commonly used to either extol or diminish teaching and recently, politicians and pundits are attempting to convince the public that replacing teachers is a viable remedy because its schools are failing.2
According to Diane Ravitch, school failure is largely a myth, and most schools are improving rather than underperforming. One reason why public perception of school failure is so rampant, she argues, can be found in the multi-billion dollar educational technology industry and its corrupt political partners,3 who highlight the less-common and more predictable cases of school failure to build false demand for an online school market.
The Test Tech Industry: Merging Test Publishing and Educational Technology
Owing to the stability of the public school market and to recent mergers with the technology industry,4 test tech can abandon the message of school failure. Demand for test tech is strong and as a result, the market is flooded with automated web-based assessment systems (that have parent and teacher advocacy groups raising concerns about student privacy).
The test tech industry provides valuable products to schools that can strengthen our democracy if the products are well designed. Testing that is precise and informative improves student achievement and gives teachers a sense of agency.5 Tests are also an important means of securing the public trust through accountability. Accountable organizations make data available so that consumers can ultimately decide if an institution is acting in their interest. Corporations and schools are compelled to secure public trust via ethical and legal principles known, respectively, as fiduciary duty and in loco parentis. Because testing is such a vital teaching function, is not surprising that the test publishing industry enjoys a multi-billion dollar market cap that should grow rapidly as it merges with technology.
Assessment Illiterate Teachers: Test Tech’s Highway into Education
In a recent co-authored article, one of Pearson’s lead scientific advisors wrote that web-driven assessment is “more efficient, effective, and consistent than procedures managed by people.”6 The message is simple: test tech will help struggling students because teachers alone cannot. Is the test tech industry conspiring to take tests away from teachers or are teachers taking testing out of their own hands? It is unclear if teachers will consent to having their most essential function removed from them—but evidence suggests that they will not resist.
Recent meta-analyses suggest that teacher preparation programs offer inadequate training in measurement and psychometrics, which is associated with assessment, data, and scientific illiteracy.7 Low knowledge and skill partially explains why teachers generally mistrust tests (especially “standardized” tests). As a remedy, attempts to install assessment literacy standards into teacher prep programs and state educational agencies have been largely ignored.8
In practice, teachers with low assessment literacy are likely to disregard test administration procedures and underestimate the impact of measurement error on instructional decisions. Uninformed users and unreliable tests can also combine to produce unrealistic instructional recommendations. For instance, students who typically perform below their peers but who obtain average test scores may not receive the instruction that they need (i.e., false negatives). Instead, instructional time and resources could be allocated to students who commonly achieve on par with their peers but who performed poorly on a test (i.e., false positives).
Teachers comprise the primary test tech market, and without assessment literacy, it may be difficult to resist the seasoned pitch of a snake oil salesman. As a result, test tech marketing teams can sell anything that purports to call itself a test and they can disregard scientific standards if they so choose (as is the current case). The National Science Foundation, like most scientific organizations, requires that test tech products undergo substantial psychometric evaluation until evidence supports large-scale dissemination. Currently, test tech is an unrestricted wild frontier of dubious products that only claim to be evidence-based. This brash approach indicates either partial or full disregard for the transparency of the scientific process and an interest in immediate profit.
Promoting Assessment Literacy to Prevent Corruption
Test tech companies like Houghton Mifflin Harcourt claim to “change lives through learning,” but does the idea behind this motto imply that teachers are not integral to learning? Does it imply that tests must be taken out of their hands? How can test tech products improve student literacy and secure the public trust as effectively as a teacher with assessment expertise?
Education does not fail in general. Instead, education fails when specific instructional processes fail, and no process is more important to teaching than gathering and using information (i.e., data) to improve student learning. If unchecked, assessment illiteracy can create a profit highway into the public schools, but data literate teachers can hold test tech to higher standards.
When public schools unintentionally purchase evidence-poor products, then promoting shareholder profit takes the place of promoting the public good and social policy becomes a proxy for corporate policy. Using test tech to remove the burden from assessment-illiterate and test-averse educators is a logical sales strategy, but an unsustainable social and educational policy.
1. Arthur C. Clarke, “Electronic Tutors,” Omni, June 1980, 76-79, http://goo.gl/hYEDsE.
2. Diane Ravitch, Blog, August 10, 2014, http://bit.ly/1DjwUMM.
3. Diane Ravitch, Blog, http://bit.ly/1t8EyXO.
4. Katherine Rushton, “Pearson’s Dominance of Textbook Market Is Under Examination,” Telegraph, October 11, 2014, http://bit.ly/1t8ERlz.