ATLANTA The stain of cheating spread unchecked across 44 Atlanta schools before the state finally stepped in and cleaned it up. But across the country, oversight remains so haphazard that most states cannot guarantee the integrity of their standardized tests, The Atlanta Journal-Constitution has found.
The AJCs survey of the 50 state education departments found that many states do not use basic test security measures designed to stop cheating on tests. And most states make almost no attempt to screen test results for irregularities.
For example, a computer analysis of erasures on test papers and a statistical analysis of improbable gains on tests are both proven ways to catch cheating. Georgia uses one of those methods, but Alabama uses neither.
The survey reveals other wildly inconsistent practices around the country: Some states require outside investigations of cheating in school districts, but most states permit districts to investigate themselves; some states look for unusual increases in test scores, but most dont; about half send out independent monitors to oversee testing, about half do not.
At the root of the inconsistencies: The United States has a clear national strategy of administering make-or-break tests to nearly every schoolchild, but it has no national strategy to ensure the tests integrity.
Ten years after the No Child Left Behind Act made testing the keystone of U.S. education policy, the federal government has yet to issue standards, guidelines or even recommendations on how to prevent and detect cheating.
For years, federal officials have ignored warnings from government watchdog agencies and testing experts that inadequate security can lead to system cheating on a scale that undermines the entire system of testing. Systemic cheating gives education officials a false understanding of student achievement and, more important, robs struggling students of access to the extra help they need to improve.
Cheating scandals have surfaced in several major cities, and a Journal-Constitution analysis of test scores earlier this year suggests a nationwide problem. The newspaper reported in March that 196 school districts exhibited patterns of suspicious test scores similar to those in Atlanta, where the patterns proved to be cheating.
And if cheating is widespread, it is likely to get worse. As more states and districts tie teacher evaluations, bonuses and pay to test results, the motivation to cheat will only increase in the years to come.
For now, the nation spends as much as $760 million a year on testing required by NCLB, but it spends little to ensure the test results are legitimate.
To spend all this money and all this energy on testing, and the one area where we havent devoted the same energy is standardizing the administration of the test to deter cheating, said Wayne Camara, vice president of research at The College Board, which administers the SAT and Advanced Placement tests. To have better or standardized procedures would limit opportunities for cheating.
IS A NATIONAL TESTING POLICE THE ANSWER?
Congress seized a measure of state and local authority over education when it passed No Child Left Behind and created the requirement for high-stakes tests. But federal officials say any directive from them to ensure that the test results are valid would be an overreach.
Camara and other testing experts say the federal government should take a more active role in helping states identify and stop cheating.
Without any kind of standardization, states are left to make it up on their own or borrow from their friends, said Scott Norton, the former assessment director for the Louisiana Department of Education. Im not proposing a set of rules that every state must follow, but I think a set of best practices would be helpful.
Secretary of Education Arne Duncan, in a recent interview with the AJC, however, said test security is best left to state and local officials.
So much of this is best done with thoughtful leaders at the state level, not a new complicated federal bureaucracy, Duncan said. I dont think anyone wants a national testing police.
The secretary also told the AJC he sees little incentive for educators to cheat.
As anyone can see from the testing scandal in Atlanta, cheating was not worth it, Duncan said. That was a disaster for the city and a major black mark. Other places have a tremendous incentive not to cheat and become embroiled in something like that.
NO INCENTIVE TO CONFRONT CHEATING
Poor testing oversight is often the result of education officials ignoring what no one wants to see.
In Atlanta, for example, accusations of cheating were met with denials not only from district administrators, principals and teachers, but also from local business leaders.
There is typically little to no incentive for anyone to take threats to test security seriously, said Greg Cizek, a professor and testing expert at the University of North Carolina at Chapel Hill. Educators are happy when test scores go up; parents are happy when their children do well; students are pleased when they are declared to be proficient; the public is assuaged when all schools appear to be increasing learning.
He said that aggressively pursuing test security measures is a lose-lose situation in most cases.
With no motivation from the federal government and with little reason to look for cheating on their own, most states designed security systems that have failed to find blatant cheating even when the evidence is right in front of them.
Records show some state officials failed to act when whistle-blowers stepped forward. Some states did nothing to investigate schools where students posted almost impossible gains on tests from one year to the next.
Part of the problem is a lack of looking. Of the 46 states that responded to the AJC survey: 24 states did not conduct an analysis looking for improbable test improvements in 2012. A statistical analysis of test results can take many forms, but all examine test results for increases in scores that are so extraordinary that they are unlikely to result from teaching alone.
Twenty-one did not look for an improbable number of changes from wrong to right answers in 2012. Some states, including Georgia, have their testing contractors feed answer sheets through a scanner that can detect erasures and evaluate whether a wrong answer was erased and replaced by a right one. A certain percentage of such changes can signal cheating.
Thirty-four states did not screen for unusually similar answers that would suggest that students copied from one another or that a teacher filled in answers on multiple tests.
The numbers are even grimmer than they appear.
Some states that conduct screening hope that just the threat of getting caught is enough to discourage educators from cheating. These states, like Texas, perform erasure analyses but dont actually use the results to initiate investigations. Texas doesnt even share the results with its school districts.
The Pennsylvania Department of Education paid its testing contractor to conduct an erasure analysis in 2009. Even though the results suggested cheating in Philadelphia, Pittsburgh, Scranton and several other districts, the department did not investigate until last year after the Philadelphia Public School Notebook, a nonprofit news organization, exposed the analysis.
Even when states do use screening results to spur investigations, state officials say they often accept explanations for the score increases like a new teacher or a new curriculum. Testing experts say these are unlikely explanations for radical score increases.
Thirty-seven states allow school districts to conduct cheating investigations of their own employees, according to the AJC survey. Testing experts say an independent agency should always investigate signs of cheating, not district officials, who benefit from increased test scores.
Many states make it easier for educators to cheat by using weak security procedures.
Forty-one allow teachers to proctor tests for their own students. The proctors responsibilities include ensuring the security of the test; for example, making sure no unauthorized materials are used, enforcing time limits and reporting irregularities.
State and district records from multiple states show that some teachers are not above guiding their students to the right answers.
A teacher at a Phoenix elementary school, for example, told a colleague that shed used red and green M&Ms; during a test to nudge students toward the right answers. If she set a red M&M; on a childs desk, that was a signal that the pupil had the wrong answer and should stop and do the problem over again. If she put a green one on the desk, that meant the child selected the right answer and should move on.
The threat that a state monitor may look in on a test can curb this kind of cheating, experts say. Yet only 25 states used independent monitors in 2012, and most of them deployed fewer than 20 monitors to cover hundreds or thousands of school. One state used only a single monitor.
These weak security systems allow administrators to shuffle unprepared students from grade to grade, while congratulating themselves for raising test scores.
HIGH STAKES NEED HIGH SECURITY
When Congress passed No Child Left Behind in late 2001, lawmakers placed tremendous pressure on educators to improve student achievement. They did not, however, pressure the states to ensure the improvements were genuine.
Thats because when the law was written, no one understood the consequences of weak safeguards, said U.S. Sen. Johnny Isakson, R-Ga., who, as a member of the U.S. House, helped draft No Child Left Behind.
I am the first to say that when debating No Child Left Behind, nobody contemplated a situation like what happened in Atlanta, Isakson said.
Some experts assert, however, that cheating was a predictable response to NCLB.
If you think you can have high stakes without high security and still have trustworthy measures, youre kidding yourself, said Scott Marion, associate director at the Center for Assessment, a not-for-profit consulting firm that works with about 30 states to develop tests and teacher accountability systems.
Marion says tests are still largely trustworthy, but cheating is a threat to the validity of scores.
Sharon Rideau, an elementary and middle school teacher in Phoenix, wrote her doctoral thesis on cheating. Rideaus survey of more than 3,000 Arizona teachers in 2008 revealed 50 percent either had cheated themselves or knew a colleague who cheated.
I think it happens much, much more now, Rideau said. It happened before NCLB and now we have all this pressure on us; its had a great impact. In addition to the Atlanta scandal, widespread cheating also broke out in Baltimore and El Paso, Texas. Currently, cheating investigations are under way in Philadelphia, Pittsburgh and other districts in Pennsylvania, in Toledo and Columbus, Ohio, and in St. Louis.
The cost of screening for cheating does pose a problem, according to the AJC survey. Of the states that never conducted any screening for cheating, 60 percent cited the expense.
But the costs of looking for cheating are minimal compared to the price tag of administering the tests. Massachusetts, a state with strong security procedures, paid $36,000 to conduct test screening and $140,000 to conduct investigations in 2012, according to the survey. Thats 0.7 percent of the $27 million the state spent to administer tests.