Standardized tests must effectively gauge student achievement without giving certain students an advantage over others. Some researchers suggest that the way the tests are composed causes particular groups of students to be more likely to misinterpret information due to cultural background, language/dialect interference and gender. In 1980, Allan Nairn and associates alleged that the Scholastic Aptitude Test (SAT) ranked test takers not by aptitude but by social class. Since testers cried foul at elitist terminology such as "yacht" and "regatta," these specific words were removed from the test in the 1970s, and the SAT has come a long way in ensuring that content is more accessible to all students. Yet in 2009, SAT scores were still divided by race, gender and family income, with white upper-middle-class males performing significantly better. The repercussions can be significant when low test scores prohibit students from accessing educational opportunities.
Basing student placement decisions on standardized test scores has historically resulted in unwarranted labeling of minorities as mentally deficient and ending up in lower ability groups. Many lawsuits since the late 1960s have argued that racial differences in test scores are due to cultural bias in test design. In the case of Larry P. vs. Riles (1972), the court determined that cultural bias in IQ tests resulted in lower scores among black students. Standardized test designers must take care not to frame items in terms of situations that require familiarity with predominately white, middle-class experiences.
English language learners (ELLs) generally perform worse on standardized tests than their native English-speaking counterparts, but the lower test scores may be less due to students' knowledge of the subject matter than to their lack of familiarity with the English in which the tests are written. ELLs may misunderstand instructions or get bogged down in irrelevant details that employ unfamiliar vocabulary. On math tests, ELLs do better on formula-based problems than on word problems, the difference suggesting that the issue is based on general language competence. The problem of linguistic bias in standardized tests was highlighted in the 1970 legal case Diana vs. California State Board of Education 1970, in which Mexican-American students gained an additional 15 IQ points when permitted to take the test in Spanish, indicating that the test measured English proficiency rather than intelligence. The case never went to court but nevertheless resulted in substantial changes to the testing process including the development of IQ tests specifically for non-English speakers and allowing students to choose the language used in their responses.
In 1992, the Mid-Atlantic Equity Center reported that boys did better than girls on the SAT even though girls performed better academically in all grades. Fast forward to 2009 and females were still scoring significantly lower than males on the SAT, according to a "USA Today" article by Scott Jaschik. The explanation may reside in test construction. Test items may be geared more closely to how males solve problems. Furthermore, boys would have an advantage if test items were set in male-oriented contexts. For example, on a math achievement test, basing certain word problems are based on a football scenario might skew the test toward boys. Wading through sports terminology that they are unfamiliar with could cost girls valuable time that would be better spent figuring out less gender-specific scenarios.
In the preparation of standardized tests, content must be safeguarded properly so that no student gains an unfair advantage by having advance access to specific test items. Making test questions public compromises the validity of results. Reputable testing organizations follow rigorous security procedures that may require test preparers to sign confidentiality agreements, keep materials under lock and key and strictly specify procedures for shipping and disposing of test materials.
While campaigning for president, Barack Obama called for "a broader range of assessments that can evaluate higher order skills," and yet the current state of standardized testing still fails to assess learning readiness or capacity for intellectual inquiry. The quest for objectivity and reliability causes standardized test designers to rely heavily on multiple choice questions, which unlike real life problems have one and only one right answer. Alternative assessments --- which take into account performance-based tasks such as portfolios --- can provide a more revealing picture of a student's true competence than whether the correct bubbles on the answer sheet are blackened. Although alternative assessments present reliability challenges, an evaluation of a widened sampling of student work can be used to supplement traditional standardized tests.