Excerpts from The New York Times
Two years ago, fifth graders taking Texas's annual standardized science test faced this multiple-choice question: "Which two planets are closest to Earth?" The four choices were "Mercury and Saturn," "Mars and Jupiter," "Mercury and Venus" and "Venus and Mars."
Simple, right? The Texas Education Agency thought so; every fifth grader should know that Venus and Mars orbit on either side of Earth's orbit (remember the mnemonic: "My Very Excellent Mother Just Sent Us Nine Pizzas"?). "Venus and Mars," therefore, would have been a good pick.
But wait, said Mark Loewe, a Dallas physicist who was curious about what students are expected to know and so took the test. The question asked which planets --- not which planets' orbits --- were closest to Earth. So the correct answer depends on when the question is asked.
"Mercury, which orbits closest to the Sun, is closest to Earth most often," Dr. Loewe said, and sure enough, during that test week in spring 2003, Mercury and Mars were the planets closest to Earth, according to the National Aeronautics and Space Administration Web site. That pair was not among the possible answers.
So is the question valid? Perhaps, since the problem was written for a typical 10-year-old, not someone with Dr. Loewe's understanding of science. On the other hand, the problem ignores the physical world woven into the question, and that might trip up brighter fifth graders.
Consider, for example, another question critiqued by Dr. Loewe on the Texas Assessment of Knowledge and Skills in 2003. Students in 11th grade were asked to calculate how much force a frog would exert against a river bank while leaping off. Dr. Loewe, who has co-written a textbook on quantum mechanics, says that when he worked out the problem, he included the frog's gravitational weight (its force during rest, which he determined by using the formula for acceleration due to gravity, which was provided at the beginning of the test). But the answer key made clear that the question writer did not expect students to consider that.
Dr. Loewe found several other problems and informed the Texas Education Agency. "Texans are ill-served by such incompetence or dishonesty," he wrote. The agency responded by hiring university professors to review exam questions, says Victoria Young, director of instructional coordination in the agency's testing office. The professors review tests that relate to their areas of expertise but are also familiar with what is taught in high school.
Nonetheless, Ms. Young disputes Dr. Loewe's specific complaints. Asked whether the planets question might have led students to Dr. Loewe's answer, she responded, "Not a realistic viewpoint, in my opinion." And of the leaping frog, she says that physics educators have told her that "only if you brought a very advanced level of college physics to the table would you know enough to know that the answer could be arrived at differently."
Not so, says Dr. Haney of Boston College. Students should have known to take the frog's gravitational weight into account, and so argued a few Texans in letters to The Fort Worth Star-Telegram, which reported on the dispute. Dr. Haney says one reason for the problem is that "a lot of people who may be writing the math and science questions may not have a deep understanding of the math and science that they are trying to test."