Question 13 of the Spring 2003, Grade 5, Texas Assessment of Knowledge and Skills (TAKS) released Science test reads as follows:
Which two planets are closest to Earth?
A Mercury and Saturn
B Mars and Jupiter
C Mercury and Venus
D Venus and Mars
Out of the 285701 students who took the test, the response frequencies for answers A, B, C, D, and no answer were, respectively, 24145, 44527, 71062, 145545, and 422.
The correct answer depends on when question 13 is asked. When the test was administered, Mercury was closest to Earth, Mars was second closest to Earth, Venus was third closest to Earth, and all of the listed answers, A, B, C, and D, were incorrect. The Texas Education Agency made the mistakes of giving credit for the incorrect answer D (even though answer D was worse than answer C) and of not throwing out question 13. The TEA wrongly increased the scores of those 145545 students who chose answer D and, with respect to the scaled scores that would result from throwing out question 13, wrongly decreased the scaled scores of the remaining 140156 students. The remedy is to remove credit for answer D, throw out question 13, and rescale the test scores based on whatever valid test questions remain.
The test was administered sometime from 28 April 2003 to 3 May 2003, during which time the distance from Mercury to Earth was no greater than 0.66 AU, the distance from Mars to Earth was no less than 0.96 AU and no greater than 1.03 AU, and the distance from Venus to Earth was no less than 1.40 AU. [The Astronomical Unit (AU) is nearly 150 million kilometers and is very close to the average distance from the Sun to Earth.]
Mercury and Mars were the two planets closest to Earth from 31 March 2003 to 14 January 2004. Answer D was incorrect from 30 March 2003 to 1 September 2005. Answer C was incorrect from 31 March 2003 to 14 January 2004. Answer D was worse than answer C because Mercury was closer than Mars to Earth from 30 March 2003 to 30 May 2003.
During the ten years from 1 July 1993 through 30 June 2003, answer C was correct 69.5% of the time, Mercury and Mars was correct 21.4% of the time, and answer D was correct only 9.1% of the time.
Over many years, the average distance from Mercury to Earth (roughly 1.04 AU) is less than the average distance from Venus to Earth (roughly 1.14 AU) and much less than the average distance from Mars to Earth (roughly 1.7 AU).
Accurate distances from the planets to Earth can be obtained from the Solar System Dynamics Group's "Ephemeris Generator
". Lower accuracy formulas for approximate positions of the major planets are given here
Failure to correct the scoring mistakes
informed the Texas Education Agency of the scoring mistakes in a letter of 7 July 2003. On 4 August 2003, the TEA issued the response TAKS Released Test Items
and issued, in "Additional Information Regarding Released Science Items", the statements
Item 13 asked students which two planets are closest to Earth. This item was
intended to measure Grade 3 science TEKS 3.11(C), which required students to
identify the planets in our solar system and their position in relation to the Sun.
The issue raised about the item is based on using the National Aeronautics and
Space Administration (NASA) website and calculating the current position of the
planets. On the day of the test, Mercury was closer to Earth than Mars due to
Mercury's orbit in relation to Earth's orbit. However, students do not study "frame
of reference" or "astronomical units of measurement" until high school. Since this
level of understanding is not expected of fifth graders, the item has only one
correct answer and is a valid question for all students tested at this level.
The TEA states that "However, upon review, these items were determined to be correct and no action was taken by the agency." This TEA statement refers to questions that include question 13. By "determined to be correct" the TEA means that answer D was correct. The TEA's claim that answer D was correct is false.
The TEA states that "The issue raised about the item is based on using the National Aeronautics and Space Administration (NASA) website and calculating the current position of the planets." This TEA statement is inaccurate. Mark Loewe
recognized that question 13
should be thrown out within 10 seconds of reading question 13
, based on the fact that the correct answer depends on when the question is asked and on the fact that, when taking the test, not even very knowledgeable fifth graders were expected to be able to determine which two planets were closest to Earth.
The TEA states that "On the day of the test, Mercury was closer to Earth than Mars due to Mercury's orbit in relation to Earth's orbit." It is true that Mercury was closer than Mars to Earth; this is the reason why answer C was better than answer D. However, the TEA statement does not address the fact that, since Venus was not one of the two planets closest to Earth, both answer C and answer D were incorrect.
The TEA states that "However, students do not study 'frame of reference' or 'astronomical units of measurement' until high school." This TEA statement is irrelevant. The reason why answers C and D were incorrect does not depend on "frame of reference" or on "astronomical units of measurement"; the reason is simply that Venus was not one of the two planets closest to Earth.
The TEA states that "Since this level of understanding is not expected of fifth graders, the item has only one correct answer and is a valid question for all students tested at this level." By "only one correct answer" the TEA means that answer D was correct. The TEA's claim that answer D was correct is false.
Fifth graders are expected to know that the planets are major components of our solar system and that Mercury, Venus, Earth, and Mars, in order of increasing distance from the Sun, orbit the Sun. It is appropriate, and the Texas Administrative Code appears to urge, that fifth graders learn "how these components relate to each other", including that distances between planets "change over time."
For many fifth graders, it is easy to understand that answer D is incorrect whenever Mercury is on the same side of the Sun as Earth and either Venus or Mars is on the other side of the Sun. For some fifth graders, it is easy to understand that answer D is incorrect most of the time. During the test, some students undoubtedly realized that answer D is sometimes incorrect and some students probably realized that answer D is incorrect most of the time.
The TEA's false claim is due to incompetence or dishonesty. Texans are ill-served by such incompetence or dishonesty.
The TEA has continued to propagate the scoring mistakes and false claims without correction.
On 15 July 2005, Mark Loewe
gave written and oral testimony to the State Board of Education
on the topic "TEA failed to correct TAKS mistakes". The SBOE has failed to acknowledge or to correct any of the TEA's scoring mistakes or false claims.
is discussed at the beginning of "None Of the Above
", by Lisa Guernsey, The New York Times
, 24 April 2005.