Friday, January 19, 2007

How good data can lead to wrong decisions

Consumer Reports has just withdrawn a report on infant car seat test results. Apparently, the testing wrecked most of the car seats. This, in turn, has been reported to cause many parents to start doubting the usefulness of infant car seats! So what happened?

The press release describes the aim of the study:
"The original study, published in the February issue of Consumer Reports, was aimed at discovering how infant seats performed in tests at speeds that match those used in the government’s New Car Assessment Program (NCAP). "

In particular, "Our tests were intended to simulate side crashes at the NCAP speed of 38 mph."
What apparently happened was that the actual test was performed at a much higher speed... Consumer Reports decided to withdraw the report and do some more testing.

Was the study design faulty? Probably not. Were the data faulty? Probably not. It looks more like a failed link between the study originators and executers. The moral is that even a solid study design and a reliable execution can lead to disastrous results if communication is broken.
Post a Comment