The Law and Statistical Thinking(The New Haven case)

One thing about classification schemes is that they should be mutually exclusive and selectively exhaustive. Translated that means that no sampling unit should belong to more than one class and that every sampling unit should be in a class. So the fact that every one of the firefighters is in a class (for the purpose of descriving their race, ethnicity, height, gender, or whatever) is a good thing.

Whether or not the test that was designed had sufficient discriminatory power is a matter for firefighting experts to decide. One group (the white firefighters) thought that it did and the other group (the non-white firefighters did not). As evidence that there was a bias built into the test the non-white group pointed out that the pass/fail ratio of white to non-white did not match the white to non-white ratio of test takers. This is a simple binomial probability problem (you set the alpha limit). I don’t know if it was examined using statistical methods, but it is a legitimate statistical question. (e.g. if the ratio of white to non-white test takers was (say) 60% to 40% of the total, what is the chance that out of 100 people taking the test 85 would be white and 15 non-white, IF the test were non biased (that is, if the null hypothesis was true).

But that was not what the Supreme Court case ended up being about. Apparently the bias was evident to some (probably non-white) test takers who threatened to sue the city on the basis that the test was unfair. Rather than reopen the original design of the test and fight the expensive lawsuit battle, the city opted to throw out the test and not use the results to promote anyone. Nobody (white or non-white) was promoted using the test.

The white firefighters sued the city saying that they had been discriminated against because they WOULD have been promoted had the test been used and the city could not simply disregard the test because of the bias issue or the threatened suit.

In a 5 to 4 decision the court agreed with the white firefighters group that they had been discriminated against by the city’s actions. It is a tricky case in the sense that since no one was promoted using the case, no individual can show that he or she should have been selected over another individual. Also it adds a burden to cities to stick by testing protocols even when they change their mind (e.g. in the face of a lawsuit).

If you are interested read the opinion and Bader-Ginsberg’s dissent see:

http://www.supremecourtus.gov/opinions/08pdf/07-1428.pdf