I was in the supermarket last week, talking to some journalists about lying. We were talking about the fact that bottled water costs more than gasoline, and that some brands cost two or three times as much as others. They suggested doing a blind taste test--pouring one of each into a glass and seeing if people could tell the difference.
Big mistake! This is the same mistake that the Pepsi Challenge forced the poor shmoes at Coke into making.
The reason it's a mistake is that in real life, there's almost never anything that's really blind. You know what container that beverage came from. You know whether the table has a white linen cloth on it--or whether you're at a luncheonette. You can see the look in the doctor's eyes when she talks to you. You can sense the confidence of the sales rep whens he brings the latest advance in ball bearing technology to your office.
Blind taste tests take the arrogant position that there is some sort of truth. I don't think there is.
No, the right taste test to do is not Brand X vs. Nationally Advertised Brand in unmarked glasses. The right test is to switch the contents but keep the labels. How does that water taste in this bottle?