The biggest problem with blind testing

Nearly all blind tests have a major problem and here I propose a solution. It makes these tests far more revealing.

What is wrong with blind tests?

Some would say "what isn't wrong with blind tests?" Here I will focus on just one aspect. I'm in favour of blind tests, although they aren't my idea of a good time.

Most blind tests suffer the same problem that plagues sighted tests. They both usually rely heavily on audio memory. As a result, in order to compare one has to process the test sample, form some kind of opinion, remember what was heard, then compare it to the next sample. This can work fine for major differences, but normally blind testing is used to try to make valid comparisons involving subtle differences.

No matter how scientifically conducted, any blind test that relies on audio memory is seriously limited. 

Trusting your eyes vs ears

Our visual acuity is generally better than our auditory perception.We pick up more information from what we see, with greater accuracy and we also generally remember what we've seen with greater accuracy than what we've heard. If presented with contradictory visual and auditory information, which one do we believe? If we see the source of a sound coming from location A but it sounds like it's coming from B, what is our conclusion? We will trust what our eyes tell us every time. Our instinct tells us that our eyes are more difficult to trick than our ears.

 

 In the image above, the left side of the gray donut appears darker than the right. However, in the image below, the only change is that the red line is removed. Now you can clearly see that this is a trick - there is no difference in tone between the two sides of the donut.

 

In order to pick subtle differences in tone, our eyes need to see the two side by side with nothing in between. A single line between is enough to allow the background to throw us off. This is a simple example, but an audio comparison includes many tricks which are not so easy to demonstrate and identify.

A blind test represents a much greater challenge, so greater care should be taken to control any factors that might skew the result.

Instant switching

Fortunately it's quite simple. If you are comparing fairly subtle differences, you need an instant switching mechanism. When this is done, audio memory is note required and tiny differences are instantly revealed. If there is a change in tonal balance, it will be noticed immediately on the switch. If the sound stage is affected, instruments will suddenly move. Any difference that can be heard will be revealed with much greater clarity. False differences are also eliminated. Previously you might think you heard a difference that was simply an auditory illusion.

Gainphiles instant level matched switch box

Shown above is a switch box that Gainphile made up for a recent listening test of active crossover units. It switches both inputs and outputs of 4 channels, where one of the inputs can be attenuated to match the other. A box like this could be used to test interconnects, DACs, active crossovers and preamps.

More about blind tests

If you are interested in blind tests, you might also like to see some examples a blind test results that have been posted online.

Blind test results >


Why is blind testing such a hot topic? >