Why ABX Testing Usually Produces Null Results with Human Subjects
by Teresa Goodwin

ABX testing has been extremely controversial since it was introduced decades ago. It is my hope to put this testing protocol in its final resting place by combining its sad history coupled the difference between how humans perceive sound and how ABX testing actually is applied.

The "X" in the ABX is either A or B, randomly selected, the listener needs to identify whether that "X" is "A" or "B". Unfortunately human beings do not have the ability to compare three sonic events sequentially. One must keep a sonic memory of sample "A" they just listened to so they can compare it to the sample "B" and then listen to "X" and try to decide if it sounds more like "A" or "B". It is the introduction of this third sound that makes it impossible for human beings since we can compare two different sounds as long as we don't wait too long however our sonic memory cannot juggle three no matter how many times one is allowed to go back and forth. Thus ABX tests usually get null results, and cause listening fatigue.

The better way to do this is to play "A" in a relaxed setting for an entire piece of music, at least five minutes and then play the same piece of music with "B" and then ask not if they sound different, but instead which one did you like? This is how most people shop for stereo equipment. Thus, it is not the methodology of ABX tests I object to, but instead their very existence.

Since the introduction of ABX double blind testing protocols many decades ago I have known they were complete and utter frauds and that is one reason I started my print newsletter in the 1980's and later my blog "The Audio Iconoclast" http://audioiconoclast.blogspot.com/

From its purpose statement "The Audio Iconoclast will challenge many deeply held beliefs in both the audio and musical communities. In music and its reproduction explaining what one hears when it is not directly measurable is not easy, the common practice is to dismiss it. This is wrong! In our world of music enjoyment there are subjectivists "music listeners" and objectivists "audio scientists" who try to measure phenomenon. Music listeners believe what they hear with their ears. Audio scientists do not believe what they hear unless they can quantify and measure it. If they cannot measure it, it does not exist and they convince themselves they are not hearing what they hear! My quest is to show the wisdom of enjoying the sound of music and accepting what one hears, even if it cannot be scientifically proven."

For example SACD would have replaced CD by now if not for ABX tests and pseudo-scientific studies in AES papers. Anyone possessing a pair of ears on the sides of their head can clearly hear the huge difference between low and high resolution for themselves however too many of them have been brainwashed not to believe their own ears and instead rely on these pseudo-scientists who over the years with ABX double-blind testing have proven:

1) All amplifiers sound the same.

2) All CD players sound the same.

3) A coat hanger sounds the same as an expensive interconnect.

4) MP3 sounds the same as CD.

5) CD sounds the same as SACD.

6) High resolution PCM sounds the same as DSD.

Remember the infamous 1987 blind listening test conducted by Stereo Review that concluded that a pair of Mark Levinson monoblocks, an output-transformerless tubed amplifier, and a $220 Pioneer receiver were all sonically identical?

Or perhaps you remember the ABX test that was likely the most damaging to these pseudo-scientists, comparing a known audibly defective amplifier to a perfectly working one? All listeners were able not only to hear the defect in the amplifier but able to describe it's distorted sound under normal listening conditions. However using ABX testing protocols none were able to identify the difference between the defective and the working amplifier with any statistical significance, thus proving beyond a shadow of a doubt that ABX testing does not work. In addition I am sure none of the participants would be willing to take home the defective amplifier, I am quite sure they would all want the perfect working one!

ABX double-blind testing should be banned by all intelligent people as the absolute scam it is. If one cannot prove the differences people experience every day of their lives when listening to music they love then any such tests are total and complete failures. But more dangerous than that they are hurting the sales of superior audio equipment, superior recordings and the musical satisfaction of gullible music lovers who believe these tests instead of their own ears because of the scientific garb they are dressed in. It is time for the real motives of the anti-high resolution crowd be revealed and their rhetoric buried forever so the masses can actually listen to high resolution with unbiased ears!

In summery ABX double-blind testing does not prove that everything sounds the same as real sonic differences are easily heard in casual listening. No, instead what ABX double-blind testing proves is that human subjects do not have the ability to compare three sonic events sequentially with any statistically significance, revealing a deficiency in short-term sonic memories of our species.

I believe even the most golden-eared audiophiles would not be able to identify differences with any statistical accuracy between an MP3 music file and a professional master recording using ABX double-blind testing protocols. This does not mean we should all only listen to MP3s on the cheapest stereos we can find, quite the contrary it means that we should enjoy the highest resolution music possible on audio equipment that we have determined to sound the best using our ears in standard casual listening evaluations.

Further reading:

ABX test

http://en.wikipedia.org/wiki/ABX_test

Blind Listening Tests are Flawed: An Editorial by Robert Harley

http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial