I'm coming 'off the fence'

[deviousalet]

With the best will in the world, there’s no way of controlling for expectation bias except by blind testing.
...
Matt

Nice post, Matt: kind, open, and respectful. This could be a fun thread.

I agree with much of what you said but I'd caution against putting too much weight on even blind testing. The scientific method is wonderful, but tests are difficult to get right even recognizing all the barriers to objectivity. Your comments about the human brain and subconscious bias are very appropriate. Add in what we know from cognitive and perceptual psychology and you can easily understand why the perfect test is a lofty goal. Science can be as much snake oil as anything else, given the humans behind it. Look at the data about the percentage of even peer-reviewed studies that are critically flawed. So I suggest remaining skeptical about blind tests and other such data even as I've worked extremely hard to create the best tests possible.

Saying that there hasn't been any valid attempts at blind testing of audio equipment (including cables) is dismissive of many good tries. If you're willing to accept the imperfection of humans then we can learn from attempts at rational evaluation. We can attempt to control for the way our minds work through more rigorous experiments, greater sample sizes, etc. This is one path. I've enjoyed reading the Big Sound '15 evaluations of headphone system and their attempts at blind testing. http://www.innerfidelity.com/category/big-sound-2015 in total and http://www.innerfidelity.com/content/big...-i-learned specifically may be of interest. I also enjoy all the measurements of loudspeakers even though many experts recognize that as at best a small part of evaluating a speaker given the flaws in such measurements. Choosing speakers by measurement alone would't be a wise approach (unfortunately for those of us attracted to rational evaluation) even if there's a strong correlation between success and speaker manufactures that do measure their speakers vs. those that don't. I have much more respect for a company that publishes test results of their equipment than those who don't. Measurably lower distortion usually does mean better sound. However as we can see with Devialet measurement without listening can lead to flaws in audio equipment. Devialet firmware updates please some people and displease others but Devialet seems to have no control of that, probably because they measure but don't listen (at least they don't admit to listening). In contrast PS Audio keeps improving their firmware to almost universal acclaim and they both measure and listen when attempting to improve the sound of their gear.

Another path is to accept that the end goal is a happy perception of music so whatever path gets someone there is also valid. Bill Lowe (the head of AudioQuest) says that the value in cables is as much about the relationship between the user and their equipment as it is about the technical details. I love his comments about equipment changes being like bringing home flowers to enhance a relationship, not that the flowers make a technical improvement but rather that your enjoyment changes through participation in the relationship. As someone of very different persuasion I don't enjoy this path and think of it as bringing Bill Lowe flowers in the form of wasted money, but I recognize that others have more of an emotional relationship with their equipment and so may gain more enjoyment through enhancing that relationship with new toys.

There's got to be an appropriate balance where we can be educated enough about the data to avoid the snake oil yet in touch with the reality of human perception and psychology enough to work with people as the not-entirely-rational being we are.