I'm coming 'off the fence'

[Jwg1749]

With the best will in the world, there’s no way of controlling for expectation bias except by blind testing. People often say things like “I’d like the cheaper cable to sound better, therefore expectation bias won’t make me prefer the more expensive one” or “I’m expecting the cables to sound the same, so I’m immune to expectation bias”. It would be nice if statements like this were true, but sadly that’s not how expectation bias works.

(In passing it’s worth noting that ‘expectation bias’ isn’t exactly the right term. What’s actually in play here is a wide range of unconscious cognitive biases, of which expectation bias is just one.)

In order to fully control for cognitive biases, you would have to be absolutely certain that you had excluded all unconscious cognitive cues that might incline you to make a false judgement. But how could you ever achieve certainty about this? You might be utterly convinced that your judgement was not affected by cognitive cues, but you cannot prove it, unless you remove the cues in the first place, by blind testing. 

You might argue (as you do in your post) that being relatively certain is good enough. Well, it isn’t. Even the smallest degree of bias, which might be totally unconscious, can be enough to skew a set of results completely.

Now consider the state of play in audio cables. We have thousands of anecdotal instances of people claiming to be able to discern differences between cables in non-blind evaluations. However, there isn’t a single halfway-reliable instance of someone being able to replicate those results in a blind test. Doesn’t that strike you as odd? And doesn’t it strike you as odd that cable manufacturers haven’t tried to use blind testing to advertise their cables? 

On the other hand, we have a wealth of evidence that shows how easily our minds are influenced by extraneous cues without our consciously knowing it. The placebo effect is the best known of these. 

It’s important to recognise that no-one is accusing anyone else of being dishonest (with the possible exception of some cable manufacturers). I’d trust you to be as honest as I’d hope I would be in the same circumstances. Nor is anyone accusing anyone else of stupidity. On the contrary, what’s at issue here is the sheer complexity and resourcefulness of the human mind, which is after all one of the most complex things in the known universe.

In these circumstances it seems to me that non-blind evaluations of hi-fi cables should be treated with amiable scepticism. Good on you for being honest about your opinions. But I hope you’ll understand why I couldn’t reasonably treat your evaluations as evidence of anything except your own honest good intentions.

Matt