Version: Firmware 13.1.3 + DOS 2.2.4 Date: 2019-02-12

[David A]

@David A, without wanting to appear perverse, what exactly do you mean by "sound" then?  Do you mean pressure waves in the air, or the experience of hearing them?

If the former, then I think I follow your train of argument, except that then it is certainly possible for people to hear a difference in sound quality when there is no difference in sound.  That can happen for any number of reasons which don't need to be rehearsed here.

If the latter then I really don't understand the distinction you're making between "sound" and "sound quality"; it's all the same subjective experience which is unique to the individual.

Finally, to clarify, I didn't say (and certainly didn't mean to imply) that people's experience of differences in sound quality has no value -- on the contrary, that's really all that matters from the point of view of hi-fi/audio as a leisure activity.

First, "sound": there's probably a fair bit of both meanings in what I wrote at various points and yes, I agree that people can hear a difference in sound quality when there is no difference in sound, but only when they do "hear" a difference in sound. If they don't "hear" a difference, they can't hear a difference in sound quality.

My point is this. We need to distinguish between 2 things, especially when it comes to testing to prove what is being reported.

Saying that there is a difference in the sound is reporting a subjective experience as you say, but you are reporting a subjective experience which should be able to be verified by objective testing. When it comes to saying that the sound you hear has certain qualities (warmth, brightness, detail, etc) you are reporting a subjective experience which may be able to be correlated with the results of objective testing because its a subjective report of characteristics of the sound. Saying that the sound quality is better/worse/equal, reports a subjective assessment rather than an experience and can't be verified by objective testing. There is an important difference in the reports. We can in principle objectively verify many subjective experiences but we can't objectively verify subjective assessments like better/worse

So, once we start talking about testing in order to verify reports that say something is producing a difference in sound, we have 2 ways of verifying that statement. The first is to undertake measurements of the sound which means the sound pressure level in the air. The second is to undertake listening tests in order to determine whether people can actually reliably distinguish a difference. 

There are problems with both of those sorts of tests. When it comes to measuring the sound pressure level we're faced with the fact that microphones don't respond to sound the way people do. There's a pressure sensitive device in a mic, and we have a pressure sensitive mechanism in our ears, but when we report what we hear we aren't just reporting the frequency and pressure differences a mic picks up, we're reporting the result of those differences after they've been processed by our brain which does things with the frequency and pressure differences from 2 ears which receive some of those differences at slightly different times because sound arrives at different angles to our head and there are differences in the length of the path the sound has travelled. That difference in time, and the slight differences in pressure they cause, tell us things that you can't tell from the signal output from a single mic. Yes, we can set up 2 mics on a dummy head and start to capture that sort of info but now the measurement process is becoming more difficult and our hearing has developed in ways which give us information which we may be able to identify from a feed from 2 mics but we need to know how to combine the information in those 2 feeds in order to be able to relate that information to what we hear. We don't always know how to do that.

When it comes to listening tests we also have problems. First, not everyone has ears that are equally sensitive/accurate. We say very glibly that we can hear from 20 Hz to 20 kHz but those numbers aren't exact and they don't apply to everyone. It becomes increasingly difficult to find people who can actually hear 20 kHz the older they are, and I've seen comments that most people over the age of 20 or so can't hear 20 kHz. Age affects hearing range and so does noise related hearing loss and other conditions. Then thee's the fact that what we pay attention to is important. We notice things when we're paying attention that we don't notice when we aren't and paying attention to one thing can mean that we don't notice other things which we could otherwise notice. How we listen and what we listen to in the sound arriving at our ears makes a difference. When it comes to listening tests we know that not everyone is going to hear something that can actually be heard so the results of listening tests are based on a level of probability. We accept that something is audible if enough people in a test sample can identify it correctly often enough to make it extremely unlikely that the correct identification isn't a random possibility but I've heard of one case where group tests didn't reach that level of reliability but one person managed to show that there was an audible difference by identifying the difference correctly with 100% accuracy every time it occurred.

It's certainly possible for beliefs to influence what people report hearing and people can report hearing a difference that they believe exists even though it doesn't exist but it's also possible for people to report not hearing a difference they don't believe exists when it does in fact exist. People make both those sorts of mistake. Measurements can show no difference but there is no difference to measure if you don't measure what is different. For example THD used to be reported as a simple % measure at rated amplifier output but that measurement doesn't show differences at output levels other than the specified output and it doesn't show differences in the harmonic structure of the distortion so there was a time when a lot of people said they could hear no difference between 2 amps with identical frequency response and the same THD measurement while others said they did hear a difference, and some people claimed that there could not be an audible difference between 2 amps with the same measurement while some people claimed there was a difference between amp A and amp B which had that same simple THD measurement. We all know how that argument panned out. There were audible differences and they didn't show in the measurement because the measurement being relied on wasn't the right measurement.

Science works in funny ways at times but science always starts with observations. Sometimes those observations are measurements which can't be explained by current knowledge at the time and sometimes those observations are people's reports of hearing something which isn't shown in the measurements being made. Sometimes the observations turn out to be wrong and sometimes the current knowledge or the measurements being made aren't complete and the observations lead to new discoveries which show that the observations were correct.

We can all make mistakes about whether or not there is a real difference for us to hear between 2 things like RAAT and AIR. There either is or is not a difference to account for the difference in observations. If there is a real difference then in principle we can measure it but just because we can measure it in principle doesn't mean that any specific set of tests we do will measure it because we simply may not do the right test, and it also doesn't mean that at the moment we can measure it. Its always possible that we need a new test to measure that specific difference.

Too many people think that all subjective reports can't be verified, either in part or in total, and too many people think that the measurements they have are the right measurements and capture everything. Neither belief is right and there's a lot more to finding out whether the people who think that there are no audible differences between RAAT and AIR or the people who think that there are audible differences between them are right, and also to finding out whether the differences, if they are shown to exist, can be correlated in some ways with the observations which are being made about what the difference actually sounds like but when it comes down to which one sounds better, if there is a difference, then that's the sort of subjective report which measurements can't provide information on.

Sorry for the long post but the issues aren't quite as simple as your questions seem to suggest.