Archimego's Harmonic Distortion internet blind test.

[Confused]

Another of Archimago's internet blind test initiatives was announced yesterday.  See link below for full details, there is no need for me to repeat everything in this post.

The last test Archimago did was a comparison between different music players, this was interesting and did produce some positive conclusions.

I have just downloaded the music files without any problems, no issues at all.

It is also worth noting that anyone trying this has until April 30th 2020 to submit their results, so plenty of time here and no need to rush.

I was fascinated by the music player results having tried the blind test myself, and I suspect this test will be just as interesting.  So I highly recommend that anyone who is interested should give this a try, it should be well worthwhile and educational.

https://archimago.blogspot.com/2020/01/i...monic.html

[daniel.avasilichioaei]

I'm in.

[thumb5]

Just had a crack at this. Interesting, and recommended -- I'll look forward to the grand unveiling!

[Confused]

For those interested but never got around to trying this (or tried it but never submitted the results), the closing date to submit results is tonight. (30th April 2020)

Even without knowing the results, I found this test to be fascinating. There is much talk of measurements elsewhere, distortion of 0.001% versus 0.016% and so on. I think tests like this are excellent, if nothing else for learning what these numbers might mean (or not mean) to your very own ears.

It does not take long to do the test, you can even decide which of the demo tracks represents your own listening habits, and do the test with the four versions of that track. Plus, one advantage of doing the test super late is you will not need to wait too long for the results.

If nothing else I am looking forward to the results, and it doesn't matter if I have scored 100% correctly, or got everything completely wrong, either way I will learn something valuable.

[Confused]

Part 1 of the results were published yesterday.

http://archimago.blogspot.com/2020/05/bl...-high.html

I have found it very interesting to look at the above versus my own listening notes. Could I easily discern the tracks in order of harmonic distortion? Frankly, no, I could not.

Looking at this in more detail though, it seams that the influence of harmonic distortion for me as a positive or a negative seems to vary depending on what specifically I am listening to. I shall certainly be interested to read the rest of the analysis.

It is also interesting to think back over the time I was listening to these tracks "blind" and also to relate this to the recent threads covering measurements. It is easy to worry about on-paper claims of THD of 0.0001 versus a measured 0.0016 or whatever, but a test like this helps to correlate this to an actual listening experience, and much higher THD numbers, which is fascinating I think.

It might also be fun to listen again now that the status of each version of the tracks has been published, to see if I can pick anything else up now expectation bias is fully primed with the actual data.

[f1eng]

Given that measurement tradition would have it that the performance of the measuring instrument needs to be at least an order of magnitude better than the device under test, distortion in this case, and the measuring instrument here is speaker/room/ear I would not expect low levels of distortion to be heard, personally, rather than imagined.
Very, very few speakers have mid band distortion of 0.1% or better, which is -60dB so expecting being able to consistently detect less than 1% distortion by listening goes against established norms.

If I set my volume control at a nice listening level with a -15dB signal, which is probably about the average of most non classical music (or non loudness wars at the other end of the scale) I can just hear a -75 dB signal in room.
The idea that I would actually be able to detect this barely audible level of sound added to actual music is absurd to me.

After my experience, post retirement, of careful level matched blind (I get my daughter to make the changes) comparisons at home most of my existing held beliefs about telling kit apart by listening have been debunked.
I still hear clear differences between microphones, record players and speakers though.
All the differences I thought existed between my very expensive cables and cheap adequate ones evaporated when I didn't know which I was listening to what a lot of money I have wasted.
It was before I went Devialet but comparing DACs, from the one in my Metric Halo recorder to a Linn Klimax all the differences I was sure i heard disappeared when the levels were matched and I didn't know which was being used.
I did hear differences between power amplifiers, but all were explainable by clipping, very high distortion in bass or by the frequency response alteration caused by high output impedance in transformer coupled valve amps and amps with low levels of feedback.

Nowadays if the distortion is better than 0.1% and SNR better than -80dB I expect a device will be audibly transparent to me playing music recordings through superb speakers in a domestic room.

I am much more relaxed now since I always hated evaluating kit when I could be listening to music.

[Jean-Marie]

Given that measurement tradition would have it that the performance of the measuring instrument needs to be at least an order of magnitude better than the device under test, distortion in this case, and the measuring instrument here is speaker/room/ear I would not expect low levels of distortion to be heard, personally, rather than imagined.
Very, very few speakers have mid band distortion of 0.1% or better, which is -60dB so expecting being able to consistently detect less than 1% distortion by listening goes against established norms.

If I set my volume control at a nice listening level with a -15dB signal, which is probably about the average of most non classical music (or non loudness wars at the other end of the scale) I can just hear a -75 dB signal in room.
The idea that I would actually be able to detect this barely audible level of sound added to actual music is absurd to me.

After my experience, post retirement, of careful level matched blind (I get my daughter to make the changes) comparisons at home most of my existing held beliefs about telling kit apart by listening have been debunked.
I still hear clear differences between microphones, record players and speakers though.
All the differences I thought existed between my very expensive cables and cheap adequate ones evaporated when I didn't know which I was listening to what a lot of money I have wasted.
It was before I went Devialet but comparing DACs, from the one in my Metric Halo recorder to a Linn Klimax all the differences I was sure i heard disappeared when the levels were matched and I didn't know which was being used.
I did hear differences between power amplifiers, but all were explainable by clipping, very high distortion in bass or by the frequency response alteration caused by high output impedance in transformer coupled valve amps and amps with low levels of feedback.

Nowadays if the distortion is better than 0.1% and SNR better than -80dB I expect a device will be audibly transparent to me playing music recordings through superb speakers in a domestic room.

I am much more relaxed now since I always hated evaluating kit when I could be listening to music.

^
|_______ This a thousand times!

cheers,

jean-Marie

[thumb5]

Part II of the results now published: http://archimago.blogspot.com/2020/06/bl...-high.html.

Interesting and thought-provoking results.  It's a shame the sample size was quite small, but the conclusion Archimago draws tentatively is pretty much in line with @f1eng's thinking:

Nowadays if the distortion is better than 0.1% and SNR better than -80dB I expect a device will be audibly transparent to me playing music recordings through superb speakers in a domestic room.

[Confused]

Part 2 of the results were published yesterday:

https://archimago.blogspot.com/2020/06/b...-high.html

There is a lot to digest here.  Certainly there is some evidence that the levels of distortion used in the test makes a difference, but it is a very long way from a statically clear correlation.

Some interesting trends though, such as those that used headphones performed a lot better than the speaker only group, which perhaps lends some weight to @f1eng 's point about speaker distortion.  That said, the "speaker only" group failed to find even the 3% distortion distortion sample "worst", and looking at the of the order of the samples best to worst, it looks almost random to me!  (Note, I am not a statistician)

Another interesting result is that the only group that correctly identified the lowest distortion track best, was the group that believed subjective preference did not correlate to THD.  Remarkable, although this was a very small group, and it begs the question what their order of subjective preference was.  Did they really pick out sample D as having the most THD (which they did) but then stated a preference for this?  (this bit not clear) 

In addition, there are many cases where even the 3% sample was not found to be "worst".  I guess this hints at why some valve amps can sound great when they have appalling on paper measurements.

So interesting results that indicate some correlation with THD.  I suspect this will be one article that I will probably re-visit and re-read a couple of times, to try to fully adsorb and think through what is there, but I am starting to think that THD relates to sound quality rather less than I previously thought.  Which then begs the question, what does?

EDIT:  I stated the above post this morning, got distracted by other things, then finished the above post.  The point being I wrote my post before actually reading @thumb5 's slightly earlier post above, although it is nice to see that we seam to have independently reached very similar conclusions.  (but yet again, thumb beats me to it ..... )

[Confused]

Part 3 published yesterday "Subjective Descriptions":

https://archimago.blogspot.com/2020/06/b...-high.html

Less analysis, but more of a fun read ...