Version: Firmware 13.1.3 + DOS 2.2.4 Date: 2019-02-12

[Soniclife]

I did the standard rew sweeps, with the mic close to the bass mid driver to reduce room interaction.  I then looked at all the measurements rew offers, and was amazed how close they all were, with no smoothing applied as well.  So lots of measurements, but no jitter test, but jitter was solved years ago in good designs, so any change there would likely be tiny.  

I would be surprised if there was an easily measurable change from a firmware that does not imply something was fixed or changed. But the implication above was something was measured, so it was worth looking into.  Sam could be a whole other thing, I didn't test that.

REW is good for measuring speakers and speaker behaviour in rooms but it's not good for measuring electronic components. No professional measures an amplifier's frequency response or distortion characteristics by measuring output through a speaker. Electronic equipment is measured using different gear and different tests to those REW offers.

I'll give you an example of why REW won't work for testing electronic equipment. It's not relevant in the RAAT/AIR case but it may give you an idea of why you would not want to use REW to measure electronic components and the signals they process.

Let's say you have an amp and you want to measure distortion. Measure it in the usual way and you get figures for the distortion level for different harmonics for harmonic distortion. Convert those figures to decibel values. The output of the amp when you do your frequency sweep is going to the combination of the level of the frequency signal and the level of the noise at the same frequency. The distortion is going to vary at different frequencies so you would expect to see this reflected in your REW measurements but your frequency sweep doesn't show the different levels of harmonic distortion at different overtones and it is only measuring one frequency at a time. The harmonic distortion levels are low, usually less than 1%. If the level of the frequency signal is x dB,  and the distortion is at the same level then the combination will measure 3 dB higher than the level of the frequency signal on its own but if the level of the distortion is 12 dB down from the level of the frequency signal, the combined level will only be 0.2 dB higher than the level of the frequency signal. If the SPL if the distortion is 12 dB dow on the test tone, the level of the distortion l n the combined signal would be around 10% of the level of the test tone in the combined signal. We can hear differences in amps with THD levels below 1% and the difference those levels of distortion would make to a frequency sweep are going to be much less than 0.1 dB in level. That's why no professional measures harmonic distortion in an electronic component by measuring speaker output, especially with REW.  The resolution of the test just isn't fine enough to reveal differences that people can and do hear in harmonic distortion, especially when speakers also produce harmonic distortion at levels much higher than those produced by electronic components and there's no way of telling from measurements of speaker output how much of the distortion you measure, if you can get a reliable measurement, is due to the speaker and how much is due to each of the electronic components in the signal path.

I'm not saying that REW is no good or that the tests you did aren't accurate to within REW's specification. I'm saying that the tests you did aren't the appropriate tests to measure things related to digital or electronic signals. They're appropriate for speaker output and room acoustics and they aren't appropriate for measuring some other aspects of audio equipment performance.

David,

I don’t disagree with what you wrote, but it’s not quite what I was trying to do.  I’ll try and explain, I might not do a good job so work with me.
We already know that our amps measure very well, so well that they are already way below audible thresholds in all major areas.  You mention distortion, as I remember it the lowest amount of distortion that can be detected in listening tests with music is 0.1%, but there is some research that shows that 0.01% might be audible under very controlled conditions with test tones, which is still way more distortion than our amps have, so if distortion was made lower again it would not matter, as we would not be able to hear the change.  On the other hand lots of people (probably including me) like a bit of distortion from time to time, of the right sort, but if that had been added to ‘improve’ the sound it would probably be of the 0.5% or higher of 2nd order type, which my tests would have shown easily, distortion was 0.2% across most of the spectrum in my test.  So I was not looking for lower distortion, and I agree I could not measure a decrease in distortion, but rather checking that it was not higher.  Now I would have fallen of my chair if Devialet had added distortion, I was really looking for anything else that stood out, something that would easily change a listening test, like a change to frequency response or playback level.  They would have easily shown up in REW sweeps, but they were not changed at all, so much so I had to double check as the lines on the graph they were so close to each other, I don’t normally get results that repeatable.  Nothing was found be me, as far as I can tell there is no measurable change, with the exception of below 100hz, were my tests are not reliable as background noise and increasing speaker distortion intrudes.
So I don’t detect any cheating that some manufacturers have indulged in over the years, which is good.