Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Version: Firmware 13.1.3 + DOS 2.2.4 Date: 2019-02-12
#41
(14-Feb-2019, 19:13)Soniclife Wrote: I did before and after measurements today, I could measure no difference that was bigger than run to run variance. I did all the tests with Sam off.

I suspect SAM may have been tweaked a bit.  I have VERY SAM-sensitive speakers and am hearing something different—more controlled less flab that I would occasionally hear when at 100 percent.  The sound reminds me of when I was demoing the Synergistic Dev Spec cable with less of an overdriven sound at the low end.  There’s just...more.  Dev Air via Roon.  On the fence about the sound of RAAT but experienced my first full dropout in weeks using Dev Air earlier this morning over WiFi.
Reply
#42
(14-Feb-2019, 19:13)Soniclife Wrote: I did before and after measurements today, I could measure no difference that was bigger than run to run variance. I did all the tests with Sam off.

But what did you measure?

The thing you need to remember about measurements is that they tell you a lot about what you measure and they tell you nothing about what you don't measure. You can measure frequency response and with the same digital data going to the Devialet you shouldn't see a difference in frequency response because RAAT and AIR should both deliver identical digital data for the music or test signal You can do waterfall plots and RT60 measurements but those tests tell you about speaker behaviour and your room's acoustic behaviour and shouldn't change with Roon vs AIR.

If there is an audible difference it should show up on testing but it won't show up if you don't do tests for the thing which is causing the difference. For years the only measurement done for distortion by one magazine was a simple measurement of the overall percentage of THD and the reviewer used to say that there couldn't be an audible difference between 2 amps with identical THD measurements . He wasn't measuring the harmonic structure of the THD and he wasn't measuring IMD. Differences in those measurements explain why 2 amps with the same percentage of THD can sound different. People didn't start measuring jitter until a year or two after the first CD players hit the market and people used to look at reviews and see distortion measurements which didn't include jitter, see no difference in the measurements, say there was no difference between player A and player B so there couldn't be an audible difference. Jitter measurements proved there was a difference and it was audible.

If there is a difference between RAAT and AIR, it should show in measurements but you are going to have to measure something where the difference shows. If you don't measure the right thing, then you won't get a difference in measurements. 2 different CD players can measure identically for THD and IMD and sound different because of jitter. If you don't measure jitter you won't see a difference between those players but if you measure jitter you will. Two amps may both show 1% THD and sound different but if you don't measure the level of each harmonic in the THD signature you won't see differences in THD that make an audible difference. As I said at the start, measurements tell you a lot about what you measure but nothing about what you don't measure. Just because some measurements don't show a difference doesn't mean anything if you're not measuring the parameter that is different.
Roon Nucleus+, Devilalet Expert 140 Pro CI, Focal Sopra 2, PS Audio P12, Keces P8 LPS, Uptone Audio EtherREGEN with optical fibre link to my router, Shunyata Alpha NR and Sigma NR power cables, Shunyata Sigma ethernet cables, Shunyata Alpha V2 speaker cables, Grand Prix Audio Monaco rack, RealTRAPS acoustic treatment.

Brisbane, Qld, Australia
Reply
#43
...and of course, hearing a difference doesn't necessarily mean there is an objective, measurable-even-in-principle difference (e.g. in the air pressure arriving at the listener's ears) anyway.
Roon (Mac Mini), Wilson Benesch Full Circle, Expert 1000 Pro CI, Kaiser Chiara
Warwickshire, UK
Reply
#44
I did the standard rew sweeps, with the mic close to the bass mid driver to reduce room interaction. I then looked at all the measurements rew offers, and was amazed how close they all were, with no smoothing applied as well. So lots of measurements, but no jitter test, but jitter was solved years ago in good designs, so any change there would likely be tiny.

I would be surprised if there was an easily measurable change from a firmware that does not imply something was fixed or changed. But the implication above was something was measured, so it was worth looking into. Sam could be a whole other thing, I didn't test that.
Roon, Rega P9 + Dynavector XX2Mk2 > 440 Pro > Sonus Faber Guarneri Evolution
Reply
#45
(14-Feb-2019, 22:09)Soniclife Wrote: I did the standard rew sweeps, with the mic close to the bass mid driver to reduce room interaction.  I then looked at all the measurements rew offers, and was amazed how close they all were, with no smoothing applied as well.  So lots of measurements, but no jitter test, but jitter was solved years ago in good designs, so any change there would likely be tiny.  

I would be surprised if there was an easily measurable change from a firmware that does not imply something was fixed or changed. But the implication above was something was measured, so it was worth looking into.  Sam could be a whole other thing, I didn't test that.

REW is good for measuring speakers and speaker behaviour in rooms but it's not good for measuring electronic components. No professional measures an amplifier's frequency response or distortion characteristics by measuring output through a speaker. Electronic equipment is measured using different gear and different tests to those REW offers.

I'll give you an example of why REW won't work for testing electronic equipment. It's not relevant in the RAAT/AIR case but it may give you an idea of why you would not want to use REW to measure electronic components and the signals they process.

Let's say you have an amp and you want to measure distortion. Measure it in the usual way and you get figures for the distortion level for different harmonics for harmonic distortion. Convert those figures to decibel values. The output of the amp when you do your frequency sweep is going to the combination of the level of the frequency signal and the level of the noise at the same frequency. The distortion is going to vary at different frequencies so you would expect to see this reflected in your REW measurements but your frequency sweep doesn't show the different levels of harmonic distortion at different overtones and it is only measuring one frequency at a time. The harmonic distortion levels are low, usually less than 1%. If the level of the frequency signal is x dB,  and the distortion is at the same level then the combination will measure 3 dB higher than the level of the frequency signal on its own but if the level of the distortion is 12 dB down from the level of the frequency signal, the combined level will only be 0.2 dB higher than the level of the frequency signal. If the SPL if the distortion is 12 dB dow on the test tone, the level of the distortion l n the combined signal would be around 10% of the level of the test tone in the combined signal. We can hear differences in amps with THD levels below 1% and the difference those levels of distortion would make to a frequency sweep are going to be much less than 0.1 dB in level. That's why no professional measures harmonic distortion in an electronic component by measuring speaker output, especially with REW.  The resolution of the test just isn't fine enough to reveal differences that people can and do hear in harmonic distortion, especially when speakers also produce harmonic distortion at levels much higher than those produced by electronic components and there's no way of telling from measurements of speaker output how much of the distortion you measure, if you can get a reliable measurement, is due to the speaker and how much is due to each of the electronic components in the signal path.

I'm not saying that REW is no good or that the tests you did aren't accurate to within REW's specification. I'm saying that the tests you did aren't the appropriate tests to measure things related to digital or electronic signals. They're appropriate for speaker output and room acoustics and they aren't appropriate for measuring some other aspects of audio equipment performance.
Roon Nucleus+, Devilalet Expert 140 Pro CI, Focal Sopra 2, PS Audio P12, Keces P8 LPS, Uptone Audio EtherREGEN with optical fibre link to my router, Shunyata Alpha NR and Sigma NR power cables, Shunyata Sigma ethernet cables, Shunyata Alpha V2 speaker cables, Grand Prix Audio Monaco rack, RealTRAPS acoustic treatment.

Brisbane, Qld, Australia
Reply
#46
(15-Feb-2019, 00:57)David A Wrote:
(14-Feb-2019, 22:09)Soniclife Wrote: I did the standard rew sweeps, with the mic close to the bass mid driver to reduce room interaction.  I then looked at all the measurements rew offers, and was amazed how close they all were, with no smoothing applied as well.  So lots of measurements, but no jitter test, but jitter was solved years ago in good designs, so any change there would likely be tiny.  

I would be surprised if there was an easily measurable change from a firmware that does not imply something was fixed or changed. But the implication above was something was measured, so it was worth looking into.  Sam could be a whole other thing, I didn't test that.

REW is good for measuring speakers and speaker behaviour in rooms but it's not good for measuring electronic components. No professional measures an amplifier's frequency response or distortion characteristics by measuring output through a speaker. Electronic equipment is measured using different gear and different tests to those REW offers.

I'll give you an example of why REW won't work for testing electronic equipment. It's not relevant in the RAAT/AIR case but it may give you an idea of why you would not want to use REW to measure electronic components and the signals they process.

Let's say you have an amp and you want to measure distortion. Measure it in the usual way and you get figures for the distortion level for different harmonics for harmonic distortion. Convert those figures to decibel values. The output of the amp when you do your frequency sweep is going to the combination of the level of the frequency signal and the level of the noise at the same frequency. The distortion is going to vary at different frequencies so you would expect to see this reflected in your REW measurements but your frequency sweep doesn't show the different levels of harmonic distortion at different overtones and it is only measuring one frequency at a time. The harmonic distortion levels are low, usually less than 1%. If the level of the frequency signal is x dB,  and the distortion is at the same level then the combination will measure 3 dB higher than the level of the frequency signal on its own but if the level of the distortion is 12 dB down from the level of the frequency signal, the combined level will only be 0.2 dB higher than the level of the frequency signal. If the SPL if the distortion is 12 dB dow on the test tone, the level of the distortion l n the combined signal would be around 10% of the level of the test tone in the combined signal. We can hear differences in amps with THD levels below 1% and the difference those levels of distortion would make to a frequency sweep are going to be much less than 0.1 dB in level. That's why no professional measures harmonic distortion in an electronic component by measuring speaker output, especially with REW.  The resolution of the test just isn't fine enough to reveal differences that people can and do hear in harmonic distortion, especially when speakers also produce harmonic distortion at levels much higher than those produced by electronic components and there's no way of telling from measurements of speaker output how much of the distortion you measure, if you can get a reliable measurement, is due to the speaker and how much is due to each of the electronic components in the signal path.

I'm not saying that REW is no good or that the tests you did aren't accurate to within REW's specification. I'm saying that the tests you did aren't the appropriate tests to measure things related to digital or electronic signals. They're appropriate for speaker output and room acoustics and they aren't appropriate for measuring some other aspects of audio equipment performance.
David,

I don’t disagree with what you wrote, but it’s not quite what I was trying to do.  I’ll try and explain, I might not do a good job so work with me.
We already know that our amps measure very well, so well that they are already way below audible thresholds in all major areas.  You mention distortion, as I remember it the lowest amount of distortion that can be detected in listening tests with music is 0.1%, but there is some research that shows that 0.01% might be audible under very controlled conditions with test tones, which is still way more distortion than our amps have, so if distortion was made lower again it would not matter, as we would not be able to hear the change.  On the other hand lots of people (probably including me) like a bit of distortion from time to time, of the right sort, but if that had been added to ‘improve’ the sound it would probably be of the 0.5% or higher of 2nd order type, which my tests would have shown easily, distortion was 0.2% across most of the spectrum in my test.  So I was not looking for lower distortion, and I agree I could not measure a decrease in distortion, but rather checking that it was not higher.  Now I would have fallen of my chair if Devialet had added distortion, I was really looking for anything else that stood out, something that would easily change a listening test, like a change to frequency response or playback level.  They would have easily shown up in REW sweeps, but they were not changed at all, so much so I had to double check as the lines on the graph they were so close to each other, I don’t normally get results that repeatable.  Nothing was found be me, as far as I can tell there is no measurable change, with the exception of below 100hz, were my tests are not reliable as background noise and increasing speaker distortion intrudes.
So I don’t detect any cheating that some manufacturers have indulged in over the years, which is good.
Roon, Rega P9 + Dynavector XX2Mk2 > 440 Pro > Sonus Faber Guarneri Evolution
Reply
#47
(15-Feb-2019, 16:30)Soniclife Wrote: David,

I don’t disagree with what you wrote, but it’s not quite what I was trying to do.  I’ll try and explain, I might not do a good job so work with me.
We already know that our amps measure very well, so well that they are already way below audible thresholds in all major areas.  You mention distortion, as I remember it the lowest amount of distortion that can be detected in listening tests with music is 0.1%, but there is some research that shows that 0.01% might be audible under very controlled conditions with test tones, which is still way more distortion than our amps have, so if distortion was made lower again it would not matter, as we would not be able to hear the change.  On the other hand lots of people (probably including me) like a bit of distortion from time to time, of the right sort, but if that had been added to ‘improve’ the sound it would probably be of the 0.5% or higher of 2nd order type, which my tests would have shown easily, distortion was 0.2% across most of the spectrum in my test.  So I was not looking for lower distortion, and I agree I could not measure a decrease in distortion, but rather checking that it was not higher.  Now I would have fallen of my chair if Devialet had added distortion, I was really looking for anything else that stood out, something that would easily change a listening test, like a change to frequency response or playback level.  They would have easily shown up in REW sweeps, but they were not changed at all, so much so I had to double check as the lines on the graph they were so close to each other, I don’t normally get results that repeatable.  Nothing was found be me, as far as I can tell there is no measurable change, with the exception of below 100hz, were my tests are not reliable as background noise and increasing speaker distortion intrudes.
So I don’t detect any cheating that some manufacturers have indulged in over the years, which is good.

My point was that although I'm hearing a difference, it's not a difference I would have expected to show up on any test done by measuring speaker output.

I think the things which would need to be tested are DAC input vs output with both RAAT and AIR, and also any differences between signal output from Roon itself to signal received by the Devialet apart from the musical data in )s and 1s. The music content seems identical, frequencies and loudness levels aren't changing, so things like packet error rates, background noise levels on the connection, timing differences in how the packets are transferred and so on are the kinds of things I'd be looking at there but then there's also the issue of showing how such changes can make a difference to the audible output.

If RAAT makes a difference, and I believe it does, proving that with measurements may not be all that easy. We still can't measure everything that's out there in the world, and we don't always measure everything we could measure but rather just the things we think will be most useful.  We may need new tests to reveal a difference in this case (audio history is littered with the development of new tests to measure things which make an audible difference but couldn't be measured at the time) or we may just need to find the right test out of the many tests we have available including those not usually used in equipment reviews.
Roon Nucleus+, Devilalet Expert 140 Pro CI, Focal Sopra 2, PS Audio P12, Keces P8 LPS, Uptone Audio EtherREGEN with optical fibre link to my router, Shunyata Alpha NR and Sigma NR power cables, Shunyata Sigma ethernet cables, Shunyata Alpha V2 speaker cables, Grand Prix Audio Monaco rack, RealTRAPS acoustic treatment.

Brisbane, Qld, Australia
Reply
#48
I am now firmly in the AIR camp. I did some comparisons between Roon and AIR and every time using AIR I experience a better sense of rhythmic drive and musicality. It pulls me into the music, where Roon leaves me a bit cold. On flight of the cosmic hippo by Belle Flack,  the bass guitar had  more realism and the presentation more juiciness. The soundstage is wider on Roon and a bit higher, but deeper on AIR . I hear more fine details on Roon, but in the end for me, AIR sounds more musical, a better sense of music making. The great thing is we have choice, 2 presentations to select from, pick your poisen ?
220 Pro/CI, Well Tempered Versalex/Kiseki PurpleHeart, QNAP HS 251+ NAS/Roon, Furutech e-TP86 power distributor, Audoquest NRG10 Power Cable, Furutech Flux-50 filter, Furutect Powerflux Power Cable, Furutech Speakerflux Speaker Cables, Sonus Faber Stradivari. 
Reply
#49
(15-Feb-2019, 23:14)David A Wrote: If RAAT makes a difference, and I believe it does, proving that with measurements may not be all that easy.

If it makes an audible difference it will show up in a controlled listening test, and we can go from belief in a difference to knowledge of a difference.  Then it gets really interesting.  Did you say somewhere you are in an audio club, if so getting their help on the double blind part will make things a lot easier.

Putting myself through blind testing was one of the most interesting things I done in audio, and I encourage everyone to have a try at something like flac vs mp3 using a software tool.  I found it a fascinating process being presented with no other input than my ears, and having to trust them alone.
Roon, Rega P9 + Dynavector XX2Mk2 > 440 Pro > Sonus Faber Guarneri Evolution
Reply
#50
(16-Feb-2019, 12:40)Soniclife Wrote:
(15-Feb-2019, 23:14)David A Wrote: If RAAT makes a difference, and I believe it does, proving that with measurements may not be all that easy.

If it makes an audible difference it will show up in a controlled listening test, and we can go from belief in a difference to knowledge of a difference.  Then it gets really interesting.  Did you say somewhere you are in an audio club, if so getting their help on the double blind part will make things a lot easier.

Putting myself through blind testing was one of the most interesting things I done in audio, and I encourage everyone to have a try at something like flac vs mp3 using a software tool.  I found it a fascinating process being presented with no other input than my ears, and having to trust them alone.

How about performing a null test to compare Roon AIR with RAAT?

http://www.galaxyclassics.com/index.php/...9-nulltest

https://www.youtube.com/watch?v=52kbgFAuu6k

Guillaume
Industry disclosure: UK distributor for Shunyata Research

220 PRO, totaldac d1 server with additional external power supply, totaldac d1-seven, Echole PSU for Totaldac, Wilson Audio Sasha 2, Shunyata Research cables, Shunyata Hydra Alpha A10 + DPC-6 v3, Various Entreq ground boxes and cables, Entreq Athena level 3 rack, 2 X SOtM sNH-10G with sCLK-EX + 10MHz Master Clock input + sPS-500 PSU, i5 sonicTransporter w/ 1TB SSD

UK
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)