"Audiophile Grade" Ethernet Switches - The new generation

[David A]

If we're trying to find out whether the ER does something objectively "real", listening reports are not evidence unless they are carried out under controlled conditions and are repeatable to some pre-defined degree of statistical significance.  Otherwise they are insignificant (to that particular question) especially as most (all?) of them so far come from a group that is self-selected for a bias towards expecting a sonic improvement.

Isn't audio about personal listening experience?  Having evidence based experiments under controlled conditions and making statistical significant claims on audio improvements of any audio product belongs to the realm of a university research thesis. Listening to music is an emotional experience which I enjoy; definitely not a scientific experiment.

Yes, audio is about our personal listening experience but @thumb5 does have a point. In this case we're talking about whether or not we can trust what people say about whether or not they hear a difference and/or whether A sounds better than B. The listening tests that @thumb5 is concerned about actually start with people doing the sort of listening that you are talking about and then statistically analysing those comments. It isn't simply a "university research thesis" type of project, it's also the type of project that gets undertaken at times in the design of the gear we use when we listen to music and it has often proved beneficial to the enjoyment we get when listening to music so don't rush to write it off.

Having said that, @thumb5 is somewhat dismissive of the reports users make on the basis that they are reports from a self selected group of people with a "bias towards expecting a sonic improvement". There's nothing one can do at the moment about self selection when the only people who have had a real chance to hear the device have only been able to do so because they bought the one they're listening to and the company is still in the process of filling all of the orders received since they released the product. One can equally question whether there's selection bias involved in any group of individuals who participate in a listening test, especially if that test is conducted by someone who strongly believes or disbelieves in the product. I doubt we're ever going to see a listening test that he would be completely happy with, simply because that sort of test is extremely expensive to set up and conduct and it's hard to get access to the budget and facilities required for such a test unless there is a compelling reason to conduct the test. Self selection should raise a flag but it doesn't guarantee that the reports it proves are less reliable than other reports. 

As someone who once had to design and conduct a research project, admittedly a very basic and very small project, I can tell you that actually designing and conducting a test that will stand up to the standards required by professional scientific journals is not easy and most of the tests you see conducted by private individuals because of their own interest fall short of that standard, in part because of budget, in part because of shortcomings in facility, and in part because of failings in the design of the test procedure. It takes a lot more than passion and good intentions to design and conduct a professional quality test and then to correctly interpret the results and actually come up with a finding that is fully supported by the results. All too often there is a tendency to make claims based on inference which end up going beyond what the data actually supports. Even professional researchers with years of research experience get it wrong from time to time when they have access to the budget and facilities required as anyone who has read the peer reviewed professional journals and observed the traps that get identified when people start trying to replicate initial studies can confirm. Proving something to scientific standards is far from easy and it's never quick. It took decades to prove some of the predictions made by Einstein in his Theory of Relativity and there is a lot more scientific interest in proving things predicted by that theory than there is in proving whether or not the ER works as claimed. I would not recommend anyone hold their breath while waiting for truly compelling evidence, one way or the other, about the ER. All we can do is to take the reports we get, look at them critically, and make our own assessments on the plausibility of each report in the knowledge that all of the reports we currently have access to fall significantly short of the standard required for scientific proof that the ER does or doesn't do what is claimed.