DCAudioDIY.com

All the mounting screws and drivers were in the box sunday morning. Check in they fell in your car, I will check mine.

If you keep complaining about a missing screw or two, you will make the person who gave away stuff regret.

Just reporting what I have onhand.

Not complaining. I can buy screws.

Thank you for the stuff.

_________________
I reject your music, and substitute my own.

Tom, it's not that the test process was flawed. The process was fine. I think what Stuart is referring to is the limited difference of the components and the limitations of the system.

Charlie and I (okay, mostly Charlie) are working on a capacitor demo for Dave B that will demonstrate an audible difference in capacitors installed in one of Dave's amplifiers similar to what you heard with the digital wires last time we met there. At least one set of capacitors (same value of course) will be so different in sound "presentation", an A/B/X won't be needed to hear it, but we can do one.

Much like those digital wires, it surprised the heck out of me.

Getting a finding that is "significant" or not is easy if power (effect/sample sizes) not been properly accounted. Without knowing more about the test procedure, I can't comment but lots of "A/B tests" seem to fail in this area. Small effects, small number of observations ends with results that are not valid regardless of whether they turned out "significant" (or not). Are the methods followed in this test available anywhere for review?

I definitely did hear the differences in wire in the test at Charlie's. There are differences in components. The advantage of double blind testing is that if the differences are subtle the double blind method will give you great confidence if you get a positive result. If you get a negative result the only thing you can say is that with the test setup, components, music, and listeners in that particular test there was no difference. Changing the conditions may change the results. However, if you repeat the test for example with a set of wires with different components, listeners, and music and still do not get a significant result, you have to ask yourself if there really are differences.

It will be interesting to see the results of the capacitor tests with the amps if in fact the values of the caps are precisely matched. If there are differences what parameter of the caps caused it, eg. differences in ESR, changes with capacitance with applied voltage, dielectric absorption, leakage inductance, etc.

With the wire test at Charlie's I'm not surprised that at the frequencies feeding the DACs that audible differences could occur. Once you get into the MHz range the wires are important. With all the double blind testing I have done over the last 40 years or so with analog signal and speaker cables at audio frequencies neither I nor other test subjects have been able to tell differences. There are audio level cables that can make a differences. For example there have been some high capacitance speaker cables that have sent some poorly designed amps into destructive oscillations. Is that the fault of the cables or the amp? You pick.

The test conditions for the crossover cap test were as follows:

1. A set of speakers built by Stuart were used for the test that occurred in David Berning's listening room. Stuart can provide the details of the speakers.
2. The crossovers were constructed so that the capacitors under test could be swapped out in the same boards so only the caps changed.
3. No ABX box or other device was used to switch the caps in and out.
4. The choice of which caps (A or B set) would be used for each trial was determined by a random sequence generator that is on the web.
5. The values of the caps were verified so they were the same for both sets.
6. Before the blind trial, listening sessions where the cap in the speaker was known were conducted for both sets.
7. To make it a true double blind test, the person changing the caps (me) was not in the room during the listening tests so I could provide no visual or other clues as to which set I had installed.
8. With the listeners in another room I installed the set of caps determined by the random sheet and then left the room after telling them the test was ready to proceed.
9. The listeners controlled the music and length of the trial. When they were through they let me know it was time for the next test, called me back, and left the room while I either changed caps or left them the same as determined by the sheet.
10. To make sure there was no clue as to whether I changed caps by the length of time I took, in every case, I took the same time to remove the caps and installed either the same caps or the second set before calling them back so the time was the same and if anyone was listening to my actions they were the same.

At the end of the test the score sheets that were used by the listeners to determine which set of caps was used for each trial were collected and scored against the original random selection sheet from the web.

One of the legitimate complaints of the test was the necessity to leave the room between trials which interfered with their memory of the previous test. I offered to use my ABX control center which would allow instantaneous comparisons but there were some who did not want anything else in the test. BTW, I have done many sessions with listeners to determine if they could hear a difference when the ABX tester was in or out of the circuit and they could not. David Berning did not trust my tester so he built his own and achieved the same results. If anyone is interested in doing further tests either with or without the test box, let me know.

Note: Tom made his post before I wrote this. No disagreement with what Tom wrote. However, I'd rather retake my old calculus classes than participate in another of these tests.

When a test is performed and fails to demonstrate expected results, the logical conclusion is that the test is flawed, or that the assumption of differences was fallacious.

Several of us heard, previously, distinct differences in listening. Yep, cap values, SPL, and self-delusion could be at play. Trouble is, these differences have been heard time and time again by different observers in differing situations. One cannot choose between paint chips in dim light.

Several things contributed, IMHO to the lack of (95%) correct results in listening.

1) The old tweeters in my speakers, Dayton PT2C-8, have a horrible HF rolloff. I chose them from a manufacturer's data sheet, which they later revised without a model change. See attached FR curves. Downloaded 2015 and 2016. This certainly made it more difficult to discern the fine details we listen for. Let's face it, all that we are listening for is fine details. Any POS stereo from Best Buy presents most of the music.



2) The time delay between "changes of caps" was way too long. It had to be at least five minutes each time when we left the room.

3) The music selection was terrible. Eva Cassidy "Dancing Cheek to Cheek", the first cut on the disc. The engineer is clearly "fiddling with the knobs" as the performance progresses, further making it difficult to hear the minute details which make a difference for some of us.

4) Personally, I'd never participated in such a test before. I never suspected that the same cap would be presented to us five or six times in a row. From my standpoint, sitting and listening, after (five minute?) intervals where we'd left the room while the "cap change" was made, was never anticipated. I kept listening for differences and expected I should hear something. I was not listening so much as trying to hear a difference so I could check out a box. Again, IMHO, this can work when choosing between red and blue, but not fine shades of pink, walking away from the canvas for minutes at a time in an adversarial situation where someone is ardently trying to show you that you are wrong.

5)A higher resolution setup with proper source material without the anxiety would be a better environment in which to make a proper evaluation.

6) Honestly, this "testing" simply removes the joy out of the whole DIY hobby for me. That's just me. If this is what the hobby is to be, I want no part of it.

Carry on and just call me stupid and self-delusional. I can handle it. [You wouldn't be the first to level such criticisms at me!]

My point about effect size is that you had 3-4 listeners? Less than 10 trials? Perhaps you would need 100 listeners to reliably detect a difference? Or single listener would need to need hours and many trials? In a system they were familiar with? With so few observations, even using a random generated observation, standard error will be large. How about uncontrolled variables.....was each listener agnostic to the finding of the other listeners in real time during each evaluation period? Etc, etc etc..

I guess my point is that getting/calculating statically significance is easy but getting valid results from any experiment involving human participants is often difficult.

https://journals.plos.org/plosmedicine/ ... ed.0020124

One of the difficulties brought on by not using the ABX switch device as I mentioned before was the long time between trials. Although for the test to be valid I could not interact with the listeners at any time including between trials. It is probable that during the wait time the listeners were conversing with each other about the results of each trial which is a no -no. In any case, even with the music used which was not my choice and the other factors, the thing that was most apparent to me was that the claimed major differences in the caps were not present. Yes, there maybe more subtle differences that further testing could uncover but that would be for a later test using an ABX box so that the comparison times could be shorter and also with the box, you can go back and forth as many times as you want between the two samples before selecting X which is only known by the box.

I will try to dig up the original results but there were more than ten trials and I believe around six listeners. I have found that around 15 trials seems to be the sweet spot with fewer being less reliable and more tending to tire the listeners out which causes confusion. It is highly recommended that never more than 25 trials be conducted in one session.

As far as the number of times that a particular selection occurs in a row, that is entirely up to the random sequence generator. Just like in Vegas, you can roll the same number many times in a row. The purpose of the test is not to try to outguess the random sequence generator, but rather to determine audible differences in the devices under test.

I wholeheartedly agree with Stuart that a higher resolution set of speakers would be desirable to determine subtle differences. I offer the eggs for a future test which will work especially well as the passive crossovers are external.

As far a Stuarts complaint that this type of testing spoils the hobby, I can understand that if it were an every day affair it would certainly be obtrusive. It takes a lot of work to set up and participate in a valid test. This is not meant to be the focus of the hobby, but only another tool to occasionally aid in determining whether a particular component or device makes an audible difference. As with any other tool you can choose to use it or not. This is a hobby and if it does not float your boat, by all means don't do it.

As far as sample size, good luck with getting 100 people to participate. For a given set of trials, any thing over 10 becomes unwieldy. To get 100 would realistically require at least 10 different occasions and it is hard enough to get even one. Also with one set of listeners, if a statically valid positive result is achieved, you can go to the bank with the assumption that there is a difference. The only need to go further would be if you as an individual who was not part of the test wanted to run it to see if you could hear the difference. Remember if only one person gets a statistically positive result, there is a difference even if others cannot hear it. Positives are easy. Null conclusions are extremely difficult to prove. If anyone wants to continue this discussion we should start another thread so it does not get lost.