AMplifer A/B Blind testing- Time to hear for myself!

[claydo]

Ok.....c'mon capn! We're dying to know!

[captainobvious]

OK, at this point, I believe everyone has received and had the chance to view the docs (and they are probably anxious to discuss), so I'm going to go ahead and post up the information for review and comments.



Without further ado, below is the content. The first post will be the description of the events and the forms handed out for the different items throughout the day. My second post will be the Excel sheet with the percentages, etc.






2/9/2014 Amplifier Blind Testing for the mobile audio community

The event began with an informal and brief introduction and to explain what we’d be doing over the next few hours. Before any actual demoing was done, it was important to ensure that all of the amplifiers were set properly for equal output voltage. With two testers as assistants, we tested each amplifier individually and set the gains to +/- 0.03v. We played some music through the speakers and then each tester was assigned a random tester number and given an initial setup/sign off sheet.

After having inspected the setup and configuration of the devices being used to implement and sample in the tests, I agree that the tests will provide a fair and unbiased setting with respect to a “level playing field” for the amplifiers in the group.

□ Agree

□ Somewhat Agree

□ Disagree



Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______



After initial evaluation, I agree that the room and environment provides a fair and unbiased setting for the amplifiers to be tested in for this group.

□ Agree

□ Somewhat Agree

□ Disagree



Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____



Please use this space for any other comments regarding the testing conditions, environment or proctoring that may be of note, either positive or negative in content. Thank you.

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

All testers checked off “Agree” to the two questions and none had criticisms in the comments section regarding any other conditions relating to the testing environment.

Next, I started off with non-blind listening evaluations of each of the 8 amplifiers used in the tests. I told the testers which amplifier was being played and a little about it. The testers were given a sheet to provide their comments on each of the amps sampled.

Audio Amplifier Blind Comparison Testing: Initial Demoing Session NON BLIND

Date:_____________________________________________ __

Tester Number: ______________________________________

ARC AUDIO XDi804

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____

DIAMOND AUDIO D7054

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____

JL AUDIO HD900/5

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

PHOENIX GOLD ELITE.4

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

SOUNDSTREAM REF4.920

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

XTANT X604

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

ZAPCO Z-150.6

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

ZED AUDIO LEVAIATHAN

Remarks:__________________________________________ __________________________________________________ ______

In an effort to avoid “listener fatigue”, we took a break preceding each of the series of tests. Next up, I provided the testers with a new sheet and proceeded with blind listening evaluations on each of the 8 amplifiers. The testers were again asked to provide their comments on each of the amplifiers they sampled, this time not knowing which was being played.

Audio Amplifier Blind Comparison Testing: BLIND LISTENING EVALUATION

Date:_____________________________________________ __

Tester Number: ______________________________________

AMPLIFIER 1

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____

AMPLIFIER 2

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____

AMPLIFIER 3

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

AMPLIFIER 4

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

AMPLIFIER 5

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

AMPLIFIER 6

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

AMPLIFIER 7

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

AMPLIFIER 8

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

Following another break, I provided a new sheet for a different comparison. In this final series of evaluations, I conducted blind “AX” tests. These tests are designed to provide a sample (reference) of amplifier A, and then an unknown “X” sample. The X sample can be either A again, or a different amplifier. The method is performed as such: In a series of 10 samples, the reference amplifier A is announced and played with a 30 second clip, followed by the X amplifier announced and played with a 30 second clip, followed by A, then x and so forth until the answers of all 10 samples are recorded by the testers. The testers were asked to pick which amplifiers they wanted to use as the reference amplifier in each of the 6 groups of tests. I selected (unknown to the testers) which amplifier would be paired up as the “different” amplifier in the comparisons for each group.

Audio Amplifier Blind Comparison Testing: A/X Shootout

In these evaluations, we will use 2 amplifiers – an “A” and a second for comparison. The same 30-45 second clip will be played on amplifier A, then the unidentified amplifier X. Amplifier X will be either amplifier A or the second different amplifier for each Trial in a group. Testers will listen to the sample on A and then determine if amplifier “X” is a different amplifier or actually the same amplifier A being played. Select a choice of X being the SAME amplifier as A, or different in a series of 10 trials per group of amplifier.

TESTER NUMBER:

Is “X” the same as amplifier A or different? (circle your selection for each of the ten trials per group)

GROUP 1: GROUP2: GROUP3:

Trial1 SAME DIFF Trial1 SAME DIFF Trial1 SAME DIFF

Trial2 SAME DIFF Trial2 SAME DIFF Trial2 SAME DIFF

Trial3 SAME DIFF Trial3 SAME DIFF Trial3 SAME DIFF

Trial4 SAME DIFF Trial4 SAME DIFF Trial4 SAME DIFF

Trial5 SAME DIFF Trial5 SAME DIFF Trial5 SAME DIFF

Trial6 SAME DIFF Trial6 SAME DIFF Trial6 SAME DIFF

Trial7 SAME DIFF Trial7 SAME DIFF Trial7 SAME DIFF

Trial8 SAME DIFF Trial8 SAME DIFF Trial8 SAME DIFF

Trial9 SAME DIFF Trial9 SAME DIFF Trial9 SAME DIFF

Trial10 SAME DIFF Trial10 SAME DIFF Trial10 SAME DIFF

GROUP4: GROUP5: GROUP6:

Trial1 SAME DIFF Trial1 SAME DIFF Trial1 SAME DIFF

Trial2 SAME DIFF Trial2 SAME DIFF Trial2 SAME DIFF

Trial3 SAME DIFF Trial3 SAME DIFF Trial3 SAME DIFF

Trial4 SAME DIFF Trial4 SAME DIFF Trial4 SAME DIFF

Trial5 SAME DIFF Trial5 SAME DIFF Trial5 SAME DIFF

Trial6 SAME DIFF Trial6 SAME DIFF Trial6 SAME DIFF

Trial7 SAME DIFF Trial7 SAME DIFF Trial7 SAME DIFF

Trial8 SAME DIFF Trial8 SAME DIFF Trial8 SAME DIFF

Trial9 SAME DIFF Trial9 SAME DIFF Trial9 SAME DIFF

Trial10 SAME DIFF Trial10 SAME DIFF Trial10 SAME DIFF

In closing, the testers were given a post-testing exit review to get feedback on the testing procedures, equipment, environment and proctoring after having gone through the full testing gamut.

Audio Amplifier Blind A/B Comparison Testing

Post Testing Exit Review



Date:_____________________________________________ __

Tester Number: ______________________________________

After the testing, I agree that the setup and configuration of the devices being used to implement and sample in the tests did provide a fair and unbiased setting with respect to a “level playing field” for the amplifiers in the group.

□ Agree

□ Somewhat Agree

□ Disagree



Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______



After completion of the testing, I agree that the room and environment provided a fair and unbiased setting for the amplifiers to be tested in for this group.

□ Agree

□ Somewhat Agree

□ Disagree



Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ _____



Please use this space for any other comments regarding the testing conditions, environment or proctoring that may be of note, either positive or negative in content, following the testing. Your comments will help to provide a better experience for future events. Thank you for your time, contributions and participation.

Remarks:__________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______

__________________________________________________ __________________________________________________ ______



Thank you to all of the testers for their time and patience in proceeding through the tests and willingness to provide feedback. Thank you to all of the donators of money, equipment and time to make this testing event possible.

-SW

[captainobvious]

And the pdf of full results.... Once it will allow more than 19k....doh!


Big Al?

[claydo]

Hey al.....help this man out........capn, yer killing me!

I have to say tho, the format of the testing seems very thorough, and well thought out. I love the a/x testing, that experiment should prove very revealing as to the ability to seperate amps by nothing but sonics......something I don't believe is possible with reasonably well built amplifiers........still can't wait for the read!

[captainobvious]

What's your email address Clay? I'll send it to you

[claydo]

You have a pm capn!

[BigAl205]

I increased the max limit to 200MB

[claydo]

Ah.....an excellent read, you put a lot of work into this, and it shows! Thank you for sharing the results, and I loved seeing the opinions of the amps, brand known.....vs not. Its amazing what preconcieved notions can influence, showing the filter between our ears and our brains. Awesome test, and something you and your testers should be proud of successfully completing.

[captainobvious]

Data file now attached. Thanks Al !

[captainobvious]

Ah.....an excellent read, you put a lot of work into this, and it shows! Thank you for sharing the results, and I loved seeing the opinions of the amps, brand known.....vs not. Its amazing what preconcieved notions can influence, showing the filter between our ears and our brains. Awesome test, and something you and your testers should be proud of successfully completing.

Thanks Clay, I appreciate the positive feedback. It was a fun time and very enlightening.