Hifi Demos: Blind or Sighted?
One of my most frequently performed tasks, as the owner of a hifi shop, is to help people choose between different hifi components, or systems. In industry speak this is known as "The Dem". It's a relatively simple concept - the client listens to some music on one system, then a change is made, and they listen to it again, and express an opinion.
Sounds pretty straightforward. After all, it's how we make all kinds of selections - shoes, cars, sofas.
However, there's a vociferous element of "rationalists" who cry out that this is "unscientific", and insist that listening tests should be carried out blind.
This stems from a deep-seated belief that the hifi industry is out to trick the consumer - to lead them down a path of spending money on things they don't need. To be fair, there has always been an element of snake oil selling in the hifi industry. With products like $8,400 rubber speaker feet, "ambient field conditioners", and pebbles which involve "atomic mechanisms in the crystals" to control resonances, it's hardly surprising that there's an element of suspicion.
The standard response given by a hifi salesperson to a skeptical customer is simple: just listen - trust your ears!
But is it that simple? No. It's not. People are very easily primed to hear things that aren't there. They're very easily influenced. A few words by the salesperson can lead the customer to expect something, which they then hear - psychological bias 101.
I remember a demo I did for a client, in his home. He was using the on-board phono stage of his Marantz amplifier, with a very nice Funk Firm turntable, with an upgraded arm, and a top-of-the range moving magnet cartridge. In the context of his system, it was definitely time to consider improving the phono stage. I had a dedicated valve-based moving magnet phono stage ready to show him. While I set it up, he went out of the room for a few minutes. I was fiddling around at the back of the amplifier, getting the possible upgrade ready. He sat down. I cued up the record, and dropped the needle.
"Wow!", he said, "That is amazing! I never could have believed what a difference a phono stage would make!".
I hadn't switched it over yet.
My customer had set his heart on hearing a difference, had mentally assigned money to the project, and consequently was primed to like the upgrade.
Was I being dishonest or disingenuous? No. Indeed I pointed out to him that I hadn't made the change yet. He blushed a little. We did the demo, and my customer did feel like he could hear a difference, but decided on balance that his money would be better spent elsewhere.
So, given that people are easily led - either accidentally, or deliberately, how can we prevent people from being duped, or duping themselves?
The simple answer is - listen blind. Set up a demo such that two components or systems are compared, but without telling the listener which is which, and without telling them what to expect. Then if they hear a difference, or indeed if they don't, it's down to their own "unbiased" sensory perception. Better still, conduct a "double blind" - where neither the person doing the demo nor the person listening knows the difference between the systems or components.
So the argument goes. At a stroke we've disarmed the nefarious salesperson, and hapless biases of the consumer. A perfect solution.
However, in every hifi shop in the country, every day, the same "sighted" A/B testing takes place. It's extremely rare for a salesperson to initiate a 'blind' comparison, and if requested to by the customer, the response is some uncomfortable mumbling, or possibly downright refusal. Surely the salesperson has something to hide?
As usual, this doesn't tell the whole story. I am sure there's an element of insecurity involved. Without the patter, and the expectations, there is indeed a chance that a customer might be unable to tell any difference between a £10 cable and a £300 cable. And that's a lot of margin that's lost. It also calls into question the expertise and worldview of the manufacturers. So, to a degree, there's no surprise that the suggestion is met with reluctance, or even hostility.
But let's look at things from a different perspective.
Firstly, a foundational principle of a blind test is that the conditions be identical. If I swap a pair of speakers, of course there's going to be a difference. The speakers have different drivers, different crossovers, different impedance, and different efficiency. The sound pressure level will be radically different. The cabinet design may be suited to a different room positioning, height, type of stand, or distance from the listener. In order for it to be a truly "fair" blind test, we need to make sure, at the very minimum, that the sound pressure level is identical, and that we've set up the speakers in line with the manufacturer's guidelines. Of course doing this within sensible time constraints, in a shop demo room, or a client's home is next to impossible, yet alone when the variable change rapidly - it's not unusual to try three or four different setups before settling on one which pleases a client. So before we start, the idea that we could even conduct a true blind test is off the cards.
Next, is the recorded observation, carried out by Hifi Choice, that a level-matched "blind test" (in which the volume is set by the demonstrator) will yield different results if, with each change, the listener is permitted to set their own preferred volume level. In this example, those who favoured quieter listening conditions were drawn to systems with the flattest frequency response, and those who preferred higher volumes were attracted to those with elevated frequency response at around 1kHz. These listeners heard different differences at their preferred levels when contrasted with the level set by the demonstrator.
Another significant consideration is that there is a clearly identified cognitive issue at play here too. When the brain lacks any context or reference point, it is much much harder to discern differences. In fact, this results in listeners going one of two ways. The listener doesn't know what changed. They don't know how big the change was. They don't know the implications of the change - does it cost or same me money. They don't know whether the change has any aesthetic or logistical impact. All they know is that something changed. When the called upon to identify a difference, people tend either to exaggerate or suppress. differences.
They exaggerate because they want or expect to hear a difference, or don't want to seem as if they can't hear a difference when they believe there should be one. They suppress differences, perhaps reasoning that they don't have good enough hearing. Alternatively, because they are inherently skeptical, or have been primed by a skeptical hifi rationalist, they expect the differences to be marginal, or non-existent, and so they hear nothing. It's also possible even opt for self-protection, reasoning that even if they suspect there is a difference, they fear the consequence, err on the side of caution. It's plainly fallacious to believe that a blind test automatically removes cognitive bias. It doesn't. It just introduces different ones.
This lack of context reminds us that when we're we're buying shoes, or wine, or a car, or a hifi system, we're not conducting a scientific experiment. We're making a choice based on emotion, aesthetics, budget, and many other considerations. Even supposing we were able to set up a test in a truly scientific way, and even supposing it were possible to carry out the demo without either the customer or the salesperson knowing which was which, this is just not how we shop, as a consumer.
Indeed, consumers go to specialist retailers because they want to draw on the expertise of the retailer. If a customer comes to see me with a certain budget, and a certain setup, and a certain room, and a certain style of music, they're not coming to take part in a peer-reviewed study - they're coming for advice. I sold my first hifi system more than fifteen years ago. I've been to countless different homes, set up countless turntables, experimented with almost infinite different combinations of components, in different sorts of environments. I've owned all manner of different systems, tried out a mind-boggling number of cables, cartridges, tweaks, and mods. It's that expertise that a prospective customer is seeking. So of course, when they want to compare loudspeakers at up to £1000, they're going to ask me to describe the differences, and want to be guided to make a decision that is right for them.
Ultimately it boils down to trust and relationship, and that's something that takes time to build. That's why at Expressive Audio, we never hurry people. We don't hustle, and hard close. We understand that the process of selecting a hifi system takes time, and is deeply personal, and often slow-burn. That's why we frequently demo in our client's homes, and why we're often willing to lend equipment out. Because we know that we can't build a business and a brand on deception and influence.
The compromise I've reached is this. I can promise that every item in my shop (apart from those items which I've taken in on part-exchange) has been listened to and compared to competitors, at great length, with a range of music, at a range of volumes, in more than one kind of environment. Often those listening tests have, to the best of my ability, been conducted under "blind" conditions - especially cables. In my case, I'm trying as far as I can to get to the truth of how components sounds, and how they work interdependently. That's a good situation in which to try to introduce blind tests. But that's very different from the context of a customer. Effectively, I've done blind tests, so as to minimise the need for the customer to do blind tests.
I can also promise that I'll leave no stone unturned when it comes to trying out different combinations, and am well-known for maverick setups which work against all odds. I would much rather sell a cheaper system that works for the client, and helps them fall in love with their music, than go for the extra pounds in the till, knowing I wasn't honest.
I'm happy to demonstrate blind (as best I can), but that I find it's an artificial enterprise. I genuinely do believe in trusting one's ears, and I genuinely believe my customers are intelligent enough to understand that more expensive doesn't always mean better.
And ultimately, I believe that the business of sales - because after all, that's what all this is about - is a collaboration between interested parties, to solve a problem together, for the maximum benefit of both parties. It's an endeavour entered into in a spirit of friendship and trust, not suspicion and fear.
Stephen Nelson-Smith is founder and principal consultant at Expressive Audio, in Chichester, West Sussex. He's particularly interested in the psychology of listening, and music.