What Do Your Fans Want? (Part 2) - The Value of Testing

When Coca-Cola introduces a new product, they don't just unleash it on the world. They start small, testing things in focus groups, and eventually a few select markets, to make sure people actually want what they've got.

Sometimes there are problems...

For example, "New Coke" was very popular in blind taste tests, but rejected when marketed under the Coca-Cola name. And "The Pepsi Challenge" in the early 1980s, had to change its method of collecting data, when more people picked Coke.

When dealing something as music, tests aren't perfect. People like music for a number of reasons, one of which is that their friends like the same music. Still, don't let that stop you from "testing" different versions of the same song, different set lists, different stage banter, and different merchandise to get an idea of what works best. Some idea is better than no clue...

In the direct marketing business, it's not uncommon to run two different marketing campaigns for the same product, at the same time. For example, if the method of reaching people were postcards, 50% of the postcards mailed would be "Option A" and the other 50% would be "Option B."

Sometimes the results, when you're testing on a small level, can be subtle... Little things can mean a big difference when you "go big" though.

If you mail 10,000 postcards, 5000 of each option, and get 37 orders from "Option A" and 43 orders from "Option B", that might not seem like a big difference, but with a mailing of 100,000 postcards, assuming you get the same percentage of orders, you'll get an additional 60 orders from mailing "Option B." And mailing 1,000,000 postcards, which isn't uncommon, can mean a difference of 600 orders.

Same work and same cost of doing business, but much different results...

You never know if you can do better than you are now if you don't test, so this is something I encourage you to look at. And once you do find something that works better, I encourage you to test something else against that, to see if you can improve things yet again.

The above method of testing is the foundation of why the process is so powerful, but something I feel most musicians will relate to is the testing that you see a comic do when he's telling jokes. 

Let's face it...some things seem funny at times, but they're not funny at other times. And what you think is funny might not be funny to somebody else.

Do how does a comic find out if most people in the audience think something is funny?

1. He tells the joke every time he's on stage.

2. He asks, "Does it get consistent laughs?"

• If yes, he keep telling it.
• If no, he tweaks it until it does get consistent laughs...or he drops it, if it can't be improved.

It's a little less scientific, but can absolutely make a huge difference in the success you're having.

Every comic goes through this process. By the time you see a guy on HBO, he's tested everything, because a big gig like an HBO special isn't the time to find out if something doesn't work. You want to do that in the small clubs, with a limited audience, so not everybody will see you fail.

Same for musicians. Get out there, try new things, fail fast, and fail often, but keep it small. Keep the stuff that works, tweaking it to be as good as it came be, while slowly expanding it to larger and larger audiences. During smaller gigs, keep trying new stuff and really tweak the things that don't work, eventually dropping it, if you can't turn it around.

Need something to test? Try these...

• stage banter/stories
• electric vs. acoustic
• melodies
• song titles
• lyrics
• hooks
• instrumentation
• long intro vs. short intro
• solo vs. no solo
• harmonies
• extra verses
• tempos
• time signatures
• styles

Have fun!

Comments

[Phil Johnson]

Comedians spend a life time testing material. But you'd be surprised how many forget to do the same with their marketing and business stuff.

For me, a joke gets 3 tries before going back to the drawing board or being dumped. And now the parts of my songs do the same.

One other tip is not testing too many variables at once. If you change too many things, you don't know what caused the effect. In the comedy world, we make sure to sandwich new material in between proven material.

This is all more concrete if you're testing business stuff. Did it result in more sales/bookings/emails/etc? But with the musical stuff it can be a little trickier. Comics have a defined goal, the laugh. But besides the laugh, we're super tuned into the audience to try and gauge what they're feeling during the laugh. I have to tap into that idea with music audiences but without the laugh as a guide. Bottom line is always watch the eyes of the crowd to see how into whatever it is you're doing at the moment.

[David Hooper]

Thanks for adding this. Agree with you about not testing too many variables at once.

Comedy is great, because the feedback is instant. Something either works or it doesn't, with little gray area. Music isn't as easy to judge in that aspect, although I think you're right in that, in the end, the sales are a great measurement of success, since people vote with their wallets. The only issue with that is that you have other variables, such as day of week, day of month, pay cycles (1st week of month shows almost always sell more merch), etc.

But still...testing works, so do it!! :)

Will look into some quick and easy tests people can do to get feedback to make a better product. At one time, I though about doing a site for musicians, similar to hotornot.com, where they could post photos of themselves, album covers, or other items to get ratings. Doing a "survey" of your list in a similar fashion would be a good way to get this kind of info.

[David Hooper]

Forgot to add... http://en.wikipedia.org/wiki/Multivariate_testing has some great info on multivariant testing.

[empresas de consultoria]

Nice post David, i totally agree with you. In my opinion one of the best ways to acquire an interesting market share is to give clients an oportunity to test your product and become an evangelist of your brand, your product or your service.

[Music Marketing Chris]

A little test I like to run is for album titles...

Before the release I will send fans a choice of four free tracks and usually one of them will get downloaded much more than the others.

I call the album the title of the most popular track and Then I know it will appeal to the fans.

This is a really quality post David!

- chris