How to Turn Your Failed Tests into Winners!

Perhaps one of the most frequent questions I get asked when presenting testing hypotheses and ideas to clients is "but... what if it fails?"

It never fails to amaze me how many people have that concern. But it is a valid one, nonetheless, and I've learned to handle it with both logic and empathy.

Why is it that people are so afraid of testing? Is it the knowledge that what we are currently doing may be wrong? Or is it the fear that change may not always have a positive outcome?

If it Ain't Broke...

The first step toward being open to the possibility of improvement and optimization is letting go of our "if it ain't broke" mentality. This overpowering fear of messing with something that's already working is, well, paralyzing!

The only way to overcome this is by shifting our focus from wanting to fix something to simply wanting progress.

Fixing something implies that something is wrong. It surrounds the process with boundaries. It defines steps, timeframes, a start, and an end.

Progress, on the other hand, is continuous. It doesn't have to start with a problem. It doesn't have to end either.

There is no "Failed" Test

Change, even as simple as a minor tweak on your website, can be terrifying. It's even more terrifying when you think it can cost you money (or even your job!).

You will get resistance. If not from yourself, it may be from your colleagues or your boss.

But you know better. You know that the cost of not testing is much higher than the knowledge you gain from a test that didn't prove your hypothesis correct.

Notice how I didn't call it a failed test? Testing is not about what's right and what's wrong. It's about what we learn.

There is no "Winning" Test Either

Really, there isn't. To stop at a winning test is a form of complacency.

A winning test is only a "winner" in a sense that it outperformed its current counterpart in predefined metric(s). But is there an even better variation than the winner? Absolutely! There always is.

Your job is to continuously test and optimize, to find a variation better than your best one.

It's OK to Fail. Really

If working with Silicon Valley startups has taught me anything, it's that failure is completely acceptable. Try. Fail. Run out of Money. Do something else... repeat!

While I have my doubts that Samuel Beckett was thinking about online testing and optimization when he said this, it's still very much applicable:

Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.

So remember that great testing results can sometimes be a result of a series of bad or "failed" tests. Embrace test failures as a necessary part of the process. Only then will you have the courage to try, try again, and succeed.