Conversion Optimization is not a validation step
I told this to a manager that I had and he told me I was wrong, after my years of experience, that optimization is a validation step, that it was there to validate the ideas that he came up with.
To further exacerbate the problem, he was sure that we had to put out the best experience possible before we even tested anything.
The problem was, he didn’t know what the best experience was. He was missing the whole point of experimentation. It’s through experimentation that we can find out the most optimal experience, because nobody really knows until we run experiments the right way to find what works.
Here is an example, using the classic button test that everyone talks about.
Validation approach: “I have a green button; I think a blue button is better. Will you run a test?”
Yes, this can be run as a test, but you will learn only one thing from this approach. If green is better than blue.
Experimentation approach: “What are you trying to accomplish?” – we are trying to improve clicks on the button and add to cart.
“Well, if you think color matters, then we should really try other colors as well, like black, red, purple, yellow, etc.”
“But you are limiting yourself to color, what if button size matters, button location, button copy, button shape? These things might also influence add to cart”
“But why stop there, it could be the images you are using, the copy of the product description, the layout of the page, the type of traffic coming to the page, and more likely, it’s a combination of a lot of things. How do we know what things influence conversion the most? We must try and figure that out, so we know what levers to pull”
Yes, this is a simple example, but apply it to any validation step. People with this mindset will try to design the best thing in their mind, oftentimes changing too many variables, tell themselves that they have a great experience and then look to CRO to validate if they are right or not. A very myopic approach.
After that type of test, they have learned nothing and have no plan for iteration or going forward with what to do to really move the needle. If conversion optimization is used in this way, you will flail about, without challenging your opinions or assumptions, occasionally getting a win but not knowing why it won, and never really learning how to drive the needle deep into conversions
