Welcome to the Ecommerce Marketing Podcast everyone. I am your host, Arlen Robinson. And today we have a very special guest, Johann Van Tonder, who has been running online controlled experiments like a/b-tests for over 10 years. He is co-author of E-commerce Website Optimization. The second edition, a revised and completely updated version, has recently been published by Kogan Page. 

Welcome to the podcast, Johan.

Thank you, Arlen.

All right. Awesome. Well, I’m so excited to talk to you today and have you on the podcast today. We’re going to be diving deep into a subject matter that you’re definitely an expert on. As far as optimization. In today’s response to ecommerce websites, we’re specifically going to be talking about conversion optimization or conversion rate optimization and some helpful strategies that can get some ecommerce businesses thinking more in lines of what they can do to increase their conversions. But before we do get into all of that, why don’t you tell us a little bit more about your background and specifically how you got into what you’re doing today.

I really stumbled into this unexpectedly and unplanned, and it happened because I was clueless. I didn’t know what I was doing, and this was the mid 2000s. So to take you back to that time quickly, it was Google Analytics had just been launched today. We take it for granted, but that’s what we’re talking about here. The dust from the dot com crash was settling around us, so it’s that kind of environment that we’re talking about. Now I find myself working at a global Internet investment firm, and we were doing some exciting things probably in spaghetti against the roof effectively and seeing what sticks.

And as you can imagine, not a lot of it’s stuck. And we made some expensive mistakes and hopefully learned some expensive lessons. But I then ended up running a unit and a small portfolio of some of these strands of spaghetti that was sort of barely holding on to the roof. So businesses that weren’t really going anywhere, and my brief was fix them, sell them or kill them. And in trying to fix them, I tried what I thought was the obvious way first. And that was using the frameworks and models that I’ve been taught in business school.

Well, the punchline is that didn’t work. I surrounded myself with a team of engineers, constantly hiring more engineering power, thinking that we can engineer a way out of it. Right. So we just build more stuff and fix stuff that we think should be fixed that didn’t work either. I found that all the things we were building, all the things we’re fixing, it just didn’t land very well. And so I went to the opposite side of the spectrum and out of desperation, literally started talking to customers, our customers, of these various businesses.

And as I say, Ga, Google Analytics has just launched. So we had that perspective. And then we had the qualitative perspective.


Suddenly things started going in the right direction. And I discovered there wasn’t a name for it at the time. But the principles that I was using at the time and the principles that I found to be successful and work really well is what today you would call Sierra got you.

Well, thank you for sharing that. And I do remember those days very well. A lot of our weeks knows that I’ve been around the block quite a bit, quite a long time, actually, rather over 20 years. So I do remember those early 2000s. And I do recall when Google Analytics was first launched, they really were almost kind of the only kids on the block.


I think you could say when it comes to analytics, really, there wasn’t too many other platforms around. I think at the time that’s right.

There were a few others, and we were using one. But it wasn’t up to the standard that Ga set in the market. And so immediately, Ga was just adopted widely because it actually worked, and it actually gave you the insights that you wanted.

Yeah, definitely. But things have definitely changed since then. Now there are a lot of players in the space. There’s a lot of analytics tools and a lot of ways to slice and dice your data, so to speak, which is good for businesses and ecommerce businesses.

Especially. The trick is and this is what we deal with every day is knowing which rabbit holes to allow yourself to be sucked into. Because the problem one of the problems with this big data that you’re talking about is there are so many of these rabbit holes, and you’ve got to think about where the effort and reward equations starts not making sense anymore.

Very true.


Definitely more rabbit holes. Because of the amount of data that is available. You do have to be very selective as far as which rabbit holes you go down and which pieces of data that you really are going to dig deep into and then make data based decisions on it that could actually change your business. So you do have to be careful.


It’s good that the data is out there and there’s these two out there, but at the same time, if you’re not strategic about it, you could definitely fall into some traps and pitfalls for sure. Well, of course, as I alluded at the beginning of the episode, we’re going to be talking about conversion rate optimization as it relates to ecommerce businesses. And I think it’s a big deal these days because what I constantly see is so many businesses these days have such a focus on driving the traffic to their website and doubling down on all types of efforts paid advertising via organic search, just driving as much traffic as possible to the site.

And then a lot of times the optimization of this traffic that they’re getting and the conversion from this traffic is almost like kind of a second thought. They’re always just focused on driving the traffic. So that’s why I think it’s definitely a big deal, and it’s a good time to focus on it. The question then becomes, if a business is doing this, they’re driving all this traffic. They haven’t really done too much to optimize it. Where do they really begin when it comes to conversion rate optimization?

I think most people when you’re asking this question would start talking about the operational stuff. Right. So looking at the data, which we’ll get to in a moment, I’m sure. And trying to discover opportunities. I think that’s not the starting point. The starting point is a step before that, which is to really understand why you’re doing this and what you’re doing. And even a step before that is to understand the needs of your senior stakeholders, understand what problems are they trying to solve and then helping them to solve those problems and address their needs with the tools and methodologies that Sierra give you.

And the reason I say that is that if you don’t have that foundation in place, then it doesn’t matter where you start or what you end up doing. It’s still born more likely than not, or it will fizzle out. You need that top management buy in, you need the support. You need the mindset to be right for this to flourish. So I’d say before we talk about the sexy stuff, that’s really the place to start.

Yeah, it definitely makes sense, because if you don’t have the proper buy in, like you said, from the higher levels there, then anything that you’re really kind of doing it really may not have. If you don’t really get the buy in from them, you can do some small piecemeal things to try to adjust things. But if the whole vision isn’t adopted or agreed upon by everybody on the whole team and on the executive side, then this kind of doesn’t make sense. You have to look at it as a whole and figure out where to go.

And this is all. And this is lessons from the trenches. This is dear lessons straight from the trenches. I’ve seen this too many times and I think it’s kind of lost over at your own peril. I think it’s an important piece.

Yeah, definitely. Now let’s assume that at this point then the business kind of has to buy in has to go ahead the green light to go forward with the plan for this. Now, is there a specific methodology to follow? We’re looking to improve the conversion rate, and if so, how they go about doing it, what tools and resources are required for it?



So this is the sexy stuff. This is the real answer that people want when they ask that previous question. So yes, there are many methodologies and frameworks, and in fact, in our book we present one such framework, but I think most of them if you look at it breaks down into the following. Firstly, looking at data and you’re looking at data, not for the sake of it, but you’re looking to find actionable insights. And what I mean by actionable insights is its observations, which leads to something you can do and in the context of experts.

So you can do something about that in order to improve a KPI conversion rate or something else in the context of CRO. What we’re talking about doing that something is usually asking a what if question. So that’s step two, we’ve looked at the data and now we ask a what if question and it’s really looking at your observations, trying to uncover opportunities, maybe areas to fix. All those are the usually very short list, and a lot of people obsess over that. But that’s not really where the money is then asking what if questions and that leads hypotheses, which really is I think it’s an overused word today, but what it means is we believe something to be true based on what we’ve seen.

And now we’re going to test it. We are going to check whether that something is true and that’s where testing comes in. And that’s the third and last step. Well, the last step really is repeat. So it becomes a virtual cycle and it will measure learn, which is lean startup. There are many manifestations of this, but that’s really what you’re doing. Understand the world of a customer, understand the opportunities, ask what if and for my processes, and then test them and don’t stop.

All right. Continually rinse and repeat for sure. Now, if we start to look at this from kind of a practical view of going through that particular methodology, because the average ecommerce website has typical structure. A lot of them have the same type of structure where, of course, they shop area, where they’ll have their shopping cart or whatever the product categories. They’ll have some other information about the business, maybe about the founder, about their mission, how they got started, and then some other resources that really will talk about things that supplement their customers.

Let’s say post purchase, maybe some informational documents things like that. So most of these websites have a typical structure. So if we’re looking at a typical structure like that, what do you really need to be mindful of as far as when you eyes first, when a customer’s eyes potential customers eyes first land on the website, what are some things that you need to do to really pull them in and then increase their chances of converting at some point, even if not right then and there. But at some point, what are the things that you need to look at to do?

I like the way you phrase this question, Alan, because you talk about the customers and thinking at it from the customer perspective. And I need to qualify something that two minutes ago, I said, you need to before you do any of this, you need to understand the perspective of your stakeholders, but that’s just to get started. That’s to get the bedrock, the foundation in place. What we’re really doing here is being customer centric. And we’re getting into the mind of the customer, getting into the mind of a user, we understanding their world as best we can.

And in your preference there, you spoke about the structure of a website and you write typical ecommerce structures exactly as you painted it. But you can turn that on its head so you can construct different models of looking at a website. So the one way is that and look at it in terms of a hierarchy. And you are on the page. And so on. The way we look at it is through a different lens, and it’s through the lens of a sales conversation. So why is that customer or that user the prospect on your site?

Presumably, they’re there to buy something from you. They have a need while they’re not there to buy something from you, they have a need and they’re there to meet that need. This is basic marketing stuff right now. If they don’t have that need, they’re in the wrong place anyway, we don’t worry about them. They’re not our target market. We are happy to lose them. They’ll click the back button, go back to Google, and that’s no concern to us. First thing is, you’ve got to understand your conversion potential because I think many folks fall into this job looking at Ga and obsessing over the low conversion numbers.

And when you start digging into it, and you start contextualizing that by, for example, looking at the real conversion potential, it’s maybe 25% or less in some cases of that 100% power that you’re looking at. So that’s the first thing, then the second thing is, as you said, understanding what goes on in the mind of the customer. And if you understand that this has nothing to do with fixing stuff on the page, this has to do with fixing stuff in the mind. It has to do with fine tuning that sales conversation.

As I said, they’re there because they have a need. How do we persuade them that our solution is the best for them if indeed it is. And that’s really the model, the lens through which I prefer to look at the site. And I’d say it’s slightly harder work, but it’s a lot more rewarding. And the reason I say it’s slightly harder work is because if you look at it as a flat structure, a two dimensional series of pages with UI stitched together, then what you’re doing is you’re going from a company centric perspective and you’re going, what can I change?

You’re asking the wrong question, how can I move stuff around this page? And the right question is what goes wrong in the mind of the prospect? And how do I fix that?

Yeah, that is some great advice. And I’ve never quite heard somebody put it that way. I’ve had, of course, other people on and talk about conversion rate optimization a lot of times. Like you said, the first thought is the notion of moving things around, restructuring the order of what people see when they first come to the site, coming up with some new landing pages, the whole nine getting people into a funnel and all of that. But none of that really matters. Like you said, unless you really can get into the mind of the customer and figure out what is really going to resonate for them and answer the questions that they’ve come to the site for.

So I think it really like you said, really, it comes down to the messaging and that conversation that you’re having with the customer and having that right conversation and figuring out really what that conversation needs to be, because like you say.

If I can jump quickly and there’s a quote that comes to mind here from Steve Blank and back to my starting story in the mid 2000 Steve Blanks book, Four Steps to Epiphany, had a huge influence on me, and it gave me the early answers of that different process to try, as opposed to the business school models. But he talks about the answers are not in the building. The answers are not in your boardroom, and we often think and this is the default. In many organizations. We think we have the answer.

We think we know our customers because we’ve seen emails from them because we’ve spoken to them on the phone because we are in touch with them. We think we know our customers. But the only way to get to know your customers is through a structured approach using specific research methodologies and what Steve Blank calls customer development and that realm of things and stepping under the skin of the customer getting into the mind, I’ll give you a quick example of what that looks like. So Henry Ford, for example, everyone knows the story.

He said, if I ask my users what they want, they’d ask for a faster horse, and we know that wasn’t the proper answer. But that’s the wrong question to ask your customers. The right question to ask is about their problems is about their needs. Consumers, customers. Users are able to articulate what problems they have, how they currently solve them and what goes wrong in trying to solve those problems. And what’s the upside that they’re looking for? And that’s what you’re looking for. If you ask customers about that, you start mining the gold.


And I think once you start asking those questions based on their response, then like you said, the jewels will be uncovered, and then you’ll understand what’s the conversations I need to have, what needs to be presented. What’s the proper messaging for sure.

This comes back to the earlier framework. So once you’ve spoken to the customers and you’ve looked at other data sources and you’ve mapped them onto each other, so you kind of build up a jigsaw. You’ve built up a picture informed by many different data sources. Then you ask the body of question. So you don’t bet the farm on your idea. You test it. And the reason you test it is that more often than not, you’re going to be wrong. The best themes in the world are wrong.

Seven to nine out of ten times. Why would you be any different? So you’ve got to test these things.

It is all about testing.

Definitely. Yeah.

I kind of heard an acronym before when it comes to optimizing sites and actually doing the testing. It’s A-B-T-I don’t know if you heard that always be testing. That’s right. Because if you don’t do that, then like you said, you’ll never really know. You could always like you said in your early example, the other companies you’re with, you can throw some spaghetti at the wall and see if it sticks if you keep doing that. But yeah, the bottom line is you have to do that. But yeah, you have to analyze the results.

What is sticking? What’s not and then be able to quickly pivot after analyzing. Because otherwise, if you’re just throwing stuff out there and they’re not really looking at the results and then are able to effectively and quickly pivot rather than leaving a certain strategy out there, then you’re not going to win. So yeah, I was a huge advocate of kind of getting some kind of quick wins. I guess if you will, based on your experience, what do you feel as in standard do’s and don’ts when it comes to conversion rate optimization, if there’s any.

Yes. I mean a lot. So I’ll give you the don’ts and then immediately the dos as well.


So the first don’t, I think, is and I see this a lot is working down a menu of Gimmicks, a menu of tricks. So we read a blog post and it worked for some other site and let’s try it because you already know what to do is we’ve already spoken about the do. Your site is different. Your site is unique, your customer base is unique and you’ve got unique context. So you could understand your world and don’t obsess forget about the competitors, because that’s another variation of this problem is that people copy the competitors.

And there’s a great saying in the industry, don’t copy the competitors. They don’t know what they’re doing either. You might be copying losing tests. So that’s the first one. The second one, I think, is obsessing over the win rate. So the number of wins out of all the tests you run, right. So five out of ten would give you a 50% win rate and so on. And that’s bad because Firstly, you’re going to be wrong more than you’re. Right. So you’re benchmarking yourself into a negative situation, you can have sleepless night.

Secondly, there is value in losses. Right. So Firstly, you can quantify that if you test something that gives you a loss that’s mitigating risk, it’s something you would have put live that would have hurt the sales on your site. You’ve managed to mitigate that risk so that you can quantify it in that way, there’s another value to it, which is learning from it. As you said earlier, some of the biggest wins I have seen have come from losing tests. And here’s the thing. There is no way you would have come upon that insight that led to the win.

Rather than running a test, it is the only way that insight would have been revealed to you. So running negative tests as part of the process is important. Part of the process to do is follow the process. Don’t obsess over the win rate, obsess over the number of tests you get out the door and speeding things up. The last one. There’s a lot that I can think of here, but the last one is if it’s perfect, then you’re launching too late. Validating the hypothesis. You’re not engineering the final solution.

That’s not what this is about. What you’re doing is you’re validating the idea, and if you get a green tick next to it, you pass it over to your devs and they’ll develop the final solution and sometimes to build on to that sometimes that minimal viable product. The test that you run the means by which to validate that assumption or idea is far simpler than you might think. It helps by abstracting a level or two up when you have an idea and thinking, what’s the core idea here that we are trying to test?

And is there an easier way to test that? Because what it allows you to do is it allows you to move faster and at lower effort, which means your ROI increases because you’re not spending as much money uncovering losses, because that’s what will end up happening.

Good stuff.


Thank you for sharing those. And the last one I like, because I see a lot of businesses also fall into that trap where they try to make it perfect before putting it out there. But you got to just do the minimal viable product or release or whatever change you make. Get it out there, get some feedback, see if it works. Are your customers receptive to this, change this change of messaging and then go from there some form of reception from that.

Then come back then.

Like I said, bring it to your development team and figure out how do we really clean this up, make it perfect, and then roll this out across the site. So I think that’s a really good one there. Well, let’s get ready to wrap things up. I’m always a huge advocate of finding out and learning from other companies that have been successful. So if there’s any companies that you’ve either dealt with that we all may know that have a successful conversion rate optimization strategy. And what kind of specific things do they do?

The first thing I’ll say is emphatically. They follow the process that we’ve spoken about, and what they don’t do is chase the magic bullets. You’ll find them misleading case studies on the web about changing buttons that brought in millions, and that kind of thing that doesn’t exist. This is constantly chipping away at it, and it’s following the process. And so the companies that are really good at this, they don’t ever chase those nonexistent magic bullets what they do specifically. And there’s a lot of data to back this up is they run more tests than the rest.

And there is a lot of data. There have been many studies, among others, Harvard Business School studies that have been done talking to companies and surveying companies who do testing. And that link is clear is the more tests you get out the door, the more success you have. So in terms of the actual specifics what they do, it’s really impossible to give you an answer. We can maybe talk about some case studies, but the web is full of them because they’re running hundreds, thousands of tests over a period.

And it’s all different things. And as I said, you come in this with a blank canvas. There’s not many of tricks to work down, so it really is a range of things. It depends on where the data leads you, and often that’s very unexpected places. And then maybe the last thing I’ll say. And if you go onto our website, you’ll find some of the logos of the companies that do this. Well, I don’t have permission to shout out their names. I won’t do that. But think of the big, successful ecommerce stores and they doing CRO or a version of Siero.

They might not call it Sierra because it’s called by many different names, and sometimes it’s just experimentation. It’s that mindset. Sierra is just one application of the experimentation and the scientific thinking mindset. But again, there’s a study from Bloomberg and one from Forestry recently showing the difference in performance between companies who do zero experimentation and those who don’t. And it is just mind blowing. If you’ve seen that stats, you will jump all over this if you haven’t yet.

Yeah, good stuff. Yeah, it does really all come down to testing, and the more tests, the better, the more test, the more data, the more abilities for you to determine what would work versus what doesn’t. Because there’s really no way to know unless you do test it. You just can’t keep doing the same thing, presenting your material, your messaging the same way and expect any different results. You have to test it because you’re going to be always getting a constant flow of customers.

Allen, just to underscore your point, an ecommerce website or ecommerce business has three options to them. The first one is doing nothing. Now, if you do nothing, well, let’s not even talk about that. You won’t be around this time next year. Number two is doing something, but not testing it. And what you’re doing there is you are going on intuition and experience and gut feeling hunches, and I’m afraid the data is against you. The odds are stacked against you. Contact me so I can send you those reports and get you on the right track.

And the third thing is what we’ve been talking about that’s the third option is doing something and testing and making sure that you’re doing the right stuff and what you’re doing by that. So one concern I hear often is people say to me, Well, we can’t afford to be slowed down by testing. We need to move fast. Well, actually, if you do testing right, you’ll work faster. You’ll move faster because you’re not over engineering. If you’re validating Hypotheses, you’re building MVPs and you meant to be getting them out at a high velocity.

So you’re able to get the idea of a roadmap in front of your customers far quicker than back in developers would have taken to Dave the whole thing. The second thing is you make sure that your developers actually work on the right stuff, because if you haven’t validated your ideas, your developers are always stretched. How do you know whether the things on the backlog on the Dev roadmap actually will move the needle? So if you validated that you increase that ROI? The third thing is you keep only the winners and you know which ones would have caused you a loss, and you’ve managed to avoid them.



Those are some great closing pieces of advice there. I think the mindset companies have, like you mentioned, is doing this stuff. Doing this testing is we don’t have time for this. We got to get stuff out there. We got to drive customers, we got to drive sales, and they think our testing process is going to slow things down. But you’re totally correct. It’s the opposite. You’re going to expedite the process by getting more data quickly, more quickly, seeing what’s working, and then be able to make those changes.

So it makes a lot of sense. Well, I definitely appreciate everything that you’ve been part of on the show, and I’ve learned a lot for sure. And the listeners have as well. I mean, these awesome tactics that I know can get businesses to get more in that mindset of what they need to do to improve their conversion rates, because it definitely is a big deal. As our ecommerce space becomes more crowded than ever, you got to do things to keep those customers on the site and then have the right conversation with the customers.

I think that’s the bottom line. Well, I would like to kind of shift gears here with my closing question just so our audience can get to know you a little bit better. But if you don’t mind sharing one closing fun fact about yourself, as you think, our audience will be interested to know about you.

Okay. So this one always gets a response. I’m a trained venomous snake handler. So the area that I grew up in is home to some of the most dangerous snakes anywhere on the planet, and I’ve been trained and certified to handle them. Still, now when I’m in the area and when I have time, and when people have snakes in their homes, they’ll call me and I’ll go out and I’ll capture the snake and release it safely where it belongs.

Wow. That’s definitely an interesting fun fact. I think that kind of ranks. If I’m thinking about my past fun facts, that ranks. I think in the top five of fun facts that I’ve had on the show. So good stuff.


You would be right at home here in Florida because we have some venomistakes, and I’ve seen some around my house. Too bad you aren’t in my area because I would have called you had you handle them. Yeah, I’ve seen a few rattlesnakes actually. Believe it or not, here in Orlando area, right around my house and some really big ones. I’ve stayed away. I’m not going near those guys.

You’re very fortunate to have them around.

You fortunate.


Well, I guess the one thing to note about the snakes that a lot of people don’t realize is that they’re definitely awesome at taking care of some of those critters, the rats, other types of nuisance types of animals. So yeah, that’s right.

Definitely keep that down if you tell me you dislike snakes, then my question to you is, do you like rats and all these things you’re talking about? And I always say to people, just one last comment. If you’d allow me people who stay in areas where there are a lot of snakes, I say to them that if you saw all the snakes that saw you, you wouldn’t sleep at night.

They were around you.

Yeah, very true. Very true. Because when I’m walking out and about, I really see them every now and then, like I said, but there’s a lot more that are in the week.


That is very true. Well, thank you for sharing that, Johanne. I really appreciate that. That’s definitely a pretty cool, fun fact. And lastly, of course, before we let you go, if you don’t mind letting us our audience know what’s the best way to reach you if they want to get somebody who wants to get in contact with you directly.

I think two ways. One is hit me up on LinkedIn and or contact me via the website, which is awahenddigital.Com. Okay, greatital.Com.

We will definitely have that in the show notes on our transcripts on our website. Well, thank you for sharing that, Johann, I appreciate you coming on today and you’ve been an awesome yesterday on the Ecommerce Marketing Podcast.

It’s been great talking to you. Thank you.

Thank you for listening to the Ecommerce Marketing Podcast.

Podcast Guest Info

Johann Van Tonder
Chief Operating Officer at AWA Digital