169: Small Tests, Giant Leaps
As a veteran of Silicon Valley with tenures at multiple massive, industry-leading tech companies, Carl Tsukahara has developed a serious taste for adventurous testing, especially in B2B. That doesn’t mean just running around pushing buttons and selecting metrics at random, it means breaking free from the common snare of planning multiple meetings and emails before adjusting a small facet of the landing page for A/B testing.
Speaking of A/B testing, Tsukahara’s career path has led him to currently serve as Chief Marketing Officer of Optimizely, which is essentially the home of the A/B test. They also call themselves “the world’s leading experimentation platform,” painting the company as a fitting place for a bold marketer and tester like Tsukahara. In this interview, Carl discusses what different companies ought to be testing for, how to properly set up certain guardrails to guide your testing, what we can learn from big brands like Amazon and Netflix, and much, much more. Check it out!
How do you think testing in marketing should be approached or planned out?
I think all of us can learn from big brands like Amazon and Netflix. These folks experiment at a crazy scale, and they do so because going into meetings and discussing things through PowerPoints and Google slide and picking everything apart often isn’t the way to go. Marketing can get bogged down there, and when you take into account the different perspectives of different people, you’ll often end up playing it safe, which is exactly what you don’t want to do with testing. You know what you do? You just put it out there—your big idea. And if you’re a big enough company, expose it to a few percentage points of your user base is pretty low risk—just find out what’s going to work. And it clears up a lot of what I’ll call “the innovation fog,” that “oh, my God, we have to have seven meetings, and then go through this 50-slide deck and do this, this, and this.” Dodge that, and you can really drive innovation.
Is there a core to your testing philosophy? What are the key metrics you’d typically want?
Sure. But, I think those metrics depend on what the company trying to do. Here’s an example. We have a large B2B customer, Fortune 500 kind of a thing, and they’ve traditionally sold a big product to big enterprises. And they basically said, “Hey, look, I’ve got to be much better at selling to down-market.” Which is much more self-service. That’s after a period of examining their sales processes and cycles, and this represented a key business goal from them. When they look at their process and see a point where they’re lacking, in this case down-market sales, they’ve got a north star for what they should be testing. The whole point is to align your testing program to that. That’s a key corporate initiative, that’s the key to everything. Find your key initiative and test to it.