Today we continue the “Test”” part of the FAST Innovation Model (Focus, Ask, Shift, Test).
In the last video I discussed why confirmation bias will cause you to run faulty experiments. In this episode I share several techniques for overcoming this issue.
Be sure to watch the previous videos!
Transcription:
Today I’m going to give you some specific techniques you can use to run better experiments, and overcome the brain’s confirmation bias…
In the past videos I talked about why the brain wants to prove our beliefs to be true. We run experiments designed to confirm or disprove our beliefs, but most of the time we prove them to be true because if we believe it’s a great idea, we’re only going to find evidence to support that idea.
However, there are some things you can do, in order to overcome this natural tendency…
The first thing is to recognize it’s how we’re wired. As human beings, we’re wired for confirmation bias. When we have a strongly held belief, we only find evidence to support it. This is what we see in our political systems. If I have a strongly held belief about a particular political issue, I’m only going to find evidence to support my beliefs. The same thing is true when it comes to innovation. So when we really can understand that this is how the brain is wired, we can at least take the first step in saying, “Hey, maybe everything I believe to be true isn’t true.”
The second step is to create a team of the devil’s advocates. These are the people who are there to help disprove your hypotheses.
In fact, one of the things that I like to do is make sure that we separate the seeker, from the solver, from the experimenter. Let me say a bit more about that. The seeker is the person who wants the problem solved. I have a problem. I need to find a solution, so I’m going to seek for a solution. Now, I’m passionate about this, so the problem is my passion is going to get in the way of my being able to conduct good experiments. It’s also going to get in the way of my being able to find good solutions. So the people I look to, to solve the problem, the people who are going to develop the breakthroughs for me, probably don’t come from my area of expertise. We talked about this in earlier videos. So what I need to do is make sure that my solvers are separate from my seekers. The solvers have to be an independent group of people, designed to help find breakthrough solutions.
But we have to go a step further. We have to separate our solvers from our experimenters. If I’m the person who came up with the great solution, I’m going to be so excited about it that I’m going to want to see it implemented. But this means that I’m going to have faulty reasoning. There’s going to be experiments that we conduct, that should disprove my hypothesis and my beliefs, but it won’t. So we need to have a group of independent experimenters, who really are designed to objectively prove or disprove the hypothesis.
When you change the way you think about experimentation, you will fundamentally change the way you think about failure because, again, failure is not a good thing. Experimentation is good, and experiments are only good when they actually prove or disprove a hypothesis in an accurate way.
The next round of videos, we will tackle some other topics related to innovation.
I look forward to seeing you soon.
Ramkumar Yaragarla says:
Hi Stephen, that was a good video. You have put out your thought on having three sets of people, the seekers, the solvers and the experimenters. It has always been that solvers had a group of people within them who would experiment, iterate and investigate new solution for innovation. You are proposing, the solvers to be independent people. So, what would their role be ? Good thought. Good takeaway for me. I guess, at the time of implementation and actual execution separating them would make more sense to get a much detailed and a good grasp of the solution.
Nice post. I happen to bump into your site from another site. Liked watching your videos. I will visit again. Cheers, Ramkumar
Stephen Shapiro says:
I’m so glad you bumped into my site. The solvers are those who are looking for solutions. What I find is that the person who wants a problem solved is often too close to the challenge and therefore has a limited range of possible solutions (expertise is the enemy of innovation). Therefore, I want to turn to a different group of people to find solutions; people who are not as close to the problem as I might be. These could be from inside the company or external; from a different department, or across the entire company.
Then, when it comes time for testing the hypotheses we need a different group to run the experiments. Otherwise confirmation bias will kick in and the experiments will be designed to confirm what they believe to be true.
I hope that helps!
Christoph Sohn says:
Hi Stephen. Thanks for your innovation minutes. I really like them, especially as they are easy to digest and hence perfect for a coffee break.
One comment on this one: it sounds like you suggest to have three different groups/persons: seekers, solvers, experimenters. From our experience, the seeker and the experimenter are ideally the same people. In reality, the business need owner may not have the resources to review & test suggested solutions and may therefore nominate a group of experimenters. However, the seeker still knows best what they are (not) looking for. Hence, the seeker should be also be (part of) the experimenter(s).
Stephen Shapiro says:
Thanks for your comment. We are for the most part aligned. I strongly believe that the seeker should not be the solver. And the solver should not be the experimenter. But the seeker CAN be the experimenter. Although I do believe that (as you point out) the seeker needs to be involved from a political perspective, I also believe that their pasts experiences will bias the results of the tests if they do not allow the experimentation to be done properly. They need to be involved, but they also need to be somewhat hands-off at the same time. Thanks!!
Fraser Liscumb says:
Natural born Innovator administrator and Analyst and actually have been part of change for the last 40 years in Canada to now innovation, R&D, start-up is now supported. More than agree with what you say. Our Hub mandate in 1996 was to change not what the system did but how it did it and 80% of what we tested is now growing in the Global marketplace, The last 20% was a more advanced 21st century form of Crowd Funding was seen as to innovative and to disruptive in 2002 to be supported and the technology to expand beyond a single community collaboration system is still not been able to transform it to go beyond a silo technology. When our Hub introduce Canadian government in 2002 to community incubator and accelerator thinking . Maybe you should send this to all Global leaders. Once they get headed in the right direction maybe we would get to a more sustainable future sooner than later.
Stephen Shapiro says:
Fraser, thanks for your thoughtful comment. As much as I would love to influence our global leaders, I am not sure how successful I would be. Why? The same reason that experiments often fail: confirmation bias. People want to believe their own perspectives a lot more than those of others. And people who are in power tend to have an even greater tendency to believe their own hype. But I will keep on doing what I can to spread the word. Thanks again!