Which will help your business be more successful: statistics or probability?
Underwriters at insurance companies use statistics to assess future risks. This is based on years of collected data.
Probability is what card counters in Vegas use to increase their odds of success. This is based on real-time, real-life experience.
If you want to play it safe, use statistics. If you want to win big, use probability.
There Are Lies, Damned Lies, and Statistics – Mark Twain
Businesses are increasingly using statistics to manage decision making, as evidenced by popular books like Tom Davenport’s Competing on Analytics and the boom in CRM system usage.
The belief is that if we gather more data we can make better decisions. But this may not be true when it comes to innovation.
If you are crunching numbers, you are probably gathering information from existing customers. This will give you insight into their buying habits, usability behaviors, and other patterns. But most likely you are only gathering data on YOUR customers. This represents the middle of the bell curve or the norm. This information may be useful in “incremental” improvement, but it will rarely lead to significant innovations.
When you move beyond the norm to the far ends of the bell curve, you will find the real interesting ideas.
Being normal is not a virtue; it denotes a lack of courage
On the far right-hand side of the curve are the market leaders; the advanced users. They may not be your customers because you can’t meet their high-end needs. Or maybe they were once your customers and they left. When someone is not a customer it is difficult to gain insights into their wants and needs. If you could somehow understand their perspectives, you may find opportunities for “advanced” innovation and insights on where the industry may be going in the near future. These innovations would be more radical, yet continuous in nature. Think of this as the Blu-ray improvement on the standard DVD (we’ll save a discussion on the demise of HD DVD for another time).
On the far left-hand side of the curve are the laggards; the less sophisticated users. Your products/services may be too advanced, too complicated, or too expensive for their needs. Again, you are probably not gathering statistics on these individuals or organizations. But here lies the greatest opportunity for discontinuous innovation. Or as Clayton Christensen would call it, disruptive innovation. If you can find a way of “dumbing down” your offering, you might find new and untapped sources of revenue. Quite often these products become the de facto standard, much like when PCs replaced the more sophisticated mainframes and mini-computers.
The problem is, it is very difficult to get data about the ends of the bell curve. Focus groups, surveys, and other traditional data gathering techniques are useless. I love this quote from Scott Cook at Intuit: “For every one of our failures, we had spreadsheets that looked awesome.” We can use numbers to justify anything we want. But quite often they justify the wrong actions.
The Probable is What Usually Happens – Aristotle
If a statistics-driven innovation model does not work, what would a probability-based model look? Probability tells me that if everything is equal, the more bets I have, the more likely one will be successful. The odds of 1 success out of 200 are greater than 1 success out of 20.
But how can you have more bets without diluting your effort and potential returns? The key is to learn as you go. This is exactly what card counters to.
Let’s contrast a more statistics-driven model with a probability-based model. To do so, we will use two exceedingly simplistic examples. With innovation model #1, you make a few “big bets” based on analytics you gathered from your customers (a statistics-driven model). Innovation model #2 is a more experiential “learn as you go” model (a probability-based model).
In both examples, let’s assume you have $100 million to bet, woops, I mean invest in innovation.
Innovation Model #1 – Big Bets: This is the most common approach and is highly driven by statistics. You identify a number of large innovations you want to invest in. For this example we’ll use 20 projects @ $5 million each. No matter how much data you have, most innovations will fail. And of the successes, most will not achieve the predicted ROI. In the end, if you are lucky, you’ll have 3 wins out of 20. This feels like putting all of your money on 35 black on the roulette table and crossing your fingers. Your successes/wins had better pay out big to cover your losses.
Innovation Model #2 – Learn As You Go: Let’s look at a different model. What if instead of 20 large projects, you have 200 smaller projects. Again, you know that most of these will fail – but you don’t yet know which ones. You initially invest a small amount ($10M or $50K per project) to test the ideas as low-risk, low-cost experiments. Based on this experience, you decide that 40% (80) of the ideas still show some promise. But you are not yet ready to bet the house. This time you allocate an addition $20M ($250K each) to do further testing. You now eliminate 70% of the projects, keeping 24 alive. You now invest another $20M (nearly $1M per project). Of these 24, you decide that 5 are real winners. At this point you have only spent half of your money and yet you were able to eliminate 195 ideas. That’s incredibly valuable information learned by doing rather than by analyzing. You now invest the remaining $50M on those 5 ($10M each).
With innovation model #1 you must “guess” which ideas will be successful up front. HP is moving to this model by consolidating 150 ventures into only 20. This feels like a bad bet. What if the 20 they chose all turn out to be duds and that the real winners were in the 130 they eliminated?
With innovation model #2, you make lots of small bets. And as the odds of success improve, you increase your bets. This is the business version of card counting. As you get to the end of the deck, you have “real life” information that guides your bets. As the odds of success increase, you increase your bets. This method works. Just ask the kids from MIT who won millions card counting in Vegas and will make millions more with their new movie, 21.
karl Staib l Your Happiness Matters says:
I think it’s a bad move too. The more that a company thinks they can focus on a few new ideas that are sure to work the more likely they’ll miss the mark. All great innovators try many ways until a few emerge. I believe it was Edison who said something like 10,000 of his projects failed, but the ones that succeeded made him famous.
d.d. says:
Reality introduces complexity, as normal.
.
(1) You may give up your “first mover advantage.” While you’re doing 2-steps of testng “low-risk, low-cost” projects, the competition is watching. In a lot of innovative new product situations, the second and third entrants to the market quickly drive prices down to the “competitive, average profits level.”
.
(2) Some projects simply aren’t anywhere near scalable to the nano level you’re suggesting. $50,000 for a project that needs engineering work plus some kind of prototype to be created … now you’re talking about no more than perhaps 3 total man-months of effort in translating an idea into a product, less if any sales test is made. Some percent of your R&D portfolio will fail simply because the money allocated (the bet) isn’t big enough to give the idea a fair test.
.
(3) Some more ideas will also perish irrespective of their merits because they get handed to the “C” team instead of the “A” team. With 200 projects to hand out, some have to be handed out to less capable, less visionary, less experienced people, and as a result the work-up of the idea will be sub-optimal. (Either hands-on innovators/developers or mgmt support.)
.
(4) Ignores effects on internal motivations and incentives. 200 projects with 5 emerging at the end means that 98.5% of the original project teams were involved with “losers.” How do you reward and keep them happy? Most organizations punish people involved in failures; some at least don’t actively punish; a very few have managed to reward people for (prudent) risk-taking, even if/when it doesn’t pay off. In the second and future years, how do you motivate folks to give their best efforts, when the statistical reality is that any project they’re handed has only a 2.5% chance of success? As a manager/engineer/geek, do you want to take on career assignments where you know up-front your chance of success is only 2.5% ??? What works as a risk-mitigating portfolio for the company does not (naturally and easily) work at the level of individual employees. Ignore that at your peril.
Stephen Shapiro says:
D.D. I appreciate your taking the time to provide such a well thought out comment!
Yes, real life is more complex than my simplistic model. And you raise some valid points. I believe that the correct answer is in fact a blended approach. Some bets are better fleshed out using a more traditional approach. However I still contend that most bets will come through the “card counting†model I describe.
I agree that not all innovations can thrive on micro funding (your point #2). Some may require more up front money. And there are some “big bets” (like building a computer chip fabrication plant) that may require lots of analysis before moving too far forward. But the same general “scaling experiment†philosophy can still prevail in most situations. The key is to institutionalize a mindset of experimentation – and to become masterful at defining small, valid, scalable experiments. This takes practice.
You will not lose “first mover advantage†(your point #1) because an iterative approach is in fact considerably faster…assuming the organization has streamlined the funding process and failure is not only tolerated, but embraced. The key is to address the cultural issues first before overlaying a new process like this (your points #3 and #4). And I recognize that changing a process is a lot easier than changing a culture. This is why the status quo typically wins out.
I will continue to write on this topic. In the meantime, if you are interested in reading about a wildly successful company who uses a similar approach, but “The Science of Success†by Charles Koch, the CEO of Koch Industries. As readers of my blog and books know, Koch Industries is the largest privately held company in the world ($90B in revenues) and has grown seven times faster than the S&P 500 for the past 40 years. Their Market Based Management philosophy is all about using a “build it, try it, fix it†approach – creating a series of small experiments that grow over time. Their philosophy is very “Darwinian†in nature. All reasonable ideas get seed money. Good ideas get more and start to cannibalize ones that aren’t performing as well.
D.D. Thanks so much for the thoughtful comment.
Antony Woods says:
If you want to know the probability of who’ll become president, forget about the statistical opinion polls, go to the bookmakers’ sites.
Stephen Shapiro says:
Antony, I think you are spot on!
Graham Horton says:
I think it’s very misleading to set the piece under the premise “probability or statistics”?
in fact, it will be statistical arguments which
prove the superiority (or rather, better applicability) of any one approach over another.
What you have written has nothing to do with probability or statistics per se. The first part of the article would more appropriately be titled “Orientation towards the fringe instead of the center”, and the second part, “One-shot or evolutionary innovation processes?”
Innovation Model #2 has been popularised by Google. One reason that it works for them, is that prototype Google products can be created very cheaply – by a single person working in their “free” hours. However, it is harder to see how individuals or small groups can build innovative prototype nuclear submarines (for example) in their off-hours for a few thousand dollars.
Although it is currently popular in the blogosphere to deprecate incremental innovations and uphold disruptive innovations as the solution to all problems, let us not forget that it is the incremental innovations that pay the bills and provide the 100$ million needed to seed the next generation of innovation projects.
All styles of innovation process mentioned in the article have their justification – different goals require different approaches.
Brendan Dunphy says:
Poor innovations succeed in the market if enough is spent to advertise and promote, driving out better innovations in the process. I think we always take a much too narrow view on the innovation process and fail to factor in this element to our thinking, as this article does too. Most organisations I work with generate far too many small projects that are not resourced, researched or promoted to succeed. This is easier than doing the depth of thinking and customer research required to spot true opportunites and deliver compelling solutions – I would suggest less is more and I think HP and others understand this need for innovation focus.