Why ‘Gut Feel is better than ‘Big Data’

Well according to the Netflix CEO, Reed Hastings, it is: “We start with the data but the final call is always gut. It’s informed intuition”. He reckons that “data science simply isn’t sophisticated enough to predict whether a product will be a hit”.Does he have a point? Or is he talking garbage?

Hastings digs much deeper into the detail on how Netflix takes an intuitive approach to portfolio management when they determine which ideas to ultimately produce. It’s not that they dismiss the use of data (that would be foolish, of course) not least because they invest millions of dollars in analytics to help them consider the huge variety of content ideas that may, or may not, be commissioned. He talks about how Ted Sarandos (Chief Content Officer at Netflix who has the enviable job of selecting original programming for the platform) evidently has the midas touch when it comes to selecting successful series. With hit shows such as ‘Making a Murderer’, ‘Black Mirror’ and ‘Narcos’ to name but a few, he clearly has 24 carat intestines. Or to put it another way (if you will indulge me for a second) Sarandos is ‘The Man With the Golden Gut’. See what I did there? Oh well, please yourself…

So basically what Hastings is alluding to is that big data can help to inform, and maybe even verify, a theory or an idea but it actually isn’t very good at predicting what is going to be a popular product. With current trends it may be able to have some degree of accuracy in foretelling extensions, but breakout hits? Not so much. That takes someone with a strong hunch or a smart intuition that something (whether it be a TV show, a gadget, a toy or an item of apparel etc.) is going to capture the zeitgeist.

It seems that Hastings is not alone in this thinking either. A Harvard Business Review article entitled ‘Instinct Can Beat Analytical Thinking’ cites the work of Gerd Gigerenzer, an eminent Psychologist who has dedicated his life’s work to the study of how some humans have the ability to make the right decisions and how we can actually learn from those same people to do it for ourselves. His assertion is that by using “heuristics, rules of thumb and other shortcuts” we can all become better decision makers and even beat the rational models developed by mathematicians and statisticians which rely on big data to make their predictions.

The main premise behind Gerd’s philosophy is built around the notion of ‘uncertainty’ in an imperfect World:

“If you’re in the world where you can calculate the risk, then statistical thinking is enough. If you go in a casino and play roulette, you can calculate how you will lose in the long run. But most of our problems are about uncertainty”

Basically it’s all about risk management and Gigerenzer tells a nifty story about a turkey to exemplify his point. Yes, you read that right, a turkey. He asks you to believe that you are a turkey (okay, I’m there with all the trimmings) and then imagine that although you are scared that humans will kill you they actually feed and care for you, day after day. The probability that you will continue to be fed and not killed increases every single day until after 100 days the trust is established and it becomes second nature so the fear disappears. That is until the day before Thanksgiving when you literally become ‘dead meat’. The moral of the story? The turkey confused the world of uncertainty with the world of calculated risk.

So how does all this relate to business? Well the Professor has worked with major international conglomerates (plus their leaders) for many years and he estimates that an astonishing 50% of their decisions are based on gut feel. However, he does concede that many business leaders would never dare to admit it. Instead, they post rationalise their decisions by employing third party consultants to corroborate the reasons why they made that particular choice. Surely a somewhat expensive way of justifying what has already been done whether it’s successful or not?

The late great Steve Jobs was a prime exponent of gut feeling over big data. Apparently he wasn’t really one to endlessly pore over reams data or crunch the numbers. His particular genius came from his innate ability to detect a trend before it even happened. The genesis of his ideas was not ignited by analytic rigour but by pure intuition and fuelled by his single minded belief that he was right.

So what’s the conclusion here? Is one better than the other? Does gut feel beat big data? Or vice versa? If so, which one wins? And in what ratio? Well if we really knew the answers to these questions then I guess that we would all be doing it…

In my opinion (albeit slightly predictable and possibly even quite dull) it’s a ‘blend’ of the two. Now, sitting on the fence is not my usual modus operandi but I do have a redeeming caveat… I believe that it works best when gut feeling comes BEFORE big data. Like with Netflix, their most popular long form content has emanated from the creative thought of an individual backed up with robust analytics to prove that there is a market for it. And I believe that the advertising / communications industry which I work in, performs in a similar way. A brilliant insight into a consumer doesn’t come from the masses of statistical data on the way humans behave, it is derived from a smart planner who uncovers a human truth and then seeks to support that hypothesis with the numbers.

So how does ‘gut feel’ play a part in your industry? From an Accountant to a Zoologist, is there a difference in the interplay between the two variables? In your chosen field, is intuition more or less important than big data? And if so, why? As ever, I’m keen to hear your point of view.

Thanks for reading! If you enjoyed the article please SHARE, SHARE, SHARE! (and maybe LIKE, COMMENT or TWEET)

I have 40,000+ followers on LinkedIn and you can FOLLOW ME via this link LinkedIn to try and win a free copy of my book How to be a Top 10 Writer on LinkedIn’ available on Amazon or visit www.linkedintop10writer.com or on Facebook 

LinkedIn ‘Top 10 Writer’ for 2015 and 2016 – ‘TOP VOICES’ FOR MARKETING & SOCIAL & ‘AGENCY PUBLISHER OF THE YEAR’ 

Also available on: BeBee.comDigital Market Asia and 247 Business News