In human cognition, decision making occurs in two main ways: a fast, involuntary, unconscious manner call System 1 by Daniel Kahneman, and a slower, more deliberate manner called System 2.
System 1 is our “gut” or intuition, why System 2 is conscious, deliberate thought and the one that we use for deeper, mathematical analysis. Let’s examine why can’t always trust our gut:
We are inconsistent. When presented with the same evidence at different times, we come to different conclusions; and when different people are presented with the same evidence, we can come to different conclusions.
We remember things that didn’t happen. Our intuition is based on a subconscious collection of data points, some of which are wrong. Our memory is made of bits and true facts, surrounded by holes that we Spackle over with guesses and beliefs.
We are not as good as we think we are. We have an illusion of validity.
We won’t give up bad data. We internalize facts, build mental models and then when new evidence contradicts those facts, we are resistant to changing our model or accepting new data. The misinformed are far more dangerous than the uninformed; misinformation is sticky. According to Shankar Vendantam, “knowledge actually doesn’t make our misinformation better, sometimes it can make our misinformation worse.”
We anchor on irrelevant data. If you’ve ever bought a car, no doubt you’ve seen the “official” sticker price pasted on the window. The sticker price is bullshit. It’s a psychological trick to get you to think in relative terms and compare the current best offer against a higher value rather than focusing on the absolute amount or on other direct evidence. Completely irrelevant numbers can act as anchors and cause us to make bad judgments.
We get tired and hungry. Our decisions are affected by other internal, extraneous factors, such as hunger, mood, and energy.
Source: Anderson, Carl. Creating a data-driven organization: Practical advice from the trenches. O'Reilly Media. 2015.