You have probably heard the phrase ‘data driven’ thrown around. But what does this really mean, when it’s not just being used as an empty buzzword? On its face, it means that your decisions depend on data that is collected. For instance, if you were deciding whether a roundabout should be installed at an intersection, you would look at the data on how roundabouts affect traffic collisions. Note that data is being used here synonymously with terms like empirical evidence or observation.
But let’s go a little deeper into what data actually means. Data is not important in and of itself, the important thing about (good) data is that it is correlated with reality. In other words, reality will statistically affect data that is observed*.* If you’re data driven, that really means that your decisions are being driven by your best estimate of reality, whatever that reality may end up being.
For example, suppose someone gives you a coin and you’re told there is a 50% chance that it’s a fair coin, and a 50% chance that it’s a trick coin that someone always lands on heads. You decide to collect some data and flip the coin five times, and each time it lands on heads. The results of this test are helpful for determining whether the coin is fair, because the reality (whether the coin is fair or not) has an effect on how likely the coin is to land on heads each time. I think most people would intuit that since it landed on heads all five times it’s more likely to be a trick coin, but you can also use Bayes’ rule to work out that the probability it’s a trick coin is now 97%.
Of course we all want to believe we have an unbiased understanding of reality, but people can go to different lengths to try to interrogate their beliefs and challenge them with new information. I discuss in other articles more on how to do this, but here I just want to discuss a heuristic for getting in the habit of data dependent thinking.
I believe one should openly acknowledge how one’s beliefs are dependent on empirical evidence. For instance, instead of only saying ‘I believe a roundabout should be installed to reduce collisions’, one should say (or at least think to themselves), ‘I believe a roundabout should be installed only if the data suggests it will reduce collisions.’ Now this belief is much more flexible to new evidence that may be discovered, and keeps you from getting entrenched in one position and dogmatically sticking to it. When you spend time fighting for a belief that is important to you, it’s very easy to become identified with that belief and become susceptible to cognitive biases like confirmation bias. I believe almost everyone could benefit (myself include) by working on becoming more humble about their beliefs, and recognizing what new evidence could arise that would change them.
Similarly, I think it’s a warning sign when hear about someone’s beliefs and they do not seem to acknowledge how their beliefs are dependent on empirical evidence. It doesn’t mean that their beliefs are incorrect, but it does make me suspicious about how rigorous their reasoning was, and if they would change their beliefs in the face of new evidence.
One common clue that a belief is not data dependent is when it’s ‘scope insensitive’. Scope insensitivity is a cognitive bias where someone ignores the size of an effect. For example, psychological studies find that when subjects are asked how much money they would be willing to pay to save 2,000 birds from an oil spill or 20,000 birds, they give numbers that are quite similar (far from a 10 times difference.) This is related to people improperly thinking per dollar.