What Makes a Decision Data-Driven?

For today’s post, I want to discuss different ways people could use the term “Data Driven Decision.” I think there is a spectrum of interpretations which often get confused for one another depending on your perspective. I am organizing my definitions roughly from where the decision is least-to-most driven by the data. However, there are definitely gray areas and overlap. Without further preamble, the definitions.

I looked at some data before I made a decision.

Unless you are making a decision that is purely based on opinion (should I buy the red coffee mug or the blue one?), most decisions fall into this category. Say you are at the grocery store and want to buy peanut butter. The decision you have to make is how many of each kind to buy, where usually you choose just one or two options to buy more than zero of. Depending on the information about price, size, and brand, you will probably change your choice at least a little bit. Say the kind you usually buy is on sale, you may decide to buy extra compared to if it is not on sale.

I do a lot of work with clients on inventory decisions. Your choice at the grocery store is exactly the situation some less sophisticated buyers make every day. Maybe they even look at how much they bought last time or how much was used over the last 6 weeks. Regardless, in this category any analysis is restricted to understanding “what happened” and does not extend into why.

I analyzed some data before I made a decision.

I think most people in the previous category would not claim to be making data driven decisions. On the other hand, people in this analysis category frequently believe they are making data driven decisions. I searched through the top hits on google and found this post with a succinct definition. In general, each of the pages referred to the following process:

1. Understand the decision you want to make.

2. Gather and analyze data that is related to that decision.

3. Make your decision based on insights you gather through the analysis process.

Looking at these steps, my key question is how the insights are supposed to be related to the decision? The process above does not answer that question, and most people have a fuzzy path from the data to the decision.

How do I know it is fuzzy? Experience from working with folks and trying to turn what they do into code. The difference between what someone says their process is and the reality becomes very obvious when you automate the analysis and then they review how much sense the outputs make.

I used data to see the consequences of my decision on my goals.

You may be familiar with this level of “data driven decisions” as scenario analysis. The idea here is to not only look for insights that you gather during your analysis, but to actually identify what the future state would be if you make certain decisions. Sophisticated decision support systems usually provide this kind of functionality, but an analyst working in excel can (and frequently does) generate scenarios as well.

The key difference between this and the previous definition is that you have reasons, not just “insight” driving your decisions. An insight might look like “Wow, when we offer someone a discount, they cancel less often.” A consequence of a decision might look like “Wow, if we offer people who threaten to cancel a discount, we will make an extra million dollars next year.”

As I mentioned in the introduction, there is grey area between these definitions. Good analysts will look specifically for insights that guide decisions. And the best way to guide a decision is to understand what options you have and what the outcomes will look like. But less skilled folks may or may not realize the purpose of focusing on data-driven-decisions is that the data actually drives towards the decision.

I used prescriptive analytics to identify the right decision given my goals.

In this level of “data driven decision” the data is undeniably driving the decision. Someone has coded an algorithm that takes data as an input and spits out decisions as an output.

While using prescriptive analytics could mean that the data is fully driving the decision, in practice people often use the outputs more as a tool for scenario analysis. This is called having the human-in-the-loop and is also a perfectly valid way to use prescriptive analytics tools.

Conclusion

In this article I described four different levels of data driven decisions with a lot of overlap between the levels. Even so, I hope you see that there is a difference between data, analysis, and insights which are specifically… let’s say “aimed” at the decision you are trying to make, and not doing so.