Who Made That Decision?

During a recent client onsite, we were discussing how to deal with some bad design choices from the past. Coming from analytics, I started to suggest some “creative” solutions that would lead to the right final information in the system.

Unfortunately for everyone, we were not working on an analytics project. We were working on an operational data project. Modifying the data is allowed, but only if it follows financial and accounting rules for how to do so. Breaking those rules is not an option unless you want to have a lot of extra meetings with your auditors and potentially pay substantial fines.

I advocate and teach a very decision-centric view of analytics. Three years ago I wrote an article that touched on the idea that unless something different happens as a result of your analysis, the work was a waste.

Contrast that with a lot of other roles, especially in domains where there are regulatory requirements. Accountants, healthcare professionals, lawyers, and so on. They need to make good decisions. But they also need to be able to back up those decisions by proving that they followed the correct process to come to their conclusions.

The difference between these groups is a bit more subtle than you might expect. Accountants are at one extreme in that they take purely historical data to make purely historical conclusions. To know a company’s profit for the year we can follow a very specific process and get (nearly?) the same answer every time, at least if we follow the accounting rules of how to do so.

Accountants are judged based on following that process exactly. This is not to say there’s no creativity or expertise in accounting, but when you’re looking at only the past, the information is right there.

On the far other end we have folks like traders. It doesn’t matter how you make the decisions you do, as long as you don’t break certain laws. Traders are almost exclusively judged on their investment performance.

In the middle we have examples like finance. Their job is to take current information and make predictions and plans. If the predictions don’t work out, they are responsible for explaining why, and then updating their plans accordingly.

Analytics work is typically somewhere between finance and trading. In some cases, the analysis will trigger a decision. In others, the analysis is an input to a decision.

Following the right process helps, but it does not guarantee a good outcome, which is a medium to large part of how this work is judged. And if an analytics professional is handing the data off to someone else to make the decision, it’s not clear at all how the two are linked. Some other person might be the one who is judged on the decision the analytics was meant to inform.

I think this disconnect is a big gap in how analytics is done today. If the decision-maker is separate from the analyst, good analytics might seem like making the decision-maker happy. It’s a black box from analysis to decision when you have a person in the middle. Though we work with black box models in analytics all the time too.

If we were great at linking the decision to the analysis, I might be out here advocating for more focus on process. I’m a proud holder of the Certified Analytics Professional – Expert from INFORMS,  and I advocate for the INFORMS Analytics Framework as a tool. But to me that all makes sense after people accept that the work they’re doing will hopefully turn into a decision, and they should guide that transformation.

Doctors make treatment recommendations based on their judgement after gathering relevant information. Maybe in analytics we should think of ourselves a bit more like residents with a doctor overseeing the final decisions. Sure, we don’t get to make the final call, but it’s still our job to figure out what the decision should be.