Opinion is Part of Problem Solving

Suppose you had access to an oracle. Any question you asked her, she would answer with perfect accuracy. Now imagine you decided to use this oracle to help you decide what problem to solve. Which questions should you ask?

We can start with the first and most obvious question “Hi Oracle, we were planning to solve Problem A, is that the right problem to solve?” She replies, “That is a matter of opinion, how do you define right?” Ok, I modified this a bit to be more oracle-like, but this is the key reason consultants constantly respond to (seemingly) simple direct questions with “It depends.” Most situations depend on context, which is organization and situation specific.

First, let’s discuss what would constitute a rigorous definition of “right.” To develop an understanding, we can start with the “diet problem.” This is a traditional problem in optimization which says: “Given a list of foods you could choose to eat, and nutrition requirements, what is the lowest-cost choice of foods that meets your needs?” If you want to dive deep and even play with the ability to change which food to consider, check out NEOS which has been kicking since at least 2007 when I used it to solve some homework in undergrad.

So, we begin with a list of food options, costs, nutrition data, and priorities. Specifically, we want the cheapest set of food that has acceptable nutrition. We define upper and lower bounds on total calories, cholesterol, sodium, etc. Also, we can specify that we are unwilling to eat more than, say, ten servings of sardines in oil a week (they set everything to ten maximum servings in the sample). We give all this information to a computer, and it can spit out “The Answer.” For the model on NEOS, we get a recommendation to eat raw carrots, baked potatoes, skim milk, peanut butter, and air popped popcorn.

If, like me, you find the recommendation to eat 3.601 servings of peanut butter (over 7 tablespoons) daily a bit alarming, you could go into the model and change the definition of right. This is what it means to have a rigorous definition of right, that there is a set of equations that specify not just your definition of “best” but also that you have included all the pieces you need to into the model.

With a shared understanding of what it takes to have a problem statement that has a “right” answer, we have a framework for our discussions with the oracle. Let’s modify our original question. “Hi Oracle, we were planning to solve Problem A, we have a single outcome we are trying to improve, and a list of constraints on which problems we could tackle. Is A the right problem to solve?” Odds are very high her answer would be “No, that is not the right problem to solve.” If you asked why, you might learn that Problem B is like A, but will be at least a bit better outcome-wise. Like the peanut butter example above, that might be because you forgot to include a constraint and B is not actually a valid option. Other times Problem B is actually better, and you just did not realize it was an option.

But what if you have two outcomes you care about? With one outcome and a list of constraints we can still try to pretend we have a true “right answer” to the problem. If I asked you “Do you really only want to minimize cost, or do you also care about how tasty the food is?” It is very hard to stick to the claim that only one outcome matters.

We can still use the framework from optimization to give us the right answer even when we have multiple goals. In optimization we call this a “multi-criteria objective function” which basically means that you have multiple outcomes (criteria) you care about. In optimization all you need to pull this off is a tradeoff value. How much more expense is worth one extra unit of tastiness? With that defined the oracle (or a computer) can give you just as right an answer as before.

But how do you define that tradeoff? In some later post I will talk more about practically wrestling with the tradeoff. However, no matter what, the answer is somewhat subjective. There is always a matter of preference when you try to combine outcome variables with different units. Individuals make these choices countless times a day. Should I buy coffee or make it? What should I have for dinner? Which task should I do first, second, third, and so on?

Economics studies “revealed preferences” as a tool to understand what people would choose given a set of options. There is a research area where we try to understand this subjective aspect of decision making. Basically, we give people choices to make and infer their preferences based on their decisions.

If tradeoffs are subjective, how can we ask our Oracle to give us the “right” problem to solve at all? Well, subjective does not mean “pretend.” It just means that we have to ask people to tell us how to balance the competing goals. And if we need to identify “right” for a group of people (say a company) we need to also have rules for how we balance each individual’s definition of right to get to an overall answer.

If all of this sounds like a lot of work, it certainly can be. Most problems are not worth having a “right” answer to at all. Just something that is good enough. But if you want to ask an oracle what to do, there is no way around specifying both the objective and subjective criteria she should consider to give you an answer.