Bias Hunter
  • Home
  • Blog
  • Resources
  • Contact

Measure Right and Cheap: Overcoming “we need more information”

1/12/2014

1 Comment

 
Ah, information. The One Thing that will solve all your problems, and make all the hard decisions for you. Or so many keep thinking: if I only had more information… Of course, in many ways, this is exactly right. More information does equal better decisions, as long as the information is – sorry for the pun – informative. Unfortunately, in many cases we either acquire the wrong information, or pass by getting the right kind of data, thinking that it’s too costly.

Thinking about that, I have a hypothesis about why the feeling “we need more information” persists:

  1. Even with information, hard decisions are still hard
  2. Information is of the wrong kind
  3. Thinking information costs too much

Even with information, hard decisions are still hard

This is really not very surprising, but there’s a common thread linking all hard decisions: they are hard. If they were easy, you wouldn’t be sitting there, thinking about the problem. No, you’d be back at home, or enjoying a run, or whatever. Decisions are hard for two main reasons: uncertainty and tradeoffs. Uncertainty makes decisions hard, but it can be mitigated with measurements. But what about those pesky cases when you can’t measure? Well, I’m going to say it flat out: there are no such cases. Sure, you can rarely get perfect certainty, but usually you can reduce uncertainty by a whole lot.

The second problem, that of tradeoffs, is the true culprit for hard decisions’ existence. Often we’re faced with situations, in which one option is more certain, but another has more potential profit. For example, when I run a race, I can start with a slower pace or harder pace. The slower pace is safer: I’ll definitely finish. The hard start pace, in contrast, is more risky: my reachable time at finish is better, but I run the risk of cramps and might not finish at all. Tradeoffs are annoying in the sense that there’s often nothing you can do about it, no measurement will save you. If you’re thinking between a cheap but ugly car, and an expensive but fancier one, what could you measure? No, you’ll just have to make up your mind about what you value.
Picture
Iron Man, Hulk, or Spider-Man? Why not all three?
Information is of the wrong kind

According to a management joke, there are two kinds of information: what we need, and what we have. I think there’s some truth in this. 
Picture
A fundamental problem with information is that not all things are equally straightforward to measure. It’s quite a lot more difficult to measure employee motivation, and a lot easier to measure the number of defect products in a batch. For this reason, a lot of companies end up measuring just the latter. It’s just so much easier, so shouldn’t we focus our efforts on that? Well, not necessarily. It’s not the cheapest measurements you ought to do, but the ones with the most impact. In his book How To Measure Anything Doug Hubbard tells that he was shocked by companies measurements: many were measuring the easy things, and had left several variables with a large impact completely unquantified! As Hubbard explains (p. 111):
The highest-value measurements almost always are a bit of surprise to the client. Again and again, I found that clients used to spend a lot of time, effort, and money measuring things that just didn’t have a high information value while ignoring variables that could significantly affect real decisions.
Thinking information costs too much

It’s an honest mistake, thinking that if you have a lot of uncertainty, you need a lot of information to help you. But, in fact, the relationship is exactly the inverse. The more uncertainty you have, the less information you need to improve the situation. If you’re Jon Snow, just spending a moment looking around will improve things!

I think this mistake has to do with looking for perfect information. Sure, the gap to perfect information is much larger here. But the point is that if you know next to nothing, you get to pick the low-hanging fruit and improve the situation with those very cheap pieces of information, while in a more advanced situation with less uncertainty, you’d need more and more complex and expensive measurements.

For example, many startups face the following question in the beginning: Is there demand for our product? In the beginning, they know almost nothing. They probably feel good about the product, but that’s not really much data. An expensive way of getting data would be to hire a marketing research firm, do a study or two about the demand, burning tens of thousands in the process. A cheaper way: call a few potential customers, or go to the market and set up a stand. You won’t have perfect information, but you’ll know a lot more than you did just a while ago! It’s good to see that the entrepreneurship literature has taken this to heart, and guys like Eric Ries are teaching also bigger companies that more costly doesn’t always equal better. Or even if it would, maybe it’s still unnecessary. Simple measurements go a long way.
1 Comment
Lesbian in Massachusetts link
10/11/2022 04:53:45

Thank you for being yoou

Reply



Leave a Reply.

    RSS Feed

    Archives

    December 2016
    November 2016
    April 2016
    March 2016
    February 2016
    November 2015
    October 2015
    September 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014

    Categories

    All
    Alternatives
    Availability
    Basics
    Books
    Cognitive Reflection Test
    Conferences
    Criteria
    Culture
    Data Presentation
    Decision Analysis
    Decision Architecture
    Defaults
    Emotions
    Framing
    Hindsight Bias
    Improving Decisions
    Intelligence
    Marketing
    Mindware
    Modeling
    Norms
    Nudge
    Organizations
    Outside View
    Phd
    Planning Fallacy
    Post Hoc Fallacy
    Prediction
    Preferences
    Public Policy
    Rationality
    Regression To The Mean
    Sarcasm
    Software
    Status Quo Bias
    TED Talks
    Uncertainty
    Value Of Information
    Wellbeing
    Willpower

Powered by Create your own unique website with customizable templates.