Bias Hunter
  • Home
  • Blog
  • Resources
  • Contact

What You See Is What You Believe

8/12/2014

0 Comments

 
The old saying everybody has heard says “don’t believe everything you read”. Current research shows while this is true, we also need a saying instructing us “don’t believe everything you see – especially on a chart”.

In their recent article Blinded with science: Trivial graphs and formulas increase ad persuasiveness and belief in product efficacy Tal and Wansink experiment with different ways of providing information about medical drugs to consumers. They have two groups of people: the ones seeing just text, and the ones seeing also a chart. The graph was trivial: it added no new information at all. The control group saw a text saying that
A large pharmaceutical company has recently developed a new drug to boost peoples’ immune function. It reports that trials it conducted demonstrated a drop of forty percent (from eighty seven to forty seven percent) in occurrence of the common cold. It intends to market the new drug as soon as next winter, following FDA approval.
The experimental group saw the same text, followed by a graph:
Picture
The result? Participants who saw the graph believed the drug to be more effective than the control group.

As it is, this result could be just due to increased information retention: graphs make it easier to remember things. Well, the authors thought the same, and replicated the study by checking for retention 30 minutes after the study. It turned out there was no difference in memory, but the same effect persisted. What could be the cause? Looking at the data, the authors found out there’s a significant interaction of belief in science and the chart effect – people with more belief in science ended up being more convinced because of the graph! Uh oh, this is worrying for us science buffs.

In their third experiment, they changed the setting by replacing the bar chart with a chemical formula. What happened was that once again, a scientifically-looking item increased the effectiveness rating. Unfortunately, the authors neglect to mention whether the interaction with belief in science was significant here 8probably not, since it wasn’t reported).

It’s understandable that a chart would make a claim seem more scientific. After all, one of the hallmarks of pseudoscience is hiding all the results behind vague words – a chart is at least clear. Unfortunately, by choosing the right measures and axes, one can design compelling yet false claims with graphs quite easily. A chart does not ensure sound science, or sound data. I guess we’ll just have to stick to trying to evaluate the actual claims carefully. Don’t believe me? Well, here’s a chart to convince you:
Picture
0 Comments



Leave a Reply.

    RSS Feed

    Archives

    December 2016
    November 2016
    April 2016
    March 2016
    February 2016
    November 2015
    October 2015
    September 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014

    Categories

    All
    Alternatives
    Availability
    Basics
    Books
    Cognitive Reflection Test
    Conferences
    Criteria
    Culture
    Data Presentation
    Decision Analysis
    Decision Architecture
    Defaults
    Emotions
    Framing
    Hindsight Bias
    Improving Decisions
    Intelligence
    Marketing
    Mindware
    Modeling
    Norms
    Nudge
    Organizations
    Outside View
    Phd
    Planning Fallacy
    Post Hoc Fallacy
    Prediction
    Preferences
    Public Policy
    Rationality
    Regression To The Mean
    Sarcasm
    Software
    Status Quo Bias
    TED Talks
    Uncertainty
    Value Of Information
    Wellbeing
    Willpower

Powered by Create your own unique website with customizable templates.