Deceiving with Data

One of my pet peeves is when people misrepresent a situation using data that appears to support their case, but almost immediately begs huge questions (if you’re paying attention).

Here’s one that ticked me off (mildly) recently. In an article (Businessweek June 17, 2010 “The Other U.S. Energy Crisis: Lack of R&D” ) about chronic US under-spending on energy research, the author is trying to make appoint in support of their position that the US is chronically underinvested.

They had the following graph:

Professor moment: Why might I ask myself – “is this a fair picture of the data?” 

I think some (or many) people might look at that chart and say “holy cr$%&p, we spend 35% of what Japan does, and less than China, our emerging global competitor. What are we thinking???”

But does this paint a fair picture of the story, even just within the constraints of the example provided? Well, relative vs. absolute comparisons matter when you’re talking about spending.

.

Do you feel differently about the picture when you look at this chart?

Public energy research, development and deployment spending as a share of GDP
2007 $ BB
Japan 3.90
US 3.68
China 1.32
France 1.08
South Korea 0.54

Well that sure looks different. On this one I feel slightly vexed that we can’t beat the Japanese, but am hardly quaking about several other countries that seemed troubling above.

Where did these numbers come from? I went to Wikipedia and looked up 2007 GDP for each country and multiplied it by the % from the BW chart. Here’s their chart turned into a table with a little more context.

  Spending as % of GDP (2007) 2007 GDP
$ TT
Total Govt Energy Spend
$ BB
Japan 0.08% 4.88 3.90
South Korea 0.06% 0.90 0.54
France 0.05% 2.15 1.08
China 0.04% 3.30 1.32
US 0.03% 13.16 3.68

This wasn’t hard and didn’t take long. In my opinion it shows a much more realistic picture. Percentages don’t buy R&D. Dollars (or Yuan, Euros etc.) do. Take South Korea. In Businessweek’s chart, they appear to invest twice what we do. Really they invest at twice the RATE we do, but off a MUCH SMALLER base. On absolute spend, the US spends 7x as much as Korea. In this case, size matters.

Don’t get me wrong, the numbers above trouble me as an American, but don’t quite paint the same crisis picture. (And they are further challenged by huge spending cited in another article in the same issue).

This kind of sloppy data usage bugs me. It’s a sign of at least two potential root causes:

1 – Laziness/poor sense of numbers.

Much has been written about American’s generally poor understanding of numbers. (One of my favorites and a book I assign undergraduates is Innumeracy: Mathematical Illiteracy and It’s Consequences.) I’m sure this is part of the story, but in this case it’s a reporter for a prominent business magazine writing for executives. Bad math isn’t the issue (I don’t think).

I think it’s laziness. We all get busy, we haven’t necessarily had teachers or mentors really push us to be “correct” and dig deeper. We’ve seen so much bad, we forget or don’t know what good looks like. In the end we lose sight of taking the extra time to dig deeper.

2 – Trying to deceive

I want to note, I am NOT talking about making up data. I’m talking about DISTORTING actual data (simply lying is another matter).

This one is simpler. I want to make a case and I am going to show the numbers in a way that supports my position and diminishes opponents, regardless of the fairness of the representation. I see this happen for “good” reasons (e.g. I need to convince people to do the right thing by creating a burning platform) and for less kind reasons (e.g. I want to win this deal to make more $).

This is not just a news or political phenomenon. I see it in business on a weekly basis. Either new students aren’t getting data they are looking at (because they are learning) or someone is trying to dress up their pitch to win investment money. Whatever. It goes on everywhere.

In this case, I can’t tell which it is and to the reader it doesn’t much matter what the intent is. Just beware the impact.

What I do know is that if you want to be successful in a business career, you better develop a sense of the scale and direction of the numbers you deal with on a daily basis. This helps develop intuition about “hinky” numbers.

So my challenge to you: dig deeper. Don’t settle for crappy data poorly constructed, either from yourself or others. Always be skeptical (and bring a calculator).

Tell me about bad data you’ve seen.

5 thoughts on “Deceiving with Data

  1. Pingback: Tweets that mention Deceiving with Data -- Topsy.com

  2. Hey Phil,

    Thanks for this article. I am doing a lot of work on metrics, numbers, diagrams, and data modeling. This article reminds me to keep a very close look at the accuracy of the weights, averages, and statistics so that I don’t create business intelligence that managers may use incorrectly.

    Thanks-
    D

    • Dustin – It’s important to keep that perspective as you develop your professional judgement. My question is always “what’s really going on here?” Good luck and have a good summer.

  3. I think it depends on what exactly you are trying to argue. Simply saying that US is under-invested in energy research and use that blue table can be misleading. This table can and does support the argument of “US needs to put more of their GDP into energy research” which can be translated as “US is under-invested”. So maybe it’s a matter of phrasing the argument rather than deceiving with data. Just my thoughts.

    • Ida – I don’t disagree. In this case, I put it to my “lazy” category from the post- rather than being cynical. If this were my student in a draft review my feedback would be relatively gentle and in the “be more precise” genre, not the “don’t be like Glenn Beck” category 🙂

      I was also reacting to the fact that in the same issue, another author cites a >$40BB spend on infrastructure and R&D. My point on the policy side would be if the US spend as % of GDP rose a few 100ths of a percent, we would be spending the most in the worl in raw $. That’s an important (and obscured by the chart) point.

      I personally feel we should be doing MUCbetter, but you want to make the case clearly (In my opinion.)

Leave a Reply

Your email address will not be published. Required fields are marked *