Friday, April 20, 2012

Don't Hate Statistics. Hate Bad Statistics

You probably don't like the word "statistics" much. You probably think of bland, boring and uninteresting things when you see or hear the word. But you and I are surrounded by statistics every day.

I fell in love with statistics after my first semester course back in college. I hated it in high school, mainly because my high school teacher sounded like Ben Stein and put us all to sleep with his monotonous voice. But in college, I ended up taking an extra semester (over the curriculum requirement for my degree track) because I wanted to learn more.

Two hugely important things I've learned to appreciate that I wanted to share with you are explained below. They pin the two most overlooked and often ignored aspects of statistical references you see and hear on the news all the time:

1) Statistical Results without a Clear "Margin of Error" mean NOTHING

2) "Cause and Effect" Claims are almost Always Bullshit

Here's why...

Margin of Error: It matters a lot.

If I tell you that the results of an opinion poll indicate people favor one political candidate over their opponent by 52% to 48%, you would think that's worthy data. Even 55 to 45. But if I then told you that the margin of error (a factor of sloppiness) was 10%, that means the real numbers could be off by 10% in EITHER direction. Net result? It's a tie. Even at 5% MOE you could consider it a tie, for the 52-48 numbers anyway.

If that were applied to mapping directions and I told you that the odds of the destination being correct were 90%, that would mean 1 out of every 10 times you'd end up in the wrong place. Not very good, is it?

Cause and Effect

One of the most common uses of statistics is to establish a claim of "cause and effect". You hear it all the time. Things like "people with red hair buy blue cars more often" and so forth. Or "people who own a Lexus have kids who enter college at a higher rate than those who don't.

But does that mean that owning a Lexus is the reason? Or is it the income level of those who own them? And how was the sample data collected? From one geographic area? Was it based on survey cards supplied by the dealerships? How accurate is that? How many people were surveyed? 10? 100? 1,000? 100,000? One?!

Do you see where I'm going with this? So often we let this crap seep into our eyes and ears, but we don't stop to ask how it was derived. You wouldn't do that with food or medicine. You'd apply some rationale as to the safety and assurance before risking the poisoning your body. But allowing bullshit statistical claims into your brain poisons your thinking. Don't do it! If food isn't cooked or packaged properly, you don't eat it (hopefully). If statistical claims aren't provided with sample basis and margin of error, call "bullshit!" and ignore them as well. Life is too short.

Post a Comment