An Obsidian Potency PSA: Don't Believe the Hype

EXTENDED BODY:
"This is another Public Service Announcement brought to you in part, by Obsidian Potency."

As much as I dislike the workload in my Research Methods class, I've learned a ton of stuff.  It has even made me bust out laughing, as I did when I read this article by Tom Hobart, president of New York State United Teachers.  I will post most of the article below:

>>My dentist is great!  He sends me reminders so I don't forget checkups.  >He uses the latest techniques based on research.  He never hurts me, and >I've got all my teeth, so when I ran into him the other day, I was eager to see >if he'd heard about the new state program.  I knew he'd think it was great.

>"Did you hear about the new state program to measure the effectiveness of >dentists with their young patients?"  I said.

>"No," he said.  He didn''t seem too thrilled.  "How will they do that?"

>"It's quite simple," I said.  "They will just count the number of cavities each >patient has at age 10, 14 and 18 and average that to determine a dentist's >rating.  Dentists will be rated as Excellent, Good, Average, Below Average and >Unsatisfactory.  That way parents will know which are the best dentists.  It will >also encourage the less effective dentists to get better,"  I said.  "Poor dentists >who don't improve could lose their licenses to practice."

>"That's terrible," he said.

>"What?  That's not a good attitude," I said.  ""Don't you think we should try to >improve children's dental health in this state?"

>"Sure I do," he said, "but there's not a fair way to determine who is practicing >good dentistry."

>"Why not?" I asked.  "It makes perfect sense to me."

>"Well, it's so obvious," he said.  "Don't you see that dentists don't all work with >the same clientele?  So much depends on things we can't control.  For >example," he said, "I work in a rural area with a high percentage of patients >from deprived homes, while some of my colleagues work in upper middle >class neighborhoods.  Many of the parents I work with don’t bring their >children to see me until there is some kind of problem, so I don’t get to do >much preventive work.  Also,” he said, “many of the parents I serve let their >kids eat way too much candy from an early age, unlike more educated >parents who understand the relationship between sugar and decay.  To top it >all off,” he added, “so many of my clients have well water that is untreated >and has no fluoride in it.  Do you have any idea how much difference early >use of fluoride can make?”  

>“It sounds like you’re making excuses,” I said.  I couldn’t believe my dentist >would be so defensive.  He does a great job.

>“I am not!”  he said.  “My best patients are as good as anyone’s, my work is >as good as anyone’s, but my average cavity count is going to be higher than a >lot of other dentists because I choose to work where I am needed most.”  

>“Don’t get touchy,” I said.

>“Touchy?” he asked.  His face had turned red.  “Try furious.  In a system like >this, I will end up being rated average, below average, or even worse.  My >more educated patients who see these ratings may believe this so-called >rating actually is a measure of my ability and proficiency as a dentist.  They >may leave me, and I’ll be left with only the most needy patients.  And my >cavity average score will get even worse.  On top of that, how will I attract >good dental hygienists and other excellent dentists to my practice if it is >labeled below average?”

>“I think you are overreacting,” I said.  “Complaining, stone-walling and >excuse-making won’t improve dental health…I am quoting from a leading >member of the DOC,” I noted.

>“What’s the DOC?” he asked.  

>“It’s the Dental Oversight Committee,” I said, “a group made up of mostly >laypersons to make sure dentistry in this state gets improved.”

>“Spare me,” he said.  “I can’t believe this.  Reasonable people won’t buy it,” >he said hopefully.  

>The program sounded reasonable to me.  “How else would you measure good >dentistry?”

>“Come watch me work,” he said.  “Observe my processes.”

>“That’s too complicated and time-consuming,” I said.  “Cavities are the bottom >line, and you can’t argue with the bottom line.  It’s an absolute measure.”

>“That’s what I’m afraid my parents and prospective patients will think.  This >can’t be happening,” he said despairingly.  

>“There you go again,” I said.  Doing this would be like grading schools and >teachers on an average score on a test of children’s progress without regard >to influences outside the school: the home, the community served, and stuff >like that.  Why would they do something so unfair to dentists?  No one would >ever think of doing that to schools.”

>I just shook my head sadly, but he had brightened.  “I’m going to write my >senator,” he said.  “I’ll use the school analogy.  Surely they will see the >point.”  He walked off with that look of hope mixed with fear and suppressed >anger that I see in the mirror so often lately.”<<

This may seem pretty random, but it is fucking hilarious - especially if you've ever been in grad school.  Every good student knows you can't base a rating system upon a single criteria - especially one that cannot be fully controlled by the people you are rating.  It is also a phenomenon I am seeing more and more in all arenas of our lives: within politics, the news, and polling.   It was either American humorist Mark Twain or British politician Benjamin Disraeli who said, "There are three kinds of lies: lies, damned lies and statistics."  It's a crime to rely on statistics without giving the source study a good once-over.

An even worse crime, however, is to take statistics and create some uninformed, bullshit program.  I guaran-damn-tee you that some well-meaning but ignorant citizen/politician saw that many kids have rotting teeth by the time they are 18.  So what happened?  They got a bunch of people who have no idea what they are talking about to design a program that makes no sense.  

Not only does this particular program make no sense, it may end up crippling an entire segment of health care in New York State.  Any time some jackass discovers some horrible statistic in this country, you can bet your bottom dollar that he/she will find some scapegoat.  Even scarier, they'll probably come up with something even worse.  A memo to the ignorant American: most problems are not easy to solve, and many of them require a great deal of individual effort on your own part to make it better.  So the next time you hear about something like this, don't believe the hype.

links: digg this    del.icio.us    technorati    reddit