Where Are The Bad Evaluators?

I’m fresh out of a weekly discussion forum on evaluation. We talked about some of the same topics that always seem to arise: the importance of knowing your client’s organizational culture, the perils of unclear boundaries, the stigma of bad evaluation. Something struck me this time about that last point: Where are all of these bad evaluators we keep referring to?

Who are these offenders who have so tainted my client that she has built up a resistance to evaluation? Did they dry up and find jobs in academia? Did they go native and become entrepreneurs? Or did they move along to another project and continue their streak of damage?

Surely it is none of us. Surely it is all of us.

The simple fact is that we rarely know the truth of how our clients view us and our work. Most often, our contract is over when a report is delivered (rightly or wrongly). We don’t typically stick around, watching how the program uses (or doesn’t) our findings and recommendations. We have no idea if they shelf our report or use it as a doorstop. If they think we suck, we wouldn’t know. If we traumatized them, we would probably be unaware.

The second simple fact is that no evaluation is perfect. Something, somewhere is bound to fall short of expectations (rightly or wrongly). So chances are that there is something “bad” about every evaluation project… I’m sure you can see where I’m headed with this.

Of course, this question implies that there are tons of bad evaluators out there who keep leaving sour tastes in our clients’ mouths. Yet even with my frequent communication with dozens of evaluators around the country, I would not be able to pinpoint any of them as particularly “bad.” I suspect this is because we are all a little bad (and not in the good let’s-go-have-a-beer-at-lunch kind of bad). We all make mistakes that irrevocably mar our projects. But we lack a safe culture that allows evaluators to talk about their own mishaps. It is far easier and professional reputable to refer to Those Bad Evaluators Over There who hurt our chances at doing good work.

Where are the bad evaluators? It is probably safe to say there is one typing this blog post. And one reading it, too.

If It Ain’t Broke

I found the cutest old-man optometrist. He puttered around the room, in cute old man fashion. He had a little cute old man mantra: “if it ain’t broke…”

Him: Are your contacts working okay for you?

Me: Sure, I guess.

Him: Well, if it ain’t broke…

Me: But aren’t you going to check my eyes???

He eventually did. But he must have repeated his mantra three or four more times during our appointment together.

It was while I was waiting for my eyes to dilate that I realized how “if it ain’t broke…” might be the worst phrase for an evaluator to hear. Why wait until things are broken to start fixing them? Waiting until things are broke means enduring a period of decline, a period of broken-ness, and a period of rebuilding to get things back up at the same operating level as previously. That sort of downtime impacts an organization’s productivity, effectiveness, and bottom line. When there are clear patterns and signposts established (especially in the eyecare industry), it would be much more efficient to watch out for those early warning signals and take action, rather than wait until it is broke. This is why evaluators are good at pattern recognition.

Now whenever I hear “if it ain’t broke…,” I cringe. Must be hard to examine my eyes that way.