Where Are The Bad Evaluators?

I’m fresh out of a weekly discussion forum on evaluation. We talked about some of the same topics that always seem to arise: the importance of knowing your client’s organizational culture, the perils of unclear boundaries, the stigma of bad evaluation. Something struck me this time about that last point: Where are all of these bad evaluators we keep referring to?

Who are these offenders who have so tainted my client that she has built up a resistance to evaluation? Did they dry up and find jobs in academia? Did they go native and become entrepreneurs? Or did they move along to another project and continue their streak of damage?

Surely it is none of us. Surely it is all of us.

The simple fact is that we rarely know the truth of how our clients view us and our work. Most often, our contract is over when a report is delivered (rightly or wrongly). We don’t typically stick around, watching how the program uses (or doesn’t) our findings and recommendations. We have no idea if they shelf our report or use it as a doorstop. If they think we suck, we wouldn’t know. If we traumatized them, we would probably be unaware.

The second simple fact is that no evaluation is perfect. Something, somewhere is bound to fall short of expectations (rightly or wrongly). So chances are that there is something “bad” about every evaluation project… I’m sure you can see where I’m headed with this.

Of course, this question implies that there are tons of bad evaluators out there who keep leaving sour tastes in our clients’ mouths. Yet even with my frequent communication with dozens of evaluators around the country, I would not be able to pinpoint any of them as particularly “bad.” I suspect this is because we are all a little bad (and not in the good let’s-go-have-a-beer-at-lunch kind of bad). We all make mistakes that irrevocably mar our projects. But we lack a safe culture that allows evaluators to talk about their own mishaps. It is far easier and professional reputable to refer to Those Bad Evaluators Over There who hurt our chances at doing good work.

Where are the bad evaluators? It is probably safe to say there is one typing this blog post. And one reading it, too.

Advertisements
Previous Post
Leave a comment

2 Comments

  1. Bad evaluators include me, you, even most if not all of the BIG NAMES in evaluation, meaning we all can (and need to) improve. I’ve often thought of proposing an AEA session called “My Evaluation Sux” where participants could share their evaluation pitfalls (whether design, instruments, etc.) and the audience would suggest ways they can make improvements – think of the learning that would go on! I would love it, especially as an independent evaluator without in-house colleagues to call upon.

    Reply
    • I’d love it, too! We tried something similar to that at our Evaluation Cafe at WMU – but no one was willing to be open about their mistakes, and this was among colleagues. Yet in a larger sense, I think people love to learn from others’ mistakes…
      Anyway, that would be a great session title. I’d love to be in the audience of that one!

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: