When Not to Use Data Driven Decision Making

In 1993 two artists set out to make a data-driven painting. I’m not kidding. They contracted with a well respected research firm, ran a call center in the midwest, and used telephone interviews to gather opinions about the type of artistic content and composition people preferred.

The poll used a stratified (by state) random sample to snag 1,001 responses, which had nearly perfect distributions in terms of income, gender, race, education, and political beliefs. Evaluators would be hard pressed to obtain the same representativeness in most of our work.

The artists calculated simple descriptive statistics and used the analysis to create a painting which reflected the desired artwork of the majority. The result is the painting below, America’s Most Wanted. Imagine it dishwasher-sized. And yes, that’s George Washington and a random group of anonymous people in the picture.

The artists debuted the painting in Soho, published it in magazines and paraded it around the country in town hall meetings. The main sense of those in the crowd, upon digesting their data-driven art, was… embarrassment. The painting was quaint. Provincial. Even ugly. One of the artists, Komar, said, “It is my hope that people who come to see our Most Wanted paintings will become so horrified that their tastes will gradually change, and another poll would gather different results for creation of paintings that do not resemble blue landscapes.”

Now this poll was examining tastes, but the same problem occurs all over the place when we have the good intention to making data-driven decisions. Henry Ford said, “If I had asked people what they wanted, they would have said faster horses.” In evaluation we often run into this predicament with needs assessments. It isn’t as simple as asking people what they need. Some evaluators have wised up to this idea, so they observe behavior instead and surmise needs from their observations. But what would the artists have observed in this study? About 50 percent of the respondents said they visited an art museum never or less than once per year. Not much to observe. Sometimes the smartest decisions come from an innovator who knows the audience base well, not from the audience itself.

By the way, the painting below is a representation of all of the worst rated features, America’s Least Wanted.

The poll was replicated in several other countries around the globe, with nearly similar results. See the resulting paintings for other locations and explore the data at http://awp.diaart.org/km/

Advertisements
Leave a comment

5 Comments

  1. Thanks Stephanie, this is great! My takeaway will be adding George Washington and a seascape to all of my cartoons from now on.
    Also, what’s up with Holland? Talk about an outlier…

    Reply
  2. Stuart Henderson

     /  June 26, 2012

    Great blog post–interesting and informative. It would make a nice Ignite session.

    Reply
  3. Very nice post Stephanie, putting observation skills to use to ‘predict’ needs in a sense. When you take the time to get to know the people/ clients you are working with that helps quite a bit with predicting needs,likes and dislikes, beyond the assessments like you noted. This has helped me to get a richer understanding of the people I work with to inform the work that I do.

    Karen

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: