The Cost of Bad Design

The next time someone asks me how good data visualization actually contributes to a better bottom line, I’m going to retell this story.

Some time ago, I sat in a meeting with 11 other people. We were reviewing evaluation findings, presented via charts, which were created by another researcher (not present). You know this scene. You’ve been there a hundred times. The group spent 20 minutes just trying to dissect a single chart. It wasn’t the data that was confusing. It wasn’t the chart type, either. But it was the little things like color and labeling that confused the 11 of us (an educated group, too, I should point out). So what did that bad design cost?

6 people are paid $600/day, which means we spent $150 on their confusion
1 person at $400/day = $17
1 person at $300/day =  $13
1 person at $250/day = $10
2 people at $1,000/day = $83

So one poor chart design cost our group $273, which is actually more than the daily rate (salary and benefits) for one of the meeting attendees.

Ouch. And this doesn’t include the time it took for the report author to develop the poor chart in the first place, or the time that person will put in to make the chart clearer. Bad design is expensive. Thus, investment in some professional development around good design or even consultation with an actual graphic design may literally pay off in the end.


What Bad Grouping Looks Like

Toy designers beware. Do not put your products into my little dude’s hands because I am an ex-teacher AND now I’m a professional evaluator AND I care about design. Look, just LOOK, at this deck of cards my son came home with. Notice the problem (aside from the fact they are shaped like snowmen)?

The issue, you noticed, is that the category markers (symbol and number) are too close and too similar to the actual counting symbols placed in the middle of the card.

Here’s the corresponding lesson for evaluation report authors: when we use headers to signal an organization to a report, make the header really really really clearly different from the narrative text. When working with headers, we can use font, color, and other emphasis techniques to make the headers clearly distinguished from the text.

Now, with these playing cards, they may not be able to change the color – but there are other options. The category markers could be much smaller. Or much larger.  Or positioned further away from the counting symbols in the middle. The counting symbols in the middle could also be squished closer together. In other words, there could be better grouping of elements (graphic designer Timothy Samara calls this technique “squish and separate” in his awesome book Design Elements: A Graphic Style Manual).

PS. The blogging schedule will cut in half to twice per month – I’ve got a book to put out! Sage has contracted with me to write a book on evaluation reporting, scheduled to be released in Spring 2014.

Eye Gaze & Image Placement

I went to my favorite city in the world a couple of weeks ago – Washington DC. Of course, I didn’t have enough time to explore everything I’d wanted, which included the new Museum of African American History and Culture. But I did catch the promotional signage in the Metro (and one for the Museum of African Art). Let’s pick them apart a bit, shall we?

In this first photo (sorry its so dim around the edges, but I was in the Metro after all), the designers used the power of eye gaze to both draw in the viewer and then direct the viewer’s attention toward the supporting text.

The photo of the artifact was taken at a slight angle, which gives it directionality and a bit of warmth. Think about how much flatter the ad would be if the photo had been taken fully face on. The directionality of the picture and the smart placement of the image so that eye gaze can support the text really unite the ad. When graphic designers say a design should have “unity” this is part of what they mean.

Now check out the eye gaze and directionality in the second ad.

What struck me about this one the most was that the eyes gazing toward the text seemed to belong to white people and that the black people in the ad were facing away, in a bit of an oppositional stance. Perhaps that’s on purpose, to underscore the struggle that the museum is honoring. Yes, let’s suppose that’s the case.

Whether intentional or not, these two ads remind us that there is power in the eyes. When we feature people in our evaluation reporting, position the photo so that the subjects are facing the text, focused internally, rather than literally having their backs turned to the content.

Moving Emphasis

As much as I advocate for minimalistic slide design, sometimes that just can’t happen. Sometimes you’ve gotta have a lot of words or a logic model or diagram to show. I previously showed one strategy for handling this – the Slow Reveal. But incrementally revealing parts of a complex image might not work for you. So here’s another strategy – Moving Emphasis.

Let’s say I was trying to explain to an audience how they can submit a proposal for the American Evaluation Association 2012 conference. My slide might look like this:

I need to show the whole screenshot here so I can illustrate the proposal submission process while also talking about the individual pieces. But instead of projecting this single screenshot, which would lose visual interest quickly, I can begin using arrows, circles, and other emphasis techniques, like this:

This was as simple as inserting an arrow shape in PowerPoint. Then, as I talk about the next bit, I move to the next slide. Same screenshot, different part emphasized:

I just inserted a text box and a circle shape to draw the viewer’s attention where I want it to go. This way, I can help my listeners better track my own words because the visual matches my verbal pace.

I like this red because it is so good at drawing attention (image how the effect would be lessened with black, for example), but you could even customize the moving emphasis to match your project’s colors.

When we use Moving Emphasis as a technique, it allows us to keep the same visual – which is really helpful in some circumstances, like websites. So we capitalize on attention-grabbing elements like circles and arrows to guide attention and maintain visual interest. Have fun!

How to Keep Parents in the Dark

Report card time in the Evergreen household! We have a kindergartener, so this is our first venture into decoding the marks representing his progress. Among other assessments, he’s given a standardized test throughout the year to measure his emergent literacy. Because it looks nothing like the rest of the report card materials, I’m assuming the standardized test company spits out this profile report. I am a newly minted PhD and it took me several attempts, spread a few days apart, to understand what this is trying to tell me:

This is chartjunk, in its finest form.

Let’s hone in on one as our example. This is the one in the upper left:

Here’s how I remade that graph:

These are the differences that make the difference:

1. I put the legend right in the graph. Previously, it was too far away.

2. I simplified the language. This test company was speaking test company-ese. I did this too in my first job. I worked the drive-thru (spelled like that) at McDonald’s and I asked the customer if he wanted his drink to be a “16 ounce.”  He had no idea what I was talking about, of course. Let’s use language that really matters to people.

3. I added an explanation about what this test measures, right under the title. Again, make it matter to people.

4. I removed the gridlines – just chartjunk there.

5. I made the text bigger. The original version of this document wasted the bottom half of this page. Now, a page doesn’t need to be crammed with information, but trading in more readable text caters to parents.

6. I took out the target bar (why a bar?) and replaced them with actual scores.

7. I took out the months along the x axis. What’s more important is the start of the year and the end of the year.

8. I removed the “progress monitoring assessment” circles. I assume these are the interim tests conducted by the teacher. I don’t really need to know these. These seem more important for the teacher or the school to know. For a parent report, I just need to know where my kid is at each report card, compared to where he’s supposed to be.

9. I removed the word “kindergarten” from the bottom of the x axis. I know my kid is in kindergarten. And it also says so in the information listed in the top left of the page. And under every other x axis.

10. I took out “DIBELS Next” from each y axis. Again, this is listed at the top of the page. Plus, this information doesn’t really mean anything to me as a parent.

11. I eliminated “FSF” from the y-axis. I’m confident this stands for “First Sound Fluency,” which is already labeled in the chart title. Redundancy removed.

12. I added color to each data point to more easily distinguish actual performance from the target. I also assigned them different icons.

13. I grayed out the text pertaining to the start of the year (which I saw at his first report card) and the end of the year, since those are really just reference points.

So I made 13 adjustments and now I’m betting more parents would understand their child’s progress. Sure, it’s a little splashier. Sure, it involves color (but would not necessarily require it, since I also used the icons to distinguish). But this profile report is generated through the testing company software. It doesn’t require extra work on the part of my kid’s awesome teacher. It requires that data companies think more like their consumer, conduct some usability testing, and communicate in ways that people can understand.

I Can Make A Window in Plain English

I can make a window to discuss your parallel logistical innovation.

Sound familiar? Didn’t some evaluation colleague say this at the conference this past year? Did *I* say this at the conference last year?

Nope. It is a randomly generated bit of gobbledygook from the Plain English Campaign. But it sure sounds like some evaluation-speak, doesn’t it? Maybe we aren’t quite as bad as this, but we sure do bolster ourselves with big words and things like this bit here:

You really can’t fail with knowledge-based organisational paradigm shifts.

It’s so close to home it cracks me up.

I found the Plain English Campaign via Sandra Fisher Martin’s TEDx talk on the Right to Understand. In her talk, Sandra illustrates how even the highest-educated citizens cannot decipher complex, jargon-laded documents from courts, legislators, and doctors. She argues that we have a right to understand the documents that impact our daily lives. Oh, evaluators, do you hear the call? We are the guilty.

She says these are the three things we should do when we are in the position of creating documents for public communication (as all evaluators are):

First, start with what information is most important. People aren’t going to read three pages to get to the point. In our world, that means we cut to the chase instead of burying findings on page 50, after we have so carefully explained our methods and limitations.

Second, use short sentences.

Third, use simple words. Write for Grandma in an unpatronizing way. I sit on the IRB at Western Michigan University and we require that consent forms are written at a 6th grade reading level, but depending on the study being explained it sometimes needs to be even simpler. Our evaluation reports should be held to the same level. It isn’t about looking smart to the CEO. It’s about getting evaluation used.

Here’s how to find out the grade level of your reports. In Office 2010, Click on Review>Language>Language Preferences>Proofing and click the check box by Show readability statistics.

Then run spellcheck to get the reading grade level. This blog post is at a 7th grade reading level – a little high.

Back to Sandra’s talk. She also points to the Plan English Campaign’s Crystal Mark, shown here:

For a fee, organizations can apply to the Plain English Campaign to have their document clarity approved, allowing them to affix the Crystal Mark on reproductions. While we may not all be able to afford their seal of approval, we can still heed their advice about the standards, which include:

  • the use of ‘everyday’ English;
  • consistent and correct use of punctuation and grammar;
  • an average sentence length of 15 to 20 words;
  • plenty of ‘active’ rather than ‘passive’ verbs;
  • explanations of technical terms;
  • good use of lists;
  • words like ‘we’ and ‘you’ instead of ‘the Society’ or ‘the applicant’;
  • clear, helpful headings, which stand out from the text; and
  • a good typesize and a clear typeface.

I do not foresee the same oversight (even if voluntary) for the field of evaluation, but we can internally hold ourselves accountable for communicating in Plain English (or Spanish or whatever you prefer). We can strive to speak with the acknowledgement that our audiences have a right to understand.

Icon Construction

Icons or symbols are an “easy” way to organize information throughout an evaluation report. I say “easy,” but I really mean easy for the reader. Symbol sets provide mental organizational structure. They often aren’t so easy for the report author, though. Usually it can be difficult to find a set of symbols through stock image sites that adequately fits the information being conveyed in the report.

Recently, I found this collection of icon sets – so awesome, lots in there, fairly general:

You’ll quickly see there’s a ton to dig through. Like the search for good images, it can often take more time than its worth to find a good match to your reporting needs. So here’s a quick way to make your own.

In this example, I’m trying to compare key indicators (rates of graduation, obesity, and home ownership) between Michigan, our nemesis to the south Indiana, and the national average. This is what the typical boring table would look like, yes?

I wanted to make similar-looking icons to represent each state. In the case of states, the shapes of each state could work, but purchasing state shapes via a stockphoto site was going to set me back about $30 (and think about the shape of Wyoming – would anyone really recognize it?) .

So I made icons myself in about 5 minutes. I started by typing the two-letter state abbreviations in a cool font (this is Gills Sans Condensed) and making them large:

Then I used “Text Fill” and selected white and “Text Outline” and selected black. I made the text box itself transparent, or no fill. Easy enough to get the cute outline letters, yes? For the shape, I just inserted a circle and used the “Shape Fill” and “Shape Outline” features:

I gave each state it’s own color to boost content organization even more. Then I placed the letters over the circles and grouped each letter with each circle:

The circles and the font really give it that feel of an icon set. Now I can use these to spark up my table:

Ok, I know it doesn’t do much to add icons to one table. But when we use the icon system throughout the report, we build a predictable organizational structure that speeds recognition and aids comprehension. See what I mean?


And I did it all without the aid of fancy graphic design software programs, with about 5 minutes of experimentation. Try it out for yourself! Enjoy!

Handling Colorblindness

Dealing with potential audience colorblindness isn’t as mysterious as it seems. (And as an added bonus, by way of handling colorblindness, you’ll also fortify your work against the dim bulb in the projector or the color settings on the presenter laptop that skew your established color scheme.)

One product, Color Oracle, can be downloaded right to your computer – it’ll run through your selected documents and show you what it would look like to someone with the various types of colorblindess. (Howeva, I couldn’t get it to download when I tried.)

Vischeck can do the same sort of thing online or on your computer. I uploaded a line graph produced with the default Excel colors:


As seen with red-green colorblindness:

This simulated image is pretty hard to follow, isn’t it? All the better to have good legend labeling. Or highlight the most important line with one color and make the rest of them gray. You can handle it.

Here I Vischecked a slide I replicated from all the others I’ve seen that make me cringe. The slide on the right shows what it would look like with red-green colorblindness (deuteranope).

Yeah, that’s terrible. But the original image on the left is pretty bad in the first place! So it isn’t really about the red-green color combination – it’s about the contrast of the two colors. Below I still used red and green to make my original slide, but note how much more readable it is both before and after Vischeck:

So don’t be afraid of color combinations – but do focus on making sure you have a light-dark pairing that will hold up no matter the projector quality, presentation laptop color settings, or audience impairment.

Slow Reveal

Most of the time I advocate for replacing words with images when presenting slideshows. But sometimes the slide just needs to have a lot of words, like this:

But when we have a lot of words on a slide and we’re truly trying to get people’s minds oriented toward our presentation content, overwhelming their field of vision with so much at once can cause them to just mentally check out (or check their email).

Here’s a strategy for presenting those word-filled slides that can better support audience comprehension – the slow reveal.

Basically, just reveal one point at a time. You start by constructing the full slide, as I’ve shown above. That slide has 3 objectives plus 1 title, so I’ll duplicate it such that I end up with four slides that look the same. Then, just delete one item from each slide, working backward:

This way, when I start at the beginning and work through the slide deck, I’m presenting just one point at a time. This lets my audience focus on what I’m saying (instead of reading ahead and not fully paying attention). The slow reveal also better helps the audience build the mental concept I’m trying to convey.

So ultimately, I end up with four times the number of slides. But don’t let that scare you. It’s the exact same amount of content, same number of minutes talking.

Hot tip: Make each element on the slide a separate text box. So the final slide shown above has four text boxes (and a star). Using separate text boxes makes it a million times easier to delete individual items so the slideshow can build. I’ve seen people try to keep all the objectives in one text box, while attempting the same type of slow reveal. But keeping them in the same text box often introduces the chance for spacing errors. If just one space is different during the slide build, it’ll totally get noticed (in a bad way). Just use separate text boxes.

Related hot tip: Don’t try to compile it all into one slide and use animation to reveal each element. Animation distracts when you want the audience to focus. Animation is also often slower than anticipated, and sometimes doesn’t work at all in certain webinar platforms. Just use separate text boxes.

Proper Placement of Chart Legends

In my dissertation study (probably the most boring four words to start any blog post), I saw a lot of evaluation reports that grouped all tables and graphs into the appendix. Tables, graphs, and other graphics really must be placed right next to the narrative describing them. Why? When we flip back and forth between pages, we impair working memory’s ability to make sense of the associated words and images. Truly, the ideal situation for our brains is extremely close placement. Whenever we have to seek-and-find to match up content, we hurt cognition. That’s why I was super excited to see Jeff Johnson talking about charts and their legends in his book, Designing with the Mind in Mind.

Jeff also pointed out how it is hard for people to distinguish legend colors when produced in default mode (so two lessons here). This is what the typical chart looks like in Excel, with some modifications I made to clean it up:

The blues and purples do get hard to distinguish, don’t they? So Jeff recommends enlarging the legend colors so they are both more easily distinguished and closer to the actual lines they are associated with, reducing the need for an actual eye movement between the line and it’s corresponding legend entry. You can’t really do this in Excel, though. So I faked it, by inserting a square, matching the color, and putting it right over the line in the original legend. See here:

It’s an improvement, yes. However, there’s still a lot of searching to match up each line to it’s legend entry. So I got this idea from Storytelling with Data to get rid of the legend entry altogether. In the example below, I’ve inserted text boxes with the correct word from the legend, so it is totally obvious which line goes with what and the need for the search-and-find in the legend is removed. Cognition supported. Oh yeah.