How to Keep Parents in the Dark

Report card time in the Evergreen household! We have a kindergartener, so this is our first venture into decoding the marks representing his progress. Among other assessments, he’s given a standardized test throughout the year to measure his emergent literacy. Because it looks nothing like the rest of the report card materials, I’m assuming the standardized test company spits out this profile report. I am a newly minted PhD and it took me several attempts, spread a few days apart, to understand what this is trying to tell me:

This is chartjunk, in its finest form.

Let’s hone in on one as our example. This is the one in the upper left:

Here’s how I remade that graph:

These are the differences that make the difference:

1. I put the legend right in the graph. Previously, it was too far away.

2. I simplified the language. This test company was speaking test company-ese. I did this too in my first job. I worked the drive-thru (spelled like that) at McDonald’s and I asked the customer if he wanted his drink to be a “16 ounce.”  He had no idea what I was talking about, of course. Let’s use language that really matters to people.

3. I added an explanation about what this test measures, right under the title. Again, make it matter to people.

4. I removed the gridlines – just chartjunk there.

5. I made the text bigger. The original version of this document wasted the bottom half of this page. Now, a page doesn’t need to be crammed with information, but trading in more readable text caters to parents.

6. I took out the target bar (why a bar?) and replaced them with actual scores.

7. I took out the months along the x axis. What’s more important is the start of the year and the end of the year.

8. I removed the “progress monitoring assessment” circles. I assume these are the interim tests conducted by the teacher. I don’t really need to know these. These seem more important for the teacher or the school to know. For a parent report, I just need to know where my kid is at each report card, compared to where he’s supposed to be.

9. I removed the word “kindergarten” from the bottom of the x axis. I know my kid is in kindergarten. And it also says so in the information listed in the top left of the page. And under every other x axis.

10. I took out “DIBELS Next” from each y axis. Again, this is listed at the top of the page. Plus, this information doesn’t really mean anything to me as a parent.

11. I eliminated “FSF” from the y-axis. I’m confident this stands for “First Sound Fluency,” which is already labeled in the chart title. Redundancy removed.

12. I added color to each data point to more easily distinguish actual performance from the target. I also assigned them different icons.

13. I grayed out the text pertaining to the start of the year (which I saw at his first report card) and the end of the year, since those are really just reference points.

So I made 13 adjustments and now I’m betting more parents would understand their child’s progress. Sure, it’s a little splashier. Sure, it involves color (but would not necessarily require it, since I also used the icons to distinguish). But this profile report is generated through the testing company software. It doesn’t require extra work on the part of my kid’s awesome teacher. It requires that data companies think more like their consumer, conduct some usability testing, and communicate in ways that people can understand.

I Can Make A Window in Plain English

I can make a window to discuss your parallel logistical innovation.

Sound familiar? Didn’t some evaluation colleague say this at the conference this past year? Did *I* say this at the conference last year?

Nope. It is a randomly generated bit of gobbledygook from the Plain English Campaign. But it sure sounds like some evaluation-speak, doesn’t it? Maybe we aren’t quite as bad as this, but we sure do bolster ourselves with big words and things like this bit here:

You really can’t fail with knowledge-based organisational paradigm shifts.

It’s so close to home it cracks me up.

I found the Plain English Campaign via Sandra Fisher Martin’s TEDx talk on the Right to Understand. In her talk, Sandra illustrates how even the highest-educated citizens cannot decipher complex, jargon-laded documents from courts, legislators, and doctors. She argues that we have a right to understand the documents that impact our daily lives. Oh, evaluators, do you hear the call? We are the guilty.

She says these are the three things we should do when we are in the position of creating documents for public communication (as all evaluators are):

First, start with what information is most important. People aren’t going to read three pages to get to the point. In our world, that means we cut to the chase instead of burying findings on page 50, after we have so carefully explained our methods and limitations.

Second, use short sentences.

Third, use simple words. Write for Grandma in an unpatronizing way. I sit on the IRB at Western Michigan University and we require that consent forms are written at a 6th grade reading level, but depending on the study being explained it sometimes needs to be even simpler. Our evaluation reports should be held to the same level. It isn’t about looking smart to the CEO. It’s about getting evaluation used.

Here’s how to find out the grade level of your reports. In Office 2010, Click on Review>Language>Language Preferences>Proofing and click the check box by Show readability statistics.

Then run spellcheck to get the reading grade level. This blog post is at a 7th grade reading level – a little high.

Back to Sandra’s talk. She also points to the Plan English Campaign’s Crystal Mark, shown here:

For a fee, organizations can apply to the Plain English Campaign to have their document clarity approved, allowing them to affix the Crystal Mark on reproductions. While we may not all be able to afford their seal of approval, we can still heed their advice about the standards, which include:

• the use of ‘everyday’ English;
• consistent and correct use of punctuation and grammar;
• an average sentence length of 15 to 20 words;
• plenty of ‘active’ rather than ‘passive’ verbs;
• explanations of technical terms;
• good use of lists;
• words like ‘we’ and ‘you’ instead of ‘the Society’ or ‘the applicant’;
• clear, helpful headings, which stand out from the text; and
• a good typesize and a clear typeface.

I do not foresee the same oversight (even if voluntary) for the field of evaluation, but we can internally hold ourselves accountable for communicating in Plain English (or Spanish or whatever you prefer). We can strive to speak with the acknowledgement that our audiences have a right to understand.

Icon Construction

Icons or symbols are an “easy” way to organize information throughout an evaluation report. I say “easy,” but I really mean easy for the reader. Symbol sets provide mental organizational structure. They often aren’t so easy for the report author, though. Usually it can be difficult to find a set of symbols through stock image sites that adequately fits the information being conveyed in the report.

Recently, I found this collection of icon sets – so awesome, lots in there, fairly general:

http://theelearningcoach.com/resources/icon-collection/

You’ll quickly see there’s a ton to dig through. Like the search for good images, it can often take more time than its worth to find a good match to your reporting needs. So here’s a quick way to make your own.

In this example, I’m trying to compare key indicators (rates of graduation, obesity, and home ownership) between Michigan, our nemesis to the south Indiana, and the national average. This is what the typical boring table would look like, yes?

I wanted to make similar-looking icons to represent each state. In the case of states, the shapes of each state could work, but purchasing state shapes via a stockphoto site was going to set me back about $30 (and think about the shape of Wyoming – would anyone really recognize it?) .

So I made icons myself in about 5 minutes. I started by typing the two-letter state abbreviations in a cool font (this is Gills Sans Condensed) and making them large:

Then I used “Text Fill” and selected white and “Text Outline” and selected black. I made the text box itself transparent, or no fill. Easy enough to get the cute outline letters, yes? For the shape, I just inserted a circle and used the “Shape Fill” and “Shape Outline” features:

I gave each state it’s own color to boost content organization even more. Then I placed the letters over the circles and grouped each letter with each circle:

The circles and the font really give it that feel of an icon set. Now I can use these to spark up my table:

Ok, I know it doesn’t do much to add icons to one table. But when we use the icon system throughout the report, we build a predictable organizational structure that speeds recognition and aids comprehension. See what I mean?

and

And I did it all without the aid of fancy graphic design software programs, with about 5 minutes of experimentation. Try it out for yourself! Enjoy!

Handling Colorblindness

Dealing with potential audience colorblindness isn’t as mysterious as it seems. (And as an added bonus, by way of handling colorblindness, you’ll also fortify your work against the dim bulb in the projector or the color settings on the presenter laptop that skew your established color scheme.)

One product, Color Oracle, can be downloaded right to your computer – it’ll run through your selected documents and show you what it would look like to someone with the various types of colorblindess. (Howeva, I couldn’t get it to download when I tried.)

Vischeck can do the same sort of thing online or on your computer. I uploaded a line graph produced with the default Excel colors:

Original:

As seen with red-green colorblindness:

This simulated image is pretty hard to follow, isn’t it? All the better to have good legend labeling. Or highlight the most important line with one color and make the rest of them gray. You can handle it.

Here I Vischecked a slide I replicated from all the others I’ve seen that make me cringe. The slide on the right shows what it would look like with red-green colorblindness (deuteranope).

Yeah, that’s terrible. But the original image on the left is pretty bad in the first place! So it isn’t really about the red-green color combination – it’s about the contrast of the two colors. Below I still used red and green to make my original slide, but note how much more readable it is both before and after Vischeck:

So don’t be afraid of color combinations – but do focus on making sure you have a light-dark pairing that will hold up no matter the projector quality, presentation laptop color settings, or audience impairment.

Slow Reveal

Most of the time I advocate for replacing words with images when presenting slideshows. But sometimes the slide just needs to have a lot of words, like this:

But when we have a lot of words on a slide and we’re truly trying to get people’s minds oriented toward our presentation content, overwhelming their field of vision with so much at once can cause them to just mentally check out (or check their email).

Here’s a strategy for presenting those word-filled slides that can better support audience comprehension – the slow reveal.

Basically, just reveal one point at a time. You start by constructing the full slide, as I’ve shown above. That slide has 3 objectives plus 1 title, so I’ll duplicate it such that I end up with four slides that look the same. Then, just delete one item from each slide, working backward:

This way, when I start at the beginning and work through the slide deck, I’m presenting just one point at a time. This lets my audience focus on what I’m saying (instead of reading ahead and not fully paying attention). The slow reveal also better helps the audience build the mental concept I’m trying to convey.

So ultimately, I end up with four times the number of slides. But don’t let that scare you. It’s the exact same amount of content, same number of minutes talking.

Hot tip: Make each element on the slide a separate text box. So the final slide shown above has four text boxes (and a star). Using separate text boxes makes it a million times easier to delete individual items so the slideshow can build. I’ve seen people try to keep all the objectives in one text box, while attempting the same type of slow reveal. But keeping them in the same text box often introduces the chance for spacing errors. If just one space is different during the slide build, it’ll totally get noticed (in a bad way). Just use separate text boxes.

Related hot tip: Don’t try to compile it all into one slide and use animation to reveal each element. Animation distracts when you want the audience to focus. Animation is also often slower than anticipated, and sometimes doesn’t work at all in certain webinar platforms. Just use separate text boxes.

Proper Placement of Chart Legends

In my dissertation study (probably the most boring four words to start any blog post), I saw a lot of evaluation reports that grouped all tables and graphs into the appendix. Tables, graphs, and other graphics really must be placed right next to the narrative describing them. Why? When we flip back and forth between pages, we impair working memory’s ability to make sense of the associated words and images. Truly, the ideal situation for our brains is extremely close placement. Whenever we have to seek-and-find to match up content, we hurt cognition. That’s why I was super excited to see Jeff Johnson talking about charts and their legends in his book, Designing with the Mind in Mind.

Jeff also pointed out how it is hard for people to distinguish legend colors when produced in default mode (so two lessons here). This is what the typical chart looks like in Excel, with some modifications I made to clean it up:

The blues and purples do get hard to distinguish, don’t they? So Jeff recommends enlarging the legend colors so they are both more easily distinguished and closer to the actual lines they are associated with, reducing the need for an actual eye movement between the line and it’s corresponding legend entry. You can’t really do this in Excel, though. So I faked it, by inserting a square, matching the color, and putting it right over the line in the original legend. See here:

It’s an improvement, yes. However, there’s still a lot of searching to match up each line to it’s legend entry. So I got this idea from Storytelling with Data to get rid of the legend entry altogether. In the example below, I’ve inserted text boxes with the correct word from the legend, so it is totally obvious which line goes with what and the need for the search-and-find in the legend is removed. Cognition supported. Oh yeah.

What Exactly is Happening in this 3D Chart?

When giving advice about chart styles, one of my friends likes to say “save 3D for the movies.” She’s right – research shows that 3D charts actually slow reader comprehension. I’ve long advocated for the use of ChartTamer as a friendly way to restrict data visualization to that which will actually support reader cognition. In a recent webinar I heard Cole Nussbaumer at Storytelling with Data give another compelling reason why we shouldn’t graph in 3D – Excel doesn’t use the front line or the back line of the 3D bar when aligning with the y axis – it graphs the midpoint of the bar. What??? Yes, she said Excel uses the middle of the 3D rendering as the determiner of the data point. I didn’t believe it. How could anything so antithetical to cognition actually survive in this sink-or-swim society? So I tried it myself:

Here is a 3D column chart of some fake data about customer satisfaction with prices in various departments of a natural health food store, before and after their physical move to a new location:

Now I know I’m breaking some serious graphing rules here, but stick with me. I super enlarged the y axis and honed in on one range of it so that we could really see where the gridlines line up with the tops of the columns.

See the blue bar in Produce? The actual data point for that blue bar is 85. Neither the front line, the back line, nor the midpoint of the top of the column are at 85. See the blue bar for Beauty? The actual data point there is 83. Again, none of the possible points of measurement on the 3D column are accurately expressing 83. Maybe more like 82.75, but definitely clearly absolutely not reaching the gridline marking 83.

I selected the exact same raw data table and created a similar column chart, just in 2D:

This time the blue bar in Produce is exactly at 85 and the blue bar in Beauty accurately represents the data point of 83.

I wasn’t able to reproduce Cole’s assertion that Excel uses the midpoint of the 3D column. But it is pretty clear that the audience misinterpretation often cited in research that results from the use of 3D charting is due to more than the complexity of analyzing in the third dimension – it is also because the columns simply aren’t accurately visualized.

My 2011 Personal Annual Report

Worst Font Contest

Two years ago I ran this tiny contest on my social networking platforms for the worst font ever. Far and away, Comic Sans was the winner. Here are the others that deserved mention, along with the messages they tend to communicate:

Jokerman: Also says “Fajitas Tonite!” (spelled with “nite” instead of “night,” for sure)

Papyrus: Also says “yoga studio” or “Egypt”

Okay, I’m joking. Kinda.

But with these examples we can see how our font choices communicate for us, outside of the actual words we type with these fonts. Still, these examples are pretty easy targets. And they are two years old!

So dig a little deeper and type your new nominee for Worst Font Ever into the comments below. Winner gets a high-five.

PS. Font nerds unite!

 

 

Kuler/Color

Here’s a procedure I use all the time to help me select color combinations for my reporting. It makes use of this great, free, online program that takes all the scientific color theory stuff and translates it for those of us without a MFA.

First, I head to my client’s website and take a screenshot of their logo.

Then I go to this cool program, called Adobe Kuler (pronounced “color,” I’m pretty sure) at kuler.adobe.com. This is a color picking website. Once you sign in, you can upload your image. Here I have my client’s logo I just stole from their website. And the program picks out the exact colors from the logo.

Once I save, I can click a little sliderule icon that gives me the RGB color codes. With those color code numbers, I can customize the palette of my word-processing and presentation software programs to match my colors to those of my client.

I just write down those RGB color codes and head over to Word. Here I’m showing a screenshot where I transferred the RGB codes from Kuler into the Custom Colors option (right-click on, say, Heading 1 in the Styles menu and select Modify, then click on the arrow by the color menu, go down to More Colors, and click on the Custom tab).

In the same area you can modify the font, justification, etc. Once you have set up your page layout the way you’d like, go to the Themes button and select “Save Current Theme.” This will allow you to access to these settings on other Word documents and even in other Office programs like Excel or PowerPoint. I like to name the theme after my client and use the theme consistently in all of my work with them.

Why go to all that trouble? Because now you have an intentional tone that communicates consistency and belonging with your client’s work, and that’s what evaluation should be.