welterweightUX

The company I work for has 1,500+ish employees. We’re geographically dispersed and run the gamut from call center agents to finance to developers to bus drivers to you get the picture.

We produce websites for varying audiences. But most of what we, as a User Experience team, have been focusing on lately has been the overhaul of several of our biggest consumer-facing sites.

Occasionally, we’re encouraged to tap internal participants for usability testing in lieu of recruiting external ones. We are 100%, definitely not alone in this. Many companies take this approach. The reason why? It makes total sense from a business standpoint.

The perfectly valid reasons for going this route are simple:

• There are a lot of people wading around in the testing pool
• A large array of demographic definitions are represented
• There are varying levels of knowledge of and expertise with the sites being redesigned
• Really, who can beat the, um, fiscal cheapness of testing people who are already working for you anyway? 

It’s always great to get feedback on a site that’s being developed, from whoever will offer up an opinion. But there are definite dangers with shirking external testing to get the opinions of the easiest and cheapest to reach, the internals.

Here are some of the pitfalls associated with testing internally:

• Internal users have an intense familiarity with the industry
• They have insider knowledge of the site being redesigned and/or the product build behind the site
• There’s an inherent bias forged by corporate politics, agendas, and allegiances
• There are standing historical preferences and prejudices
• There are a decidedly finite number of testing resources to draw from. Fresh viewpoints will soon be made stale by overuse.

If you’re testing externally, you get:

• New, casual, or politically-disconnected users
• Those who are can compare and contrast your product to their favored marketplace competitors
• A diversity of actual consumer voices
• Lack of internalize baggage and assumptions

That’s not to say that you should never test internally. You should, especially early on. Internals can help uncover sore thumb or pebble-in-the-shoe issues. You can then quickly fix those issues before braving the blessed avalanche of  external testing.

Later in the process, internals can also be used to validate more detailed, nuanced usability issues. The more eyes and opinions, the deeper the proof sinks into the bowl of pudding.

Buuuut…internal users should never be the sole validation point for an external product. To let that happen would just be a huge flipping of the finger at the brave soldiers who will use your product where it counts: The sloppy battlefield of the real world.

I wrote an article called (Not) Seducing Users for my friend Nicole’s content strategy blog, contente.org. It’s about setting restrictions on your writing for the sake of your users.

Here’s a snippet:

Oddly, after a while, you’ll internalize the restriction. You’ll start creating paired back prose at first stab. It’ll become habit. You’ll write better when you know you can write better, always. [Read the whole shebang]

Am I the only person in the world who actually likes Google Buzz? Sometimes it seems that way.

That said, I don’t find it all that useful. Why? Because the information I see on it is insanely redundant.

The whole connected sites concept is a great, but most peoples’ lack of Buzz adoption (and adoration) make it so you’re forced to see the same stuff by the same people in multiple places.

Here’s an example: My friend Jennie takes rad photos. She posts them on Flickr, which she’s connected to Buzz. She’s the only person I follow who connects the two sites. So to see my other friends’ photos, I need to go straight to Flickr. Where I also see Jennie’s photos again.

Kind of pointless, eh?

I’m going to go out on a limb and say that the main reason Buzz sucks is because we—all of us—aren’t using it right. By “right” I mean “enough”.

It would be awesome if we could have it as a clearinghouse for all of our social media and blogging activities. And also a place to microblog independently. Basically, one site for all of the shit we’re interested, all of the stuff we want to keep track of and comment on, all our likes and loves and things of note. Basically, huggy RSS on collaborative steroids.

Unfortunately, I don’t see this happening. Buzz is already becoming a low whine, puttering its way into the internet ghetto. Are you sad at all? Did you even notice?

Half inspired by the $5 Guerrilla User Test concept, half by the need to quickly and cheaply get user feedback to quell internal debate, @yellowledbedder and I hit up AJ Bombers in downtown Milwaukee last Wednesday afternoon. We were armed only with a pair of laptops and a question: Which approach is more effective, user?

A bit of background
We’ve been working on developing a new section for our consumer-facing websites. The section has refinement tools that allows users to get to more qualified, personalized search results.

In the consumer world, refinements traditionally cascade down the left side of the page. Take, for example, how CarMax does it:

But during our research, we found that some of the biggies, like Zappos, are starting to place these tools across the top of the page, above the products, like this:

The evils of untested innovation
Innovation is good. Don’t get me wrong. Our UX team, some of our designers, and most of the business teams personally preferred the new-fangled horizontal approach.

But UXers, being persnickety as we are, said ardently that our preferences couldn’t make the call. Users had to decide. So we pitched the idea for a guerrilla study and got energetic support from the organization to go ahead.

Here’s what we did
We came up with the idea of testing this way on Friday afternoon, for testing the following Wednesday.

These were our steps, from concept to execution.

Friday:

1. Got approval to buy participants lunch (up to $15) for 15 minutes of their time.
2. Contacted @ajbombers (a Milwaukee restaurant well-connected in the social media realm) to see if we could take up table space for a few hours.
3. Sent out a tweet that included my email address for participants to contact me for more info.
4. Had AJ Bombers retweet my tweet.

Within minutes, I had several responses. I contacted the respondents immediately to let them know I’d be following up Monday with details.

When I got into work Monday, there were several more emails waiting! The rest of the day went something like this:

1. Contacted respondents to let them know available times, asking them to identify the 3 times that worked best with their schedule.
2. Began slotting participants into available times as they responded.
3. Followed up by confirming times and giving details about where we’d be and generally what they could expect.

Testing day, Wednesday, went like this:

1. Tied up all loose recruiting ends, like getting zero hour substitutions for people who could no longer attend. Social media made this super easy.
2. Got to the restaurant when they opened, set up, ordered a soda, and waited for our first participant.
3. Tested 10 participants from between 11:20am and 3pm-ish.

The result
What happened was not at all what we expected. Preference for horizontal vs vertical was split precisely 50/50. There was no trend based on age, gender, profession, etc. It was a pure matter of preference.

Now, this may sound inconclusive. But really, it’s anything but.

We can now confidently tell our clients that they can use the placement that best suits their business needs. Both approaches were understandable, noticeable, and usable.

Oh, and the total cost of the test (aka, the restaurant tab, including tip) was under $200.

How often can you walk away with such inexpensive surety? Not very.

I quickly plugged our findings into an executive summary, which we shared with the larger team Friday morning. They were pleased as punch with the results.

Gorilla vs guerrilla vs go-for-reala testing
My partner-in-UX-crime, Mike, recently wrote a blog post outlining the difference between gorilla testing (old, classic, lab testing) and guerrilla testing (agile, informal, out “in the field”). It’s worth a read.

I agree that different realms of technology require different methodologies. But if you’re looking to get quick feedback on a consumer-facing website or app, meeting users out in the real world— which is rife with verisimilitudinous distraction—is bar-none the best thing you can do. Plus it’s cheap. You get the answers you need, now. No expensive recruiting, no huge pay-outs, no sterile lab settings. Just down and dirty results.

Really, I see this as more than just guerrilla testing. I see it as go-for-reala user testing. Because it’s a real deal (also, I just really like puns). Yo? Ahem…

Shout outs
Thanks to AJ Bombers for being amazingly accommodating! And thanks to our participants for being great guinea pigs and proving the effectiveness of the go-fo-reala approach to user testing.

My credit union, UWCU, kind of rules.

In addition to having shockingly low credit card/loan interest rates and stellar customer service (I’ve never once been frustrated), their web branch rolled out a great new set of money management tools last month. These tools give users the ability to keep track of how they spend their money by adding personalized categories to transactions.

Basically, when you spend money, the system applies a label to it. It looks like this:

The first time you make a transaction with a source, the system runs logic to auto-categorize it for you. After it posts to your account, you can log in and use the Category drop-down to change the category.

The next time you make a transaction with that source, the web branch remembers how you categorized it and will apply that same label.

The system has a set list of labels, but you can also add your own to make them relevant to how you spend your dough.

Awesome, eh?

Now if only I could get them to start automatically depositing thousands of dollars in my account each week, I’d be content as a peach.

Anti-redesign
There’s been a lot written lately about shirking comprehensive site redesigns in favor of slower, incremental roll outs of enhancements.

The reasons for this are simple: You can be nimbly reactionary. You can test out a concept outside the usability lab (which has has its inherent awkwardness) to see how it performs. If it does well, you can sit on it. Or better, refine it further to make it even better. If it fails, you can scrap it, start from scratch, then roll out something altogether new.

And then continue the circle.

I’m an advocate of this. It makes sense. Keeps you on your toes and honest about the fact that good enough can always be improved.

A different idea altogether…
A few weeks ago, I was the website of a local optician, Optix on Downer. I’m not sure how long their site has been live, but if you look at it, you’ll notice the following image in the left gutter near the page footer:

When you click it, you get the old site. Strange, eh? But interesting.

Part of me likes this idea. It says: Visitor, we know we may have thrown you a wrench by changing stuff around on you, but we like you, so we’ve provided you with an out to get back to the site you’re used to.

Part of me hates this idea. It says: Visitor, you’re a stodgy, unchanging beast. We want to make you happy with our new site, but you’re awfully hard to please. Also, we know there’s something wrong with this new site, so here’s an out. Feel free to never take the time to explore this new thing we spent so much time and money developing. We’re not going to make enhancements anyway.

Pros and cons of offering an out
Optix almost certainly doesn’t have a dedicated web department who can review analytics, do heuristic reviews, or use the bevy of other techniques out there to determine what’s failing and respond to it.

They’ve come up with an interesting approach to the full redesign, by allowing visitors to access the older, possibly more familiar site. But by doing this, they’re making new users ask too many questions: What am I missing on this new site that was on the old one? Do I have to troll that one, too, in order to find the info I’m looking for?

The new site has crisp, concise content, a sensible content architecture, and is pretty dapper looking to boot. Optix should be happy with how they’ve evolved and not focus on continually paying homage to the flashy dinosaur wreckage of their past.

The truth is, sometimes a full, significant redesign makes sense. I feel kind of icky saying that, but I guess occasionally you need to come face-to-face with the alteratives before you can face the facts.

Still reeling from the incredible accessibility presentation I attended last night.

While demonstrating how a screen reader works, Scott Mayer, a blind man from American Family’s usability department, almost off-handedly said, “A usable site IS an accessible site.”

To me, that was the night’s most cogent statement.

As UX people, I wondered, do we truly give accessibility the consideration it deserves? Especially given that (and here’s where I become US-centric) nearly 20% of people in the US have some sort of disability. Think about that. 20%. Are we helping to make the internet easy to use for such a huge chunk of the population?

Ah, but me, I’m not a numbers person. To me, this is a more tangible way to look at it: If you lost your eyesight today or were seriously injured in an accident or were stricken with a debilitating disease or lost your hearing, etc, would you be able to use the site you’re designing or redesigning?

And think outside the stuff you create.

Say there’s a radio station that has lots of shows you like to stream online. Over time, you lose your ability to hear. Does the station’s site support you? If not, do you just move on, never looking back, la-di-da-ing as you go? I doubt it. You’d demand a transcript or something equally concrete to keep you connected to the programs you love.

Carol Voss from IndependenceFirst, where the presentation was held, said that her co-workers, 50+% of whom are disabled, jokingly refer to her as a TAB (Temporarily Able Bodied). That’s a compelling way to look at it. It means we’re all seconds away from losing everything we take for granted. And in the event that happens, we should never be wrenched away from the resources we value.

If, as we claim, our goal is to make usable and even user-friendly websites, we have to expand our sense of what defines a user. Sure, it’s impossible to design for everybody all the time, but there are definite considerations that can be made within every project that will make life easier for the person that you could easily become.

In an effort to get usability even more deeply engrained all of our company’s projects, I recently put together a flow chart of usability touch points. This is what I came up with—we’re not 100% there yet, but this is what my vision for the future looks like.

How does this match up to what you do? Am I not doing something I should be doing? Or doing something I shouldn’t be? If so, leave a comment!

Here’s an admission: I don’t read usability books.

This isn’t a proclamation of anti-intellectualism; it’s the result of a deficit of time. In the past I’ve worried a lot about not having read all the books I “should” read, but the transition into this new decade has me wondering: Does it really matter?

Think about the blistering speed of change these days (duh). Think about the technologies at our fingertips that allows us keep up on morphing trends in usability. We have Twitter, RSS feeds, mailing lists, white papers, and so on.

Hopefully, all usability people are actively reading and sharing articles socially, attending conferences and webinars when budget allows it, participating in local usability group presentations—oh, not to mention spending their day deep in the trenches. Given this, is it crucial in 2010 to hunker down with usability books at night, highlighter in hand, in order to keep in tip-top professional shape?

And is it critical to have read the “classic” usability tomes to know what’s what?

If you’ve gleaned and internalized the underlying principles of usability, allow yourself to be flexible to the intricacies of context and are open to change, do you really need books, where information/ideas can be dated while the ink’s still drying?

This recent critique of Jakob Neilsen’s new eyetracking book illustrates my point. As does a few coworkers’ criticism of the need for the recent Smashing Magazine book about best practices in modern Web design (not that it doesn’t look gorgeous).

Does the inherent datedness of print nullify its importance or at least our need to consume it?