Traditional Usability Testing Might Be Causing You Problems

Hi all.
When you hear somebody talking about testing with users, what do you think about? You are likely picturing traditional usability testing. In it, a participant navigates a website with a facilitator while being recorded.

If you’re not quite as old and decrepit as I am, you might imagine remote usability testing being conducted over a platform like Zoom or Lookback. Nevertheless, you are probably envisioning the same fundamental interaction.

The Pros and Cons of Facilitated Usability Testing

For a long time, this was how I conducted the majority of my user testing. However, this approach has big downsides. These include the complexity of scheduling sessions and the time needed to run them.

Facilitated usability testing is a great chance to get to know users and ask in-depth questions. But, I use it less and less due to the downsides. Most of the time, I test to answer a specific question. Traditional usability testing is rarely the best way to get my answer.

Today, there are many other options. They let us understand user behavior with much less effort and time than traditional usability testing. However, when discussing testing, most people see it as time-consuming and hard to organize. So, they can be reluctant to run testing.

Alternatives to Facilitated Usability Testing

With that in mind, I thought it might be useful to share some of the testing methods that I tend to favor and when I use them. Hopefully, this list will prove helpful to you.

• In unfacilitated testing, a user completes a task using a prototype. They do so while speaking out loud. No facilitator is present. This method is ideal for assessing if users can successfully perform a specific action.
• Surveys can answer many questions. They can show if a user will like the design. They can also show what a user expects from your website. I rarely undertake a project without incorporating a survey at some stage.
• A top task analysis is a type of survey used exclusively to identify what users want to know or do on your website and prioritize these tasks. It is an invaluable tool for determining the information or actions your user interface should emphasize.
• A first-click test is a great tool to use when all you have is a static mockup. It helps determine if users comprehend the interface. They can show how they would navigate it without needing a full prototype.
• A 5-second test is another test you can run using a single-page mockup, like a site homepage. It helps determine if users understand the page and if they notice critical page elements, such as your call to action.
• Preference testing involves users choosing their preference between versions of a design. They may also choose between your design and your competitors. I will sometimes run a preference test if I want to prove that users will be drawn to a particular design aesthetic.
• Card sorting can ensure that your categories make sense to users. It works for information architecture or product categories. Whenever I work on a site’s structure, I almost always run some form of card sorting.
• Tree testing is used to validate an existing information architecture to ensure that users can find content. I use it either to check a site structure I have created or to prove that an alternative structure will not work.
• Heatmaps provide aggregated data on user behavior across a website. They can highlight behaviors such as clicks, scrolling, misclicks, rage clicks, and more. I use them primarily to identify underperforming pages.
• Session recordings are recordings of real users interacting with your website or app. They are helpful in identifying problems with under-performing pages.
• A/B testing allows you to trial small changes on your website to test their performance. It is ideal for testing how changes to text, imagery, or design elements impact user behavior.

Would you be interested in me carrying out some user testing on your app or site? If so, drop me an email.

Think Twice

These tests can be set up and run much faster than traditional usability testing. They are better for addressing specific concerns or questions. Moreover, most of these tests can be conducted either for free or by utilizing a free tier of a paid product.

So next time you consider testing and conclude that you don’t have the time or resources, I would encourage you to think again. Equally, if you are considering running facilitated usability testing, ask yourself if it is the best tool for the job.