One Is Not Done: 5 Reasons To Do Usability Testing More Than Once

By: Pamela Ong & Nikhil Shah

Congratulations! You finished conducting usability testing with representative users and have launched the final recommended changes. Your product or service (e.g., website, intranet, application, etc.) is up and running. You have seen and heard the difference it has made for your users. All is well and management is happy!

Fast forward 6…12…24 months, and in that time new content has been posted, there were a few changes in policies, and a slew of new users have joined your organization. You might ask yourself, is my organization’s digital service still working for its users? With all these changes, are people creating new workarounds to access and understand content? Do some things just not make sense anymore?

You have done usability testing once before but now you are wondering whether that is enough or if it is worth repeating.

We believe that usability testing should not be a one-and-done process. Here are 5 reasons why doing usability testing more than once is important to make better design decisions for your users:

1. You are not your user!

This point is not new and in fact, you may have heard this one already! We are mentioning it as a reminder because it is so important. Making assumptions about what users want and need will only set you up for disaster. In consulting with users and asking them for feedback, you can expect a high return on investment from usability testing.

Test. Iterate. Repeat.

Testing with as little as 5 users can give you plentiful feedback on changes that are required to make a more pleasant user experience - Conducting your next round of usability does not have to be a complicated and lengthy process!

2. The people who use your product/service may change over time

People can change positions within an organization, move to other Government of Canada departments, retire, or leave for another job altogether at the drop of a hat (if you are struggling to retain your staff, check out our article on why UX training may be the solution for retention).

Sometimes these organizational changes are out of your control. However, something that is within your control is the ability to provide a better user experience to your new users. Whether they have come in from another department, are brand new to the federal government, or parents coming back from parental leave, they all need to quickly understand the content, navigate around the website/intranet and hopefully, do it without too many support tickets!

Additionally, the needs of your current users may change over time. They may be required to use some services and touchpoints in their daily working life while some others may be accessed during later stages of their career or life. Think of the manager who suddenly needs to conduct a performance review, a close contact who requires medical leave, or someone finding support after being harassed. Your users’ needs can change in a snap depending on their circumstances. You may even be able to think of your own experiences where your needs changed over time. Ask yourself, “Were my needs being addressed on the services I used?”

Providing frequent testing shows your new and current users that you care about the systems that they use daily and that their feedback is not just being heard but acted upon as well.

3. Increase in mobile users and remote workers  

Phones and tablet usage has dramatically increased over the last decade. In fact, according to Statistics Canada, 88% of internet users aged 15 and up in Canada have a smartphone. And with the majority of GoC employees working from home, this poses unique strains on the Intranet to support self-service and answer questions that would have otherwise been answered by the colleague in the office next door.  Ensuring that we design for users on different screen sizes is important to accommodate their versatile and flexible needs. 

Often, we tell our clients to design with a “mobile-first” approach. What does this mean? Compared to our laptop/desktop screens, mobile screens are significantly smaller. Where laptop/desktop screens provide a comfortable amount of real estate for information and usually require minimal scrolling, mobile screens often squish all this information into a smaller interface. In effect, this means more scrolling and less space to tap and move around the screen. Without careful consideration in your mobile design, the experience for desktop users versus mobile users could be extremely different. Designing for mobile means being very particular on what and how much content users see and interact with. 

Usability testing on mobile devices often generates surprising findings about some of the unseen barriers to task completion. All that being said, it is crucial to test your website or intranet with mobile devices today, to ensure it is well optimized for your users tomorrow. 

4. It can be done quickly and can save you money in the long run 

You may be thinking to yourself, “Do I have enough time for this?” You will be glad to hear that you already covered most of the groundwork during your first round of usability testing! By using the same tasks that you have already carefully curated, we can test users (potentially the same ones) with the same tasks a second time. Minimal effort is required on your part! 

Then, we can compare the results of the first round (as a baseline) against the results of the second round. This is called benchmarking.  By conducting benchmark tests periodically, we can track progress/impact over time and use the data to make informed design decisions. 

By taking this iterative approach, we can avoid minor band-aid fixes that may potentially snowball into a larger issue, ultimately saving you time and money in the long run. 

5. You have data to present to management/stakeholders for future buy-in 

Let’s be honest, if we want to maintain a product/service and test with users regularly, it’s going to require funding. And oftentimes, this sort of buy-in from management/stakeholders can be difficult. So how do you explain the importance of frequent testing in a way that resonates with their organizational goals? Let data lead the way!  

By doing usability testing more than once, we will be collecting both qualitative and quantitative data to demonstrate improvements over time (i.e., benchmarking). This data can be a powerful tool to demonstrate impact to management/stakeholders.  Furthermore, calculations can be made to directly connect these improvements to having reduced calls to call centres, emails and other support lines, increased compliance, and enabling people to find answers more quickly to save time and increase efficiencies. Thus, data can help us to advocate and provide a strong business case for future buy-in. 

BONUS TIP. Future-proofing in a rapidly changing digital landscape 

If there is one thing that we have learned from the COVID-19 pandemic, it is that change is constant and can happen in an instant. As new technologies emerge, policies shift, accessibility standards are updated, and our reliance on digital and remote work increases, making sure that our products/services are up to standard with our current realities is vital. But how can we do this if things are changing so rapidly? 

By conducting regular usability testing, we can implement small proactive changes, rather than reactive changes. This ensures that your product/service can be ready for any future change that might disrupt our current system. 

How Jumping Elephants Can Help 

Testing with users is not something that is just done once in a product life cycle. In fact, one company ran 12 tests in 12 weeks! We know this kind of frequency is not possible for many organizations and departments, but at Jumping Elephants, anything is possible. 

We can work with your team to easily re-test your website/intranet or other digital content to get quality feedback within budget and on a deadline.  

To be able to take advantage of this offering, ensure: 

• You have done usability testing with Jumping Elephants in the last 2 years. 

• There have been changes on your digital service since initial testing; and that 

• You want to provide a fantastic user experience to your users. 

If this sounds like something you would like to take advantage of or have questions about, book a quick 30-minute call with Alasdair Stuart-Bell, Partner & Lead - Human Centred Research and Design at Jumping Elephants.