Ask Alasdair
This is an opportunity to ask our Human-Centered Research and Design Lead questions about all things UX. Check back monthly to see Alasdair’s quick takes and expert insights on UX topics.
Every month we will take some of our most frequently asked questions and include them in a poll. Are there specific questions from our list that you would like Alasdair to answer? Vote in the poll to make your voice heard ➡️
Want to ask Alasdair a question? Send us an email with the subject “Ask Alasdair” and your question or fill out this form. Your question may be answered in an upcoming edition!
-
Check back soon for answers from Alasdair!
-
1) How do we kill the fear of creativity without killing creativity itself?
In March I was invited to be a guest speaker by the Agora network at Shared Services Canada to talk about managing innovation. Agora – an employee led network – asked me to discuss how we kill the fear of change without also eliminating creativity. It was an amazing discussion; I was very impressed with Shannon Kenney’s openness and understanding of the topic and had lots of fun with our back-and-forth. But I also really enjoyed the insightful and challenging questions that came from the audience. [Oh, and when Rob introduced me he called me a “Creative genius”, so I was compelled to love everything about the event.]
In terms of our focus for the discussion, the conclusion that we came to was that as humans we can’t expect to simply kill what is a very natural fear. Instead, we need to recognize and accept fear and know that it might even be a valuable motivator. But equally important, for innovation to succeed and not be stopped by fear, we need to foster a culture that values experimentation and accepts failure as a natural part of the creative process. We collectively agreed that the best way to achieve this is by establishing a safe environment for people to express their ideas and take risks without fear of judgement or retribution. We also agreed that this needs to start at the top and that leaders and managers must set the tone by encouraging and rewarding innovation. And of course, the chat recognized that success required certain essential ingredients, namely time, funding, and expertise which must all be made available to support creative endeavors and help bring ideas to fruition. I’m thrilled to have had this amazing opportunity to get an insider’s view into what cool things SSC is doing and how they are truly engaging the entire SSC team.
2) Is measuring UX maturity of value to the public sector?
The short answer is yes, it is absolutely important to understand UX maturity at both the organizational level and the system-wide level. UX maturity has been around for years but is frequently seen as a competitive advantage in the private sector, and not frequently discussed in the context of the public sector. I think this is a mistake...
A UX maturity model is a framework that describes the level of maturity an organization or a team has in terms of user experience. It is often seen to measure a company’s desire and ability to successfully deliver user-centered design and to define the path to beating their competitors. But many of our public sector clients have been exploring their UX maturity for several unbelievably valuable reasons:
Improving user satisfaction - Good UX can improve user satisfaction services, leading to higher trust and confidence.
Increased usage - User-friendly interfaces and easy-to-use services can increase the usage of services, resulting in better citizen engagement and increased access to public services.
Cost savings - Good UX can reduce the cost of delivering services by reducing the need for costly support services and customer service staff.
Better outcomes - By improving user understanding and engagement with public services, good UX can help to achieve better outcomes, such as increased compliance with regulations and higher participation rates in GAC programs and activities.
Positive impact on society - By providing citizens with user-friendly and accessible public services, good UX can have a positive impact on society, leading to greater inclusion and participation in GAC activities.
How do departments, ministries, agencies and the public sector as a whole get to a higher state of maturity? You measure, understand where you are today, and then set a goal. For those of you who wonder what a truly mature organization looks like, I can point you to a few, but generally, a mature UX practice typically involves a user-centered approach to service design, which includes conducting user research, activities like creating user personas, and testing service designs with real users. But it also involves having a well-defined design system, with established design patterns, guidelines /policies, and standards that ensure consistency and coherence across an organization's digital services. Finally, a mature UX practice involves a culture of continuous improvement, with processes in place to measure and evaluate the effectiveness of its digital services, and to make iterative improvements based on user feedback and data. Hard? Maybe. Worthwhile? Absolutely!
3) How do you know what kind of UX research to do for a client?
A first step in working with any client is to spend time speaking with them to understand their UX needs and goals. Sometimes clients don’t know their goals or don’t know exactly how to articulate them, while in other cases they have all the answers. Fortunately, we’ve done and seen lots and can likely help them figure that stuff out or maybe just refine it a little based on some candid back-and-forth. Once that is done, we can develop a research plan that includes one or more of the following types of research approaches:
Generative or Exploratory Research: This type of research is used to gather insights and ideas to generate new concepts and hypotheses. It is often conducted at the beginning of a project, before any specific design or solution has been identified. An example of generative research might be conducting user interviews or focus groups to gather insights about how people use a particular product or service. This type of research is useful when you need to generate new ideas and understand the context in which your users are operating.
Descriptive Research: This type of research is used to describe and define the characteristics of a particular group or phenomenon. It is often used to answer questions about what is happening or what exists. An example of descriptive research might be conducting a survey to understand how many people in a particular demographic use a specific technology. This type of research is useful when you need to understand the characteristics of a particular group or phenomenon.
Evaluative Research: This type of research evaluates the effectiveness of a particular design or solution. It is often conducted after a design or solution has been developed to determine if it is meeting users' needs. An example of evaluative research might be conducting usability testing to determine whether a new website design is easy to use and meets the needs of users. This type of research is useful when you need to evaluate the effectiveness of a particular design or solution.
Causal Research: This type of research is used to determine cause-and-effect relationships between different variables. It is often used to answer questions about why something is happening or to test hypotheses about the relationships between different variables. An example of causal research might be conducting a study to determine whether a particular intervention leads to a specific outcome. This type of research is useful when you need to understand the causal relationships between different variables.
There is never one “right” way to design the research approach, and there is no silver bullet. In fact, two skilled practitioners might recommend two very different approaches to address the same project for some very valid reasons. But you should keep in mind that you can choose to use the wrong research approach, which will give you answers to the wrong questions. So please don’t hesitate to ask if you are unsure of the best approach for you/your project. We’ve all been there.
4) How do you select research participants that are diverse and ensure that the research is inclusive?
Selecting a diverse pool of UX research participants is critical to creating inclusive products and services that cater to a broad range of users. To achieve this, it is essential to define the scope of your research and identify the user groups you want to target. Aim to recruit participants from diverse backgrounds, including different ages, genders, cultural backgrounds, and abilities. We developed our database to be diverse to support clients in exactly this way. When creating recruitment criteria, use inclusive language and avoid exclusionary terms that could discourage certain groups from participating.
It's also crucial to prioritize accessibility as a fundamental pillar of UX when selecting research participants. Accessibility involves designing products and services that are usable by people with different abilities. Thus, it is essential to include participants with disabilities in your research to identify accessibility issues early in the design process. When conducting user testing, consider the needs of all participants and provide necessary accommodations, such as sign language interpreters, captioning, or audio descriptions.
Finally, creating a welcoming and inclusive research environment can help ensure that all participants feel comfortable sharing their thoughts and experiences. Be mindful of cultural differences, avoid making assumptions about participants' experiences based on their background, and create a safe space where participants can ask questions and share their opinions freely. By prioritizing diversity, inclusivity, and accessibility in UX research, we can create products and services that cater to a broader range of users and ultimately provide better user experiences.
-
1) What UX books/literature are currently on your nightstand? - why?
I’m actually most interested in a book that I have yet to get... but let me start by saying I also have a number of books by my bed including Think Like a Rocket Scientist: Simple Strategies You Can Use to Make Giant Leaps in Work and Life by Ozan Varol which is fun, accessible and interesting. I’ve been working through a few on designing and selling consulting services - but I tend to read articles in places like Smashing Magazine, UX Matters and then in Medium and Wired to name a few. I’m also interested in what other businesses are talking about and frequently go to the Neilsen Norman Group website and the Interaction Design Foundation. I also spend a little too much time on LinkedIn and Reddit looking at what UX/HXD and other aligned communities are talking about.
But I’m most curious to get into Don Norman’s new book Design for a Better World which should arrive in my mailbox very soon. He’s not alone talking about the mess the world is in and how we should and indeed must be designing to think about nature, the environment, broader systems and our impacts. Since I suspect he will do it in a meaningful and relevant way I’m excited to jump in!
2) What challenges do you see when clients do their own recruiting?
At the heart of good research is bringing real users - who either have experience with your service/product or match the target audience - and getting their ideas and feedback. We are sometimes asked whether our client can or should do all their own recruiting. Our advice is that it depends. While it seems to be straightforward, barriers to simple, seamless recruiting can create significant problems and even derail the entire project. Some of the recent and frequent problems that we have seen our clients experience include:
Legislation places limits on communicating with your audience or the information you can collect about them.
Rules that limit the values or outright ban compensating participants for their time.
Plans to reach the target audiences directly or through third parties is harder or takes more time (we’ve seen it take months) than expected.
There is no existing pool of people to recruit from, or the pool is too small or biased, or it is old and not well managed.
Clients don’t want to use their relationships with potential participants to invite them to the exercise.
Administrative or logistical challenges around inviting, booking and replacing participants.
Since we can get elephants to jump, we’re pretty sure we can successfully work through any and all of these issues with you!
3) Why do our clients outsource their UX?
Over the last few months, the news has been full of discussions about how consultants are taking advantage of the Canadian public service. I can’t pretend I’ve never seen it, but I will say that I have seen lots of good reasons why folks look for outside help. They include:
Lack of internal expertise – Some clients simply don’t have the necessary expertise or resources in-house to develop a high-quality user experience.
Cost-effectiveness – External resources can be more cost-effective than hiring full-time employees, saving the costs of recruiting, hiring, training, and providing benefits to new employees. Outsourcing is also often cheaper than bringing someone on and then not keeping them busy.
Time savings - Outsourcing can save time, allowing our clients to focus on their core business activities. Plus, an experienced team will deliver a faster, more efficient and potentially more complete project.
Access to a wider talent pool - Outsourcing can provide access to a wider talent pool of folks with diverse skill sets and experiences, generating more innovative and effective solutions.
Flexibility - Outsourcing provides clients with the flexibility to scale their needs up or down as required.
Validation - In some cases, clients outsource because they want a neutral third party who will collect and use evidence to guide decision making.
Whatever the reason for outsourcing, look for a company you feel you can trust and respect.
Last Updated: Match 17, 2023
-
1) There has been a real concern in recent years about groupthink in UX work? What is your take?
I sometimes have to blame groupthink, but we don't have to accept it!
Once in a while our team is asked to "UX" or fix services, products, or solutions that are a BIG fat mess and we wonder how anyone could have given the project a go-ahead and how it made it this far. I can sometimes point to groupthink as the reason for money and time being wasted on a project that I know didn't need to go so wrong.
The NNGroup recently posted about Groupthink in UX Work and I think they did a great job of tackling a lot of the key issues and offering a number of good preventative approaches. Calling groupthink out makes us all a little more accountable and should remind us that we don't have to be a go-along-sheep. However, I did notice a few gaps that I think they overlook:
One-on-one stakeholder interviews - Interviews with those that have a vested interest in the project outcome allow individuals to define their goals and needs before being influenced by others. I see huge value in tracking their feedback and sharing it as part of our findings as this helps reduce the tendency to just gravitate to one shared opinion.
Audience-based usability research - While I will intentionally leave the methodologies vague, research with the audience will also help refine and define what needs to be developed/designed and becomes the "evidence-based shield" against poor ideas and bad groupthink.
Environmental scans/comparative analysis - Whatever you call it, environmental scans, a comparative or competitive analysis can be very helpful in bringing new ideas to the table or validating existing concepts. I've seen this be super effective at moving people away from "the way we've always done it", and effectively introducing an alternative approach.
I'm with the NNGroup - stand on guard against groupthink and know that there are ways to mitigate it!
2) Why include stakeholders and who do you invite?
Good design needs to understand the goals of the organization to truly be effective. It is not enough just to build a product or service because your users want/need it (much as I’d love to, just because you all like ice cream, does not mean we’re gonna start offering it). We find that voice of the organization (VoO) by talking to folks in the know. They go by many names, but we tend to refer to these people as stakeholders. We frequently get asked how to find these people. We generally recommend that when choosing stakeholders, you consider:
Senior managers whose support is required to make the project/content/design a reality
Those people – and they could be anywhere in the organization - who have a vested interest in the project/content/design and will be impacted
People who are "squeaky wheels" and who would like to be heard and involved in the project. As an aside, not only does including them give them the opportunity to feel heard, but we have heard some amazing ideas from this group
Our typical approach for getting at the VoO is to invite them to an hour-long interview (the higher up the chain we go the less time we have, so sometimes it is more like a 20-minute discussion). We are often also asked for a sense of the kinds of questions we normally ask as this might impact what is chosen. Our interviews often focus on things like:
Who the stakeholders see as the target audience
What they think the audience is trying to accomplish/their needs and tasks
What goals/objectives they have for the project/content/design
How they would measure the success of the project/content/design
And before you ask, we tend to suggest that the interviews be one-on-one. Groups can work, but they generally result in one person leading the discussion (typically the peep with the highest salary), or a scenario where everyone divides the 60 minutes equally (e.g., in a group of 4 giving each person 15 minutes to have their say, which is often not nearly enough).
3) What is the perfect number of participants for usability testing?
Clients often struggle to define (or guess) the perfect number of participants for their usability testing research exercises. That makes total sense since the answer really is “it depends,” but that is not always obvious.
When we set up a usability testing project, we will typically go through a decision-tree exercise of “how many” with our clients. We start with the recognition that as few as 5 or 6 participants can work well, as explained in the 2012 Neilsen Norman Group article. But when the client needs to explore more than 10 tasks (which is our recommended sweet spot for reasons I will explain in another post), and/or when they have more than one key and diverse audience, we will typically bump the number up to at least 12, depending on the number of audiences and/or tasks that are required. In many cases we have suggested 12-15 completed sessions when clients want a representative number of sessions in French (normally 25% of the total number of participants), or if they want to test with folks who use assistive devices or have disabilities (which is super important, and all too often overlooked!!!). For many clients we target 18-20 because they want to have a high level of confidence that they have found the uncommon as well as more common problems. Many Government of Canada clients also like this approach because the Digital Transformation Office at the Treasury Board of Canada Secretariat is also a fan of this range, particularly when running both a baseline (on the original version) and validation round (on the improved version) of usability research.
4) Why do you recommend 10 tasks/participant when conducting usability testing?
Understandably, clients want to get as much out of their research investment as possible, and while our tag line, is “Anything is possible”, there are limits! One of those limits is that a participant be asked to do no more than about 10 tasks during their session.
There are several reasons for this proposed upper limit. The first is that 10 tasks or exercises is a lot to do, especially in a somewhat strange and stressful environment where you want to look and sound coherent, please the stranger who is moderating the session, and find the correct answer. Usability testing can be genuinely exhausting for the participant and moderator alike!
But we also know that if someone does not find the answer within about 5 minutes the task has probably already failed. And in the real world, if you are struggling with a task for more than about 3 minutes, you will probably leave and try to find the answer through Google or maybe ChatGPT. So, at the 5-minute mark we (politely) move folks on to the next task. Capping the tasks at 5 minutes and having no more than 10 tasks also means that we can generally estimate that each session will take no more than an hour (with 5 minutes of welcome/warm up and 5 minutes of debrief at the end of the session). That hour “target” takes some of the guesswork out of the booking process, but also allows folks to determine the relative worth of their time when we tell them how much they will get paid for the session. Obviously, we make exceptions to this rule of thumb, but they are minor since we have found 10 to be our happy place!
Last Updated: February 21, 2023