The Ultimate Guide to User Testing
And How To Facilitate A Successful Usability Test
Usability testing is way to test the validity and limitations of a website or prototype. They allow design teams to identify what works and what doesn’t by observing a user’s behaviour, body language and emotions as they run through carefully selected tasks.
Why is User Testing important?
We all project our own wants and assumptions onto others. That said, as designers, rarely are we actually the end users for the product we design. This is why we test with others – to ensure our products and websites are designed objectively with their intended audiences in mind.
User tests reduce chance of building the wrong product, or a product the wrong way. When it comes to website or product design, this means saving time, money and other precious resources. For example, a problem uncovered in the prototyping phase costs drastically less than fixing that same problem later in the development phase. Other reasons to facilitate a usability test on a website, product or prototype are:
- Identify flaws
- Inform new product features
- Test assumptions made during Discovery and early Design stage
- To see how it meets the user’s expectations
- To see how successful participants are at accomplishing key task.
- Gather user reactions and feedback about the product
Testing Methods for Usability Tests
In general, there are three types of usability testing methods, each one with its pros and cons.
- Moderated In-Person: These tests are ideal as they allow the facilitator and observers to catch things you may not if the test is done remotely. They can often also be the most expensive way to test a website or product.
- Moderated Remote: The user and facilitator are in different locations. The test occurs via screen share software, like Google Hangouts, Skype, WebEx, etc.
- Unmoderated Remote: Software is used to administer tasks directly to the end user. Tasks are designed and programmed by a designer, and participants are selected based on a set of criteria the designer chooses. Often, video recordings are provided to the designer of the the participant’s test.
Common Types of Usability Tests
Different usability tests are used to test different things. Below we’ve broken down some of the most common tests and what they reveal about a specific product or website.
Preference Test
Preference Tests ask the participants to choose between two design alternatives. At Massive, we use these when testing logo concepts, homepage concepts and pivotal design elements.
The goal with a preference test is to understand which design users prefer when given a specific criteria. For example, when preference testing a logo we may ask “Which design do you think looks more professional?” Preference testing can also be done on website elements, like navigation or form type.
5 Second Test
A Five Second Test displays a design for just five seconds. Before the participant views the design, we may ask them: “As you look at this page, focus on whether this “fits” as being authentically <Client Name>.”After the five seconds is up, the participant is asked a number of questions, such as:
- “What words would you use to describe this brand after viewing that page?”
- “What did you like best about the page?”
- “What would you change about the page?”
Click Test
A ‘Click Test’ records where users click on a designs. A participant is asked to follow an instruction, such as, “Where would you click to find information on bottle recycling?”. Click tests or often used to test multiple variations of a design to see which one is most intuitive.
Nav Flow Test
Using a ‘Nav Flow Test’ helps determine whether participants can successfully navigate a design. Specifically they help measure the success (and failure) rate of calls-to-action and navigation links at each step. They also help us understand how users navigate between different sections of a website/product, and which items require more/less prominence.
Choosing the Right Participants for a Usability Test
Now that you’ve decided to run a user testing session, you’ll need to identify volunteer participants to test. Look for participants that match the groups of people the site or product is designed to reach. If you have Buyer Personas, great! Find participants that reflect those personas. Participants will likely reflect a range of ethnicities, and interests. Finding a solid mix of participants helps mitigate the chance of producing false positives. You’ll want to ensure the tasks you create transcend these different personas to avoid spending time explaining things that users with the right domain knowledge know naturally.
The most common when selecting user testing participants is choosing ones who are “the best”. Instead, you want to get a list of participant names from a variety of places since your demographic is likely more diverse than that core group of deeply involved individuals.
How many participants should you test?
Begin by compiling a list of people willing to participate in a user testing session. You’ll need different participants for each round of testing, and finding them can be hard if you narrow your search too much, so start with as many people as you can.
Each session requires three participants, so you’ll want to have a minimum of three participants to start, but as many as 30 participants who agree to take part.
Finding User Testing Participants
The best way to find participants is to recruit actual end users. Often, recruitment happens at the places they frequent. Be sure to cover a variety of locations to ensure you’re getting a good mix of participants.
How to Recruit Participants
The follow is a simple signup form that can be used as a paper form, or converted into a webform. The key is to keep the form simple, and request further information as needed once you are ready to test.
Book enthusiasts! Shape the next publiclibrary.com!
Give us your input and We’ll give you a $50 Prepaid Visa gift card.
Public Library is redesigning it’s website and wants your input! Tell us what you think ofpubliclibrary.com and how we can improve it. In exchange for an hour of your time, we’ll give you a $50 prepaid Visa gift card.
Interested? Leave your email below and we’ll be in touch before our next feedback session.
Your Name: ______________________________
Your Email: ______________________________
Preparing for a User Testing Session
Creating Tasks.
To run a user testing session you’ll need to come up with 5-10 of the most important things people will need to do with the site or product. What are the things that people must be able to do or the project will be a failure? As a project progresses, there are often things that keep you up at night, things that you wonder about or suspect are going to cause trouble. You might even have a list of things that existing customers complain about, or commonly have trouble doing. Once you have narrowed it down to 10-15 tasks, you will be able to use them to create your testing scenarios to use in your sessions.
Generally, For a 1 hour session you should have between 10-15 tasks.
Creating Scenarios
Once you have tasks chosen, you’ll need to use them to create scenarios. A scenario gives the user all of the context that they will need to have to complete the task. It tells them who they are, what they need to do and gives them information they need to have to complete it but wouldn’t naturally have (like a username and password to login with).
- Task: “Sign up for a newsletter which sends you new realty listings in Scottsdale.”
- Scenario: “You’ve just accepted a job in Phoenix AZ. After a lot of research, you’ve decided Scottsdale is the area you’d like to move to. Find the latest real estate listings for Scottsdale.
Write a Script
You need a script. Scripts ensure you’re not introducing a new variable into different sessions. This is especially important in cases where there are multiple facilitators. If you can’t memorize your script, write it down and let the participant know you’re reading “to make sure you cover everything”. Each session should cover the exact same points.
A script will also ensure you explain the permission form correctly, which is important.
Run Through Your Test as a Practise
These practice runs highlight things that aren’t working and should be fixed before the session. They also help identify issues that may arise during the session, like task ambiguity.
Prepare Your Necessary Documents
Each task and scenario should be printed on half a sheet of paper, in a large visible font. These are for your user to reference as they consider a task.
- Print the questions on a single sheet of paper to distribute to each observer of the session so that they can follow along.
- Print out instructions for observers with spots for top three usability problems at the end of each session.
- Print out recording consent form that the participants will need to sign
Purchase gift cards
We recommend giving each participant a Visa/MC gift card (minimum $50) as a thank you for their time.
Additional Resources for User Testing
Time requirements
Plan 1-2 hours for writing tasks and scenarios for your user tests. If you have time, test your scenarios out with a colleague. This will ensure that your scenarios are clear and understandable and that they’ll give you the feedback needed.
Make sure you give yourself enough time to analyze the results for your report. A general rule of thumb is for every hour of video to be reviewed, give yourself that same amount of time to review.
Remote User Testing
Sometimes you can’t run a user testing session in person. In these instances, remote testing is a great way to go. Below are a few of the tools we use to conduct a remote user test:
Related Reading
If you’re designing a website or working on a new product, here’s some additional reading you may be interested in:
- Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days
- Facilitator’s Guide to Participatory Decision-Making
- The Design of Everyday Things
- Don’t Make Me Think, Revisited: A Common Sense Approach to Web Usability
- 100 Things Every Designer Needs to Know About People
- The Paradox of Choice: Why More Is Less
- Low-fi or High-fi? Choosing the Right Kind of Prototype
- Testing for eCommerce
Go Out And Test!
Hopefully this post has helped you better understand usability testing, and how testing a product or website is a critical step in designing a usable product. Like all things, testing takes practice. Some will fail miserably, but in time, you’ll develop your own style and way of extracting user data. Now all that’s left is for you to go our and test your product!
Categories | Content Strategy , Design , Experience Design , Strategy , Trends , User Testing , UX Design , Website Design