How do you set up a remote user test?

Step 1: Inform the users and check their ability to participate

As designers, we are stuck to our devices 24/7, making it easy to forget that not everyone is best friends with their computer. Therefore, it’s important to keep in mind that participants might not be comfortable with digital tools and calls at all. A big mistake is to assume that people are comfortable participating in a remote user test, or that they even know what to expect. Sometimes basic computer knowledge can be a hurdle. We’ve had to deal with a participant who hadn’t started up her computer for years, and didn’t know what a browser was. Literally.

So first of all, inform your participants that the test will be remote, and not just with a phone call. Check whether they are able to do an online interview:

• • Does the user have a working computer with video, sound and internet connection?
  • Does the user know how to use his/her computer?

Also, don’t forget to inform your participants about the software you will be using, and how it works. We always send our participants a manual on how to use Google Hangouts.

Step 2: Set up the online environment to analyse during the user tests. 

Everyone who has ever done a full day of user tests knows that time flies. Before you know it, it is time for the next test. It’s important to organise a way to capture your valuable insights and iterations right away to avoid mixing them up with other participants. In the real world you can use a mirror room to analyse what’s being said in real time, as you see in most UX-labs. In the digital world, there are plenty of tools that are able to replace the post-its and physical walls. We prefer Miro but Mural is also fine. Preparing a structure for your digital environment makes it easier for both you and your client to instantly capture useful insights during the interviews. 

This is how we set up the online environment:

• • Allocate certain ‘areas’ of your board or canvas to specific topics or tasks. For example, you could create ‘areas’ for quotes and raw notes, hypotheses that you want to validate or prototype screens. Importing your prototype screens allows you (and the team) to immediately collect feedback on the screens. 
  • Use different colour post-its for every interview. It will make your life easier. Trust us.

Step 3: Inform your clients about their contribution

At Koos, we usually invite our clients to our ‘mirror room’ to observe the user tests. This way, they can directly hear what the user has to say about a prototype. Although your online environment is well-structured (assuming you carefully executed step 3), it’s still a new way of witnessing a user test for most clients. Therefore, it’s important to inform the external project team, and let them know how everything works, what you expect from them and set some ground rules. Remember, you’re asking as much from them as you do from the participants!

Don’t forget to tell them to turn off their microphone and camera so they do not interrupt the user test! 

Only invite one team member to witness the real-time user test 

In our remote user test, we had 3 pairs of eyes watching every move and clicking through a prototype. One interviewer, one researcher recording the screen and analysing, and one witness from the client side. This can be a bit intrusive and uncomfortable for a user. Thus, communicate with your clients that only one person is allowed to join every user test. Tell the rest of the team that there will be recordings which will be shared immediately after the user test so everyone can listen to it afterwards.

Ask your clients to join the online analysis

Tell the entire team that they are expected to join in the online analysis environment to share their insights. Give them access to this online environment, for example by creating day passes. (See step 3 for how to set up this online environment). Give them a small instruction before the test day itself, so they know what is expected of them and how everything works.

Step 4: Appoint one interviewer and be prepared

You’re now almost good to go to conduct a remote user test yourself. Yay! 

However, take into account that you are not only an interviewer at this point, but also a customer support agent in case something does not work. So make sure you are well prepared and know the ins and outs of the video software you are using. Make sure you are all set, including a topic list, structured online analysis environment and a working prototype. Always double check, and do a trial run! 

Now it’s time to start communicating with the participants. Take your time to get the participants comfortable and settled before diving into your prototype. Some participants can become nervous when using technology like video calls or digital prototypes for the first time. More often than not, they don’t know how it works. Take your time to explain it to them. We made sure to give all participants a good old fashioned phone call 15 minutes in advance, to help them set up everything.

Tip: You can also set up screen sharing right after the introduction so you can guide them more easily. 

One of the advantages of conducting your user test online is that you can use screen sharing. You can ask your participant to share his/her screen, giving you insight into how they use the prototype. If you have a digital interactive prototype, screen sharing gives you a good idea on how the participant is using your prototype. You can see their mouse and scrolling behaviour which gives you the perfect opportunity to ask questions. 

“I see that you immediately clicked on this button, why is that?”

Every project is different

Even though these four steps are really helpful, not every project is the same. That’s why we adjust our remote approach to fit every project. We are excited to keep on developing remote user testing skills. Do you want to know more about how we do remote user tests? Reach out so we can talk about how we can help your organisation.