I’m working as a user researcher on GOV.UK Verify, the new way for people to prove they are who they say they are when using government services online.
In this post, I’ll share why we’ve found it's important to augment lab research with talking to people using the live service.
A research lab environment has definite advantages
A research lab environment has definite advantages; we do lab-based research at least once every 2 weeks.
The lab provides a controlled setting for conducting sessions. We’re able to easily observe how people interact with our prototype and take note of usability problems around particular areas or features.
Importantly, because our lab has a substantial observation suite, our project team is able to watch sessions in real time. This means that everyone in the project team gets to know our users first hand, not just the researchers.
But lab research does have limitations
Whilst the lab has many benefits, it’s not a true-to-life representation of the context in which users will complete a task.
Our service requires people to answer questions about their personal details, such as their bank account number. Because we’re researching a prototype and recording the session, we don’t use actual details. This makes it hard to get authentic responses about privacy issues.
If people feel uncomfortable about continuing the process, the research facilitator will aim to address these concerns. However, because we want to get feedback on all the latest updates to the prototype, we’ll encourage the participant to keep moving.
I’m very alert to the kind of questions asked. I was uncomfortable, and would've stopped immediately.
We try to overcome these things through good facilitation but, even better, we step away from our prototype and get out of the lab.
Combining quantitative and qualitative approaches
Lab research offered us insight into the participant's conceptual and emotional response to the prototype. We learned about issues which may cause users to drop out.
Analytics helped us understand where users of the live service were dropping out.
We wanted to build on this knowledge by speaking with live users - people who were experiencing the service in their natural context.
To access live users, we chose phone interviews, which were simple to set up, quick to run and allowed us to gather valuable qualitative insights.
Live research deepened our understanding of certain issues
Live-user research broadly supported what we’d discovered in the lab, but it also deepened our understanding of certain issues.
Time and context was important
In the lab, we set participants a task to complete, allowing them enough time to get it done. The lab is a quiet and calm environment, which frees the participant from any distractions.
With live users, we found that people were accessing the service in environments more demanding of their time and attention. People who failed to register, complained of having a lack of time and were accessing the service between other tasks.
It was the middle of work. I spent no more than 5 minutes on it.
In comparison, those that were successful, deliberately set aside ample time.
I didn't know how long it would take. I set aside an hour, and it only took 10 mins.
Trust in certified companies
As part of proving their identity, our service requires people to choose a certified company.
In the lab, we learnt that people are more likely to pick a company they trust. If people don’t know or trust any of the offered companies, the facilitator will acknowledge these concerns and encourage the participant to continue.
Similar to lab research, live-service users chose companies based on trust and were more successful in registration. However, those who had no trust were likely to abandon the process.
I don't know this company, and that's the only reason I left.
Stepping away from our prototype paid off
All research approaches are useful but, by using several different research methods, we were able to get a more indepth and multi-dimensional view of our user's behaviour.
Work with your analytics experts, speak to live users by getting on the phones or into the field, and keep on working in the lab.
One last tip: catch live users straight off the service
Our participants had to go through data protection processes before we could speak to them, and we knew that the longer we left the interviews, the less likely people were to remember what they did.
We made every effort to speak to people as soon after using the service as possible.
This research helped to create the user interface which introduces the service, and the public beta went live on 14 October 2014.