https://userresearch.blog.gov.uk/2017/01/11/there-is-a-whole-world-of-users-out-there-just-waiting-to-be-discovered/

There is a whole world of users out there just waiting to be discovered

I recently worked on a discovery project looking into the needs of international users on GOV.UK. We knew from analytics that GOV.UK receives around 9 million visits a year from outside the UK. We wanted to understand the needs of our international users, so that we could better meet them. Rob’s blog post talks more about why we kicked off this project.

Part of the project involved identifying and defining the sort of content international users might be interested in. Lucy’s blog post looks at this process.

What I want to tell you about is how we conducted user research to better understand our international users, and what we found.

A timeboxed, short, phased approach

With only a six-week period for our discovery we created a research plan to help us work quickly.

This plan comprised 4 steps:

  1. identify the user groups of international content
  2. find and review any existing research by GDS and other departments about international users
  3. interview users that match our international content user groups
  4. use the satisfaction survey on GOV.UK to validate our findings

Who our international users are

An image with text saying users pointing at a world map

We needed to start by defining what we meant by ‘international’ users. We felt the simplest way we could group people was by their nationality and where they lived.

This gave us these four groups.

The four user groups for international content on GOV.UK.
British citizens Not British citizens
Living in the UK Group 1:
British and living in the UK
Group 3:
Not British and living in the UK
Living outside the UK Group 2:
British and living outside the UK
Group 4:
Not British and living outside the UK

We found we had a lot of existing research on group 1, British people living in the UK.

We found that there was very little existing research on group 2, British citizens outside the UK, so we would need to do new research.

When looking at group 3, non-British people living in the UK, we had a question of whether or not their needs were the same as people in group 1. We struggled to find an answer from existing research, so we realised we needed to talk to them too.

We learnt that the most common reason group 4, non-British people living outside the UK, would use GOV.UK is to apply for a visa to come to the UK. As there was a separate stream of work with GDS and UK Visas and Immigration looking at these users we felt it wasn’t a priority, especially with our short time scales.

Therefore our research focused on groups 2 and 3: British citizens living outside the UK and non-British citizens living in the UK.

Speaking to an audience outside the UK

For this strand of research, we worked with a recruitment panel to find British citizens living outside the UK. The panel was a list of people in the UK, so we asked them to refer us to friends and colleagues living abroad.

Our interviews were conducted over the phone and we had problems distinct to this type of research. Other than the usual time delays and connectivity issues we weren’t able to observe any behaviour and context of use.

Interviewing non-British citizens in the UK

We could easily find and observe this audience so we were able to do 6 face-to-face interviews. We split these between EU and non-EU nationals to see if there are any differences in attitude between the two.

We had seen existing research looking at how this audience used GOV.UK, but very little about why. So this is what we focused on.

To find this out we didn’t specify any pages or services in advance for the research, as we wanted to know what this user group was using GOV.UK for. Our participants volunteered things they had done on the site and showed us what the experience was like.

Interviewing in this way meant we got to find out what the motivation to use GOV.UK was for this user group. We then compared this with our knowledge of British nationals using GOV.UK to see if there were any differences.

We found that ultimately the motivations and experiences were the same for everything, other than visas and immigration.

Using the surveys to validate our findings

On GOV.UK we ask every 50th visitor if they would like to complete a survey to provide feedback on their experience of the site. Our plan was to use this survey and tailor the questions specifically to our project. That way we could validate the findings we made during our discovery.

Using the responses we were able to show the scale of the user groups we defined and also their perceived satisfaction with the content they viewed.

Having this larger-scale data from the survey and qualitative findings from the interview meant we finished our discovery confident with the findings we’d made.

What we’ll do next

At the end we identified key areas of GOV.UK that need improvement for international audiences. The roadmap for this year is taking shape, with multiple opportunities to improve the way we meet the needs of international users.

Most importantly we found that in order to be better at international research we need to improve our knowledge of conducting research globally. The phone interviews gave us insights but we couldn’t build an understanding of user behaviours using GOV.UK and services.

The cross-government user research community are running a meetup early this year to discuss how we can better support each other to develop these skills. This will be an exciting chance for government to get better at researching internationally.

Keep in touch. Sign up to email updates from this blog. Follow David on Twitter.