I recently worked on a discovery project looking into the needs of international users on GOV.UK. We knew from analytics that GOV.UK receives around 9 million visits a year from outside the UK. We wanted to understand the needs of our international users, so that we could better meet them. Rob’s blog post talks more about why we kicked off this project.
Part of the project involved identifying and defining the sort of content international users might be interested in. Lucy’s blog post looks at this process.
What I want to tell you about is how we conducted user research to better understand our international users, and what we found.
A timeboxed, short, phased approach
With only a six-week period for our discovery we created a research plan to help us work quickly.
This plan comprised 4 steps:
- identify the user groups of international content
- find and review any existing research by GDS and other departments about international users
- interview users that match our international content user groups
- use the satisfaction survey on GOV.UK to validate our findings
Who our international users are
We needed to start by defining what we meant by ‘international’ users. We felt the simplest way we could group people was by their nationality and where they lived.
This gave us these four groups.
| British citizens | Not British citizens | |
|---|---|---|
| Living in the UK | Group 1: British and living in the UK |
Group 3: Not British and living in the UK |
| Living outside the UK | Group 2: British and living outside the UK |
Group 4: Not British and living outside the UK |
We found we had a lot of existing research on group 1, British people living in the UK.
We found that there was very little existing research on group 2, British citizens outside the UK, so we would need to do new research.
When looking at group 3, non-British people living in the UK, we had a question of whether or not their needs were the same as people in group 1. We struggled to find an answer from existing research, so we realised we needed to talk to them too.
We learnt that the most common reason group 4, non-British people living outside the UK, would use GOV.UK is to apply for a visa to come to the UK. As there was a separate stream of work with GDS and UK Visas and Immigration looking at these users we felt it wasn’t a priority, especially with our short time scales.
Therefore our research focused on groups 2 and 3: British citizens living outside the UK and non-British citizens living in the UK.
Speaking to an audience outside the UK
For this strand of research, we worked with a recruitment panel to find British citizens living outside the UK. The panel was a list of people in the UK, so we asked them to refer us to friends and colleagues living abroad.
Our interviews were conducted over the phone and we had problems distinct to this type of research. Other than the usual time delays and connectivity issues we weren’t able to observe any behaviour and context of use.
Interviewing non-British citizens in the UK
We could easily find and observe this audience so we were able to do 6 face-to-face interviews. We split these between EU and non-EU nationals to see if there are any differences in attitude between the two.
We had seen existing research looking at how this audience used GOV.UK, but very little about why. So this is what we focused on.
To find this out we didn’t specify any pages or services in advance for the research, as we wanted to know what this user group was using GOV.UK for. Our participants volunteered things they had done on the site and showed us what the experience was like.
Interviewing in this way meant we got to find out what the motivation to use GOV.UK was for this user group. We then compared this with our knowledge of British nationals using GOV.UK to see if there were any differences.
We found that ultimately the motivations and experiences were the same for everything, other than visas and immigration.
Using the surveys to validate our findings
On GOV.UK we ask every 50th visitor if they would like to complete a survey to provide feedback on their experience of the site. Our plan was to use this survey and tailor the questions specifically to our project. That way we could validate the findings we made during our discovery.
Using the responses we were able to show the scale of the user groups we defined and also their perceived satisfaction with the content they viewed.
Having this larger-scale data from the survey and qualitative findings from the interview meant we finished our discovery confident with the findings we’d made.
What we’ll do next
At the end we identified key areas of GOV.UK that need improvement for international audiences. The roadmap for this year is taking shape, with multiple opportunities to improve the way we meet the needs of international users.
Most importantly we found that in order to be better at international research we need to improve our knowledge of conducting research globally. The phone interviews gave us insights but we couldn’t build an understanding of user behaviours using GOV.UK and services.
The cross-government user research community are running a meetup early this year to discuss how we can better support each other to develop these skills. This will be an exciting chance for government to get better at researching internationally.
Keep in touch. Sign up to email updates from this blog. Follow David on Twitter.