If you want to get an overview of how your service is being used and have no website analytics, there is no need to panic. Here are 4 steps that will help.
When we think about the data we hold on our services, the first thing that comes to mind is often website analytics. But there are other valuable and occasionally overlooked types of data that can be really useful to user researchers.
I’ve used several different types of user data which have given me a much richer understanding of user needs than I could have gleaned from web analytics alone.
Search logs, support tickets and social media all hold valuable insights, and gathering them can help get your whole team involved in the user research process.
Step 1: Start with a research question
Whoever helps you get hold of the data will ask what you need to know so they can get the right data for you. Being clear about the questions you need to ask of the data will also help you when you come to analyse it. There are a lot of handy things in the Service Manual about how to work with your team to create research questions.
Step 2: Choose the right tools for the job
It’s a good idea to work out which data source you need, rather than trying to grab hold of all the data that’s available. Think about how users interact with your service. You may already have captured this information in a journey map. These are a few of the sources I’ve used:
User accounts
Accounts are where your users get up close and personal with your service. Used responsibly, backend data from accounts can tell you who your users are. For example, you might collect information about which organisation they belong to or what work they do. This information can help you segment them.
Account data can also tell you how frequently an account is accessed, or how often something is published. If you combine this with your segments you can start to get a picture of which types of users do which things.
Search logs
By looking at keywords or phrases in search logs, you can find out what your users are looking for on your site and how often. Sometimes you can find whole sentences expressing how a user feels about the service or what problems they’re having. This is gold dust.
Analyse the way the service or information is described in these places. Which words are used frequently? Which words are used together?
Compare your analysis of search logs with the content you have on your site. You will start to see if a user’s understanding of what you do is different from your team’s. If it is, then you can fix it.
Support tickets, contact forms and social media
This is where users tell you what they want but also what’s going wrong. Analyse all three in a similar way to search logs. Here is a great blog post explaining the method Government as a Platform used to analyse their support tickets.
Bring it all together
Use backend data in combination with support tickets to check if users are doing what you expect. If they’re not, it might mean there’s a problem. For example, users might be sending support tickets instead of applying for new accounts through your system.
Use data responsibly
Whichever data you access, if it’s likely to contain personal information, make sure you speak to whoever is responsible for data management in your organisation and always check your service’s privacy policy.
Step 3: Find a developer who can help
So you have your questions and an idea of which data might hold the answers. How do you get hold of it? Not all of us are lucky enough to work with a data scientist or performance analyst, but many work with developers. I’ve had the great fortune to work with some amazing developers who pulled data from databases, transformed it into lovely spreadsheets and helped with the analysis. You never know what data skills might be on your team.
Step 4: Finally, analyse the data
Now you’ve got your spreadsheet. Go back to the questions you started with. Here are some things to try:
Make a copy and experiment
Playing with the data will help you become familiar with it and work out what information you can get from it. But always copy it first. Then you will not have to worry about accidentally deleting a row or column.
If you’re worried about sorting a column and getting things out of order then highlight a row. You’ll soon see if things have gone wrong.
If the data you’d like to use is large and cumbersome to analyse then take a small amount and play with it. This will help you find out if it’s worth accessing the whole thing.
Count and compare things
A couple of years ago someone taught me how to make a pivot table in Excel. Suddenly, counting things and comparing them using spreadsheets became much easier. There are lots of tutorials online to learn how to make them, but – once again – ask around. You never know who in your team may be a spreadsheet whiz. By counting and comparing things you’ll start to create segments which will give your data context and meaning.
Look for things that stand out
Something that looks unusual or seems to contradict the rest of the data might suggest a problem with the service, but it may also mean there’s a problem with the data. Check it. If it’s not a problem with the data, it might suggest something that needs further research.
Bring it together with your qualitative data
Once you’ve done your analysis, put it together with your other research. It will give you a fuller picture and may point to gaps in your knowledge.
What I learned – some final thoughts…
So, there are my 4 steps. Before signing off, here are a few final thoughts to leave you with.
User research with data is a team sport and this provides several benefits:
- the more brains that are involved, the more likely it is that mistakes will be picked up early
- the team owns the research because they have helped create the findings; this makes the research more likely to be understood
- more people in the team will be thinking about what users are doing with the service, not just what the service is doing
However, the 4 steps I’ve laid out are not always straightforward, and data analysis can be time-consuming, so you’ll need to weigh up the benefits for your own projects carefully before diving in.
I hope you find this useful. There are all sorts of data sources that can be useful for a user researcher. What other sources of data have you used and how have you used them? Share your thoughts in the comments below.