I’m working as a user researcher in the Digital Inclusion team at GDS. Over the past few months, a key part of my work has involved sizing and identifying the characteristics of digitally-excluded service users, and understanding their support needs.
This work has been funded by a number of government departments and involved a UK-wide quantitative approach.
When a research agency makes sense
When we do service design projects, we do most of the user research in-house. Occasionally, we work on projects in which we need to learn about the sizes and needs of different markets. These projects need a different methodology - something more like market research, and with a much larger sample size that's nationally representative (>1000 respondents). In these instances, we tend to bring in outside help.
We collaborate with experts
Working with an agency also means that we’re able to collaborate with experts and bring in vital skills and experience that would otherwise be difficult to access in government.
No time to sit back and relax
Although the agency is responsible for the fieldwork, delivery and analysis of the data, there’s still a large amount of work required of GDS and stakeholders in departments.
This involves, amongst other things: writing briefs, agreeing on objectives, questionnaire design, agreeing analysis structure, keeping an eye on the budget, and, above all, making sure that stakeholders have significant project buy in.
In this post, I’ll expand on a few of these things, and talk about what I’ve found useful in working with an agency to ensure the project is delivered on time and provides the information we need.
Make sure your research brief is clear
It’s essential, when conducting any research, to have a clear understanding of the need for the research. This includes the background and context, definitive objectives, suggested methodology/sample, timings, and budget.
When using an agency, it’s even more vital that the aims of the research are clearly set out, so that the agency has a good understanding of what’s required. This will enable them to provide a clear proposal that meets the objectives, is delivered on time - for the correct budget - and adds value.
It’s also important that stakeholders have adequate opportunity to contribute to the brief. This will help ensure that their core requirements are covered, that everyone’s priorities are transparent, and that it’s clear what the project will deliver.
Make sure the project scope is clear
A successful project relies on the scope of the research being clear.
Also, it’s important that everyone buys into it. Make sure to allocate a significant proportion of project time for sign off between stakeholders and the agency.
If the scope of the research changes, it commonly results in increased cost and timings - something we always want to avoid.
Understand your budget and watch it closely
The scope of the project will always be dictated by the amount of money available. It’s important to understand exactly what your budget can deliver, and where additional money will be required if the project scope increases.
For this reason, it’s important to keep a close eye on elements that can lead to the budget spiralling. These might be, an increase in the number and/or complexity of questions or a larger or different sample - particularly very niché or hard-to-reach samples where penetration in the population may be low.
Failure to keep an eye on these kinds of changes may result in a project becoming much larger, and more costly, than originally planned.
If this happens, refocus on the original research objectives and cut back on anything in the research approach that doesn’t address them. This often means cutting back on the questionnaire length and/or question complexity, but can also require prioritising the sample of people you need to interview, particularly key subgroups.
Questionnaire design: make the most of your questions
When designing the questionnaire, it’s important that the agency is involved early on so they can provide advice on structure, length, and phrasing of questions.
It’s also important that the objectives of the research are kept front of mind, this will help ensure that each question has a clear purpose and addresses the objectives of the research.
Be careful to avoid duplication: avoid asking questions that address similar points or will give you similar data.
Finally, it’s always tempting to add ‘just one more question’. Remember that ‘one more question’ may not add value to the research outputs and will always add costs. Depending on the length of the question and the number of people being asked it, you can expect £1k - 4k per question.
Think about how the data will be analysed
Set aside time to think about how the data will be analysed. The agency should take the lead on this, but it’s vital that stakeholders are clear on what subgroups are of interest so that findings can be tailored to address specific user groups.
This should be finalised prior to fieldwork starting so the agency can set up data processing guidelines, and the data is easily available for analysis.
Agree on a general structure for the presentation of data
It’s important to agree with the agency a structure for how research findings will be presented.
Ideally this should be discussed prior to the fieldwork being completed so there’s time for stakeholders and the agency to feedback on suggestions. However, sometimes, it makes sense to finalise a presentation structure once toplines are available so you understand what the data is telling you.
Failure to agree on the general structure of the presentation of data ahead of preliminary analysis can often led to additional work, and the final outputs being delayed.
It’s also important to agree on the extent to which the agency should make recommendations. This will help ensure that any recommendations given are realistic and appropriate to the business context.
In conclusion
Projects with multiple stakeholders are always time consuming, even if the research is being conducted by an agency. You can rely on a lot of resource going into the initial setup and delivery of the findings. However, with joint ownership and the forging of team spirit between all parties, it can be rewarding, both in the way in which you work as a team and also in the results it delivers.
Keep in touch. Sign up to email updates from this blog. Follow Nick on Twitter.
(Nick's quiet on Twitter. You might prod him into conversation.)
1 comment
Comment by Jane Reid posted on
Excellent work. On the DWP project I am assigned to we work with agencies a lot, as part of the User Research role (scoping a new qualitative commission right now) and agree with everything you say! We find using agencies particularly useful for conducting research with people who prefer not to talk us directly about certain issues. It also gives us more scope to get the findings back using more engaging methods, for example; smartphone video 'selfies' the power of these really helps get stakeholder buy-in to the findings.