We produce a lot of video
At GDS, we produce at least 200 hours of user research video per month. If participants give their permission, we record their user research sessions in our GDS user research lab, in external labs and during field trips. Video offers a raw and untranslated record of what happened in a research session.
But we don’t use it very much
Our researchers agree that video is valuable and must be kept - it’s unthinkable to throw it away. However, video’s currently too time consuming to be of any real use in our agile workflow.
… as a researcher, videos are useful for clips to share immediately but after that it’s rarely (if ever) revisited. It’s just too time consuming.
This post is about we've learned in trying to change that.
We found 3 reasons why we need a video library
We learned that our user researchers need a video library for 3 reasons:
- For secure storage of their videos
- To refer back to and work with video data more efficiently
- To share research, and collaborate more easily with team members
Securely storing video is a good thing, but it's not enough
We asked: Could we speed up our researchers’ day-to-day work with a better video solution? And even better: Could we make video data available to researchers across project teams, pan-GDS, and potentially even across government? If so, we’d be able to build up a body of evidence and make correlations across research in ways currently unavailable to us.
Having a repository is a good thing… Whenever anyone leaves the team, Tara gets their [drive] with all their videos. The research disappears. … [This] limits our capacity to build on a body of evidence. We’re reinventing the wheel all the time.
We looked at what’s available in the video library industry
We researched the video library industry to find out what’s currently available and where video technology is going.
We found an off-the-shelf and cloud-based video library that offered all the bells and whistles common to most top-level suites, but it was unique in that it was aimed at user research environments specifically. It seemed like a perfect match to our needs.
We did a pilot
We’re committed to iterative and agile working, so before making a long-term commitment to the new video library, we did a pilot.
We tried the solution for 3 months and with 3 GDS teams. One of the teams was running a very tight agile research schedule, another was doing thematic research, and the third was doing something in between.
The pilot enabled us to learn risk-free:
- Whether this particular solution was the right one for us
- What video library features were actually important to us in the long term
- Whether our hypothesis was right: If video data is less time consuming, teams will make more use of it/gain more value from it.
In running the pilot, this is what we learned.
Safe storage of video is essential
During the first few weeks of the pilot, I asked researchers what they thought of the solution. I was expecting a discussion around some of the fancier features the library offered, but this conversation was typical:
Researcher: “I love it.”
Me: “What do you love about it?”
Researcher: “Just that I’ve got somewhere central and secure to store my videos. It takes such a weight off my shoulders.”
Me: “Anything else?
Researcher: “Not yet.”
Our researchers are deeply respectful of the participants they work with and value the video evidence they gather. Security of storage is their first priority.
The onus is on us to make our videos secure and get them somewhere where it’s a common area and other people can see them too.
Like Maslow's hierarchy of needs, once secure storage is looked after, researchers look for the ability to edit, share and collaborate on content more easily.
A balance needs to be struck: the video library needs to provide secure storage (to OFFICIAL level) and yet remain accessible enough to allow for collaboration and sharing of content.
Give me a one-button experience
I just want to know that my videos are stored somewhere safe. Any other details, such as metadata, I can deal with those later when I’ve got more time. If I’ve got more time.
Many of our researchers are working on fast-paced projects. The video library needs to make their most crucial task, uploading video - and often several at once - as easy as possible.
Researchers want to click one button to upload.
Search cannot be reliant on metadata entry
While researchers acknowledge that adding metadata to video content would likely make the library more useful in the long term, they often don’t have the time to add it at upload.
Some key things we learned about metadata entry:
- Quick upload is first priority, so metadata entry can't be forced.
- If the metadata is not immediately and personally useful to a researcher, they won't add it.
- It's unlikely that a researcher will return to add metadata once a video is uploaded.
Search therefore cannot be reliant on handcrafted metadata. Instead, search by way of transcriptions (not always ideal, as described below) or, even better, using automatic metadata extraction (technology that's commercially available) is likely the best way forward.
Transcription-based search is expensive and slow
Expensive
If you’re going to leverage transcriptions for search, accurately time-posted transcriptions are needed. The cost of a UK-produced (important for information security reasons) and time-posted transcription is on average £100 per hour of talk time. If you’re storing hundreds of hours of video per month, a library that relies on transcription for search is not realistic from a budget perspective.
Slow
Unless you're going to pay oodles for an express service, transcriptions take 3 days to be delivered. For many of our teams, that's just too slow. Teams observe research in the lab and review and analyse the very next day; they need to be able to pinpoint moments in a video then and there. Three days later might be useful for future searches, and for teams running thematic research programmes, but it's of little use to those running a rapid research schedule.
Still, transcriptions do have value
Some teams feel transcriptions are really important and they’re a regular part of their process:
Transcripts are great for skimming through the transcripts. It’s definitely useful to have the video and the transcript with one another. A time posted transcriptions is the way to go.
Other teams were lukewarm, but still found them useful:
Search via transcription isn’t something I need all the time, but maybe once every couple of months then it would be really useful to that.
Transcriptions have one more benefit: they can be used to produce video subtitles, which some teams find useful in show and tells.
A video library should offer time-posted transcriptions as an option.
4 hours p/week editing videos: we can make better use of our time
Each of our researchers typically spends around 4 hours per week editing videos for show and tells and presentations.
Some researchers felt that being able to edit within the video library was essential:
I had to pull out video clips to explain the themes we'd found in our research. I had 4 x 2-hour video clips to prepare and it took me half a day. This usually takes me almost 2 days.
Other researchers have become adept at using external editing suits. These researchers acknowledge the value of being able to find specific points in a video, but don’t see editing capabilities within a video library as essential.
Most video libraries offer built in video editing sufficient to our needs and its time-saving potential makes it a worthwhile addition to a long-term solution. In fact, time saving on its own makes a good business proposition for a video library.
Audio files are important too
In some research instances, particularly in field research, researchers record audio files. During the pilot, it became clear that audio files should also be catered for within the ‘video library’. Our researchers really need an A/V library. That's easy to accommodate.
A dichotomy of sharing
On the one hand, researchers talked about a central and shared video library as a positive thing. There’s an assumed benefit in being able to search a collective library of research. On the other hand, researchers expressed concern about who could access their videos.
I’ve got a responsibility to the respondent. … I need to put the video into a safe place, but I don’t have the time this week to do all the obscuring [of personal data] stuff. As much as I love my colleagues, I might not want them to see those videos.
In its simplest form, the video library should allow user researchers to set permissions for their videos on upload. With long term use of a video library, it will be interesting to see whether anxieties around sharing change, and if so, how.
Non-user researchers don’t use the system
Although non-user researchers showed enthusiasm at pilot launch, in reality they didn’t use the system unless prompted by a researcher.
We’ve learned that the primary audience is the user researchers: first the researcher running the research, then researchers in their immediate team.
Don’t try to make it all things
Notes should remain on sticky notes. We don’t want a pair-note kind of thing.
We have excellent non-digital and visual processes for note taking, analysis and sharing (such as user research walls). Researchers expressed the importance of keeping those workflows material, we shouldn’t hide those processes or outputs in digital vaults.
Also, a user research A/V library won't necessarily solve all user research insights/outputs needs. But it will look after two of the most difficult of them: video and audio.
Where to next?
As a result of the pilot, we now know that an A/V library should do three things and do them well:
- Securely store A/V content
- Make it searchable
- Make it shareable
It should be an exercise in 'Keep it simple, stupid.'
We’re currently planning next steps, talking to suppliers, stakeholders and punting for funds. Should we get all of this right, we’ll go ahead with an alpha.
If you’ve got any questions, or stuff to share about your user research A/V content, please drop a line. I'd love to hear from you.
Keep in touch. Sign up to email updates from this blog. Follow Kate on Twitter.
4 comments
Comment by Lisa Renery Handalian posted on
Hi, Kate - greetings from across the Pond. 🙂
I really appreciated this entry (among so many others on the blog!). I am a senior researcher conducting Listening Labs (a la Mark Hurst/Creative Good) with an individual user every Friday; it is (virtually) attended by members from our Engineering, Product, Marketing, and Analytics teams. During the sessions, as they listen and watch the user's screen (via GoToMeeting), everyone is expected to listen in and take down notes, comments, exclamations, ideas, etc. We use the chat/productivity tool Slack for this purpose. The beauty of Slack is that it gives you a timestamp at each comment - which, in turn, enables us to pinpoint the approximate spot in the video recording. This prevents us from having to search through the video, while also liberating me from having to take notes or keep track of key insights in the video.
Cheers!!
Lisa | Ancestry.com
Comment by Kate Towsey posted on
Hi Lisa, ahoy back over the pond! Great to hear you found the post interesting. User research A/V content is a bit of 'geekdom' for me - there aren't crowds of people interested in this subject. (Yet!) That's a very interesting use of Slack. It sounds like an excellent way to get remote observers participating in live research whilst also gathering their observations in one place. For on-site observers, we prefer that they write observations and quotes on sticky notes - one data point per sticky note - this way sticky notes can be used the next day in analysis without additional preparation. Still, I love the idea of time markers in Slack. Very clever.
Comment by Simon posted on
A really good blog. Thank you for sharing.
I'd be interested in any tips people have on editing video in general. My skills are very basic, I use iMovie to hack off the front and back of a clip and stick a title on it. That's it.
Comment by Kate Towsey posted on
Thanks Simon. Most GDS researchers just use iMovie and pretty much do what you do. One or two researchers use Camtasia. I've noted editing tips as a topic for a future blog post.