The Finding Things team recently ran a mock tagging workshop. We did this to learn how content designers apply subject tags to content to say what it’s about.
Here’s how we did it and what we learned.
How we ran the session
We asked a group of (mostly) writers from different organisations to look at content from the audit of early years that helps organisations prepare their content for inclusion in the GOV.UK taxonomy.
In a group session we asked everyone to choose 1 or more terms from the draft taxonomy to describe what each page is about. The draft taxonomy contained 8 terms drawn from user research, log analysis and content audits.
They entered the terms in a shared Google sheet. We weren’t testing an interface, so we deliberately abstracted the mechanics of how it might work. We wanted to keep people focused on the meaning, not the technology.
Once they’d tagged each page, we asked how confident they felt about their choices, and whether they thought a new tag was needed.
After the individual tagging, we brought the group back together to discuss what they’d done. It worked like this:
- people paired up to mark on paper how often their tagging was the same: this was to get people thinking about how they approached the task and to quickly give us rough data on consistency
- We held a brief retrospective, using the ‘liked, lacked and learned’ format, to capture people’s thoughts, feelings and ideas
Preparing for the session
We tested the tagging task with a few people from around GDS before the workshop as a pilot, to help us find any problems with how it worked. This showed us that:
- we hadn’t added enough columns in the sheet to allow people to choose as many tags as they needed
- people sometimes wanted to tag to a higher level in the taxonomy if they felt the content was about lots of things (for example, ministerial speeches, which sometimes touch on different areas)
What we learned
We learned a number of different things about tagging and the taxonomy that will help us in the weeks and months to come.
Granularity versus consistency
Tagging was mostly consistent between editors. But we know that consistency reduces as the set of available tags grows. This shows the balance we’ll need to strike between the granularity of terms versus consistency of tagging.
On average people applied 1.5 tags to each page. We know from other research on GOV.UK that different users use different terms to find the same content, and that having more tags can help users find what they need. So we’ll need to make it easy to for editors to tag content with multiple terms.
Tagging is sometimes easy
In some areas, it’s quick and easy to tag content. For example, if the title of a page includes ‘Early Years Foundation Stage’ we can be sure that’s what it’s about.
If an organisation has a single function which corresponds to a term in the taxonomy, any content it publishes is likely to be about that subject. For example, most of Ofsted’s content is (unsurprisingly) about inspections.
And sometimes it’s hard
It’s harder to tag content that relates to different areas. For example, some Ofsted content is specifically about inspections of early years providers and some relates to other kinds of inspections.
We also discovered some terms people felt were lacking, and terms that could be interpreted differently by different people. For example, some people thought ‘Childminders’ included childminding agencies, and some didn’t. Also, the distinction between ‘Nurseries and pre-schools’ and ‘Children’s centres’ wasn't clear.
What’s next
We’re working on the education taxonomy at the moment, taking account of what we learned from this session. To make sure tagging is consistent across education content, we want the terms in the taxonomy to be as self-explanatory as possible.
And knowing how many tags people typically apply to each page will help when we’re designing the interface for tagging.
Thanks
Thank you to colleagues from the Standards and Testing Agency, Education Funding Agency, Ofsted and the Department for Education who generously gave their time.
Keep in touch. Sign up to email updates from this blog. Follow Alan on Twitter.
Featured image by @glynlowe, Library of Congress Classification - Reading Room, CC BY 2.0
2 comments
Comment by Luisa posted on
Interesting article! I'm working with tags too and I was wondering if you can answer a couple of questions:
- Are you considering imposing a limit of tags to a given piece of content/page?
- Are the tags created before hand and selected from an existing list or can users create their own tags on the spot? If so, how do you deal with variations of the same tag?
Cheers
Comment by Alan Maddrell posted on
Hi Luisa
Great questions. We haven't got far enough to be able to answer them yet though. The interface we build will need to encourage publishers to only apply tags that accurately describe what the content is about. The tags we'll have are likely to be fairly specific, so the issue of having a long list of tags on a single piece of content shouldn't occur. We do have the problem of what to do when content is about more than one subject (eg some political speeches). We either tag to a higher level (for example 'education') or a longer list. We're tending towards the former at the moment, but it's early days.
The plan is currently to have a pre-defined list of tags in a centrally managed taxonomy. However, people will need to be able to easily suggest new tags.