³ÉÈËÂÛ̳

Unless you know exactly what you’re after, finding the ³ÉÈËÂÛ̳ content that is exactly right for you can be a little like looking for a needle in a haystack. Some challenges we face at the ³ÉÈËÂÛ̳ are:

1. We produce a vast quantity of content, at a rough estimate, around 2000 pieces of new content a day. In terms of our ‘shop window’, we have a good number of promotional areas and slots – on homepages, in our apps, on our schedules and on our social media feeds – but simply not enough to accommodate all our content.

2. When content makes it to a prominent promotional slot, it will, of course, get good traffic, but there is no guarantee that the person for whom that content is most relevant will see it at the time. And once that content loses its place it will be hard to find, indeed, no one will know it exists unless they are armed with determination and the greatest of digital search skills.

3. Colleagues working in curation have an incredible ability to seek out the best content and ensure that it is labelled in a way that maximises its impact, but they simply can’t read, watch or listen to all of our output. There is just too much of it.

So far, so familiar.

Personalised recommendations, fuelled by the power of machine learning, are what every forward-thinking media and tech organisation is doing to put the content that audiences would most enjoy right in front of them. The ³ÉÈËÂÛ̳ has automatic recommendations in , on and on some news language services like , which use strategies like content similarity, popularity and collaborative filtering.

The curatorial challenge for the ³ÉÈËÂÛ̳ becomes more interesting and complex, because our guiding principles relate so strongly to delivering public service value to the audience. We cannot simply work out what would get the most clicks, and show that to our audiences (although ensuring content is attractive to our audiences is important). Instead, we are required:

• To provide impartial news and information to help people understand and engage with the world around them
• To support learning for people of all ages
• To show the most creative, highest quality and distinctive output and services
• To reflect, represent and serve the diverse communities of all of the United Kingdom’s nations and regions and, in doing so, support the creative economy across the United Kingdom
• To reflect the United Kingdom, its culture and values to the world

These requirements, written into our  are why editorial values are woven into our Machine Learning Engine Principles and why those working in machine learning work closely with editorial teams. I recently conducted a deep exploration of what public service curation means to an organisation like the ³ÉÈËÂÛ̳, so that we can begin to identify some of the signals that can help us as we build next level public service recommendations services.

Around 80 editorial staff, involved in some way with digital curation or content creation, took part in the discussion. Around 40 Design & Engineering colleagues observed those discussions, mostly those working in areas related to data science and engineering, metadata and research. Five criteria for public service curation emerged:

1. We want our content to reach and engage as large an audience as possible. We have a role in the national conversation by bringing the most important and resonant stories of the day to the attention of our audiences, as well as prioritising content, that is popular and has universal appeal for the greatest number of people. Examples of this might include content that brings people together like Strictly Come Dancing or content of global importance like Seven Worlds, One Planet, both of which have an impact on the national and cultural conversation. Popular and impactful content also represents good value for money.

2. In a seemingly contradictory move, we don’t always optimise for peak popularity. We make and promote content that aims to appeal to different audiences, groups, communities, regions and perspectives. We want to showcase content that feels personal. Take the podcast Netballers, which is about a sport with less national impact than, say, Premier League football. In terms of feeling relevant, it’s a win for women, young and BAME audiences. And it reflects an aspect of British culture. By its nature, it doesn’t get as many downloads as the Peter Crouch podcast, but it serves to inform and entertain those with a passion for netball.

3. We strive always to bring audiences something new - newly released music, new writers, new presenters, untold stories, news about events and stories of national significance, information about emerging technologies, research, discoveries, perspectives, major drama series, ways of storytelling. This quest for the new is the kindling that starts the national and cultural conversation in the first place. The ³ÉÈËÂÛ̳ produces a lot of this type of content, but it is not easily identified by engines which are based on collaborative filtering alone*.

4. We provide useful, helpful, practical information, explainers and fact checks, which makes us a trusted source of information: that could be a news story, or revision notes for a Chemistry GCSE, or recipes for a quick mid-week meal, or sports results.

5. We have enormous breadth and depth which means that there is something for everyone: there is variety by topic, tone, format, duration, location, level of expertise and age suitability and relevance.

Our long term ambition is to use this thinking to build recommendation systems which can broaden our audience’s horizons, providing different perspectives and stories and experiences that they might not otherwise have come across. But this is complex - and a recommender that can effectively deal with as multi-faceted an editorial issue as impartiality is extremely challenging and will take significant time to develop.

For now, curation in the context of our recommenders involves upholding our editorial values and finding ways to surface the most relevant and compelling content for each user. All the content the ³ÉÈËÂÛ̳ makes for our UK audiences, one way or another, is public service, so our recommendations will of course always have a public service flavour.

I’m excited that we have already built models with business rules about increasing breadth (in the Sounds recommender) and depth (in the World Service recommenders), and as well as reflecting editorial values around sensitivity. For example, on the Sport recommender, due for release next year, we’ve taken the curatorial decision that content from rival teams will not be shown together in a set of recommendations.

We can learn so much about public service by sharing the editorial point of view as we iterate and refine our approach. I’ll be working in collaboration with editorial colleagues and data scientists to ensure these public service curation criteria inform the ³ÉÈËÂÛ̳’s future recommendations engines.

Lastly, we also recognise that the power of machine learning - of which recommendations is a part - can only get us so far. Machines cannot understand all the subtleties, complexities and nuance of editorial decision making. An algorithm will have trouble identifying what is entertaining or fresh or authentic without significant human assistance. A machine can help only up to a point to accurately tag content in a metadata system before a human verifies the machine’s choices and hits ‘publish’.

So at the ³ÉÈËÂÛ̳, we’ll be maintaining a human hand in content creation and discovery. We need both humans and machines to best serve our audiences; editorial colleagues are highly skilled at making and promoting our content and machines can help amplify those skills. More and more, what recommendations can do is help locate the needle in the haystack in the first place - and exactly the right needle for you.

*Which is why we have built a factorisation machine for our Sounds recommender which combines collaborative filtering with content matching. For more detail see Developing personalised recommendation systems at the ³ÉÈËÂÛ̳.