³ÉÈËÂÛ̳

Here in the Distribution Technologies team for ³ÉÈËÂÛ̳ A&M Interactive, we look at how best to distribute media and metadata across A&M for current and future platforms; we also look at how to syndicate our content to external partnerships and the public.

Feeds are a great way of reusing content more easily in an automated way. You're probably familiar with the example of RSS feeds from blogs or podcasts, which save you having to visit different sites to collect the information you want. In A&M we reuse and reversion many different feed formats, not just RSS, to save us duplicating work across different platforms.

Feeds Hub is one of our new projects focusing on registering, reusing and reversioning data feeds.

It is an open-source project that aims to share its solutions publicly. ³ÉÈËÂÛ̳ Audio & Music Interactive will be working with our FM&T colleagues, an independent development company called LShift, and the wider Open Source community to create this new technology.

In order to deliver Feeds Hub we'll not only be calling on our FM&T colleagues and an independent development company called LShift, but we also aim to engage the wider Open Source community and make this an Open Source project; this means we'll not just be helping the ³ÉÈËÂÛ̳ make sense of using Feeds, but helping others with the same problem. In a world where we talk about sharing technology, and giving back to our audience, this seems the most sensible way to approach the project.

However, creating content in this way is usually just the beginning of the process.

Often when you find content you want to use, it's not quite the way you want it, so you spend time processing it yourself for your own project's needs. For example, A&M might want to reuse a web article for mobile, but simply publish a feed of the headline and the first paragraph and provide a link to the full story. We also might want to attach a smaller, low resolution image to the story or attach some metadata to make the story work on mobile. On digital television we might want to reproduce the story with a different set of reversioning rules for that feed. Another part of the ³ÉÈËÂÛ̳ might want to take the same story and again apply a completely different set of rules to suit a service they are building. Then someone else comes along, sees your project producing a nice feed and takes it and processes it again in different way for their own needs.

Chaos reigns when something breaks at the start of the chain - maybe the original web story feed is turned off because no one knows it is being used by anyone else or the original feed is tweaked to add some extra information in the metadata. Downstream, systems that have been built on top of it now break and it is not always clear to those that need to fix them where the original source data has come from.

The number of new projects across the ³ÉÈËÂÛ̳ starting to use feeds in creative ways is growing very quickly - just think of spaghetti... on a massive scale.

So what do we do? What are the options?

We could go down the route of gathering together a centralised 'Feed Usage' committee with members across the ³ÉÈËÂÛ̳, to 'federate' feeds so that they are all produced in the same way but, in practice, this never truly works and is likely to stifle creativity. Often it is quite difficult to convince people to work together when they have already experienced the freedom of doing what they want - often they are concerned that their projects will be delayed. Not all feeds sources that we use or want to use are under our control, things like Twitter, Flickr, blogs, etc. Federation will never solve all our problems anyway - for example, it can't help when a source feed is turned off, it doesn't monitor failures.

We think there's another option that will be much more effective - Feeds Hub. We've been working on it for some time, and it's currently at the implementation stage.

Some of the more high level requirements we aim to solve with Feeds Hub include giving data source owners who register feeds proper usage statistics on how and where their data is used and how effective the feed is (stats on click-throughs). This allows them to take a call on whether the feed should continue to exist. By logging the hierarchy of feed usage and reuse, the system will enable data source owners to contact people that have registered their interest in using their feed - the 'feed consumers' - which will be useful for when source owners wish to pull data feeds or alert consumers to changes in the feed that might impact their systems. Consumers and Feed Managers (those that register or create feeds) alike can subscribe to monitoring alerts about when things break, as they inevitably will, allowing them to take action - perhaps in an automated way. We also want to give people the ability to view feed history so they can check if it is editorially appropriate for their project, how it has changed over time and how stable it has been to date. There will be a basic interface for transforming data so that you won't need to be a software engineer who understand complex code to edit or create new feeds. The advantages of this will be widespread - from supporting our publicly available websites, to our backend broadcast chains.

Feeds Hub is underway, we're currently trying to pull all these features in to the plan - we hope to be able to update you on the project through a number of sources in the coming months - via the RadioLabs blog, ³ÉÈËÂÛ̳ Backstage and also through the communities around RabbitMQ and some other open source groups.

We're excited, I hope you are too.

Alan Ogilvie, Jacqueline Phillimore - Distribution Technologies, Audio & Music Interactive.