³ÉÈËÂÛ̳

Applications - whether on second screens or on TV sets - offer the possibility of access to viewers' activity data, personal data, and social network data so that TV providers such as advertisers, schedulers and programme-makers may soon know more about their viewers, enabling them to offer greater personalisation. Our role in NoTube has been to come up with ideas that illustrate how the integration of TV and the Web, and particularly the integration of TV and the Social Web, might make watching TV in the future a more enjoyable and interesting experience from the viewer's point of view. Throughout the project we have been thinking of new ways in which people might want to , while trying to avoid obvious and near-term applications. Based on these scenarios we've created several working prototypes, combining technology components from other parts of the project.

N-Screen: drag and drop sharing and TV control

The most recent and polished of our prototypes is , a playable Web application designed for use on second screens. N-Screen aims to help small groups of people explore large collections of videos together in real time, whether they're in the same room together or in separate locations. Anyone can set up an N-Screen group and invite specific friends to join it. Group members can then drag and drop suggestions for things to watch to each other. Once the group has chosen what to watch together, the selected video can then be dragged and dropped to a TV icon to start playing it on a traditional shared TV screen. N-Screen's 'Connected TV' is a web page capable of playing streaming and on-demand HTML 5 and flash video. Users connect to a specific TV via a pin number. Multiple TVs in different locations can play the same content simultaneously.

N-Screen Drag and Drop second screen user interface

N-Screen integrates various developed in different parts of the project (personalised, programme-to-programme, collaborative filtering, quasi-random suggestions) into one end-user interface, providing viewers with multiple ways of finding interesting content to watch. The idea is that this hybrid approach can help to make TV more enjoyable by suggesting interesting new content that a person might like - for example, a TV programme that they wouldn't have normally come across otherwise. The role of this kind of serendipity in helping people to discover new things that they find appealing (whether it be films, books or music for example) has become a current hot topic on the Web.

Part of the quest for serendipitous content discovery is driven by the growing trend for 'social' recommendations based on friends' tastes and preferences. One of the motivations for using N-Screen is wanting to share ideas for good things to watch - with a group of friends, or with specific individuals in the group - and similarly to receive suggestions from friends. However, rather than automatically sharing things with everyone in your social graph ('frictionless sharing'), in N-Screen sharing is more targeted: it is limited to specific friends that you've invited to your group, and is based on your tacit knowledge about their likes and dislikes.

In one sense, N-Screen acts as a different kind of remote control interface to the TV. It also acts as an interactive companion to the TV, with the ability to present related information at a specific point in a programme. It is also a personalised and social filter to available television and radio programmes, whether those are live or on-demand, or from the ³ÉÈËÂÛ̳ or other providers such as TED or the Internet Archive.

Beancounter: putting you in control of your personal data

The other main prototype we have been working on recently is NoTube's 'Beancounter', developed in collaboration with several of our project partners. Beancounter is a user-profiling service that implicitly determines a person's interests based on their activities on the Social Web. This interests profile is useful because it can be automatically used as input for personalised TV recommendations. The data that the Beancounter has collected and analysed about someone can also be presented back to them in attractive and meaningful ways. Beancounter-like services are becoming more and more relevant across the Web, with the growth in data mining of activity and Social Web data as input to recommendations and other personalised services. Beancounter puts the user in control of her own personal data so that they can choose how much to share with other organisations. We have been gathering user feedback about the Beancounter over the last few weeks, looking at how people feel about their data being collected and used in this way: you can read more about that .

Themes

These two applications encapsulate many of the core themes of the project. From the user's perspective these include: finding something to watch when faced with an overwhelming choice, the role of social recommendations, perceived trade-offs between personalisation in exchange for loss of privacy, and how best to interact with the Web and TV simultaneously. These are all issues we have seen become more mainstream during the lifetime of the project as the Web and TV have become more closely integrated.

Metadata and the Semantic Web

Behind the scenes, the project set out to demonstrate how Semantic Web technologies could be used as a tool to connect TV content and the Web through Linked Open Data. Since the ³ÉÈËÂÛ̳'s part of the project has focused on technologies that can help people interact with each other while watching TV, for us this has mostly involved building on previous ³ÉÈËÂÛ̳ work by extending the infrastructure for using URLs to identify specific programme episodes. We were able to use ³ÉÈËÂÛ̳ data to create a inspired by to go from the crid identifiers available over the air on TV to the http URLs the ³ÉÈËÂÛ̳ provides, and turn this into a service, and to use this and other data to create applications such as N-Screen and Beancounter.

As we argued in (with Mo McRoberts) to W3C's Web and TV workshop, being able to access basic metadata about programmes - particularly URLs for programmes - is very important for Social TV use cases, because links are the basic currency of social media, and when you share a link you want it both to uniquely identify the thing you want to talk about (so other people can share it), and also provide more information about it (so people can find out more about it). Making a small amount of metadata available not only enables interesting applications to be built which help people explore what's available, but also helps people share what they found with others: it can act as a personal advert for the programme.

Synchronisation and Pairing

Given the growing importance and potential of second screen applications, some of the technical questions we needed to address were: How do devices find out about each other and communicate with each other, and how can we make the process very simple? What are good technologies for syncing TV and other metadata, and sharing information in real time?

We chose XMPP as a scalable and mature infrastructure on which to build applications. XMPP (or 'Jabber') is an XML messaging technology with useful capabilities such as permissioning and contact lists as well as multiple open source implementations, including client libraries in Javascript. In some of our early prototypes we used XMPP permissioning mechanisms to pair devices with TVs (for example using audio signals, QR codes, or numeric pins displayed on screen), the idea being that you could become 'friends' with your TV by having information such as a pin only available to devices in close proximity and thereby restrict the devices that could control it.

For N-Screen we used the anonymous group chat feature of XMPP, experimenting with pushing some of the permissioning out into social mechanisms that people already have, so for example, people can watch together apart by sharing a private link, which enables real-time 'conversations' between the TV(s) and the group of human participants 'I'm playing a documentary on ³ÉÈËÂÛ̳ Four with the following link...', 'I'm changing the channel to ³ÉÈËÂÛ̳ One' and so on. Behind the user interface to N-Screen is a small javascript library, buttons.js, that wraps some XMPP commands in a simpler API that makes it easier to write UIs that send and react to different kinds of messages.

The precision of synchronisation is an important aspect of these kinds of second screen systems. If you need your second screen device to show you a separate audio track you need sub-frame-rate precision, but for the 'sharing programmes' 'watching together apart' and 'providing additional information' usecases the precision can be much lower, and XMPP group chat works well.

Some Conclusions

Probably the most interesting part of the project has been the user testing. It was very clear from our research that people want to watch TV together. They like talking about TV - they like people having watched the same thing as them, and older people miss the days when people were much more likely to have watched something they had. To get the social benefit they don't have to watch it at the same time as others but sometimes this is fun. People like recommendations from friends, either personally or via social networks: social networks here are an extension of face to face interactions; but not all recommendations from friends are equally good. People will trade-off privacy and personalisation, especially if they see the benefit - they are getting used to systems that do this, such as Amazon.

Example Beancounter analytics based on Twitter activity

As TVs become more complicated, interacting with them using remote controls is becoming much harder. In particular, user experience between applications is inconsistent, text input is difficult, and reading text is difficult. There are no conventions yet for tablets as first screen controls, so their behaviour is not well understood in this context, although in general people enjoy using tablets and find them intuitive. We expect that second screen devices (tablets and smartphone) will continue to play a major role in complementing TV.

Developer Communities and APIs to TV

On a final personal note, coming from Web backgrounds, it has been very interesting to see the cultural differences between the Web and TV developer communities. TV development is relatively slow-moving for very good reasons: people do not update their TVs very often, and reliability has always been very important. Specifications are typically not generally available and are very long. Web technology by contrast, iterates very quickly, often in public, and specifications are (often) short and freely available. Open specifications and particularly open data and APIs mean that collaboration can be lightweight and applications can be built very quickly. This open nature of the Web contrasts very sharply with the closed and controlled world of TV, and the differences in perspectives limit the amount of integration.

There is huge potential to harness the abilities of Web developers to make TV more interesting, but for now, interacting or synchronising at all is very difficult and dependent on techniques such as audio fingerprinting. From a developer perspective a major incentive to the creation of second screen or on-screen applications would be a simple, Web-developer-friendly, cross-platform API, that enabled Web developers to control a TV and get information about what is playing on it. Our colleagues in another R&D team have produced the Universal Control specifications in order to try and address this potential.

There is also some integration happening at the technological level. Widgets and applications on smart TVs typically use Web technologies. Perhaps most interestingly, Web browsers in TVs are becoming very full-featured, to the extent that on some TVs, the N-Screen 'television' (which is just a web page) can in fact be an actual television (albeit in a web browser). We think one future development may be that TVs get dumber rather than smarter, with interactions on mobile devices/tablets which have better user experience. If there are no Web-developer-friendly APIs to TV, then HTML5 browsers are one way this could be implemented.

Future work

Although NoTube has finished, both ³ÉÈËÂÛ̳ R&D and VU will continue to work on some of these themes. The FI-Content project is undertaking work related to user data, and work on real-time data and recommendations continues in the upcoming VistaTV project. We and our colleagues in NoTube have blogged extensively about our work over at .

We would like to thank all of our NoTube colleagues for making NoTube such an enjoyable and interesting project.