The document discusses the BBC's efforts to automatically classify television programs by mood. It describes the BBC archive containing over 1 million items and 650,000 television programs. It also outlines the current methods used for program retrieval, including keywords (LonClasss) and standard categorization. The document then focuses on the development of a mood-based classification system using audio, video, and text analysis features. Ground truth data was collected from the public to evaluate programs based on adjectives from affective theory. Future work aims to combine affective and semantic analysis and generate highlights from sports matches.

1. R&D © BBC 2012
Automatic Mood Classification of TV
Programmes
Sam Davies, Jana Eggink, Denise Bland
BBC Research & Development

2. R&D © BBC MMVIII
British Broadcasting Corporation Archive
• BBC I&A
– Perivale, London
– >1,000,000 items
• ~ 650,000 TV
• ~ 350,000 Radio
• ~ 1.5 million hours
– Since 1922
• BBC Redux
– online
– 300,000 hours of TV and radio
– Since 2007
• BBC Written Archive
– 4 ½ miles of documents
– Caversham, Reading

3. R&D © BBC MMVIII
Current Programme Retrieval
• Infax
– > 1,500,000 programmes
• LonClasss
– > 52,000 concepts
• “pop music”, “Iraq”, “criticism
of growing plants for biofuels”,
“Dover Castle communications
centre”, “fake psychics”,
“posters of Ariel Sharon”.
– BBC Redux

4. R&D © BBC MMVIII
Current Programme Retrieval – BBC Internal
• Infax
– > 1,500,000 programmes
• LonClasss
– > 52,000 concepts
• “pop music”, “Iraq”, “criticism
of growing plants for biofuels”,
“Dover Castle communications
centre”, “fake psychics”,
“posters of Ariel Sharon”.
– BBC Redux
– BBC Snippets

5. R&D © BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation

6. R&D © BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation
• bbc.co.uk/programmes
– More episodes
– Editorially chosen similar
programmes

7. R&D © BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation
• bbc.co.uk/programmes
– More episodes
– Editorially chosen similar
programmes
• Link key contributors
– Editorially identified
– Automatically linked

8. R&D © BBC MMVIII
Mood Based Classification – System overview
Feature Extraction
• Video & Audio Analysis
– Colour histogram, motion
detection, brightness.
– Spectral audio components
• Object identification
– Faces, animals, objects
(Tardis)
– Gunshots, laughter, screaming

9. R&D © BBC MMVIII
Mood Based Classification: Ground truth collection
• Ground Truth Collection
– Video
• 200 members of public from varied
demographic
• 250 programmes
• Asked to classify programme clips
based around adjectives taken
from Affective Theory

10. R&D © BBC MMVIII
Mood Based Classification - GUI

11. R&D © BBC MMVIII
Mood Based Classification - GUI

12. R&D © BBC MMVIII
Other mood based features - Music
• Ground Truth Collection
– Video
• 200 members of public from
varied demographic
• 250 programmes
– Music
• MusicalMoods
– 20,000 members of
public
– 60 theme tunes

13. R&D © BBC MMVIII
Other mood based features - Text
• Identify mood of any text on three axis:
– Valence (positive/negative) e.g. triumphant, love, paradise
– Arousal (amount of emotion instilled) e.g. rage, thrill, explosion
– Dominance (power) e.g. winner, confident, admired
• Increases dimensionality of sentiment analysis
• Use on large datasets negates requirement for syntactical analysis
• Subtitles are more correct than derived metadata (automatic speech transcripts, machine
vision, machine listening)
• Fast, scalable
• Domain independent

14. R&D © BBC MMVIII
Other mood based features - Text

15. R&D © BBC MMVIII
Other mood based features: Text
Precision 0.95
Recall 0.91
F1 Score 0.93

16. R&D © BBC MMVIII
Future areas - Combination of Affect and Semantic

17. R&D © BBC MMVIII
Future areas - Highlights Generation
• Sports matches
– Audio based analysis
– Two stages
• Live match identification
• Interesting section detected
– Accuracy of 78%
– Looking currently to include
social media to increase
accuracy.

18. R&D © BBC MMVIII
Publications & more info
• Davies, S., Bland, D. & Grafton R (2010) “A Framework for Automatic Mood Classification of TV Programmes”
presented at SAMT 2010.
• Davies, S. (2010) “Interestingness Detection in Sports Audio Broadcasts” presented at IEEE ICMLA 2010
• Knoiusz, P. & Mikolajcyzk, K. (2011) “Soft Assignment Of Visual Words As Linear Coordinate Coding And
Optimisation Of Its Reconstruction Error” presented at ICIP 2011
• Knoiusz, P. & Mikolajcyzk, K. (2011) “Spatial Coordinate Coding To Reduce Histogram Representations, Dominant
Angle and Colour Pyramid Match” presented at ICIP 2011
• Davies, S. & Bland, D. (2011) “An Improved Framework for Affective Classification and Browsing of Large Scale
Broadcast Archives” presented at ACM SIGIR 2011
• Mann, M. & Cox, T. (2011) “Music Mood Classification of Television Theme Tunes” presented at ISMIR 2011
• Davies, S., Mann, M., Cox, T. & Allen, P. (2011) “Musical Moods: A Mass Participation Experiment for Affective Music
Classification” presented at ISMIR 2011
• Eggink, J. Allen, P. & Bland, D. (2011) “A Pilot Study for Mood-Based Classification of TV programmes” presented at
ACM SIGAC 2011
• Eggink, J. & Bland, D. (2012) “A Large Scale Experiment for Mood-Based Classification of TV Programmes”
presented at ICME 2012
• Available at http://www.bbc.co.uk/rd/publications/whitepapers.shtml

19. R&D © BBC MMVIII
Thank you
• Questions
• Contact;
– sam.davies@bbc.co.uk