Paul Murphy gave a presentation about unlocking data trapped in audio and video files. He discussed trends like the end of the Gutenberg era and humans communicating more through sound and images instead of just text. Current tools are better at manipulating text than audio and video. Transcription and annotation of audio/video files using artificial intelligence could help extract more data by providing context, but context is also its own type of data. The presentation recapped that trends, technology, and context are important factors and that automation of transcription and annotation is the goal, though the state of the art is improving. Paul then solicited any questions and mentioned his company is hiring.

1. Unlocking Data
Trapped in
Audio & Video
Files
Paul Murphy
paul@clarify.io
@prmurphy

3. Córdoba, Argentina

4. Milan, Italy
Paris, France

5. London, UK

6. So what’s
wrong?

7. Context
• My accent
• The way I look
• Bio (maybe)

8. My Background (why am I here?)
Banking » Telephony » Analytics
When things
important, they’re oral
Oral means no trace
Except that that’s no
longer true

9. Why am I excited?
• It’s a hard problem
• It has to be solved (trends!)
• I founded a company to do that

10. Trends
• End of the Gutenberg Pause
• Humans communicate through
sound & images
• Text is an optimization
• Text is dead
• Massive amounts of data are
now being stored in A/V files

11. Where are we today?
• Lots of tools for manipulating text
• Almost no tools for manipulating
audio & video

12. Where are we today?
• Lots of tools for manipulating text
• Almost no tools for manipulating
audio & video

13. How can we compute on A/V?
• Transcription
• Annotation

14. Turks are great!
• Human transcription (APIs)
• Manual annotation

15. Can those be automated?
• Transcription
• Annotation
» ASR
» Artificial vision
Of course!
(or I wouldn’t be here)

16. ASRs
• AT&T
• IBM Watson
• Vocapia
• Speechmatics
• …

17. Vision
• Clarifai
• Orbeus
• Image Vision Labs
• Face++
• …

18. State of the Art
• Last 4 slides

19. Context (input)
• Telephony vs. wideband
• Conversation vs. voicemail
• Music vs. speech
• English vs. Spanish
Better & better data

20. Context (output)
• Gender?
• Identity?
• Relationship?
• Emotion?
• Location?
Need audio.
Transcripts & annotations
aren’t enough.

21. Recap…why does context matter?
• Context helps us extract
more & better data
• Context is data
We compute on data

22. Recap…where are we today
• Trends
• Technology
• Context

23. Recap…where are we today

24. Recap…where are we today

25. Recap…where are we today

26. Recap…where are we today

27. Paul Murphy
paul@clarify.io
@prmurphy
Thank you!

28. Any Questions?
PS. We’re hiring!
Paul Murphy
paul@clarify.io
@prmurphy