Some of this year's exciting trends and cautionary tales from the Interactive Conference at this 2017's SXSW festival. Presented with my Teneo colleagues Mark Wainwright and Tom Cook.
Key Trends
• Mobile Messaging
• The Evolution of Video
• VR/AR – don’t believe the hype
• Connected Lives
• DigiHealth
• Ethics 2.0
• Voice as the UI
• Under the surface of AI
• Polivation and short termism
The newest form of Dark Social is mobile messaging. Dark social having x3 impact of Facebook newsfeeds.
Keeping within the vein of dark social and mobile messaging a session lead by Vivian Rosenthal of mobile messaging agency Snaps in the US
Larger audience than social media
98% open rates in the US
40% of US check messaging before any other apps on their phone
iMessage = 200,000 messages per second
What opportunities exist for brands:
Chatbots
Branded Keyboards
iMessage Apps
Chatbots have come of age but what’s the true value to brands and businesses?
Ultimately there’s a great opportunity for Chatbots to usurp mobile apps, google search and potentially even people…
Chatbot utility
Key consideration for businesses
We need to be careful not to become lazy and offload tasks we don’t want to deal with to bots. Four key areas that bots are being used most at the moment:
Customer service
Can be a heated moment that needs human interaction and bots or AI can’t always be trusted here to deliver an effective solution beyond regurgitated phrases and cyclicar logic
That said some elements like opening hours, store locations and contact details could easily be funneled through chatbot technology
E-Commerce, Entertainment, Companionship and Support
There was an ongoing theme at this year’s SXSW: Now that we’ve developed all of these interesting and creative technologies how can we use them for the betterment of society?
So it should be no surprise that when you consider the current political landscape that immigration is a topic.
The panel discussion Can I Order a Drink via my T-Shirt? Touched on the huge societal beneficial impacts offered by bots. James Thompson of Diageo NA discussed Visa Bot as the perfect marriage between tech and social purpose and app designed to help immigrants within the United States in their Visa applications and matters of legal residence.
E-Commerce has huge opportunity in the Chatbot space and we heard how Nike Jordan and Marriott Hotels in the U.S. both used chatbot technology to engage with consumers in ongoing conversation on social to build affinity and drive conversions.
Shopsping created an automated personal shopping assistant that had their entire catalog uploaded to the functionality of the FB chatbot allowing for clever searching, specification and utimate conversions.
We’re moving out of the browse mobile, purchase desktop. Consumers faith in mobile shopping has increased and comes at a perfect time to driving this type of interaction.
BIG question though – WHO DO WE HOLD TO ACCOUNT THE ACTIONS OF AI?
Other chatbots of note include:
Hendrick’s Adopt a Cucumber Chatbot, TransferWise, Stoptober Bot
With the advent of the iMessage App Store:
Opportunity for brands to created branded sticker packs
Increase awareness in every day life for consumers
Dunkin Donuts key takeaways –
Consumer insights for creative
Link with existing mobile apps
Do fewer, better
Media to aide discovery
Branded keyboards
The Love Your Curls branded emoji keyboard tapped into a key audience insight – no curly headed emojis
To build brand love with their shampoo customers Dove helped solve this problem
(Emojis originally from Japan so only straight hair)
The Dove keyboard earned the brand 1billion impressions globally according to Mindshare’s Alana Diehl
- VR and AR have backed themselves into a corner – and only have themselves to blame.
Video is evolving:
360 cameras were everywhere at SXSW. At one session someone held this up and the speaker thought they were asking a question. Distinct lack of selfie-sticks. Is this the future?
VR is becoming more familiar
360 is not VR – 360 is easy VR is not
Best in class doesn’t exist yet
Infancy of landscape – good knowledge sharing community
Worth investing in proper specialised VR directors / production teams
VR pods were everywhere at SXSW
Motion added next level to VR experiences but it’s not the VR that we’ll have in our homes
Safety concerns when using VR need to be addressed
Facebook Oculus’ VR For Good Project that included – Spectrum Laboratory allowed users to experience autism as a child
Immersive narrative is key for a good VR and considering the 360 spectrum at all times not just focusing on 2D view with 3D surrounds - Weaker example came from The Mummy VR
- VR and AR have backed themselves into a corner – and only have themselves to blame.
2016 at SXSW marked a breakout year for VR and AR – big announcements from most of the big players in this field
Expectations at an all time high – examples of VR use popping up all over the place
This was backed up with predictions that VR / AR could be a “$150 billion business by 2020”
However, we seem to have hit a plateau where VR / AR could be accused of stagnating – 2017 is arguably therefore a make or break year…
And there’s also too much of this
Not everyone wants to pull on a haptic suit every time they experience VR
Too much emphasis on finding the WOW factor for VR
AR is not much different, Pokemon Go marked a landmark moment for AR – mass global participation in the use of AR functionality
But what was the follow-up? How has this been turned into something genuinely useful
Niantic rant…
Where is the application of AR to create smart cities and urbanised spaces?
AR is entering mass market.
Pokemon Go lauded as first outing of AR in gamification.
Lots more practical uses that we witnessed at SXSW beyond using a phone screen or hololens glasses
Implications for every facet of life
Motoring – HUD from startups like Navdy
Educational – National Geographic’s usage was inspiring
Medical – Understanding bodily functions
Expensive show pieces at present
Opportunity to enhance every day life
To succeed in VR we need to take a step back and think about what it is that makes VR and AR special
Immersive storytelling
As a medium for storytelling, immersing people into situations or environments they otherwise wouldn’t be able to experience, the power of VR is huge
Great stories will always be shareable, and this is the key takeout for brands
By all means consider VR, it has huge potential – but only if you keep things simple instead of looking for the WOW moment
What story do you want to tell? How can you use VR to tell it in a different, more immersive way.
E.g….
Wearables are starting to add proper value to our lives
From Visa’s NFC payment through sunglasses to Snapchat’s Snapbot wearables were visible around Austin
What’s more exciting is the innovation in wearables and how they’re getting smarter in their connection
Google’s and Levi’s Jacquard shows real sophistication in wearables
Responsive threads woven into jacket denim
Bluetooth transmitting button
New gesturesfor phone users including: brush away, brush toward, tap, double-tap
Application reaches far beyond clothing – upholstery, bedding, limits are endless
Jacquard is expected to go to market Sept/October 2017 with price tag from $350-400USD
The Internet of Things You Don’t Own Panel
When you buy items now from your smartphone to your coffee machine you don’t really own them
Licensing the software – Digital Rights Management exists even within our coffee makers
Keurig example that tied users into using expensive brand coffee pods
Big conversation in the US in relation to John Deere tractors and the software that power them
Apple keenly interested as it effects them too: Apple TOS doc is 20,000 words – longer than Hamlet but surrenders RTR
Nebraska, Wyoming and Kansas currently considering bills around the entitlement to repair your own machine – customise as required
But how far is too far?
Cloud Pets seemed like a neat idea, but as Aaron Perzanowski from the Case Western Reserve University explained turned out to be lacking security.
The toys allowed parents and children to connect via a Wi-Fi enabled stuffed animals to send messages while Mammy or Daddy is away for business.
The $40 cuddly CloudPets feature microphones and speakers, and connect to the internet via an iOS or Android app on a nearby smartphone or tablet.
Families can use the fake animals to exchange voice messages between their children, friends, and relatives.
Due to a hole in the authentication over 2million messages between parents and their kids were hacked and held to ransom.
So how safe is this new Internet of Things?.... Well.
How far is too far?
We-connect long-distance connected sex toy that tracks usage, speed and all sorts of intimate details against a personalised account.
Consider if this information got hacked?
Partnerships / transparency and trust is key for strategising IOT for Diageo
GoogleX tech moving into areas where expertise already exists – e.g. auto industry
Tech company and government – pushes boundaries of comfort zones
Transparency is important in the new landscape of wearables to mitigate privacy concerns and fear of data usage
But what about where we need to share information more freely?
VP Biden discussed the challenges of the US Health System and the Biden Foundation
The Cancer Moonshot features a Public Cancer Database
It’s been accessed 200m times
Cloud storage donated by Amazon Cloud Services
Aim at SXSW was to reach out to tech innovators to help make this a reality
Health is the biggest opportunity for wearable tech but needs to be accessible for all.
Wifi Connected Blood Pressure Monitor and New Braille smartwatch launched.
However new ethics must be thought through…
Penn State University research used Twitter firehouse data to determine likelihood of heart disease for users based on lifestyle
App to determine blood alcohol levels – Diageo to disable cars to prevent drink driving. Should they have access / permission to do this? Ethical dilemma even when trying to do good.
People need to be allowed to own their own data.
Vinome – DNA analysis
Other health and wellness innovations we spotted at SXSW included the Panasonic House where they teamed up with the Parsons College of Art and Design.
Three examples included:
Sleepwise – wearable health and wellness product to assist in comfortable nights sleep
Interacts with your connected home devices to control light / heat / noise in the home and monitor movements during the night
CalorieCo – a microwave sized device that can scan loose foodstuffs with an infrared laser and identify the amount of calories / carbs / protein contained within and then synch with diet tracking apps to log food intake
Digital Cobbler – a device that scanned your foot in 3D and then custom 3D printed footwear specifically matching the contours of your feet to aid any mobility issues or special requirements
One key topic discussed at length in the health space was Innovation outpacing Legislation:
Vinome – a starup from California taking DNA samples to recommend wine
MyAncestry.com
Bio-security needs a security update panel speaker Ed You from the FBI discussed the challenges around:
3D printing of diseases as a bioterrorism risk
DNA based security risks – retina, biometric scanning
Need for legislation around DNA Data protection to catch up
Onus on industry to use caution and help bring public service on the journey when developing new tech
UK is country with highest “Views are my own” on Twitter so we’re used to usual data protection but this is a new battlefield
Social inclusion and diversity in tech:
To pivot from the worrying doom and gloom of bioterrorism there were great examples and a swell towards how we can use our new innovations for good including:
The Human Rights Foundation asking for USB donations to smuggle content from the outside world into DPRK North Korea in a project they’re calling Flash Drives For Freedom
Largely the content featured is soap operas showing romance and joy – two emotions not present in North Korean lives
And in an enlightening session on How an Emoji Becomes an Emoji Jenny 8 Lee explained to us the story of a 19 Year Old non-muslim German girl working on the world’s first Hijab emoji submission showing inclusiveness and racial diversity.
We’re currently experiencing an evolution in way we communicate with computers and smart devices
As humans we’ve always aspired to communicate with computers through speech – this is something that is evident from sci-fi (Hal, Jarvis etc.)
Evolution of User Interfaces – from computer language to human speech
Graphical UI > Conversational UI > Voice UI > No UI
Today, most of us become bilingual from a early age - we quickly pick up both computer & human language capabilities
The drive to No UI is to move beyond this, to rely only on spoken natural language to communicate with a computer - i.e. no translation being required between human and computer
Given this evolutionary process is tied to the technology available, the process began on a graphical level with screens, keyboard and mouse
Graphical UI doesn’t come naturally like voice, but screens are a valuable resource for sharing information - they allow us to see a large array of information and give us more choice
The move to Conversational UI came with the arrival of basic AI assistants like Siri and Cortana – moving from purely Graphical UI to the ability to ask very basic questions of your device, however still heavily reliant on a screen
Conversational UI gained traction in the shadow of app fatigue – the fact that apps no longer provide an efficient enough UX for us
The last 12-months have seen us take the first tentative steps into Voice UI – the manifestation of this being voice-first home assistants such as Amazon Echo and Google Home
These are currently the closest examples to the No-UI paradigm (pure voice input/voice output)
Why now? Technological advances and Big Data
Voice-first UI is finally taking off because of advances in the technology used to train deep neural networks – enabling them to take and understand much, much larger sets of speech data
As a result, devices suddenly became much better at identifying words from the sounds of our voices, stitching together meaning and speaking back to us.
Less raging at Siri – more thanking Alexa
With continuing advances in machine learning, word error rate continues to decline and our voice-first devices continue to improve
Also, the more users these devices get, the more data will be sent to Amazon and Google’s data engineers and the better the devices will become
Reasons for acceleration - Advances in speech technology + Internet of Things = Acceleration
Also, there’s gold in them there hills - the artificial intelligence market is expected to grow from $400 Million in 2014 to $5 Billion by 2020.
Hence GAFA involvement – Google, Apple, Facebook, Amazon
Why the move to voice?
Speech came first before writing, we’re made for speech – speech is an innate ability as opposed to a learnt one
Humans have always dreamed about speaking with computers in our natural language
Voice is a logical next step in ‘Anticipatory Design’
Amazon Echo and Google Home have given us a taste of what this could mean - ownership predicted to quadruple to 33 million in 2017, with 30% of device-based interactions by voice in 2018
Why? - Voice captures a much more textured view of who we are – a 3-D view of you based on your behaviours
This is where personalisation comes in:
Successful personal assistants need to be able to seamlessly stitch your life together - to form this textured view of who you are
Personalisation is at the heart of this for three reasons:
Compounding benefits of personalisation - each data point our assistant collects is a personal investment, incentive to choose a device and keep it close
Reconciling and prioritising different needs - having one personal assistant to stitch our fragmented digital identities together, reconcile differences and prioritise needs > many personal assistants to further fragment our digital identities, create further differences and compete over needs
Emotional attachment - our tendency to anthropomorphise means we develop an attachment to our assistant, making us more loyal to it – pet dog example
Physical limitations
1. Usage – at this stage, voice is more convenient in the home for three reasons:
Ecosystem – at the moment most connected devices (lights, speakers, refrigerators) are for the home
Convenience - the home is the place where you’re least likely to have your smartphone handy at all times, most likely to just use your voice
Privacy - you’re much more like to speak to a voice-assistant inside the comfort of your own home
2. Screen reliance
We still use voice-first devices as an extension of our screens - music, lights, timers
When you want to up-skill a voice-first device the process remains screen-based – creating a disconnect between using a voice-first devices and downloading new skills
Language limitations
Three primary reasons for the setbacks – all relate to the question of what makes a conversation?
Dialogue - conversing, not commanding
Range - weaving between domains
Emotion - understanding and expressing emotion
But…
Dialogue
Today, voice interactions are often limited to one-sentence or command utterances.
But, advances in 'Deep Reinforcement Learning' have the potential to improve and lengthen human-machine dialogue.
Range
Today, the best dialogue systems are limited to a specific domain - i.e. booking flights, ordering a taxi, completing shopping orders etc.
But, advances in 'Deep Transfer Learning' and 'General AI' will improve “open-domain” dialogue - making voice-first dialogue systems smarter about a wider range of topics and capable of contextual reasoning over the course of a long conversation.
Emotion
Today, we don’t incorporate emotion into our voice-first devices.
But, with advances in affective computing, voice can consistently recognise and generate prosody - the patterns of stress and intonation in a language.
In conclusion, voice as a UI is getting there – but it’s not quite there yet
We’ve already heard how voice is going to be one of the big areas of growth in tech
It’s seamless and convenient, but how transparent is it? How much do we know about what it is listening?
WTF IS AN ALGORITHM
There have been isolated reports of Alexa / Home responding when it has not been commanded "Do you work for the CIA?"
Google got into trouble for including ads for Beauty and the Beast alongside people’s ‘My Day’ update (weather, calendar, etc.)
Google denied it was an ad to The Verge. Twice.
This represents one of the areas of concern people have around current AI
Kate Crawford, Dark Days: AI and the rise of fascism
AI is good at centralising power
Facebook is the default muslim registry in the world
Even if you claim to be a muslim, machine learning will spot you’re not really // Based on FB likes, we can predict your religion based to an accuracy of 86% // Cambridge University
Never being held to account. Who is to stop Google or Amazon doing these things? This is a benign example, but no policy (more of that later) exists.
IT can create distance from the impact of your decisions
AI systems are not transparent, they give a false sense of accuracy and neutrality
http://www.theverge.com/2016/8/19/12552384/chicago-heat-list-tool-failed-rand-test
http://www.theverge.com/2016/8/19/12552384/chicago-heat-list-tool-failed-rand-test
the city’s futuristic "heat list" — an algorithm-generated list identifying people most likely to be involved in a shooting.
the list’s algorithm identifies people by looking not only at arrests, but also whether someone is socially connected with a known shooter or shooting victim. The program also has a kind of pre-crime feature in which police visit people on the list before any crime has been committed.
RAND discovered that CPD wasn’t using the list as a way to provide social services; instead, CPD was using it as a way to target people for arrest.
A lot of talk at SXSW about how Government and policy is not keeping pace with technology - Particularly with new US administration
Terah Lyons, Whitehouse Office of Science and Technology:
Creating a friendly policy environment // Govt invests 1/8th of what industry invests in AI
Not appointed a director for the Office of Science and Technology Policy, given that they haven’t appointed a chief technology officer for the United States
https://techcrunch.com/2017/03/26/trump-to-create-white-house-office-for-american-innovation-to-be-headed-by-kushner/
More public private partnerships were talked about
This office seems to be more about outsourcing departments that are difficult to fix, e.g. Veteran’s Affairs
Panels on health where Government representatives are open to pitches from technology companies to help them with problems they can’t solve: e.g. inaccurate self reporting on diet and alcohol // using data to alleviate the ‘burdens of disease’ // helping kids meet physical activity guidelines
Issue with Governments is they are often short term and prone to knee jerk reactions
Big fear around AI is that it could get shut down following a public outcry – why the narrative we’ve talked about is quite dangerous
“At the moment, I think it's certainly as big a risk that we have a GMO moment, and there's a powerful reaction against the technology which prevents us from reaping the benefits, which are enormous. I think that's as big a risk as the risks from the technologies themselves.” Stephen Cave, Director of The Centre for the Future of Intelligence at Cambridge University
This is also true of education and research –
Investment is going into short term projects that generate immediate results
Tech and AI in particular has a white male problem, and a skills shortage
Governments should be looking at investing more in STEAM, encouraging more women to get involved to redress the balance
The debates around encryption and responsibilities ladder up to a topic that we’ll explore in even more detail in the full session.
Firstly, who’s responsibility is it to ensure that the data owned by GAFA etc. that is powering its AI and machine learning technology is secure
Who’s jurisdiction does it fall under? Where are the standards for companies to adhere to? Where are the checks and balances, beyond sustained media campaigns that demand action
DARPA has been doing some work around a letter grade (A, B, C,) ranking the security and the use of personal data
Platforms didn’t know that surveillance companies were using APIs Geofeedia, Snaptrends, Babel Street. Stopped it, but it’s a grey area
Secondly, at the moment we sign-up once for most online services, and agree to something without reading it; as companies use our data and map our activity for even more complex purposes. Again, where are rules or policies that manage this?
Dieter Zetsche, Daimler AG / Mercedes Benz - “People just need accept that we need their data”
Cory Docttorrow - “We’re footing the bill for our own surveillance” // who is enforcing transparency on companies?
Going the other way, i.e. no encryption but full transparency won’t work either. But in an age of increasing threats of cyber security
White House VR
Car in a vending machine for Carvana
Painting with cold air using thermochromatic paint
Facebox
PoweredByPaper
3D printing
DubSmash booths