SXSW INTERACTIVE 2017
Over the years, SXSW was an ideal event to gauge and project consumer-centric tech
trends. From Twitter empowering consumers in 2007, Foursquare focusing on location
in 2009, social proximity with Highlight in 2012, and live streaming via Meerkat in 2015.
2017 focused on the rise of intelligent systems from a content perspective and
immersive experiences bridged physical to digital.
Marketing is quickly shifting from disruptive tech to acceleration through intelligent
systems. It’s less about the latest app fad, and more about how quickly the combination
of data, intelligent systems and smart environments are going to impact consumer
behavior in the future.
The trends we covered at SXSWi 2017 outline the continual evolution of marketing in
2017, and the acceleration of intelligent systems. Join myself and Ian Beacraft, VP,
Digital Strategy, as we outline the trends we saw in Austin, Texas.
We view trends through the consumer-centric filters of connection, cognition and
immersion.
• Connection – Trends that reimagine how we connect, enable and empower
consumers
• Cognition – Trends where machine-based intelligence will disrupt and redefine data
assets and how we work
• Immersion – Trends that align technology and presence to evoke emotion, entertain
and power commerce
The technology featured at SXSWi 2017 aligns with our view of the coming intelligence
revolution built on new data types that will simplify complex tasks, predict need states
and usher in new forms of computing that will radically alter how we connect with both
consumers and intelligent proxies.
Tom Edwards
Chief Digital Officer, Agency
2
Cognition
Connection Immersion
Zone of
Convergence
EVOLUTION OF
STORYTELLING
AUGMENTED INTELLIGENCE
EMOTIVE ROBOTICS
INTERACTIVE VIDEO
SOCIAL PROXIMITY
SIMPLIFIED CONVERSATIONAL EXPERIENCES
TO BOT OR
NOT
MIXED REALITY
PROXY WEB
8K VR
4D VR
MULTIPLAYER VR
TRANSCULTURAL AI
PROJECTION MAPPING
To Bot or Not
6
Connection!
A number of content tracks, meet-ups and discussions at SXSW were
tied to all forms of bots. From designing the ideal bot experience to
mapping the role that bots can play along the consumer journey, to the
fact that mobile messaging platforms have become the new face of
social media.
My caveat to all of this is that it still comes down to the role that a bot
can play on behalf of a brand. It is less about brand personification and
more about mapping simplified utility leading to predictive experiences.
The other point to consider is the evolving landscape of messaging
platforms that will combine conversational aspects supported by bots
as well as richer web view experiences within the platforms.
Implications: With more brands considering multiple facets of
conversational experiences as well as commerce, it is important to
align strategy with objectives. Web views will be ideal for top of the
funnel discovery while bot-based interactions can provide pre- and
post-purchase support as well as progressive profiling from a data
perspective.
Bots were a hot topic of conversation at SXSW 2017. A majority of the focus was on
experience design. SXSW also launched their own event-based bot to assist
discovery of content and programming.!
Voice = Accessibility
8
Connection!
Conversational- and voice-based systems have been a part of every
major show in 2017; with Alexa winning the day at CES. At SXSW, the
focus was less on the integration of voice-based user experiences and
more about the impact that voice will have on the future of the web.
Natural language processing provides more accessibility for people of
all ages, cultures and socio-economic backgrounds. This is important
to consider as we continue to shift from text-heavy experiences to
natural language processing.
Implications: Gartner recently highlighted that by 2020, 30% of web
browsing sessions will be done without a screen. Voice first browsing
and the idea of Zero UI experiences is quickly becoming a reality. It is
important for brand marketers to consider the role that voice-based
experiences can play as an extension of their traditional campaigns.
With the rapid shift from everything is connected to everything is
intelligent, as well as the proliferation of voice services through other
entities such as cars, home appliances and more, do not
underestimate the potential of voice-based services.
By 2020, 30% of web browsing sessions will be done without a screen.*!
*Gartner Symposium/ITxpo 2016!
Social Proximity
10
Connection!
At SXSW 2012, Highlight was the runaway app of the event. But
Highlight was a bit ahead of its time. The accuracy tied to proximity
was definitely lacking.
For SXSW 2016, while visiting the Capital Factory, I got to experience
NewAer’s peer-to-peer social proximity platform. While this experience
was beacon-based, their experiences can also be leveraged without
hardware for a truly peer-to-peer experience.
By mapping elements of the social graph with personalized physical
experiences, they created an environment that not only was
personalized to my liking, but also identified others with similar tastes.
Implications: For those organizations that are focused on enhancing
physical or shopper experiences, personalized and/or peer-to-peer
experiences add another level of connection to bridge digital to
physical experiences.
During a visit to the Capital Factory, we saw NewAer’s social proximity platform in action.
From automatically adding me to queues, to personalized drink experiences, their peer-to-
peer personalization platform’s accuracy was impressive.!
Non-Linear Video
12
Connection!
Consumer consumption of video content is at an all time high and
interactive and non-linear video platforms were on display at SXSW
2017.
Interactive video is an ideal approach to enhance a consumer's
experience, as well as give the impression of personalization through
consumer choice with a tactic known as “branching,” which is mapping
multiple linear videos together to create unique pathways forward.
As interactive video experiences evolve, we also saw the seamless
integration of contextual hot spots that can map directly to conversion
experiences.
Implications: Consider non-linear and interactive video experiences
as an entry point into branded experiences, especially for brand home
pages. Having the ability to empower consumers while also driving
progressive profiling is key to higher engagement and collecting more
behavioral data.
Wirewax was on the SXSW trade show floor and their interactive video platform has evolved
to include both “branching,” which is "choose your own adventure" style navigation, as well
as contextual hot spots in video.!
Data Design + ML
15
Connection!
Machine Learning was a hot topic at SXSW 2017. Brands such as
Disney discussed how they are using machine learning to inform future
innovation in their parks.
80% of data is unstructured, and machine learning provides an
opportunity to quickly identify perceptions, occasions and themes that
can enhance a brand narrative.
Mapping the data landscape, storytelling through data, and aligning
intelligence from data can inform new ways to communicate and
message with consumers.
Implications: For the past 15 months, my data design team has
focused on aligning emerging artificial intelligence systems and
algorithms to inform our quantitative and qualitative research that then
aligns with our world-class proprietary data assets, including the
largest transactional data set in the world.
Data design is incredibly important to create the framework for
personalization at scale.
While on the trade show floor we came across Hello Soda, which is a machine learning
platform that maps psycholinguistics of users who have authenticated via a social login. This
can enhance research panels as well as proivide a better understanding of 1st party data.!
Emotive Robotics
17
Connection!
In regard to content programming, artificial intelligence was one of the
primary topics of discussion, from ethical design to the importance of
overlaying emotional intelligence into AI-based systems.
Designing robot communications to feel more intuitively human while
being relevant for each type of engagement is key and the concept of
emotive robotics continues to advance at SXSW.
Emotive robotics is based on systems and devices that can recognize,
interpret, process and simulate elements of human behavior.
Implications: As intelligent systems and assistants move beyond their
current narrow AI focus, we will begin to see emotional intelligence
being applied to these systems.
This will also lead to the rise of bots as proxies for consumers, and
marketers will have to contend with both consumer-centric marketing
journeys, as well as system-based journeys.
There were many forms of emotionally intelligent systems on display at SXSW. One of the
best experiences was with Kuri. Emotive eyes and a friendly disposition provide a glimpse
into the not so distant future. !
Intelligence Augmentation
19
Connection!
One of the key themes discussed during SXSW Interactive was the
intersection of human and artificial intelligence.
There is a natural friction between artificial intelligence vs. augmenting
human intelligence. Some fear that AI will displace those in traditionally
creative positions.
Others believe that intelligence augmentation is key to unlocking the
full potential of human intuition and creativity with systems then
carrying out behaviors.
Implications: Intelligence augmentation is already a key component
for how we approach the creation of data-driven creative and
strategies.
By leveraging systems to support analysis and planning while leaving
the human at the center of the interaction is key. Striking the right
balance can unlock new creative territories and opportunities to
personalize at scale. The key is to understand the goals for systems
and outcomes and map both an AI & IA strategy, accordingly.
Discussions about Augmented Intelligence impact the advertising industry as a whole, as
systems are being developed to both enhance and potentially replace existing systems.
Understanding how to maximize human creativity through intelligence augmentation is key
vs. replacing with AI.!
Proxy Web
21
Connection!
In the very near future, we as consumers will have intelligent systems
serving the role of a proxy for interaction with other intelligent systems.
These systems will communicate with each other in order to streamline
information exchange, replenishment of frequency-based purchases
and align situational awareness with our desired experiences to adjust
and predict our needs. We talk about this through the lens of the
intelligence revolution.
Implications: Understanding that most marketing of today is based on
reactive data sets, we are quickly accelerating towards intelligent
systems that are contextually aware. Data is the fuel for AI, and
defining a strategy that incorporates reactive data of today with
affective data of tomorrow is key.
Once contextual awareness is achieved, we will see the rise of the
proxy web and, then, ultimately general intelligence. From a marketing
perspective, it is important to understand that strategies will need to be
defined that are consumer-centric, as well as account for general
assistant and proxy-based strategies.
We saw multiple examples of a Proxy Web future. While on the show floor I saw a group
called NexTremer. They have developed a multimodal dialogue system that is adaptable
across various form factors such as robots and mobile devices. !
Four Dimensional VR
Virtual reality demos have been a dime a dozen at tech trade shows
within the past six months, however, the vast majority of them merely
translated 360-degree video into a headset experience, or, at the very
best, built CGI worlds the user could interact with. At SXSW we found
a number of creators taking things to the next level with experiences
that involve multiple senses and custom built apparatus that engage
the user’s body in the experience in novel ways.
Implications: These experiences are a preview of what’s to come for
both in-home and commercial VR experiences. As VR gains adoption
and the many devices designed to stimulate other senses such as
touch hit the consumer market, people will begin to expect immersive
experiences that fool more than the eyes. In addition, they’ll expect
commercial grade experiences to be far more immersive and
expansive in order to consider them worthwhile. The bar is raising
quickly for those who want to wow their audience.
24
Previous: The Mummy VR experience simulates the zero-gravity in hydraulic chairs fitted
with vibration motors to enhance sound and mimic turbulence. !
!
Above: Birdly simulates a bird in flight with hydraulic platform, contextual audio, and a
dynamic fan to blow wind against the player.!
Multi-player VR
Virtual Reality may still be an Alpha stage technology, but the
complexity and quality of experiences available is rising quickly.
What’s impressive about the recent growth in multi-player experiences
is the programming wizardry necessary to track two bodies in motion
while rendering a virtual space.
Many of the demonstrations require large physical spaces free from
obstruction, but online multi-player eliminates this necessity and
makes these experiences far more approachable to owners of a VR
headset.
Implications: As headsets come down in price and VR experiences
on mobile devices improve, we’ll see a massive uptick in the number of
multi-player experiences and consumers playing them. We’re also
likely to see multi-player VR become part of the e-Sports ecosystem,
challenging the notion of e-Sports competitors as armchair athletes
and bringing spectators into the mix.
26
Previous: Attendees of the Capital Factory VR lab play head-to-head in a game that
combines elements of tennis and Brick Breaker.!
!
Above: Light Strike from Groove Jones is reminiscent of the disc battle in Tron.!
8K VR
8K TV’s may cost you mid-five figures and lack any content to take
advantage of their resolution, but that hasn’t stopped companies from
developing prototype devices that are 8K capable. In fact, these
platforms may make 8K content a viable standard even before 8K
broadcast becomes an option.
The objective of relatively high resolutions for current headsets is to
help avoid motion sickness and provide a generally smooth
experience– not exactly an extremely high bar, but a decent place to
start. As 8K content starts to appear in VR, these experiences will
become extraordinarily immersive, and make it nearly impossible to
detect individual pixels no matter how hard you look.
Implications: For VR to become a mass-medium, the resolution will
have to improve significantly (among a number of other hurdles).
However, once it does, the expectation that content is optimized for VR
will be as commonplace as the idea of “mobile first” is today.
Also, combined with 5G capabilities, the live streaming of events
around the world in VR may be even more attractive to some than
physically being there.
28
Previous: Tokyo Victory immersive “VR” experience in 8K celebrates the coming Tokyo
Olympic games. Despite a massive screen, pixels were impossible to spot.!
!
Above: The PiMa VR headset boasts 8K capability, which is significantly higher than either
Oculus or Vive.!
Mixed Reality
Mixed Reality brings together the physical and virtual worlds to create
new environments and experiences where physical and digital objects
interact in real-time.
At SXSW we saw a few experiences that leveraged the capabilities of
the Microsoft HoloLens, Vive, and Oculus to allow the wearers to
manipulate their physical surroundings through digital interactions.
Implications: As popular as VR had become in the media, many
believe Augmented Reality and Mixed Reality will be have far bigger
implications for consumers in the coming years. While marketers
currently have to hard-code experiences in highly controlled
environments that allow participants to control the physical world
through a digital interface, the adoption of Mixed Reality through
mobile and other devices will establish a platform upon which these
experiences can be rapidly prototyped and published for mass
consumption.
31
Previous: A festival goer experiences “telekinetic powers” in Invisible force using a
HoloLens and Mio Armband to blow up a foam Jenga tower.!
!
Above: The Jenga tower immediately before its demise.!
AR: Transcultural
Communication
One particular display of augmented reality from Grani VR studios was
interesting enough to warrant its own feature in our recap. The AR/VR
studio specializes in creating platforms for communication and
collaboration within virtual environments and conferencing across
language barriers.
Project Sonata’s AR demo used natural language processing to
translate speech to text and then translate into the wearer’s native
language in real-time, allowing people to communicate complex
concepts without deep understanding of another’s language.
Implications: These types of use cases are beginning to make their
way to mobile phones and as opportunities for AR become more
commonplace, people will be able to navigate foreign countries and
communicate with others in a non-native tongue without previous
knowledge of the language. This will have a profound impact on
language learning, a potential increase in foreign travel, and further
drive the effects of globalization.
33
Previous: HoloLens wearers communicate by speaking Japanese and English to each other
with perfect understanding of what the other is saying.!
!
Above: Grani Labs demos a VR experience meant to showcase new modes of collaboration!
Evolution Of
Projection Mapping
Projection mapping has gained notoriety as a novel way to overlay a
physical space with light-based imagery to transform the environment.
Up to now, this type of technology required extremely expensive
projectors and control units to coordinate their imagery.
With the introduction of interactive and “touch” based projectors like
Sony’s Xperia Touch, projection mapping and highly engaging
“screenless” experiences are likely to find their way to more venues
than ever. We found several installations across the SXSW campus,
each with a different execution.
Implications: These devices may still be outside the budget of the
everyday consumer, but they’re cheap enough for even small
businesses to install and build experiences for. We anticipate these
devices will be popular with retail and experiential spaces.
35
Previous: The $1,600 Sony Xperia Touch turns a table into an interactive music experience.!
!
Above: A Spider Man wall climbing experience that leverages projection mapping to make
the environment interactive.!
CLOSING THOUGHTS
While the past few years have spurred chatter about whether SXSW is still relevant, the
festival still provides tremendous value to those willing to traverse the show floor and
put effort into researching the best keynotes and panels to attend. Even without a
breakout hit like Twitter or Meerkat owning the news cycle, there’s plenty to see if
you’re looking for creative applications of the technologies that were in beta or concept
stage at previous years’ CES and Mobile World Congress trade shows. While the other
two shows are focused on the future gadgets, platforms, and infrastructure of
experience, SXSW is great for getting taste of what these things look like when tamed
by creatives and released into the wild.
What we saw during our attendance was a clear nod to the importance of AI and
intelligent systems from the content programming, with dozens of panels addressing all
manner of topics, from the ethics of AI to the human relationship with machines. The
experiences we encountered had a decidedly Immersive focus, with hundreds of Virtual
Reality and Augmented Reality spread throughout Austin. The best of these found new
ways of involving the senses instead of transposing the established conventions of
other media like video and the web. If what we experienced here is any indication of
what we’ll see outside these walls in the near future, and there’s reason to believe it is,
we’re in for a wild ride.
Ian Beacraft
Vice President, Digital Strategy
36