- Stay aware of upcoming AR shopping trends
- Understand the state of Augmented Commerce in 2019
- Learn how Face AR addresses challenges and drives sales
- Explore Face AR software possibilities in Try-On apps and AR Store Mirrors
- Check the best Face AR examples in eCommerce and retail
Need a virtual try-on app or software? Visit https://www.banuba.com/
Augmented Reality (AR) product experiences are heading for the mainstream. Leading retailers are successfully using Face AR technology to give consumers innovative ways to try on and experience products using cameras.
How do brands capitalize on Face AR to drive sales, reduce product returns and engage with younger consumers?
At Banuba we explore today's possibilities for Face AR, its future and the implications for e-commerce and retail. We also include groundbreaking case studies in Face AR commerce, with examples of our approach and high standards.
2. Augmented Reality (AR) product experiences are heading for the mainstream. Leading
retailers are successfully using Face AR technology to give consumers innovative ways to
try on and experience products using cameras.
Consumers’ expectations are growing fast too. Gartner predicts that 100 million
consumers will be using augmented reality for shopping, in-store and online, by 2020 [1].
Yet despite all the excitement, few people are aware of what ‘Face AR’ and ‘Augmented
Reality Commerce’ mean.
What is Face AR technology? How do brands capitalize on Face AR to drive sales, reduce
product returns and engage with younger consumers?
This paper explores today's possibilities for Face AR , its future and the implications for
e-commerce and retail. We also include groundbreaking case studies in Face AR
commerce, with examples of our approach and high standards.
About this paper
4. Future AR shopping trends
Mary Meeker's report on Digital trends 2019 outlines images and visual content as the
prime means of communication among people today [2].
Over 50 per cent of Twitter impressions now include posts with images, video or other
visual content where it used to be text-only. 55 per cent of consumers have bought
something online after discovering it on social media [3].
Today's trends shape tomorrow's standards. In 2019 we are seeing some interesting trends
that may help us get a glimpse of the future consumer and the role of AR.
Brands will continue to capitalise on social media and visual content generated by users.
Technology and camera advances will encourage a surge in virtual product experiences.
1. Consumer demand for visual content increases
5. With rapid adoption of AR, cameras will become the key touchpoint for every part of
shopping. Self-expression, self-identity and community building will remain the most
important drivers of AR camera apps for shopping, fashion and self-care.
Snapchat and Facebook AR filters and ads make discovering products fun and
non-intrusive, compared with other advertising formats. Virtual try-on apps, reached on
mobile or website, will make product choice a smooth process. Smart interactive mirrors
will add engagement and help to drive audience in-store.
2. The camera as the customer touchpoint
6. 91 per cent of consumers prefer brands which provide
personalized offers and recommendations [4].
In the future of retail, shopping can be much more
personalized, smooth and tailored to your individual needs
and characteristics. The AI-empowered tools extracting data
f rom a customer's face will improve content targeting and
product recommendations helping consumers to
communicate with the brand personally.
In cosmetics and skincare, AI beauty advisors will track
customers’ conditions or progress making them stick with
the brand.
3. Personalisation is King
7. As we look ahead, building social currency may be the most crucial investment retailers
can make. The extent to which people share a brand or information about it as part of
their everyday lives will define the success of organic marketing.
Augmented reality allows the conversation between brands and consumers to unfold
naturally. It won’t be about buzz but meaningful experiences around the brand – ones
that connect emotionally and turn the audience into advocates.
4. Social currency in the 21st century
8. The Trend Hunter reports that adolescent activism is growing across all aspects of our life
[5]. Gen Z takes an active part in sharing their views, and brands can’t stand aside.
With so much information targeted at consumers every day, brands will need to go for AR
marketing campaigns with a meaningful goal, a story that makes a difference, to break
through the noise and deliver their messages effectively.
5. Brands going for Gen Z with AR storytelling
9. Over the past 10 years, technology has dramatically simplified how we discover,
experience and buy things.
When shopping online, we once had to rely on 2D images and text descriptions.
Then we moved on to video reviews and influencer marketing, where people experienced
products in real time.
Now we're entering the era of Augmented Commerce (aCommerce) where we can
experience products ourselves virtually without leaving home – via the camera.
Website
eCommerce
2010
Mobile user interface (UI)
mCommerce
2016
Camera
aCommerce
2018
Tough competition for top
rankings in Google Search
Costly ads and marketing
Non contextual product
representation with 2D, text,
video content
In-app pushes based on user
location
Smaller screens make it hard
to visualize products
Complex navigation among
extensive product choice
limits discovery
Personalized selfie-powered
product discovery
High level of engagement
and novelty with 3D, AR
content
Real-time product
visualization & AI based
recommendations
Virality through
user-generated content
sharing on social media
Augmented commerce today
10. Augmented Commerce or aCommerce is buying
and selling goods using augmented reality to
visualise them virtually in the real-world
environment before purchase.
In the context of aCommerce, Face AR technology
allows products to be visualised with informative
graphical overlays and demonstrations. Consumers
can virtually try on in 3D products such as makeup,
glasses, accessories, jewellery, headwear and hair
colouring.
Face AR technology shifts attention from a simple
representation of the product to a personal
experience of it.
Too many clicks result in fewer conversions, and
brands have been focusing on providing the least
amount of steps possible during the online
shopping experience.
Face AR simplifies shopping to two steps – trying
on products virtually, then buying those that suit
you best. Unlike with eCommerce and
mCommerce, users can browse hundreds of items
without getting bored.
“AR commerce can work for anything that requires the
user to build their understanding and opinion of it prior
to holding its physical manifestation in their hands.”
— Gracie Page, innovation lead at VMLY&R.
11. The shopping path has changed.
of consumers would be more likely to shop
with a retailer using AR technology.48%
of retailers plan to use augmented reality within three years. Given the
fast-changing market trends and many brands adopting AR, these
numbers are likely to increase.
32%
users would rather interact with AR content
than watch a 30-second video [6].78%
Why now?
AR content is 2.6 times more engaging than static 2D
imagery and 2.1 times more engaging than video [7].
12. The global success of Snapchat and now TikTok have shown a tremendous interest in AR
applications among people. What’s more important, it has also shown the power of the
AR camera apps to engage people and immerse them into other realities.
The shopping sector is one of the first to transform AR f rom a gimmick into something
genuinely useful.
AR makes the buyer’s experience much more captivating, enabling unrivaled
"one-to-one" interaction with sellers through the camera. It becomes far easier to discover
products, bringing novelty and convenience to shopping, directly tackling pain points for
both consumers and brands.
Lack of personalization, product returns, and the problems of attracting younger
‘Generation Z’ consumers are just some of the challenges retailers can meet using Face
AR technology.
Given too many products to choose f rom, consumers experience difficulties navigating a
retailer’s site or store and finding items that would match them. They are likely to give up
or leave without purchase at the peak of their f rustration.
Consumers want shopping
to be personalized every
time they encounter a
brand – on the web,
mobile and in-store.
Personalisation
80 per cent of consumers
are more likely to buy f rom
a brand offering a
personalized experience.
Only 2 per cent of
marketers report that their
company has personalized
all their channels [8].
The rise of Face AR Commerce
13. Employing computer vision and deep learning
algorithms, the Face AR technology enables AR
experiences that are not only built around the customer
but personalized for his or her individual needs.
By analyzing facial characteristics, Face AR algorithms
can find products that are the best fit for a customer
and showcase them in real-time.
The intuitive interfaces allow quicker navigation, making
it easier to find the right product.
Consumers can browse more products via swipes on
mobile apps and in-store mirrors or instantly launch the
web try-on feature.
"It can be overwhelming coming into our stores or shopping online,
but virtual try-on makes it easy to help you find your favorite shade
and save you time."
— Mary Beth Laughton, EVP of Omni Retail at Sephora
“
15. Market research puts ‘appeal to millennials’ and ‘strong digital platforms’ among the
main reasons for the success of global beauty brands [10].
Gen Z represents up to $143 billion in buying power and in a few years will be the largest
consumer group in the US [11].
Their trendsetting abilities cannot be ignored – how teenagers interact with products and
services affects the spending of their families and friends to the tune of $600 billion a
year.
Traditional marketing strategies fall flat with Gen Z. They have a shorter attention span
than other generations. Raised in the smartphones and social networking era, they shop,
communicate and live online.
The novelty of AR technology, its accessibility on mobile and the selfie-focused camera
experience lets brands hold the attention of “digitally native” consumers longer and
convert them at a higher rate.
By letting consumers take and share selfies with the product, brands can capitalize on
the long-term virality of user-generated content and native advertising without extra
marketing costs.
Attracting Gen Z
16. “Gen Z are perhaps the most
brand-critical, bullshit-repellent,
questioning group around and will call out
any behavior they dislike on social media.
(Little wonder brands are quaking in their
boots.)”
— Lucie Greene, Worldwide Director of the Innovation Group
17. Consumers may not know what they want
and how they want it, so
brands must think ahead.
The biggest challenge?
"If what you're building doesn't make their shopping experience
faster or easier or more fun, it's probably not worth the investment."
— Brian Solis, an analyst at Altimeter
“
18. Banuba’s Face AR augmented reality platform is a set of software
development kits (SDK) and technologies that let brands build and deploy
AR try-on experiences and applications for web, mobile and in-store.
With Face AR technology, consumers can visualise products and find the
best fit using the 'try before you buy' experience with cameras.
What is Banuba’s Face AR
Platform?
Provides real-time virtual product experience that feels real.
Requires no prior photo upload or video recording needed – one-tap try-on.
Allows for realistic 3D product representation of colours and shapes.
Reaches younger audiences with immersive experiences
Advertises products with branded 3D masks and AR filters
Increases brand engagement
Harnesses virality as user content is shared on social networks
Drives traffic organically without expensive adverts
Can be experienced on mobile, web and in-store.
•
•
•
•
•
•
•
•
•
Face AR key features and benefits
19. Brands build mobile and web apps from scratch or integrate Face AR into their existing
apps using our suite of development tools. This lets them introduce real-time AR
shopping for customers.
How Face AR platform works
AR Experience Software suite
Virtual product try-on based on real-time
face detection and tracking
Beauty try-on and face modification
experiences based on face segmentation
and tracking
AI product recommendations based on
analysis of facial characteristics like skin,
hair colour, age and gender
Face Filters SDK
AR Beauty SDK
Face Analytics SDK (demo)
20. Try-on apps & experiences
enabled with Face AR
Customers can find the perfect frames
by trying them on using real-time AR.
Eyewear and sunglasses of any shape
and color
Real-life lighting and shadowing
effect
Size adjusted based on the device
distance
Measure pupil distance for eye
prescription
•
•
•
•
Glasses & sunglasses
Showcase beauty products in 3D with mobile
apps, web AR plugins or in-store beauty
mirrors.
Lipstick, eyelashes, eyeshadows, eyeliner
and foundation, hair and eye recolor
Automatic fitting with precise face
segmentation
Natural beautification for a better user
experience
All colors and shades adjusted for lighting
Real-time or photo post-processing
Makeup recommendation based on skin
and hair color detection
•
•
•
•
•
•
Makeup
21. Demonstrate different hair colors to
help customers find the perfect style.
Magic-mirror hair recoloring
All-round color palette from
natural to lurid
Hairstyle detection
Existing hair color detection to
predict results of recoloring.
•
•
•
•
Hair color
Visualize how small items will look
on a user's face.
Earings, accessories, face jewels
and piercing try-on
Precise item fitting
Physics support (items move
naturally when the user shakes or
inclines the head)
Size adjustment – users can move
close to the camera to see the
item in detail
•
•
•
•
Jewellery
22. Let consumers wear accessories virtually,
shop with AR and share on social media.
3D items applied automatically
Hats, headbands, helmets, scarfs
Any form, shape, color and design
Virtual items throw shadows and
cover the face
•
•
•
•
Headwear & accessories
Try on your smartphone
23. How Face AR Addresses
Challenges and Drives Sales
Sales increase
Brands actively embracing augmented reality technology have already seen
a measurable positive affect. Face AR increases the time consumers spend
with a brand – on mobile and online – and spark impromptu purchases.
Lack of personalization
The intuitive AR experience enabled by the camera brings unrivaled
personalization for customers. The product recommendation can be tailored
using AI algorithms and face analytics data like age, gender, skin and hair
color, to guide product choice.
Gen Z engagement
Gen Zs prefer visual media and easy-to-perceive content. Face AR allows
many varied AR scenarios – from realistic try-ons to captivating AR ad filters,
which keep younger audiences engaged.
Costly marketing
The Face AR platform inspires users to generate and share their own
content, allowing brands to reach new audiences organically.
Competition
The novel virtual product experiences enabled with the Face AR
platform help brands to stand out and stay competitive.
24. Omnichannel Face AR
experiences right where
consumers are
Smartphones have become the go-to channel for online shopping.
49 per cent of consumers use mobile phones for shopping online
and in-store. Mobile augmented reality is estimated to top 2.5 billion
installed base and $70 billion to $75 billion revenue by 2023 [12].
So brands have an amazing opportunity to build positive brand
sentiment by reaching consumers with unique, compelling and
targeted AR experiences on mobile.
Face AR platform allows for real-time try-on experience that works
like a magic mirror.
Users can try on products through the mobile camera, find their
perfect color match or get it recommended with AI algorithms
based on their facial characteristics.
Mobile
Most mobile AR experiences require smartphone applications. For
consumers, this involves downloading an app and allowing access to personal
data. This creates a significant barrier to distribution and raises concerns over
privacy.
Face AR for the web lets AR content be viewed and shared through mobile
and desktop web browsers, allowing for a frictionless shopping experience.
Instead of downloading a new app, users can simply click on a link to open
a page in their browser or press the “Try On” button on the product page,
activate their web camera and try on items virtually.
WebAR
25. In apparel, clothing, accessories and cosmetics, many brands have
successfully allowed customers to virtually try on products. AR fitting
rooms and mirrors can increase sales by taking an omnichannel
approach.
The Face AR technology not only saves the time customers spend
going to fitting rooms but lets them browse many more items when in
the store.
Using analytics, smart mirrors become non-intrusive consultants,
advising on the most suitable cosmetics, skincare products, hair color
and even fashion and clothing.
Virtual product testing keeps shoppers engaged, adding fun, novelty
and convenience, so retailers stand out from their competitors.
In-store
27. What makes the Face AR
try-on apps truly magic?
The first buzz around AR try-on apps emerged in 2016. The news
feeds bet big on augmented reality in shopping and the idea of
wearing products virtually.
However, most early try-on applications failed to meet the
expectations, because of immature technology and ineffective
implementation.
Manual facial feature adjustment, poor UI and performance,
especially on Android, and unrealistic product representation were
the main complaints.
The result? Consumer disappointment, waste of budget and no
ROI.
Magic mirror vs manual photo editing
29. Unlike the early try-on apps, Face AR technology allows a real-time
f rictionless AR experience that can be accessed on a wide range of
devices, not only top-level iPhones.
3D face tracking
Three core technologies underlie Face AR:
Face segmentation 3D rendering engine
Face AR tech explained
30. Most face-tracking solutions are based on landmark
tracking. Landmarks map key features such as eyes,
eyebrows, nose and mouth. The more landmarks are
detected, the more feature-rich AR experiences can be
overlayed onto the face.
However, landmark tracking requires significant
computing power which is why most early AR apps fail to
provide quality AR camera experiences on mobile devices
with limited CPU, memory, and power – the so-called
"constrained devices".
Face AR uses technology that recognises 37 characteristics
represented as 3D models of the face.
Instead of wasting capacity for landmark alignment and
tracking, we have trained the technology to “see” the face
in 3D.
We have selected the optimal number and set of face
morphs to extract all the data needed and robustly track
the face in the camera. This includes 30 models with
different facial expressions and anthropometry plus 7
parameters that describe the position of the face in a
photo (head tilts and face angles).
This innovative approach provides much faster
performance and better accuracy – the algorithms need to
process only 37 numbers.
The significant capacity saving and technology
optimisation let us achieve the best 3D mobile face
tracking on the market.
3D face tracking explained
31. We run algorithms several times within one frame to
process the noises that result in jitter-free user experience
without lag.
Less battery drain and device heating due to the saved
capacity.
Optimised specifically for mobile to function as well on
iPhone-Xs and $100-Androids, allows brands and developers
to maximise their audience.
Device support starts with iPhone 5S, Android 5.0 with
Camera 2 API and Open GL ES 3.0. We cover 97 per cent of
all iOS devices and 80 per cent of all Android ones.
Supports low lighting, 360-degree device rotation and up to
70 per cent occlusion that results in a seamless experience.
Users can wear glasses, have poor lighting, move and incline
their heads while experiencing Face AR apps.
We use balanced datasets that include faces of different
ages, genders and nationalities. Our Face AR experiences
work well for all types of skin colour and facial features.
How we achieve industry-best face tracking
•
•
•
•
•
•
•
As more and more apps build on real-time augmented reality,
consumers see selfie uploading and facial feature adjustment
in photos as outdated.
The facial segmentation on our face-tracking engine, lets Face
AR technology detect the right areas, such as like lips, hair or
eyebrows, to add the virtual product automatically with high
accuracy.
Face segmentation neural networks
32. Unless the product looks and feels realistic, the virtual product experience won’t satisfy
customers. Delivering high-quality AR content for virtual try-on apps is one of the most
important and difficult tasks for retailers.
We’ve developed the most powerful 3D rendering engine to allow life-like product
experiences to happen virtually. It can not only transfer realistic colours but adjust the
virtual item’s look for lighting and even behaviour.
Virtual items change size based on the user's position in the camera, throw shadows and
support physics. When a user bends their head the earrings move too. When your raise
your head, the sunglasses mirror the light just like real glasses.
Advanced 3D renderer capabilities
Graphical technologies AR experience
3D modelling & animation
Image-based lighting
Physically based rendering
Animation billboards
Morphing
Physics
High dynamic range imaging
Video textures
Multisample anti-aliasing
Sprite animation
LUT post-processing
3D items can be viewed f rom different
angles
Real-world light capture
Life-like object behavior
Real-time size adjustment
Items’ shadowing with head-down
tilts
Mirroring and bleak with head-up tilts
Transfer all possible color grades,
changing the hue, saturation and
brightness, adding contrast or making
certain colors ‘pop’.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
33. In March 2018 L’Oréal acquired AR company ModiFace, using its
software to power its 34 brands with beauty AR experience.
Successful use cases
“We really believe that the future of beauty will be
digitalised and tech beauty and in this area, we have been
really the first to start and we are ahead of the game and
for us, it is priority number one.”
Jean-Paul Agon, CEO and Chairman at L'Oréal.
“
Deliverables
Makeup Genius app to apply preset looks or try specific L'Oreal
products in different shades.
Style My Hair mobile app which lets users try on different
hairstyles.
AR Makeup Try-on app for NYX Cosmetics, a subsidiary of
L'Oréal.
Virtual skin assessment platform deployed on L’Oréal’s Vichy
web and mobile sites.
•
•
•
•
•
Results
Downloaded over 35 million times
Become the №1 lifestyle app in 47 countries.
260 million virtual product trials that led to conversions increase
Engagement time on that site doubled
Conversion rates tripled
↑
↑
↑
↑
↑
34. Sephora worked with ModiFace on its first Virtual Artist mobile app
that made a revolution in 2016.
Olay noticed that a third of women couldn't what they needed
when browsing the shelves and did not make a purchase. They
rolled out an AI-powered skin advisor platform to help women
understand their skin type and choose the right products.
“Every digital product or tool or experience that we
introduce is done to make shopping for our consumer more
fun and more efficient–to help engage her, educate her,
and entertain her."
Mary Beth Laughton, Sephora.
“
Results
200 million shades tried on
Over 8.5 million visits to the feature
Wear lipstick and eyeshadows right in the app
Virtual tutorials on how to contour, apply highlighter and create
eyeliner looks on your face.
↑
↑
↑
↑
Results
Reduced the bounce rate for online visitors by one third and
increased average basket size
2x in conversion rate
5-min session length
Improved involvement of younger demographic
Used over 1.2 million times
5,000 to 7,000 users every day
↑
↑
↑
↑
↑
↑
35. Followed by a successful test of lip products, the cosmetics brand
Macy’s plans to extend its mobile virtual try-on service to more
products.
Virtual Try-On Mirrors in its physical stores across the country.
“2/3 of online traffic and half of digital sales come f rom
mobile devices, making its app users the "most loyal
customer,"
Jill Ramsey, Chief Digital Officer at Macy's.
“
Results
Expects a 31 per cent sales increase due to AR-powered in-store
experience.
YouTube’s AR beauty try-on campaign launched in 2019.
According to prior testing, 30 per cent of viewers launched the AR
try-on feature in the YouTube iOS app. The average session length
has reached 80 seconds.
↑
↑
↑
The virtual jewellery try-on app brought a significant boost in sales
specifically among millennials.
Results
n 2018, the 40 per cent of CaratLane’s business was coming f rom
the AR-powered app.
↑
36. About Banuba
Banuba is the camera company on a mission to empower creative expression and visual
communication in people through AR apps built with our Face AR platform.
Our Face AR platform combines several SDKs that brands and developers use to build the
most immersive mobile, web and real-world AR experiences.
Face Filters, Beautification, Avatar, and Analytics SDK are all based on our innovative face
tracking technology that functions robustly both on iPhone-Xs and $100-Androids, with
low lighting, wide angles and up to 70 per cent occlusion.
The result? The magic-mirror experience that fascinates everyone.
Brands and developers can benefit from our Face AR platform, developing camera-first
apps that innovatively engage users and clients, inspire easy content creation and help
increase audiences organically.
We use artificial intelligence and deep learning to teach today’s camera see and sense
the world the way humans do. And for the future? We look towards the 21st-century
camera that will sees things beyond human vision.
100+ amazing people. Developers, data scientists, creative minds and visionary leads –
we're techno-optimists who believe in a better, AR-fueled tomorrow, one that we make
happen today.
Team
37. How to get in contact with us
We are always looking for new challenges, points of view and people to collaborate with. If
you’re a brand, developer or think you’d be a right fit for our team then do not hesitate to
contact us on the following channels:
Email
info@banuba.com
Facebook
facebook.com/banubadevelopment/
LinkedIn
linkedin.com/company/banuba-development/
Twitter
twitter.com/BanubaFaceAR
YouTube
youtube.com/channel/UCnXZnhDdFv-1VC47PCY9Ptw