Paul Kruszewski (wrnch): Using AI to Build Enchanted Worlds: Frictionless Digitization of Human Motion & Behaviours from Standard Video for AR/VR Applications

•

1 gefällt mir•218 views

A talk from the Develop Track at AWE USA 2018 - the World's #1 XR Conference & Expo in Santa Clara, California May 30- June 1, 2018. Paul Kruszewski (wrnch): Using AI to Build Enchanted Worlds: Frictionless Digitization of Human Motion & Behaviours from Standard Video for AR/VR Applications We will describe in detail how to use AI to digitize human motion and behavior from standard video cameras without traditional motion capture or depth sensing hardware. We will present a live "MagicMirror" in which an audience member can be beamed from a mobile phone camera into an Augmented Reality application. http://AugmentedWorldExpo.com

Technologie

Using AI
to build
Enchanted Worlds
Markerless motion capture
from standard video for AR/
VR/MR/XR Applications

Extended Reality (AR/VR/MR)
Opportunity
•  The next compute platform
•  The perfect interactive fusion of
the real and synthetic world
Problem
•  XR systems are digital
•  Humans are physical
•  How to digitize humans?

State of the Art: HUGE FRICTION
Current Solutions
•  Speciality Cameras
•  Wearables
Shortcomings
•  Expensive to own / operate
•  Time consuming to set up / use
•  Requires speciality sensors / suits
•  Quickly obsolete

Solution: Use Deep Learning turn ordinary
cameras into Markerless motion capture
systems
Completely Frictionless
•  No speciality suits / sensor
•  No set up time
Unlimited Configurations
•  Camera agnosticism allows many different
form factors to maximize MR applications
Future Proof
•  Deep learning means the system is always
getting better
•  AI processors means the system is always
getting faster

Alternatives to AI Markerless Motion Capture
Scalability
Frictionlessness
Leap Motion
RealSense
Kinect

Unique Features: wrnchAI
Accurate
•  Proprietary DL Networks
•  Synthetic 3D Data Pipeline
Robust
•  Biomechanical 3D human model
•  Body, Hands and Face
Fast
•  Optimized for real-time across NVIDIA
and mobile AI processors
•  Unity integration for rapid development

Proprietary Synthetic Data
Training Pipeline
Public and
Proprietary
Real Data
Proprietary
Synthetic
Data
Proprietary
DL Models
wrBrain
wrnchAI Training Framework

The “ABC”s of Teaching a Camera to
Digitize Human Motion
?

Key Points Detection
“J” “A” “M” “E” “S” “M” “A” “G” “G” “I” “E”

Stitching Together Body Parts into
Individual Skeletons
“James”
“Maggie”

Inferencing Pipeline:
Lightning Fast and Feature Rich
Motion Capture
Behavior Capture
Segmentation
2D
Skeletons
3D
Skeletons
IDs Locations Gestures Activities
2D
Video
TensorRT
NPE

Location Based MR: Samsung Flagship Store

Weitere ähnliche Inhalte

Was ist angesagt?

Augmented Reality

Leap motion

Augumented reallity

Leap motion

Marker Based Augmented Reality

Arshiya Sayyed

Augmented reality in spine surgery

ealkhatib

Future force presentation_to_ministry_of_defence

Auggy Augmate

Leap motion

Jay Poojara

i-Mouse

Pradnya Phulore

Spar 2011 Aldo Facchin

AldoFacchin

Augmented Reality

MeghanaVeronica

Vision System and its application,Problems

Nikhil Chavda

For the full video of this presentation, please visit: http://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/mar-2016-member-meeting-daqri For more information about embedded vision, please visit: http://www.embedded-vision.com Wenyi Zhao, Ph.D., Director of the Vision and Sensor Group at DAQRI, delivers the presentation, "Augmented Reality for Industrial Productivity," at the March 2016 Embedded Vision Alliance Member Meeting. Zhao discusses how his firm’s computer-vision-enabled augmented reality helmet is being used to dramatically improve productivity in industrial applications.

"Augmented Reality for Industrial Productivity," a Presentation from DAQRI

Edge AI and Vision Alliance

Augmented reality

Sakshi Chawla

Augmented Reality Presentation

SJSU

Augmented reality(my ppt)

Srilakshmi Alla

Ftir (nitish ranjan)

Nitish Ranjan

6th sense technology

arvind carpenter

Project Seminar on Leapmotion Technology

Abhijit Dey

Leap Motion Capabilities Overview

Modicum

Was ist angesagt? (20)

Augmented Reality

Leap motion

Augumented reallity

Leap motion

Marker Based Augmented Reality

Augmented reality in spine surgery

Future force presentation_to_ministry_of_defence

Leap motion

i-Mouse

Spar 2011 Aldo Facchin

Augmented Reality

Vision System and its application,Problems

"Augmented Reality for Industrial Productivity," a Presentation from DAQRI

Augmented reality

Augmented Reality Presentation

Augmented reality(my ppt)

Ftir (nitish ranjan)

6th sense technology

Project Seminar on Leapmotion Technology

Leap Motion Capabilities Overview

Ähnlich wie Paul Kruszewski (wrnch): Using AI to Build Enchanted Worlds: Frictionless Digitization of Human Motion & Behaviours from Standard Video for AR/VR Applications

Visug: Say Hello to my little friend: a session on Kinect

Visug

2016 AR Summer School - Lecture 5

Mark Billinghurst

Kinect sensor

bhoomit morkar

Lecture 4: VR Systems

Mark Billinghurst

AWE USA 2018 Startup Pitch: Isadore Katz with Suometry

AugmentedWorldExpo

Gam02 kinect1, kinect2

DotNetCampus

Emerging vision technologies

Qualcomm Research

This talk will tie the different usage opportunities to imaging requirements for a Mixed Reality system for both consumer and enterprise markets in the coming years and dive into system design aspects like placement, location, types of image sensors, bandwidth requirements for tethered and wireless HMD scenarios and the processing pipeline architecture with the critical technology building blocks from multiple camera sources distributed between a HMD and a host system. Example end usages like Obstacle Avoidance and Avatar Navigation with Mixed Reality headsets will be used to provide a use case decomposition view from capture to application.

MIPI DevCon Bangalore 2017: Imaging Systems Design for Mixed Reality Scenarios

MIPI Alliance

Mobile AR Lecture 10 - Research Directions

Mark Billinghurst

Virtual Mouse

Vivek Khutale

Mobile Augmented Reality

Marios Bikos

3D perception is crucial for understanding the real world. It offers many benefits and new capabilities over 2D across diverse applications, from XR and autonomous driving to IOT, camera, and mobile. 3D perception with machine learning is creating the new state of the art (SOTA) in areas, such as depth estimation, object detection, and neural scene representation. Making these SOTA neural networks feasible for real-world deployment on mobile devices constrained by power, thermal, and performance has been a challenge. Qualcomm AI Research has developed not only novel AI techniques for 3D perception but also full-stack AI optimizations to enable real-world deployments and energy-efficient solutions. This presentation explores the latest research that is enabling efficient 3D perception while maintaining neural network model accuracy. You’ll learn about: - The advantages of 3D perception over 2D and the need for 3D perception across applications - Advancements in 3D perception research by Qualcomm AI Research - Our future 3D perception research directions

Understanding the world in 3D with AI.pdf

Qualcomm Research

Computer vision - Applications and Trends

Kshitij Agrawal

Synthetic environment

Ullas Gupta

Sixth sense technology ppt

Mohammad Adil

virtual reality Information-160422181930.pdf

21107117

AIDC India - AI Vision Slides

Intel® Software

Augmented reality (Access virtual world)

chirag thakkar

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2020/12/machine-learning-for-the-real-world-what-is-acceptable-accuracy-and-how-can-you-achieve-it-a-presentation-from-arm/ For more information about edge AI and computer vision, please visit: https://www.edge-ai-vision.com Tim Hartley, Director of Product and Marketing at Arm, presents the “Machine Learning for the Real World: What is Acceptable Accuracy, and How Can You Achieve It?” tutorial at the September 2020 Embedded Vision Summit. The benefits of running machine learning at the edge are widely accepted, and today’s low-power edge devices are already showing great potential to run ML. But what constitutes acceptable accuracy when applied to real-world, real-time use cases? In this talk, Hartley explores what constitutes acceptable detection accuracy for specific use cases, and how this can be measured. Looking at which ML models are meeting the challenges and which fall short, he focuses on how techniques like transfer learning can help fill the gaps when weaknesses in detection accuracy are found.

“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/06/tensilica-processor-cores-enable-sensor-fusion-for-robust-perception-a-presentation-from-cadence/ Amol Borkar, Product Marketing Director at Cadence, presents the “Tensilica Processor Cores Enable Sensor Fusion for Robust Perception” tutorial at the May 2023 Embedded Vision Summit. Until recently, the majority of sensor-based AI processing used vision and speech inputs. Recently, we have begun to see radar, LiDAR, event-based image sensors and other types of sensors used in new AI applications. And, increasingly, system developers are incorporating multiple, heterogeneous sensors in their designs and utilizing sensor fusion techniques to enable more robust machine perception. In this presentation, Borkar explores some of the heterogeneous sensor combinations and sensor fusion approaches that are gaining adoption in applications such as driver assistance and mobile robots. He also shows how the Cadence Tensilica ConnX and Vision processor IP core families and their associated software tools and libraries support sensor fusion applications with high performance, efficiency and ease of development.

“Tensilica Processor Cores Enable Sensor Fusion for Robust Perception,” a Pre...

Edge AI and Vision Alliance

Ähnlich wie Paul Kruszewski (wrnch): Using AI to Build Enchanted Worlds: Frictionless Digitization of Human Motion & Behaviours from Standard Video for AR/VR Applications (20)

Visug: Say Hello to my little friend: a session on Kinect

2016 AR Summer School - Lecture 5

Kinect sensor

Lecture 4: VR Systems

AWE USA 2018 Startup Pitch: Isadore Katz with Suometry

Gam02 kinect1, kinect2

Emerging vision technologies

MIPI DevCon Bangalore 2017: Imaging Systems Design for Mixed Reality Scenarios

Mobile AR Lecture 10 - Research Directions

Virtual Mouse

Mobile Augmented Reality

Understanding the world in 3D with AI.pdf

Computer vision - Applications and Trends

Synthetic environment

Sixth sense technology ppt

virtual reality Information-160422181930.pdf

AIDC India - AI Vision Slides

Augmented reality (Access virtual world)

“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...

“Tensilica Processor Cores Enable Sensor Fusion for Robust Perception,” a Pre...

Mehr von AugmentedWorldExpo

A talk from the Training & Education Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Bill Meyer (Virtual Science Center): Turning Kids on to STEM by Letting Them Discover How VR and AR Tricks Our Brain How exactly does Virtual and Augmented Reality game human perception so things we know aren’t there seem to exist? Can simple and robust public learning experiences empower and delight people of all ages to experiment and discover the answers for themselves? The answer is yes. This talk will present examples and discuss how UX best practices can be applied to create effective inquiry-based learning experiences that unpack seemingly complex modern technologies. Through iterative design coupled with repeated visitor testing, we deconstructed the surprising perceptual science and technology behind VR and AR into 16 cool hands-on experiences that spark “aha” moments and minimize the need for wordy explanations. We keep concepts intuitive, playful, and fun to understand. Our traveling exhibition goal is to show visitors they're smarter than they suspect, inspire students to aim for STEM careers, and help teachers lead deeper NGSS and Common Core dives both in and out of the classroom. This 45 second video shows an example exhibit and part of our visitor testing process: https://youtu.be/Sgh0Sys57lY Without needing to learn the terms, visitors discover the secrets of how things like Motion Parallax, Stereo Vision, 3D Audio Spatialization, Outside-In, and Inside-Out Tracking work in concert to make virtual, augmented and mixed reality experiences possible. https://awexr.com

Bill Meyer (Virtual Science Center): Turning Kids on to STEM by Letting Them ...

AugmentedWorldExpo

A talk from the Gaming & Entertainment Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Nick Klingensmith (Microsoft): Increasing Immersion with the MR Lighting Tools To-date, most 3D graphics have been rendered on opaque displays. Now, not only are we putting holograms into the real world, but we're also doing it on additive displays! How do we render holograms that look good in this environment, and how do we make them look like they belong? The Mixed Reality Lighting Tools are a collection of easy Unity based solutions, from replicating your environment's lighting, to casting believable additive shadows. Come find out how easy it is to make your Mixed Reality application more immersive! https://awexr.com

Nick Klingensmith (Microsoft): Increasing Immersion with the MR Lighting Tools

AugmentedWorldExpo

A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Rokid: Design a seamless solution for AR glasses Qi Xiong | Rokid Paris Fan | Rokid Nan Wang | Rokid Gonglue Jiang | Rokid In this talk, we will discuss how Rokid creates a seamless AR experience from two aspects: ergonomic design(hardware) and systematic UI/UX design(software). The presentation will also share the insights of the entire product development cycle of Rokid Glass: from crafting product appearance with strict constraints, deploying user interfaces on a new and unique AR device, to the case studies and progressive design of real implementations. The talk offers a glimpse of the future AR design methodology by demonstrating the superpower enabled by design thinking. Rokid Glass is an AI-enabled, all-in-one AR glasses already in mass production, with seamless user experience which widely adopted by the industry. https://awexr.com

Rokid: Design a seamless solution for AR glasses

AugmentedWorldExpo

A talk from the Marking & Sales Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Tom De Vos | Fat Boy Dalia Lasaite | CGTrader Augmented Reality is having a profound impact on the way people shop online. By enabling consumers to interact with and try products before purchasing as they would in a brick and mortar store, Augmented Reality has been proven to increase sales conversions, reduce product returns and extend online engagement. This presentation will examine the latest technologies, trends and strategies for deploying AR for e-Commerce. https://awexr.com

How AR is Transforming the Online Retail Experience

AugmentedWorldExpo

A talk from the Enterprise Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Chloe Doyeon Kim (MAXST): The Current and Future State of MAXST Platform MAXST, a persistent challenger, is proudly specialized in Augmented Reality working with prominent Korean enterprises, such as Hyundai, Samsung Display, Samsung Electronics, and KT. MAXST’s cloud-based AR platform pursues innovation in terms of AR development tools, AR production costs, and profitability. We also believe Visual SLAM is critical basis for Future AR. We will reveal our stories and insight to the immersive world of today and tomorrow. https://awexr.com

Chloe Doyeon Kim (MAXST): The Current and Future State of MAXST Platform

AugmentedWorldExpo

A talk from the Start-up Pitch Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Owning Augmented Space Neil Mandt | The Layer Group Brian Wassom | Warner Norcross + Judd LLP "How can I own the AR 'air rights' around my physical property?" This may be the most commonly asked legal question around AR content. It is gaining new momentum and importance today, as location-based AR finally turns a profit, and as the quest for the Open AR Cloud takes shape. Finding the answers requires crossing the boundaries of multiple legal disciplines and practical concerns, and will not always be easy. This session will equip you to understand the legal risks and give you strategies to protect the integrity of both physical space and digital expression. https://awexr.com

AWE USA 2019: Owning Augmented Space

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Victor Sun (Shadow Creator): How to turn Mixed Reality “headset” into “glasses”? Microsoft has released HoloLens 2 on MWC. I think it is the benchmark for mixed reality devices, but it only reaches the pass mark. In order to be used by ordinary consumers, headsets must be turned into glasses to eliminate their fears of tech equipment. https://awexr.com

Victor Sun (Shadow Creator): How to turn Mixed Reality “headset” into “glasses”?

AugmentedWorldExpo

Chris Pickett (DigiLens): XR is Hard: Here’s Why

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Antti Sunnari (Dispelix Ltd): Full-color, single-waveguide near-eye displays for AR glasses and MR headsets For years, many have tried to develop a see-through, near-eye display technology that combines beautiful design, excellent image quality and scalable mass production with high yields. At Dispelix, we’ve figured it out. Dispelix is helping product companies to create beautiful AR glasses based on single-waveguide full-color displays with superior image quality and mass manufacturability. https://awexr.com

Antti Sunnari (Dispelix Ltd): Full-color, single-waveguide near-eye displays ...

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Jian Liang (HiScene): AR for Industry in China: From Concepts to Real Applications AI/AR industry has attracted attention never seen before of academia and industry, into which numerous talents and resources have been invested. However, academic achievements are not equal to products, which need to be adjusted and optimized in technology, engineering, product, etc. according to specific application scenarios. This talk will share with you some difficulties, misconceptions and experience in commercializing AR based on HiScene’s practice. https://awexr.com

Jian Liang (HiScene): AR for Industry in China: From Concepts to Real Applica...

AugmentedWorldExpo

A talk from the Gaming & Entertainment Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Augmented reality meets Television -Ellen DeGeneres' Game of Games AR App Daniel Leary | The Ellen DeGeneres Show Alexa Hook | The Ellen DeGeneres Show Paul Hake | HitPoint Inc. Ellen DeGeneres, America's most loved comedian and popular TV host, entered the augmented reality era with the "Game of Games" show. Over 900,000 app users get to tune in and play along with Ellen with AR by unlocking prizes that jump from the TV screen into viewers' phones. The app has hit 741,614 AR gift box markers scanned from NBC broadcasts of Game of Games, during the weekly Play Along Live game. At AWE, HitPoint Studios gives an insight into the creative and technical process of building the "Game of Games" Game app and how to connect traditional TV shows with AR technology. https://awexr.com

Augmented reality meets Television -Ellen DeGeneres' Game of Games AR App

AugmentedWorldExpo

Jennifer Cook (Dreamscape Immersive): Dreamscape Immersive Presentation

AugmentedWorldExpo

A talk from the Enterprise Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. 2 Partners sharing 1 vision for smart operators Mark Fleischer | AMA XPERTEYE INC. Peter Verstraeten | Proceedix The partnership between XpertEye and Proceedix offers a best-in-class solution to empower smart operators and technicians. The combination leverages both mobile technology and smart glasses for remote assistance and instruction and inspection execution. https://awexr.com

AWE USA 2019: 2 Partners sharing 1 vision for smart operators

AugmentedWorldExpo

Phil LaFond (Bosch Automotive Service Solutions Inc.): Bosch Technical Traini...

AugmentedWorldExpo

A talk from the Main Stage at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Scott Montgomerie (Scope AR): AR’s Influence on the Workforce of Tomorrow: Job Eliminator or Creator? As the speed of technology continues to accelerate automation in the manufacturing world, the inevitable question of whether or not the “human touch” will become obsolete is top of mind for workers. The fear that robots and smart technologies will take everyone's jobs is prevalent, but not necessarily true. AR has the power to be a job creator, not a job eliminator. Its ability to make anyone an instant expert can in fact increase job security by quickly helping workers become more proficient in tasks. Learn about real-world use cases where AR is making people better, and safer, at their jobs, and explore why enterprises who create a workplace that’s augmented, not automated, will be the leaders of tomorrow. https://awexr.com

Scott Montgomerie (Scope AR): AR’s Influence on the Workforce of Tomorrow: Jo...

AugmentedWorldExpo

A talk from the Main Stage at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Hugo Swart (Qualcomm, Inc.): How 5G, Distributed Processing and Technology Advancements are Accelerating the Timeline to a Ubiquitous XR World XR has the potential to become one of the world’s most disruptive computing platforms in the coming decade. Over the past year, we’ve seen significant strides in technology and with the introduction of new XR form factors such as standalone, tethered viewers and dual-mode HMDs – there has been an acceleration of XR opportunities, use cases and adoption. In this keynote presentation, Hugo Swart, Head of XR at Qualcomm Technologies, Inc. will speak to cutting-edge technological developments and how Qualcomm has met previous challenges, with a forward outlook on how strong ecosystem collaboration, the proliferation of 5G and AI will fast-track the industry timeline to an XR ubiquitous world. https://awexr.com

Hugo Swart (Qualcomm, Inc.): How 5G, Distributed Processing and Technology Ad...

AugmentedWorldExpo

A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Valentin Heun (PTC): One Reality - A 3rd generation AR User Interface Many of our technological innovations seem to expand the computational abilities without the human in the loop. Augmented Reality is one of the view technologies that promises technological advances that enable us humans a better connection with the ubiquitous technology around us. We are not there yet to make use of such a user interface. We currently have individual islands of interactivity via AR. This conference may have many spatial demos as well, however, a single ubiquitous layer that penetrates all screens, surfaces, machines, and environments are not available. This is what the PTC Reality Lab researches. The core of this research is the ability to use AR in a scalable distributed platform to bridge among technological domains to allow the weaving of multiple technologies into one user experience. The resulting user experience is called One Reality. The goal for One Reality is to provide one seamless, ubiquitous flow of user experience for all technologies that touch the user’s decision making. This talk will iterate on the different stages of Augmented Reality User Interfaces and identifies One Reality as the 3rd stage in the evolution of AR user interfaces. This visionary talk aims to inspire the audience to think beyond the known applications for Augmented Reality. https://awexr.com

Valentin Heun (PTC): One Reality - A 3rd generation AR User Interface

AugmentedWorldExpo

A talk from the Training & Education Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Anton Ebert (Siemens Healthcare): Challenges and approaches on integrating a cutting edge technology in modern healthcare businesses This session will provide insights and a basis for further discussions on how technologies like Augmented and Virtual Reality are being tested and explored in the healthcare sector. https://awexr.com

Anton Ebert (Siemens Healthcare): Challenges and approaches on integrating a ...

AugmentedWorldExpo

A talk from the Humanities Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Sarah Hill (Healium by StoryUP Studios): Healing Stories Powered by Biometrics Users aren't just watching your content, they're feeling it with their wearables. Brain patterns and heart rate are XR storytelling's new inputs. Peek inside the world's first VR & AR content channel powered by a brain-computer interface and heart rate to learn about a new study on XR's therapeutic impact on anxiety. https://awexr.com

Sarah Hill (Healium by StoryUP Studios): Healing Stories Powered by Biometrics

AugmentedWorldExpo

A talk from the Humanities Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Iva Leon (ARVR Women and Allies): Building community with social VR Social VR platforms like Facebook Spaces can help community builders and purpose-driven organizations innovate the way they engage their members and bring people together. Find out how ARVR Women uses social VR to elevate the visibility of their members and add value to the group. https://awexr.com

Iva Leon (ARVR Women and Allies): Building community with social VR

AugmentedWorldExpo

Mehr von AugmentedWorldExpo (20)