Lecture 4 from the COMP 4010 course on AR/VR. This lecture reviews optical tracking for AR and starts discussion about interaction techniques. This was taught by Mark Billinghurst at the University of South Australia on August 17th 2021.
A lecture give on AR Tehchnology taught as part of the COMP 4010 course on AR/VR. This lecture was taught by Mark Billinghurst on August 10th 2021 at the University of South Australia.
Lecture prepared by Mark Billinghurst on Augmented Reality tracking. Taught on October 18th 2016 by Dr. Gun Lee as part of the COMP 4010 VR class at the University of South Australia.
Lecture 5 in the COMP 4010 class on Augmented and Virtual Reality. This lecture was about AR Interaction and Prototyping methods. Taught by Mark Billinghurst on August 24th 2021 at the University of South Australia.
This document provides a summary of a lecture on perception in augmented and virtual reality. It discusses the history of disappearing computers from room-sized to handheld. It reviews the key concepts of augmented reality, virtual reality, and mixed reality on Milgram's continuum. It discusses how perception of reality works through our senses and how virtual reality aims to create an illusion of reality. It covers factors that influence the sense of presence such as immersion, interaction, and realism.
Lecture 10 in the COMP 4010 Lectures on AR/VR from the Univeristy of South Australia. This lecture is about VR Interface Design and Evaluating VR interfaces. Taught by Mark Billinghurst on October 12, 2021.
COMP4010 Lecture 4 - VR Technology - Visual and Haptic Displays. Lecture about VR visual and haptic display technology. Taught on August 16th 2016 by Mark Billinghurst from the University of South Australia
Lecture 11 of the COMP 4010 class on Augmented Reality and Virtual Reality. This lecture is about VR applications and was taught by Mark Billinghurst on October 19th 2021 at the University of South Australia
A lecture give on AR Tehchnology taught as part of the COMP 4010 course on AR/VR. This lecture was taught by Mark Billinghurst on August 10th 2021 at the University of South Australia.
Lecture prepared by Mark Billinghurst on Augmented Reality tracking. Taught on October 18th 2016 by Dr. Gun Lee as part of the COMP 4010 VR class at the University of South Australia.
Lecture 5 in the COMP 4010 class on Augmented and Virtual Reality. This lecture was about AR Interaction and Prototyping methods. Taught by Mark Billinghurst on August 24th 2021 at the University of South Australia.
This document provides a summary of a lecture on perception in augmented and virtual reality. It discusses the history of disappearing computers from room-sized to handheld. It reviews the key concepts of augmented reality, virtual reality, and mixed reality on Milgram's continuum. It discusses how perception of reality works through our senses and how virtual reality aims to create an illusion of reality. It covers factors that influence the sense of presence such as immersion, interaction, and realism.
Lecture 10 in the COMP 4010 Lectures on AR/VR from the Univeristy of South Australia. This lecture is about VR Interface Design and Evaluating VR interfaces. Taught by Mark Billinghurst on October 12, 2021.
COMP4010 Lecture 4 - VR Technology - Visual and Haptic Displays. Lecture about VR visual and haptic display technology. Taught on August 16th 2016 by Mark Billinghurst from the University of South Australia
Lecture 11 of the COMP 4010 class on Augmented Reality and Virtual Reality. This lecture is about VR applications and was taught by Mark Billinghurst on October 19th 2021 at the University of South Australia
This document provides an introduction to extended reality technologies from Mark Billinghurst, the director of the Empathic Computing Lab at the University of South Australia. It outlines Billinghurst's background and research interests. It then provides an overview of the class, including assignments, equipment available, and the lecture schedule. The lecture schedule covers topics such as augmented reality, virtual reality, the metaverse, and the history of AR/VR.
Lecture 9 of the COMP 4010 course in AR/VR from the University of South Australia. This was taught by Mark Billinghurst on October 5th, 2021. This lecture describes VR input devices, VR systems and rapid prototyping tools.
This document discusses various techniques for prototyping augmented reality interfaces, including sketching, storyboarding, wireframing, mockups, and video prototyping. Low-fidelity techniques like sketching and paper prototyping allow for rapid iteration and exploring interactions at early stages. Higher-fidelity techniques like interactive mockups and video prototypes communicate the look and feel of the final product and allow for user testing. A variety of tools are presented for different stages of prototyping, from sketching and interactive modeling in VR, to scene assembly using drag-and-drop tools, to final mockups using design software. Case studies demonstrate applying these techniques from initial concepts through to higher-fidelity prototypes. Overall the document
Lecture 12 in the COMP 4010 course on AR/VR. This lecture was about research directions in AR/VR and in particular display research. This was taught by Mark Billinghurst on September 26th 2021 at the University of South Australia.
The final lecture in the 2021 COMP 4010 class on AR/VR. This lecture summarizes some more research directions and trends in AR and VR. This lecture was taught by Mark Billinghurst on November 2nd 2021 at the University of South Australia
This document discusses augmented reality technology and visual tracking methods. It covers how humans perceive reality through their senses like sight, hearing, touch, etc. and how virtual reality systems use input and output devices. There are different types of visual tracking including marker-based tracking using artificial markers, markerless tracking using natural features, and simultaneous localization and mapping which builds a model of the environment while tracking. Common tracking technologies involve optical, magnetic, ultrasonic, and inertial sensors. Optical tracking in augmented reality uses computer vision techniques like feature detection and matching.
Lecture 2 in the 2022 COMP 4010 Lecture series on AR/VR and XR. This lecture is about human perception for AR/VR/XR experiences. This was taught by Mark Billinghurst at the University of South Australia in 2022.
Lecture 7 from the COMP 4010 class on AR and VR. This lecture was about Designing AR systems. It was taught on September 7th 2021 by Mark Billinghurst from the University of South Australia.
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
The document discusses using virtual avatars to improve remote collaboration. It provides background on communication cues used in face-to-face interactions versus remote communication. It then discusses early experiments using augmented reality for remote conferencing dating back to the 1990s. The document outlines key questions around designing effective virtual bodies for collaboration and discusses various technologies that have been developed for remote collaboration using augmented reality, virtual reality, and mixed reality. It summarizes several studies that have evaluated factors like avatar representation, sharing of different communication cues, and effects of spatial audio and visual cues on collaboration tasks.
A four lecture course on how to build AR and VR experiences using Unity, Google Cardboard VR SDK and Vuforia. Taught by Mark Billinghurst from May 10th - 13th, 2016 in XI'an, China
Lecture 8 of the COMP 4010 course taught at the University of South Australia. This lecture provides and introduction to VR technology. Taught by Mark Billinghurst on September 14th 2021 at the University of South Australia.
COMP 4010 Lecture 8 on an Introduction to Augmented Reality. This lecture provides a basic introduction to AR. Taught by Gun Lee on September 17th 2019 at the University of South Australia.
Talk given by Mark Billinghurst to Bajaj Finance Limited in India, on May 9th 2020. The talk describes AR and VR applications, example AR/VR applications in financial services, and potential research directions.
This lecture discusses presence in virtual reality. It defines presence as the subjective experience of being in a virtual environment rather than the physical one. Presence is influenced by how immersive a VR system is at stimulating the senses through sights, sounds etc. to generate realistic sensations. High presence leads to greater engagement from users and more natural reactions. The lecture compares presence to immersion and outlines different dimensions and methods of measuring presence, highlighting the importance of multi-sensory stimulation for creating strong feelings of presence.
Lecture 6 on the COMP4010 course on AR/VR. This lecture describes prototyping tools for developing interactive prototypes for AR experiences. The lecture was taught on August 31st 2020 by Mark Billinghurst at the University of South Australia
COMP 4010 Lecture 9 providing an overview of Augmented Reality Technology. Taught by Mark Billinghurst on October 8th 2019 at the University of South Australia.
Lecture 6 of the COMP 4010 course on AR/VR. This lecture is about designing AR systems. This was taught by Mark Billinghurst at the University of South Australia on September 1st 2022.
Lecture 4 in the 2022 COMP 4010 lecture series on AR/VR. This lecture is about AR Interaction techniques. This was taught by Mark Billinghurst at the University of South Australia in 2022.
A lecture on VR systems and graphics given as part of the COMP 4026 AR/VR class taught at the University of South Australia. This lecture was taught by Bruce Thomas on August 20th 2029.
This document provides an introduction to extended reality technologies from Mark Billinghurst, the director of the Empathic Computing Lab at the University of South Australia. It outlines Billinghurst's background and research interests. It then provides an overview of the class, including assignments, equipment available, and the lecture schedule. The lecture schedule covers topics such as augmented reality, virtual reality, the metaverse, and the history of AR/VR.
Lecture 9 of the COMP 4010 course in AR/VR from the University of South Australia. This was taught by Mark Billinghurst on October 5th, 2021. This lecture describes VR input devices, VR systems and rapid prototyping tools.
This document discusses various techniques for prototyping augmented reality interfaces, including sketching, storyboarding, wireframing, mockups, and video prototyping. Low-fidelity techniques like sketching and paper prototyping allow for rapid iteration and exploring interactions at early stages. Higher-fidelity techniques like interactive mockups and video prototypes communicate the look and feel of the final product and allow for user testing. A variety of tools are presented for different stages of prototyping, from sketching and interactive modeling in VR, to scene assembly using drag-and-drop tools, to final mockups using design software. Case studies demonstrate applying these techniques from initial concepts through to higher-fidelity prototypes. Overall the document
Lecture 12 in the COMP 4010 course on AR/VR. This lecture was about research directions in AR/VR and in particular display research. This was taught by Mark Billinghurst on September 26th 2021 at the University of South Australia.
The final lecture in the 2021 COMP 4010 class on AR/VR. This lecture summarizes some more research directions and trends in AR and VR. This lecture was taught by Mark Billinghurst on November 2nd 2021 at the University of South Australia
This document discusses augmented reality technology and visual tracking methods. It covers how humans perceive reality through their senses like sight, hearing, touch, etc. and how virtual reality systems use input and output devices. There are different types of visual tracking including marker-based tracking using artificial markers, markerless tracking using natural features, and simultaneous localization and mapping which builds a model of the environment while tracking. Common tracking technologies involve optical, magnetic, ultrasonic, and inertial sensors. Optical tracking in augmented reality uses computer vision techniques like feature detection and matching.
Lecture 2 in the 2022 COMP 4010 Lecture series on AR/VR and XR. This lecture is about human perception for AR/VR/XR experiences. This was taught by Mark Billinghurst at the University of South Australia in 2022.
Lecture 7 from the COMP 4010 class on AR and VR. This lecture was about Designing AR systems. It was taught on September 7th 2021 by Mark Billinghurst from the University of South Australia.
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
The document discusses using virtual avatars to improve remote collaboration. It provides background on communication cues used in face-to-face interactions versus remote communication. It then discusses early experiments using augmented reality for remote conferencing dating back to the 1990s. The document outlines key questions around designing effective virtual bodies for collaboration and discusses various technologies that have been developed for remote collaboration using augmented reality, virtual reality, and mixed reality. It summarizes several studies that have evaluated factors like avatar representation, sharing of different communication cues, and effects of spatial audio and visual cues on collaboration tasks.
A four lecture course on how to build AR and VR experiences using Unity, Google Cardboard VR SDK and Vuforia. Taught by Mark Billinghurst from May 10th - 13th, 2016 in XI'an, China
Lecture 8 of the COMP 4010 course taught at the University of South Australia. This lecture provides and introduction to VR technology. Taught by Mark Billinghurst on September 14th 2021 at the University of South Australia.
COMP 4010 Lecture 8 on an Introduction to Augmented Reality. This lecture provides a basic introduction to AR. Taught by Gun Lee on September 17th 2019 at the University of South Australia.
Talk given by Mark Billinghurst to Bajaj Finance Limited in India, on May 9th 2020. The talk describes AR and VR applications, example AR/VR applications in financial services, and potential research directions.
This lecture discusses presence in virtual reality. It defines presence as the subjective experience of being in a virtual environment rather than the physical one. Presence is influenced by how immersive a VR system is at stimulating the senses through sights, sounds etc. to generate realistic sensations. High presence leads to greater engagement from users and more natural reactions. The lecture compares presence to immersion and outlines different dimensions and methods of measuring presence, highlighting the importance of multi-sensory stimulation for creating strong feelings of presence.
Lecture 6 on the COMP4010 course on AR/VR. This lecture describes prototyping tools for developing interactive prototypes for AR experiences. The lecture was taught on August 31st 2020 by Mark Billinghurst at the University of South Australia
COMP 4010 Lecture 9 providing an overview of Augmented Reality Technology. Taught by Mark Billinghurst on October 8th 2019 at the University of South Australia.
Lecture 6 of the COMP 4010 course on AR/VR. This lecture is about designing AR systems. This was taught by Mark Billinghurst at the University of South Australia on September 1st 2022.
Lecture 4 in the 2022 COMP 4010 lecture series on AR/VR. This lecture is about AR Interaction techniques. This was taught by Mark Billinghurst at the University of South Australia in 2022.
A lecture on VR systems and graphics given as part of the COMP 4026 AR/VR class taught at the University of South Australia. This lecture was taught by Bruce Thomas on August 20th 2029.
This document summarizes a presentation about mobile augmented reality (AR). It discusses that AR enhances the real environment by combining real and virtual elements in real-time. Popular applications of AR include overlaying information in manuals, tourism maps, and educational materials. Current challenges to AR include photorealistic rendering, user perspective rendering, and occlusion handling with real objects. Popular AR devices discussed include Google Glass, Microsoft Hololens, Meta Spaceglasses, and AR apps on smartphones. The document also covers tracking techniques, including marker-based, markerless, and simultaneous localization and mapping methods.
Lecture 10 from a course on Mobile Based Augmented Reality Development taught by Mark Billinghurst and Zi Siang See on November 29th and 30th 2015 at Johor Bahru in Malaysia. This lecture provides an overview of research directions in Mobile AR. Look for the other 9 lectures in the course.
Overview of Computer Vision For Footwear IndustryTanvir Moin
Computer vision is an interdisciplinary field that focuses on enabling computers to interpret and analyze visual data from the world around us. It involves the development of algorithms and techniques that allow machines to understand images and videos, just as humans do.
The main goal of computer vision is to create machines that can "see" and understand the world around them, and then use that information to make decisions or take actions. This can involve tasks such as object recognition, scene reconstruction, facial recognition, and image segmentation.
Computer vision has a wide range of applications in various fields, such as healthcare, entertainment, transportation, robotics, and security. Some examples include medical image analysis, autonomous vehicles, augmented reality, and surveillance systems.
In recent years, the development of deep learning techniques, particularly convolutional neural networks (CNNs), has greatly advanced the field of computer vision, allowing machines to achieve state-of-the-art performance on various visual recognition tasks.
The second lecture from the Augmented Reality Summer School talk by Mark Billinghurst at the University of South Australia, February 15th - 19th, 2016. This provides an overview of AR Technology.
Lecture 2 from a course on Mobile Based Augmented Reality Development taught by Mark Billinghurst and Zi Siang See on November 29th and 30th 2015 at Johor Bahru in Malaysia. This lecture provides an introduction to Mobile AR Technology. Look for the other 9 lectures in the course.
This document describes a vision assisted pick and place robotic arm guided by image processing concepts for object sorting. It discusses introducing a robotic arm that can pick objects from one location and place them in another using machine vision. The document covers concepts like image acquisition, processing, object identification, and control signal transfer. It provides details on how a webcam captures images that are converted to grayscale and binary before edge detection and other processing to find object boundaries and centroids. This allows generating control signals to guide the robotic arm via a controller. Applications are in automated industries like assembly and potential enhancements are also discussed.
This document describes a vision assisted pick and place robotic arm guided by image processing concepts for object sorting. It discusses introducing a robotic arm that can pick objects from one location and place them in another using machine vision. The document covers key concepts like image acquisition, processing, object identification, and control signal transfer. It provides details on how a webcam captures images that are converted to grayscale and binary before edge detection and other processing to find object boundaries and centroids. Control signals are sent via an interface to guide the robotic arm based on image analysis. Potential applications and advantages like consistency and hazardous task handling are also summarized.
The document discusses principles of computer vision and its applications. It is a lecture by Dr. Vanessa Camilleri from the University of Malta on computer vision fundamentals and techniques. The key topics covered include object detection methods, stages of computer vision like image acquisition and processing, and examples of computer vision applications in various domains like manufacturing, healthcare, transportation and more.
This document discusses challenges, techniques, and lessons learned from shooting the film Europa in 3D. It describes the homemade stereo rig used and pre-visualization process. Key challenges included limited camera control, focus issues, and slower movement. Extensive visual effects were needed. Lessons included underestimating post-production scope, allowing more student involvement, and considering if 3D truly enhances the narrative. In the future, better planning and a better rig could improve the process.
Lecture 8 in the COMP 4010 course on AR and VR. This lecture gives an overview of Augmented Reality technology. Taught by Mark Billinghurst on October 5th, 2017 at the University of South Australia
This document provides an overview of an introduction to machine vision course. The course introduces concepts of machine vision including image formation and filtering. It addresses machine vision techniques such as feature detection, extraction, and pattern recognition. Students will explore applications and learn about enabling technologies. The course involves assignments, midterm and final exams to assess learning outcomes including understanding, applying, analyzing, evaluating, and designing approaches related to machine vision. Related fields, optical illusions, sample applications, software, and resources are also discussed.
1) Prof. Toon Goedemé discusses how computer vision can enable drones to operate autonomously by allowing them to map environments, localize themselves, avoid obstacles, detect and track objects.
2) Two main computer vision technologies for drones are visual simultaneous localization and mapping (SLAM) and object detection algorithms like Viola-Jones.
3) Example projects demonstrate autonomous orchard inspection and Cametron, a system using multiple drones to automatically film events like a virtual camera crew.
This document proposes a real-time system for designing and modeling using only a webcam. The system uses probability masks and optical flow to track hand movements in real-time and use them to draw on video chat. It outlines an approach using skin color probability masks, optical flow probability masks, and Lucas-Kanade optical flow to track regions of interest frame-by-frame. Future work includes improving performance, converting 2D drawings to 3D models in real-time, augmenting the 3D model onto the video in real-time, and sharing the model over video chat.
Mika Kaukoranta presents what computer vision is and how it can be utilized in software testing by gaining high-level understanding from digital images or videos.
Computer-Vision based Centralized Multi-agent System on Matlab and Arduino Du...Aritra Sarkar
This document summarizes an internship project involving a centralized multi-agent system using computer vision and Arduino microcontrollers. The project aims to track multiple mobile agents with an overhead camera for collective task completion. Key aspects include interfacing robots and cameras using Matlab and Arduino, developing image processing to identify robot locations, and simulating swarm behaviors like surrounding objects, escorting objects, and maze solving. The system is designed to be robust, scalable, and adaptable at low cost for potential applications in planetary exploration, construction, mining, emergency response, and other domains.
Augmented reality (AR) is an interactive experience of a real-world environment where computer-generated perceptual information enhances real-world objects. This can include visual, auditory, haptic, somatosensory, and olfactory enhancements. Examples include superimposing images, adding information/models, and altering appearances. Key uses of AR include training/education through dynamic instructions, entertainment through virtual experiences, and e-commerce through virtually trying on products. There are two main types of AR systems - monitor-based which displays enhancements on a screen, and see-through which allows users to see enhancements overlaid on the real world directly through devices.
How to easily improve quality using automated visual inspectionDesign World
Mis-registered parts, out of tolerance parts or defective assemblies are costly mistakes in today’s manufacturing environments. Reducing scrap by catching deviations in the manufacturing process early are key to keeping profit margins high.
Automated inspection using Vision Sensors provide 100% inspection. Learn how the VeriSens Vision sensors ease of use combined with powerful inspection tools catches detects during assembly. Join us in an educational based webinar to demonstrate how to improve quality using automated visual inspection.
Watch this webinar to learn:
-What is a vision sensor?
-What type of applications are suited for vision sensors
-How to easily setup a vision application with VeriSens vision sensors
This document provides an overview of feature detection techniques in machine vision, including edge detection, the Canny edge detector, interest points, and the Harris corner detector. It describes how edge detection works by finding discontinuities in images using masks and correlation. It explains that the Canny edge detector is an optimal method that uses Gaussian smoothing and non-maximum suppression. Interest points are localized features useful for applications like image alignment, and the Harris corner detector computes gradients to find locations with dominant directions, identifying corners.
Ähnlich wie Comp4010 Lecture4 AR Tracking and Interaction (20)
Keynote talk by Mark Billinghurst at the 9th XR-Metaverse conference in Busan, South Korea. The talk was given on May 20th, 2024. It talks about progress on achieving the Metaverse vision laid out in Neil Stephenson's book, Snowcrash.
These are slides from the Defence Industry event orgranized by the Australian Research Centre for Interactive and Virtual Environments (IVE). This was held on April 18th 2024, and showcased IVE research capabilities to the South Australian Defence industry.
This is a guest lecture given by Mark Billinghurst at the University of Sydney on March 27th 2024. It discusses some future research directions for Augmented Reality.
Presentation given by Mark Billinghurst at the 2024 XR Spring Summer School on March 7 2024. This lecture talks about different evaluation methods that can be used for Social XR/AR/VR experiences.
Empathic Computing: Delivering the Potential of the MetaverseMark Billinghurst
Invited guest lecture by Mark Billingurust given at the MIT Media Laboratory on November 21st 2023. This was given as part of Professor Hiroshi Ishii's class on Tangible Media
Empathic Computing: Capturing the Potential of the MetaverseMark Billinghurst
This document discusses empathic computing and its relationship to the metaverse. It defines key elements of the metaverse like virtual worlds, augmented reality, mirror worlds, and lifelogging. Research on the metaverse is still fragmented across these areas. The document outlines a vision for empathic computing systems that allow sharing experiences, emotions, and environments through technologies like virtual reality, augmented reality, and sensor data. Examples are given of research projects exploring collaborative VR experiences and AR/VR systems for remote collaboration and communication. The goal is for technology to support more natural and implicit understanding between people.
Empathic Computing: Designing for the Broader MetaverseMark Billinghurst
1) The document discusses the concept of empathic computing and its application to designing for the broader metaverse.
2) Empathic computing aims to develop systems that allow people to share what they are seeing, hearing, and feeling with others through technologies like augmented reality, virtual reality, and physiological sensors.
3) Potential research directions are explored, like using lifelogging data in VR, bringing elements of the real world into VR, and developing systems like "Mini-Me" avatars that can convey non-verbal communication cues to facilitate remote collaboration.
Keynote speech given by Mark Billinghurst at the ISS 2022 conference. Presented on November 22nd, 2022. This keynote outlines some research opportunities in the Metaverse.
Empathic Computing and Collaborative Immersive AnalyticsMark Billinghurst
This document discusses empathic computing and collaborative immersive analytics. It notes that while fields like scientific and information visualization are well established, little research has looked at collaborative visualization specifically. Collaborative immersive analytics combines mixed reality, visual analytics and computer-supported cooperative work. Empathic computing aims to develop systems that allow sharing experiences, emotions and perspectives using technologies like virtual and augmented reality with physiological sensors. Applying these concepts could enhance communication and understanding for collaborative immersive analytics tasks.
This document discusses how metaverse concepts can be applied to corporate learning and leadership development. It defines the metaverse and outlines its key components: virtual worlds, augmented reality, mirror worlds, and lifelogging. Traditional corporate learning is described as instructor-led, group-based, and discrete. The document proposes applying metaverse concepts like learning in the flow of work, just-in-time learning, and adaptive personalized learning. Specific applications explored are virtual reality for skills and soft skills training, augmented reality for hands-on training, lifelogging for adaptive training, and mirror worlds for capturing real-world tasks.
Empathic Computing: Developing for the Whole MetaverseMark Billinghurst
A keynote speech given by Mark Billinghurst at the Centre for Design and New Media at IIIT-Delhi. Given on June 16th 2022. This presentation is about how Empathic Computing can be used to develop for the entre range of the Metaverse.
keynote speech by Mark Billinghurst at the Workshop on Transitional Interfaces in Mixed and Cross-Reality, at the ACM ISS 2021 Conference. Given on November 14th 2021
Lecture 11 of the COMP 4010 class on Augmented Reality and Virtual Reality. This lecture is about VR applications and was taught by Mark Billinghurst on October 19th 2021 at the University of South Australia
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframePrecisely
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
3. Augmented Reality Definition
• Combines Real and Virtual Images
• Both can be seen at the same time
• Interactive in real-time
• The virtual content can be interacted with
• Registered in 3D
• Virtual objects appear fixed in space
4. Augmented RealityTechnology
• Combines Real and Virtual Images
• Needs: Display technology
• Interactive in real-time
• Needs: Input and interaction technology
• Registered in 3D
• Needs: Viewpoint tracking technology
5. Example: MagicLeap ML-1 AR Display
•Display
• Multi-layered Waveguide display
•Tracking
• Inside out SLAM tracking
•Input
• 6DOF wand, gesture input
6. AR Display Technologies
• Classification (Bimber/Raskar 2005)
• Head attached
• Head mounted display/projector
• Body attached
• Handheld display/projector
• Spatial
• Spatially aligned projector/monitor
7. Bimber, O., & Raskar, R. (2005). Spatial augmented reality: merging real and virtual worlds. CRC press.
DisplayTaxonomy
8. Types of Head Mounted Displays
Occluded
See-thru
Multiplexed
18. Magic Mirror AR Experience
• See AR overlay of an image of yourself
19. AR RequiresTracking and Registration
• Registration
• Positioning virtual object wrt real world
• Fixing virtual object on real object when view is fixed
• Calibration
• Offline measurements
• Measure camera relative to head mounted display
• Tracking
• Continually locating the user’s viewpoint when view moving
• Position (x,y,z), Orientation (r,p,y)
20. Sources of Registration Errors
•Static errors
• Optical distortions (in HMD)
• Mechanical misalignments
• Tracker errors
• Incorrect viewing parameters
•Dynamic errors
• System delays (largest source of error)
• 1 ms delay = 1/3 mm registration error
21. Dynamic errors
• Total Delay = 50 + 2 + 33 + 17 = 102 ms
• 1 ms delay = 1/3 mm = 33mm error
Tracking Calculate
Viewpoint
Simulation
Render
Scene
Draw to
Display
x,y,z
r,p,y
Application Loop
20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms
22. Reducing dynamic errors (1)
•Reduce system lag
•Faster components/system modules
•Reduce apparent lag
•Image deflection
•Image warping
23. Reducing dynamic errors (2)
• Match video + graphics input streams (video AR)
• Delay video of real world to match system lag
• User doesn’t notice
• Predictive Tracking
• Inertial sensors helpful
Azuma / Bishop 1994
28. Why Optical Tracking for AR?
• Many AR devices have cameras
• Mobile phone/tablet, Video see-through display
• Provides precise alignment between video and AR overlay
• Using features in video to generate pixel perfect alignment
• Real world has many visual features that can be tracked from
• Computer Vision is a well established discipline
• Over 40 years of research to draw on
• Old non real time algorithms can be run in real time on todays devices
29. Common AR Optical Tracking Types
• Marker Tracking
• Tracking known artificial markers/images
• e.g. ARToolKit square markers
• Markerless Tracking
• Tracking from known features in real world
• e.g. Vuforia image tracking
• Unprepared Tracking
• Tracking in unknown environment
• e.g. SLAM tracking
30. Visual Tracking Approaches
• Marker based tracking with artificial features
• Make a model before tracking
• Model based tracking with natural features
• Acquire a model before tracking
• Simultaneous localization and mapping
• Build a model while tracking it
31. Marker Tracking
• Available for more than 20 years
• Several open-source solutions exist
• ARToolKit, ARTag, ATK+, etc
• Fairly simple to implement
• Standard computer vision methods
• A rectangle provides 4 corner points
• Enough for pose estimation!
33. Key Problem: Finding Camera Position
• Need camera pose relative to marker to render AR graphics
Known image Image in Camera view Overlay AR content
34. Goal: Find Camera Pose
• Knowing:
• Position of key points in on-screen video image
• Camera properties (focal length, image distortion)
36. Coordinates for Marker Tracking
Marker Camera
•Final Goal
•Rotation & Translation
1: Camera Ideal Screen
•Perspective model
•Obtained from Camera Calibration
2: Ideal Screen Observed Screen
•Nonlinear function (barrel shape)
•Obtained from Camera Calibration
3: Marker Observed Screen
•Correspondence of 4 vertices
•Real time image processing
37. Marker Tracking – General Principle
1. Capturing image with known camera
2. Search for quadrilaterals
3. Pose estimation
from homography
4. Pose refinement
Minimize nonlinear
projection error
5. Use final pose
37
1
3
2
4
5
Image: Daniel Wagner
39. MarkerTracking – Fiducial Detection
• Threshold the whole image to black and white
• Search scanline by scanline for edges (white to black)
• Follow edge until either
• Back to starting pixel
• Image border
• Check for size
• Reject fiducials early that are too small (or too large)
40. MarkerTracking – Rectangle Fitting
• Start with an arbitrary point “x” on the contour
• The point with maximum distance must be a corner c0
• Create a diagonal through the center
• Find points c1 & c2 with maximum distance left and right of diag.
• New diagonal from c1 to c2
• Find point c3 right of diagonal with maximum distance
41. MarkerTracking – Pattern checking
• Calculate homography using the 4 corner points
• “Direct Linear Transform” algorithm
• Maps normalized coordinates to marker coordinates
(simple perspective projection, no camera model)
• Extract pattern by sampling and check
• Id (implicit encoding)
• Template (normalized cross correlation)
42. Marker tracking – Pose estimation
• Calculates marker pose relative to the camera
• Initial estimation directly from homography
• Very fast, but coarse with error
• Jitters a lot…
• Iterative Refinement using Gauss-Newton method
• 6 parameters (3 for position, 3 for rotation) to refine
• At each iteration we optimize on the error
• Iterate
43. Outcome: Camera Transform
• Transformation from Marker to Camera
• Rotation and Translation
TCM : 4x4 transformation matrix
from marker coord. to camera coord.
44. Tracking challenges inARToolKit
False positives and inter-marker confusion
(image by M. Fiala)
Image noise
(e.g. poor lens, block
coding /
compression, neon tube)
Unfocused camera,
motion blur
Dark/unevenly lit
scene, vignetting
Jittering
(Photoshop illustration)
Occlusion
(image by M. Fiala)
49. Visual Tracking Approaches
• Marker based tracking with artificial features
• Make a model before tracking
• Model based tracking with natural features
• Acquire a model before tracking
• Simultaneous localization and mapping
• Build a model while tracking it
50. Natural Feature Tracking
• Use Natural Cues of Real Elements
• Edges
• Surface Texture
• Interest Points
• Model or Model-Free
• No visual pollution
Contours
Features Points
Surfaces
51. Natural Features
• Detect salient interest points in image
• Must be easily found
• Location in image should remain stable
when viewpoint changes
• Requires textured surfaces
• Alternative: can use edge features (less discriminative)
• Match interest points to tracking model database
• Database filled with results of 3D reconstruction
• Matching entire (sub-)images is too costly
• Typically interest points are compiled into “descriptors”
Tracking 51
Image: Gerhard Reitmayr
Image: Martin Hirzer
54. Tracking by Keypoint Detection
• This is what most trackers do…
• Targets are detected every frame
• Popular because tracking and detection
are solved simultaneously
Keypoint detection
Descriptor creation
and matching
Outlier Removal
Pose estimation
and refinement
Camera Image
Pose
Recognition
55. Detection and Tracking
Detection
Incremental
tracking
Tracking target
detected
Tracking target
lost
Tracking target
not detected
Incremental
tracking ok
Start
+ Recognize target type
+ Detect target
+ Initialize camera pose
+ Fast
+ Robust to blur, lighting changes
+ Robust to tilt
• Tracking and detection are complementary approaches.
• After successful detection, the target is tracked incrementally.
• If the target is lost, the detection is activated again
56. What is a Keypoint?
• It depends on the detector you use!
• For high performance use the FAST corner detector
• Apply FAST to all pixels of your image
• Obtain a set of keypoints for your image
• Describe the keypoints
Rosten, E., & Drummond, T. (2006, May). Machine learning for high-speed corner detection.
In European conference on computer vision (pp. 430-443). Springer Berlin Heidelberg.
59. Descriptors
• Describe the Keypoint features
• Can use SIFT
• Estimate the dominant keypoint
orientation using gradients
• Compensate for detected
orientation
• Describe the keypoints in terms
of the gradients surrounding it
Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D.,
Real-Time Detection and Tracking for Augmented Reality on Mobile Phones.
IEEE Transactions on Visualization and Computer Graphics, May/June, 2010
60. Database Creation
• Offline step – create database of known features
• Searching for corners in a static image
• For robustness look at corners on multiple scales
• Some corners are more descriptive at larger or smaller scales
• We don’t know how far users will be from our image
• Build a database file with all descriptors and their
position on the original image
61. Real-time Tracking
• Search for known keypoints in the video
• Create the descriptors
• Match the descriptors from the
live video against those in the database
• Brute force is not an option
• Need the speed-up of special data structures
Keypoint detection
Descriptor creation
and matching
Outlier Removal
Pose estimation
and refinement
Camera Image
Pose
Recognition
62. NFT – Outlier removal
• Removing outlier features
• Several removal techniques
• Simple geometric tests
• Is the keypoint rotation invariant?
• Do keypoints remain relative to each other?
• Homography-based tests
Rotation Invariant
63. NFT – Pose refinement
• Pose from homography makes good
starting point
• Use Gauss-Newton iteration
• Try to minimize the re-projection error
of the keypoints
• Typically, 2-4 iterations are enough..
64. NFT – Real-time tracking
• Search for keypoints in the video image
• Create the descriptors
• Match the descriptors from the
live video against those in the database
• Remove the keypoints that are outliers
• Use the remaining keypoints
to calculate the pose of the camera
Keypoint detection
Descriptor creation
and matching
Outlier Removal
Pose estimation
and refinement
Camera Image
Pose
Recognition
70. 3D Model BasedTracking
• Tracking from 3D object shape
• Align detected features to 3D object model
• Examples
• SnapChat Face tracking
• Mechanical part tracking
• Vehicle tracking
• Etc..
74. Taxonomy of Model Based Tracking
Lowney, M., & Raj, A. S. (2016). Model based tracking for augmented reality on mobile devices.
75. Marker vs.Natural FeatureTracking
• Marker tracking
• Usually requires no database to be stored
• Markers can be an eye-catcher
• Tracking is less demanding
• The environment must be instrumented
• Markers usually work only when fully in view
• Natural feature tracking
• A database of keypoints must be stored/downloaded
• Natural feature targets might catch the attention less
• Natural feature targets are potentially everywhere
• Natural feature targets work also if partially in view
76. Visual Tracking Approaches
• Marker based tracking with artificial features
• Make a model before tracking
• Model based tracking with natural features
• Acquire a model before tracking
• Simultaneous localization and mapping
• Build a model while tracking it
78. Tracking from an Unknown Environment
• What to do when you don’t know any features?
• Very important problem in mobile robotics - Where am I?
• SLAM
• Simultaneously Localize And Map the environment
• Goal: to recover both camera pose and map structure
while initially knowing neither.
• Mapping:
• Building a map of the environment which the robot is in
• Localisation:
• Navigating this environment using the map while keeping
track of the robot’s relative position and orientation
79. Parallel Tracking and Mapping
Tracking Mapping
New keyframes
Map updates
+ Estimate camera pose
+ For every frame
+ Extend map
+ Improve map
+ Slow updates rate
Parallel tracking and mapping uses two
concurrent threads, one for tracking and one
for mapping, which run at different speeds
80. Parallel Tracking and Mapping
Video stream
New frames
Map updates
Tracking Mapping
Tracked local pose
FAST SLOW
Simultaneous
localization and mapping
(SLAM)
in small workspaces
Klein/Drummond, U.
Cambridge
81. Visual SLAM
• Early SLAM systems (1986 - )
• Computer visions and sensors (e.g. IMU, laser, etc.)
• One of the most important algorithms in Robotics
• Visual SLAM
• Using cameras only, such as stereo view
• MonoSLAM (single camera) developed in 2007 (Davidson)
83. How SLAMWorks
• Three main steps
1. Tracking a set of points through successive camera frames
2. Using these tracks to triangulate their 3D position
3. Simultaneously use the estimated point locations to calculate
the camera pose which could have observed them
• By observing a sufficient number of points can solve for both
structure and motion (camera path and scene structure).
84. Evolution of SLAM Systems
• MonoSLAM (Davidson, 2007)
• Real time SLAM from single camera
• PTAM (Klein, 2009)
• First SLAM implementation on mobile phone
• FAB-MAP (Cummins, 2008)
• Probabilistic Localization and Mapping
• DTAM (Newcombe, 2011)
• 3D surface reconstruction from every pixel in image
• KinectFusion (Izadi, 2011)
• Realtime dense surface mapping and tracking using RGB-D
86. LSD-SLAM (Engel 2014)
• A novel, direct monocular SLAM technique
• Uses image intensities both for tracking and mapping.
• The camera is tracked using direct image alignment, while
• Geometry is estimated as semi-dense depth maps
• Supports very large-scale tracking
• Runs in real time on CPU and smartphone
88. Direct Method vs. Feature Based
• Direct uses all information in image, cf feature based approach
that only use small patches around corners and edges
89. Applications of SLAM Systems
• Many possible applications
• Augmented Reality camera tracking
• Mobile robot localisation
• Real world navigation aid
• 3D scene reconstruction
• 3D Object reconstruction
• Etc..
• Assumptions
• Camera moves through an unchanging scene
• So not suitable for person tracking, gesture recognition
• Both involve non-rigidly deforming objects and a non-static map
91. Combining Sensors andVision
• Sensors
• Produces noisy output (= jittering augmentations)
• Are not sufficiently accurate (= wrongly placed augmentations)
• Gives us first information on where we are in the world,
and what we are looking at
• Vision
• Is more accurate (= stable and correct augmentations)
• Requires choosing the correct keypoint database to track from
• Requires registering our local coordinate frame (online-
generated model) to the global one (world)
93. Types of Sensor Fusion
• Complementary
• Combining sensors with different degrees of freedom
• Sensors must be synchronized (or requires inter-/extrapolation)
• E.g., combine position-only and orientation-only sensor
• E.g., orthogonal 1D sensors in gyro or magnetometer are complementary
• Competitive
• Different sensor types measure the same degree of freedom
• Redundant sensor fusion
• Use worse sensor only if better sensor is unavailable
• E.g., GPS + pedometer
• Statistical sensor fusion
www.augmentedrealitybook.org Tracking 93
94. Example: Outdoor Hybrid Tracking
• Combines
• computer vision
• inertial gyroscope sensors
• Both correct for each other
• Inertial gyro
• provides frame to frame prediction of camera
orientation, fast sensing
• drifts over time
• Computer vision
• Natural feature tracking, corrects for gyro drift
• Slower, less accurate
95. Robust OutdoorTracking
• HybridTracking
• ComputerVision, GPS, inertial
• Going Out
• Reitmayr & Drummond (Univ. Cambridge)
Reitmayr, G., & Drummond, T. W. (2006). Going out: robust model-based tracking for outdoor augmented reaity. In Mixed and
Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on (pp. 109-118). IEEE.
98. ARKit – Visual Inertial Odometry
• Uses both computer vision + inertial sensing
• Tracking position twice
• Computer Vision – feature tracking, 2D plane tracking
• Inertial sensing – using the phone IMU
• Output combined via Kalman filter
• Determine which output is most accurate
• Pass pose to ARKit SDK
• Each system compliments the other
• Computer vision – needs visual features
• IMU - drifts over time, doesn’t need features
99. ARKit –Visual Inertial Odometry
• Slow camera
• Fast IMU
• If camera drops out IMU takes over
• Camera corrects IMU errors
101. Conclusions
• Tracking and Registration are key problems
• Registration error
• Measures against static error
• Measures against dynamic error
• AR typically requires multiple tracking technologies
• Computer vision most popular
• Research Areas:
• SLAM systems, Deformable models, Mobile outdoor tracking
102. More Information
Fua, P., & Lepetit, V. (2007). Vision based 3D tracking
and pose estimation for mixed reality. In Emerging
technologies of augmented reality: Interfaces and
design (pp. 1-22). IGI Global.
106. AR Interaction
• Designing AR Systems = Interface Design
• Using different input and output technologies
• Objective is a high quality of user experience
• Ease of use and learning
• Performance and satisfaction
107. Typical Interface Design Path
1/ Prototype Demonstration
2/ Adoption of Interaction Techniques from
other interface metaphors
3/ Development of new interface metaphors
appropriate to the medium
4/ Development of formal theoretical models
for predicting and modeling user actions
Desktop WIMP
Virtual Reality
Augmented Reality
108. Interacting with AR Content
• You can see spatially registered AR..
how can you interact with it?
109. Different Types of AR Interaction
• Browsing Interfaces
• simple (conceptually!), unobtrusive
• 3D AR Interfaces
• expressive, creative, require attention
• Tangible Interfaces
• Embedded into conventional environments
• Tangible AR
• Combines TUI input + AR display
110. AR Interfaces as Data Browsers
• 2D/3D virtual objects are
registered in 3D
• “VR in Real World”
• Interaction
• 2D/3D virtual viewpoint control
• Applications
• Visualization, training
111. AR Information Browsers
• Information is registered
to
real-world context
• Hand held AR displays
• Interaction
• Manipulation of a window
into information space
• Applications
• Context-aware information
displays
Rekimoto, et al. 1997
114. Current AR Information Browsers
• Mobile AR
• GPS + compass
• Many Applications
• Wikitude
• Yelp
• Google maps
• …
115. Example: Google Maps AR Mode
• AR Navigation Aid
• GPS + compass, 2D/3D object placement
116.
117. Advantages and Disadvantages
• Important class of AR interfaces
• Wearable computers
• AR simulation, training
• Limited interactivity
• Modification of virtual
content is difficult
Rekimoto, et al. 1997
118. 3D AR Interfaces
• Virtual objects displayed in 3D
physical space and manipulated
• HMDs and 6DOF head-tracking
• 6DOF hand trackers for input
• Interaction
• Viewpoint control
• Traditional 3D user interface
interaction: manipulation, selection,
etc.
Kiyokawa, et al. 2000
122. Advantages and Disadvantages
• Important class of AR interfaces
• Entertainment, design, training
• Advantages
• User can interact with 3D virtual
object everywhere in space
• Natural, familiar interaction
• Disadvantages
• Usually no tactile feedback
• User has to use different devices for
virtual and physical objects
Oshima, et al. 2000