SlideShare ist ein Scribd-Unternehmen logo
1 von 83
Downloaden Sie, um offline zu lesen
Video Fields: Fusing Multiple Surveillance Videos into a
Dynamic Virtual Environment
Ruofei Du, Sujal Bista, Amitabh Varshney
The Augmentarium| UMIACS | University of Maryland, College Park
{ruofei, sujal, varshney} @ cs.umd.edu
www.VideoFields.com
image courtesy: university of maryland, college park
Introduction
Surveillance Videos - Monitoring
image courtesy: www.icsc.org
Introduction
Surveillance Videos – Shopping Centers
image courtesy: wikipedia
Introduction
Surveillance Videos - Airports
image courtesy: wikipedia
Introduction
Surveillance Videos – Train stations
image courtesy: university of maryland, college park
Introduction
Surveillance Videos - Campuses
image courtesy: university of maryland, college park
Introduction
Surveillance Videos - Conventional
image courtesy: theimaginativeconservative.org
Introduction
Surveillance Videos – Cognitive Burden
image courtesy: university of maryland, college park
Introduction
Surveillance Videos – Fusing & Interpreting
Related Work
Fusing Multiple Static Photographs
Related Work
Fusing Multiple Static Photographs
Related Work
Fusing Multiple Static Photographs
Related Work
Fusing Multiple Static Photographs
Related Work
Fusing Multiple Static Photographs
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
RGB
Related Work
Fusing Multiple Dynamic Videos
RGB
RGBD
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
SIGGRAPH 2016
Wednesday, 3:30-4:00 PM
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Related Work
Fusing Multiple Dynamic Videos
Our Approach?
Video Fields
Video Fields
Introduction
Video Field
Introduction
Video Field
Conception, architecting & implementation
Video Fields
A mixed reality system that fuses multiple
surveillance videos into an immersive virtual
environment,
Integrating automatic segmentation of
moving entities
Video Fields Rendering
Real-time fragment shader processing
Two algorithms to fuse multiple videos
Early & deferred pruning
These methods use voxels and meshes respectively
to render moving entities in the video fields
Achieving cross-platform compatibility by
WebGL + Three.js
smartphones, tablets, desktop, high-resolution
large-area wide field of view tiled display walls, as
well as head-mounted displays.
System Overview
Architecture
Video Fields Flowchart
Architecture
Video Fields Flowchart
Architecture
Video Fields Flowchart
Architecture
Video Fields Flowchart
Background Modeling
Motivation
• Provide a background texture for each camera
• Identify moving entities in the rendering stage
• Reduce the network bandwidth requirements
Background Modeling
Gaussian Mixture Models (GMM)
Background Modeling
Advantages [Stauffer and Grimson]
More adaptive with:
• different lighting conditions,
• repetitive motions of scene elements,
• moving entities in slow motion
Architecture
Video Fields Flowchart
Segmentation
Moving Entities
Background Modeling
Gaussian Mixture Models (GMM)
Architecture
Video Fields Flowchart
Visibility Test
Plus Opacity Modulation
Architecture
Video Fields Flowchart
Video Fields Mapping
Overview
Video Fields Mapping
Challenges
1. Vertex in the 3D models -> Pixel in the texture space
2. Pixel in the texture space -> Vertex on the ground
• The second is useful for projecting a 2D segmentation
of a moving entity to the 3D world
Video Fields Mapping
Projection Mapping
Video Fields Mapping
Perspective correction
Video Fields Mapping
Depth Map / Hashing Function
Early Pruning for Rendering
Moving Entities
Voxels
Deferred Pruning for
Rendering Moving Entities
Billboards
Visual Comparison
Early Pruning vs. Deferred Pruning
View-dependent Rendering
View-dependent Rendering
View-dependent Rendering
View-dependent Rendering
Experimental Results
Early Pruning vs. Deferred Pruning
Experimental Results
Early Pruning vs. Deferred Pruning
Experimental Results
Early Pruning vs. Deferred Pruning
Visual Comparison
Early Pruning vs. Deferred Pruning
Future Work
Scale Up - Hundreds of cameras
Future Work
Bandwidth Problem
Future Work
Holoportation with RGB cameras
Acknowledgement
Augmentarium Lab | GVIL | UMIACS
Acknowledgement
NSF | Nvidia | MPower | UMIACS
Video Fields
www.Video-Fields.com
Thank you! Questions or comments?
Ruofei Du and Amitabh Varshney
Augmentarium Lab | GVIL | UMIACS
Web3D 2016

Weitere ähnliche Inhalte

Ähnlich wie Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment

Harvesting Crowdsourced Mobile Videos under Bandwidth Constraint
Harvesting Crowdsourced Mobile Videos under Bandwidth ConstraintHarvesting Crowdsourced Mobile Videos under Bandwidth Constraint
Harvesting Crowdsourced Mobile Videos under Bandwidth Constraint
University of Southern California
 
Fiera sicurezza trs_20101117-19_all
Fiera sicurezza trs_20101117-19_allFiera sicurezza trs_20101117-19_all
Fiera sicurezza trs_20101117-19_all
trsitaly
 
Video saliency-recognition by applying custom spatio temporal fusion technique
Video saliency-recognition by applying custom spatio temporal fusion techniqueVideo saliency-recognition by applying custom spatio temporal fusion technique
Video saliency-recognition by applying custom spatio temporal fusion technique
IAESIJAI
 

Ähnlich wie Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment (20)

Harvesting Crowdsourced Mobile Videos under Bandwidth Constraint
Harvesting Crowdsourced Mobile Videos under Bandwidth ConstraintHarvesting Crowdsourced Mobile Videos under Bandwidth Constraint
Harvesting Crowdsourced Mobile Videos under Bandwidth Constraint
 
International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)
 
Video copy detection using segmentation method and
Video copy detection using segmentation method andVideo copy detection using segmentation method and
Video copy detection using segmentation method and
 
Action_recognition-topic.pptx
Action_recognition-topic.pptxAction_recognition-topic.pptx
Action_recognition-topic.pptx
 
Ppt safecities axis_ agentvi_jvp_en_1505
Ppt safecities axis_ agentvi_jvp_en_1505Ppt safecities axis_ agentvi_jvp_en_1505
Ppt safecities axis_ agentvi_jvp_en_1505
 
IRJET - Applications of Image and Video Deduplication: A Survey
IRJET -  	  Applications of Image and Video Deduplication: A SurveyIRJET -  	  Applications of Image and Video Deduplication: A Survey
IRJET - Applications of Image and Video Deduplication: A Survey
 
VVC Project.pptx
VVC Project.pptxVVC Project.pptx
VVC Project.pptx
 
VVC Project.pptx
VVC Project.pptxVVC Project.pptx
VVC Project.pptx
 
Multi-View Video Coding Algorithms/Techniques: A Comprehensive Study
Multi-View Video Coding Algorithms/Techniques: A Comprehensive StudyMulti-View Video Coding Algorithms/Techniques: A Comprehensive Study
Multi-View Video Coding Algorithms/Techniques: A Comprehensive Study
 
VVC_PPT.pptx
VVC_PPT.pptxVVC_PPT.pptx
VVC_PPT.pptx
 
High Quality Video Simulation from Still Images
High Quality Video Simulation from Still ImagesHigh Quality Video Simulation from Still Images
High Quality Video Simulation from Still Images
 
OBJECT-MEDIA: FROM PERSONALISATION TO A SEAMLESS TV/VR CONVERGENCE
OBJECT-MEDIA: FROM PERSONALISATION TO A SEAMLESS TV/VR  CONVERGENCEOBJECT-MEDIA: FROM PERSONALISATION TO A SEAMLESS TV/VR  CONVERGENCE
OBJECT-MEDIA: FROM PERSONALISATION TO A SEAMLESS TV/VR CONVERGENCE
 
Deploy Live Video and Machine Learning for Rapid Decision-Making & Budget Eff...
Deploy Live Video and Machine Learning for Rapid Decision-Making & Budget Eff...Deploy Live Video and Machine Learning for Rapid Decision-Making & Budget Eff...
Deploy Live Video and Machine Learning for Rapid Decision-Making & Budget Eff...
 
Fiera sicurezza trs_20101117-19_all
Fiera sicurezza trs_20101117-19_allFiera sicurezza trs_20101117-19_all
Fiera sicurezza trs_20101117-19_all
 
20160221 va interconnect_pub
20160221 va interconnect_pub20160221 va interconnect_pub
20160221 va interconnect_pub
 
DVO FAQ - Streaming Video
DVO FAQ - Streaming VideoDVO FAQ - Streaming Video
DVO FAQ - Streaming Video
 
Parking Surveillance Footage Summarization
Parking Surveillance Footage SummarizationParking Surveillance Footage Summarization
Parking Surveillance Footage Summarization
 
An Stepped Forward Security System for Multimedia Content Material for Cloud ...
An Stepped Forward Security System for Multimedia Content Material for Cloud ...An Stepped Forward Security System for Multimedia Content Material for Cloud ...
An Stepped Forward Security System for Multimedia Content Material for Cloud ...
 
Video saliency-recognition by applying custom spatio temporal fusion technique
Video saliency-recognition by applying custom spatio temporal fusion techniqueVideo saliency-recognition by applying custom spatio temporal fusion technique
Video saliency-recognition by applying custom spatio temporal fusion technique
 
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Robust face recognition from multi...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Robust face recognition from multi...IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Robust face recognition from multi...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Robust face recognition from multi...
 

Mehr von Ruofei Du

Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
Ruofei Du
 
Fusing Multimedia Data Into Dynamic Virtual Environments
Fusing Multimedia Data Into Dynamic Virtual EnvironmentsFusing Multimedia Data Into Dynamic Virtual Environments
Fusing Multimedia Data Into Dynamic Virtual Environments
Ruofei Du
 
Montage4D: Interactive Seamless Fusion of Multiview Video Textures
Montage4D: Interactive Seamless Fusion of Multiview Video TexturesMontage4D: Interactive Seamless Fusion of Multiview Video Textures
Montage4D: Interactive Seamless Fusion of Multiview Video Textures
Ruofei Du
 
Deliberately Planning and Acting for Angry Birds with Refinement Methods
Deliberately Planning and Acting for Angry Birds with Refinement MethodsDeliberately Planning and Acting for Angry Birds with Refinement Methods
Deliberately Planning and Acting for Angry Birds with Refinement Methods
Ruofei Du
 

Mehr von Ruofei Du (11)

Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Soc...
 
Geollery: A Mixed Reality Social Media Platform
Geollery: A Mixed Reality Social Media PlatformGeollery: A Mixed Reality Social Media Platform
Geollery: A Mixed Reality Social Media Platform
 
Fusing Multimedia Data Into Dynamic Virtual Environments
Fusing Multimedia Data Into Dynamic Virtual EnvironmentsFusing Multimedia Data Into Dynamic Virtual Environments
Fusing Multimedia Data Into Dynamic Virtual Environments
 
Montage4D: Interactive Seamless Fusion of Multiview Video Textures
Montage4D: Interactive Seamless Fusion of Multiview Video TexturesMontage4D: Interactive Seamless Fusion of Multiview Video Textures
Montage4D: Interactive Seamless Fusion of Multiview Video Textures
 
CTUAAA Summit 2017 Schedule
CTUAAA Summit 2017 ScheduleCTUAAA Summit 2017 Schedule
CTUAAA Summit 2017 Schedule
 
交大历史与梅竹赛
交大历史与梅竹赛交大历史与梅竹赛
交大历史与梅竹赛
 
Chinese Caligraphy 品读书法·感悟中华 (2010)
Chinese Caligraphy 品读书法·感悟中华 (2010)Chinese Caligraphy 品读书法·感悟中华 (2010)
Chinese Caligraphy 品读书法·感悟中华 (2010)
 
Statistics for K-mer Based Splicing Analysis
Statistics for K-mer Based Splicing AnalysisStatistics for K-mer Based Splicing Analysis
Statistics for K-mer Based Splicing Analysis
 
基于视频的疲劳驾驶检测系统
基于视频的疲劳驾驶检测系统基于视频的疲劳驾驶检测系统
基于视频的疲劳驾驶检测系统
 
Online Vigilance Analysis Combining Video and Electrooculography Features
Online Vigilance Analysis Combining Video and Electrooculography FeaturesOnline Vigilance Analysis Combining Video and Electrooculography Features
Online Vigilance Analysis Combining Video and Electrooculography Features
 
Deliberately Planning and Acting for Angry Birds with Refinement Methods
Deliberately Planning and Acting for Angry Birds with Refinement MethodsDeliberately Planning and Acting for Angry Birds with Refinement Methods
Deliberately Planning and Acting for Angry Birds with Refinement Methods
 

Kürzlich hochgeladen

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Kürzlich hochgeladen (20)

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 

Video Fields: Fusing Multiple Surveillance Videos into a Dynamic Virtual Environment