Augmented Reality and Virtual Reality: Research Advances in Creative Industry

Researcher: Zi Siang See
Augmented Reality and Virtual Reality:
Research Advances in Creative Industries
Zi Siang See
Research: zisiangsee@sunway.edu.my
Industrial collaboration: zisiang@reina.com.my
Research Workshop Seminar
Faculty of Arts, Sunway University
Research Workshop Seminar: Zi Siang See
Centre for Research Creation in Digital Media
22 June 2016 – University of Reading
Research &
collaboration

Workshop
1) Introduction
2) Demonstration
3) Work-in-progress
4) Discussion
Research &
collaboration

Professional Background:
Industrial involvements since 2003/04

Researcher: Zi Siang SeeVirtual Reality 360 (VR360) | Presenter: Zi Siang See
Research Interest
• Virtual Reality 360 (VR360)
• Augmented Reality
• Interactive Multimedia
• Human Computer Interaction
• Computational Photography
• Imography Research
Research and Specialization

Work: McDonald’s Malaysia
Creative: NagaDDB
Operation: Quality Ingredient Story
Creative Imaging: Reina Imaging

Researcher: Zi Siang SeeDigital Imaging (DI) image manipulation for advertising usage, high technical finishing | Presenter: Zi Siang See

Researcher: Zi Siang SeeDigital Imaging (D() for Advertising Visuals| Presenter: Zi Siang See

Researcher: Zi Siang SeeAdvertising visual | Presenter: Zi Siang See

Researcher: Zi Siang SeeDigital Imaging (DI) for Advertising Visuals| Presenter: Zi Siang See

Researcher: Zi Siang SeeObject Virtual Reality (Object VR) | Presenter: Zi Siang See

Discover MINI
Enter at your own risk of losing control

www.vsmm2016.org

“Creativity is not an option, it’s an absolute necessity”
Sir Ken Robinson 2014

VISION
The Centre seeks to be a cornerstone for transformative digital media
research, scholarship and innovation to advance Sunway University onto the
global new media-scape.
MISSION
To facilitate synergy between the arts and sciences, cultivating an innovative
media research-creation environment.
To create significant collaborations with key local and international centres
and labs to catalyze new partnerships.
To motivate a new generation of students and researchers in the domain of
creative digital media (CDM) merging core attributes of
the arts, sciences, humanities, media, and technology.
Vision & Mission

Centre for Research-Creation in Digital Media

UC Berkeley
Collaboration

The Project
Founding Projects

Figure: VSMM2016 also features the system by Prof Sarah Kenderdine

Introduction
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve

Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
• Touch
• Scent
• Taste
• Sight
Introduction
Trend in Industries
• Mixed and Augmented Reality
• Human Computer Interaction
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve

Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard.

Figure: Sensorama (Morton L Heiling, 1962)

Major changes in computing platforms
Desk /Table Pocket Palm /Hand Head /Eye Ubiquitous

Figure: augmented reality, key examples from search engine
Mixed, Augmented & Virtual Reality

Virtual Reality
Figure: NASA, 1988 Virtual Environment System

Development
Figure: Super cockpit project, Tom Furness.

Near-eye-display and fully augmented interactive display experience
How far can we go?

Augmented Reality
Figure: augmented reality experience via google glass project

Augmented Reality
• Simulation of interface at www.glasssim.com
• Exploration in modern wearable interface (fashionable, UX practical)

Google Glass
• Google Glass, developer version, 2013
• Can be configured on user’s customized or personal spectacles.

Recon Jet
• Available 2014?
• Sporty design
• Cost “friendly”
• Android compatible
• Smartphone connectivity

Figure: potential areas, google glass project

Augmented Reality
• Augmented reality (AR) permits computer aided visualization that works with
human interaction (Billinghurst, 2012).
• Billinghurst, M. (2011). The Future of Augmented Reality in Our Everyday Life.
• Damian, I., Kistler, F., Obaid, M., Bueling, R., Billinghurst, M. and Andre, E.
(2013) Motion Capturing Empowered Interaction with a Virtual Agent in an
Augmented Reality Environment.
Figure: Human Interface Technology Laboratory (HITLab), University of Canterbury, NZ.

Mixed Reality
• Human Pacman: a mobile wide-area entertainment system based on physical,
social, and ubiquitous computing. (Cheok et al., 2004)
• Digital Taste Interface project, Electronic taste stimulation (Ranasinghe et al.,
2011)
• Mediating intimacy in long-distance relationships using kiss messaging
(Saadatian et al., 2013)
Figure: Mixed Reality Lab, the future of internet.

Introduction: Mixed & Augmented Reality
• Displays, tracers, and AR systems in general need to become more accurate,
lighter, cheaper, and less power consuming (Azuma, 2001).
• Augmented reality (AR) is related to the concept of virtual reality (VR); VR
attempts to create an artificial world that a person can experience and explore
interactively, predominantly through his or her sense of vision, but also via
audio, tactile, and other forms of feedback (Höllerer and Feiner, 2004)
Figure: wearable computing, Steve Mann.

Possibilities
• Human Pacman: a mobbile wide-area entertainment system based on physical,
social, and ubquitos computing. (Cheok et al., 2004)
• Digital Taste Interface project, Digital taste and smell communication
(Ranasinghe et al., 2011)
• Digital Smell and Taste Communication, City University London.

Experiment
Augmented reality medical laboratory
(example)
http://code.wikitude.com/medicalmurmurs
Hearable AR

Augmented Reality: Various Approaches
• Various marker-based or markerless AR tracking approaches has
been discussed (Billinghurst, 2012; Cheok et al., 2004; Lieberknecht
et al., 2009)
• Image-based AR tracking (Lieberknecht et al., 2009)
• CAD /3D object feature tracking (Metaio, 2014)
• Face tracking (Metaio, 2014)

Augmented Reality using High Fidelity Spherical Panorama with HDRI
(Authoring process for AR environment recognition)
Spherical Panorama facilitated with HDRI
(Cubic projection)
Cropped /selective cubic facade
working with image-based AR
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications.
http://dx.doi.org/10.1145/2818427.2818445
http://www.zisiangsee.com/research/zisiangsee_research_p009.pdf

AR for Mobile and Wearable
Figure: handheld experience of markerless AR.

Figure: handheld experience of markerless AR (video).

Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.

Google Cardboard
• Carl Zeiss just announced VR One, a virtual reality headset for use with a
smartphone. It is a viewer designed to work with phones between 4.7 and 5.2
inches (Zeiss, 2014).

Google Cardboard (v1)
• Carl Zeiss just announced VR One, a virtual reality headset for use with a
smartphone. It is a viewer designed to work with phones between 4.7 and 5.2
inches (Zeiss, 2014).
Programmable
NFC Tag
Low cost resin
/plastic lens
Mobile device as
computing module
Endless configurable size
and shapes (and materials)
Magnetometer

Google Cardboard (v2)
more reading at http://www.gizmag.com/google-cardboard-2-review-initial/37777/
Presses the
screen
Figure: google cardboard v2 supports larger phone.

What do I see?

Samsung Gear VR
• Samsung has launched Gear VR in October 2014, co-developed with Oculus.
Gear VR can house specific mobile device (Note 4) for providing virtual reality
and augmented user experience (Samsung, 2014)
• Control interface included

Samsung Gear VR
• VR /AR for social media
• UX prototyping

Figure: able to produce 360 images or videos, for HMD and other interface usage.
Gear 360

Oculus Rift
Figure: Oculus, acquired by Facebook

Android VR
Figure: Android VR collaborating with other developers
Google has announced eight hardware partners that will make Daydream-ready phones,
including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel.

Metaio
Figure: Metaio, acquired by Apple
Metaio was one of the most technologically-ambitious AR organization, acquired by Apple in 2015

Media Companies’ Interest
Corporate media investment into AR/VR statup - infographic.

User Experience

Figure: European researchers used 3D-modeling and Instagram to recreate Mosul Museum destroyed in civil war.

User Experience

User Experience (Concept)
• Concept video produced by Infinity AR, 2014.
• Interface design, UX, daily applications.
Hyper-reality by Keiichi Matsuda

Mixed, Virtual, Augmented Reality
• Ikea AR

• McDonald’s McMission AR

• Penguins NAVI

• Octagon 4D (Animal series)

• Octagon 4D (Octaland series)

• Concept video produced by Infinity AR, 2014.
• Interface design, UX, daily applications.

Archiving our world in 3D
• LIDAR 3D Scanning
• Characteristics: trained users, costly, precision CAD content.

• Quiver
• Physical involvement during creation process
• Visual prototyping /conceptualizing

Gartner’s Hype Cycle

Definitions
Virtual Reality
• Virtual Reality (VR) which can be referred to as immersive
multimedia or computer-simulated life, replicates an environment
that simulates physical presence in places in the real world or
imagined worlds and lets the user interact in that world. Virtual
reality artificially creates sensory experiences, which can even
include sight, hearing, touch, and smell.
Augmented Reality
• Augmented Reality (AR) mixes a live real-world view with virtual
interactive content on a mobile or wearable device. One of the key
enablers for this is tracking technology, such as computer vision
techniques for tracking off pre-defined markers or markerless
images.

VR AR
Source and reinterpreted from
http://www.augment.com/blog/which-headset-is-right-for-you/

Research Possibility
Explore into
• Method and apparatus for augmented reality
• Method and apparatus for virtual reality
• Method and apparatus for mixed reality
• Ubiquitous computing
• Obstacles and issues
• Technology management
• Education technology
• Engineering and concept design
• Collaboration /Multi-disciplinary
• User interface (UI) design
• User experience (UX)

Everything is achievable
through technology

Research Collaboration & Publications
• Lee Xia Sheng, Chung Pui Yan and Zi Siang See (2016) Irregular Shaped Building Design Optimization with Building Information Modeling.
Proceeding in 4th International Building Control Conference 2016, Universiti Malaya. [indexed in SCOPUS]
• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia
2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.
• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI – Demonstration.
SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.
• Zi Siang See, Adrian David Cheok. (2015) Virtual Reality 360 Interactive Panorama Reproduction Obstacle and Issue. Virtual Reality, Volume 19, Issue
2, pp 71-81, Springer. [indexed by ISI, SCOPUS]
• Lee Xia Sheng, Lee Ching Hua, Cheah Shyn Yi and Zi Siang See (2014) Architectural Transformation of a Night Market Stall. International Conference
on Innovation and Technology for Sustainable Built Environment 2 2014, Universiti Teknologi MARA (UiTM) Perak.
• Zi Siang See, Khairul Hazrin Hashim, Harold Thwaites, Lee Xia Sheng, Ooi Wooi Har. (2012). Retrieving Extended High Dynamic Range From Digital
Negative Image - An Experiment On Architectural Photo Imaging. World Academy of Science, Engineering and Technology, Vol 6(8) August 2012,
pp. 1833-1841. [indexed by EBSCO]
• Zi Siang See, Lee Xia Sheng. (2011). Non-bias Architectural Image Archive using High Dynamic Range Approach. 2nd International Conference on
Research and Innovation in Information Systems 2011 (ICRIIS’11) [CD Proceedings], Kuala Lumpur, Malaysia, 23-24 November. [indexed by SCOPUS,
IEEE Xplorer]
• Zi Siang See. (2011). Non-bias Scientific Imaging Archive for Urban Morphology Research. Exchanging Views on How to Sustain Urban and Rural
Environments in Aspects of Technology, Social and Ecology in Green Innovation Research Group (GIRG) [CD-ROM], Universiti Teknologi Malaysia
(UTM), Malaysia.

Thank you
Research Discussion
research collaboration
www.zisiangsee.com

Facebook: Zi Siang See
research collaboration
www.zisiangsee.com

Augmented Reality and Virtual Reality: Research Advances in Creative Industry

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (11)

Ähnlich wie Augmented Reality and Virtual Reality: Research Advances in Creative Industry

Ähnlich wie Augmented Reality and Virtual Reality: Research Advances in Creative Industry (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Augmented Reality and Virtual Reality: Research Advances in Creative Industry