1.
Takashi Yoshinaga
Institute of Systems, Information
Technologies and Nanotechnologies
(ISIT)
Talk @ Universitat Oberta de Catalunya, 06.04.2016

2.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

3.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

4.
Augmented Reality (AR) is a term for real-world
environment whose elements are augmented by
computer generated information.

5.
AR is technology to superimpose
digital information into real scene viewed by user.

6.
Position and orientation of user or target
are necessary to superimpose objects precisely.
We can use mainly 2 types of techniques.
Image Recognition Geo Localization

7.
AR is available at any time if you have
Smartphone, PC (with Web Cam) or HMD.
Smartphone
or Tablet PC
PC with
Web Camera
HMD
(Head Mouted Display)

8.
AR Is Available For Everyone!

9.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

10.
AR is used in many fields such as:
• Marketing
• Travel Navigation
• Maintenance and Repair
• Entertainment
• Medical
• Education
etc...

11.
Pages of catalog are used as a target to superimpose furniture.
https://youtu.be/8_yXEluXLbU

12.
https://youtu.be/IK4-zPD_25U
Virtual penguins take travelers from the station to the aquarium.
GPS and Compass are used to recognize user’s position/direction.

13.
Annotations and internal structure are superimposed on a machine.
Procedures of maintenance is shown by video and 3D animation.
https://youtu.be/nHfY56lHZjU

14.
This application brings AR experience to coloring book.
https://youtu.be/EGMjsYHD7Ak

15.
http://quivervision.com/es/
Quiver - 3D Coloring App

16.
It was developed for viewing an animated
3D model of a Triceratops with photo-realistic quality.
.
https://youtu.be/LmShjXsIwA8

17.
Letting doctors and surgeons examine a holographic view of
patient’s internal anatomy which are reconstructed from CT/MRI.
AR allows for more advanced pre-operative imaging studies.

18.
Kinect v2
https://youtu.be/1jbvnk1T4vQ
Kinect detects user’s 3D shape and motion in real time.
Users simply move in front of Kinect. (Markerless!)
3D clothes are superimposed on a user.
Simple hand gestures allow user to choose outfits.

19.
 Leap Motion senses movement of hands and fingers.
 EMG sensor senses muscle activity of user (tighten or relax).
 Bullet is shot from tip of finger when hand is grasped tighten.
https://youtu.be/sLXA0Ngu7pw

20.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

21.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion
-> Echography

22.
Probe
Feature
 Echography is the medical
image diagnosis technique
using ultrasound.
 Image of internal organ is
obtained just by putting
probe on a patient’s body.
Difficulty
 Field of view is not wide.
 Anatomical knowledge is
required when physician
uses probe to get echogram.
 Training is very important.
Cross section

23.
We have proposed 2 kinds of AR systems which
can visualize probe operation of skilled physician.
(1) Recorded operation of a probe in previous.
(2) Real-time instruction from remote environment.
Web Camera
Probe
AR Interface
Rotate
Move

24.
Green plane is probe operation
recorded by skilled physician
https://youtu.be/Dpuc243qCzw

25.
Skilled Physician
Instruction via GUI
Patient’s Side
We confirmed unskilled operator could acquire good image
by referring instruction from remote environment.
https://youtu.be/WV99AmD-A0E

26.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion
-> Drawing Navigation

27.
Visualization of instruction has potential
not only for medical use but also educational use.
For example, AR navigation system of
drawing picture or writing text are useful.
Navigation

28.
Teacher records drawing scene as video in previous.
Student imitates the video superimposed on a paper.
(1)Slant view (LEFT) is transformed to top view (RIGHT).
(2)Drawing procedure of teacher is recorded as a video.

29.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

30.
Previous:
Programming skill was required to develop AR app.
Now:
Programming skill is NOT required because
some development tools enables us to create app.
Zappar

31.
① Please access to Google play or App Store
② Search Zappar and Install it into a smartphone
[Requirements]
• iPhone, iPad
(iOS 5.1.1~)
• Android
(2.3~)

32.
Demonstration

33.
• What is Augmented Reality (AR)?
• Introduction of AR Applications
• My Works - AR for Learning
- Medical Skill Training
- Education
• Development Tools
• Conclusion

34.
• AR is already available technology.
- with smartphone, pc, etc…
• Many kinds of Application is published.
• AR has potential for use of learning.
- Medical Operation
- Drawing Picture
• Creating AR content has became easy.