SIXTH SENSE TECHNOLOGY

1. SIXTH SENSE
TECHNOLOGY
GUIDED BY:
Prof. JISNA BASTINE
PRESENTED BY:
JISMI K JACOB
S8 IT 18
3/27/2014
1JECC IT Department

2. CONTENT
• Introduction
• History
• Components
• Applications
• Working Principle
• Related Technologies
• Advantages And Disadvantages
• Future Enhancement
• Conclusion
• References
3/27/2014 2JECC IT Department

3. INTRODUCTION
• Sixth Sense is a wearable gestural interface.
• Sixth Sense is a gesture-driven computing platform.
• It augments the physical world with digital information.
• It bridges the gap by bringing intangible, digital
information out into the tangible world.
• Device coupled with a projector, camera and a phone.
• All the information stored on the web.
3/27/2014 3JECC IT Department

4. WHY SIXTH SENSE?
Sixth Sense prototype implements several applications that
demonstrate the usefulness, viability and flexibility of the
system.
3/27/2014 4JECC IT Department
Sixth Sense Technology - connecting the digital world
with the physical world seamlessly, eliminating hardware
devices.

5. HISTORY
• Steve Mann is the father of
Sixth Sense Technology who made
of wearable computer in 1990.
• He implemented the neck worn
projector with a camera system.
• It was implemented by Pranav
Mistry, a PhD student in the Fluid
Interfaces Group at the MIT Media
Lab.
3/27/2014 JECC IT Department 5

6. COMPONENTS
• Camera
• Projector
• Mirror
• Smart Phone
• Color Marker
• Microphone
3/27/2014 JECC IT Department 6

7. CAMERA
• Input device of system.
• It acts as a digital eye of the system.
• It captures the scene the user is looking at.
• Captured data is passed to mobile computing device.
• Software program processes the video stream data captured
by the camera.
• It tracks the locations of the colored markers at fingers.
3/27/2014 JECC IT Department 7

8. PROJECTOR
• Output device of the Sixth Sense system.
• Mobile device provides the data to be projected.
• Projector unit runs on a rechargeable battery.
• It have a tiny LED projector.
• GUI of the selected application, projects onto any surfaces.
• It projects related information of user interacts with
from the Internet just-in-time.
3/27/2014 JECC IT Department 8

9. MIRROR
• Mirror reflects data from the projector.
• Mirror reflects image on to the desired surface.
• Mirror dangles pointing downwards from the neck.
• User manually can tilt the mirror to change the location
of the projection.
• Mirror overcoming the limitation of the limited projection
space of the projector.
3/27/2014 JECC IT Department 9

10. MOBILE COMPUTING DEVICE
• Processing device carried in user’s pocket.
• Software program runs on this device.
• Camera, projector and the microphone are
connected to the device using wired/wireless connection.
• Mobile device given data in turn to the projector.
• Mobile computing device is also connected to the Internet.
• A Web-enabled smart phone in the user’s pocket processes
the video data.
3/27/2014 JECC IT Department 10

11. COLOR MARKER
• Marker placed at the tip of the user’s fingers.
• Red, green, blue and yellow colored tapes
helps webcam to recognize the hand gestures.
• Movements are interpreted as gestures, that acts as the
instruction for the projected application interface.
• Camera tracks the locations of the colored markers.
• Supports multi-touch and multi-user interaction.
3/27/2014 JECC IT Department 11

12. MICROPHONE
• Optional component of the Sixth Sense.
• Required when using a paper as a interactive surface.
• User clips the microphone to the paper captures the sound
signals of user’s touching the paper.
• Data passed to mobile computing device.
• Sound signal captured by the microphone provides time
information.
3/27/2014 JECC IT Department 12

13. APPLICATION
1. Make A Call
2. Call Up A Map
3. Check The Time
4. Get Flight Updates
5. Drawing Application
6. Zooming Features
7. Check The EMAIL
8. Take Pictures
9. Get Book Information
10. Get Product Information
11. Feed Information On People
12. Create Multimedia Reading Experiences
3/27/2014 JECC IT Department 13

14. 3/27/2014 JECC IT Department 14
1. Make A Call
• To project a keypad onto your
hand.
• Use that virtual keypad to make
a call.
2. Call Up A Map
• we can call up the map of our
choice.
• Use thumbs and index fingers
to navigate the map.

15. 3/27/2014 JECC IT Department 15
3. Check The Time
• Draw a circle on your wrist.
• Get a virtual watch that gives
the correct time.
4. Create Multimedia
Reading Experiences
• Programmed to project related
videos onto newspaper articles
you are reading.

16. 3/27/2014 JECC IT Department 16
5. Get Flight Updates
• System will recognize your
boarding pass.
• Display real time information
about your flight over ticket.
6. Feed Information On
People
• Display relevant information
about a person.

17. 3/27/2014 JECC IT Department 17
7. Drawing Application
• Tracking the fingertip
movements.
• User draw on any surface.
8. Zooming Features
• User can zoom in or zoom
out using intuitive hand
movements.

18. 3/27/2014 JECC IT Department 18
9. Get Production Formation
• Sixth Sense recognizes
products we pick up, then
feeds us information on those
products.
10. Get Book Information
• The system can project Amazon
ratings on that book.
• Gives relevant information.

19. 3/27/2014 JECC IT Department 19
11. Take Pictures
• Creating a picture frame tells
the camera to snap a photo.
• Desired photos can project
them onto a surface.
12. Check The EMAIL
• Drawing an @ symbol in the air
allows you to check your email.

20. WORKING PRINCIPLE
3/27/2014 JECC IT Department 20
Images
Pictures
Gestures
Info On
Surface
Working Of The Sixth Sense Device
captureSend for
processing
Send info
Project image
Reflect onto
desired location

21. •Work like mobile wearable interface.
• Camera, mirror and projector connected to a Bluetooth
smart phone in user’s pocket.
• Camera captures hand gestures and sent to the Smartphone.
• Projector projects the output image on to any surface.
• Mirror dangles near to the projector, reflects image onto the
desired surface.
• Digital information is placed in the physical world.
3/27/2014 JECC IT Department 21
Contd…..

22. RELATED TECHNOLOGIES
1. Gesture Recognition
2. Computer Vision
3. Radio Frequency Identification
4. Augmented Reality
3/27/2014 JECC IT Department 22

23. 1. Gesture Recognition
• Capture hand gesture recognition.
• Humans interact with device without
using any external interfacing devices.
• Colored bands are used for hand pose tracking.
• Neural network approaches or statistical templates are the
commonly used techniques.
3/27/2014 JECC IT Department 23

24. 2. Computer Vision
•Extract information from an image.
•It basically includes four processes :
•Recognition: Object have useful data or not.
•Motion Analysis: Estimation of motion where an image
sequence is processed continuously.
•Scene Reconstruction: Recreate a 3D image from the image.
•Image Restoration: To remove noise from an given image
using filters to get quality image.
3/27/2014 JECC IT Department 24

25. 3. Radio Frequency Identification
• Transmit identity of an object using tag.
• Reader is needed to retrieve data from tag.
• Reader passes the information in digital form.
• Reader collision -occur when signals from two or more
readers overlap.
• Tag collision -occurs when many tags are present in a
small area.
3/27/2014 JECC IT Department 25

26. 4. Augmented Reality
• Virtual experience added over real world.
• AR superimpose graphics for every perception.
• Adds graphics, sounds, haptic feedback and
smell to the natural world as it exists.
Three components of AR :
1.Head mounted Display-View superimposed graphics created
2.Tracking System-Tracks the user’s eye and head movements.
3.Mobile Computer- computing device that fits in user’s hand.
3/27/2014 JECC IT Department 26

27. ADVANTAGES DISADVANTAGES
• Portable • Hardware limitations of the devices
• Support Multi touch and
Multi user interaction
• Many phones will not allow the
external camera feed in real time.
• Saves time spent on
searching information.
• Post processing can occur however.
• Mind map the idea anywhere • Failure of any components makes
functioning difficult.
• Recognizes user’s freehand
gestures and saves electricity.
• It is difficult to wear the interface
every time.
• Cost Effective • Number of hardware components.
Sixth Sense Technology

28. FUTURE ENHANCEMENTS
Imagine the world where Sixth Sense Technology is
applied everywhere.
• Elimination of colored markers.
• 3D gesture tracking and assured security.
• Sixth Sense work for disabled person.
• Use of contact lens and laser projectors.
• It could be helpful in rendering defense services.
3/27/2014 JECC IT Department 28

29. CONCLUSION
• Sixth sense helps us recognize the world around us.
• It makes the applications user-friendly and portable too.
• It helps in displaying the information automatically.
• Simple hand gestures can help to bring the world close to.
• It keeps us technologically awake and connects us with
the world via this newest invention.
• Sixth sense could make the world magical.
3/27/2014 JECC IT Department 29

30. REFERENCES
[1] P. Mistry, P. Maes. Sixth Sense – “A Wearable Gestural
Interface”, In SIGGRAPH Asia 2009.
[2] P. Mistry. “The thrilling potential of Sixth Sense
technology”, TED India 2009. Mysore, India.
[3] Rao, S.S.. “Sixth sense technology”, Communication and
Computational Intelligence (INCOCCI), on 2010.
[4] S.Pradeep , Pandithurai. “Sixth sense technology”, In
Proceedings of the International Conference on 2010.
3/27/2014 JECC IT Department 30