What is known about human stereopsis and how does it impact VR design? Ron Herschel's presentation reviews what appears to be the nonmaterial aspects of human vision such as perception, imagination, memory, and comprehension. Gain insights from his speculations on the minimum space-time requirements for VR such as viewer motion (eye tracking), resolution, and parallax. His presentation covers gaming VR display and other commercial applications including remote control robotic surgery. Presented as part of the VR-AR Track at Power of Play 2017 hosted by the Washington Interactive Network.

1. 10TH ANNUAL VIDEO GAME AND INTERACTIVE ENTERTAINMENT CONFERENCE
Presented by and
Ron Hershel
President
Hershel Consulting
Human Vision and
Virtual Reality

2. Human Vision and Virtual Reality
Ron Hershel
Hershel Consulting
Corvallis, OR
ronh9@comcast.net

3. The Challenge of VR
 The goal of VR is to reproduce a visual scene to the viewer
that is visually indistinguishable from reality
 Without shortcuts, that goal is extremely challenging:
110o FOV VR retinal display (1’ acuity) = 34Mb/frame
x 30 frames/sec x 2 eyes
= 2Gb/sec which is 50X HD Blu-ray (40Mb/sec)

4.  Resolution peaks at 1 arcmin or better near the center
of vision and drops rapidly towards the periphery
 A visual scene is created by a sequence of saccades or
rapid eye movements between fixation points (FP)
where vision is masked during each saccade trajectory
Shortcuts from Vision Science
 Accommodation insures the best visual acuity at each
FP while giving little information about depth
 Eye tracking and RAD compression offer a 1000:1 bitrate
reduction w/o sacrificing quality
 Saccades are reduced to fading out of the last RAD while
fading in the new
 A fixed focal plane or surface can be used in VR displays
 Human vision is very sensitive to parallax Eye tracking should locate pupil position to within 0.2mm

5. Retinal Acuity Distribution, RAD
Acuity peaks at the center of
vision (0) and drops off to 50%
at a 0.5 radius. Acuity then
declines rapidly to 1% at 50.
linear
scale
0.05 0.5 5 50
Radius
RetinalAcuity
1.0
0.1
0.01
0.5
1.0
0.5
0
0 25 50

6. RAD Compression
1000
800
600
400
30 50 70 90 110
FOV (degrees)
CompressionRatio
Performing RAD compression,
ETVR can realize nearly a 1000:1
bitrate reduction compared to a
110 FOV VR without eye
tracking.

7. FR1
FR2
FR3
Visual imagery, is constructed from a sequence
of “snapshots” in space and time, obtained by
rapid eye movements between successive FP’s,
called saccades whose trajectories are ballistic
(no mid-course corrections).
Microsaccades correct trajectory errors as well
as perform other vital functions and occur within
a 0.5 radius, called the fixation region (FR).
Saccades

8. RAD1
When the saccade velocity exceeds a few
degrees per second, vision is masked from
the brain. You can prove this to yourself.
Look into a mirror and alternate your
focus between the left and right pupils.
You will notice no movement of your eyes!
RAD2
RAD3
Saccade Masking
In ETVR, saccadic motion is reduced to fading
out the last RAD and fading in the next.

9. Eye Tracking Virtual Reality, ETVR
 In summary, ETVR provides authentic VR experience at very
low bitrates, using shortcuts provided by vision science.
 Uses a display that tracks viewer fixation regions, coupled
to retinal acuity compression algorithms, to reduce bitrates
1000:1 as compared to VR without eye tracking.
 The adaptive display relies on high speed (1kHz) eye tracking
to accurately predict saccadic trajectories with the potential
to sizably reduce visual latency.
 When other shortcuts are employed, bitrates are further
reduced to < 512Kb/sec for high quality FOV 110 VR

10. Data Compression
1 VR User
Server
w/o ET
4000 VR Users
Server
ETVR
Download

11. Where Do We Go From Here?
 To discover new applications of ETVR to enhance
visual interaction in work, play, and science.
 To explore saccadic patterns that can be linked to a
variety of health conditions, including vision.
 To further advances in remote controlled robotic tools
to support remote surgical procedures, HASMAT and
defense applications, and modern manufacturing.

12. And to unfold the secrets of vision…
Memory, imagination, thought, and comprehension
in vision require astronomical speed and memory
and hence must occur in another reality.
Thank You