Immersive technologies and HCI Innovation

1. Immersive Technologies and HCI Innovation
The Impact on the Knowledge Professional
David Wortley FRSA
Senior Research Fellow
Immersive Technology Strategies
Leicester, United Kingdom
david@davidwortley.com
Abstract—This paper describes the trends in Human Computer
Interface Innovation driven by the immersive technologies of II. HUMAN INTELLIGENCE AND MACHINES
video games, virtual worlds and social networks and questions
the potential impact of these innovations on society in general A. Historical Perspective
and the knowledge professional in particular. In the early days of any new machine or technology
designed and built by humans, the operation of that machine to
Keywords-component; immersive technologies, serious games, optimum performance required operator skill, experience,
interfaces, personalisation, knowledge
understanding and intelligence. To a large extent, the skill and
experience of the human operator compensated for deficiencies
I. INTRODUCTION in the man-machine interface. In almost every sphere of human
Immersive Technologies such as serious games, virtual activity, there has been a remarkable transformation in the level
worlds and social networks are a very influential driver for of operator skill, intelligence and knowledge required to make
innovation because of their success in engaging the effective use of machines and technology. These are just a few
discretionary time, attention and income of their users. This examples.
investment in time, attention and money translates itself into
the commercial value of research and development of devices B. Transport Technologies
and applications which better engage technology users and/or
make products and services accessible to the widest possible
consumer market. These devices and applications are shaping
our use of technology in ways which are developing highly
personalised relationships between users and various digital
devices such that the devices themselves provide a unique
interface experience to the user which increasingly requires
lower levels of skill and knowledge to access because the
domain knowledge is embedded in the device and/or
application.
These new types of relationships between humans and
computers are maturing rapidly towards a more human-human
and/or human-deity relationship rather than a man-machine
relationship based on ergonomics and ease of use. In practice
this means that devices and applications need less and less
skill, experience and knowledge so that where previously British Rail Steam Locomotive Oliver Cromwell
humans used their intelligence to get the most from machines,
Transport technologies are one example of how humankind
machines and technology increasingly use their intelligence to
has sought to break down the barriers of time, space and state
get the best from human beings. This paper and presentation to enable people to travel further and faster. Where once
will illustrate by example some of the recent development in humans harnessed the power of animals to achieve this goal,
HCI being shaped by innovations in immersive technologies with the industrial revolution and steam power, mankind could
which combine very natural, almost invisible interfaces with operate a machine to transport people and goods to new places
artificial intelligence, image processing, sensor devices and at faster speeds. The early steam engines required a whole
user profiling / personalisation tools. spectrum of human senses, intelligence and experience to
operate efficiently and the necessary skills and/or knowledge
would take years of training and practice to acquire. Without

2. this skill, experience and knowledge the transport systems of The invention of devices like Disk Fax is a good example
yesteryear would not function at all. of ergonomics and technological innovation embedding human
knowledge and intelligence into a machine and thereby
Today, most of this skill and intelligence has been deskilling a process and/or removing the need for human
embedded in the machines themselves so that many modern intelligence. What human intelligence is required then becomes
trains do not even require a driver, planes fly long distances on the domain of the small number of individuals responsible for
autopilot and our cars and lorries require lesser amounts of skill inventing and developing the device.
and intelligence to operate and this trend will continue
From a societal perspective, this potentially creates an
C. Telecommunications Technologies unprecedented environment in which knowledge is not
disseminated and replicated globally but is concentrated in a
The operation of a telephone from a user perspective has
handful of individuals. Because this concentration of
always been a relatively straightforward affair that could
knowledge and power is a historical quantum leap and
easily be picked up by a novice user but the technology and dislocation from the past, its potential impact on society is
infrastructure to connect people and information together unpredictable.
across long distances has followed the same pattern as
transport technology. Data communications in the early digital
age required a substantial amount of skill and knowledge to be D. Computing Technologies
able to effectively and reliably achieve a connection that could
transfer digital data from one device to another
III. PREPARE YOUR PAPER BEFORE STYLING
IBM 5120 Desktop Computer
It is difficult to imagine how much computing technologies
have changed in the last thirty years, not just in their price
performance, but also in the interface and relationship with end
users. In 1982, IBM were selling desktop business computers
like the IBM 5120 with its 64k memory and its 2.4 MB floppy
disk storage for over £5,000 (equivalent to £12,500 today) to
small businesses and offering what was described as a next
Disk Fax File Transfer System generation programming language called Business Report
Application Development System (BRADS) that enabled non-
Computer to computer communications using data modems programmer business people to create their own applications.
required both a knowledge of data communications software These systems used a keyboard text interface with a
and of the modem devices themselves to be able to effectively monochrome display and no graphics or networking
programme the data communications link. The Disk Fax shown capabilities.
above is a prime example of how an innovation in an interface
device can totally transform not only the speed and reliability Today, tablet computers like the iPad with 64GB memory
of a function, but also totally remove any requirement for can be purchased for around £600 with highly intuitive
operator skill or knowledge. graphical interfaces and access to a global wealth of free or
very low cost applications and powerful connectivity.
To transfer files between locations using a Disk Fax simply
required the operator to insert a data disk, type in the telephone The latest generation of these devices has very powerful
number of the remote Disk Fax and press the send button. The personalization capabilities which enable the ordinary user to
process became totally automatic and this technology only tailor their own completely bespoke system that allows them to
became obsolete when the computer files sizes to transfer high connect and communicate on a global basis. This
quality images and video were beyond the capacity of floppy personalization, combined with artificial intelligence has a
disks. profound impact on the knowledge professional.

3. III THE ROLE OF VIDEO GAMES AND INTERFACE IV IMMERSIVE TECHNOLOGIES, PERSONALISATION
TECHNOLOGIES AND HCI
The global video games market size was estimated to be In parallel with the innovations in video games, virtual
$60.4 billion in 2009 and is forecast to rise to over $70 billion worlds and social networking applications have also been
by 2015. Video games generate more revenue than either the driving innovations that have impacted interface technologies.
movie industry or the music industry and because of their The success of the whole spectrum of immersive technologies
commercial value, the market for video games is highly evidenced by the number of users of virtual worlds like World
competitive and commands substantial investment in of Warcraft and Second Life and subscribers to social media
emerging technologies which provide competitive advantage sites like Facebook, MySpace, Twitter and Youtube is in no
to the developers. small way attributable to the ability of these applications to be
personalised by the user. This personalisation gives the user
One of the most fruitful Research and Development areas control of how they are represented in these virtual spaces
for such competitive advantage is in the design of new either through their personal profile or the design of the avatar
interface technologies which can broaden the market potential alter ego.
and/or introduce engaging and innovative ways of interacting
with the game environment. One of the best examples of such This control of what is effectively the user’s interface with
innovation is the Nintendo Wii wireless controller and motion the application, with its ability to control not only the content
tracking device. This new form of games console interface that is presented to the user but also the distribution and
with its intuitive interaction really opened the doors to dissemination of content to a potentially global audience,
generations of new users of all ages because of its ease of use introduces a new dimension to HCI – Personal Relationships.
and instant on-screen feedback.
Personal Relationships with computing technologies are a
The Nintendo Wii was a breakthrough in interface fast growing development area for HCI applications.
technologies that was quickly followed by devices like the Computing in the areas of video games, virtual worlds and
Playstation Muve controller and the Microsoft Xbox Kinect social networking has a strong emphasis on technology
device, both of which use three dimensional motion tracking developments which recognise the user and dynamically shape
to control game activities. the interface to the user according to the individual’s
preferences, capabilities and interests. In this way, the
technology is learning to mimic the nature of human to human
relationships in which our behaviour evolves and modifies
over time as we get to know people.
Through devices like the Mirosoft Kinect with its voice
and facial recognition technologies, as well as other proactive
biometric security technologies, computing devices are getting
better at being able to identify an individual and therefore self-
configure according to the knowledge of that individual. In
addition, once a user is “recognised” and logged-on, other
background artificial intelligence technologies begin to
monitor the way the individual uses the technology in order to
shape the interface and content to the individual.
This phenomenon of personalisation begins to manifest
itself in areas such as targeted advertising based on user
profiles and browsing patters but will extend into many other
areas such as learning and assessment in which the computer
plays the role of a virtual coach or mentor.
Mindwave brainwave controller V. CONCLUSION
In addition to the interface devices which operate by The technologies described in this article signal a reversal
motion tracking, major advances are being made in the use of in the nature on man-machine relationships. For all of
brainwave monitoring technologies and linking these to the mankind’s history, human beings have needed to use their
control of video games and other applications. These examples skill, intelligence, experiences and senses to control machines
provide evidence that market driven innovation within the and technologies. Knowledge professionals had a premium
games industry can be very influential in the development of role in society to not only refine and advance human
more intuitive and user friendly human computer interfaces capabilities, but also to transfer the knowledge to future
which combine sensor and motion tracking technologies with generations. Today the same skills, intelligence, experience
either handheld devices embedded with positional technology and senses are being embedded into devices and the
or human body recognition technologies. infrastructure that supports them. In this process, mankind’s
dependence on machine intelligence will continue to grow and
its dependence on the human knowledge professional reduce.