Human computer interaction, interaction design and experience design have not yet digested the new changes brought about by post digital, hybrid techniques and methodologies. Combining current technologies with the practice of ancient pedagogies is beginning to create possibilities that go beyond the 'normal' repertoire of human cognition & ways of experiencing the-world. Context engineering (Smith, 2013) is an experience design practice that attempts to break the perceptual conventions that limit innovation within the social and knowledge construction agendas. Field Of View (FOV) technologies like the Oculus Rift http://www.oculusvr.com will not only alter the gaming landscape but will redefine fundamental human experiences in areas like film, education, architecture, and design. The FlyVIZ headset is another FOV technology http://www.slashgear.com/flyviz-headset-brings-360-degree-vision-to-the-party-17261196 that augments the sense of sight by giving the user 360-degree vision. The 360 view is compressed to fit into a human's usual 180 degree field of view, it takes fifteen minutes for the brain to adjust to this new way of seeing and ‘accept it as normal’. The eye now adjusts so readily to the effects of linear perspective that it is considered to be an innate process. The evolution of our perception alongside our architectural understanding is dictated by spatial conventions that have been built into our cultural constructs since the Renaissance. (Panofsky 1924) in his book ‘Perspective as a Symbolic Form’ established a parallel between the history of spatial representation and the evolution of abstract thought. Vision and optical experts (Radley 2013) note that it is rather curvilinear perspective that has a strong foundation in reality as the human visual field has a natural curvilinear shape. As a result abstract reasoning which is developed through the use of media tools and media content that are constructed using these ‘incorrect’ conventions can now be augmented and re-engineered with other more ‘realistic’ forms of spatial literacy. Likewise when the digital blends with the physical, in particular in relation to spatial perception, a hybrid reality appears and the wide and varied impact on consciousness now requires investigation. Context engineering is understood as an intermediality practise for exploring perceptual augmentation by giving us control over our senses, allowing us to adjust them in real time. Context engineering works by combining the affordances of analogue and digital technologies in order to provide fundamentally new ways of seeing and knowing.

1. Experience (Re)Design Techniques
using Innovative Mobile Assistive
Technologies and Creative
Context Engineering
Carl Smith
Director / Senior Research Fellow
Learning Technology Research Institute
LMU
London Metropolitan University
Learning Technology Research Institute

2. A major research theme at LTRI is designing
hybrid technologies and methodologies for the
creative industries and creative researchers.
Context
engineering (Smith, 2013) is an
experience design practice that attempts to break
the perceptual conventions that limit innovation
within the social and knowledge construction
agendas.
London Metropolitan University
Learning Technology Research Institute

3. Objective 1 of FP7 Challenge 8.1: ICT for
Creativity and Learning + Horizon 2020 Research priorities:
Potential of technology in human creative processes.
Progress towards a formal understanding of creativity with a
view to advancing the measurable capability of computers.
Improved efficiency of creative processes.
Roadmaps for future research and innovation in the creative
industries.
Improved competitive position of the European cultural and
creative industries - closer dialogue between research and
industry.
London Metropolitan University
Learning Technology Research Institute

4. CRe-AM Creativity Research Adaptive Roadmap.
CRe-AM aims to bridge communities of creators with
communities of technology providers, in a collective, strategic
intelligence/road mapping effort.
Developing, enhancing, and mainstreaming new ICT
technologies and tools by addressing the needs of different
sectors of the creative industries (e.g. art, culture, epublishing,
design etc.)
Ranked #1 in call
Kick off meeting in Nantes – will feed into Horizon 2020.
London Metropolitan University
Learning Technology Research Institute

5. FlyVIZ headset - 360-degree super(vision)
 Giving users the power
to see all around them
at once. What does this
mean?
 15 min for brain to
adjust. No nausea.
 The FlyVIZ can only
give you 360-degree
vision: you must choose
how to use it.
London Metropolitan University
Learning Technology Research Institute

6. Creating entirely new senses – Neil Harbisson can
hear colour with a prosthetic device.
London Metropolitan University
Learning Technology Research Institute

7. Artist Neil Harbisson is completely colour-blind
so he create a device to allow him to see colour.
 Simple device transforms light waves into sound waves using a
webcam, computer and a pair of headphones with software that
would translate any colour in front of me into a sound. £50
 At the beginning he experienced strong headaches because of the
constant input of sound, but after five weeks his brain adapted to it.
 “Changed the way I perceive art. Now I have created a completely
new world - currently working on seeing ultraviolet, which is very
important because it can damage our skin.”
London Metropolitan University
Learning Technology Research Institute

8. London Metropolitan University
Learning Technology Research Institute

9. “We shape our tools, and thereafter our
tools shape us” - Marshall McLuhan
London Metropolitan University
Learning Technology Research Institute

10. Ancient Technology with Modern Pedagogy
London Metropolitan University
Learning Technology Research Institute

11. Modern Technology with Ancient Pedagogy
London Metropolitan University
Learning Technology Research Institute

12. London Metropolitan University
Learning Technology Research Institute

13. Tools, Methodologies and Techniques that enhance
Collaboration, Creativity and Knowledge Construction.
1) Hybrid Reality / Intermediality.
2) Analogue and Digital Context Engineering.
3) Body hacking / Reality hacking.
London Metropolitan University
Learning Technology Research Institute

14. 1) Hybrid Reality - Deformscape
Architect Thom Faulders:
explores interfaces
between space,
perception, and context.
He situates the practice
of architecture within a
broader context of
performative research
and material
investigations that
negotiate dynamic
relationships between
users and environments.
London Metropolitan University
Learning Technology Research Institute

15. Intermediality - mvrdv cloud
http://www.designboom.com/architecture/mvrdv-the-cloud/
London Metropolitan University
Learning Technology Research Institute

16. Wiesflecker Architecture - Crinkled Wall
London Metropolitan University
Learning Technology Research Institute

17. Adrian Cheok. Professor of Pervasive
Computing: Multi-Sensory Internet
London Metropolitan University
Learning Technology Research Institute

18. PrioVR – Oculus + Body Sensors = Full
Body Immersion.
London Metropolitan University
Learning Technology Research Institute

19. 2) Context Engineering: made possible
through the datafication of everything
London Metropolitan University
Learning Technology Research Institute

20. Clay Vision
London Metropolitan University
Learning Technology Research Institute

21. Analogue and Digital Context Engineering
London Metropolitan University
Learning Technology Research Institute

22. A revolution in spatial literacy: Access to
new spaces: Implications for what can be
known
London Metropolitan University
Learning Technology Research Institute

23. Fractured View = No common ground /
(Cultural Heritage) = Reduction in Creativity
/ Knowledge Construction.
London Metropolitan University
Learning Technology Research Institute

24. London Metropolitan University
Learning Technology Research Institute

25. Re-enabling focus in music listening
-slowlistening.com
A pair of
headphones
that force its
wearer to focus
on the music by
dropping the
volume if they
move.
London Metropolitan University
Learning Technology Research Institute

26. 18 months
“We decided we
had to give them
the whole album
for free – but with
a catch. You had
to be dancing to
the music in order
to listen to it all.”
London Metropolitan University
Learning Technology Research Institute

27. Context Engineering - Conversation Spacification
Same Height Parties :
artist Hans Hemmert
explores the notion of
how first impressions are
made, as any height
related impressions are
subverted. Usually when
talking to people it’s
suggested you approach
them at their level. Eye
contact is constantly
recommended as a way
to bond with people.
London Metropolitan University
Learning Technology Research Institute

28. Context Engineering
Innespace
Productions
have created a
series of unique
dolphin-inspired
submersible
boats that can
jump, dive and
roll like real
dolphins.
London Metropolitan University
Learning Technology Research Institute

29. London Metropolitan University
Learning Technology Research Institute

30. 3) Body hacking / Reality Hacking – Eidos Outcomes
Context engineering: What if we had the same control over our senses – if we could adjust them in
realtime? What new experiences could this make possible.
London Metropolitan University
Learning Technology Research Institute

31. Lorenz Potthast's Decelerator helmet
"Decelerator," the avant
garde piece of headgear
does just that: it has a
camera that feeds video to
the head-mounted display
inside, with the wearer (or
someone else) able to
control the speed of the
video playback with a
remote.
London Metropolitan University
Learning Technology Research Institute

32. Perceptual Augmentation Devices
Exploring perceptual
expansion through
sensory substitution
and augmentation.
The function of these
devices is to expand
the body's senses.
London Metropolitan University
Learning Technology Research Institute

33. Plug-in architecture - Walter Pichler
Absolute Architecture: For Pichler and
Hollein, architecture was not what it
enables, nor what in encloses, but what
it is.
London Metropolitan University
Learning Technology Research Institute

34. Flyhead-environment-transformer
Haus-Rucker-Co were
exploring on the one hand,
the potential of architecture
as a form of critique, and on
the other the possibility of
creating designs for
technically mediated
experimental environments
and utopian cities.
London Metropolitan University
Learning Technology Research Institute

35. HYUNGKOO LEE – Objectuals
Re-imagining
what already
exists and using
perspective to
shape reality
instead of
changing reality
itself is
something that
intrigues me.
London Metropolitan University
Learning Technology Research Institute

36. Tetris video game fixes lazy eye
The McGill University team
discovered the popular tilematching puzzle could train
both eyes to work together.
Works better than
conventional patching of
the good eye to make the
weak one work harder.
www.bbc.co.uk/news/health22245620
London Metropolitan University
Learning Technology Research Institute

37. FOV technologies- Oculus Rift
London Metropolitan University
Learning Technology Research Institute

38. Apply different types of perspective / lenses to break
learnt conventions by reprogramming vision
London Metropolitan University
Learning Technology Research Institute

39. Avegant's Virtual Retinal Display
London Metropolitan University
Learning Technology Research Institute

40. Body Hacking / Reality Hacking
Karolinska Institute - Ehrsson Body Illusions :
Seeing through another POV
London Metropolitan University
Learning Technology Research Institute

41. London Metropolitan University
Learning Technology Research Institute

42. Mirror Neurons: Implications from Neuroscience
London Metropolitan University
Learning Technology Research Institute

43. Oculus – Among the Sleep
Among the Sleep, where
players experience the
world through the eyes of
a two-year-old child.
Things seem bigger —
you'll need to look up to see
a doorknob, for instance —
and all you can really do is
stumble around like a
toddler, walking or crawling
away from danger.
London Metropolitan University
Learning Technology Research Institute

44. Julius-Von-Bismarck - Topshot-Helmet –
Inducing OBE
London Metropolitan University
Learning Technology Research Institute

45. Olaf Blanke – Neuromodulation via Luis Girae
London Metropolitan University
Learning Technology Research Institute

46. London Metropolitan University
Learning Technology Research Institute

47. London Metropolitan University
Learning Technology Research Institute

48. Macroscopic Visualisation
• The macroscope effectively
provides the overview and the
local point of view of the
research object simultaneously.
• Within one field of view, to be
both in the world and to see
yourself in it. The power of
looking through, and occupying,
your own field of vision.
• We can see through satellites
now.
London Metropolitan University
Learning Technology Research Institute

49. Augmenting the understanding of an art work
Looking at Jackson
Pollock whilst listening to
jazz that Pollock himself
was listening to when he
made the artwork deeply
effected the viewers
understanding of the
artwork.
Enrich the content itself
with the context (of how
the original artwork was
created)
London Metropolitan University
Learning Technology Research Institute

50. 3D Movie ‘Hugo’ Cured Neuroscientist’s
Stereoblindness
Bruce Mcnaughton had lived
his entire life “stereoblind”,
which meant he was unable to
perceive depth correctly.
This new effect stayed with
him after he walked out of the
theatre, because unlike most
regular moviegoers, Bruce had
been unable to perceive
everyday objects with accurate
depth.
London Metropolitan University
Learning Technology Research Institute

51. Weak Signal: Importance of peripheral vision.
Removing peripheral vision is a reliable way of
hacking the brain.
Scientists have found that there is a neurotransmitter
in the peripheral nervous system, that is crucial to
focus and memory.
This is a problem with augmented reality which
tends to focus on only what is directly in front of you.
London Metropolitan University
Learning Technology Research Institute

52. Summary
Focus on feeling rather than creating more cognitive
dissonance = not living in the moment is leading to diminished
reality.
Take advantage of the plasticity of brain and the plasticity of
the senses.
Learn more about human perception and the interaction
between different sense modalities.
Fundamental problem is that we think of the technological
solution before defining the human need.
New economy coming out of context augmentation rather than
content augmentation.
London Metropolitan University
Learning Technology Research Institute

53. Challenges: Stop the Cyborgs are concerned with the
ethics of being always on and always augmented.
London Metropolitan University
Learning Technology Research Institute

54. London Metropolitan University
Learning Technology Research Institute

55. London Metropolitan University
Learning Technology Research Institute

56. The McCollough effect: a phenomenon of
visual perception.
London Metropolitan University
Learning Technology Research Institute

57. Experience design: Moving from ‘Information
Communication’ to ‘Experience Communication’
London Metropolitan University
Learning Technology Research Institute

58. Summary
Post-digital/hybrid reality design uses these tools to
humanise technology via social and cultural applications.
The combination of hybrid reality tools and context
engineering practises are fundamentally changing the way
we interact, allowing us to access new ways of seeing and
knowing.
Fluidity of thinking relates to fluidity of movement (Maher)
Design problem: Replacing imagination with
graphics/visualisation?
London Metropolitan University
Learning Technology Research Institute

59. Email:
carl.smith@londonmet.ac.uk
Facebook:
facebook.com/behindthebeats
Twitter: behindthebeats
http://www.makototojiki.com/
London Metropolitan University
Learning Technology Research Institute