Invited talk at Espacios Educativos Especulares (Educational Reflected Spaces) EEE Anual Meeting 27-28 May 2013 - Universidad Carlos III de Madrid, Spain

1. Towards blended reality on
collaborative laboratory activities
using smart objects
School of Computer Science & Electronic Engineering
University of Essex, UK
Anasol Peñ a-Ríos
acpena@essex.ac.uk
1

2. • Introduction
• Blended learning landscapes
• Scenario: Collaborative laboratory activities
• Interreality environment (Conceptual model &
implementation)
• Blended reality distributed system
• Collaboration projects
• Summary
Overview of the presentation
2

3. About me
3
Anasol Peña-Ríos

4. Introduction
The Economist Intelligence Unit, “The future of higher education: How technology will shape learning,” The Economist, 2008.
6

5. Use of technology in academic settings
(Some challenges)
Promote adoption of new technologiesadoption of new technologies
by academic faculty members and students
Adapt technologies, pedagogicalAdapt technologies, pedagogical
methods and evaluationsmethods and evaluations enabling this
integrated solutions for people in
geographically dispersedgeographically dispersed locations
Change the way to experienceexperience
education from one-dimensionaleducation from one-dimensional
(physical) to multi-dimensionalmulti-dimensional (physical
and virtual) education in an integrated way
BlendedBlended
LearningLearning
LandscapesLandscapes
7

6. Blended learning landscapes
Virtual
Learning
Environmen
t
Blended
Learning
Environment
Real Learning
Environment
An infrastructure
that works as a link
between this
environments able to
reflect information
(data, changes, etc)
on real time
Learning activities that
complements virtual
and real world making
possible for students
and tutors to move
between these two in a
seamless way
8

7. Blended
Reality
This might be limited by the capacity of user’s presence and engagement to a single reality
at a time (Lifton's "Vacancy problem"1) a consequence of the user’s real immersion.
9
User's interaction in real-
time with two different
environments (real/local
and virtual/distant) extend
them to work as if they
were one, by blending
traces of one into the other
unconsciously (often
seemingly as simultaneously).
1. J. Lifton and J. Paradiso, “Dual Reality: Merging the Real and Virtual,” Lecture Notes of the Institute for Computer Sciences,
Social Informatics and Telecommunications Engineering, vol. 33, no. 1, pp. 12-28, 2010

8. Scenario (Work in progress)
COLLABORATIVE
LABORATORY ACTIVITIES
10

9. Laboratory activities for distance learners
Which are the options for remote laboratory activities?
• Simulations
• Virtual laboratories
• Remote laboratories
• To test conceptual knowledge
• To work collaboratively
• To interact with equipment
• To performing analysis on
experimental data
Laboratory
activities
11

10. Examples
University of Bristol - LabSkills
University of Leeds - Virtual Labs
Challenges for current technology:
•Limited interaction with real equipment
•Use of idealised datasets
•Restricted collaborative interaction
Freie Universitat Berlin –
Universit Technology Enhanced Textbook
12

11. Interreality Environment
1 - J. van Kokswijk, Hum@n: Telecoms and Internet as Interface to Interreality : a Search for Adaptive Technology and
Defining Users, Bergboek, 2003
A collection of interrelated devices in the real world, a virtual
environment and software agents that allow users to complete activities
at any point of Milgram’s Virtuality Continuum to achieve user
perception of integration between the physical and the virtual world.
13
Interreality.- “a hybrid total experience between reality
and virtuality”1

12. Interreality Portal (Definition)
In the educational context can be used:
• As a learning environment
• For geographically dispersed students
• Using mixed reality laboratory activities
14
Human-computer interface (HCI) which captures data
obtained in real-time, and processes this data so
it can be mirrored in the virtual world linking
both worlds.

13. Interreality Portal (implementation)
15

14. A "real" immersive
environment
•Gives to the learners the
sensation of "being there"
A 3D virtual environment
•3D GUI made on Unity3D, a
cross-platform game engine
used to create interactive 3D
content.
xReality objects
1- Immersive Display Group (www.immersivedisplay.co.uk)
2 - V. Callaghan, «Tales From a Pod» on Creative Science Workshop,
Nothingham, 2010.
1, 2
ImmersaStation
Interreality Portal (implementation)
16

15. 17
Smart objects
• coupled to their virtual
representation (dual
reality state)
• updated and maintained
in real time
1.- Kortuem, F. Kawsar, D. Fitton and V. Sundramoorthy, “Smart objects as building blocks for the Internet of
things,” IEEE Internet Computing, vol. 14, no. 1, pp. 44-51, 2010
Autonomous physical/digital objects
augmented with sensing, processing, and
network capabilities” which can
interpret their local situation and
status, and can communicate with other
smart objects and interact with human
users 1
+Xreality
objects
=Smart
objects

16. Interaction between physical and virtual
elements within an environment
• Each object, (real and virtual), is
complete by itself and can exist
without the other.
• Dual reality state enriches
both objects by the bi-directional
process that can reflect, influence
and merge real-time information.
MIT Media Laboratory – Dual reality
lab 2008
18
CrossReality (xReality)

17. xReality objects & virtual objects
• Real world objects that have
a virtual representation
• Dual reality state
xReality objects Virtual objects
• Only exist inside a virtual environment
• Rules and behaviours but these depend
on the existence of the virtual world.
19
Anasol Peñ a-Ríos, V. Callaghan, M. Gardner & M. J. Alhaddad, «Remote mixed reality collaborative laboratory activities: Learning
activities within the InterReality Portal»on Web Intelligence and Intelligent Agent Technology (WI-IAT), 2012 IEEE/WIC/ACM
International Conferences - The Intelligent Campus International Symposium (IC’12), vol.3, no., pp.362,366, 4-7, Macau, China, 2012.

18. xReality objects (Implementation)
* Fortito (www.fortito.mx)
20
**Raspberry Pi Foundation (www.raspberrypi.org)
Main module Components
• Fortito’s* Buzz-Board Educational Toolkit
• 30 pluggable boards that can
interconnect and communicate between
each others via I2C protocol
• Comprises different sensors and
actuators to allow the creation of diverse
physical mashups
• Raspberry Pi **
• Detects other components and
works as a hub to connect
them to the interreality system

19. Interreality Portal
•Buzz boards
http://youtu.be/wblqY-M0uKA?t=38s
•Interreality Portal
http://youtu.be/3oqdr7
nIG7Y
http://youtu.be/4AZw_
gcbAzw?hd=1
21

20. Implementation (architecture) (single dual
reality)
22
Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed-
reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.

21. Implementation (single dual reality)
23
Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed-
reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.

22. Multiple dual realities
24
• Management of multiple dual reality states
• Addition of a collaborative layer to allow two or more learners in
different geographical locations participate on the laboratory activity
+Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «End-user programming & deconstrutionalism for collaborative mixed
reality laboratory co-creative activities»on 2nd European Immersive Education Summit (EiED’12), Paris, France, 2012.

23. Blended reality distributed system
(Conceptual model)
25
Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed-
reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.

24. End-user programming
The interaction between users within collaborative mixed
reality learning activities can be analysed from two
different angles:
•From the learners view
•From the instructors view.
The learning environment should allow users to create
and execute learning activities regardless their expertise
on computers.
26

25. • Homogeneity & structure to
the creation of mixed reality
learning activities
• Adding collaborative
learning to the use of
xReality objects
Learner’
s view
Instructor’
s view
3D
Learning
environmen
t
End-user programming
27

26. Learning Activities
Learning activities within the InterReality Portal are
structured as a sequence of activities based on the IMS
Learning Design specification (Units of Learning-UoL).
CollaborativeCollaborative
interactioninteraction
betweenbetween
learnerslearners
EmphasiseEmphasise
ComputingComputing
fundamentalsfundamentals
Problem-Problem-
basedbased
learninglearning
VirtualVirtual
objectsobjects
(SW)(SW)
Internet-of-Internet-of-
ThingsThings
applicationsapplications
xRealityxReality
objectsobjects
(HW)(HW)
ComputerComputer
sciencescience
projectsprojects
28

27. Deconstructed model
• Based on the disaggregation of physical/logical
devices and services.
• Propose the creation/identification of a number of
elementary services (atomic functions) which can
be combined in various ways to create complex
functions (nuclear functions).
29

28. Deconstructionism
Role Atomic function Nuclear function
Learner Objects available in the
environment, actions
available (programming
statements).
An Internet-of-Things
project
Instructor Resources available in
the environment,
activities available
(sequence of activities).
A Unit of Learning
(UoL)
Technical Infrastructure Processes, threads,
processors or FPGA
xReality toolkit and
system
30
Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «End-user programming & deconstrutionalism for collaborative
mixed reality laboratory co-creative activities»on 2nd European Immersive Education Summit (EiED’12), Paris, France, 2012.

29. Implementation phases
31
Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Towards the Next Generation of Learning Environments: An
InterReality Learning Portal and Model»on 8th International Conference on Intelligent Environments 2012 (IE'12), Guanajuato,
Mexico, 2012.

30. Scale Up project
• A collaboration between King Abdulaziz University, KSA and
the University of Essex,UK.
32

31. Summary
• Core elements of our mixed reality activities
• Combine them using end-user programming
to create an educational mixed reality object
• Immersive mixed reality learning environment
• Grounded on collaborative, constructionalist,
problem-based learning theories
• Offers the possibility of creating collaborative
laboratory activities for geographically
dispersed students
InterReality
Portal
xReality &
Virtual objects
34

32. Challenges
Pedagogical challenge: a constructionist student-centred
method to create laboratory activities for distance
learners
• A representation of
a deconstructed
worldTechnical
challenges
• Learners
• Instructors
• A set of components that
can be shared and
combined
• Distribution of objects
between different immersive
environments
• Blended
reality
distributed
system
35

33. Challenges (2)
• To solve this we have proposed the use of
a deconstructionist architecture.
Physical model of
distributed xReality
objects in an
immersive learning
environment
Pedagogical
model of
constructionist
laboratory
activities
A solution forA solution for
distributeddistributed
mixed realitymixed reality
laboratorieslaboratories
36

34. Questions
•Thank you for your attention!
acpena@essex.ac.uk
acpena@ieee.org
37