This document discusses the relationship between computational fields and augmented reality. It begins by defining computational fields as a programming model that views distributed devices as approximating a continuous computational environment. It then provides an overview of augmented reality and its basic concepts. The document proposes that computational fields and augmented reality can be combined by using augmented reality as a visual interface for computational fields, allowing fields to be visualized and input to be provided via augmented reality interactions. It presents this relationship from both directions, with computational fields also serving as an enabling technology for augmented reality applications by supporting virtual entities that exist across physical space. An example of aggregate triage is briefly described to demonstrate this combined approach.
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
Computational Fields meet Augmented Reality: Perspectives and Challenges
1. Computational Fields meet Augmented Reality:
Perspectives and Challenges
Danilo Pianini, Angelo Croatti, Mirko Viroli, Alessandro Ricci
{danilo.pianini, a.croatti, mirko.viroli, a.ricci}@unibo.it
Alma Mater Studiorum—Universit`a di Bologna a Cesena
Spatial and COllective PErvasive Computing Systems (SCOPES)
September 21, 2015 - Cambridge, USA
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 1 / 25
2. Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 2 / 25
3. Introduction Motivation
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 3 / 25
4. Introduction Motivation
Why Augmented Reality and Computational Fields?
Differences
They are definitely different entities
CF is a programming abstraction. AR is a mean of interaction.
CF focuses on collectivity, AR (mostly) on the single user.
Commonalities
They share a common context of application
Both are devoted at environments pervaded with computational
devices
Both are bound to the physical world
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 3 / 25
5. Augmented Reality Basics
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 4 / 25
6. Augmented Reality Basics
Definition
Augmented Reality
is a technology that allows the user to see the real world, with virtual
objects superimposed upon or mixed with the real world [Azu97].
The ultimate goal is a seamless integration of reality and virtuality
Mostly via see-through devices
Images from Microsoft Hololens presentation video
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 4 / 25
7. Augmented Reality Basics
Degrees of augmentation
GHOST
AGENT
GHOST
BODY STREET
LAMP
PLAYER
BODY
PLAYER
ASSISTANT
AGENT
MIRROR WORLD
PHYSICAL WORLD
8
Augmentation can happen at different levels:
1 Unrelated to the elements in the user’s FOV
E.g. displaying a map on a visual overlay
2 Dynamically associated to elements in the user’s FOV
E.g. displaying historical information next to a monument in FOV
3 Entirely virtual and interactive elements on real world
E.g. the two images of the previous slide, the ghost game [RPTC15]
Image on left: DashWare. Image on right from [RPTC15]
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 5 / 25
8. Computational fields Introduction to aggregate programming and computational fields
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 6 / 25
9. Computational fields Introduction to aggregate programming and computational fields
Manifesto of aggregate computing
Main observation
In a world with ever increasing number of deployed devices, one
conveniently views a dense aggregation of interacting devices as a discrete
approximation of the space of computational environment through which
they are distributed.
neighborhood
device
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 6 / 25
10. Computational fields Introduction to aggregate programming and computational fields
Manifesto of aggregate computing
Main observation
1 the reference “machine” for pervasive computing processes is
abstracted to be the continuum of computational devices;
2 the reference “elaboration process” is the manipulation of a physically
distributed data structure;
3 how computation is carried on by the cooperation of devices in that
region is hidden “under-the-hood” of the model/platform
neighborhood
device
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 7 / 25
11. Computational fields Introduction to aggregate programming and computational fields
Pervasive continuum and computational fields
Aggregate computing
From “what value the single device computes”...
...to “what distributed data structure the pervasive fabric computes”
The notion of computational field arises [MZ09, BB06]
⇒ a map from the space to (structured) values
typically evolving over time, possibly self-stabilising
better “understood” on continuous domains
neighborhood
device
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 8 / 25
12. Computational fields Languages for aggregate computing
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 9 / 25
13. Computational fields Languages for aggregate computing
Existing languages
MIT Proto [BB06] is the most known and successful
Developed at MIT and maintained at BBN Technologies
Functional language, LISP-like syntax (I know you hate it too)
All devices run the same program
Computation happens in rounds:
Every device sleeps for some time
Processes the messages received from the neighbours
Executes its program
Sends all the neighbours its result
Complex operational semantics
Difficult to maintain and extend
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 9 / 25
14. Computational fields Languages for aggregate computing
Field Calculus
A “distillate” of Proto
Provides a lightweight operational semantics [VDB13]
((((LISP-like syntax))))
Still a functional language
Simple enough to formally prove properties, powerful enough to be
universal (proved!)
Theoretical object, no runtime nor simulation tool provided
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 10 / 25
15. Computational fields Languages for aggregate computing
Protelis [PVB15]
Ordinary language features
Functional language
Same operational semantics of the field calculus
C-family syntax with infix operators
Java interoperability: static methods imports and calls, method
invocation with dynamic binding
Higher order functions (functions as arguments, lambdas)
Dynamic code
Suggested read [BPV15]
Jacob Beal, Danilo Pianini, and Mirko Viroli. Aggregate programming for
the internet of things.
IEEE Computer, 48(9), 2015
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 11 / 25
17. Computational fields Languages for aggregate computing
Architecture
Implementation, distribution
Based on Xtext
Eclipse plugin
Now distributed through Maven Central a
Integrated with Alchemist [PMV13]
Easy to port to new platforms
a
artifact: org.protelis:protelis
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 13 / 25
18. Augmented Fields Augmented reality as visual interface for fields
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 14 / 25
19. Augmented Fields Augmented reality as visual interface for fields
See the fields
Inject the ability to use AR to see the ongoing CF computation
How
Read the program output, display it properly
Requires basically no intervention on the CF program
Why
The mapped field may be directly mapped to the user (e.g. a crowd
warning system)
Useful for seeing the shape of a field while developing
Limitations
Some fields cannot be trivially mapped to a graphical output
How would you map a field of anonymous functions?
No interaction
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 14 / 25
20. Augmented Fields Augmented reality as visual interface for fields
User view
Crowd detection scenario: change the perceived hue in such a way that
dangerously dense areas are properly advertised.
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 15 / 25
21. Augmented Fields Augmented reality as visual interface for fields
Developer view
Crowd detection program development: see the field.
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 16 / 25
22. Augmented Fields Augmented reality-based input for CF program
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 17 / 25
23. Augmented Fields Augmented reality-based input for CF program
Weak interaction
Provide input to a CF program through AR interaction
How
Connect the (processed) AR input to the “sensors” of a CF program
Requires basically no intervention on the CF program
New information must be added to an existing device participating
the aggregate
Why
Some system settings could be very naturally set by means of a
gesture
Limitations
The new information is constrained in a device (a single point in
space)
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 17 / 25
24. Augmented Fields Augmented reality-based input for CF program
Strong interaction
GHOST
AGENT
GHOST
BODY STREET
LAMP
PLAYER
BODY
PLAYER
ASSISTANT
AGENT
MIRROR WORLD
PHYSICAL WORLD
8
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 18 / 25
25. Augmented Fields Augmented reality-based input for CF program
Strong interaction
Inject new information that lives in a “Mirror World”
How
Modify the CF program to “sustain” the existence of entities in areas
where no device is located
e.g. with a field of mappings between virtual entities and their position
Why
Support for highest degree of augmented reality
Mixed real-virtual collective applications
Limitations
The program must be conceived with this kind of interaction in mind
Harder to realize
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 19 / 25
26. Augmented Fields Computational fields as enabling technology for AR applications
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 20 / 25
27. Augmented Fields Computational fields as enabling technology for AR applications
A look the other way around
We have discussed increasing degrees of interaction between augmented
reality and computational fields, up to a “strong” interaction, but...
...are computational fields a valid abstraction for creating such programs?
Progression
“Weak” interaction is almost a free lunch
“Strong” interaction seems powerful, but requires CF programs to be
designed with AR in mind
AR is no longer confined to be an I/O component of the system, it
impacts the “business logic” of the application instead
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 20 / 25
28. Augmented Fields Computational fields as enabling technology for AR applications
Mirror Worlds on Augmented Fields
Basic strategy:
Switch from a simple field of mirrored entities to a field of entities with
location
Exploit CF programming to keep the entities alive and their status
aligned on multiple devices
If entities have their own behavior, mobile code support is required
[DVPB15]
Reusable building blocks can be exploited in order to devise a
“standard” solution [BPV15]
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 21 / 25
29. Augmented Fields Example
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 22 / 25
31. Conclusion Conclusion and future work
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 23 / 25
32. Conclusion Conclusion and future work
Conclusion
We noted that both computational fields and augmented reality work
best in environments densely populated with computational devices
We have explored increasing degrees of integration
AR as UI ⇒ not hard, no change in aggregate programs
Local input from AR devices ⇒ still not hard, still no change
Injection of (possibly proactive) augmented entities, ⇒ requires CF
programs to be properly designed
Modern aggregate programming languages support code mobility and
reusability
We believe that CF are a promising programming abstraction for
supporting AR applications, and languages and technologies are maturing
Future work
Build a working demo is the most effective way of experimentally confirm
our hypothesis.
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 23 / 25
33. References
References I
Ronald Azuma.
A survey of augmented reality.
6(4):355–385, 1997.
Jacob Beal and Jonathan Bachrach.
Infrastructure for engineered emergence on sensor/actuator networks.
IEEE Intelligent Systems, 21(2), 2006.
Jacob Beal, Danilo Pianini, and Mirko Viroli.
Aggregate programming for the internet of things.
IEEE Computer, 48(9), 2015.
Ferruccio Damiani, Mirko Viroli, Danilo Pianini, and Jacob Beal.
Code mobility meets self-organisation: A higher-order calculus of computational fields.
In Susanne Graf and Mahesh Viswanathan, editors, Formal Techniques for Distributed
Objects, Components, and Systems, volume 9039 of Lecture Notes in Computer Science,
pages 113–128. Springer International Publishing, 2015.
Marco Mamei and Franco Zambonelli.
Programming pervasive and mobile computing applications: The tota approach.
ACM Transactions on Software Engineering and Methodologies, 18(4), 2009.
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 24 / 25
34. References
References II
Danilo Pianini, Sara Montagna, and Mirko Viroli.
Chemical-oriented simulation of computational systems with Alchemist.
Journal of Simulation, 2013.
Danilo Pianini, Mirko Viroli, and Jacob Beal.
Protelis: Practical aggregate programming.
In Roger L. Wainwright, Juan Manuel Corchado, Alessio Bechini, and Jiman Hong, editors,
Proceedings of the 30th Annual ACM Symposium on Applied Computing, Salamanca,
Spain, April 13-17, 2015, pages 1846–1853, Salamanca, Spain, 2015. ACM.
Alessandro Ricci, Michele Piunti, Luca Tummolini, and Cristiano Castelfranchi.
The mirror world: Preparing for mixed-reality living.
IEEE Pervasive Computing, 14(2):60–63, 2015.
Mirko Viroli, Ferruccio Damiani, and Jacob Beal.
A calculus of computational fields.
In Carlos Canal and Massimo Villari, editors, Advances in Service-Oriented and Cloud
Computing, volume 393 of Communications in Computer and Information Sci., pages
114–128. Springer Berlin Heidelberg, 2013.
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 25 / 25