Ichiro Satoh's talk at the Awareness Workshop - SASO 2012, Lyon, France

1. Experiences from Context-aware
Services in Real World
National Institute of Informatics, Japan
Ichiro Satoh
Email: ichiro@nii.ac.jp
Ichiro Satoh

2. Self-Introduction
Ichiro Satoh
Professor, National Institute of Informatics
Visiting researcher of Rank Xerox (at Grenoble) From TV news on my experiments in
shopping moles and department stores.
Chairs or program committee members of major
conferences on ubiquitous computing, e.g.,
PerCom and Ubicomp.
Ichiro Satoh

3. National Institute
of Informatics (NII) (Manga)
Publishers area
Akihabara
(electric/geeks/manga/
NII otaku town)
NII
1.5km
The Imperial
Palace
NII has PhD course / internship programs
Tokyo Station If you are interested in the program,
NII is 1.5km-near from Tokyo Station please contact with me
Ichiro Satoh

4. Outline
n Experiment on context-aware services
n Experience from context-aware services at the real world
n Conclusion
This presentation focuses on experiences rather than system
issues.
Ichiro Satoh

5. Middleware for Context-aware
Services
n Our middleware can maintain the locations of computing devices
and software for defining services in addition to those of physical
entities and places in a unified manner.
n Application-specific services are defined and executed within
virtual counterparts within the location model.
n The middleware can naturally provide a location-aware service
discovery mechanism and service policy specification language.
service provider networked light service provider
software control and software deployment software
3.00 3.00 3.00
user-bound proxy light-bound desk-bound
computer counterpart counterpart counterpart counterpart
Specification
3.00
3.00
3.00
3.00
3.00
of the user’s
requiring room-bound room-bound room-bound room-bound
3.00 9.00
3.00 3.00 services counterpart counterpart counterpart counterpart
floor-bound
counterpart
3.00
3.00
9.00
The model also maintains the location and capabilities of
computers and the relocation of software. Ichiro Satoh

6. Experiments
n We had done several experiments in public museums:
n The National Science Museum, and etc.
n Museums have many visitors whose knowledge about
exhibitions are different.
n Annotation about exhibitions should be provided for
visitors dependently on their knowledge and experience,
including the exhibitions that they previously viewed
the National Science Museum (Tokyo, Japan) Ichiro Satoh

7. Experiment
n In the National Science Museum,
each spot was in front of the fossils
of dinosaurs in an exhibition room.
n When a visitor enters in a spot, the
system selected and played audio-
based annotation about dinosaurs
according to the combination of the
current spot, past spots, and his/her
selecting course.
n The experiment has done for two
weeks.
n It had more than 200 participants per-
day and was evaluated with a group of
262 participants.
Ichiro Satoh

8. Experiment (cont’d)
n Each visitor wore a hat equipped with RFID tags.
n Each spot had an RFID reader to detect the presence of a
visitor within it.
Speaker
RFID Tag(s)
Flat antenna for
RFID Reader
I am a member of ISO standardization committee
for RFID and real-time locating technology
Ichiro Satoh

9. How to evaluate context-aware
services
n There is no unified metrics to evaluate context-aware
services.
n It is difficult to evaluate the effectiveness of context-
aware services individually.
n (Only) one solution is to compare between our approach
and existing approaches.
n You should provide context-aware services, which can
be fairly compared from other approaches.
No evaluation is no academic research.
Ichiro Satoh

10. Evaluation
To evaluate the effectiveness of our context-aware visitor-guide
system, we experimented four audio/text-annotation systems.
A) Mobile Phones/PDAs (iPod Touch)
To listen audio-annotation, users input numbers attached to
exhibits (120 testees)
B) Location-aware PDAs (iPod Touch)
When RFID readers detect RFID-tags attached to PDAs, the
system automatically enables PDAs to play audio-annotation
dependent on their current spots. (56 testees)
C) Paper-based posters
Text-based annotation located near the exhibitions. (50 testees)
D) Our experiments (262 testees)
Annotation content were common among the four experiments to offer a
unified metrics in comparison of these approaches.
Note: the above numbers are the number of testees, which are only 8-12 years
children for the reason of a single evaluation metrics. Ichiro Satoh

11. Evaluation (cont’d)
After visitors experienced each courses, we assigned tests to
participants to evaluate the achievement of their learning.
The percentage of their correct answers:
A) Mobile Phones/PDAs: 53%
B) Location-aware PDAs: 52%
C) Paper posters: 66%
D) Our experiments: 68%
Problems:
1. A’s problem : half-visitors inputted incorrect numbers and
listened incorrect annotations but were aware of their
mistakes.
2. B’s problem: most visitors paid attention to their PDAs
instead of exhibitions.
Ichiro Satoh

12. From experiment, we learned..
n Context-aware services should play leading or extra roles.
n All technically interested approaches are not appropriate in the
real world.
n Researchers may misunderstand because people tend to
evaluate new approaches to be interested, but such
approaches may not be effective.
n Many traditional approaches have been used because of they
are effective.
Ichiro Satoh

13. From experiment, we learned..
How to inform services
n It is difficult for users to find/know who/
where/when context-aware services are
available.
n On the other hand, several users were
surprised at the sudden activation of when
context-aware services.
n We need to instruct users who, where, when,
what, and how context-serviced provided.
To avoid this problem, we put marks at
spots to explicitly specify where
context-aware services are available.
Ichiro Satoh

14. From experiment, we learned..
How to support to legacy spaces
n It is more difficult to make legacy spaces than to build new
smart spaces, because of their constraints.
n There may be no space to deploy devices, e.g., sensors,
in legacy spaces, including homes and offices.
n Legacy spaces may lack power-line and networks.
n Context-aware services must make existing spaces smart
without losing any utility of the original spaces.
Museums required all devices
to be invisible from visitors.
Ichiro Satoh

15. From experiment, we learned..
How to manage
n Management systems are important.
n Context-aware services need
numerous heterogeneous sensors
and devices.
n Public spaces,
including museums,
have no spaces for
management systems.
n They lack any professional
administrators.
Backyard of a typical experiment for smart rooms.
It is often larger than its target rooms.
Ichiro Satoh

16. From experiment, we learned..
How to manage (cont’d)
n There may be no space for system
management in real spaces.
n Our experiments supported GUI-based
management systems in PDAs.
n It enabled curators to monitor and
control context-aware services by
using their portable devices instead
of any stationary terminals.
Ichiro Satoh

17. From experiment, we learned..
Availability and Reliability
n Context-aware services are required to be provided anytime.
n e.g., the experiment was required to continue to work for 6
days because the devices were deployed close to definitely
precious specimens.
n There is no silver bullet to develop reliable systems.
n Context-aware systems are a mission-critical system.
n In fact, student-level programming skills may not be enough.
n I designed and implemented all middleware systems in
out experiments.
In fact, our middleware and services could support more than 200
visitors per-day and continue to work without any reboot and
configuration for every 6 days.
Ichiro Satoh

18. From experiment, we learned..
Availability and Reliability
n Sensing systems should be selected according to their
measurement errors.
n Lateration systems can locate users' positions, but they may
sometimes return incorrect positions far from their real
positions.
n They may provide incorrect users with incorrect services.
n RFID-tag-based proximity systems can detect the existences
of users within specified areas but they occasionally lose the
positions of tags.
n They may not provide users with services even when they
are in spaces where they should receive the services.
n The latter’s problem is better than the latter’s problem in the real
world.
n Least suffering strategy rather than best-effort one.
Ichiro Satoh

19. From experiment, we learned..
Availability and Reliability
n Smart surroundings are stupid and scatterbrain.
n Sensing systems are not perfect.
n Champion data in experiments are meaningless.
n The technology needs to support least suffering strategy
instead of best-effort one.
n High reliability and availability
are needed.
n How to notify failures of smart
surroundings to users.
n I also designed a new antenna for the
experiment.
Ichiro Satoh

20. From experiment, we learned..
How to notice system errors
n Sensing and computing systems are not perfect.
n System errors must be inevitable.
n Two solutions:
n To recover systems errors as much as possible,
n To enable users to notice system errors.
This is a good example of notifications of failures in reliable systems
Ichiro Satoh

21. From experiment, we learned..
Deployment and installation
n We cannot have long time to deploy equipment at the real world.
n Museum required us to deploy and install our system for one
day (close day).
n Devices can be easily replaced by new ones.
n Context-aware systems should be robust from rainy, wind, dust,
and mischievous attacks.
n They should be able to be packed and assembled for logistics.
Ichiro Satoh

22. From experiment, we learned..
Deployment and installation
n The current version enables end-users to easily install and
configure context-aware services by using (semi)self-
tuning / diagnosis / healing mechanisms.
n We lend our systems to non-professional peoples to
evaluate their installation.
n In fact, artists could setup and provide their media-
art works without our help at fine art museums.
Ichiro Satoh

23. From experiment, we learned..
How to right human errors
n Context-aware services are required to guide visitors to their
right behaviors.
n Visitors often do unexpected behaviors.
n But, most existing user-assistant systems assume that
people always follow their assistances exactly.
n To right human errors, we need to
warn users in various approaches.
n When users miss visual signs,
we may need to right them in
another approach,
e.g., audio-based one.
n In the experiment, 95% programs are exceptional handling
for human, sensing, or system errors.
Ichiro Satoh

24. From experiment, we learned..
n Context-aware services in public spaces take care of
handicapped / elder / child persons.
n Context-aware services must avoid to become obstacles
for them.
n Cabling and devices may prevent their movement.
n All visitors are not gentle.
n Mischievous children, complainers, etc.
n Context-aware services may incur resentments.
n Some voluntary museum guides felt that our context-aware
services deprived their activities.
n Audio-guide systems in many museums have been
operated by third-parties.
Ichiro Satoh

25. Proactiveness vs. Spookiness
n Context-aware services are required to be proactive:
n Computing systems can anticipate our needs
and act on our behalf.
n They need to understand the user’s context and how it
changes over time.
n People want proactive services but may feel spooky or officious.
n I think, there may be an implicit gap between proactive and
spooky/officious services.
Ichiro Satoh

26. Conclusion
n Context-aware systems depend on the real world.
n They should be evaluated in the real world.
n Real requirements for context-aware computing can be studied
only from real experiences.
n in real spaces, with real users, for real applications,
n There is a large gap between prototype-/laboratory-level and
real systems.
n Most issues in the latter do not appear in the former.
n Many research issues in the gap.
We cannot know real problems without real experiences.
Ichiro Satoh

27. Question
Experiment (is hard…)
Ichiro Satoh