Ambient Intelligence is an emerging discipline that brings intelligence to our living environments, makes those environments sensitive to us, and adapt according to the user’s needs. By enriching an environment with appropriate sensors and interconnected devices, the environment would be able to sense changes and support decisions that benefit the users of that environment. Such smart environments could help to reduce the energy consumption, increase user’s comfort, improve security and productivity, etc. One specific area of interest is the application of ambient intelligence in Ambient Assisted Living, where the home environment provides assistance with daily living activities for people with disabilities. In my presentation, I will provide a review of the technologies and environments that comprises ambient intelligence, as well as how changes in the environment are reflected in the overall design of an adaptive ambient intelligence environment.

1. ICAIS’14
Adaptive Ambient
Intelligence and Smart
Environments
DR AHMAD LOTFI
School of Science and Technology,
Nottingham Trent University

2. ICAIS’14
Outline of Talk
• Definitions
• Intelligent Environment, Ambient Intelligence and Ambient Assistive
Living
• Statement of the problem
• Tools & Infrastructure
• Robots, wearable and mobile devices, intelligent environments and
smart homes
•State of research in this area
• Activities Classification and Abnormality Detection
• Predictive Ambient Intelligent Environment
• Concluding remarks
2

3. ICAIS’14
Intelligent
Environments
Room Temperature
Room Occupancy
Home Tablet / Controller

4. ICAIS’14
1. Definition
• Intelligent Environments are spaces
with embedded systems and
information and communication
technologies creating interactive
spaces that bring computation into the
physical world.
• Intelligent environments are spaces in
which computation is seamlessly used
to enhance ordinary activity.
http://en.wikipedia.org/wiki/Intelligent_environment
Popular Mechanics
Magazine (Dec, 1950)
4

5. ICAIS’14
Intelligent
Office
Light Control
Room Temperature
PC/Laptop Control
Office Occupancy

6. ICAIS’14
Smart Home
Light Control
Room Temperature
Bed Usage
Daily Activities
Security
Heating
Room Occupancy

7. ICAIS’14
http://www.research.philips.com/
Ambient
Intelligence

8. ICAIS’14
2. Definition
• Ambient Intelligence refers to a digital environment that proactively
supports people in their daily lives.
• It is an emerging discipline that brings intelligence to our living
environments, makes those environments sensitive to us, and adapt
according to the user’s needs.
• By enriching an environment with appropriate sensors and
interconnected devices, the environment would be able to sense
changes and support decisions that benefit the users of that
environment.
• Such smart environments could help to reduce the energy
consumption, increase user’s comfort, improve security and
productivity, etc. One specific area of interest is the application of
ambient intelligence in Ambient Assisted Living, where the home
environment provides assistance with daily living activities for
people with disabilities. 8

9. ICAIS’14
Ambient Intelligence
Characteristics
• Context Awareness
• Personalise
• Immersive
• Adaptive
9

10. ICAIS’14
Exit Here !
Personalised Ambient Display

11. ICAIS’14
Immersive
Ambient
Intelligence
11

12. ICAIS’14
Ambient
Assisted
Living

13. ICAIS’14
3. Definition
• Ambient Assisted Living (AAL) is the
use of information and communication
technologies in a person's daily living
and working environment to enable
individuals to stay active longer and live
independently into old age.
• This could be as simple as an alarm to
remind a person to take medication or
as sophisticates as a mobility scooter or
electric wheelchair to help with daily
shopping.
13

14. ICAIS’14
Sources: CORDIS, TSO “Carers of Elderly People – Summary of the Background Evidence” Alzheimer’s Society
What is the Problem?
• By 2050, people aged 65-79 are expected to make up almost 1/3 of the
population in Europe
• Over that period, the population of very elderly (80+) will rise by 180%
• One in 50 people aged 65-70 have a form of dementia, rising to one in
five over 80
• Currently there are 7 million carers of the elderly in the UK alone.
14

15. ICAIS’14
Challenges
• Social communication: easy access to phone and video conversation to
stay in touch with family and friends, overcoming social isolation
• Daily shopping, travel, social life, public services: easy access to order
goods online e.g. when reduced mobility makes physical shopping
more difficult
• Safety: making sure entrance doors and windows are locked/closed
when leaving the house or sleeping; checking for water or gas leaks;
and turning all but one light off when going to bed, etc.
• Reminders: memory problems tend to be associated to ageing and
thus support may be needed in taking medication and fulfilling
household tasks
• Tele-health: wearable and portable systems for monitoring and
diagnosis, therapy, supporting treatment plans.
• Tele-care: providing medical care remotely. The sensors automatically
raises the alarm by contacting, via a call centre, a family member,
friend, neighbour or warden.
15

16. ICAIS’14
Tools & Infrastructure
What makes Ambient Assisted Living
(AAL) possible?
A. Assistive & Social Robotics
• … a robot that interacts and
communicates with humans or other
devices …
B. Wearable & Mobile Devices
C. Intelligent Environments
and Smart Homes
• ... digital environment that proactively
supports people in their daily lives 16

17. ICAIS’14
ASSISTIVE & SOCIAL ROBOTICS
Tools & Infrastructure
17

18. ICAIS’14
Social Robots
• Semi-autonomous robots
(mobile platforms, humanoid,
…)
• Interacting with people in their
own space
• Augmenting healthcare givers
• Providing rehabilitation and
lifestyle support
• Inexpensive, user-friendly and
reliable
18

19. ICAIS’14
Social Robots (Examples)
VGo Anybots QB DoubleRobot MantaroBot
19

20. ICAIS’14
Assistive Robots (Examples)
User Assisted
Mobility Scooter
Ambient Assisted
Living
http://www.youtube.com/watch?v=8GG0wiAqLHk
20

21. ICAIS’14
WEARABLE & MOBILE DEVICES
Tools & Infrastructure
21

22. ICAIS’14
• Applications
• Health monitoring
• Navigation and stray prevention
• Mobile persuasive technologies
Wearable & Mobile Sensors
LifeShirt By
Vivometrics®
iHealth - Wireless
Activity and Sleep
Tracker
Epidermal Electronics
Smart Cane
22

23. ICAIS’14
INTELLIGENT ENVIRONMENTS AND
SMART HOMES
Tools & Infrastructure
23

24. ICAIS’14
IE & Ambient Assisted Living
Activities
of Daily
Living
(ADL)
Kettle
Microwave
Fridge
Doors
Food cupboards
Cutlery drawer
Bed
Room occupancy
24

25. ICAIS’14
Ambient Assisted Living - the
possibilities
25

26. ICAIS’14
Unobtrusive “Activities of Daily
Living” Monitoring System
• To gather data on the routine activities of elders e.g.,
getting out of bed, going to the bathroom, preparing
meals, taking medications etc. without altering the
elders' normal behaviour
• Preferably wireless sensors should be used, along with
a small computerised receiver to collect data that are
then analysed and posted to a secure central web site
for viewing by the carer/relative
• The adult children of frail elders living alone and at a
distance can be sent reports or alerts daily/weekly in
the form of e-mail or phone calls.
26

27. ICAIS’14
Existing Solutions
27

28. ICAIS’14
Activities of Daily Living
(Example)
http://www.tunstall-life.com/
28

29. ICAIS’14
• to help elders gain independence and
to remain in their own homes longer
than they might otherwise
•understanding human behaviour
from low-level sensory data
• actively offering prompts and other
forms of help as needed
• assistive to human caregivers
Our Research Aim
29

30. ICAIS’14
Monitoring and Interaction
System Architecture
• Data collection
• Communication
• Wired; X10, CEBus, LonWorks,
HomePlug
• Wireless; ZigBee, Wi-Fi,
Proprietary Wireless (license free
433 MHz, 868 MHz, 915 ISM)
• Monitoring and behaviour
analysis
• Behaviour prediction 30

31. ICAIS’14
Data Collection
• Passive Infra-red Sensors (PIR) or motion detectors
• Sensitive to the movements of living objects. They are
normally used to monitor the occupancy of different
areas.
• Door/Window entry point sensors
• On/Off switches which can detect the open and closed
status of a door/window.
• Electricity power usage sensors
• Monitor the activity of electrical devices by measuring
their electrical current consumption.
• Bed/sofa pressure sensors
• To measure the presence in and usage of these areas.
• Flood sensors
• Provide early warning of overflows and leaks.
31

32. ICAIS’14
Occupancy Signal – Single
Occupancy
Time
LoungeSensor
Time
KitchenSensor
0
1
• No parallel activity of PIRs
• No uncertainty on where the occupant is.
32

33. ICAIS’14
Binary Data Representation
• Not easy to work with original binary signals
• Investigated solutions are:
• Combined activity of daily living signal
• Start time and Duration
• Start-time and Stop-time
33

34. ICAIS’14
Start-time and Duration
Bedroom Kitchen
Lounge Corridor
34

35. ICAIS’14
Start-time and Duration for
Bedroom Signal
35

36. ICAIS’14
Start-time and Duration for
Front Door entry point
36

37. ICAIS’14
Occupancy Signal – Multiple
Occupancy
Time
LoungeSensor
0
1
• Signal with Uncertainty representing
an area with multiple occupant.
37

38. ICAIS’14
Single or Multiple Occupancy
Scenario
No parallel activities
Parallel activities
38

39. ICAIS’14
Hour
Behavior pattern
Room transition
Pattern Recognition
Hour
Monday pattern
39

40. ICAIS’14
ACTIVITIES CLASSIFICATION
AND ABNORMALITY DETECTION
Research Area
40

41. ICAIS’14
User Activities Outlier
Detection
• To detect any deviation in the day-to-day behavioural patterns
of occupants using binary data generated from low level
sensors.
• To develop a good understanding of the normal behaviour
and distinguish any abnormalities and possible trend in the
behavioural changes.
• To examine the application of distance measures, Principal
Component Analysis (PCA) and Fuzzy rule-based system in
identifying the abnormality within the behavioural patterns of
an occupant in a smart home.
41

42. ICAIS’14
Outlier Detection System –
Proposed System
Proposed outlier detection system.
42

43. ICAIS’14
Outlier Detection System –
Distance Measures
• Distance measure is defined to measure the degree of differences/distance
between two vectors.
• Only binary distance measures are used since most sensory data collected
from a real environment are based on occupancy sensors presenting in long
sequence of binary data.
• If Ai and Bi are two binary vectors i.e. have the ith feature value either 0 or 1.
• Binary distance measures are based on , where is the number
of occurrences of matches with i in the first pattern and j in the second
pattern at the corresponding positions.
• This technique is not sufficient for real time binary signals. There are many
parallel activities with many instances.
)1;0,( ∈jiSij ijS
S01
S10
S00S11
A
B
43

44. ICAIS’14
Outlier Detection System - PCA
• As new data arrives, it is quite difficult to make a decision on how far or
how close it is. The distance matrix will become a very long matrix.
• It is better to handle the outliers or anomalies using a multivariate
approach where outliers are detected considering all features of the
multidimensional data.
• Principal component analysis (PCA) can be used to reduce the high
dimensionality of the data and ultimately is able to identify any outlier
or abnormality.
• PCA takes the input matrix and transforms it into Eigen vectors e1 , e2 ,
..., ek, and associated Eigen values λ1, λ2, …, λk, where k is the number of
principal components (PCs).
• Using the PCs, two statistical index measures i.e. Hotelling’s T-squared
and Square Prediction Error (SPE) along with their control limits are
computed.
• If both measures exceeds their control limits then the status of the
process is abnormal. Otherwise, the status of the process is considered
as normal when both indices are less than their limits. 44

45. ICAIS’14
Outlier Detection System - PCA
• Hotelling’s T-Squared (T2) measures the squared norm of the current sample
from the centre of the normal data points region.
• The limit of T2 index with a confidence level α is:
where the F (k, N−k, α) corresponds to the probability point on the F-distribution
with (k, N − k) degrees of freedom.
• The Square Prediction Error (SPE) index measures the projection of the data
points on the residual subspace.
• The limit for SPE index which denotes the upper control limit for SPE index with a
confidence level α.
where and .
XV VXT
TT
Λ
−
=
12
),,(
)1(2
lim
αkNkF
kN
Nk
T −
−
−
=
XVVXXX
T
kk
TT
r −−= = rr
T
SPE −=
hhhhC
SPE
a
0
1
2
1
002
1
20
1lim
)1(
1
2


−




++=
θ
θ
θ
θ
θ
θ
θθ 2
2
31
0
3
2
1−=h∑ +=
=
m
kj
i
ji 1λθ
45

46. ICAIS’14
Outlier Detection System-FRBS
• A Fuzzy Rule Based System (FRBS) can report the user
activities in terms of linguistic variables instead of the raw or
pre-processed sensor data.
• Fuzzy if-then rules of the following configuration are
employed for the modelling of the PCA statistical measures to
identify the outliers;
• The universe of input variables is limited to and .
• The final outlier rank is decided based on the rank of the
outlier for each sensor.
CrankoutlierBSPEATR i
jistheni
jisandi
jisif
j
jj
i
j
~~~
2
_: →
)(_ _min1
rankoutlier j
p
j
rankoutlier
=
=
T
2
lim SPElim
46

47. ICAIS’14
Outlier Detection System-FRBS
• Compute the degree
of memberships for
both Hotelling’s T2
and SPE indices as the
inputs of FRBSs and
outlier_rank as the
output.
Membership labels for input and output variables for
the back door sensor.
47

48. ICAIS’14
Outlier Detection System-FRBS
IF THEN
SPE T2 Outlier_rank
Low Low Normal
Medium Low More or Less Normal
High Low Medium
Low Medium More or Less Normal
Medium Medium Medium
High Medium Slight Outlier
Low High Slight Outlier
Medium High Slight Outlier
High High Extreme Outlier
Table of fuzzy rule for outlier rank identification.
48

49. ICAIS’14
Case Studies
• Case 1
• This environment is monitored using JustChecking Monitoring system.
• The data is collected for 14 months.
• It is based on a single elderly occupant
• Front and back door sensors, lounge, kitchen, bedroom, bathroom and
upstairs motion sensors are used.
• Case 2
• An elderly person was first prescribed some medications
• After a few days, her medications were replaced
• This environment is monitored using JustChecking Monitoring system.
• Case 3
• The elderly person uses a walker support to help her in moving around her
apartment.
• Four motion sensors are used: lounge, kitchen, bedroom and corridor
sensors. Two door entries are used: bathroom and the main entrance.
• The data is collected for a couple of weeks where holidays and weekends are
not included.
49

50. ICAIS’14
Extremely Outlier NormalMore or Less Normal
Scattered plot for the 1st and 2nd principal components of the back door entry sensor data used in
case study I with classification.
Results on Case Study (1)
50

51. ICAIS’14
Extremely Outlier NormalMore or Less NormalSlightly Outlier
Results on Case Study (1)
Scattered plot for the 1st and 2nd principal components of the lounge motion sensor data used in
case study I with classification.
51

52. ICAIS’14
Slightly OutlierNormal More or Less Normal
Results on Case Study (2)
Scattered plot for the 1st and 2nd principal components of the bedroom motion
sensor data used in case study II with classification.
52

53. ICAIS’14
Extremely OutlierNormal
Results on Case Study (3)
Scattered plot for the 1st and 2nd principal components of the front door entry sensor data
used in case study III with classification.
53

54. ICAIS’14
PREDICTIVE AMBIENT INTELLIGENT
ENVIRONMENT
Research Area
54

55. ICAIS’14
Behaviour Modelling
Techniques
• Hidden Markov Model (HMM) - is a statistical
model in which the occupant behaviour is
assumed to be a Markov process.
• Recurrent Neural Network (RNN) - The neural
network based approaches use large time series
data sets to learn the relationship between the
input data and output data.
• The accuracy of both statistical and neural network
based methods degrade rapidly with increasing
prediction lead time. 55

56. ICAIS’14
Kitchen Lounge
0.70.3
0.2 0.8
• Two states : ‘Kitchen’ and ‘Lounge’.
• Transition probabilities: P(‘Kitchen’|‘Kitchen’)=0.3 ,
P(‘Lounge’|‘Kitchen’)=0.7 , P(‘Kitchen’|‘Lounge’)=0.2,
P(‘Lounge’|‘Lounge’)=0.8
• Initial probabilities: say P(‘Kitchen’)=0.1 , P(‘Lounge’)=0.9 .
Hidden Markov Model (HMM)
56/15

57. ICAIS’14
Recurrent Neural Network
(RNN)
• The neural network based approaches use large time
series data sets to learn the relationship between the
input data and output data.
• The accuracy of both statistical and neural network
based methods degrade rapidly with increasing
prediction lead time.
• The occupancy signal represents a time series
• Binary time series
• Analog time series
• Time series prediction techniques are investigated to
predict the occupant behaviour.
• 2 hours ahead prediction is the target.
57

58. ICAIS’14
Recurrent networks
• Focused Time Delay Neural
Network (FTDNN)
• Layered Recurrent Neural Network
(LRN)
• Non-linear Autoregressive
netwoRk with eXogenous (NARX)
• Echo State Network (ESN)
• Back Propagation Through Time
(BPTT)
• Real Time Recurrent Learning
(RTRL)
• Long Short Term Memory (LSTM)
58

59. ICAIS’14
6 Hours Ahead Prediction
Using ESN
Predicted values for bedroom
occupancy sensor
Predicted values for corridor
occupancy sensor
59

60. ICAIS’14
2 Hours Ahead Prediction Using
NARX
60

61. ICAIS’14
Concluding Remarks
• Ambient Intelligence is foreseen to be
present everywhere in the future world and
to ease human living.
• To build an environment which will be
natural, informative and caring from human
perspective.
• Attempt to combine the best aspects of the
techniques used.
“The perfect hybrid would be British police, German mechanics, French cuisine,
Swiss banking and Italian love. However, German police, French mechanics, British
cuisine, Italian banking and Swiss love would be a bad one.” Lotfi Zadeh
61

62. ICAIS’14
Publications
http://LOTFI.net

63. ICAIS’14
Current Projects
• AAL (Ambient Assistive Living Joint Programme)
– Intelligent Care Guidance and Learning
Services Platform for Informal Carers of the
Elderly
• KTP (Knowledge Transfer Partnerships) - Energy
Efficiency in Social Housing in Partnership with
Nottingham City Homes Ltd.
http://iCarer-project.eu/

64. ICAIS’14
AHMAD LOTFI
School of Science and Technology,
Nottingham Trent University
Email: ahmad.lotfi@ntu.ac.uk
T H A N K S