1. Information Engineering in the
Age of the Internet of Things
1
Payam Barnaghi
Institute for Communication Systems (ICS)/
5G Innovation Centre
University of Surrey
Guildford, United Kingdom
Digital Catapult, December 2015

2. “A hundred years hence people will be so avid
of every moment of life, life will be so full of
busy delight, that time-saving inventions will
be at a huge premium…”
“…It is not because we shall be hurried in
nerve-shattering anxiety, but because we shall
value at its true worth the refining and restful
influence of leisure, that we shall be impatient
of the minor tasks of every day….”
The March 26, 1906, New Zealand Star :
Source: http://paleofuture.com

3. 3
IBM Mainframe 360, source Wikipedia

4. Apollo 11 Command Module (1965) had
64 kilobytes of memory
operated at 0.043MHz.
An iPhone 5s has a CPU running at speeds
of up to 1.3GHz
and has 512MB to 1GB of memory
Cray-1 (1975) produced 80 million Floating
point operations per second (FLOPS)
10 years later, Cray-2 produced 1.9G FLOPS
An iPhone 5s produces 76.8 GFLOPS – nearly
a thousand times more
Cray-2 used 200-kilowatt power
Source: Nick T., PhoneArena.com, 2014
image source: http://blog.opower.com/

5. Computing Power
5
−Smaller size
−More Powerful
−More memory and more storage
−"Moore's law" over the history of computing, the
number of transistors in a dense integrated circuit
has doubled approximately every two years.

6. Smaller in size but larger in scale
6

7. The old Internet timeline
7Source: Internet Society

8. Connectivity and information exchange was
(and is ) the main motivation behind the
Internet; but Content and Services are now
the key elements;
and all started growing rapidly by the
introduction of the World Wide Web (and
linked information and search and discovery
services).
8

9. Early days of the web
9

10. Search on the Internet/Web in the early days
10

11. 11
AnyPlace AnyTime
AnyThing
Data Volume
Security, Reliability,
Trust and Privacy
Societal Impacts, Economic Values
and Viability
Services and Applications
Networking and
Communication

12. 12
Sensor devices are becoming widely available
- Programmable devices
- Off-the-shelf gadgets/tools

13. Internet of Things: The story so far
RFID based
solutions
Wireless Sensor and
Actuator networks
, solutions for
communication
technologies, energy
efficiency, routing, …
Smart Devices/
Web-enabled
Apps/Services, initial
products,
vertical applications, early
concepts and demos, …
Motion sensor
Motion sensor
ECG sensor
Physical-Cyber-Social
Systems, Linked-data,
semantics, M2M,
More products, more
heterogeneity,
solutions for control and
monitoring, …
Future: Cloud, Big (IoT) Data
Analytics, Interoperability,
Enhanced Cellular/Wireless Com.
for IoT, Real-world operational
use-cases and Industry and B2B
services/applications,
more Standards…

14. Real world data
14

15. Data in the IoT
− Data is collected by sensory devices and also crowd sensing
resources.
− It is time and location dependent.
− It can be noisy and the quality can vary.
− It is often continuous - streaming data.
− There are several important issues such as:
− Device/network management
− Actuation and feedback (command and control)
− Service and entity descriptions.

16. IoT data- challenges
− Multi-modal, distributed and heterogeneous
− Noisy and incomplete
− Time and location dependent
− Dynamic and varies in quality
− Crowdsourced data can be unreliable
− Requires (near-) real-time analysis
− Privacy and security are important issues
− Data can be biased- we need to know our data!
16
P. Barnaghi, A. Sheth, C. Henson, "From data to actionable knowledge: Big Data Challenges in the Web of Things," IEEE Intelligent Systems,
vol.28 , issue.6, Dec 2013.

17. Making IoT data widely available and (re-)usable
− Machine-readable and/or human interpretable meta-data
− Open IoT data portals (static and streaming)
− Open APIs and discoverable interfaces
− Discoverable data and patterns (and resources)
− (easily) Sharable and connectable data
− Opportunistic and ad-hoc discovery, matching and mash-up
solutions
− Quality and fit-for-purpose data
− Trust, privacy and security aware data collection, sharing and
access solutions
17

18. Device/Data interoperability
18
The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

19. Heterogeneity, multi-modality and volume are
among the key issues.
We need interoperable and machine-interpretable
solutions…
19

20. A bit of history
− “The Semantic Web is an extension of the current web in
which information is given well-defined meaning, better
enabling computers and people to work in co-operation.“
(Tim Berners-Lee et al, 2001)
20
Image source: Miller 2004

21. Semantics & the IoT
−The Semantic Sensor (&Actuator) Web is an
extension of the current Web/Internet in which
information is given well-defined meanings, better
enabling objects, devices and people to work in co-
operation and to also enable autonomous
interactions between devices and/or objects.
21

22. Semantic Descriptions in Semantic (Web) World
22

23. 23
The world of IoT and Semantics

24. Example: IoT-A information model
24

25. 25
Some good existing models: SSN Ontology
Ontology Link: http://www.w3.org/2005/Incubator/ssn/ssnx/ssn
M. Compton, P. Barnaghi, L. Bermudez, et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics,
2012.

26. 26
But why do we still not have fully
integrated semantic solutions in the IoT?

27. Semantic Web these days…
27

28. Several existing models
28
A simple Google search for “IoT data model” returned around 3,720,000 results.

29. 29
We have good models and description
frameworks;
The problem is that having good models and
developing ontologies are not enough.

30. 30
Semantic descriptions are intermediary
solutions, not the end product.
They should be transparent to the end-user and
probably to the data producer as well.

31. A WoT/IoT Framework
WSN
WSN
WSN
WSN
WSN
Network-enabled
Devices
Semantically
annotate data
31
Gateway
CoAP
HTTP
CoAP
CoAP
HTTP
6LowPAN
Semantically
annotate data
http://mynet1/snodeA23/readTemp?
WSN
MQTT
MQTT
Gateway
And several other
protocols and solutions…

32. Publishing Semantic annotations
− We need a model (ontology) – this is often the easy part for
a single application.
− Interoperability between the models is a big issue.
− Express-ability vs Complexity is a challenge.
− How and where to add the semantics
− Where to publish and store them
− Semantic descriptions for data, streams, devices (resources)
and entities that are represented by the devices, and
description of the services.
32

33. 33
Simplicity can be very useful…

34. Hyper/CAT
34
Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html
- Servers provide catalogues of resources to
clients.
- A catalogue is an array of URIs.
- Each resource in the catalogue is annotated
with metadata (RDF-like triples).

35. Hyper/CAT model
35
Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

36. 36
Perhaps complex models are (sometimes) good
for publishing research papers….
But they are often difficult to implement and use
in real world products.

37. What happens afterwards is more important
− How to use the data (documentation, tools, interfaces)
− How to index and query the annotated data
− How to make the publication suitable for constrained
environments and/or allow them to scale
− How to query them (considering the fact that here we are
dealing with live data and often reducing the processing time
and latency is crucial)
− Linking to other sources
37

38. The IoT is a dynamic, online and rapidly
changing world
38
isPartOf
Annotation for the (Semantic) Web
Annotation for the IoT
Image sources: ABC Australia and 2dolphins.com

39. Tools and APIs- e.g. Sense2Web
39P. Barnaghi, M. Presser, K. Moessner, "Publishing Linked Sensor Data", in Proc. of the 3rd Int. Workshop on Semantic Sensor Networks (SSN),
ISWC2010, 2010.

40. Tools and API – e.g. FIWARE IoT Discovery Generic
Enabler
40http://catalogue.fiware.org/enablers/iot-discovery/documentation

41. Model documentation- e.g. SAO ontology
41
http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/sao

42. Providing flexible APIs- e.g. SAOPY
42
http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/saopy.html

43. 43
Creating common vocabularies and
taxonomies are also equally important
e.g. event and unit taxonomies, common
formats for representing location,
vocabularies to describe proximity and
relations between Things and their data.

44. 44
We should accept the fact that sometimes
we do not need (full) semantic
descriptions.
Think of the applications and use-cases
before starting to annotate the data.

45. An example: a discovery
method in the IoT
time
location
type
Query formulating
[#location | #type | time][#location | #type | time]
Discovery ID
Discovery/
DHT Server
Data repository
(archived data)
#location
#type
#location
#type
#location
#type
Data hypercube
Gateway
Core network
Network Connection
Logical Connection
Data

46. An example: a discovery method in the IoT
46
S. A. Hoseinitabatabaei, P. Barnaghi, C. Wang, R. Tafazolli, L. Dong, "Method and Apparatus for Scalable Data Discovery in IoT Systems",
US Patents, 2015.

47. 101 Smart City Use-case Scenarios
47
http://www.ict-citypulse.eu/page/content/smart-city-use-cases-and-requirements

48. 48
Semantic descriptions can be fairly static on the
Web;
In the IoT, the meaning of data and the
annotations can change over time/space…

49. Dynamic Semantics
<iot:measurement>
<iot:type> temp</iot:type>
<iot:unit>Celsius</iot:unit>
<time>12:30:23UTC</time>
<iot:accuracy>80%</iot:accuracy>
<loc:long>51.2365<loc:lat>
<loc:lat>0.5703</loc:lat>
</iot:measurment>
49
But this could be a function of
time and location;
What would be the accuracy
5 seconds after the
measurement?

50. Dynamic annotations for data in the process chain
50
S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

51. Dynamic annotations for provenance data
51
S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

52. 52
Metadata (Semantic) or higher-level event
descriptions can also be learned and
created automatically.

53. Extraction of events and semantics from social media
53
City Infrastructure
Tweets from a city
https://osf.io/b4q2t/
Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, Amit P Sheth, "Extracting City Traffic Events from Social Streams",
ACM Transactions on Intelligent Systems and Technology, 2015

54. Ontology learning from real world data
54Frieder Ganz, Payam Barnaghi, Francois Carrez, "Automated Semantic Knowledge Acquisition from Sensor Data", IEEE Systems
Journal, 2014.

55. Overall we need semantics and metadata in
the IoT and these play a key role in providing
interoperability.

56. However, we should design and use the
data publication, access, sharing and their
associated metadata models carefully and
consider the constraints and dynamicity
of the IoT environments.

57. Data Lifecycle
57
Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment: Challenges and
opportunities of data driven systems for building, community and city-scale applications,
http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

58. Information Engineering for the IoT:
Design Commandments
58

59. #1: Design for large-scale and provide tools and APIs.
#2: Think of who will use the data and how when you
design your models.
#3: Provide means to update and change the semantic
annotations.
59

60. Smart data collection
− Smart data collection
− Sooner or later we need to
think whether we need to
collect that data, how often
we need to collect it and
what volume.
− Intelligent data Processing
(selective attention and
information-extraction)
60
(image source: KRISTEN NICOLE, siliconangle.com)

61. Image sources : The dailymail, http://helenography.net/, http://edwud.com/

62. Designing for real world problems

63. #4: Create tools, open APIs and datasets for
validation, evaluation, and interoperability testing.
#5: Create common vocabularies and provide
documentation.
#6: Of course you can always create a better model,
but try to re-use existing ones as much as you can.
63

64. Reference Datasets
64http://iot.ee.surrey.ac.uk:8080/datasets.html

65. Importance of Complementary Data
65

66. Open Data/Open APIs
−Open data is often misinterpreted as free data
and publically available data.
−You can have open data, but with the right
access controls, with trust and privacy.
66

67. #7: Link your data and descriptions to other existing
resources.
#8: Define rules and/or best practices for providing the
values for each attribute.
#9: Remember the widely used (semantic) models on
the Web are simple ones like FOAF.
67

68. Best Practices: an example (early draft)
68

69. Spatial Data on the Web- Best Practices (early draft)
69

70. #10: Design for different audience (data consumers,
developers, providers) and think about real impact and
sustainability.
#11: Specify (and encourage others to do the same)
data governance and privacy procedures, explain the
ownership and re-use rules, and give control to the
owners of data
70

71. 71Source LAT Times, http://documents.latimes.com/la-2013/
Future cities: A view from 1998

72. 72
Source: http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/
Source: wikipedia
Back to the Future: 2013

73. 73

74. Users in control or losing control?
74
Image source: Julian Walker, Flicker

75. #12: Semantics and information engineering are only
one part of the solution and often not the end-product
so the focus of the design should be on creating
effective methods, tools and APIs to handle and process
the semantics.
75

76. Technical challenges and research directions
76

77. Technical (and non-technical) Challenges
− Creating common models to represent, publish, and
(re-)use and share IoT data.
− creating an IoT data market and data-driven innovation
− Developing standards
− Providing best practices, demonstrators and open data
portals for streaming and dynamic IoT data.
− Provide governance, dependability, reliability, trust and
security models.
77

78. Research challenges
−Transforming raw data to actionable-information.
−Machine learning and data analytics for large-scale, multi-
modal and dynamic (streaming data).
− Making data more accessible and discoverable.
−Energy and computationally efficient data collection,
aggregation and abstraction (for both edge and
Cloud processing).
78

79. Research challenges (continued)
−Integration and combination of Physical-Cyber-Social
data.
−Use of data for automated interactions and
autonomous services in different domains.
−Resource-aware and context-aware security, privacy
and trust solutions.
79

80. Desire for innovation
80
Driverless Car of the Future (1957)
Image: Courtesy of http://paleofuture.com

81. In conclusion
− IoT information engineering is different from common models of web data
and/or other types of big data.
− Data collection in the IoT comes at the cost of bandwidth, network,
energy and other resources.
− Data collection, delivery and processing is also depended on multiple
layers of the network.
− We need more resource-aware data analytics methods and cross-layer
optimisations (Deep IoT).
− The solutions should work across different systems and multiple platforms
(Ecosystem of systems).
− Data sources are more than physical (sensory) observation.
− The IoT requires integration and processing of physical-cyber-social data.
− The extracted insights and information should be converted to a feedback
and/or actionable information.
81

82. Let’s hope
−The Internet of the Future will be
−For everyone, everywhere, available at anytime,
−People will have control on their data
−Data will be used for helping people
−Smart applications will contribute to a better life
and to a better use of of our resources in the
world!
82

83. Other challenges and topics that I didn't talk about
Resilience and
reliability
Noise and
incomplete data
Cloud and
distributed computing
Networks, test-beds and
mobility
Mobile computing
Services
83

84. IET sector briefing report
84
Available at: http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

85. 85
Useful information:
http://www.raeng.org.uk/publications/reports/connecting-data-driving-productivity

86. Q&A
− Thank you.
http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/
@pbarnaghi
p.barnaghi@surrey.ac.uk