WoTSF: A Framework for Searching in the Web of Things (WoT)

WoTSF: A Framework
for Searching in the Web of
Things
M. Younan, S. Khattab, and R. Bahgat
May 11th 2016
M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof
Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems,
ACM,Cairo,Egypt,May,2016.

1 • Introduction
2 • Related Work
3 • The Architecture of the WoTSF
4 • The Implementation of the WoTSF
5 • Experimental Evaluation
6 • Conclusion and Future Work
Agenda

INTRODUCTION

Introduction (WSN -> WoT)
• Wireless Sensor Network (WSN)
• Daily life integration of embedded devices
– Convert things to Smart Things (SThs)
– No.of devices reaches order of billions in 2020.
– No.of users < no.of devices.
• The Internet of Things (IoT).
– Huge Sensory Data (data stream).
• The Web of Things (WoT).
– Current web tools and services.
• Contribution Idea:
– Searching in the WoT increases its popularity

Introduction
• Features of the WoT (Searching)
– Different formats.
– Non-standardized naming
– Huge and dynamic Sensory Data.
– Daily life integration (normal user)
• Simple query language
• Real time queries
• Interested in high level knowledge (summary)
• Main Points:
– Crawling
• LWoTSEs’ Data –> following features of the WoT dataset presented in
[1][2].
– Indexing
• WoTSF: a Framework for searching in the WoT (High level indices)
Crawling
Indexing and
Searching

RELATED WORK

Related Work (Selection)
 Traditional Search engines
o Google Optimizations [3]
o AJAX Crawler [4]
 DiscoWoT (Mayer and Dominque. 2011)[5].
 Shodan SE (www. Shodanqh.com,2015)[6] .
 Dyser SE (Ostermaier et al.- 2010)[7].
 Zhang et al. (2015) present a framework for a
distributed range-query search[8]

Dyser
• Dyser uses periodic patterns for each sensor
(L,O,W,P), where
L: period length O: offset W: sensor output P: probability value
E.g., (7-Days, 2, empty, 0.5)
• Expansion of such periodic pattern is (K*L +O)
Su Mo Tu We Th Fr Sa
17 18 19 20 21 22 23
24 25 26 27 28 29 30
For Example: If crawling process
starts in 20-4, then Index record:
(Occ., ID_01, 25-4-2016, empty, 0.5)
# recorded patterns in Dyser index will be >= # SThs

Dyser Index-> WoTSF Index
S01: (7-Days, 1, empty, 0.8)
S02: (7-Days, 1, empty, 0.8)
S03: (7-Days, 2, empty, 0.5)
S04: (7-Days, 2, empty, 0.9)
S05: (7-Days, 2, empty, 0.6)
S06: (7-Days, 3, empty, 0.7)
S07: (7-Days, 3, empty, 0.2)
S08: (7-Days, 3, empty, 0.4)
S09: (7-Days, 3, empty, 0.9)
S10: (7-Days, 4, empty, 0.3)
S11: (7-Days, 5, empty, 0.8)
S12: (7-Days, 6, empty, 0.6)
S13: (7-Days, 7, empty, 0.5)
S14: (7-Days, 7, empty, 0.7)
WoT01: (7-Days, 1, empty, 0.8, S01)
WoT01: (7-Days, 1, empty, 0.8, S02)
WoT01: (7-Days, 2, empty, 0.9, S04)
WoT01: (7-Days, 3, empty, 0.9, S09)
WoT01: (7-Days, 4, empty, 0.3, S10)
WoT01: (7-Days, 5, empty, 0.8, S11)
WoT01: (7-Days, 6, empty, 0.6, S12)
WoT01: (7-Days, 7, empty, 0.7, S14)
Using aggregation function: ‘Max.’
1
2
3
4
5
6
7
WoTSF: High Level Index

THE PROPOSED WOTSF
ARCHITECTURE

The WoTSF Architecture
Query life cycle
of the WoTSF
Crawling (WoT)
Google Opt.
(server-root file)
Save time:
crawl less no.of
pages
parse less no. of
pages
(Union format)
(High Level)

The WoTSF Architecture (Query Processing)
Query
q2
WoT_1
WoT_1
M.Ds
WoT_2
M.Ds
WoT_3
M.Ds
DS_1
Ind.
DS_2
Ind.
DS_3
Ind.
•Select WoTSEs APIs
•Send sub-queries
Key:
sensor type
b
c
d
……
a
q2
Master Indices
•Generate sub-queries

The WoTSF Architecture (Index Structure)

The WoTSF Architecture (Index Structure)
mimiMI
SISI SI
D D D D D
S
WWW
WWW
WWW
A
R S A
R
SThs-Level
(individual
WoTSEs)
WoT-Level
(WoTSF)
Master
Indices
Secondary
Indices
Storage
Dynamic
Pages

The WoTSF Implementation (Ranking)
• Sensor State probability
 𝑆𝑆 𝑖 = 𝐾 ∗ 𝐶𝑆 𝑖 + 1 − 𝐾 𝑃𝑖
K-> constant (0:1), CS-> current state, P->Probability
• Entity State probability
 𝐸𝑆 𝑖 =
1
𝑛
𝑆𝑆 𝑗𝑛
𝑗=1
N-> number of sensors
 𝐸𝑆(𝑖) = 𝐷(𝑗) ∗ 𝑆𝑆(𝑗)𝑛
𝑗=1
D(j) -> impact factor of sensor j on ES(i)

THE WOTSF IMPLEMENTATION

The WoTSF Implementation (Crawling)
(b) Gateway
Start
Receive new_value
Update prediction
model (Sr)
no.of.
consecutive
changes >
M
Calculate Entity State
SrVal_Stability (Sr,new_value) =
(cur_rec_time - last_rec_time(Sr))/interval
no.of
records
> N
Replace (oldest_value,
new_value
Add
new_value
Update
(EoI, State)
newChange
(EoI, State)
Listen to Sensors
(a) Sensor
Start
Read Cur_Value
Compare
Cur_value with
previous value
(range)
Send Cur_Value
Change
Wait Period (T)
Yes
Crawling Levels:
– WoTSE
– WoTSF

The WoTSF Implementation (Indexing)
• The WoTSF is evaluated using Real dataset:
We use Kth percentile (historical data) -> single record per
time unit.
e.g.
Suppose, a temperature
sensor reads a set of
values {5, 20, 21, 20,
20, 21, 23, 20, 21, 23}
then,
average=19.4
50th = 20

• The WoTSF is evaluated using random values:
– prediction model (‘7-Days’)
– SThs of type ‘occupancy’ in 10 WoT networks, ->
10,000 SThs
– aggregation function ‘Max’ (empty room).
– expanding the quadruple predictions
The prediction model type -> periodical crawling processes.

EXPERIMENTAL EVALUATION

(a) (b)
In case of distinct SThs values
• Dyser Index (a)
– Count -> Max reading (STh level)
• WoTSF index (b)
– Count -> Max reading (WoT level)
Local search engine
WoTSF

The WoTSF Implementation (Searching)
• The WoTSF prototype:
– implements autosuggestions like dyser.
– filters buildings (static part) -> WoTSEs’ APIs
– Evaluation is done on the dynamic part of the query (WoTSEs) using:
• Building - Level (Master Indices)
• SThs – Level (Secondary Indices)
(a) (b) (c)

The WoTSF Evaluation (Index size)

Experimental Evaluation (WoTSF & Dyser)
• Index Size

• Processing Time
- WoTSF indices (High Level) - Dyser Indices

• Result Accuracy
Accuracy =
𝐶𝑜𝑟𝑟𝑒𝑐𝑡 𝑉𝑎𝑙𝑢𝑒𝑠 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠)
𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑓𝑎𝑙𝑠𝑒 𝑎𝑛𝑑 𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠)
Consistency =
𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠)
𝑇𝑜𝑡𝑎𝑙 𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝑓𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠)
Building ID Room ID State Prediction Value
100
101
Empty
0.8
102 0.4
103 0.7
200
201 0.3
202 0.5
203 0.4
Each building represents a single WoT network and hosts an occupancy sensor in each room
Search
Engine
Index
Size
WoTSF 2
Dyser 6

The WoTSF Evaluation (Index size)
Search Engine
Number of Results
all First K-Result = 4 Probability >= 50%
WoTSF 2 2 2
DSE 6 4 3
WoTSF and DSE search results according to different query types.
Search
Engine
Results
Number of Results
all First K-Result = 4
Probability >=
50%
WoTSF
Room 101 0.8   
Room 202 0.5   
DSE
Room 101 0.8   
Room 103 0.7   
Room 202 0.5   
Room 102 0.4  
Room 203 0.4 
Room 201 0.3 
WoTSF and DSE searching results (list of rooms) according to different query types.

The WoTSF Evaluation (Summary)
Criteria DSE WoTSF
Granularity of
master index
Device (STh) level Network (e.g., building) level
Case:
Speed Search
 All available results
 More time for ranking.
 top values per WoT network.
 Faster search
 Indices more up-to-date
Case:
Accurate
Results
 Indices and accuracy: based on
prediction models.
 Consumes more time
 Indices: up-to-date.
 Accuracy: high
Pros
 Less network overheads.
 More consistent results
 Small and semi-dynamic indices.
 Less time for (crawling, parsing,
indexing).
Cons
 Larger indices.
 Harder to keep indices up-to-
date.
 More time for crawling,
parsing, and indexing
 Tradeoff between search speed
and result accuracy

CONCLUSION AND
FUTURE WORK

• The WoTSF:
 works on the top of Dyser
 supports simple query language
 saves time consumed for crawling, parsing and indexing
 builds High level indices
 Supports two types of search:
• Search in high level indices (speed search).
• Search in secondary indices using LWoTSEs’ APIs (Accurate search)
 increasing accuracy <-> indices are up-to-date
 keeping individual LWoTSEs handle their network.
Conclusion

Future Work
• WoTSF has limitations such as:
– It consumes more time in accurate search type.
• Network overhead
• Merging and Ranking results.
– Scheduling the crawling processes
• to balance network overhead and result accuracy
– Using semantic technology: will be helpful for interoperability.
– Considering other aggregation functions
• LWoTSE has limitations such as:
– Extracting prediction models from SThs historical data
– Dynamic discovery.
– Solving problem of using multiple formats partially (crawling)
but not by meaning -> (ontologies)

References (Selected)
[1] M. Younan, S. Khattab, and R. Bahgat, "An Integrated Testbed Environment for the Web of
Things," in ICNS 2015 : The Eleventh International Conference on Networking and Services,
ISBN: 978-1-61208-404-6, Rome, Italy, May, 2015, pp. 69-78.
[2] M. Younan, S. Khattab, and R. Bahgat, "Evaluation of An Integrated Testbed Environment for
the Web of Things ", in IntSys15v8n34 : International Journal On Advances in Intelligent
Systems, v 8 n 3&4, December, 2015.
[3] Google. (2010, Jan.) Search Engine Optimization (SEO) - Starter Guide.
[4] P. Suganthan G C, "AJAX Crawler," in Data Science & Engineering (ICDSE), International
Conference on. IEEE, Cochin, Kerala, July 2012, pp. 27-30.
[5] S. Mayer, D. Guinard, "An Extensible Discovery Service for Smart Things," in in Proceedings of
the 2nd International Workshop on the Web of Things (WoT 2011), ACM, San Francisco, CA,
USA, June, 2011, pp. 7-12.
[6] (2015, Jan.) shodan search engine. [Online]. www.shodanhq.com
[7] B. Ostermaier, K. Romery, F. Mattern, M. Fahrmairz, and W. Kellererz, "A Real-Time Search
Engine for the Web of Things," in The 2nd IEEE International Conference on the Internet of
Things (IoT), Tokyo,Japan, November. 2010, pp. 1-8.
[8] C. Zhang, T. Zhang, and M.Wang, "A Distributed Range Query Framework for the Internet of
Things," in 18th International Conference on Intelligence in Next Generation Networks (ICIN
2015), IEEE 2015, ISBN 978-1-4799-1866-9, Paris, France, February, 2015, pp. 83-88.

QUESTIONS ?
THANKS

WoTSF: A Framework for Searching in the Web of Things (WoT)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (18)

Ähnlich wie WoTSF: A Framework for Searching in the Web of Things (WoT)

Ähnlich wie WoTSF: A Framework for Searching in the Web of Things (WoT) (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

WoTSF: A Framework for Searching in the Web of Things (WoT)