This presentation is devoted to the "ShoBeVODSDT: Shodan and Binary Edge based vulnerable open data sources detection tool or what Internet of Things Search Engines know about you" research paper developed by Artjoms Daskevics and Anastasija Nikiforova and presented during the The International Conference on Intelligent Data Science Technologies and Applications (IDSTA2021), November 15-16, 2021. Tartu, Estonia (web-based). Read paper here -> Daskevics, A., & Nikiforova, A. (2021, November). ShoBeVODSDT: Shodan and Binary Edge based vulnerable open data sources detection tool or what Internet of Things Search Engines know about you. In 2021 Second International Conference on Intelligent Data Science Technologies and Applications (IDSTA) (pp. 38-45). IEEE.

1. SHOBEVODSDT: SHODAN AND BINARY EDGE BASED
VULNERABLE OPEN DATA SOURCES DETECTION TOOL
OR
WHAT INTERNET OF THINGS SEARCH ENGINES KNOW
ABOUT YOU
The International Conference on Intelligent Data Science Technologies and Applications (IDSTA2021)
November 15-16, 2021. Tartu, Estonia (web-based)
Artjoms Daskevics, Anastasija Nikiforova
“Innovative Information Technologies” Laboratory, Programming Department
Faculty of Computing, University of Latvia

2. AIM
To propose an OSINT-based (Open Source Intelligence) tool for non-intrusive testing of open data sources inspecting their
vulnerabilities and their extent.
is the data source visible outside the organization?
what data can be gathered from open data sources (if any) and what is their “value” for attacker and fraudsters?
whether these data can pose the risks to organization using them to deploy an attack?
This allows both a comprehensive analysis of unprotected data sources, falling into a list of predefined data sources, or a
specific IP or IP range to examine what can be seen from the outside of the organization about the data source in use
The use of Open Source Intelligence (OSINT) tools, more precisely the Internet of Things Search Engines (IoTSE) should
allow the tool to inspect a list of predefined data sources on their vulnerabilities and their extent
ShoBeVODSDT
Shodan- and Binary Edge- based vulnerable open data sources detection tool

3. ShoBeVODSDT
ShoBEVODSDT uses mainly the passive assessment (non-intrusive testing), which is characterized by its
low level of intrusiveness;
the data sources concerned are not thoroughly and actively tested.;
the tool refer to the most likely and potentially existing bottlenecks or weaknesses which, if the fourth stage
of the penetration testing, namely the attack, would take place, could be revealed and exposed.
ShoBeVODSDT
Shodan- and Binary Edge- based vulnerable open data sources detection tool
ShoBeVODSDT

4. ShoBeVODSDT SCOPE
What will be inspected?
8 types of data sources– MySQL, PostgreSQL, MongoDB, Redis, Elasticsearch,
CouchDB, Cassandra and Memcached.
Three types of sources
relational databases,
NoSQL databases, both types, document-oriented,
column-oriented and key-value databases
data stores.
How it will be inspected?
OSINT tools or, more precisely, Internet of Things (IoT) search engines (IoTSE)
Shodan and BinaryEdge, which search for and index publicly available and accessible open data sources

5. Database Primary database model Connection data Default port
MySql Relational DBMS IP address, port, username, password 3306
PostgreSql Relational DBMS IP address, port, authentication data (if supports connection with a
password)
5432
MongoDB Document store IP address, port, username, password 5984
Redis Key-value store IP address, port, authentication data (if access control is enabled) 27017
Elasticsearch Search engine IP address, port 6379
CouchDB Document store IP address, port, authentication data (if anonymized access is not
enabled)
9200
Cassandra wide-column store IP address, port, authentication data 9160
Memcached key-value store IP address, port 11211
DATA SOURCES, THEIR MODELS AND CONNECTION DATA

6. ShoBeVODSDT ACTION
searches for files in a “checked” folder that corresponds to
the service and country being checked;
opens the file and checks IP address using the “check”
class method associated with the service;
if the connection has been successful, the IP address is
stored in „good/<service_name> _ <country>.txt”, if failed -
the IP address and error information are stored in the
„bad/<service_name>_ <country>.txt”.
Step I
IP address search (gather)
uses BinaryEdge and Shodan libraries to find
service IP addresses that belong to an user-defined
country;
combines results from BinaryEdge and Shodan
by eliminating duplicates;
saves results in the
“parsed/<service_name_>_<country>.txt”;
Step II
IP address check
Step III
Retrieving information from an IP
address (parse)
searches for files in a “parsed/good” folder that corresponds to the
service and country to be checked;
opens the file and tries to reconnect. If the connection was successful -
tries to download the information from the database. For each type of
database, the is different;
saves the information in the “parsed” ,“<IP_ ADDRESS>.txt”.

7. TOOL ARCHITECTURE
The search class includes a class constructor where a Shodan or
Binary Edge client is initialized using a valid API key and
search method to obtain data from Shodan or Binary Edge*.
*In the case of Binary Edge, a page number to search for IP addresses should
also be provided.
The service class includes a class constructor where a separate
service client tries to establish the new connection. Two
functions :
(1) “check”, which returns an error if the connection was
unsuccessful or “true” if it was successful
(2) “parse”, which attempts to download all information
from the database.

8. ShoBeVODSDT IN ACTION
Use-case - data on Latvia, Estonia and Lithuania (Baltic States)
15180 IP addresses were processed,
Lithuania (7453)
Estonia (5352)
Latvia (2375)
98.43% of the addresses have failed to connect
Category Description
0 failed to connect
1 has managed to connect but failed to gather data or information
2 has managed to connect, but the database is empty
3 has managed to connect by gathering system data or non-sensitive information
4 has managed to connect and gather sensitive data
5 compromised database
✔ the further actions took place with 1.57% or 93 IP addresses only

9. ShoBeVODSDT IN ACTION
“2” and “3” – the most popular categories – good point, i.e. while these
data sources are open, these data are not of very high importance to
attackers and fraudsters, although they can facilitate their attacks,
8% of data sources contain data that could be used by attackers,
12% of them have already been compromised
most empty and compromised databases belong to Elasticsearch.
most databases that store sensitive data belong to Memcached, but it is also a
leader in databases where sensitive data are not stored (category “3”).
Memcached and ElasticSearch have the highest number of open data sources
with higher “value” of data gathered from them in almost all categories, except for
relatively poor results demonstrated by the MongoDB for the number of
compromised databases and Redis for data sources storing sensitive data.

10. FUTURE WORKS
The list of used IoTSE may be extended to other well-known Search Engines such as Censys, ZoomEye etc. to allow more extensive
investigation and determine whether the number of IoTSE has an impact on the results.
Similarly, the number of data sources can be supplemented by other data sources identified as the most popular; especially given
Oracle and MS SQL are somteimes found to have the highest number of vulnerabilities.
Although our aim was to propose the tool for investigating databases only, further studies may also cover other “types of devices”,
such as Network Equipments, Terminal, Server, Office Equipment, Industrial Control Equipment, Smart Home, Power Supply
Equipment, Web Camera, Remote Management Equipment, Blockchain and industrial based connected devices in the cloud.
At the moment, the future study aims to apply the tool to specific countries of Latvia, Lithuania and Estonia and to carry out
extensive investigation on the current state of data sources and their security. This will allow conclusions to be drawn on differences
in country patterns, i.e. whether the technological development of Estonia will be also seen in this matter. It will draw more objective
conclusions on the less protected-by-design data sources.

11. RESULTS AND CONCLUSIONS I
The paper proposes a tool called ShoBeVODSDT - Shodan- and Binary Edge- based vulnerable open data sources
detection tool, for non-intrusive testing of open data sources for detecting their vulnerabilities. ShoBeVODSDT:
supports the identification of vulnerabilities at early security assessment stages and does not require the
implementation of active and possibly disruptive techniques;
uses two IoTSE (Shodan and Binary Edge) by extending their features with the advanced capabilities built
in it;
allows inspecting 8 predefined data sources - MySQL, PostgreSQL, MongoDB, Redis, Elasticsearch,
CouchDB, Cassandra and Memcached, on their vulnerabilities and their extent.
While the tool covers 8 data sources representing both rational databases, NoSQL databases and data stores, it is
designed to be easily scalable by extending the publicly available code  https://github.com/zhmyh/ShoBEVODST
https://www.eosc-hub.eu/open-science-info

12. RESULTS AND CONCLUSIONS II
The total number of open data sources available to everyone (who wants to access them) is not very high, i.e. less than 2% of
the data sources scanned.
BUT, there are data sources that may pose risks to organizations, since external users can access the information that can be
used for further attacks. For 12% of ispected data sources this has already taken place.
Security features built into the database allow to protect against unauthorized access, but there are databases with low
security features, where we were able to connect to nearly all IP addresses by retrieving information from them. Even more, in
some cases the databases, which do not use security mechanisms, have been already compromised.

13. THANK YOU FOR
ATTENTION!
QUESTIONS?
For more information, see ResearchGate
See also anastasijanikiforova.com
For questions or any other queries, contact
me via email - Anastasija.Nikiforova@lu.lv