Presentation by Parwinder Singh Technical Specialist, NEC Technologies India Pvt Ltd. FIWARE Global Summit 23-24 October 2019 - Berlin, Germany

1. The Scorpio NGSI-LD Broker: Features and Supported
Architectures
Parwinder Singh
NEC Technologies India
Martin Bauer
NEC Labs Europe

2. 2 © NEC Technologies India Private Limited 2019
Agenda
▌Introduction
▌Architecture – Layered View
▌Architecture Implementation Considerations
▌Scorpio Architecture – An Implementation View
▌System Functional Flow
▌Security Architecture
▌Federation Architecture – High Level View
▌Federation Architecture – Implementation View
▌Feature Highlights
▌Technologies Context Used
▌Summary

3. 3 © NEC Technologies India Private Limited 2019
Introduction
▌NGSI-LD API specifications approved by ETSI GS CIM
l It is in public domain since last year and latest version is ETSI GS CIM 009 V1.2.1.
▌NGSI-LD provides APIs for Context Information Management that
l Enable close to real-time access
l Based on Core Meta Model, Domain & Cross Domain specific ontologies driven data model
l Enable semantic modelling & many other features ....
▌Fast adoption of NGSI-LD specs require
l A complete understanding of standards.
l Designing, development & knowledge transfer.
l Support of all functional non functional requirements.
▌NEC has achieved the first reference implementation of NGSI-LD specs worldwide.
l We refer it as “Scorpio Broker“.
l Microservices oriented architecture that enables flexible deployment and scalability.
l Support of different deployment architectures.
l Integrated support for temporal operation.
l Efficient support for geo-queries.
l Robust implementation adhere to best practices & latest technology trends of industry.

4. 4 © NEC Technologies India Private Limited 2019
Storage
Manager
Entity
Manager
LD Context
Resolver
Subscription &
Notification Manager
History
Manager
Context
Registry
Storage
NGSI-LD Compliant Interfaces
(REST/HTTP/BinaryTCP Call)
Applications/ IoT Entities/
Consumers/Producers
Query
Manager
NGSI-LD
Clients
Service
Interfaces
(External/Internal)
Operations
Management
Resource
Connectors
(Local/Remote)
Resources
(Local/Federated)
Authentication &
Authorization
Service API
Gateway
Service Discovery
& Registration
Context Source
(Broker)
Context Provider
[Message Bus with Realtime Data Persistence]
Storage
Connector
Context Registry
Connector
Context Source or
Provider Connector
Health Check
& Monitoring
Context Registry
Manager
Scorpio Architecture – Layered View
NGSI-LDETSICompliantAPIsScalableSecure
MicroservicesorientedFaulttolerantExtendable

5. 5 © NEC Technologies India Private Limited 2019
Architecture Implementation Considerations
Monolithic Architecture Microservices Architecture
• Large and complex
• Scalable issues
• Huge changes
impact
• Adaptability &
Extensibility issues
• CI/CD difficulties
• Decomposed independent manageable services
• Parallel development & deployment
• Easy to upgrade, scale & fault isolation per
microserviceDrawbacks
HTTP
Rest
Highlights
Common Issues
• Complexity due to distributed in nature
• Multiple databases and data synchronization
challenges
• Multiple service endpoints exposed & related
management overhead
• Monitoring & inter service communication
overhead
v
• API Gateway acts as proxy for all
registered Microservices
• Inter Microservices communication
can be rest or on some message
channel based
• Implementation can be designed
through message channel to use
common DB
Highlights
Inter Service Communication Channel

6. 6 © NEC Technologies India Private Limited 2019
METHOD: POST/GET/DELETE/PATCH
/entities /subscription /csources /csourceSubscription
HTTP REST APIs (http://ip:port/ngsi-ld/v1/<operation>)
ServiceRegistration&
Discovery
Entity Manager
Subscription
Manager
Query
Manager
Context Registry
Manager
Topic (Consists of Partitions & Offsets)
DB
Discovery
Health Check &
Monitoring
Service API Gateway
Microservices
Spring Cloud Services
Kafka Cluster
Pub/Sub calls to Kafka
Storage
Manager
Scorpio Architecture – An Implementation View

7. 7 © NEC Technologies India Private Limited 2019
HTTP REST APIs (http://ip:port/ngsi-ld/v1/<operation>)
ServiceRegistration
&Discovery
Entity Manager
Subscription
Manager
Context Registry
Manager
CR Topic
Entities
Topic
Discovery
Health Check &
Monitoring
Service API Gateway
Subscription
Topic
Create/Update/
Append TopicE1
E1
S(E1)
M(E1)
DB
5
1
2.2
3
4.1 4.2 5.1
2.1
4.3
E1
2
4.4
System Functional Flow
5.4
5.3
6
METHOD: POST/PATCH
/entities,
/entities/{entityId}/attrs
Query
Result Topic
Query Topic
5.3
Query
Manager
Storage
Manager
/ GET
Microservices
Spring Cloud Services
Kafka Cluster
Pub/Sub calls to Kafka
5.5

8. 8 © NEC Technologies India Private Limited 2019
Federation Architecture – High Level View

9. 9 © NEC Technologies India Private Limited 2019
Context
Registry
Federation Scope at Broker-A
GeoCoverage – GL1 GeoCoverage – GL2
Data Query(type “T“ with Geo-scope) Registry
Manager
Broker-A (Higher in Hierarchy)
Note: Borker-A & B are similar in terms
of components. Broker B is shown in simplified
view for better clarity
Scorpio
Broker-B
(Lower in
Hierarchy)
Context
RegistrySubscription/
Notification/
Query Data for GL2 scope
Broker B has type data “T“
at Geolocation “GL2“ scope
Broker A has type data “T“
at Geolocation “GL1“ scope
Csource Registration
Scorpio Broker
(Entity,
Subscription,
Query, Storage)Aggregated Query Response
Federation Architecture – Implementation View

10. 10 © NEC Technologies India Private Limited 2019
Feature Highlights
▌Microservices Design Orineted
▌Flawless interservices communication
▌NGSI-LD driven real time data context processing
▌Federation Support
▌Advanced & Temporal Query Support
▌History Support
▌Generic Security Framework(Keyrock & keycloak)
▌OSS driven implementation
▌Proper Documentation
▌Different storage capability
▌Comply all non functional requirements
▌Easy to integrate with 3rd party applications
▌Unit Testing Embedded modules
▌Postman Driven Testing API Scripts
Scorpio
Broker

11. 11 © NEC Technologies India Private Limited 2019
Scorpio is a FIWARE Incubated GE
▌Scorpio is now Open Source Software
▌Licensed under BSD 4 Clause License
▌Easy docker container generation from source via Maven integration
lProfile for individual micro service docker containers
lProfile for an all in one container with all Scorpio micro services
ldocker-compose files available for easy setup with Postgres and Kafka
▌Docker container also available at docker
https://hub.docker.com/r/scorpiobroker/scorpio
▌Source code is available at
https://github.com/ScorpioBroker/ScorpioBroker

12. 12 © NEC Technologies India Private Limited 2019
Technologies Context Used

13. 13 © NEC Technologies India Private Limited 2019
Summary
▌First reference implementation of NGSI-LD APIs (as Scorpio Broker).
▌Implementation has followed best practices & adhere to latest technologies.
▌All non-functional concerns have been addressed at design & development phases.
▌Federation support is implemented to provide federation among multiple Brokers with
different geo-locations.
▌“Scorpio” will enable fast adoption of NGSI-LD standards in the market.
Next Steps:
▌Functional advancements, Rigorous Testing & Performance benchmarking.
▌Non-Functional & Security aspects hardening.
▌Pre-onboarded Use cases demonstration.
▌Hybrid Cloud enabled Scorpio API services for live experience by anyone.

14. Thanks…
Contact:
Email: parwinder.singh@india.nec.com
***Ask for Scorpio demo at NEC booth

15. BACKUP SLIDE

16. 16 © NEC Technologies India Private Limited 2019
Discovery
Entity Manager
End User Requests HTTP/REST APIs
Subscription
Manager
ServiceRegistration
&Discovery
Auth.Service
Kafka Cluster
DB
Other
Microservices..
Token Validation
JWT Token
Refresh/Revoke
Resource Access with Token Relayed
Authentication/
Redirect
Resource
Rendered
User Authorization/Code/
Grant/Permissions /Redirect
Security Architecture
RedirectionRedirect
Service API Gateway
Intercept/Filter/SSO/Forward/Secure Context
1
2
3
4
5
6
78
9
Resource
Request
Microservices
Spring Cloud Services
Kafka Cluster
Pub/Sub calls to Kafka
*SSO – Single Sign On

17. 17 © NEC Technologies India Private Limited 2019
Microservices Realization Perspective

18. 18 © NEC Technologies India Private Limited 2019
Broker 4 area
Query area (a,b,c,d)
Broker 3 area
Broker1 Area
Broker2
Area
Distance
Federation Query Handling
Relational Query Area (at distance 8KM)
Query Example: Near Min distance
?type=Vehicle&georel=near,minDistance==8000&geometery=Polygon&coordinates=[a,b,c,d]
Overall Geo scope Boundary for federation
Disjoint Area of Three
Brokers
B4 Area contains B3
area & also have
intersection area with
B1
Output:
Entities associated to
type vehicle lies in all
Brokers area that are in
scope of federation is:
- Broker1
- Broker2