Stephen Cantrell, kdb+ Developer at Kx Systems “Kdb+: How Wall Street Tech can Speed up the World" You can see some additional notes here: https://github.com/cantrells/berlin_kdb_demo?files=1

1. HOW WALL STREET TECH CAN SPEED UP THE WORLD
STEPHEN CANTRELL – 13th JULY 2017
TECH MEETUPS BERLIN

2. • Widely adopted in financial services over two decades
• Now in - Hi-TechManufacturing, Utilities, Telco, Energy, Life Sciences,
Earth Observation
• Software & industry solutions, consulting and implementation services
Known for…
• Processing & analysis of large volumes of real-time and historical
time series data
• Extreme performance - low latency
• Integrates with and co-exists with other technologies
• Scalability without requiring significant infrastructure change
• Provide the fastest, most efficient, most flexible tools and dashboards
• Worldwide leader in high-volume, high-performance databases
13 Global
Offices
Inc. NYC, Singapore,
London & Tokyo
1800
Employees
and growing…
Subsidiary of
First Derivatives
plc
Listed on LSE & AIM
ABOUT US

3. 3
120+ CLIENTS

4. OTHER VERTICALS
Health & Life Sciences
Automotive
Utilities
Space and Telco
Retail
Earth Observation
Geospatial Data Analytics
Anomaly Detection
Genomics Data Processing
Connected Health
Patient Record Analytics
Telemetry Analytics
Anomaly Detection
Performance Analytics
Sensor Analytics
Smart Meters Data
Predictive Analytics
Manufacturing
Retail Analytics
Marketing Optimization
Customer Journey
Edge Computing
Multivariate Analysis
Fault Detection

5. kdb+ column based time-series database
with in-built programming language q

6. CORE TECHNOLOGY
• World’s fastest time-series columnar database
• Streaming, real-time and historical data in one platform
• Runs on Linux, Windows, Solaris, and MacOS
• Runs on commodity hardware, cloud, edge devices/appliances
• Expressive query (qsql) and programming language (q)
• In-memory compute engine for Complex Event Processing
• Column-level compression and sensor data noise filtering
• Integrates easily into legacy systems for performance augmentation
• Multi-core / Multi-processor / Multi-thread / Multi-server

7. Kx PERFORMANCE SNIPPETS
Streaming Process and store 4.5 million bulk events/second/core
Scan Search in-memory tables at 4 billion records/second/core
Batch Ingest data at 10 million records/second/core
Store Accumulate 10 trillion data points (3 PB) for NYSE
Usage Trusted by 17 of the 20 world’s top investment banks
Volume Daily volumes of 1.6 TB of streaming data per day
Scale From Raspberry Pi, edge devices, to 20,000 cores on AWS Cloud
Performance Top performing time-series database according to STAC Research
Footprint Tiny 500 KB memory profile (exploits L1/2 Cache)
Latency Sub millisecond latency for streaming event processing

8. FROM CHIP
TO CLOUD

9. File & DB
Sources
File & DB
Sources
Scalability, High Availability, and Fault Tolerance
Native Lambda/HTAP Architecture
Stream for Kx Application Framework
Kx for Flow
Kx for Surveillance
Kx for Algo
Kx for DaaS
Kx for Utilities
Kx for Cyber
Kx for Pharma
Vertical
Market Solutions
Kx for Sensors
Kx for Telco
Real-Time
Sources
Core (kdb+)
In Memory
Database
Historical
Database
In Memory
Database
Historical
Database
q language &
qsql scripting
Develop, configure,
deploy, and manage
solutions
Control for Kx Monitor for Kx
Scan, monitor and alerting
of issues in software and
hardware
Third-Party Interoperability
Pub/sub, SOA, ODBC, JDBC, web sockets
R, Python, MatLab,
Java, C#, C/C++
Analyst for Kx
Query, explore, transform,
and import without
programming
Dashboards for Kx
Build real-time
visualizations for multiple
devices
File & DB
Sources
Batch
Loader
Batch
Loader
Stream
Feed
Handler
Stream
Feed Handler
Stream Engine
Ticker Plant
Stream and
Ingestion
Engine
Complex Event
Processing
• Queries
• Transforms
• Alerts
• Control signals
• Notifications
• Micro-services
Kx ARCHITECTURE

10. ENTERPRISE
INTERFACES
ODBC/
JDBC
Web
Services
Python &
Perl
R & Matlab
TCP Sockets
& Web
Sockets
C#/.NET
Java/Scala
C/C++
CDC

11. STREAM
CONTROL
ANALYST
DASHBOARDS
Dashboards for
Kx provides rich
visualization of
real-time
streaming,
intraday and
historical
business data.
Analyst For Kx
provides a
complete real-
time data
transformation,
exploration and
discovery
workflow.
Control for Kx
allows
organizations to
manage their
application
landscape, instantly
determining what
processes run, who
owns them, their
parameters and
functionality.
Stream for Kx is a
proven data
management
solution that can
be deployed
locally, in the
cloud, or as part of
a wider Big Data
architecture.

12. DEMO 2 – NYC MOTOR VEHICLE COLLISONS
• Data is available at: https://data.cityofnewyork.us/Public-
Safety/NYPD-Motor-Vehicle-Collisions/h9gi-nx95
• Over 1 million rows of historical data between 2012 and 2017
• Weather Data Available at:
https://www7.ncdc.noaa.gov/CDO/dataproduct (up to 2013)
• Daily Summary Data for a station located in Central Park

13. kdb+ Demo
Loading data from a source
Or if you prefer k

14. Reading data into a kdb+ session

15. Accident Black Spots

17. Month by month comparison

19. Accidents due to slippery pavement

20. Saving your data

21. Benchmarks
o STAC benchmarks: https://stacresearch.com/kx; includes independently
verified benchmarks of the technology using common capital markets use
cases.
o Intel solution brief:
http://www.intel.com/content/www/us/en/processors/xeon/real-time-financial-
analysis-with-kx-systems-brief.html
o Gartner paper on Kx technology: https://kx.com/gartner-download.php
o Community
o Kx Wiki: http://code.kx.com/wiki/Main_Page
o Kx Community: http://kxcommunity.com/
o Kx Github: http://kxsystems.github.io/
RESOURCES

22. 14
Kx TECHNOLOGY FUND
Int. Office
Access
£10 Mil
Funding
Kx Tech
Access
Sales &
Marketing
Support
Logistical
Support
Learn more at www.kx.com/kx-technology-

23. 18
RECRUITMENT
o Opportunities to travel & live
internationally
o Paid for city accommodation
o Competitive salary and expenses
package
o Health & travel insurance programme
o Generous retirement savings
programme

24. CONTACT
Stephen Cantrell
Kx Engineer
scantrell@kx.com
https://kx.com/
https://kx.com/download/
http://code.kx.com/qref/
Free 32 bit download