Overview of some kdb+ and R integration methods.

1. KDB+ AND R INTEGRATIONRORY WINSTON

2. KDB+ AND R INTEGRATION
AGENDA
▸ Why integrate R/kdb+
▸ Basic R integration via CSV import/export
▸ R integration via qserver
▸ Integrating R math functionality into kdb+
▸ Generating kdb+ data with R

3. KDB+ AND R INTEGRATION
WHAT IS R?
▸ DSL For Statistical Techniques and
Data Analysis
▸ Integrated programming and EDA
(Exploratory Data Analysis)
environment
▸ Open-Source
▸ Thousands of contributed
packages

4. KDB+ AND R INTEGRATION
WHY USE R?
▸ Comprehensive Environment
▸ Tooling
▸ Command-Line
▸ IDE
▸ Web
▸ Thousands of Packages
▸ Rich Reporting/Graphing/Analysis Capabilities

5. KDB+ AND R INTEGRATION
PART ONE: CSV IMPORT/EXPORT
▸ CSV import/export
▸ In kdb+:
▸ save `:table.csv
▸ In R:
▸ read.csv(‘table.csv’, header=TRUE)

6. EXAMPLE #1
CSV IMPORT/EXPORT

7. KDB+ AND R INTEGRATION
CSV IMPORT / EXPORT
▸ Simplest approach
▸ Interchange with any other system
▸ No type information
▸ Manual overhead
▸ Can be a slow/laborious process

8. KDB+ AND R INTEGRATION
PART TWO: THE QSERVER LIBRARY
▸ Available from
▸ http://code.kx.com/wsvn/code/cookbook_code/r/
▸ Consists of:
▸ A native library containing kdb+ connectivity
▸ An R wrapper
▸ Binaries for Windows/Linux/OSX

9. KDB+ AND R INTEGRATION
THE QSERVER LIBRARY
qserver
kdb+layer
Rlayer

10. KDB+ AND R INTEGRATION
BASIC QSERVER OPERATIONS
▸ Connect to a kdb+ instance:
▸ open_connection( host, port, credentials )
▸ Returns a handle
▸ Execute a query:
▸ execute( handle, query )
▸ Close connection:
▸ close_connection( handle )

11. EXAMPLE #2
QSERVER

12. KDB+ AND R INTEGRATION
QSERVER INTEGRATION
▸ Most Efficient
▸ Can Manage Multiple Connections
▸ Can Be Wrapped Into An R Library
▸ Easy to Extend With Custom R Code

13. KDB+ AND R INTEGRATION
PART THREE: LEVERAGING THE R MATH LIBRARY
▸ R Comes With A Standalone Math Library
▸ Contains A Large Range Of Base Statistical/Math Routines
▸ E.g. Base Probability Functions/RNG
▸ Can Be Packaged Into a Library And Reused in kdb+
▸ I have packed a library and a set of routines on:
▸ https://github.com/rwinston/kdb-rmathlib

14. KDB+ AND R INTEGRATION
PART THREE: LEVERAGING THE R MATH LIBRARY
▸ Example In R:
▸ rnorm(1000,5,2)
▸ Generates 1000 random variables with a mean of 5 and
standard deviation of 2
▸ In q, using the rmathlib integration:
▸ norm[1000;5;2]
▸ R’s math library contains hundreds of useful base functions

15. EXAMPLE #3
RMATHLIB

16. KDB+ AND R INTEGRATION
USING R TO GENERATE Q CODE
▸ Generate q code from R
▸ Similar in principle to CSV export
▸ See
▸ http://www.theresearchkitchen.com/archives/776

17. EXAMPLE #4
Q CODE GENERATION

18. THANKS