An overview of how TIBCO integrates dynamic, interactive visual applications in Spotfire with predictive and advanced analytics in the R language, using TIBCO Enterprise Runtime for R--our R-compatible, enterprise-grade platform for the R language.

1. Extending the Reach of R
to the Enterprise
Lou Bajuk-Yorgan
Sr. Dir., Product Management
TIBCO Spotfire
1
© Copyright 2000-2014 TIBCO Software Inc.

2. Extending the Reach of R to the Enterprise
• TIBCO, S+, and embracing R in Spotfire
• Challenges of R for Enterprise applications
• TIBCO Enterprise Runtime for R (TERR)
• Benefits for organizations (and individuals) who use R
• Examples of TERR integration and performance
• Learn more and try it yourself
-2

3. Our Journey to TERR
•
John Chambers developed the S language at Bell Labs
– Starting in the mid 70’s
•
Insightful (Statsci) founded to commercial S as S+ in 1987
– The “plus”: statistical libraries, documentation, and support
– Later focus on commercial users, ease of use, server integration
•
R: development begun by Ross Ihaka and Robert Gentleman at University of
Auckland in mid 90’s
•
Insightful acquired by TIBCO in 2008
– Spotfire (for Data Discovery and Visualization) acquired in 2007
•
Focus shifted to applying Predictive Analytics in Spotfire
– Step 1: Embrace R
-3

4. Predictive Analytics with Spotfire
Easily provide targeted, relevant predictive analytics to business users to
improve decision making
•
Ensure compliance & proper usage
•
Share best practices and consistent workflows
•
Get the answer & do “What If?” analyses when needed
•
Leverage investments in R, S+, SAS, MATLAB, …
Powerful Predictive Analytics tools for Spotfire analysts
•
Integrated into Spotfire workflows
•
Easily create, evaluate, and share Predictive Models
•
Add Forecasts with a single click
Benefits of Predictive Analytics to a spectrum of users
•
Increase confidence & effectiveness in decision-making
–
–
–
Reduce uncertainty
Discover meaningful patterns, important data
Maximize ROI
•
Anticipate and react to emerging trends
•
Reduce/manage risk
–
•
Scenario planning, forecasts, fraud detection
Forecast specific behavior, preemptively act on it
–
Increase upsell, decrease churn

5. Embracing R
•
Spotfire Statistics Server
–
Integration of R & S+ into Spotfire
applications
•
–
Later added SAS® & MATLAB®
Leverage the interactive visualizations
of Spotfire
•
Contribute to the R community
•
Well received—but our Enterprise
customers need more
–
–
-5
R provides tremendous benefits to
statisticians
But large enterprises are often
challenged to leverage that value

6. Enterprise Challenges for R
•
Core R engine struggles with Big Data
–
–
•
R was not built for enterprise usage and integration
–
–
•
Built as an academic tool for research and teaching
Software vendors attempting to use R in ways it was never intended
GPL great for statisticians, but limits enterprise innovation and investment
–
–
•
Customers don’t use R, or reimplement R code in specialized libraries or other languages
Lose agility & consistency, delay time to production, lose opportunities
Viral open source licensing risks commercial IP
Large vendors avoid tight integration due to open source concerns
Free to acquire, but costly to maintain
–
–
Version incompatibilities, variable quality in packages
Lack of enterprise-level technical support
6
© Copyright 2000-2013 TIBCO Software Inc.

7. TIBCO Enterprise Runtime for R (TERR)
•
Unique, enterprise-quality implementation of the R language
–
Fundamentally different
•
•
•
–
TIBCO IP: Not open source/GPL
•
•
•
–
Independent implementation
Licensable for embedding and redistribution by partners
Enables implementation of transparent big data handling
Broad compatibility with R functions and 1400+ CRAN packages
•
•
New architecture, developed from the ground up
Based on our long history and expertise with S+
Faster, more robust and more memory-efficient than R
Ongoing effort to broaden our coverage of R
Extends the Reach of R to the Enterprise
–
–
–
Develop in R, deploy on TERR
Rapidly iterate prototyping to production without recoding/retesting—more rapidly respond to
changing business conditions
Easily integrate R-language analytics consistently across organization—into grids, BI
applications, event-driven analytics, etc.
© Copyright 2000-2013 TIBCO Software Inc.

8. Leveraging TERR
TERR in Spotfire
TERR in
Statistics Services
Embeddable
TERR Engine
Ad hoc tools and interactive applications powered by advanced analytics
• Spotfire Analytics platform: interactive visualization & data discovery,
easily build and share applications, broad data access, etc.
Distributed analytics
• Managed pools of engines
• Load balancing, queuing, failover, parallelization, etc.
• High level APIs for loose integration, data i/o (C#, Java)
• Central management of analytics, R packages
Custom (tight) integration, batch, existing grids, etc.
• Faster than R, more robust, better memory management, fully
supported
• Low level APIs for tight integration
• Integrated into TIBCO products: CEP, Cloud Compute, …
© Copyright 2000-2013 TIBCO Software Inc.

9. Providing Value for individuals who use R
•
Not seeking to displace R from statistician’s
desktops
–
•
Contribute to the R community
–
–
•
As we port from S+ or develop for TERR
• Supports “Develop in Open Source R, Deploy
on TERR”
• E.g., splusTimeSeries, splusTimeDate, sjdbc
TERR Developer Edition
–
–
–
-9
Sponsor useR conferences, contribute to R
Foundation
Contribute bug reports and propose fixes to R core
Contribute packages to CRAN
–
•
Enterprise platform for the deployment and
integration of your work—without having to rewrite
it!
Full version of TERR engine for testing code prior to
deployment
• Compatible with RStudio & ESS Emacs
Free for non-production use
Supported through Community site

10. Example 1: TERR vs. R Raw Performance
One specific example
• Non-optimal, non-vectorized, real-world R script
• For loop with row by row processing
for (i in seq(1,length=nrow(df))) {
…process each customer record…
}
Results
• TERR is ~35x faster for 50K rows, 150x faster for 500K rows
• No code modification required
We are looking for more real-world performance tests!
• On average 2-10x faster than R in microtests

11. Example 2: Spotfire Forecast Tool
•
Forecast Tool
– Easily add Forecasts to
Visualizations by right click menu
– Advanced users can tune settings
– Uses embedded TERR engine
•
Benefits
– Extend the power of Predictive
Analytics for ad hoc analysis to all
Spotfire users
– Easy entry point to Spotfire
Predictive Analytics

12. TERR integration with TIBCO StreamBase
•
Event-Driven analysis in TIBCO Spotfire
Event Analytics
–
•
Apply predictive models in real-time
decision making
–
–
–
–
•
Process monitoring, analysis, and
optimization
Best marketing offer
Customer churn
Predictive Maintenance
Yield optimization
Rapidly develop and iterate models in
production
–
Respond to changing opportunities
and threats
12

13. TIBCO Cloud Compute Grid
•
High performance computing on the cloud
– Available on TIBCO Cloud Marketplace
– TERR, Java and .NET computations
•
Robust DataSynapse GridServer architecture
– Used by Wall Street to manage 10K’s nodes
– Java, .NET, and REST APIs (JSON)
•
Perfect for pure computational work
– Vastly easier to use for applications like Monte Carlo
simulations than Map-Reduce
– Run complex statistical models multiple orders of
magnitude faster than open source R on a single
computer
– Unparalleled scalability without upfront capital
investment
•
Easy to get started
– Uses your Amazon EC2 account

14. Demos
• TERR in Spotfire
– Fraud Detection Application
– Data Functions: using the R language in Spotfire
– Forecast Tool

15. Learn more and Try it yourself
•
TERR Community at TIBCOmmunity.com
–
–
–
–
•
TERR Developer Edition
–
–
•
Full version of TERR engine for testing code prior to deployment
Supported through TIBCOmmunity, download via tap.tibco.com
TIBCO Cloud Compute Grid
–
•
Resources, FAQs, Forums
Details of R coverage
Product documentation & download
More info at spotfire.tibco.com/terr
https://marketplace.cloud.tibco.com
We want your feedback and input!
–
–
–
Real world performance tests
Package & R coverage prioritization
Via TERR Community, or contact me lbajuk@tibco.com or @loubajuk