Left Brain, Right Brain: How to Unify Enterprise Analytics

Welcome

Host:
Eric Kavanagh
eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr The Briefing Room

Mission

!   Reveal the essential characteristics of enterprise software,
good and bad

!   Provide a forum for detailed analysis of today s innovative
technologies

!   Give vendors a chance to explain their product to savvy
analysts

!   Allow audience members to pose serious questions... and get
answers!


JANUARY: Big Data

February: Analytics

March: Open Source

April: Intelligence


Big Data

Twitter Tag: #briefr
NEW SOURCES
New Insights
NEW
Challenges

The Briefing Room

Copyrighted property. May not be copied or downloaded without permission from 123RF Limited.

Analyst: Robin Bloor

Robin Bloor is
Chief Analyst at
The Bloor Group

robin.bloor@bloorgroup.com


Teradata Aster

!   Teradata is known for its data analytics solutions with a
focus on integrated data warehousing, big data analytics
and business applications

!   It offers a broad suite of technology platforms and
solutions; data management applications; and data mining
capabilities

!   Teradata Aster is its MapReduce platform to handle big data
analytics on multi-structured data


Steve Wooledge

Steve is Senior Director
of Product Marketing for
Teradata Aster and
has 10 years of industry
experience.


Bringing Big Data into the Light:
Teradata Big Analytics Appliance
Steve Wooledge – Sr. Director, Product Marketing, Teradata Aster

January 2013

TOPICS

WHAT IS DIFFERENT ABOUT BIG DATA ANALYTICS?

MAKING BIG ANALYTICS & DISCOVERY FAST AND EASY

TERADATA ASTER BIG ANALYTICS APPLIANCE

Confidential and proprietary. Copyright © 2012 Teradata Corporation.
10 Confidential and proprietary. Copyright © 2012 Teradata Corporation.

What is Different about
Big Analytics and Discovery?


The Lytro and Big Data


“Interactive, Living Pictures”


See Your Business in High-Definition
Big Analytics & Discovery Unlocks Hidden Value

Classic BI
Structured & Repeatable Analysis

Business determines what IT structures the data to
questions to ask answer those questions

“Capture only what’s
needed”

IT delivers a platform for Big Data Analytics
storing, refining, and Multi-structured & Iterative Analysis Business explores data for
analyzing all data sources questions worth answering

“Capture in case it’s
needed”

Iterative Analytics Accelerates Discovery

Analytical
Idea

Operational DB
or EDW
Operationalize
or Move On
5x Zero-ETL Data
Load/Integration

Faster
Discovery Process
with Aster -
Evaluate vs. Days
Hours
Results SQL and non-SQL
Analysis


Need for a Unified Data Architecture for New Insights
Enabling Any User for Any Data Type from Data Capture to Analysis

Java, C/C++, Python, R, SAS, SQL, Excel, BI, Visualization

Reporting and Execution
Discover and Explore
in the Enterprise

Capture, Store and Refine

Audio/ Web & Machine
Images Docs Text CRM SCM ERP
Video Social Logs


Big Data Comes with BIG HEADACHES

“ Even free software like Hadoop is causing
companies to spend more money…Many CIOs believe
data is inexpensive because storage has become
inexpensive. But data is inherently messy—it can be
wrong, it can be duplicative, and it can be irrelevant—
which means it requires handling, which is where the
real expenses come in.

”
“ Through 2015, 85% of Fortune 500 organizations will
be unable to exploit big data for competitive advantage.

Source: The Wall Street Journal. “CIOs’ Big Problem with Big Data”. Aug 2012
Source: Gartner. “Information Innovation: Innovation Key Initiative Overview”. April 2012
”

UNIFIED DATA ARCHITECTURE
Data Scientists Quants Customers / Partners Front-Line Workers
Engineers Business Analysts Executives Operational Systems

LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS

Big Data Analytics DISCOVERY INTEGRATED
PLATFORM DATA WAREHOUSE

Big Data Management
CAPTURE | STORE | REFINE

AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP


TERADATA UNIFIED DATA ARCHITECTURE

LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS

DISCOVERY INTEGRATED
PLATFORM DATA WAREHOUSE


AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP


TERADATA UNIFIED DATA ARCHITECTURE

VIEWPOINT LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS SUPPORT

DISCOVERY Aster Teradata INTEGRATED
PLATFORM Connector DATA WAREHOUSE

Aster Connector for SQL-H SQL-H Teradata Connector
Hadoop for Hadoop

Aster Loader Teradata Loader

20 ConfidentialVIDEOproprietary. Copyright © 2013 Teradata Corporation.
AUDIO & and IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP

Shift from a Single Platform to an Ecosystem

“Big Data requirements are solved by a range of
platforms including analytical databases, discovery
platforms and NoSQL solutions beyond Hadoop.”

Source: “Big Data Comes of Age”. EMA and 9sight Consulting. Nov 2012.


How Does Big Analytics and Discovery
Add Business Value?


Customer Behavior Analysis

BI Tools Database Tools Monitoring Tools

EMAIL ONLINE STORE VISION
BRANCH CALL CENTER CUSTOMER CUSTOMER
CORRESPOND- BANKING PLATFORM
TELLER DATA DATA PROFILE DATA SURVEY DATA
ENCE DATA DATA DATA


Events Preceding Account Closure


Events Preceding Account Closure

SELECT * FROM npath (
ON (
SELECT …
WHERE u.event_description IN (
SELECT aper.event FROM attrition_paths_event_rank aper
Interactive Analytics
ORDER BY aper.count DESC LIMIT 10)

…
)
Reducing the “Noise”
to find the “Signal”
PATTERN ('(OTHER|EVENT){1,20}$')
SYMBOLS (…) RESULT (…)
)
) n;


How Do We Make
Big Analytics & Discovery Possible?


Key Requirements of a Discovery Platform

Highly Efficient & Performant Big Data Platform
1 That Allows Quick Iterations

Hybrid Capabilities that supports SQL, statistics,
2 and new MapReduce analytics

Significant Out-of-the-Box Analytical Functions
3 that Minimize Development

Democratize Big Data & Maximize Enterprise Adoption


Teradata Aster Big Analytics Appliance
First Deeply Integrated SQL, MapReduce and Hadoop Appliance
UNIQUE FEATURES

1.  Integrated, modular Aster Database and 100% Open-Source
Hortonworks HDP
2.  First and only ANSI SQL & HCatalog integration via SQL-H™
3.  Industry’s only ANSI-standard SQL & MapReduce integration
via SQL-MapReduce®
4.  Industry’s most manageable & supportable Apache Hadoop
appliance via Teradata Viewpoint™ & TVI™
5.  Most complete MapReduce App Portfolio with 70+ pre-built
MapReduce functions
6.  Fully engineered and supported by Teradata, with Level-4
support by Hortonworks world-class Hadoop team

Benefits
•  Leverage existing investments in standard BI, ETL tools & people with SQL skills
•  Industry’s highest performance platform for Big Analytics
•  Lowest TCO (technology + people), highest ROI, and fastest time to value


Teradata Aster Analytics Portfolio
The App Store of Big Data

PATH ANALYSIS TEXT ANALYSIS
Discover Patterns in Rows of Derive Patterns and Extract
Sequential Data Features in Textual Data

STATISTICAL ANALYSIS
High-Performance Processing SEGMENTATION
of Common Statistical Discover Natural Groupings
Calculations of Data Points

MARKETING DATA
ANALYTICS TRANSFORMATION
Analyze Customer
Transform Data for More
Interactions to Optimize
Advanced Analysis
Marketing Decisions


Unified Big Data Analytics Architecture
Integrated Analytics and Navigation
BI Tools, SQL, ETL

BIG ANALYTICS
TERADATA IDW
APPLIANCE
Unified Big Analytics
Architecture

Behavior Sentiments

Multi-
Discovery
Structured Platform
Data

Facebook
Unstructured
Twitter
Data
Pinterest

Social
Revenue
Media

Iterative
Operationalized Best Decision
Information
Analytics Possible
Discovery


Teradata Aster Big Analytics Appliance
Solution Value Add

NEW •  Single vendor for lowest TCO
SQL BI Analytic Hadoop
•  Common system management tools
Tools SQL Apps Tools
Troubleshooting, and Support
Common Management,

Aster MapReduce •  Supports standard BI and ETL tools
Portfolio of Functions •  Use Hadoop tools like Hive and Pig
Hive, Pig,
SQL-
SQL MapReduce …
•  Analytics Library w/ 70+ functions
NEW SQL-H •  SQL interface to MapReduce and
Hadoop

•  Pre-tuned HDFS and MapReduce
parameters for Big Data workloads
Aster Database •  Store and manage data in Apache
Hadoop or Aster Database

NEW InfiniBand (40 GB/s) Interconnect Fabric •  Processing, storage, and networking
designed for Big Data workloads
Big Analytics Appliance Hardware •  40 GB/s InfiniBand network
NEW


ESG Benchmark Report Summary
Third Party Validation of Aster and Hadoop “Fit”
Scope
•  Identical hardware for Aster and Hadoop
•  Clickstream, sentiment, and traditional retail data
•  Compare “time to insight” and “time to develop”

RESULTS
Discovery Process: Analytics: Development: Loading: Transforms:
Aster Aster 35x Faster Aster Hadoop Hadoop
5x Faster (range: 4–416x) 3x Faster 1.8x Faster 1.3x Faster

Hadoop MapReduce 32 Hours Aster 5x Faster
Discovery Cycle-Time
Aster SQL-MapReduce 6 Hours (Development + Execution Time)

FULL REPORT AVAILABLE AT www.asterdata.com/esg


Comparing Advanced Analytic Development
and Execution
Example: Determine Spikes In Hourly Pageviews
Apache Hadoop Teradata Aster
“By using SQL-MapReduce, “This is also why the execution
Asterand find allMRfewerpageviews/hr to
1 takespages <100 steps pagename
•  Write Java job to group records by
time•  in Aster nPath is SQL as regular expressions
1 •  Input parameters in much faster.”
Use
Aster
developbyanalytics” hour fields
•  Sort the yy/mm/dd and
•  Single Pass of the data
•  Java reduce phase to place all same-keyed
records into temporary arrays •  SQL handles group-by, counts, sorts
2 Execute
•  MapReduce perform regular pattern matching
•  Compute counts for low/high/low hourly page
views over a sequence of rows

“Rather than using MapReduce “Map•  or Reduce relational table data
3+
•  Create custom partitioner
3
Outputs written to requires
•  Create custom grouping comparator
processing for each step in the shuffling andtools to visualize results
•  Create custom key comparator
•  Use SQL or BI
produces higher
analysis, SQL is used in place latency than SQL”
Execute •  Execute each Mapper and Reducer
of a•  Map (or Reduce) phase
Multiple passes of data

and•  MapReduce is used only in
Save output to flat files making it unstructured,
stepsDB interfaces (e.g. ODBC/JDBC)
5
that cannot be
•  No relational semantics and preventing use of

expressed in with other tools (e.g., SSH/FTP)
•  Retrieve results SQL.”

Development Time: 4 hours
Source: Enterprise Strategy Group, Lab Validation Report, September 2012
Development Time: 1 hour (4x faster)
Execution Time: 149 seconds Execution Time: 3 seconds (50x faster)


Teradata Aster Big Analytics Appliance—
Key Innovations


Aster SQL-H™
A Business User’s Bridge to Analyze Hadoop Data

NEW
Aster SQL-H Gives Analysts and
Data Scientists a Better Way to Aster: SQL-H
Analyze Data Stored in Hadoop
•  Allow standard ANSI SQL access to Hadoop
Hadoop data MR

•  Leverage existing BI tool and enable

Data Filtering
self service

Data
Hive HCatalog

•  Enable 50+ prebuilt SQL-MapReduce
Apps and IDE
Pig

Hadoop Layer: HDFS


Hortonworks Data Platform
Enterprise-Ready Hadoop

The ONLY 100% open source
data platform for Hadoop

•  Tightly aligned with core Apache code lines
•  All code committed back to open source
•  Engineered integration with Teradata Viewpoint and Ambari
•  HCatalog - centralized metadata services for easy data sharing
•  Dependable full stack high availability
•  Capacity scheduler for better multi-tenancy
•  Intuitive graphical data integration tools


Teradata Viewpoint Integration
Easier, Faster, and Better System Management

Common Management Console for
Aster, Teradata and Apache Hadoop

Aster-Specific Query Portlets
Portlets •  Query Monitor
•  Aster Node
Monitoring Admin Portlets
•  Aster Completed •  Teradata System
Processes •  Roles Manager

Trend/ Other Portlets
•  System Health
Visualization •  Canary queries
Portlets •  Aster Alerting
•  Capacity
Heat Map
•  Metrics Graph
•  Metrics Analysis


Teradata Vital Infrastructure (TVI)
Integrated hardware & software solution for systems
management
PROACTIVE RELIABILITY, AVAILABILITY, AND MANAGEABILITY

1U server virtualizes system and cabinet management software
Server Management VMS
•  Cabinet Management Interface Controller (CMIC)
•  Service Work Station (SWS)
•  Automatically installed on base/first cabinet

VMS allows full Eliminates need Supports TVI Support for
rack solutions for expansion Teradata Aster and
without additional racks, reducing hardware and Hadoop
cabinet for customers’ floor Aster/Hadoop
traditional SWS space & energy software
costs

62–70% of Incidents Discovered through TVI


How Can You Get Started?
Aster Express


Making it easy to try Aster Big Analytics Solutions
Aster Express, Aster Live, Aster Big Analytics Appliance

Aster Live
Aster Express Aster
Big Analytics
Appliance


Aster Express Tutorials Make it Easy to Start
www.asterdata.com/asterexpress


Teradata Aster Big Analytics Appliance Summary
Bring Big Data to Life with Big Analytics & Discovery

INDUSTRY’S FIRST UNIFIED BIG ANALYTICS APPLIANCE

UNIFIED INTERFACES FOR ITERATIVE SQL AND MAPREDUCE ANALYTICS

TERADATA-TRUSTED RELIABILITY, AVAILABILITY & MANAGEABILITY

EASY TO DEPLOY, MANAGE & USE

Get Started Now! asterdata.com/AsterExpress


When to Use Which?
The best approach by workload and data type
Processing as a Function of Schema Requirements and Stage of Data Pipeline

“Simple math
Data Pre-
Low Cost at scale” Joins, Analytics
Processing,
Storage and (Score, filter, Unions, (Iterative and Reporting
Refining,
Fast Loading sort, avg., Aggregates data mining)
Cleansing
count...)

Financial Analysis, Ad-Hoc/OLAP
Stable Teradata/ Enterprise-Wide BITeradata
Teradata Teradata
and Reporting
Teradata Teradata
Schema Hadoop
Spatial/Temporal
Active Execution

Interactive Data Discovery Aster
Evolving Aster / Aster / (SQL +
Schema
Hadoop Web Clickstream, Set-Top Box Analysis
Hadoop Hadoop Aster Aster Aster
MapReduce
CDRs, Sensor Logs, JSON Analytics)

Social Feeds, Text, Image Processing Aster
Format,
No Schema
Hadoop
Hadoop Audio/Video Storage and Refining
Hadoop
Hadoop Hadoop Aster
Aster Aster Aster
(MapReduce
Analytics)
Storage and Batch Transformations


When to Use Which?
The best approach by workload and data type
Processing as a Function of Schema Requirements and Stage of Data Pipeline

“Simple math
Data Pre-
Low Cost at scale” Joins, Analytics
Processing,
Storage and (Score, filter, Unions, (Iterative and Reporting
Refining,
Fast Loading sort, avg., Aggregates data mining)
Cleansing
count...)

Stable Teradata/
Hadoop Teradata Teradata Teradata Teradata Teradata
Schema

Aster
Evolving Aster / Aster / (SQL +
Hadoop Hadoop Hadoop Aster Aster Aster
Schema MapReduce
Analytics)

Aster
Format,
Hadoop
Hadoop Hadoop
Hadoop Hadoop Aster
Aster Aster Aster
(MapReduce
No Schema Analytics)


Ease of Development and Reuse
Analytic Foundation : 70+ out-of-the-box modules
Modules Business-ready SQL-MapReduce Functions
•  nPath: complex sequential analysis for time series analysis and
behavioral pattern analysis
Path Analysis •  Sessionization: identifies sessions from time series data in a single
Discover patterns in rows of pass over the data
sequential data •  Attribution: operator to help ad networks and websites to distribute
“credit”

•  Histogram: function to provide capability of generating
•  Decision Trees: Native implementation of parallel random forests.
•  Approximate percentiles and distinct counts: calculate
Statistical percentiles and counts within specific variance

Analysis •  Correlation: calculation that characterizes the strength of the
relation between different columns
High-performance processing of •  Regression: performs linear or logistic regression between an output
common statistical calculations variable and a set of input variables
•  Averages: calculate moving, weighted, exponential or volume-
weighted averages over a window of data

Relational •  Graph analysis: finds shortest path from a distinct node to all other

Analysis nodes in a graph
•  Tokenization: splits strings into individual words to assist text
Discover important relationships processing
among Confidential and proprietary. Copyright © 2013 Teradata Corporation.
46 data

Ease of Development and Reuse
Analytic Foundation : 50+ out-of-the-box modules
Modules SQL-MapReduce Analytic Functions

•  Text Processing: counts occurrences of words, identifies roots, &
Text Analysis tracks relative positions of words & multi-word phrases
•  Text Partition: analyzes text data over multiple rows
Derive patterns in textual data
•  Levenshtein Distance: computes the distance between two words

•  k-Means: clusters data into a specified number of groupings
•  Canopy: partitions data into overlapping subsets within which k-
Cluster means is performed

Analysis •  Minhash: buckets highly-dimensional items for cluster analysis
•  Basket analysis: creates configurable groupings of related items
Discover natural groupings of data from transaction records in single pass
points
•  Collaborative Filter: predicts the interests of a user by collecting
interest information from many users

Data •  Unpack: extracts nested data for further analysis
•  Pack: compress multi-column data into a single column
Transformation •  Antiselect: returns all columns except for specified column
Transform data for more advanced •  Multicase: case statement that supports row match for multiple
analysis cases


Perceptions & Questions

Analyst:
Robin Bloor


Big Data Is About Analytics
DATA AIN’T WHAT IT USED TO BE
Machine generated data (logs)
Web data
Social media data
Public data services
Supply chain data
Real-time data flows

THE ANALOGY OF STRIP-MINING IS
RELEVANT BECAUSE THE SCALE OF DATA
ANALYTICS HAS EXPANDED DRAMATICALLY

The Bloor Group

The Data Analytics Issue

The Bloor Group

What Hadoop Is NOT
A MULTIUSER HIGHLY
TUNED ENGINE

AN ANALYTICS
PLATFORM

A SOLUTION

But it IS:
A USEFUL, FLEXIBLE
AND VERY ECONOMIC
DATA STORE – WITH
PLUG-INS The Bloor Group

About Data Analytics
It is all about TIME TO INSIGHT – as long as that
is followed by action

Fast time to insight requires FLEXIBLE
management of high performance data flows -
for the benefit of the data analyst

The data analyst needs to be able to
MARSHAL the data

Then maybe, just maybe, he will deserve the
title of DATA SCIENTIST
The Bloor Group

Clearly the Teradata Aster Big Analytics Appliance is a
powerful data flow engine, so:

!   How does Aster Data achieve its performance
lift with MapReduce?

!   How is it most usually deployed?

!   Can it do data cleansing in flight?

!   Can it perform analytic tasks?

The Bloor Group

!   Why an appliance? What is gained and what is
sacrificed?

!   Which sectors/businesses do you expect to be
able to make best use of this technology?

!   Which companies/products do you regard as
competitors (either direct or near)?

!   Which companies/products do you partner with?

!   How does the appliance fit in the cloud?

The Bloor Group

Upcoming Topics

This month: Big Data

February: Analytics

March: Open Source

April: Intelligence
www.insideanalysis.com


Thank You
for Your
Attention


Left Brain, Right Brain: How to Unify Enterprise Analytics

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Left Brain, Right Brain: How to Unify Enterprise Analytics

Ähnlich wie Left Brain, Right Brain: How to Unify Enterprise Analytics (20)

Mehr von Inside Analysis

Mehr von Inside Analysis (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Left Brain, Right Brain: How to Unify Enterprise Analytics