Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Jovian Data Amazon Final Version
1. Analytics at the Speed of Thought Satya Ramachandran Vice President of Engineering Anupam Singh Chief Technology Officer April 14, 2010 2460 North First Street, Suite 170, San Jose, CA 95131 408-433-9383 www.joviandata.com
9. Agenda JovianDATA Company Overview JovianInsights – The Power of Analytics JovianDATA Cube Storage Innovations in Advanced Analytics using commodity clusters Analytics Lifecycle Management Innovations in Cloud Infrastructure Management
10. Avoiding Expensive Data Processing Usage based Automatic View Materialization Avoid Network I/O Multi-Dimensional Partitioning Reduce Disk I/O By Materializing Expensive Groups
13. Managing CapEx with Role Based Clusters SINGLE CLUSTER FOR DATA CLEANSING, LOAD AND QUERY 15TB 100 NODES Monthly Cost = $28,800
14. Managing Cap-Ex with Role Based Clusters UI Ad Server Data, Search Engine Data 2 hours daily for load on 10 nodes 8 hours daily for query on 5 nodes Monthly Cost = $2,052 DATA CLEANSING QUERY LOAD MODEL HIBERNATE MODEL
24. But only when you need it to hold down operating costsNode1 Node2 Node3 Node4 P34 P1 P12 P1 P22 P3 P3 P12 P12 P22 P34 P22 Nodeset1 P3 P34 P1 P3 Temp1 Temp2 P34 P1 P22 P12
26. Provision Tera Scale Applications in Minutes Without Application Isolation Data for all advertisers is kept ‘live’ on 50 nodes Campaign Manager needs to run heavy duty reports for a Big Advertiser 50 live nodes per month = $14, 400 FUNNEL ANALYSIS FOR CLIENT
27. Provision Tera Scale Applications in Minutes Application is provisioned in parallel from S3/EBS into EC2 Campaign Manager requests Application Provisioning for a Specific Advertiser 50 nodes for fortnightly analysis = $320 FUNNEL ANALYSIS FOR CLIENT HIBERNATED MODEL
28. Summary Reducing CapEx with Role based Temporary Clusters on EC2 10x Cost Savings with EC2 usage Dynamic Provisioning with Selective Replication on EC2 10x Performance on EC2 replication Application Isolation with Application Hibernation on S3/EBS 100x Cost Savings with EC2-S3