In this webinar, we show how you can migrate and access data with Amazon Redshift using SnapLogic integration.
To learn more, visit: http://www.snaplogic.com/.
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
How to Migrate and Access Your Data with Amazon Redshift
1. July 2013
Maneesh Joshi, Senior Director Product Marketing & Strategy
How to Migrate and Access Your
Data with Amazon Redshift
2. SnapLogic – Company Background
Select
Customers
• Founded 2006
• Headquarters: San Mateo, California
• Key Partnerships:
Global
Operations
Premiere
Investors
For the Era of “1,000 Apps in the Enterprise”
Elastic
IntegrationTM
3. Elastic IntegrationTM
Distributed Architecture
• Execute integration
pipelines (workflows) in
cloud, or on-premise or
both
Secure Bridge
• Securely orchestrate
business processes that
traverse cloud & ground
assets
3
4. Simplifying Access with Hub and Spoke
Users
Mobile
Enterprise
Cloud Big Data
Data Center
ESB RDBMS
5. Data Containerization with Snaps
BUY
• SnapStore
• Certified and supported
by SnapLogic
BUILD
• SDK + API
• Java, Python
• Customer, Partner or
SnapLogic
6. Any Data as a Service Architecture
Structured Data
Unstructured
Data
DB
Collect Translate & Enrich Distribute
1 32
Data
View
6
7. Amazon Redshift Connectivity
• Most efficient since co-
located with Redshift –
SnapLogic runs on
Amazon AWS
• Hub-and-Spoke
architecture
• On-prem Apps to
Redshift
• SaaS to Redshift
• Supports Dynamo and S3
• One-step process
One-step Loading Process
Most Efficient Since Co-
located with Redshift
9. World’s largest subscriber network
Core Tech for Leading Media Brand
• IT transformation mirrors shift to
streaming video, from DVD
• SaaS everywhere, identity & data on-
premise (for now)
• Why they chose SnapLogic
• Workday, Servicenow, ‘n’ websites
• Containerizations with Snaps
• Productivity, re-use and control
• SnapLogic is the platform for big
data initiatives as well
9
Connectivity to cloud should run in the cloudLike Vmware and Nicera – software defined networking…Separation btn data plane and control plane – way to run things in world todayPut data plane on premise – such as ERP systems (politically and functionally)Conversely – fewer people want to keep Siebel, etc and so just move that to cloudONLY SnapLogic can do both, elasticallyData shipping and function shippingShip the question to the data – 10 PB of dataPrice of gas over 4th of July – elasticity enables the data to move to the question!
1990sValuable data was being generated but was really living in silo’d environments. The term MDM was not even coined till 2003As long as you could connect different systems together via a nightly, or sometimes even a weekly feed, that was pretty darn awesome!Technologies like ESBs, EAIs, ETLs… flourished.Data was mostly structured. Sitting in RDBMS systems2000sNetwork speeds increasedCosts went downPlayers like Salesforce and NetSuite started getting traction from SMB marketImmense value on cost and agilityFlexibility of to subscribe vs. perpetual licenses2005: Consumer / Social dataFB, Twitter, LinkedIn, amazon.com consumer reviews…Humans generating massive amounts of preference data, likes and dislikes, Data was different: Non-relational unstructured. Real-time dataHuge volumes: PetabytesProviding immense value to the business on their customers2010: MachineRFID tags. Various other sensors, weblogs. ArcSight got bought out for $1.5B by HPMassive amounts of dataExabytesSplunk had a successful IPO last monthSnap LogicThese 4 sources create an Impendence mismatch!Good luck doing all of this with an ESB Structured vs. unstructuredStreaming vs. batchPetabytes and Exabytes vs. GigaBytesPull vs. pushHub and spokeUnprecedented opportunity & desire to use dataData silos (data fragmentation) unavoidableLegacy Apps, Cloud Apps, and Hadoop are driving thisDifferent locations, protocols, formats, and architecturesData is more distributed & less accessible (less useful)Compounding due to volume & variety of apps & dataESB is just another connectionEnterprises must share data between their appsCollect, combine, process data into valuable informationCompetitive advantage will become necessity for survivalsnapLogic = data sharing platform
Apple Like Model – we offer an API and about 200 SnapsBuild or BuyEasy to build w Java or Phython – An intern out of school built snaps in 4 daysBuild or Buy – Containerazation of accessAbstraction of the end point – so you do not need to know everything
3rd day on the job2 Dozen customers in a variety of industries,IT because that’s great for start ups – IT companies are early adopters and risk takers.Media (web architecture), Retail (Facing disruption from Amazon looking to increase their competitive advantage using software solutions and better access to informationWe are being managed by the security group at GE – GE Security: Apptio performed an independent security assessment on us before going live. RBC, AMEX took that further and performed even more rigorous data plane and control plane security analysis. (Single channel, port usage.)
Just came on board as initial pilotShift from DVD streaming78K Mar 13 around ServiceNow, Active Directory, etc.HR / employeesFix Big Data probs in future$78k – more business (BUT HOW MUCH)?NON HR INTEGRATION