Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

(ARC305) How J&J Manages AWS At Scale For Enterprise Workloads

3.605 Aufrufe

Veröffentlicht am

Johnson & Johnson is a global health care leader with 270 operating companies in 60 countries. Operating at this scale requires a decentralized model that supports the autonomy of the different companies under the J&J umbrella, while still allowing knowledge and infrastructure frameworks to be shared across the different businesses. To address this problem, J&J created an Amazon VPC, which provides simplified architecture patterns that J&J's application teams leveraged throughout the company using a self-service model while adhering to critical internal controls. Hear how J&J leveraged Amazon S3, Amazon Redshift, Amazon RDS, Amazon DynamoDB, and Amazon Kinesis to develop these architecture patterns for various use cases, allowing J&J's businesses to use AWS for its agility while still adhering to all internal policies automatically. Learn how J&J uses this model to build advanced analytic platforms to ingest large streams of structured and unstructured data, which minimizes the time to insight in a variety of areas, including physician compliance, bioinformatics, and supply chain management.

Veröffentlicht in: Technologie

(ARC305) How J&J Manages AWS At Scale For Enterprise Workloads

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Keith Blizard, Bob Tordella October 2015 Self-service Cloud Services How J&J Is Managing AWS at Scale for Enterprise Workloads ARC305
  2. 2. What to Expect from the Session - Reviewing Enterprise Challenges & Incorporating Cloud Capabilities - Provide approach for enabling Enterprise Controls - Example Architecture & Implementations - Example Patterns (HPC & Workspaces) - Lessons Learned
  3. 3. J&J is a Global Health Care Leader More than 270 Operating Companies in 60 Countries, with 126,000 employees Selling Products in more than 175 Countries The world’s sixth-largest consumer health, pharmaceuticals, and biologics company The world’s largest medical devices and diagnostics business
  4. 4. Big Company, Big Challenges Thousands of Systems Complex IT Ops Limited Financial Impact Cloud Patterns & Acceleration Automated IT Cost Transparency Current State of Enterprise IT Cloud Strategy Offers Agility
  5. 5. Transformation to a Flexible Hybrid Cloud Strategy N. America DC Provides complete infrastructure platform through Amazon Web Services and integrated with J&J processes and policies On-Premise Cloud (OPCx)Virtual Private Cloud (VPCx) Provides a highly flexible reference architecture (built on VMware stack) to deliver ‘on-demand’ VMs inside our Enterprise Data Centers or Co-location facilities in each region Europe DC AP DC Compliance Data Protection Operation Transparency Speed + Agility N. America Region Europe Region AP Region
  6. 6. Virtual Private Cloud (VPCx) Vision Empower the business by providing an integrated, scalable, secure self-service cloud IT platform that enables agility, enforces policy, and accelerates best practices Enable Agility • Self Service • Rapid Provisioning • Capacity Mgmt. • Full stack Availability Ensure Policy • AD Integration • J&J AMIs • Enterprise Logging • Backup & Retention • Firewall & Security Rules Accelerate Best Practice • Monitoring & Alerts • VM Scheduling • Encryption • Software Config. Mgmt.
  7. 7. Enterprise Control without the Bottleneck Preventative Controls Detective Controls Core principles for security, compliance & management Enforce Least Privilege Approach Log Everything J&J Identity & Group Management J&J Network Extension Enforce our Images Account Isolation
  8. 8. xbot Big Data Account Workspaces Account Xbot / Management Architecture AWS Services VPCx Help Assurance Monitor VPCx DB xbot Admin AD Console Billing AWS Console Billing Project Owners VPCx Administrators HPC Account • Centralized Policy Enforcement - xbot • Each Application Account is completely isolated from each other • Controls are executed through both Assurance and Enforcement tests run every 10 minutes • Tickets are created for drift to allowable values
  9. 9. Enterprise Control - Queue Management & Automation Work Queue Work Items API Execution @ Each Account: List, Info, Delete, Update, Setup, Admin, Login Metadata: Project Details, Allowable Cloud Objects, Chargeback, Acceptable Values Ex: HPC Account Ticket System
  10. 10. image = project.get_ec2_images(project_info['Id'], region, image_ids=image_id) images = [] for img in image_objs: unserialized_obj = binascii.a2b_qp(img['image']) images.append(img) instance_info[key][i.id]['Name']=i.tags.get('Name', '') instance_info[key][i.id]['Env'] = i.tags.get('Environment', '') instance_info[key][i.id]['Hostname'] = i.tags.get('Hostname', '') instance_info[key][i.id][’ImageId'] = i.tags.get(’ami-id', ‘’) If instance_info.img_id != allowable value error.name = ‘instance-value-error’ error.value = instance_info create_support_ticket(error.name=‘instance-value-error’) Sample Control – Only Allowing Approved Images
  11. 11. Amazon DynamoDB – Project Metadata
  12. 12. Amazon DynamoDB – Project Level Exceptions
  13. 13. CLI – Automation – Member Info User Level Information And access list
  14. 14. CLI – Automation – Project Info Project Lists including account-code and friendly name
  15. 15. CLI – Automation – Project Info Project Metadata Project Level Service Listing
  16. 16. CLI – Automation – Adding Services Adding New Service for this Project
  17. 17. CLI – Automation – Project Info New Service Added with corresponding IAM roles, policies
  18. 18. App AWS Account (002) Core Project Services Users Alarms HPC App AWS Account (002) Core Project Services Users Alarms HPC App AWS Account (002) Core Project Services Users Alarms HPC AWS Account & Infrastructure Layer Control Xbot Account Payer Account (Consolidated Billing) Consolidated Billing Xbot Administration Scalable to 1000s of accounts App AWS Account (001) Core Project Services Users Alarms HPC App AWS Account (002) Core Project Services Users Alarms HPC
  19. 19. Core
  20. 20. Project
  21. 21. Services
  22. 22. Users
  23. 23. Alarms
  24. 24. HPC
  25. 25. Operating System & Database Layer Control Xbot Account App AWS Account (001) RDS Amazon Redshift EC2 Operating System Database
  26. 26. Managing Amazon Redshift Controls Encrypt Sensitive Data Work Queue Work Items Account Metadata: Ex: HPC Account Ticket System Checks 100s of accounts every 10 min for new instance; enforces policy AD Security Group Sync xbot KMS
  27. 27. Sample Control ― Managing Redshift audit policy requires: # rotate_master_passwords=1hour # apply_cw_metrics=95%CPUutil>60mins;85%DiskUsed>60mins;HealthStatus<1=10mins # require_ssl=True # enable_user_activity_logging=True; bucket_name=RegionalS3LogBucket # backup_retention_period=35days # modify_cluster(master_user_password=newpassword) # publicly_accessible=False # add_tags=‘Environment’;’Production’ # rotate_user_passwords=90days # sync_users=(conn.rscluster) ## add users, set groups, revoke public schema ## drop users, move schema ownership
  28. 28. User Federates into Account User creates Cluster
  29. 29. Cluster Created Within 10 minutes, xbot takes over Master User Master User Password is reset by xbot every hour
  30. 30. Master User takes over, abstracts itself by syncing with AD Security Groups tied to that AWS Account
  31. 31. Begins to build a Profile / Group Grants various permissions to group and associates DBAs
  32. 32. Revokes Access to Public Schema to ensure least privilege
  33. 33. Xbot detects new Cluster; applies CloudWatch Alarms
  34. 34. Alarms
  35. 35. Xbot enables logging & sets the maximum backup retention
  36. 36. Xbot updates Parameter Group for SSL & User Activity Logging Xbot resets the parameter group within 10 minutes to enforce policy
  37. 37. Xbot notifies users of the changes to their environment
  38. 38. Enterprise Log Management Queries logs out of DB Rotates logs every week Temp Location for Log Movement Elastic Load Balancing S3 Amazon Redshift Data Pipeline EMR CloudFrontCloudTrail Config EC2 RDS Regional S3 Logging Bucket No API Action to send DB user Activity Logs to S3 Regional S3 Logging Bucket Copies to S3 Bucket
  39. 39. EC2 Elastic Load Balancing S3EBS Amazon Glacier RDS Amazon Redshift Compute Storage & Content Delivery Database AWS Components Orchestrated DynamoDB Amazon Kinesis Data Pipeline EMR VPC Direct Connect Auto Scaling CloudFront ElastiCache CloudFormation CloudWatchCloudTrail IAM SESSNSCloudSearch SQSSWF Python (boto) WorkSpacesWorkDocs Directory Service Trusted Advisor Config Networking Management Tools Enterprise Applications
  40. 40. Common Architecture Pattern for Big Data or HPC us-east-1 (10.X.X.X/25) us-east-1a 10.X.X.0/27 us-east-1b 10.X.X.32/27 Connected VPC VPC Peering Amazon S3 Win/Lin EC2 DynamoDB us-east-1 (10.X.X.X/19) Disconnected VPC for EMR IGW us-east-1a 10.X.0.X/21 us-east-1b 10.X.7.X/21 us-east-1c 10.X.15.X/20 Burst High Performance Computing (HPC) workloads in Private Address Space in same Account Take advantage of multiple subnets / AZs for Spot Instance Pricing Common Use Cases • Statistical Analysis on large data sets; e.g. Genomic Sequencing • Transformations of large complex data sets for Advanced Analytics (Sales & Supply Chain) • Machine Learning engines on unstructured or non-relatable data Large volumes of Structured & Unstructured Data Direct Connect VGW On-Premise Internal Data SourcesAdmins OIA
  41. 41. J&J DCs JJNET MFA SCCM Site & DP J&J Resources J&J Facility Zero Client ELB Workspaces Account Infra Comp Account Core Infra Account Zero Client Account Teradici Connection Manager Workspaces Architecture Patterns Comments • Global implementation across NA, EMEA and AP • Infrastructure components living within AWS for scale, performance and management • J&J Network extended into AWS
  42. 42. Tradeoff / Lessons Learned - DevOps is heavily recommended for approach to cloud. Focus on velocity of new capabilities & operational improvements - Security Engagement and Partnership is critical - Identify, Design and remain Diligent with your Cloud Principles - Early evaluation with CMP – focus has been too much on IaaS & Provisioning only - Partnership with 3rd Party is crucial (Log Management, Web Application Firewall, Utilization & Spend) - Training of Enterprise IT Users is critical
  43. 43. Key Takeaways - Lean into PaaS services - Enable agility of the cloud to your end users through self-service - Automate your enterprise controls - Unleash power of the cloud for small to large patterns
  44. 44. Thank you! Contact Details: Keith Blizard – kblizard@its.jnj.com Bob Tordella - btordell@its.jnj.com
  45. 45. Remember to complete your evaluations!