Talk by Sriguru V, Engineer, Altran Engineering Solutions on the topic "Lessons learnt in CI/CD with AWS serverless architecture" at AWS Community Day, Bangalore 2018
This is the overview of the solution context we are going to discuss on today
Solution has been developed for a North American OEM who develops High end products for Education, Government and Corporate customers
We have a front end application which helps the end users of multiple tenants and OEM to work with connected products
We have adopted serverless approach and this session is going to talk about the various lessons learnt in the implementation of this solution
Development was done using agile practices
Speed in deployments
Integrated testing of Unit Tests, E2E Tests, API Tests
Minimal time of Build, Test and deployment process
Ability to recreate the pipeline infrastructure
We first setup of dev environment with Dev Branch
Adopted Jenkins with plugins for managing the Build and deploy jobs
Basically using the Web hooks to trigger the unit tests and then build process and subsequently
Upon successful build deploy to S3 Bucket
We had some security threats and cost impact of the running instances
Customer is interested in going more serverless to understand granularity of cost by leveraging AWS services
Security of EC2 instances
Some of the team members has to maintain the EC2 instance for the Jenkins jobs which also to be monitored and managed
Upgrade plug-ins, etc.,
After identify some of the challenges as outlined in the previous slide on Jenkins we restored to use AWS services
We were getting into the need for different environments for different stakeholders
Dev - Dev Team
QA – testing Team
Pre Production – UAT/Business Team
Production – Customers
This also defined our approach on code deploy and manual handoffs required for approval
We have replaced Jenkins with AWS services
The docker containers will be running in the code pipeline
Separate credentials with IAM polices for dealing with Code pipeline execution
Code Build Services would be adopted for Build & Tests, Deployment and Cache Invalidation to complete the deployment
This would get replicated for different environments
Serverless code, API Test Scripts, Unit Test Scripts, E2E Test Scripts, Database migration scripts are stored in the GitHub repository
Test libraries and its dependencies (Example: npm) are managed to be more lightweight through custom built docker containers
Used tools like Gulp/Yarn for managing dependencies and deploy:
Obfuscation
Cache Invalidation (CloudFront)
Deployment Caching
Code build process should handle obfuscation, minification of the front end code
Headless chrome adoption for running the End to Tests
Integration of test results for Slack Notifications
Integrate the Qmetry test results with AWS Pipeline through S3 Buckets
Managing the pipelines for managing cross account issues between dev and production