Manish Sharma discusses Fair Isaac Corporation's migration of its myFICO consumer website to AWS. Key reasons for migrating included improving reliability by solving for multiple points of failure and improving security. The migration took 7 months and involved "lift and shift" of applications along with upgrades. Lessons included using small expert teams, automating processes, and designing for security and compliance from the start. Steps outlined for effective security and compliance on AWS included access control, network restrictions, auditing S3 buckets, patching, and using a web application firewall.

1. © 2015 Fair Isaac Corporation. Confidential.
This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s expr ess consent.
SVC204
Giving credit where credit’s due: myFICO’s cloud transformation
Manish Sharma
VP of Engineering - Consumer scores, FICO

2. © 2015 Fair Isaac Corporation. Confidential. 2
Agenda
• A brief intro to FICO
• Why did we migrate myFICO to AWS?
• Lessons from our migration journey
• Steps for an effective compliance & security
posture

3. © 2015 Fair Isaac Corporation. Confidential. 3
20
15
10
5
0
1990 1994 1998 2002 2006 2010
Fraud Losses
27 Million FICO® Scores purchased every
day
FICO®
Scores are used in over 90% of US
consumer credit lending decisions.
FICO®
Scores are used today in more than
20 countries on 5 continents.
250 million US consumeraccounts get free
FICO®
Scores throughtheir lenders
27M
90%
250M
20+

4. © 2015 Fair Isaac Corporation. Confidential. 4
www.myfico.com
Consumer facing online business that
offers Credit Reports with FICO Scores,
Credit & Identity Monitoring Subscriptions
to US based consumers
Send hundreds of thousands of credit
alerts to users every day
Handle sensitive PII and credit data –
Security is paramount
17Yrs

5. © 2015 Fair Isaac Corporation. Confidential. 5
Why did we migrate myFICO to AWS?

6. © 2015 Fair Isaac Corporation. Confidential. 6© 2015 Fair Isaac Corporation. Confidential. 6
Why did we choose to migrate?
Improve Reliability Solve for multiple pointsof failure
Improve Security Solve for evolving threats and lowerrisk tolerance
Foster Innovation CI/CD, Data Analyticsto drive product innovation

7. © 2015 Fair Isaac Corporation. Confidential. 7
Poor reliability: On-premises infrastructure just could not cope
3-4 bursts a day
Requests Need < 2 sec response times
VMs over-subscribed
Frequent outages, high latency
Poor Monitoring
Weeks to provision storage
Manual dependencies
Create tickets
No visibility on costs
Not modern
…
…
So we moved to the publiccloud: AWS

8. © 2015 Fair Isaac Corporation. Confidential. 8
v
SQL execution times 50%
File Processing batch jobs 6X
Compute environment way faster thanon premises
Provisioned IOPS, beefierinstances
Fasternetwork throughput within VPC, better DB
performance
Benefits on day 1 after cutover
Beefier EC2 instances, faster network throughput
within VPC
Page Load times (peak) 33%

9. © 2015 Fair Isaac Corporation. Confidential. 9
Handling burst-y traffic on AWS
3-4 bursts a day
Requests
# Instances
Lambda fn warms up Instances
Auto-scaling not fast enough
Need < 2 sec response times
Algorithm calculates # needed
Memcached ElastiCache

10. © 2015 Fair Isaac Corporation. Confidential. 10
v
Earlier Now
Weekly failures in batch jobs →Zero failures
5 teams in IT →3 DevOps Engineers
Poor monitoring →State of the art monitoring
Unknown costs →Predictable, low cost
Matrixed support teams →AWS Enterprise Support
Benefits 6 months after cutover

11. © 2015 Fair Isaac Corporation. Confidential. 11
v
Earlier Now
Frequent outages →1 brief outage in 1 year
1 deployment every 2 weeks →CI/CD: Daily deployments
4 months PCI Compliance audit
→2 months for PCI
Compliance
Storage-constrained Data
warehouse
→5x sized data fully managed
warehouse (Amazon
Redshift)
2 QA environments →10+ QA environments
Patching nightmare →Automated patching
Benefits 12 months after cutover
Platform
to drive
Innovation

12. © 2015 Fair Isaac Corporation. Confidential. 12
Lessons from our migration journey

13. © 2015 Fair Isaac Corporation. Confidential. 13
v
6 “R”s – strategies for migrating applications
Retain Do nothing (for now)
Retire Get rid of (decommission)
Re-Purchase Move to SaaS
Re-Host “Lift-and-Shift”
Re-Platform “Lift-Upgrade OS, DB, Java etc.-and-Shift”
Re-Architect Refactor to Cloud Native design/services
Pattern
Most
popular

14. © 2015 Fair Isaac Corporation. Confidential. 14
DB migration: Full + Nightly Diff + Hourly
transaction logs
Bake: import data, test application for 1 week
Purge data and re-import data for final cutover
Use AWS Support: Well Architected Reviews,
Infrastructure Event Management
Prototype a multi-AZ VPC design using
hardened images
AWS training & awareness
Small teams of SME and talented
engineers better than one large team
Prototype Execute
Bake &
Test
Cutover
Lift-Upgrade-and-Shift lessons – What worked

15. © 2015 Fair Isaac Corporation. Confidential. 15
Manage access & upkeep from 1 VPC
Shared Services VPC Application
VPC 1
Data
Warehouse
VPC
Application
VPC 2
Corporate
Network
VPC Peering
Bastion
Domain Controller/AD
Vulnerability Scanners
Patch Management
Account Federation
X
Internet
Internet
Internet
Create a “Shared
Services” VPC
VPN

16. © 2015 Fair Isaac Corporation. Confidential. 16
Private subnet
Public subnet
Avoid internal micro-services traffic going over the internet
VPC
Availability Zone 1 Availability Zone 2
Service A
Application Load Balancer
(Internal)
Instances
Service A
Service C Service B
Application Load Balancer
(External)
Private
Hosted Zone
Public Hosted Zone
Use 2 ALBs: one in
Private subnet, other in
Public subnet
Route 53 resolves the
correct IP for internal and
external services
Faster performance
More secure
Betterfor monitoring

17. © 2015 Fair Isaac Corporation. Confidential. 17
Pets = servers that can never go down
Cattle = servers that can fail but auto-recover
Easier to convert pets into cattle after
you’re in AWS.
Two choices we’re happy we made
Use Envelopeencryption: encrypt your
data keys (not data) with AWS KMS.
Helps with multi-region, multi-cloud or hybrid
architectures

18. © 2015 Fair Isaac Corporation. Confidential. 18
Don’t use Self-signed digital certs
… otherwise all clients will need to import cert authority into Trust store during bootstrap
Instead use a public CA signed cert with multiple SANs for your internal services
Be mindful of the DNS cutover
Tried to move root domain to AWS via Corp Name Servers
Root domains can only be moved by Domain Registrars
Monitoring will be noisy after cutover
Assume lots of effort to bring noise down
Use redundant KPIs to monitor key services
All else fails at least send an email in the first few days after cutover
Things that didn’t work

19. © 2015 Fair Isaac Corporation. Confidential. 19
Steps for an effective Compliance and
Security posture

20. © 2015 Fair Isaac Corporation. Confidential. 20
• We chose to create one AWS account
per environment: QA, Stage and
Production
• Secure your AWS AWS account root
user with a hardware 2-Factor key-fob
• IAM based users OK but Federated
logins better
• 2-factor authentication using Okta
verify, Duo etc.
Secure your access
to the AWS
environment.
- before you go live!

21. © 2015 Fair Isaac Corporation. Confidential. 21
• Treat your Corporate network as dirty
• Restrict access to AWS environment via
Bastion hosts only
• Restrict outbound access from all
instances using whitelists
• Prevent N-E-S-W movement using
Security Groups, NACLs, Subnets and
VPC peering
Design your
network to restrict
access and
movement

22. © 2015 Fair Isaac Corporation. Confidential. 22
All outbound traffic is
routed via Squid Proxy
• All protocols
• All ports
HTTP/S traffic white listed
via Squid rules
Non - HTTP/S traffic white
listed via IP tables
VPC
Availability Zone 1 Availability Zone 2
Squid Proxy Squid Proxy
Instances
Network Load
Balancer
Route Tables Route Tables
Internet
Instances
HTTP HTTPTCP
/UDP
TCP/
UDP
Squid Proxy to whitelist outbound access
Biggest advantage: Allows
you to whitelist by domains

23. © 2015 Fair Isaac Corporation. Confidential. 23
• Require separate credentials for
access to AWS (because Corp n/w is
dirty)
• Mandate long passwords 14+ chars or
longer
• Terminate idle sessions after 15
minutes
Credentials can be
stolen or harnessed
from sessions

24. © 2015 Fair Isaac Corporation. Confidential. 24
• Wide open S3 buckets a leading cause
of breaches
• Create a “baseline” of bucket visibility
and permissions
• Automate an audit for any changes
• AWS Trusted Advisor service can help
detect weak permissionsas well
Audit your S3
buckets
- multiple times a day

25. © 2015 Fair Isaac Corporation. Confidential. 25
Application & Security Monitoring
Monitors
DataDog creates
ServiceNow ticket
Elastic Load
BalancerCloudTrail
CloudWatch
Logs
Amazon S3 Alerts
SOC Email Distribution List
Amazon S3
AWS WAF
Metrics
DashboardsWindows
Instances
Linux
Instances
Squid
Instances
Database
Instances
Huge eco-system of managed services:
myFICOELK-basedSIEM in 3 weeks

26. © 2015 Fair Isaac Corporation. Confidential. 26
v
Frameworks
OS, Database etc.
Effort to
patch
Tomcat, Java etc.
Zero
day
Patch, patch, patch
• Automate OS and Database patching
using Ansible, Chef, WSUS etc.
• Containers for above-the-OS
• Upgrading frameworks most often
overlooked – requires discipline
• Use managed services where
possible

27. © 2015 Fair Isaac Corporation. Confidential. 27
• Automatically block IP addresses and
user-agents for 2 – 4 hours via Lambda
• > 50 errors/minute
• > 10 login failures per minute
• OWASP top 10 checklist
• Protect your origin servers by “signing”
requests from theWAF
• Reputation lists – refreshed hourly, use
Lambda
Defend the front
door with an
effective AWS WAF

28. © 2015 Fair Isaac Corporation. Confidential. 28
v
In summary…
• myFICO was migrated in 7 months
• 2+ years since go-live
• Small teams > Large teams
• No excuse for not Automating,
Monitoring
• Meeting Security & Compliance
needs easier on the cloud

29. © 2015 Fair Isaac Corporation. Confidential. 29
Thank you!