This presentation is based on a real life experience migrating Oracle E-Business Suite R12.1 production to AWS. We will talk about: - Certification basics. - How to architect. Recommendations. - Advanced configurations. - R12.2. - Microsoft Azure and Oracle Cloud review. - Horizontal auto-scaling. Is this a supported configuration?

1. Running Oracle EBS
in the cloud
ANDREJS PROKOPJEVS
Lead Applications Database Consultant

2. About me
© 2016 Pythian 2
Apps DBA from Riga, Latvia.
Speaking SQL since 2001.
In Oracle world since 2004.
“In love” with Oracle EBS since 2006.
Andrejs Prokopjevs
Lead Applications Database Consultant
At Pythian since 2011
@aprokopjevs
prokopjevs@pythian.com
https://www.pythian.com/blog/author/prokopjevs/

3. ABOUT PYTHIAN
Pythian’s 400+ IT professionals
help companies adopt and
manage disruptive technologies
to better compete
© 2016 Pythian 3

4. TECHNICAL EXPERTISE
© 2016 Pythian 4
Infrastructure: Transforming and
managing the IT infrastructure
that supports the business
DevOps: Providing critical velocity
in software deployment by adopting
DevOps practices
Cloud: Using the disruptive
nature of cloud for accelerated,
cost-effective growth
Databases: Ensuring databases
are reliable, secure, available and
continuously optimized
Big Data: Harnessing the transformative
power of data on a massive scale
Advanced Analytics: Mining data for
insights & business transformation
using data science

5. Systems currently
managed by Pythian
EXPERIENCED
Pythian experts
in 35 countries
GLOBAL
Millennia of experience
gathered and shared over
19 years
EXPERTS
11,800 2400
© 2016 Pythian 5

6. THE TOP 1.5% GLOBALLY
© 2016 Pythian 6
2015
Resumes
Reviewed:
12,711
Behavioral
Interviews
Conducted:
394
Technical
Tests
Sent: 4062
Passed: 562
Job Offers
Made: 189
Accepted:
174

7. We are hiring !!!
Please visit:
https://www.pythian.com/careers/
HOT !!! Oracle Apps DBA
position in APAC region.
© 2016 Pythian 7

8. Agenda
• Certification basics.
• How to architect. Recommendations.
• Advanced configurations.
• R12.2.
• Microsoft Azure and Oracle Cloud review.
• Horizontal auto-scaling. Is this a supported configuration?
© 2016 Pythian 8

9. Certification basics
© 2016 Pythian 9

10. What is cloud?
• Google it - cloud computing is a type of Internet-based computing that provides
shared computer processing resources and data to computers and other devices on
demand.
• In other words, some virtualization platform service, which provides required resources on-
demand, on a shared hardware base.
• Hosting version 2? Not only that. Virtual hardware is not everything what you get today.
• IaaS
• SaaS
• PaaS
© 2016 Pythian 10

11. What is cloud?
© 2016 Pythian 11

12. What is cloud?
• Pros
• You do not need to think about your hardware, it's support, electricity, UPS or cooler in your
data center room, more. This is your cloud service provider responsibility.
• Hardware pool - sharing and distribution of the load.
• Transparency.
• "Pay per use" model.
• No termination fees.
• Cons
• Security?
▪ Design wise
▪ Encryption
▪ Single Point of Disaster (AWS console)
• IAM
• Root account
• Possible network latency with cloud service provider. © 2016 Pythian 12

13. What is cloud?
• Public cloud
• Private cloud
• Hybrid cloud
© 2016 Pythian 13

14. Certification
• SaaS or PaaS
• Ready solution. You don’t need to think about it.
• Example: Amazon RDS (DBaaS).
• IaaS
• Same rules apply if that would be your local physical
hardware. No changes.
• You are The Captain of your ship. ;)
© 2016 Pythian 14

15. Licensing
• Licensing Mantra
• http://www.oracle.com/us/corporate/pricing/cloud-licensing-070579.pdf
• AWS EC2, AWS S3, MS Azure.
• Customers are required to count each virtual core as equivalent to a physical core
▪ Database Standard Edition: <=4: vCore = processor licensed, >4: closest vCore multiple of 4 * 0.25 =
processors licensed
▪ Database Enterprise Edition: vCores * 0.5 = processors licensed (each virtual core is considered
equivalent to a physical core).
▪ Named User Plus
• Licensing models
• Pay per use rates, included in Cloud service provider pricing model (Amazon - only SE1 and
SE2)
• BYOL (Bring Your Own License)
• VMware – Sorry ! Nothing changed. You need to license the whole hardware pool.
• Cost efficiency: do your own calculations !
© 2016 Pythian 15

16. Oracle E-Business Suite
• Quick overview
• AWS EC2
▪ Amazon RDS for Oracle - NOT supported !
▪ Sorry, only IaaS.
• MS Azure
▪ Only IaaS.
• Oracle Cloud
▪ Something special is there. But we’ll talk about it a bit later.
© 2016 Pythian 16

17. How to architect.
Recommendations.
© 2016 Pythian 17

18. Overview
• Our talk will be based on ”wrong cloud” Amazon EC2
• Oracle E-Business Suite R12.1
• Standard 1 Apps Tier & 1 DB Tier configuration
• High level overview
© 2016 Pythian 18

19. Important !
• You are the Solution Architect
• How you design it will be the way the system is going to run
© 2016 Pythian 19

20. #1: Network
• Everything should start with a proper Network design
• Amazon VPC
• http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html
• Region / Availability Zones
• Subnets
• Network interfaces
• Route tables
• Security
• Network ACLs
• Security Groups
• Internet Gateway
• Virtual Private Gateway
• IPSec VPN tunnel with your on-premise network
© 2016 Pythian 20

21. #1: Network
• IP address
• Private IP addresses – dynamic. Can’t be reserved and may change during instance restart.
• Public IP addresses – dynamic, but a static IP address can be assigned via Elastic IP feature.
▪ Note: Public IP traffic is routed through the public internet.
• DNS
• By default provided by Amazon
• Each region has it’s own sub-domain
• Automatically updated via internal DHCP
▪ But contains Private IP address in the name: ec2-10-10-10-1.eu-central-1.compute.amazonaws.com
• Route 53
▪ Configure your own DNS
▪ Assign names to instances, not IP addresses
© 2016 Pythian 21

22. #2: Instances
• Instance – actual virtual machine.
• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html
• Instance types
▪ General purpose
▪ Compute optimized
▪ Memory optimized
▪ Storage optimized
▪ Accelerated Computing
• It’s all about your “hardware” power
▪ vCores and Elastic Compute Units, T2 instances with CPU credits / burst
▪ Memory
▪ GPU
▪ more
© 2016 Pythian 22

23. #2: Instances
• Instance types #2
• On-Demand (default)
• Spot Instances
• Reserved Instances
• Dedicated Hosts
• Mainly affects pricing.
• AMIs (Amazon Machine Images)
• Image of the instance
• Public and Private repositories
© 2016 Pythian 23

24. #2: Instances
• Example that can be used for your Oracle E-Business Suite
• Apps Tier: m3.xlarge
▪ 4 vCores
▪ 13 ECUs
▪ 15 GB Memory
• DB Tier: r3.4xlarge
▪ 16 vCores
▪ 52 ECUs
▪ 122 GB Memory
• What AMI to use?
▪ EC2 comes with RHEL, SLES, Windows Server official package selection (pay per use licensing model)
▪ Oracle Linux available via Public AMI repository. Use for free, but don’t forget about ULN licensing terms.
• 122 GB RAM for EBS database?
▪ Sometimes memory size is not the primary pattern what instance type to select
▪ Each instance type has IOPS and throughput limits.
© 2016 Pythian 24

25. #3: Storage
• Documentation reference.
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Storage.html
• Storage types
▪ Amazon EBS (Elastic Block Store).
• Standard SAN-like disk volumes. Can be mounted to one EC2 instance at a time.
• SSD: gp2, io1 (PIOPS). HDD: st1, sc1.
• Different Throughput and IOPS limits.
▪ Amazon EC2 Instance Store (Ephemeral)
• Local disks.
• IMPORTANT: All data is lost once Instance is stopped or restarted.
▪ New! Amazon EFS (Elastic File System)
• NAS analog from Amazon – NFS.
• Note: Not available yet in all regions.
▪ Amazon S3
• By default used for storing AMIs and Amazon EBS disk snapshots.
© 2016 Pythian 25

26. #3: Storage
• Example that can be used for your Oracle E-Business Suite
• Apps Tier file system
▪ Elastic File System is Preferable, if scale-out plans are there.
▪ Elastic Block Store – standard option for Single-Tier approach.
▪ Make your own “NAS server” instance, mount Amazon EBS disks, and export them via NFS.
▪ gp2 standard type is absolutely enough. Based on IO credits / burst
• DB Tier
▪ Elastic Block Store
▪ Local File-System or ASM.
▪ No universal recommendation on IOPS / throughput. You MUST test, benchmark, and evaluate your own
system performance per requirements.
▪ Hint: Database Smart Flash Cache feature can greatly improve your performance. Configure it on Instance
Store (Ephemeral) disks.
SQL> show parameter db_flash_cache
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_flash_cache_file string /dev/xvda1
db_flash_cache_size big integer 200G
SQL> © 2016 Pythian 26

27. #4: IOPS
• Most trickiest part to understand.
• IOPS / Throughput depends not only on Instance and
Volume type, but also on the size of the volume.
• Provisioned IO (io1) volumes are not always the efficient
option for IO intensive workload.
▪ Larger in size gp2 volumes can provide similar or better IO
performance than smaller io1.
▪ Less cost.
▪ Can experiment putting multiple gp2 into LVM drives.
• Wrong type chosen can introduce huge performance issues.
▪ Example: sc1
• gp2 / io1 based on 16KiB I/O size, st1 / sc1 based on 1 MiB
• Oracle Database: multiblock reads
▪ DB IOPS <> EC2 IOPS
© 2016 Pythian 27

28. #4: IOPS
• Classical reaction after long reading of the
IOPS docs in different areas (EC2, EBS,
RDS).
• Experiment
• Benchmark
• Find better setup that will suit your needs
© 2016 Pythian 28

29. #5: Backup / Recovery
• Documentation reference.
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html
• EC2 provides EBS snapshot feature only
▪ Apps tier: Do snapshots for Apps Base file system.
▪ DB tier: Setup RMAN on dedicated volumes. Do periodic snapshots. Or NAS.
▪ Can be automated via CLI.
© 2016 Pythian 29

30. #6: Monitoring
• CloudWatch.
• https://aws.amazon.com/cloudwatch/
• Free:
• Very limited.
• Not-free:
• Complete monitoring solution for the instances.
• Alarms.
• SWS / SNS integration.
• But not very versed for Oracle.
• You can use your own custom monitoring.
• Setup EM agents.
© 2016 Pythian 30

31. #7: Amazon RDS
• You can still use Amazon RDS for Oracle for other components.
• Oracle Fusion Middleware 12c
• Not supported with Oracle Fusion Middleware 11g, but with some “tweaks” you can still get
your Metadata Repository loaded into RDS instance.
▪ Main issue: no SYS as SYSDBA access. Replaced with ORACLE_MASTER and RDS PL/SQL API.
• Example:
• Oracle Access Manager and Oracle Internet Directory for Single Sign-On.
© 2016 Pythian 31

32. Advanced
configurations
© 2016 Pythian 32

33. #1: Oracle RAC
• Is NOT natively supported *.
• No shared storage.
• No multicast support for interconnect.
• But… non-official solutions.
• Deploying Scalable Oracle RAC on Amazon EC2
▪ https://aws.amazon.com/articles/7455908317389540
▪ * - became official guide by Amazon.
• Flashgrid VSAN solution.
• Do your own interconnect via OpenVPN, try to adopt Amazon EFS (NFS) via direct NFS.
© 2016 Pythian 33

34. #2: Apps Multi-Tier / PCP
• No limits.
• Shared file-system can be implemented via EFS or own NFS solution.
• Or use non-shared file system approach using separate Amazon EBS volumes.
• Elastic Load Balancing
• https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/what-is-load-
balancing.html?icmpid=docs_elbv2_console
• Has all requirements to front-end Oracle E-Business Suite
▪ Sticky session
▪ SSL
▪ more
© 2016 Pythian 34

35. #3: External Tier
• Same as standard Apps Multi-Tier.
• DMZ network restrictions can be implemented via a dedicated VPC security group.
© 2016 Pythian 35

36. #4: SSL
• Native SSL can be implemented in the same way as on-premise.
• AWS Certificate Manager is available to maintain the certificates.
• Can be just front-ended to Elastic Load Balancer with SSL termination.
© 2016 Pythian 36

37. #5: Integrations
• Evaluate your network latency having Hybrid cloud.
• For heavy data exchange systems integrated it might be good to move them to your
Oracle E-Business Suite on the same side.
• Example:
• Data Warehouse
• ETL
• Or adjust your expectations and SLA.
© 2016 Pythian 37

38. #6: Disaster Recovery
• Availability zones.
• Regions.
• IaaS – similar “on-premise” approach.
• For initial clone, Amazon EBS volume snapshots are transferable.
• Amazon RDS for Oracle – built-in feature (DBaaS).
© 2016 Pythian 38

39. #7: Encryption
• Implement TDE.
• Amazon RDS for Oracle (not for Oracle Ebiz) also supports that.
• Or implement encryption of Amazon EBS volumes on the storage layer.
© 2016 Pythian 39

40. R12.2
© 2016 Pythian 40

41. What is different with R12.2?
• Everything that is related to R12.1.
• ADOP
• Private IP addresses
▪ Number of FND tables must contain valid IP addresses of all nodes.
▪ EC2 Instance restart changes the private IP – it must be updated.
• Validation of the /etc/hosts
▪ Same private IP addresses – hosts file has to enlist them, and with required format.
• IO throughput planning is important
▪ Heavy Apps run / patch file system synchronization through online cycles.
▪ DB workload handling edition objects.
▪ Example: “fs_clone force=yes” run duration on 220 GB standard gp2 volume: 42 minutes.
• You can setup a custom OS service
• Run during instance startup to update hosts file and FND table with $(hostname -I) value.
• Any other node dependencies via AWS CLI.
© 2016 Pythian 41

42. Microsoft Azure
© 2016 Pythian 42

43. Microsoft Azure
• Same IaaS.
• In respect to Oracle E-Business Suite it has all main analogs available.
• Virtual Network
• Linux Virtual Machine support
• Load Balancing
• Storage
• Site Recovery
• more
• Important note: Compute limits.
• Main disadvantage comparing EC2 and Azure was the compute option availability. EC2
provided more powerful capacity options. Almost 10 times.
• Things got changed a bit.
▪ 20 vCore max quota (30 West Europe region). But can be extended via support request.
▪ 100+ GB RAM
▪ Enough storage options © 2016 Pythian 43

44. Oracle Public Cloud
© 2016 Pythian 44

45. Oracle Public Cloud
• Announced and released few months back.
• Same IaaS approach.
• But… there are additional PaaS options.
• Database Cloud Service
• Exadata Cloud Service
• Easy multi-node provisioning for Oracle EBS.
• “Lift and Shift”
• One-button your on-premise instance migration to
cloud.
• Marketing. J
• Quick deployment for rapid development
needs (provisioning).
• Completely based on Oracle’s own stack.
© 2016 Pythian 45
Old picture. 12.2.6
images are available
now too.

46. Oracle Public Cloud - Integrations
• As we know, Oracle EBS is really bound to
Oracle’s own products in terms of integration
requirements.
• With other cloud service providers it’s IaaS and
install / patch / maintain everything yourself.
• Welcome to Integrations Cloud Service.
© 2016 Pythian 46

47. That’s it about Oracle Public Cloud
• It’s still too early to tell anything interesting with technically proven stuff.
• No real implementation production experience.
• Information is still being released and published.
• Follow:
• https://www.facebook.com/groups/EBS.SysAdmin/
• https://twitter.com/Oracle_EBS
• Getting Started with Oracle E-Business Suite on Oracle Cloud (Doc ID 2066260.1)
• Recommendation: Do not rush!
• Implement
• Test
• Evaluate
• Migrate
© 2016 Pythian 47

48. Horizontal auto-scaling
© 2016 Pythian 48

49. What is scaling?
• Vertical scaling
• Increase of the compute resources
▪ Change instance type with more RAM, more
CPU power
• Might dramatically increase your cost
• Still has limits. J
• In respect to Oracle EBS
• Increase your JVM memory heap size
• Increase your JVM process count
• R12.2: Add more oacore_serverX WLS
servers
© 2016 Pythian 49

50. What is scaling?
• Horizontal scaling
• Increase of the compute resources
▪ Add more nodes
• Flexibility
• In respect to Oracle EBS
• Add more apps tiers.
© 2016 Pythian 50

51. Challenge
• Oracle E-Business Suite is very depended on it’s internal configuration repository
(FND_NODES)
• Nightmare with R12.2 and ADOP.
• Vertical scaling
• No issues. Just go and tune the instance.
• Horizontal scaling
• Manual effort with “perl adclonectx.pl addnode”.
▪ Find the best combination of pairs file to minimize post manual correction of the context file and instance
configuration.
• What if I need to delete the node?
• Downtime, setup_clean, re-run of all autoconfigs on each node in 2 rounds.
• R12.2 additional: patch fs context file re-upload to the database.
© 2016 Pythian 51

52. What if we want to automate it?
• Business case:
• As for a simple web-server we would like to automatically add web nodes on-demand, if
existing stack is overloaded.
• EC2 Auto Scaling Documentation
• https://aws.amazon.com/documentation/autoscaling/
• Auto-Scaling group is created
• Minimum, maximum, and desired node count parameters
• Scaling plans – when and how to scale. Specific time, or can be based on CloudWatch
monitoring. Rules for scaling in.
• Launch configurations: Instance configuration.
▪ Create your private AMI from existing Apps tier.
• Automatically adding your new node created into the Elastic Load Balancer configuration.
© 2016 Pythian 52

53. What if we want to automate it?
• What can happen when we start a new node created from AMI?
• We can put a custom first-boot shell script which will handle certain actions.
• cloud-init standard method.
▪ By default, user data and cloud-init directives only run during the first boot cycle when you launch an
instance.
• Pricing for Auto-Scaling
• It is free.
• You pay only standard rates running your additional EC2 instances.
• Limits
© 2016 Pythian 53

54. Adding a new Web Node
• OS setup
• Software, kernel, limit configuration is coming from created private AMI.
• Hostname.
▪ We need to generate a unique hostname, set it on the server, DNS, update /etc/hosts (R12.2).
• Mount the Apps Base storage layer.
• Apps setup
• Generate required pairs file.
• perl adclonectx.pl addnode
• Run AutoConfig on all existing nodes. Or, at least, “generatetns” step.
• Optional: if sqlnet.ora tcp.invited_nodes restrictions are enabled, “lsnrctl reload
LISTENER_NAME” has to be issued on all existing nodes.
• Optional: any custom post configuration
• Service restart on new created node.
• Elastic Load Balancer – automatically.
• Will enable the node once the TCP socket ping succeeds.
© 2016 Pythian 54

55. Removing a Web Node
• Instance termination
• Elastic Load Balancer will drop the removed instance from it’s configuration
automatically.
• Apps setup
• Master node to trigger the node deletion process.
• R12.1
▪ No process of node deletion.
▪ Concurrent Service Managers configured for new node can be just disabled from back-end.
▪ OAM dashboard – sorry, you can’t avoid the red status there.
▪ File system cleanup.
• R12.2
▪ Abandon the node.
▪ perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-node
▪ Optional: “txkSetAppsConf.pl -configoption=removeMS” on existing nodes if OHS is pointed to all
WebLogic nodes in the cluster, including the removed one.
▪ File system cleanup.
© 2016 Pythian 55

56. Re-use of a removed Web Node
• What if we do not want to generate new unique node names every time, and just
reuse the same integers available?
• R12.1
• Easy. But we should not delete the INST_TOP then.
• Re-enable the Concurrent Service Managers.
• Just launch the instance, configure the OS part, and restart the services.
• R12.2
• Abandoned node – only a complete re-add. Even if we do not delete the node and do not
delete the file system content.
© 2016 Pythian 56

57. Periodic housekeeping
• Once you have a downtime window, it’s worth to go through “setup_clean” process
to have everything re-registered clean and drop garbage.
© 2016 Pythian 57

58. R12.2 considerations
• Abandoned node / Delete node process introduction is useful.
• But what happens if we scale-out or scale-in during an open ADOP online patch
cycle?
• It worked during a test case. But I wouldn’t give a 100% guarantee that all is going to be good
during the next cutover. I was lucky.
• Scale-in process is Abandon process. It is handled during the cycle correctly.
• And what can happen if we do all these our activities during a real phase execution,
like cutover?
• Not tested, but expect everything will break in a quite major manner.
• CloudWatch doesn’t know anything about what’s going on in Oracle EBS.
• Maybe Auto-Scale group should be disabled during these maintenance activities.
© 2016 Pythian 58

59. Summary
• Oracle EBS implementation on cloud is more less straightforward.
• Mostly IaaS. Oracle Public Cloud offers SaaS / PaaS for Oracle Database and
other integrations, like SOA.
• IO performance is the main bottleneck risk. Good design, evaluation of estimates
and testing are very important.
• Security is important.
• Cost efficiency is a subject of detailed calculation.
• Auto-scaling - interesting possibilities exist, but not supported yet.
© 2016 Pythian 59

60. THANK YOU
Q & A
© 2016 Pythian 60