Provisioning Oracle Fusion Middleware Environments with Chef and Puppet

Provisioning Oracle FMW
Environments with Chef & Puppet
CON7629
Mark Nelson, Edwin Biemond
Fusion Middleware Platform Team
October 02, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

• Mark Nelson
– Architect, Oracle
3

• Edwin Biemond
– Consulting Member Technical Staff, Oracle
4

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
5

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
6

What is possible Now!!!

Focus on Puppet & Chef, which are the most popular
provisioning tools
• > 10.000.000 nodes
• Fastest growing companies in the US
• Stand-alone (Client) & Master (Server)
• Open Source & Enterprise / Premium
– Free < 10 Nodes (Puppet)
– Free < 5 Nodes (Chef)
• Manage Linux, Solaris & Windows
• Cloud plugins, VMware, Google, Openstack,
Azure & Amazon
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 8

Chef components overview
9

Next major release
• SOA architecture
• Services
• Easy to scale up
• Own release lifecycle
• Runs in a JVM
• 3 times faster
• Build with clojure, same as PuppetDB
• JRuby
10
Puppet component overview

• Administrator / Enterprise focus
• Forge.puppetlabs.com > 2700
Modules
• Language, Puppet DSL & Ruby
• Manifest-> Some Puppet DSL Code
• Puppet Modules
• Facter
• Type/Provider -> Ruby
• Developer focus
• Supermarket.getchef.com> 1600
Cookbooks
• Language, Chef DSL & Ruby
• Recipe -> Some Chef DSL Code
• Chef Cookbooks
• Ohai
• Resource/Provider
– Light -> Chef DSL
– Heavy -> Ruby

file{'/etc/resolv.conf':
content => template('mod/my_resolv.conf.erb'),
owner => 'root',
group => 'root',
mode => '0644',
}
$files = ['bacon', 'eggs', 'sausage']
file{$files:
content => "#{title} is delicious!",
}
package{'bash':
ensure => latest,
}
template '/etc/resolv.conf' do
source 'my_resolv.conf.erb'
owner 'root'
group 'root'
mode '0644'
end
['bacon', 'eggs', 'sausage'].each do |type|
file "/tmp/#{type}" do
content "#{type} is delicious!"
end
end
package 'bash' do
action :upgrade
end
12
Some DSL Code

So how does it work!!
• We have to create some recipes/manifests which describes the To Be
situation & not how to achieve it
• Execute chef-apply or puppet apply
• Chef/Puppet retrieves the current state and determines what the
necessary actions should be
• Executes the actions (apply only deltas)
13

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
14

Experiences
Chef & Puppet challenges & solutions
15

WebLogic 12c Changes
• All jars, no O.S. specific installer
– Jars can be used on every Operating System
• Shared oracle_common which is part of WebLogic Infrastructure
– ServiceBus installer +/- 400Mb
• Repository Creation Utility is now part of WebLogic Infrastructure
–  FMW Database repository tightly connected to the domain (because of OPSS)
• WebLogic Restful Management Services
16

Oracle Tools like OPatch, RCU or BSU are not idempotent
• 2nd Times always fails
– It breaks Chef or Puppet
– Hard to handle this with the standard Puppet/Chef exec resource
– Slow, inventory command is always faster then apply and let it crash
• For example OPatch utility
– Detect current state with opatch lsinventory
• Every utility has a different output ( XML, CSV or text ouput )
– Determine the right action
– Skip, Install or Remove the patch with opatch apply
– Check the outcome
17

One Time
• Not really the Chef or Puppet way
• One big or many WLST scripts
• Use exec of Chef or Puppet, almost
impossible to control (Facter/Ohai)
• Execute this once or need to catch
the BeanAlreadyExists exception
• Re-use your old scripts
• Easy to get out of sync
Model driven
• WebLogic Chef/Puppet Resources
• The Model is always up-to-date
• Need to know some Ruby
• Map WLST or the WLS Rest
Interface (12.1.3) to Chef or Puppet
Attributes
• 4 WLST scripts per WLS resource,
retrieve, create/update and destroy
• Big effort to have all WLS resources
18
One time provisioning or Model driven

• BSU, OPatch, AdminServer &
ManagedServer Control
• Machine, Domain, Server, Channels
• (Dynamic) Cluster, Coherence,
Server Template
• User, Group, Authentication
providers
• Deployment
• Datasource
• File Persistence
• JMS server, JMS module,
Connection Factory, Queue, Topic,
SubDeployment, Quota, Foreign
Server
• SAF agents, Imported Destination,
Remote Context, Error handler
• Workmanagers
• Virtualhost
19
Puppet WebLogic Resources ( Total of 37)

Puppet apply Puppet resource wls_server SoaServer1 --edit
20
Model driven resource demo

WLST Performance
21
• Startup of WLST takes times-> even with
skipWLSModuleScanning
• Connect() or readDomain(‘xx’) is slow
• With > 10 WLST scripts or with multiple
Domains you will have some bad performance
• Don’t schedule a Chef or Puppet run every XX
minutes

WLST Domain Daemon process
22
• For Every domain
• Is already connected to the Domain
• Auto start & destroy in every Puppet/Chef run
• More complexity -> need to know when it is finished or
when it fails
• Can’t handle invalid WLST/Python scripts
• It’s Fast -> 2 Domains with 8 clusters and > 100 Queues in
just a few minutes ( normal 30-60 minutes)

Naming of WebLogic artifacts
• In Chef or Puppet a resource should be unique. There can be only one
Firewall Service or one Package Bash
• In WebLogic you can have multiple Domains and a Queue only has to
unique inside a JMS Module.
• Need to have a WebLogic Title convention and implement Title pattern in
all the Chef/Puppet Resources
23

Orchestration of FMW HA Environment
• A High Available FMW environments needs at least
– A Database configuration with Oracle RAC or Oracle Dataguard ( 10 a 30 min )
–WebLogic AdminServer node which depends on the Database ( 5 a 25 min )
– 2 or more WebLogic nodes which depends on the AdminServer ( 5 a 10 min )
• Chef or Puppet can’t orchestrate multi-node environments
•  Can implement some try / catch together with a repeat & wait
– AdminServer should wait for a Database with a FMW Repository
–WebLogic Nodes should wait for a Domain pack on the AdminServer
• We need something on top of Chef/Puppet which can handle this
24

Puppet & Chef Sandbox environments
Desktop
• Easy to simulate environments
• Destroy & Re-Create
• Direct feedback, Change Log levels
• Remote shell access
• Only need some MB of configuration data
• Retrieve the latest Modules or Cookbooks
25
Oracle
VirtualBox
Vagrant
Shared Folder
Puppet / Chef code
VM
VM
Template
Puppet/Chef
Client
Oracle
JDK
WebLogic
Database
Github
Forge
Supermarket
yum / wget
• Out of the box with Chef Test Kitchen
• Puppet -> requires some Vagrant
configuration

• Download & Extract zip files requires a lot of memory or a
Oracle Big Files
26
big swap file
• Takes a lot of time to download plus extract & requires
disk space
• It is slow in every run because Puppet/Chef file resource
checks for differences (locally and remote)
• Use a Shared Drive
• Skip the File resource step with some Puppet/Chef Facts

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
27

Continuous Delivery
28

Separate environment specific variables from your code
• Puppet
– Facter
– Parameter manifest class
– Hiera ( Multiple levels Node, Env, Common ) with a YAML, Database, JSON Backend
• CHEF
– OHAI
– Roles
– Environments
– Databags
– Attribute
29

Promotion of infrastructure
• Versioning (Git Suite -> Gitbucket, Gitlab or Github) for infrastructure
configuration & Development code
– Issues
– Pull Requests
• Multi environment on the master (Development, Test, Staging, Prod)
• Retrieve Modules/Cookbooks from Git or use Forge or Supermarket
– Puppet -> Puppet Librarian, R10k
– Chef -> Berkshelf, Chef Librarian
30

Model your environments
• Define Building blocks or Roles
– Better to understand
• just assign to nodes
– Easier to maintain
• Consist of small modules
• Optimal re-use
– Auto-adapt to the environment
• Developer desktop
• High Available production environment
31

Standardize your VM Base Image
Packer.io is a tool for creating identical machine images for multiple
platforms from a single source configuration
–Works with PXE boot
– Download ISO/Net install
– Installs Chef or Puppet client
– Different post-processors
• Vagrant for VMware or Oracle Virtualbox
• Amazon
• Docker
• OpenStack
• Google
– Only have to provide a Kickstart file
32

Continuous Environment Overview – Modules/Cookbooks
33
Git
•Clone
Code
•RVM (Parallel)
•Chef version
•Lint, RSpec
Smoke Test
•Vagrant
•Librarian
Approve
•Archive
•Version
Distribute
•Repository

Continuous Environment Overview – Building blocks
34
Git
• Clone
Smoke Test
• Vagrant
• Librarian
Approve
• Archive
• Version
Distribute
• Repository

Continuous Environment Overview – Test 2 Production
35
Test
VMs
Cookbooks
Recipes
Staging Production
Test
Environment 1
Test
Environment 2

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
36

Test/Code Quality frameworks
• Rubocup for Ruby coding style
• Chef
– Foodcritic for Chef coding style
– ChefSpec/Serverspec
• Puppet
– Puppet-Lint for Puppet coding style
– Rspec
• RVM/Bundler to test the different
Ruby & Puppet/Chef
combinations
• Sublime or IntelliJ plugins
37

Test in the cloud with Travis CI
Test against different version
of
• Ruby
• Chef/Puppet
Test for
• Lint
• RSpec Tests
• Code Coverage
Opensource

Robot Framework
• Test your FMW environment if it contains what you expect.
– Can’t fully rely on Puppet or Chef output
• With Robot you can add tests in readable and understandable text
• Libraries
– Selenium for WebLogic Web applications tests
– HTTP/Rest Requests for WebLogic Restful Management Services
39

Test WebLogic applications with Robot

Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
41

• JDK7
– URandom fix, JCE Security policy
• ORAWLS
– Install, Patch, RCU, Domain, Cluster, WLS Native Types
• ORADB, ORA_RAC
– Install, ASM, RAC, Patch, Instance, RCU 11g
• ORACLE
– DB Native Types like Tablespace, Role, User & Init params
• > 10k Downloads
• Many Contributions/Pull
requests
• Many Big Enterprise
companies are using it
• Developers with
Vagrant/Virtualbox
Download locations
• Forge.puppetlabs.com
• Github.com/biemond
• Github.com/hajee
42
Opensource Puppet Modules

Conclusion
Finally we have total control
over FMW Environments
without any human
interaction.
And experience the
provisioning of new FMW
environments in minutes
and push changes to DTAP

Q & A
edwin.biemond@oracle.com
mark.x.nelson@oracle.com
44

Provisioning Oracle Fusion Middleware Environments with Chef and Puppet

Provisioning Oracle Fusion Middleware Environments with Chef and Puppet

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Provisioning Oracle Fusion Middleware Environments with Chef and Puppet

Ähnlich wie Provisioning Oracle Fusion Middleware Environments with Chef and Puppet (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Provisioning Oracle Fusion Middleware Environments with Chef and Puppet