3. IT exists to run applications
automation
infrastructure
Scalable
Uber-Easy
Reliable
Fast
4. IT exists to run application
Reality:
Based on worst
principles
borrowed from
human
middleware
automation
infrastructure
5. Micromanagement
Do this sequence of things
do do do do do do do do do do
do do do do
Automation has
been an attempt
at industrialization
of
micromanagemen
t practices
6. heat automation
detailed
abstraction
neutro
n
detailed
abstraction
nova
detailed
abstraction
cinder
detailed
abstraction
swift
detailed
abstraction
glance
………..
Detailed
Interactions
Do, do, do, do
Detailed
Interactions
Do, do, do, do
Detailed
Interactions
Do, do, do, do
Detailed
Interactions
Do, do, do, do
Detailed
Interactions
Do, do, do, do
Detailed
Interactions
Do, do, do, do
(over and over again)
Lots of HOW
Too much
Unnecessa
ry detail
+
Closed
Coupling
7. simplicity, at times, has
heat automation
detailed
abstraction
neutro
n
detailed
abstraction
nova
detailed
abstraction
cinder
complications
detailed
abstraction
swift
detailed
abstraction
glance
complexity
………..
domain details leak
into the automation
layer
and its enforcement
mechanisms
It was OK when these things were very
simple, but it breaks down as the featureset
expands….
8. detailed
abstraction
neutro
n
detailed
abstraction
nova
simplicity, at times, has
domain detail
complexity
detailed
abstraction
cinder
complications
app guy
detailed
abstraction
swift
detailed
abstraction
glance
………..
heat automation
9. but IT exists to run applications….
detailed
abstraction
neutro
n
detailed
abstraction
nova
detailed
abstraction
cinder
app guy
detailed
abstraction
swift
detailed
abstraction
glance
………..
heat automation
“I’d like to run this app
that has the following
requirements on
infrastructure, services
and other apps.. with
these characteristics!”
My app looks like this:
intent
of what app should
be and what it needs
Intent Is lost in unnecessary domain specific
details
10. intent
Abstraction
Portability
Self-containment
No leakage of unnecessary knowledge across apps
this is how I expose
myself to other
apps/components/s
ervices
This is how
I need to
consume
infra
This is my
application
component
some other app
some other app
storage
requirements
compute
requirements
placement
requirements
image
rules
scaling
rules
booting/init
rules
v
m
v
m
….
v
m
some
app/compo
nent/service
What
apps/components/servic
es do I depend on?
Network and netsec
are implicit
a real application consists of many of these
12. neutro
nova cinder swift glance
n
heat automation
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
governance
app guy
intent
of what app
should be and
what it needs
domain specific intent
influence
13. neutro
Consistent continuous enforcement
Localized isolated domain specific decisions
Simplicity and Scale!
Under control of governance
nova cinder swift glance
n
heat automation
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
governance
intent
of what app
should be and
what it needs
domain specific intent
influence
14. neutro
nova cinder swift glance
n
heat automation
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
intent
abstraction
congress
TOSCA
group-based policy intent
How do we get there?
17. Goal of OpenStack Abstractions
Capture the “infrastructure needs” of an
application independent of the
complexities of how the underlying
infrastructure is implemented.
18. Let us review the use-case
2-tier (Web & App) PCI-compliant app deployed either in production or for dev
a) Developer intent:
● External to Web Tier: Open port 80, use LB
● Web to DB Tier : Open port 8800
● Existing Services : MySQL, Events correlation Bigdata App, Billing, and Monitoring
b) Cloud operator intent:
● Production : Allow internet access
● Production : DMZ firewall inspects external traffic
● Dev : Allow access to internal proxy
● Remediation : Infected application must be quarantined
c) Compliance Officer intent:
● Security : PCI Firewall for Billing
19. How you build it today…
Create logical networks for app-tier and web-tier
What subnets to use? IT Trouble Ticket
What routing to setup between these network? IT Trouble Ticket
On app tier, creates security groups to drop all traffic other than port 80, and
Monitoring traffic – creates Trouble Ticket for monitoring team to ask what ports to open
DB, Bigdata traffic – creates Trouble Ticket for DB/Bigdata team to ask what ports to open
Payment service traffic – creates Trouble Ticket for Payment team, hope PCI needs are met by those
rules
May have to create FWaaS for requirements like auditing or stateful inspection
Load balance traffic, create VIP using LBaaS
Create Floating IP for VIP
Create Firewall using FWaaS for external traffic
Creates Trouble Ticket for infra team to setup rules for external access
Needs rules to drop all external traffic, not clear where to set that up? As routing rules? As FW rules?
One more Trouble Ticket!
External traffic needs both FW and LbaaS rules now, how to order them?
Creates Trouble Ticket for support team on how to set them up to be ordered correctly
Need to quarantine infected VMs
Create scripts to update security groups on demand for quarantine needs
Automate all of this as scripts, and whenever any of the dependencies change (infra
or other apps), update the scripts as needed.
20. Where we are today…
We have good abstractions for representing logical
networks/routers/services
However, our abstractions are very low level. We spread
application description across multiple abstractions like:
L2/L3 address allocation
Routing rules
Security Groups
Service configuration
Requires manual reconciliation
Can we do better?
21. What Abstractions are needed?
2-tier (Web & App) PCI-compliant app deployed either in production or for dev
a) Developer intent: The application must
● Close all ports but 80 for the web tier
● Use LB for access to the web tier
● Allow no network communication to DB tier except from the web tier
● Use existing user db for provisioning
● Use existing bigdata app for events
● Use existing payment service for billing
● Use existing monitoring app for billing
b) Cloud operator intent:
● Applications deployed for production may have access to the internet.
● Applications deployed for development should not have internet access.
● Any traffic from internet must be inspected by a DMZ firewall.
● Any infected application must be quarantined
c) Compliance officer intent:
● All traffic to the payment service must be inspected by an audited firewall rule set
22. Groups
A new 2-tier PCI app (app tier and web tier) can be deployed either for production or for development.
a) Developer intent: The application must
● Close all ports but 80 for the web tier
● Use LB for access to the web tier
● Allow no network communication to DB tier except from the web tier
● Use existing user db for provisioning
● Use existing bigdata app for events
● Use existing payment service for billing
● Use existing monitoring app for billing
b) Cloud operator intent:
● Applications deployed for production may have access to the internet.
● Applications deployed for development should not have internet access.
● Any traffic from internet must be inspected by a DMZ firewall.
● Any infected application must be quarantined
c) Compliance officer intent:
● All traffic to the payment service must be inspected by an audited firewall rule set
Groups
23. Traffic Classifiers
A new 2-tier PCI app (app tier and web tier) can be deployed either for production or for development.
a) Developer intent: The application must
● Close all ports but 80 for the web tier
● Use LB for access to the web tier
● Allow no network communication to DB tier except from the web tier
● Use existing user db for provisioning
● Use existing bigdata app for events
● Use existing payment service for billing
● Use existing monitoring app for billing
b) Cloud operator intent:
● Applications deployed for production may have access to the internet.
● Applications deployed for development should not have internet access.
● Any traffic from internet must be inspected by a DMZ firewall.
● Any infected application must be quarantined
c) Compliance officer intent:
● All traffic to the payment service must be inspected by an audited firewall rule set
Groups
Classifiers
24. Policy Tags
A new 2-tier PCI app (app tier and web tier) can be deployed either for production or for development.
a) Developer intent: The application must
● Close all ports but 80 for the web tier
● Use LB for access to the web tier
● Allow no network communication to DB tier except from the web tier
● Use existing user db for provisioning
● Use existing bigdata app for events
● Use existing payment service for billing
● Use existing monitoring app for billing
b) Cloud operator intent:
● Applications deployed for production may have access to the internet.
● Applications deployed for development should not have internet access.
● Any traffic from internet must be inspected by a DMZ firewall.
● Any infected application must be quarantined
c) Compliance officer intent:
● All traffic to the payment service must be inspected by an audited firewall rule set
Groups
Classifiers
Policy
Tags
25. Policy Actions
A new 2-tier PCI app (app tier and web tier) can be deployed either for production or for development.
a) Developer intent: The application must
● Close all ports but 80 for the web tier
● Use LB for access to the web tier
● Allow no network communication to DB tier except from the web tier
● Use existing user db for provisioning
● Use existing bigdata app for events
● Use existing payment service for billing
● Use existing monitoring app for billing
b) Cloud operator intent:
● Applications deployed for production may have access to the internet.
● Applications deployed for development should not have internet access.
● Any traffic from internet must be inspected by a DMZ firewall.
● Any infected application must be quarantined
c) Compliance officer intent:
● All traffic to the payment service must be inspected by an audited firewall rule set
Groups
Classifiers
Policy
Tags
Actions
26. Group-Based Policy Model
Policy
Tags
Policy Rules Set
Policy Rule
Policy Rule
Policy Rule
Group
Classifier
Classifier
Action
Action
Service Chain
28. Architecture Approach
o Simplification of automation
o Capture user intent declaratively
o Separation of concerns by role
o Reusable constructs
29. Architecture
1. Neutron Driver
supports any
existing Neutron
plugin / ML2
driver
2. ODL Driver
Other infrastructure components
(future) supports
ODL GBP
Swift …
3. Native drivers will
be supplied by
vendors
CLI Horizon Heat
Group Policy
Neutron Driver GBP Drivers
Neutron
Existing
Plugins and
ML2 Drivers
Group Policy
Nova Cinder
To be defined
30. Application Policy
My App
Servers
App Server
RuleSet
High Availability
Service Chain
My Web
Servers
Web Server RuleSet
31. Existing Application Consumer
Policy
My App
Servers
App Server
RuleSet
High Availability
Service Chain
My Web
Servers
Web Server RuleSet
App DB
RuleSet
App
Database
User DB
RuleSet
User
Database
Payment
RuleSet
Payment
Service
Monitoring DB RuleSet
Monitoring
Service
Monitoring
V2 Service
Existing Applications Security Service Chain
32. Infrastructure Policy Layering
My App
Servers
App Server
RuleSet
My Web
Servers
Web Server RuleSet
App DB
RuleSet
App
Database
User DB
RuleSet
User
Database
Payment
RuleSet
Payment
Service
Monitoring DB RuleSet
Monitoring
Service
Monitoring
V2 Service
Existing Applications
Outsid
e
Acces
s
Security Service Chain
High Availability
Security Service Chain
Service Chain
GBP renders
the Services
into a
composed
Chain
37. References
GBP wiki page to get more information on how to
install and work with GBP:
https://wiki.openstack.org/wiki/GroupBasedPolicy
GBP Design Summit Session:
Tuesday November 4, 2014 12:05 - 12:45
Dufy (Le Meridien)
http://kilodesignsummit.sched.org/event/98dc4255384e340
682137c8a7ee7e60d#.VFKCJYt4r4w
Try it out
https://wiki.openstack.org/wiki/GroupBasedPolicy/InstallDe
vstack
Hinweis der Redaktion
Before we start on the topic of policy, I think we need to level set ourselves on why we are all here. We manage infrastructure and infrastructure EXISTS to run applications. An applications need to be deliverable in a way that is scalable, easy, reliable, and fast!
----- Meeting Notes (10/14/14 11:18) -----
we must automate application processes and well in infrastructure underneath.
So, we did what smart people do – we developed automation. But our automation tools to date are mostly borrowed from human middleware. We took the things we used to type into our consoles and we embedded them in cripts and tools. Even our APIs are so level as they are not much help here. The end result is indeed automated but hardly scalable, easy, reliable.
How many of you like being micromanaged? Do you think tools do better micromanaged as well?
In reality what we’ve done is the industrialization of these bad practices.
----- Meeting Notes (10/14/14 11:20) -----
this is not the answer
Now lets think about this in openStack. OpenStack was designed to offer a set of abstraction layers across different types of systems and architectures.
But the abstractions today are still too detailed and low level for automation:
networking I’m specifying L2/L3, Devops is great until its time to manage BGP…
The result is to close a coupling between the automation and the underlying infrastructure.
----- Meeting Notes (10/14/14 11:30) -----
couple
----- Meeting Notes (10/14/14 11:30) -----
What is happening is that the details are leaking from the individuals systems trhough to the automation.
This was ok when systems were simple but more and more domain details leak into the automation.
This introduces complexity and reduces effectiviness. Complexity is the the enemy of automation and scale.
And who does this impact the most? The poor application guy who has to define his services and build automation against them. He needs to understands a lot about how each of these detailed layers works and interacts. It is NOT easy or fast
But remember IT exists to run applications. APplication guy know how his application looks. He knows how it interacts with other things. By the time you get to the interfaces of the cloud platform, all you think about is implementation details. THe intent is lost. THIS IS BAD. This is root of the problem we are facing.
So what should we be doing? We have to capture intent. Networking is the worst but not the only culprit here.
----- Meeting Notes (10/14/14 11:30) -----
AN pplicaiton comprises of multiple components – each of them is compromised of units of compute with the same behavior. You have to scale them up and down and control them through a set of rules.
Then you have to know how thye consume infrastructure.
(re-arrange animation)
app
upper left reasons
then say there are many of these. They are defined by different people. There is no one person who knows everything.
If you rely on this model and clearly identify rules of consumption, networking and security becomes implicit.
----- Meeting Notes (10/14/14 11:30) -----
each of these surfaces should be indepdnent inputs where roles and division of labor are honored
An unlike orchestration, this is a continuous enforcement loop. Each of these surfaces change and the system must react in any dynamic environment.
For example, if there is a failure, you should be able to reconverge without manual intervention.
We have to rethink the abstractions we are using to describe things to better capture intent. Automation tools must provide a set of intent-based interfaces to reflect the structural requirements of the application as well as how application consume other resources.
Other systems must expose their own intent-based abstractions enforce its own policy loop in an autonomous domain specific manner.
ANd all enformcement should be influenced and enforced through a set of gov and ops constraints.
This is how we really make things scalable, easy, reliable, and fast!