SlideShare a Scribd company logo
1 of 40
Download to read offline
K8Guard
An Auditing System For Kubernetes
Medya Ghazizadeh
Linux Foundation ONS March 26 - 29, 2018
Los Angeles, California
About Me
Apache V2 Legal and Disclaimers !
Can: Commercial Use, Modify, Distribute, Sublicense, Private Use, Use Patent
Claims, Place Warranty.
Can NOT: Hold Liable, Use Trademark
Must: Include Copyright, Include License, State Changes, Include Notice.
K8Guard was built in the same retail store that
makes this
The Name K8Guard
Pre-K8S Journey
● Migrated from run time configuration (Chef) to Netflix’s Immutable Pattern.
○ Convert chef cookbooks to RPMs
○ Prevent run time errors
● Use Spinnaker to deploy everywhere
● Canary Deployments and Chaos Monkey-ed everything.
● Promote 12 Factor App Design.
Three Things About K8S
Kubernetes is like the best tetris player
for your infrastructure
The Shiny Problem
definition
Shiny Problem 1: * in Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: *
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
Shiny Problem 2: 15 GB Image size
Why is that a problem?
How to avoid big image sizes?
Shiny Problem 3: Using Externally hosted images
- Deployment failed image failed to pull from docker hub
- Malicious code in Un-Scannable images
Shiny Problem 4: Root/Privileged Containers
Shiny Problem 5: Mount a HostPath volume
“hostPath”: A hostPath volume mounts a file or directory from the kubernete’s
host node’s filesystem into your pod. This is not something that most Pods will
need, but it offers a powerful escape hatch for some very very rare applications.
Shiny Problem 6: Single Replica
I admit this is the most controversial one !
Some people just don't need failover.
K8S Clusters
Problem Before
Facing Black
Friday
Efficiency
Security
Stability
Availability
A Kubernetes Prepared for Black Friday !
Keep The Lights On Even in Disaster !
Solution
Inspired By
The-Twelve-Factor
I. Codebase II. Dependencies III. Config IV. Backing
services V. Build, release, run VI. Stateless Processes VII.
Port binding VIII. Concurrency IX. Disposability X. Dev/prod
parity XI. Logs XII. Admin processes
Solution 1 - WRONG !
Wake up when with a pager’s alert and figure out
why k8s cluster is slow or down in the middle of
the night.
Set up 1-1 meeting with app teams, preach for
them the twelve factor app design and best
practices. Sending long email explaining back
and forth to each app team’s which does a
mistake.
Solution 2 - Scan/Action/Report Automatically
An Automated way to keep the K8S clusters form violations.
- Detect developer’s violations,
- Notify and warn them of their violation
- Take action on them on the violations
- Generate Metrics and Searchable reports for audits.
First Lets Defining Violation Types
Image Size Efficiency 5 GB image size
Image Repo Security Downloading
image from a shady repo
Extra Capabilities Security Setting UID/GUID
Privileged Mode Security Root containers
Single Replica Availability Not 12-factor app
Invalid Ingress Security/Stability Having "*” in ingress
Mount Host Vols Security/Stability Mounting kubernetes system files
No Owner Security No owner
annotation for namespace
Required Entity Security/Stability Required pod not deployed
Required Annotation Security/Stability Required annotation for
Enforce your policy on K8S entities
1. Deployments,
2. Pods
3. Jobs/CronJobs
4. Daemonset
5. Ingresses
6. Namespaces
K8Guard - An Auditing System For Kubernetes
K8Guard Discover
● Discover service, when in messaging mode, finds violations and puts them on a kafka topic.
and also discover API mode, is able to serve without depending on kafka. you can hit the end
points to get JSON response.
K8Guard Action
WHAT KIND OF ACTIONS DOES IT TAKE?
● Notifies the namespace owner (email, hipchat, …).
● After X amount of notifications, it will do a hard action such as:
○ Scale bad deployments down to zero.
○ Suspend bad jobs.
○ Delete bad ingress
Note that there is a safe mode - which only notifies and does not do hard actions.
Nasty Email
K8Guard Report
Report service will generate a human readable and searchable report of all the past violation
actions.
/metrics
Discover API generates 19 metrics which are accessible at /metrics, and you can hook a monitoring
system like Prometheus to collect them and then generate pretty Grafana dashboards.
Violation metric examples:
● The number of all deployments.
● The number of bad deployments.
● The number of all pods.
● The number of bad pods.
Performance metrics examples:
● The number of seconds took to return all images from Kubernetes api.
● The number of seconds took to return all deployments from Kubernetes api.
Example Grafana Dashboard
Why Golang?
- Statically linked libraries
- Ship Binary, and run it on Docker Scratch !
- Small language, New libraries,
- Strongly encourages good practices (error handling, formatting, ...)
- Kuberentes is written in Go itself.
Batteries Included !
Developer Friendly !
How many times you tried to run
a github project and built failed ?
Some projects not only don’t give
you the batteries, they won’t
even give you an idea what kind
of batteries you have to buy.
Just push the code and good
luck !
Makefile !
make help
make build-deploy-minikube
Open Sourcing K8S
Target embraces open source.
second open source project at Target. After
winnaker
«Только все знают всё»
Thank You Open source Contributions
- Daemonsets
- HelmCharts
- Bug fixes
- Reddis and minimal technoogy stack
Open Sourcing Challenges !
There are no bad pull requests! There are only a
bad merges !
Future of
K8Guard
- Engine Rule.
- Plugin System.
- Cloud Messaging.
- Delegate notification
- Consolidate
Technology Stack
Keep in touch, Make your first PR!
● https://github.com/k8guard
● https://k8guard.github.io
● Twitter: @medyadaily

More Related Content

What's hot

DevOps Spain 2019. Beatriz Martínez-IBM
DevOps Spain 2019. Beatriz Martínez-IBMDevOps Spain 2019. Beatriz Martínez-IBM
DevOps Spain 2019. Beatriz Martínez-IBMatSistemas
 
Gitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operationsGitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operationsMariano Cunietti
 
GitOps - Modern best practices for high velocity app dev using cloud native t...
GitOps - Modern best practices for high velocity app dev using cloud native t...GitOps - Modern best practices for high velocity app dev using cloud native t...
GitOps - Modern best practices for high velocity app dev using cloud native t...Weaveworks
 
10 Steps to Cloud Happiness
10 Steps to Cloud Happiness10 Steps to Cloud Happiness
10 Steps to Cloud HappinessAll Things Open
 
Image Scanning Best Practices for Containers and Kubernetes
Image Scanning Best Practices for Containers and KubernetesImage Scanning Best Practices for Containers and Kubernetes
Image Scanning Best Practices for Containers and KubernetesDevOps.com
 
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...Daniel Bryant
 
You Want to Kubernetes? You MUST Know Containers!
You Want to Kubernetes? You MUST Know Containers!You Want to Kubernetes? You MUST Know Containers!
You Want to Kubernetes? You MUST Know Containers!VMware Tanzu
 
5 Kubernetes Security Tools You Should Use
5 Kubernetes Security Tools You Should Use5 Kubernetes Security Tools You Should Use
5 Kubernetes Security Tools You Should UseDevOps.com
 
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...DevOps.com
 
Continuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteContinuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteWeaveworks
 
Kubernetes für Workstations Edge und IoT Devices
Kubernetes für Workstations Edge und IoT DevicesKubernetes für Workstations Edge und IoT Devices
Kubernetes für Workstations Edge und IoT DevicesQAware GmbH
 
GitOps A/B testing with Istio and Helm
GitOps A/B testing with Istio and HelmGitOps A/B testing with Istio and Helm
GitOps A/B testing with Istio and HelmWeaveworks
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykDevOps.com
 
Red Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftRed Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftKangaroot
 
Git in the Enterprise: How to succeed at DevOps using Git and a monorepo
Git in the Enterprise: How to succeed at DevOps using Git and a monorepoGit in the Enterprise: How to succeed at DevOps using Git and a monorepo
Git in the Enterprise: How to succeed at DevOps using Git and a monorepoGina Bustos
 
Deploy multi-environment application with Azure DevOps
Deploy multi-environment application with Azure DevOpsDeploy multi-environment application with Azure DevOps
Deploy multi-environment application with Azure DevOpsAndrea Tosato
 
DevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -CloudbeesDevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -CloudbeesatSistemas
 
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison DowdneySetting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison DowdneyWeaveworks
 
Go for Operations
Go for OperationsGo for Operations
Go for OperationsQAware GmbH
 
Hands-on GitOps Patterns for Helm Users
Hands-on GitOps Patterns for Helm UsersHands-on GitOps Patterns for Helm Users
Hands-on GitOps Patterns for Helm UsersWeaveworks
 

What's hot (20)

DevOps Spain 2019. Beatriz Martínez-IBM
DevOps Spain 2019. Beatriz Martínez-IBMDevOps Spain 2019. Beatriz Martínez-IBM
DevOps Spain 2019. Beatriz Martínez-IBM
 
Gitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operationsGitops: a new paradigm for software defined operations
Gitops: a new paradigm for software defined operations
 
GitOps - Modern best practices for high velocity app dev using cloud native t...
GitOps - Modern best practices for high velocity app dev using cloud native t...GitOps - Modern best practices for high velocity app dev using cloud native t...
GitOps - Modern best practices for high velocity app dev using cloud native t...
 
10 Steps to Cloud Happiness
10 Steps to Cloud Happiness10 Steps to Cloud Happiness
10 Steps to Cloud Happiness
 
Image Scanning Best Practices for Containers and Kubernetes
Image Scanning Best Practices for Containers and KubernetesImage Scanning Best Practices for Containers and Kubernetes
Image Scanning Best Practices for Containers and Kubernetes
 
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
 
You Want to Kubernetes? You MUST Know Containers!
You Want to Kubernetes? You MUST Know Containers!You Want to Kubernetes? You MUST Know Containers!
You Want to Kubernetes? You MUST Know Containers!
 
5 Kubernetes Security Tools You Should Use
5 Kubernetes Security Tools You Should Use5 Kubernetes Security Tools You Should Use
5 Kubernetes Security Tools You Should Use
 
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes S...
 
Continuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteContinuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event Keynote
 
Kubernetes für Workstations Edge und IoT Devices
Kubernetes für Workstations Edge und IoT DevicesKubernetes für Workstations Edge und IoT Devices
Kubernetes für Workstations Edge und IoT Devices
 
GitOps A/B testing with Istio and Helm
GitOps A/B testing with Istio and HelmGitOps A/B testing with Istio and Helm
GitOps A/B testing with Istio and Helm
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and Snyk
 
Red Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftRed Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShift
 
Git in the Enterprise: How to succeed at DevOps using Git and a monorepo
Git in the Enterprise: How to succeed at DevOps using Git and a monorepoGit in the Enterprise: How to succeed at DevOps using Git and a monorepo
Git in the Enterprise: How to succeed at DevOps using Git and a monorepo
 
Deploy multi-environment application with Azure DevOps
Deploy multi-environment application with Azure DevOpsDeploy multi-environment application with Azure DevOps
Deploy multi-environment application with Azure DevOps
 
DevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -CloudbeesDevOps Spain 2019. David Cañadillas -Cloudbees
DevOps Spain 2019. David Cañadillas -Cloudbees
 
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison DowdneySetting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
 
Go for Operations
Go for OperationsGo for Operations
Go for Operations
 
Hands-on GitOps Patterns for Helm Users
Hands-on GitOps Patterns for Helm UsersHands-on GitOps Patterns for Helm Users
Hands-on GitOps Patterns for Helm Users
 

Similar to K8Guard - An Auditing System For Kubernetes

Continuous Security for GitOps
Continuous Security for GitOpsContinuous Security for GitOps
Continuous Security for GitOpsWeaveworks
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kuberneteskloia
 
OSMC 2022 | Current State of icinga by Bernd Erk
OSMC 2022 | Current State of icinga by Bernd ErkOSMC 2022 | Current State of icinga by Bernd Erk
OSMC 2022 | Current State of icinga by Bernd ErkNETWAYS
 
Monitoring Your AWS EKS Environment with Datadog
Monitoring Your AWS EKS Environment with DatadogMonitoring Your AWS EKS Environment with Datadog
Monitoring Your AWS EKS Environment with DatadogDevOps.com
 
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...Oleg Shalygin
 
10 tips for Cloud Native Security
10 tips for Cloud Native Security10 tips for Cloud Native Security
10 tips for Cloud Native SecurityKarthik Gaekwad
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architecturesnine
 
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...sparkfabrik
 
Ship code like a keptn
Ship code like a keptnShip code like a keptn
Ship code like a keptnRob Jahn
 
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Applitools
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices Hendri Karisma
 
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Kubernetes and real-time analytics - how to connect these two worlds with Apa...Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Kubernetes and real-time analytics - how to connect these two worlds with Apa...GetInData
 
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)QAware GmbH
 
Running Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech TalksRunning Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech TalksAmazon Web Services
 
Exploring the GitHub Service Universe
Exploring the GitHub Service UniverseExploring the GitHub Service Universe
Exploring the GitHub Service UniverseBjörn Kimminich
 
Wie macht man aus Software einen Online-Service in der Cloud
Wie macht man aus Software einen Online-Service in der CloudWie macht man aus Software einen Online-Service in der Cloud
Wie macht man aus Software einen Online-Service in der CloudAarno Aukia
 

Similar to K8Guard - An Auditing System For Kubernetes (20)

Continuous Security for GitOps
Continuous Security for GitOpsContinuous Security for GitOps
Continuous Security for GitOps
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kubernetes
 
OSMC 2022 | Current State of icinga by Bernd Erk
OSMC 2022 | Current State of icinga by Bernd ErkOSMC 2022 | Current State of icinga by Bernd Erk
OSMC 2022 | Current State of icinga by Bernd Erk
 
vinay-mittal-new
vinay-mittal-newvinay-mittal-new
vinay-mittal-new
 
Where should I run my code? Serverless, Containers, Virtual Machines and more
Where should I run my code? Serverless, Containers, Virtual Machines and moreWhere should I run my code? Serverless, Containers, Virtual Machines and more
Where should I run my code? Serverless, Containers, Virtual Machines and more
 
Monitoring Your AWS EKS Environment with Datadog
Monitoring Your AWS EKS Environment with DatadogMonitoring Your AWS EKS Environment with Datadog
Monitoring Your AWS EKS Environment with Datadog
 
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
 
10 tips for Cloud Native Security
10 tips for Cloud Native Security10 tips for Cloud Native Security
10 tips for Cloud Native Security
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
 
Enabling NFV features in kubernetes
Enabling NFV features in kubernetesEnabling NFV features in kubernetes
Enabling NFV features in kubernetes
 
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...
Drupal Dev Days Vienna 2023 - What is the secure software supply chain and th...
 
Ship code like a keptn
Ship code like a keptnShip code like a keptn
Ship code like a keptn
 
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
 
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Kubernetes and real-time analytics - how to connect these two worlds with Apa...Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
 
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
 
Running Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech TalksRunning Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech Talks
 
Balaji Resume
Balaji ResumeBalaji Resume
Balaji Resume
 
Exploring the GitHub Service Universe
Exploring the GitHub Service UniverseExploring the GitHub Service Universe
Exploring the GitHub Service Universe
 
Wie macht man aus Software einen Online-Service in der Cloud
Wie macht man aus Software einen Online-Service in der CloudWie macht man aus Software einen Online-Service in der Cloud
Wie macht man aus Software einen Online-Service in der Cloud
 

Recently uploaded

Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 

Recently uploaded (20)

Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 

K8Guard - An Auditing System For Kubernetes

  • 1. K8Guard An Auditing System For Kubernetes Medya Ghazizadeh Linux Foundation ONS March 26 - 29, 2018 Los Angeles, California
  • 3. Apache V2 Legal and Disclaimers ! Can: Commercial Use, Modify, Distribute, Sublicense, Private Use, Use Patent Claims, Place Warranty. Can NOT: Hold Liable, Use Trademark Must: Include Copyright, Include License, State Changes, Include Notice.
  • 4. K8Guard was built in the same retail store that makes this
  • 6. Pre-K8S Journey ● Migrated from run time configuration (Chef) to Netflix’s Immutable Pattern. ○ Convert chef cookbooks to RPMs ○ Prevent run time errors ● Use Spinnaker to deploy everywhere ● Canary Deployments and Chaos Monkey-ed everything. ● Promote 12 Factor App Design.
  • 8. Kubernetes is like the best tetris player for your infrastructure
  • 10. Shiny Problem 1: * in Ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test spec: rules: - host: * http: paths: - path: /foo backend: serviceName: s1 servicePort: 80
  • 11. Shiny Problem 2: 15 GB Image size Why is that a problem? How to avoid big image sizes?
  • 12. Shiny Problem 3: Using Externally hosted images - Deployment failed image failed to pull from docker hub - Malicious code in Un-Scannable images
  • 13. Shiny Problem 4: Root/Privileged Containers
  • 14. Shiny Problem 5: Mount a HostPath volume “hostPath”: A hostPath volume mounts a file or directory from the kubernete’s host node’s filesystem into your pod. This is not something that most Pods will need, but it offers a powerful escape hatch for some very very rare applications.
  • 15. Shiny Problem 6: Single Replica I admit this is the most controversial one ! Some people just don't need failover.
  • 16. K8S Clusters Problem Before Facing Black Friday Efficiency Security Stability Availability
  • 17. A Kubernetes Prepared for Black Friday !
  • 18. Keep The Lights On Even in Disaster !
  • 20. I. Codebase II. Dependencies III. Config IV. Backing services V. Build, release, run VI. Stateless Processes VII. Port binding VIII. Concurrency IX. Disposability X. Dev/prod parity XI. Logs XII. Admin processes
  • 21. Solution 1 - WRONG ! Wake up when with a pager’s alert and figure out why k8s cluster is slow or down in the middle of the night. Set up 1-1 meeting with app teams, preach for them the twelve factor app design and best practices. Sending long email explaining back and forth to each app team’s which does a mistake.
  • 22. Solution 2 - Scan/Action/Report Automatically An Automated way to keep the K8S clusters form violations. - Detect developer’s violations, - Notify and warn them of their violation - Take action on them on the violations - Generate Metrics and Searchable reports for audits.
  • 23. First Lets Defining Violation Types Image Size Efficiency 5 GB image size Image Repo Security Downloading image from a shady repo Extra Capabilities Security Setting UID/GUID Privileged Mode Security Root containers Single Replica Availability Not 12-factor app Invalid Ingress Security/Stability Having "*” in ingress Mount Host Vols Security/Stability Mounting kubernetes system files No Owner Security No owner annotation for namespace Required Entity Security/Stability Required pod not deployed Required Annotation Security/Stability Required annotation for
  • 24. Enforce your policy on K8S entities 1. Deployments, 2. Pods 3. Jobs/CronJobs 4. Daemonset 5. Ingresses 6. Namespaces
  • 26. K8Guard Discover ● Discover service, when in messaging mode, finds violations and puts them on a kafka topic. and also discover API mode, is able to serve without depending on kafka. you can hit the end points to get JSON response.
  • 27. K8Guard Action WHAT KIND OF ACTIONS DOES IT TAKE? ● Notifies the namespace owner (email, hipchat, …). ● After X amount of notifications, it will do a hard action such as: ○ Scale bad deployments down to zero. ○ Suspend bad jobs. ○ Delete bad ingress Note that there is a safe mode - which only notifies and does not do hard actions.
  • 29. K8Guard Report Report service will generate a human readable and searchable report of all the past violation actions.
  • 30. /metrics Discover API generates 19 metrics which are accessible at /metrics, and you can hook a monitoring system like Prometheus to collect them and then generate pretty Grafana dashboards. Violation metric examples: ● The number of all deployments. ● The number of bad deployments. ● The number of all pods. ● The number of bad pods. Performance metrics examples: ● The number of seconds took to return all images from Kubernetes api. ● The number of seconds took to return all deployments from Kubernetes api.
  • 32. Why Golang? - Statically linked libraries - Ship Binary, and run it on Docker Scratch ! - Small language, New libraries, - Strongly encourages good practices (error handling, formatting, ...) - Kuberentes is written in Go itself.
  • 33. Batteries Included ! Developer Friendly ! How many times you tried to run a github project and built failed ? Some projects not only don’t give you the batteries, they won’t even give you an idea what kind of batteries you have to buy. Just push the code and good luck !
  • 34. Makefile ! make help make build-deploy-minikube
  • 35. Open Sourcing K8S Target embraces open source. second open source project at Target. After winnaker
  • 37. Thank You Open source Contributions - Daemonsets - HelmCharts - Bug fixes - Reddis and minimal technoogy stack
  • 38. Open Sourcing Challenges ! There are no bad pull requests! There are only a bad merges !
  • 39. Future of K8Guard - Engine Rule. - Plugin System. - Cloud Messaging. - Delegate notification - Consolidate Technology Stack
  • 40. Keep in touch, Make your first PR! ● https://github.com/k8guard ● https://k8guard.github.io ● Twitter: @medyadaily

Editor's Notes

  1. Matches the coming containers with best space for them.
  2. Some people who jump on the latest trendy shiny tool, miss out on the best practices.
  3. Large number of developers at Target, all excited to serve blackfriday traffic on K8S
  4. With all those problems we had, affecting 4 category, we needed to make sure will survive black friday’s traffic.
  5. Why 3 microservices ? the design was organic, we wanted separation of permissions and scopes of actions.