NixCon Berlin 2015 - Nix at LogicBlox

Smart database for next-generation applications
Nix and LogicBlox

INTRODUCTION
©2014. LogicBlox. All Rights Reserved.
▪ Rob Vermaas
▪ IRC: ikwildrpepper
▪ Email: rob.vermaas@gmail.com

MY NIX HISTORY
▪ Contributed to:
▪ Nixpkgs, Hydra, NixOps, Nix
▪ First Nix related commit
▪ August 2004, adding Octave
▪ Infrastructure support
▪ NixOS Foundation

LOGICBLOX
▪ Develops a state-of-the-art database system that is used to develop applications
for many top retailers and banks
▪ 150 people, all over the world
▪ USA, Canada, Tunisia, The Netherlands, Greece, United Kingdom, Belarus, Sweden
▪ LogicBlox Nix-ers:
▪ Eelco Dolstra
▪ Rob Vermaas
▪ Martin Bravenboer
▪ Shea Levy (former employee)
▪ Evgeny Egorochkin (contractor)

WHY DO I WANT TO USE NIX?
▪ Nix protects me against me
▪ Nix let’s me do things multiple times consistently, even on different machines
▪ Nix exposes the things I forget
▪ Nix, the one language to rule them all

HOW TO NIXIFY A COMPANY
Step 1:
Find a nice company
Step 2:
Apply Nix everywhere
Step 3:
Profit
▪ 3 step plan to Nixify a software company

HOW TO NIXIFY A COMPANY
Step 1:
Find a nice company
Step 2:
Apply Nix everywhere
Step 3:
Profit
▪ 3 step plan to Nixify a software company
Build
Test
Deploy

Nix
NIX - THE BIG FOUR
Build Test Deploy

Nix
NIX - THE BIG FOUR
Hydra
Build Test Deploy

Nix
NixOps / NixOS
NIX - THE BIG THREE
Hydra
Build Test Deploy

Nix
NIX - THE BIG THREE
Hydra
Build Test Deploy
▪ How to start? Too much to do! Can’t change everything immediately!
NixOps / NixOS

NIXIFYING EXISTING BUILDS
▪ Closed source software
▪ Buildscripts everywhere!
▪ Hardcoded locations
▪ Binary files/libraries
▪ Long builds
▪ Huge builds
▪ Network access
▪ Not just 1 language

HYDRA - CONTINUOUS BUILDS

HYDRA - CONTINUOUS BUILDS
▪ Why Hydra?
▪ All builds consolidated in one system, with one language to describe all
▪ Less maintenance
▪ Better traceability
▪ Reproducibility of build
▪ Caching of builds, very useful when composing builds
▪ Integrates with our deployment tools
▪ Terminology
▪ Projects
▪ Jobsets / Jobs
▪ Builds
▪ Features
▪ Easy to clone jobsets and build branches
▪ Ability to compose multiple builds as one logical build

HYDRA - WHAT DO WE BUILD
▪ Hydra structure:
▪ 19 active projects
▪ 205 active jobsets

HYDRA - BUILDING PLATFORM
▪ What is built?
▪ Builds, debug builds, regressions tests,
benchmarks, applications, binary releases
▪ Continuous builds
▪ Each commit -> ~ 90 builds, ~ 3 buildhours
▪ Nightly integration builds
▪ Each commit -> ~ 300 builds, ~ 120
buildhours

HYDRA - BUILDING CLIENT APPLICATIONS
{ platform_release ? "3.9.5"
, increment ? ./.
}:
with (import <config> {});
let
platform = getPlatform platform_release;
in
genericAppJobset {
build =
buildLBConfig {
name = "increment-${version increment}";
src = increment;
buildInputs = [ logicblox bloxweb ];
};
extraNixPath = { inherit increment; };
}

HYDRA - BUILDING CLIENT APPLICATIONS

Everything needed for deployment
▪ LB application (installer)
▪ Software dependencies (logicblox, bloxweb, nixpkgs)
▪ Operating system (nixpkgs)
▪ System configuration libraries for Logicblox (lbdevops)
▪ Machine definitions (increment)
HYDRA - CHARON_CLOSURE (EHHHM, NIXOPS_CLOSURE)

HYDRA - BUILDING MACHINE IMAGES
▪ Also used for NixOS EC2/GCE/VB images:
▪ <nixpkgs/nixos/lib/make-disk-image.nix>
with (import <nixpkgs> {});
vmTools.runInLinuxVM (
stdenv.mkDerivation {
name = "need-root";
buildCommand = ''
echo Do stuff only root can do
'';
})

HYDRA - TESTING ON OTHER DISTRIBUTIONS
fedora19x86_64 =
testDownloadPackage
(diskImageFuns.fedora19x86_64 {
size = 2048;
extraPackages = [
"bash"
"java-1.7.0-openjdk"
"python"
"procps-ng"
];
});

HYDRA - BUILDING DOCUMENTATION

HYDRA - BENCHMARKING
▪ Dedicated benchmarking team
using nix as benchmark composer
▪ Tracking performance over time
▪ Evaluate performance of PR’s
▪ Improvements
▪ Scheduler
▪ Hydra EC2 provisioner
▪ Prioritization of builds
Software
Hardware
D
ata
CPU x Memory x Storage
System
x
Version
x
Benchmark
Scale x Data Set

BOB / HYDRA
▪ 2011 - 2013: Started out with 3 linux machines, up to 8 machines
▪ On-premise hardware in the main office in Atlanta
▪ High initial investments
▪ Need IT people to handle hardware issues
▪ Need fast internet connection to scale out
▪ 2013-2015: 21 x Linux, 2 x Mac OS X
▪ 8 general purpose
▪ 12 dedicated benchmark
▪ Can scale up on Amazon EC2, when needed
▪ https://github.com/NixOS/hydra-provisioner

BOB / NUMBER OF BUILDS

BOB / NUMBER OF BUILD HOURS

NIXOPS
Tool to deploy networks of NixOS machines
▪ Single formalism for packaging, provisioning and configuration management
▪ Easy to add software to deployment
▪ Based on Nix package manager
▪ Expressive configuration language
▪ Supports separation of logical and physical aspects of a deployment
▪ Reuse spec for multiple cloud backends
▪ Combined configuration and provisioning
▪ Interaction between config and infrastructure

NIXOPS
+ Mixed spec
Logical
spec
+ VirtualBox
physical spec
+ EC2
physical spec
+ GCE
physical spec
EC2
Google
Physical
hardware
EC2

NIXOPS
▪ What to deploy?
▪ Where to deploy?
{ machine =
{ config, pkgs, ... }:
{ imports = [ ./nix-homepage.nix ]; };
}
{ machine =
{ deployment.targetEnv = "ec2";
deployment.ec2.accessKeyId = "ec2";
...
};
}

NIXOPS - ARGUMENTS
{ account ? "logicblox-dev"
, region ? "eu-west-1"
, instanceType ? "r3.xlarge"
, name
, enableBatch ? false
, production ? false
, ...
}:
{
…
}

NIXOPS
▪ Do we use NixOps?
▪ Hell yeah!
▪ Networks/clusters with up to 54
machines per cluster

NIXOPS
▪ Hell yeah!
▪ Last 30 days
▪ 100 ‘nixops create’
▪ 88 ‘nixops delete’
▪ 1700 ‘nixops deploy’
▪ 300 ‘nixops destroy’
▪ 40 different people running deployments

NIXOPS
▪ Hell yeah!
▪ Last 30 days
▪ 100 ‘nixops create’
▪ 88 ‘nixops delete’
▪ 1700 ‘nixops deploy’
▪ 300 ‘nixops destroy’
▪ 40 different people running deployments
Type # Where?
Internal servers ~ 50 EC2, GCE,
Hetzner, on-
premise
hardware
Client applications > 500 EC2, GCE
Dynamically scaled 0-1500 EC2

NIXOPS - OUR CURRENT SETUP
▪ Deployment servers
▪ Log in via ssh, switch to specific deploy user via sudo, containing a group’s deployments
▪ Different deployment server for development / production

NIXOPS - OUR CURRENT SETUP
▪ Issues
▪ Hard to use for non-technical users (project-managers, sales)
▪ Easy access to provider (AWS/GCE) credentials
▪ Inconsistent deployments (e.g. managing code clones)
▪ Hard to track what is going on

DEPLOYMENT - NIXOPS - OUR CURRENT SETUP
▪ Issues
▪ Hard to use for non-technical users (project-managers, sales)
▪ Easy access to provider (AWS/GCE) credentials
▪ Inconsistent deployments (e.g. managing code clones)
▪ Hard to track what is going on
▪ New project: NixOps Dashboard
▪ Web UI for NixOps , started November 1st
▪ Will be open-sourced once we have a usable version
▪ Created by Oussama Elkaceh and me.

DEPLOYMENT - NIXOPS DASHBOARD
▪ Improved security
▪ By limiting the direct access to environments that have keys to access Cloud resources. Users
will use the Dashboard (RESTful API) to perform any deployment actions.
▪ Deeper operational visibility
▪ Centralized, consistent and detailed view of the real-time status of all machines and
deployments.
▪ Audit trail
▪ Easy access and insight into what is happening and what has happened with regards to
deployments. Also, all performed operations are tracked and visible to administrators.
▪ Usability, Consistency, Predefined Actions and Operations
▪ Anyone can access and view the state of the existent deployments / historical deployments.

SUMMARY - LOGICBLOX AND NIX
▪ Nix / Hydra
▪ Reproducible, composable builds, traceability
▪ NixOS
▪ Reproducible system configuration
▪ NixOps
▪ Automation of provisioning, creating reproducible networks
of machines, one click deployment!
▪ Nix helps us, so we help Nix!
▪ Giving back by contributing to:
▪ Nix, Nixpkgs, Hydra, NixOps
▪ Supporting the infrastructure running Nix*
▪ Binary caches, EC2 capacity

THANK YOU.

NixCon Berlin 2015 - Nix at LogicBlox

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to NixCon Berlin 2015 - Nix at LogicBlox

Similar to NixCon Berlin 2015 - Nix at LogicBlox (20)

Recently uploaded

Recently uploaded (20)

NixCon Berlin 2015 - Nix at LogicBlox