Why do we need to be different like everyone else? Introducing the metric system to the UNIX countries of the world.

1. Universal Userland
Why do we need to be different just like everyone else?
Engineering
seanc@joyent.com
Sean Chittenden
@SeanChittenden

2. Getting Started
• Packer - packer.io
• Image building utility for Vagrant (VMware, Virtualbox, bhyve), AWS, Triton, GCE, Azure
• An appreciation for version controlled software
• Pick your poison: git, hg, fossil
2

3. Agenda
1. Why Part 1
2. Bootstrap an image using Packer
3. Iterate using go-coreutils
4. Why Part 2
5. Iterate using rust-coreutils
6. Parting Thoughts
3

4. Why? Part 1: What is an OS?
• Is it a kernel?
• Is it a userland?
• Is it the applications?
• Is it a release?
• Is it a distribution?
• Is it an ethos and shared set of values?
• Is it a community?
4

5. What is an OS, Kernel Edition
• CDNs? Yes. "The network stack is everything."
• Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door."
• Embedded systems? Yes. "Userland is never installed."
• Hosting Providers? Yes. "Application containment."
• Network Security Deployments? Yes. "Hardened network stack."
• Users? If it's stable, "what's a kernel?"
• Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged."
• Developers? "That's why we're here (or to support the kernel)."
• Management? Maybe.
5

6. What is an OS, Kernel Edition
6
• CDNs? Yes. "The network stack is everything."
• Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door."
• Embedded systems? Yes. "Userland is never installed."
• Hosting Providers? Yes. "Application containment."
• Network Security Deployments? Yes. "Hardened network stack."
• Users? If it's stable, "what's a kernel?"
• Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged."
• Developers? "That's why we're here (or to support the kernel)."
• Management? Maybe.
YES

7. What is an OS, Userland Edition
7
• CDNs? Partially.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "Familiarity for customers is important."
• Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code."
• Users? "I like my ls(1) and top(1)*."
* if [ "`uname -s`" = "SunOS" ]; then alias top="prstat"; fi

8. What is an OS, Userland Edition
8
• CDNs? Partially.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "Familiarity for customers is important."
• Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code."
• Users? "I like my ls(1) and top(1)."
• Operators? Yes. "This is how I deploy, maintain, investigate, and manipulate the kernel."
• Developers? It depends on who you ask. "Team effort!"
• Management? "Productivity? Effectiveness?"

9. Solving science with an Oracle
9

10. Solving science with an Oracle
10
Cloudy with a chance of Linux admins.
Management
Human Resources
Recruiting
College Grads

11. Solving science with an Oracle
11

12. What is an OS, Applications Edition
12
• CDNs? Partially. "We use nginx and/or varnish, so yeah."
• Appliance vendors? No. "Everything that matters is BSDL or proprietary already."
• Embedded systems? No. "What's a userland application?"
• Hosting Providers? Yes. "Customers can't be bothered to build 3rd party apps."
• Network Security Deployments? Yes. "Wireshark is an app, but tcpdump is kernel, right?"
• Users? "Give me X11, emacs/vi, and a webserver. Or give me death."
• Operators? Yes. This is how I deploy, maintain, investigate, and manipulate the kernel.
• Developers? "Well duh, how else would someone install Chromium?"
• Management? "Productivity? Effectiveness?"

13. Solving science with an Oracle
13

14. What is an OS, Release Engineering Edition
14
• Is an OS the release and stability guarantee?
• CDNs? No.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "IPU and upgrades are important: every customer is a snowflake."
• Network Security Deployments? No.
• Users? Yes. "IPU and upgrades on my laptop is important."
• Operators? Depends: legacy thinking, yes. Modern: not really.
• Developers? "We need to protect our users from themselves, so yes."
• Management? "Productivity?"

15. What is an OS, Distribution Edition
15
• Is an OS the distribution?
• CDNs? No.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? No.
• Network Security Deployments? No.
• Users? Yes. "IPU and upgrades on my laptop is important."
• Operators? Depends: legacy thinking, yes. Modern: not really.
• Developers? "We need to protect our users from themselves, so yes."
• Management? "We don't have to do that work anymore, right?"

16. Bootstrap an Image
16
$ git clone -q https://github.com/sean-/BSDCan-2017.git
$ cd BSDCan-2017
$ make install-cfgt
go get -u github.com/sean-/cfgt
$ which packer envchain
/Users/seanc/go/bin/packer
/opt/local/bin/envchain
$ make packer-build TEMPLATE=freebsd-userland-go.json EXTRA_ARGS=-on-error=abort
cfgt -i freebsd-userland-go.json5 -o freebsd-userland-go.json
envchain tf_triton_test
packer build
-var "image_name=my-freebsd-image"
-var "image_version=1.0.0"
-on-error=abort
freebsd-userland-go.json
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363
triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64

17. Bootstrap an Image
17
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363
triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
triton: Checking integrity... done (0 conflicting)
[snip]

18. Bootstrap an Image
18
triton: ------------------------------------------------------------------------
triton: install: /bin/cat -> /bin/cat.base
triton: install: /root/go/bin/cat -> /bin/cat
triton: install: /bin/date -> /bin/date.base
triton: install: /root/go/bin/date -> /bin/date
triton: install: /bin/echo -> /bin/echo.base
triton: install: /root/go/bin/echo -> /bin/echo
triton: install: /bin/expr -> /bin/expr.base
triton: install: /root/go/bin/expr -> /bin/expr
triton: install: /bin/mkdir -> /bin/mkdir.base
triton: install: /root/go/bin/mkdir -> /bin/mkdir
triton: install: /bin/mv -> /bin/mv.base
triton: install: /root/go/bin/mv -> /bin/mv
triton: install: /bin/pwd -> /bin/pwd.base
triton: install: /root/go/bin/pwd -> /bin/pwd
triton: install: /bin/rm -> /bin/rm.base
triton: install: /root/go/bin/rm -> /bin/rm
triton: install: /bin/rmdir -> /bin/rmdir.base
triton: install: /root/go/bin/rmdir -> /bin/rmdir
triton: install: /bin/sleep -> /bin/sleep.base
triton: install: /root/go/bin/sleep -> /bin/sleep
triton: install: /bin/sync -> /bin/sync.base
triton: install: /root/go/bin/sync -> /bin/sync
triton: install: /sbin/md5 -> /sbin/md5.base
triton: install: /root/go/bin/md5sum -> /sbin/md5
triton: install: /sbin/sha1 -> /sbin/sha1.base

19. Bootstrap an Image
19
triton: install: /sbin/sha1 -> /sbin/sha1.base
triton: install: /root/go/bin/sha1sum -> /sbin/sha1
triton: install: /root/go/bin/sha224sum -> /sbin/sha224
triton: install: /sbin/sha256 -> /sbin/sha256.base
triton: install: /root/go/bin/sha256sum -> /sbin/sha256
triton: install: /sbin/sha384 -> /sbin/sha384.base
triton: install: /root/go/bin/sha384sum -> /sbin/sha384
triton: install: /sbin/sha512 -> /sbin/sha512.base
triton: install: /root/go/bin/sha512sum -> /sbin/sha512
triton: install: /usr/bin/basename -> /usr/bin/basename.base
triton: install: /root/go/bin/basename -> /usr/bin/basename
triton: install: /usr/bin/dirname -> /usr/bin/dirname.base
triton: install: /root/go/bin/dirname -> /usr/bin/dirname
triton: install: /usr/bin/env -> /usr/bin/env.base
triton: install: /root/go/bin/env -> /usr/bin/env
triton: install: /usr/bin/factor -> /usr/bin/factor.base
triton: install: /root/go/bin/factor -> /usr/bin/factor
triton: install: /usr/bin/false -> /usr/bin/false.base
triton: install: /root/go/bin/false -> /usr/bin/false
triton: install: /usr/bin/groups -> /usr/bin/groups.base
triton: install: /root/go/bin/groups -> /usr/bin/groups
triton: install: /usr/bin/head -> /usr/bin/head.base
triton: install: /root/go/bin/head -> /usr/bin/head
triton: install: /usr/bin/logname -> /usr/bin/logname.base
triton: install: /root/go/bin/logname -> /usr/bin/logname
triton: install: /usr/bin/tail -> /usr/bin/tail.base

20. Bootstrap an Image
20
triton: install: /root/go/bin/tail -> /usr/bin/tail
triton: install: /usr/bin/tee -> /usr/bin/tee.base
triton: install: /root/go/bin/tee -> /usr/bin/tee
triton: install: /usr/bin/touch -> /usr/bin/touch.base
triton: install: /root/go/bin/touch -> /usr/bin/touch
triton: install: /usr/bin/true -> /usr/bin/true.base
triton: install: /root/go/bin/true -> /usr/bin/true
triton: install: /usr/bin/tsort -> /usr/bin/tsort.base
triton: install: /root/go/bin/tsort -> /usr/bin/tsort
triton: install: /usr/bin/wc -> /usr/bin/wc.base
triton: install: /root/go/bin/wc -> /usr/bin/wc
triton: install: /usr/bin/whoami -> /usr/bin/whoami.base
triton: install: /root/go/bin/whoami -> /usr/bin/whoami
triton: install: /usr/bin/yes -> /usr/bin/yes.base
triton: install: /root/go/bin/yes -> /usr/bin/yes

21. go-coreutils: Initial thoughts?
21
triton:
triton: Rebuilding locate database:
triton: flag provided but not defined: -128
triton: Usage of head:
triton: -bytes int
triton: output the first K bytes
triton: -c int
triton: output the first K bytes
triton: -help
triton:
triton: Usage: head [OPTION]... [FILE]...
triton:
triton: Print the first 10 lines of each FILE to standard output. With more than one FILE, precede each with a header
giving the file name. With no FILE, or when FILE is -, read standard input.
triton:
triton:
triton: Mandatory arguments to long options are mandatory for short options too.
triton:
triton: -help display this help and exit
triton: -version output version information and exit
triton:
triton: -c, --bytes=K
triton: output the first K bytes; or use -n +K to output starting with the Kth byte.

22. go-coreutils: Initial thoughts?
22
triton: -n, -lines=K
triton: output the first K lines; or use -n +K to output starting with
triton: the Kth
triton:
triton: -q, -quiet, -silent
triton: never output headers giving file names
triton:
triton: -lines int
triton: output the first K lines (default 10)
triton: -n int
triton: output the first K lines (default 10)
triton: -qnever output headers giving file names
triton: -quiet
triton: never output headers giving file names
triton: -silent
triton: never output headers giving file names
triton: -version
triton:
triton: head (go-coreutils) 0.1
triton:
triton: Copyright (C) 2014, The GO-Coreutils Developers.
triton: This program comes with ABSOLUTELY NO WARRANTY; for deprintTails see
triton: LICENSE. This is free software, and you are welcome to redistribute
triton: it under certain conditions in LICENSE.
triton:

23. go-coreutils: Initial thoughts?
23
• Needs to bake more
• Probably not going anywhere
• GPLv3ed code
• Suspicion: No commercial incentive or expectation of quality

24. Why? Part 2: What is Progress?
24

25. Science: A lens through which we learn
• Whereas research has shown that to a considerable extent children learn by imitating the
behavior of adults, especially those they are dependent upon; and the use of corporal
punishment by adults having authority over children is likely to train children to use physical
violence to control behavior rather than rational persuasion, education, and intelligent forms
of both positive and negative reinforcement;
• Whereas research has shown that the effective use of punishment in eliminating
undesirable behavior requires precision in timing, duration, intensity, and specificity, as well
as considerable sophistication in controlling a variety of relevant environmental and
cognitive factors, such that punishment administered in institutional settings, without
attention to all these factors, is likely to instill hostility, rage, and a sense of powerlessness
without reducing the undesirable behavior;
• Therefore, be it resolved that the American Psychological Association opposes the use of
corporal punishment in schools, juvenile facilities, child care nurseries, and all other
institutions, public or private, where children are cared for or educated (Conger, 1975).
American Psychological Association
http://www.apa.org/about/policy/corporal-punishment.aspx
25

26. Software?
26

27. Software Performance?
Performance? Find regression, fix regression.
https://github.com/freebsd/freebsd/commit/89c59fbb17d89068626d6214477d40c81d43a025
27

28. Software Usability?
28
$ echo baz | tee foo > bar
$ echo foo >> foo
$ echo bar >> bar
$ diff foo bar
1c1
< foo
---
> bar
$ date
Sat May 10 15:49:02 UTC 2017
$ diff -u foo bar
--- foo2017-06-10 15:48:47.261401000 +0000
+++ bar2017-06-10 15:48:51.157525000 +0000
@@ -1,2 +1,2 @@
baz
-foo
+bar

29. Could we have learned something?
29

30. Software Usability?
30
The UNIX system family tree: Research and BSD
---------------------------------------------
First Edition (V1)
|
Second Edition (V2)
|
Third Edition (V3)
|
Fourth Edition (V4)
|
Fifth Edition (V5)
|
Sixth Edition (V6) -----*
|
|
|
Seventh Edition (V7) |
|
1BSD
32V |
2BSD---------------*
/ |
/ |
/ |
3BSD |
| |
4.0BSD 2.79BSD
| |
4.1BSD --------------> 2.8BSD
| |
4.1aBSD ----------- |
| |
4.1bBSD |
| |
*------ 4.1cBSD --------------> 2.9BSD

31. Software Usability?
31
/ | |
Eighth Edition | 2.9BSD-Seismo
| | |
+----<--- 4.2BSD 2.9.1BSD
| | |
+----<--- 4.3BSD -------------> 2.10BSD
| | / |
Ninth Edition | / 2.10.1BSD
| 4.3BSD Tahoe-----+ |
| | |
| | |
v | 2.11BSD
Tenth Edition | |
| 2.11BSD rev #430
4.3BSD NET/1 |
| v
4.3BSD Reno
|
*---------- 4.3BSD NET/2 -------------------+-------------*
| | | |
386BSD 0.0 | | BSD/386 ALPHA
| | | |
386BSD 0.1 ------------>+ | BSD/386 0.3.[13]
| | 4.4BSD Alpha |
| 386BSD 1.0 | | BSD/386 0.9.[34]
| | 4.4BSD |
| | / | |
| | 4.4BSD-Encumbered | |
| -NetBSD 0.8 | BSD/386 1.0
| / | | |
FreeBSD 1.0 <-----' NetBSD 0.9 | BSD/386 1.1
| | .----- 4.4BSD Lite |
FreeBSD 1.1 | / / | |
| | / / | |
FreeBSD 1.1.5 .---|--------' / | |
| / | / | |

32. Software Usability?
32
FreeBSD 1.1.5.1 / | / | |
| / NetBSD 1.0 <-' | |
| / | | |
FreeBSD 2.0 <--' | | BSD/OS 2.0
| | |
FreeBSD 2.0.5 | BSD/OS 2.0.1
| .------------------ 4.4BSD Lite2 |
| | | | | | |
| | .-----|------Rhapsody | | | |
| | | | NetBSD 1.3 | | |
| | | | OpenBSD 2.3 | |
| | | | BSD/OS 3.0 |
FreeBSD 2.1 | | | |
| | | | NetBSD 1.1 ------. BSD/OS 2.1
| FreeBSD 2.1.5 | | | |
| | | | NetBSD 1.2 BSD/OS 3.0
| FreeBSD 2.1.6 | | | OpenBSD 2.0 |
| | | | | | |
| FreeBSD 2.1.6.1 | | | | |
| | | | | | |
| FreeBSD 2.1.7 | | | | | |
| | | | | NetBSD 1.2.1 | |
| FreeBSD 2.1.7.1 | | | | |
| | | | | |
| | | | | |
*-FreeBSD 2.2 | | | | |
| | | | | |
| FreeBSD 2.2.1 | | | | |
| | | | | | |
| FreeBSD 2.2.2 | | | OpenBSD 2.1 |
| | | | | | |
| FreeBSD 2.2.5 | | | | |
| | | | | OpenBSD 2.2 |
| | | | NetBSD 1.3 | |
| FreeBSD 2.2.6 | | | | | |
| | | | | NetBSD 1.3.1 | BSD/OS 3.1

33. Software Usability?
33
| | | | | | OpenBSD 2.3 |
| | | | | NetBSD 1.3.2 | |
| FreeBSD 2.2.7 | | | | | |
| | | | | | | BSD/OS 4.0
| FreeBSD 2.2.8 | | | | | |
| | | | | | | |
| v | | | | OpenBSD 2.4 |
| FreeBSD 2.2.9 | | | | | |
| | | | | | |
FreeBSD 3.0 <--------* | | v | |
| | | NetBSD 1.3.3 | |
*---FreeBSD 3.1 | | | |
| | | | | BSD/OS 4.0.1
| FreeBSD 3.2----* | NetBSD 1.4 OpenBSD 2.5 |
| | | | | | | | |
| | | | | | | | |
| | | | | | | | |
| FreeBSD 3.3 | | | | NetBSD 1.4.1 | |
| | | | | | | OpenBSD 2.6 |
| FreeBSD 3.4 | | | | | | |
| | | | | | | | BSD/OS 4.1
FreeBSD 4.0 | | | | | NetBSD 1.4.2 | |
| | | | | | | | |
| | | | | | | | |
| FreeBSD 3.5 | | | | | OpenBSD 2.7 |
| | | | | | | | |
| FreeBSD 3.5.1 | | | | | | |
| | | | | | | |
*---FreeBSD 4.1 | | | | | | |
| | | | (?) | | | |
| FreeBSD 4.1.1 | | / | | | |
| | | | / | | | |
| FreeBSD 4.2 Darwin/ | NetBSD 1.4.3 | |
| | Mac OS X | OpenBSD 2.8 BSD/OS 4.2
| | | | | |
| | | | | |

34. Software Usability?
34
| | 10.0 NetBSD 1.5 | |
| FreeBSD 4.3 | | | | |
| | | | | OpenBSD 2.9 |
| | | | NetBSD 1.5.1 | |
| | | | | | |
| FreeBSD 4.4-. | | NetBSD 1.5.2 | |
| | | Mac OS X | | | |
| | | 10.1 | | OpenBSD 3.0 |
| FreeBSD 4.5 | | | | | |
| | | | | | BSD/OS 4.3
| FreeBSD 4.6 | | | OpenBSD 3.1 |
| | | | NetBSD 1.5.3 | |
| FreeBSD 4.6.2 Mac OS X | | |
| | 10.2 | | |
| FreeBSD 4.7 | | | |
| | | NetBSD 1.6 OpenBSD 3.2 |
| FreeBSD 4.8 | | | | |
| | | | NetBSD 1.6.1 | |
| |--------. | | | OpenBSD 3.3 BSD/OS 5.0
| | | | | | |
| FreeBSD 4.9 | | | | OpenBSD 3.4 BSD/OS 5.1 ISE
| | | | | | |
| | | | | NetBSD 1.6.2 |
| | | | | | |
| | | | | | OpenBSD 3.5
| | | | | v |
| FreeBSD 4.10 | | | |
| | | | | |
| FreeBSD 4.11 | | | |
| | | | |
| `-|------|-----------------|---------------------.
| | | |
FreeBSD 5.0 | | | |
| | | | |
FreeBSD 5.1 | | | DragonFly 1.0
| | | | |

35. Software Usability?
35
| ----- Mac OS X | | |
| 10.3 | | |
FreeBSD 5.2 | | | |
| | | | | |
| FreeBSD 5.2.1 | | | |
| | | | |
*-------FreeBSD 5.3 | | | |
| | | | OpenBSD 3.6 |
| | | NetBSD 2.0 | |
| | | | | | | DragonFly 1.2.0
| | Mac OS X | | NetBSD 2.0.2 | |
| | 10.4 | | | | |
| FreeBSD 5.4 | | | | | |
| | | | | | OpenBSD 3.7 |
| | | | | NetBSD 2.0.3 | |
| | | | | | | |
*--FreeBSD | | | | v OpenBSD 3.8 |
| 6.0 | | | | | |
| | | | | | |
| | | | | NetBSD 2.1 | |
| | | | | | |
| | | | NetBSD 3.0 | |
| | | | | | | | DragonFly 1.4.0
| | | | | | | OpenBSD 3.9 |
| FreeBSD | | | | | | |
| 6.1 | | | | | | |
| | FreeBSD 5.5 | | | | | |
| | | | | NetBSD 3.0.1 | DragonFly 1.6.0
| | | | | | | |
| | | | | | OpenBSD 4.0 |
| | | | | NetBSD 3.0.2 | |
| | | | NetBSD 3.1 | |
| FreeBSD 6.2 | | | |
| | | | | DragonFly 1.8.0
| | | | OpenBSD 4.1 |
| | | | | DragonFly 1.10.0

36. Software Usability?
36
| | Mac OS X | | |
| | 10.5 | | |
| | | | OpenBSD 4.2 |
| | | NetBSD 4.0 | |
| FreeBSD 6.3 | | | | |
| | | | | |
*--FreeBSD | | | | | DragonFly 1.12.0
| 7.0 | | | | | |
| | | | | | OpenBSD 4.3 |
| | | | | NetBSD | DragonFly 2.0.0
| | FreeBSD | | 4.0.1 OpenBSD 4.4 |
| | 6.4 | | | |
| | | | | |
| FreeBSD 7.1 | | | |
| | | | | DragonFly 2.2.0
| FreeBSD 7.2 | NetBSD 5.0 OpenBSD 4.5 |
| | | | | |
| | Mac OS X | | | |
| | 10.6 | | | |
| | | | | NetBSD | DragonFly 2.4.0
| | | | | 5.0.1 OpenBSD 4.6 |
| | | | | | | |
*--FreeBSD | | | | | | |
| 8.0 | | | | | | |
| | FreeBSD | | | NetBSD | |
| | 7.3 | | | 5.0.2 | DragonFly 2.6.0
| | | | | | OpenBSD 4.7 |
| FreeBSD | | | | | |
| 8.1 | | | | | |
| | | | | | | DragonFly 2.8.2
| | | | | | OpenBSD 4.8 |
| | | | | *--NetBSD | |
| FreeBSD FreeBSD | | | 5.1 | |
| 8.2 7.4 | | | | | DragonFly 2.10.1
| | | | | | OpenBSD 4.9 |
| `-----. Mac OS X | | | | |

37. Software Usability?
37
| 10.7 | | | | |
| | | | | | OpenBSD 5.0 |
*--FreeBSD | | | | | | |
| 9.0 | | | | NetBSD | DragonFly 3.0.1
| | FreeBSD | | | 5.1.2 | |
| | 8.3 | | | | | |
| | | | | | NetBSD | |
| | | | | | 5.1.3 | |
| | | | | | | | |
| | | | | | NetBSD | |
| | | | | | 5.1.4 | |
| | | | | | OpenBSD 5.1 |
| | | Mac OS X | `----. | |
| | | 10.8 | | |
| | | | NetBSD 6.0 | | |
| | | | | | | | OpenBSD 5.2 DragonFly 3.2.1
| FreeBSD | | | | | NetBSD | |
| 9.1 | | | | | 5.2 | |
| | | | | | | | | |
| | | | | | | NetBSD | |
| | | | | | | 5.2.1 | |
| | | | | | | | | |
| | | | | | | NetBSD | |
| | | | | | | 5.2.2 | |
| | | | | | | | |
| | | | | | | |
| | | | | | NetBSD | |
| | | | | | 6.0.1 | |
| | | | | | | OpenBSD 5.3 DragonFly 3.4.1
| | | | | | NetBSD | |
| | | | | | 6.0.2 | |
| | | | | | | | |
| | | | | | NetBSD | |
| | | | | | 6.0.3 | |
| | | | | | | | |
| | | | | | NetBSD | |

38. Software Usability?
38
| | | | | | 6.0.4 | |
| | | | | | | | |
| | | | | | NetBSD | |
| | | | | | 6.0.5 | |
| | | | | | | |
| | | | | |`-NetBSD 6.1 | |
| | FreeBSD | | | | |
| | 8.4 | | NetBSD 6.1.1 | |
| | | | | | |
| FreeBSD | | NetBSD 6.1.2 | |
| 9.2 Mac OS X | | | |
| | 10.9 | | OpenBSD 5.4 |
| `-----. | | | | DragonFly 3.6.0
| | | | | |
*--FreeBSD | | | NetBSD 6.1.3 | |
| 10.0 | | | | | |
| | | | | | | DragonFly 3.6.1
| | | | | | | |
| | | | | | | |
| | | | | | | DragonFly 3.6.2
| | | | | NetBSD 6.1.4 | |
| | | | | | | |
| | | | | | OpenBSD 5.5 |
| | | | | | | |
| | | | | | | DragonFly 3.8.0
| | | | | | | |
| | | | | | | |
| | | | | | | DragonFly 3.8.1
| | | | | | | |
| | | | | | | |
| | | | | | | DragonFly 3.6.3
| | | | | | | |
| | FreeBSD | | | | |
| | 9.3 | | | | |
| | | | NetBSD 6.1.5 | DragonFly 3.8.2
| | Mac OS X | | |

39. Software Usability?
39
| | 10.10 | | |
| | | | OpenBSD 5.6 |
| FreeBSD | | | |
| 10.1 | | | DragonFly 4.0.1
| | | | | |
| | | | | DragonFly 4.0.2
| | | | | |
| | | | | DragonFly 4.0.3
| | | | | |
| | | | | DragonFly 4.0.4
| | | | | |
| | | | | DragonFly 4.0.5
| | | | | |
| | | | OpenBSD 5.7 |
| | | | | DragonFly 4.2.0
| FreeBSD | | | |
| 10.2 | | | |
| | OS X NetBSD 7.0 | |
| | 10.11 | OpenBSD 5.8 |
| | | | | DragonFly 4.4.1
| FreeBSD | | OpenBSD 5.9 |
| 10.3 | | | |
| | NetBSD 7.0.1 | |
| | | | DragonFly 4.6.0
| | | | |
| | | | |
*--FreeBSD macOS | OpenBSD 6.0 |
| 11.0 10.12 | | |
| | NetBSD 7.1 | |
| | | | DragonFly 4.8.0
| | | OpenBSD 6.1 |
| | | | |
FreeBSD 12 -current | NetBSD -current OpenBSD -current DragonFly -current
| | | | |
v v v v v

40. Software Usability?
40
Time
----------------
Time tolerance +/- 6 months, depending on which book/article you read; if it
was the announcement in Usenet or if it was available as tape.
[44B] McKusick, Marshall Kirk, Keith Bostic, Michael J Karels,
and John Quarterman. The Design and Implementation of
the 4.4BSD Operating System.
[APL] Apple website [http://www.apple.com/macosx/]
[BSDI] Berkeley Software Design, Inc.
[DFB] DragonFlyBSD Project, The.
[DOC] README, COPYRIGHT on tape.
[FBD] FreeBSD Project, The.
[KB] Keith Bostic. BSD2.10 available from Usenix. comp.unix.sources,
Volume 11, Info 4, April, 1987.
[KKK] Mike Karels, Kirk McKusick, and Keith Bostic. tahoe announcement.
comp.bugs.4bsd.ucb-fixes, June 15, 1988.
[KSJ] Michael J. Karels, Carl F. Smith, and William F. Jolitz.
Changes in the Kernel in 2.9BSD. Second Berkeley Software
Distribution UNIX Version 2.9, July, 1983.
[NBD] NetBSD Project, The.
[OBD] OpenBSD Project, The.
[QCU] Salus, Peter H. A quarter century of UNIX.
[SMS] Steven M. Schultz. 2.11BSD, UNIX for the PDP-11.
[TUHS] The Unix Historical Society. http://minnie.tuhs.org/Unix_History/.
[USE] Usenet announcement.
[WRS] Wind River Systems, Inc.
[dmr] Dennis Ritchie, via E-Mail
Multics 1965
UNIX Summer 1969
DEC PDP-7
First Edition 1971-11-03 [QCU]
DEC PDP-11/20, Assembler
Second Edition 1972-06-12 [QCU]
1965

41. Software Usability?
41
Multics 1965
UNIX Summer 1969
DEC PDP-7
[snip]
FreeBSD 11.0 2016-10-10 [FBD]
NetBSD 7.1 2017-03-11 [NBD]
DragonFly 4.8.0 2017-03-27 [DFB]
OpenBSD 6.1 2017-04-11 [OBD]
2017

42. Software Usability?
42
Multics 1965
UNIX Summer 1969
DEC PDP-7
[snip]
FreeBSD 11.0 2016-10-10 [FBD]
NetBSD 7.1 2017-03-11 [NBD]
DragonFly 4.8.0 2017-03-27 [DFB]
OpenBSD 6.1 2017-04-11 [OBD]
2017-1965=??

43. Software Usability?
43
Multics 1965
UNIX Summer 1969
DEC PDP-7
[snip]
FreeBSD 11.0 2016-10-10 [FBD]
NetBSD 7.1 2017-03-11 [NBD]
DragonFly 4.8.0 2017-03-27 [DFB]
OpenBSD 6.1 2017-04-11 [OBD]
2017-1965=52

44. Oracle: Could We Have Learned Something?
44
Yes definitely

45. Why? Part 2: What is Progress?
45

46. Why? Part 2: Why the quagmire?
• Pluralistic Base Existed
• Ended up with a base that is only made up of self-bootstrapped bits
• Deprecated everything not C or sh(1)
• Deorbiting base to ports
• Why are base utilities limited to C and sh(1) again if we're building base with
poudriere(1)?
46

47. Automation to the Rescue
• Building releases image is now "easy" by comparison
• Debian GNU/kFreeBSD was a big effort
• Gentoo FreeBSD was a big effort
• Rust/FreeBSD? Pretty trivial
47

48. Bootstrap an Image
48
$ make packer-build TEMPLATE=freebsd-userland-rust.json EXTRA_ARGS=-on-error=abort
cfgt -i freebsd-userland-rust.json5 -o freebsd-userland-rust.json
envchain tf_triton_test
packer build
-var "image_name=my-freebsd-image"
-var "image_version=1.0.0"
-on-error=abort
freebsd-userland-rust.json
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
[snip]
triton: gmake: Entering directory '/root/coreutils'
triton: cargo build --features "arch base32 base64 basename cat chgrp chmod chown chroot cksum comm cp cut dircolors
dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install kill link ln logname
mkdir mkfifo mknod mktemp more mv nice nl nohup nproc od paste pathchk pinky printenv printf ptx pwd readlink realpath
relpath rm rmdir seq shred shuf sleep sort split stdbuf sum sync tac tail tee test timeout touch tr true truncate tsort
tty uname unexpand uniq unlink uptime users wc whoami yes" --release --no-default-features
triton: Updating registry `https://github.com/rust-lang/crates.io-index`
triton: Updating git repository `https://github.com/rust-lang/libc.git`
triton: Updating registry `https://github.com/rust-lang/crates.io-index`

49. Bootstrap an Image
49
triton: Finished release [optimized] target(s) in 0.66 secs
triton: cargo build --release -p unexpand
triton: Compiling unexpand v0.0.1 (file:///root/coreutils/src/unexpand)
triton: Finished release [optimized] target(s) in 0.57 secs
triton: cargo build --release -p uniq
triton: Compiling uniq v0.0.1 (file:///root/coreutils/src/uniq)
triton: Finished release [optimized] target(s) in 0.56 secs
triton: cargo build --release -p unlink
triton: Compiling unlink v0.0.1 (file:///root/coreutils/src/unlink)
triton: Finished release [optimized] target(s) in 0.57 secs
triton: cargo build --release -p uptime
triton: Compiling uptime v0.0.1 (file:///root/coreutils/src/uptime)
triton: Finished release [optimized] target(s) in 0.56 secs
triton: cargo build --release -p users
triton: Compiling users v0.0.1 (file:///root/coreutils/src/users)
triton: Finished release [optimized] target(s) in 0.56 secs
triton: cargo build --release -p wc
triton: Compiling wc v0.0.1 (file:///root/coreutils/src/wc)
triton: Finished release [optimized] target(s) in 0.63 secs
triton: cargo build --release -p whoami
triton: Compiling whoami v0.0.1 (file:///root/coreutils/src/whoami)
triton: Finished release [optimized] target(s) in 0.57 secs
triton: cargo build --release -p yes
triton: Compiling yes v0.0.1 (file:///root/coreutils/src/yes)
triton: Finished release [optimized] target(s) in 0.61 secs
triton: mkdir -p /usr/obj/bin

50. Bootstrap an Image
50
triton: install /root/coreutils/target/release/arch /usr/obj/bin/arch; install /root/coreutils/target/release/base32
/usr/obj/bin/base32; install /root/coreutils/target/release/base64 /usr/obj/bin/base64; install /root/coreutils/target/
release/basename /usr/obj/bin/basename; install /root/coreutils/target/release/cat /usr/obj/bin/cat; install /root/
coreutils/target/release/chgrp /usr/obj/bin/chgrp; install /root/coreutils/target/release/chmod /usr/obj/bin/chmod;
install /root/coreutils/target/release/chown /usr/obj/bin/chown; install /root/coreutils/target/release/chroot /usr/obj/
bin/chroot; install /root/coreutils/target/release/cksum /usr/obj/bin/cksum; install /root/coreutils/target/release/
comm /usr/obj/bin/comm; install /root/coreutils/target/release/cp /usr/obj/bin/cp; install /root/coreutils/target/
release/cut /usr/obj/bin/cut; install /root/coreutils/target/release/dircolors /usr/obj/bin/dircolors; install /root/
coreutils/target/release/dirname /usr/obj/bin/dirname; install /root/coreutils/target/release/du /usr/obj/bin/du;
install /root/coreutils/target/release/echo /usr/obj/bin/echo; install /root/coreutils/target/release/env /usr/obj/bin/
env; install /root/coreutils/target/release/expand /usr/obj/bin/expand; install /root/coreutils/target/release/expr /
usr/obj/bin/expr; install /root/coreutils/target/release/factor /usr/obj/bin/factor; install /root/coreutils/target/
release/false /usr/obj/bin/false; install /root/coreutils/target/release/fmt /usr/obj/bin/fmt; install /root/coreutils/
target/release/fold /usr/obj/bin/fold; install /root/coreutils/target/release/groups /usr/obj/bin/groups; install /
root/coreutils/target/release/hashsum /usr/obj/bin/hashsum; install /root/coreutils/target/release/head /usr/obj/bin/
head; install /root/coreutils/target/release/hostid /usr/obj/bin/hostid; install /root/coreutils/target/release/
hostname /usr/obj/bin/hostname; install /root/coreutils/target/release/id /usr/obj/bin/id; install /root/coreutils/
target/release/install /usr/obj/bin/install; install /root/coreutils/target/release/kill /usr/obj/bin/kill; install /
root/coreutils/target/release/link /usr/obj/bin/link; install /root/coreutils/target/release/ln /usr/obj/bin/ln;
install /root/coreutils/target/release/logname /usr/obj/bin/logname; install /root/coreutils/target/release/mkdir /usr/
obj/bin/mkdir; install /root/coreutils/target/release/mkfifo /usr/obj/bin/mkfifo; install /root/coreutils/target/
release/mknod /usr/obj/bin/mknod; install /root/coreutils/target/release/mktemp /usr/obj/bin/mktemp; install /root/
coreutils/target/release/more /usr/obj/bin/more; install /root/coreutils/target/release/mv /usr/obj/bin/mv; install /
root/coreutils/target/release/nice /usr/obj/bin/nice; install /root/coreutils/target/release/nl /usr/obj/bin/nl;
install /root/coreutils/target/release/nohup /usr/obj/bin/nohup; install /root/coreutils/target/release/nproc /usr/obj/
bin/nproc; install /root/coreutils/target/release/od /usr/obj/bin/od; install /root/coreutils/target/release/paste /

51. Bootstrap an Image
51
triton:
triton: Rebuilding locate database:
$ echo It worked
ls(1), stat(1), and who(1) don't work and install(1) is missing options. Plus more WIP problems.

52. Parting Thoughts
• Swapping out syscall(2) tables is somehow okay with us
• Do we package each binary?
• Can we version binaries? We do shared objects and APIs
• Squatting on the filesystem is not versioning or an API
• What history? Nuke and pave. No instance up older than 3 years.
• Need tight coupling? Depend on a specific package.
• How can we embrace plurality (read: R&D)
• SmartOS and OmniOS uses different userlands, all slightly different
• What history? Nuke and pave. No instance up older than 3 years.
52

53. Parting Thoughts
• Writing BSD licensed software gives you access to the guild of like-minded
commercially motivated individuals who require:
• quality
• debugability
• observability
• performance
• shipability
53

54. Parting Thoughts
• Exporting an OSes base utilities takes excessive effort as the modern
programming languages have shown
• Embrace the future
• If we could start over knowing we had 10M LoC and >1K contributors, what and
how would we build an OS?
• If the ideal is different than what we're doing now, let's begin to plan how to get to
an ideal situation
Post Talk Comment: "This is like introducing the metric system to the UNIX countries
of the world."
54
This resonated with me

55. Engineering
seanc@joyent.com
Sean Chittenden
@SeanChittenden
Thank you! Questions?
https://github.com/sean-/BSDCan-2017
https://www.slideshare.net/SeanChittenden/universal-userland