Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Ähnlich wie Packaging is the Worst Way to Distribute Software, Except for Everything Else
Ähnlich wie Packaging is the Worst Way to Distribute Software, Except for Everything Else (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Packaging is the Worst Way to Distribute Software, Except for Everything Else
- 1. Packaging is the Worst Way to Distribute Software, except for everything else Ryan McKern | Puppet Labs mckern@puppetlabs.com
- 2. Who is this?
- 3. What do I do now? Release Engineering at
- 4. Maybe you've used our products?
- 5. What have I done? System Administration … for 13 years.
- 6. I’ve probably been your customer
- 8. "Prejudice is a great time saver. You can form opinions without having to get the facts." Attributed to E.B. White, Source unknown
- 9. Let's talk about software!
- 10. Distributing software sucks Shipping new platforms is so hard Cross-platform packaging is so hard Unpredictable user-space is so hard Moving the packaged bits is so hard
- 11. Everything is so hard
- 12. Who among us knows this pain? sad@roberto Downloads $ wget -‐-‐quiet http:// ftpmirror.gnu.org/gcc/gcc-‐4.9.1/gcc-‐4.9.1.tar.bz2 sad@roberto Downloads $ tar xjf gcc-‐4.9.1.tar.bz2 sad@roberto Downloads $ cd gcc-‐4.9.1/ sad@roberto Downloads $ ./configure ./configure: line 532: sed: command not found ./configure: line 1371: sed: command not found ./configure: line 1920: sed: command not found ./configure: line 2291: sed: command not found configure: error: cannot run /bin/sh ./config.sub ./configure: line 361: sed: command not found ./configure: line 310: sort: command not found
- 13. This was a problem because the customer's time has value
- 14. Behold! ryan@animatronio ~ $ sudo rpm -‐Uvh http://my.mirror.co/pub/ el/7/x86_64/nano-‐2.3.1-‐10.el7.x86_64.rpm Retrieving http://my.mirror.co/pub/el/7/x86_64/ nano-‐2.3.1-‐10.el7.x86_64.rpm Preparing... ################################# [100%] Updating / installing... 1:nano-‐2.3.1-‐10.el7 ################################# [100%] ryan@animatronio ~ $
- 16. What's so great about packages?
- 17. Dependency management calculon ~ # apt-‐get install cmake Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: cmake-‐data emacsen-‐common libarchive12 libnettle4 libxmlrpc-‐core-‐c3 The following NEW packages will be installed: cmake cmake-‐data emacsen-‐common libarchive12 libnettle4 libxmlrpc-‐core-‐c3 0 upgraded, 6 newly installed, 0 to remove and 51 not upgraded.
- 18. Verification bender ~ # mv /usr/share/man/man8/applydeltarpm.8.gz ~/ bender ~ # rpm -‐V deltarpm missing d /usr/share/man/man8/applydeltarpm.8.gz bender ~ #
- 19. Distribution ryan@tinnytim ~ $ gem push erniebert-‐0.1.0.gem Pushing gem to BetterThanRubyGems.org... Successfully registered gem: erniebert (0.1.0) ryan@tinnytim ~ $ gem install erniebert Fetching: ffi-‐1.9.6.gem (100%) Building native extensions. This could take a while... Successfully installed ffi-‐1.9.6 Fetching: erniebert-‐0.1.0.gem (100%) Successfully installed erniebert-‐0.7.1 1 gem installed ryan@tinnytim ~ $
- 20. What could be better about packages?
- 21. Sometimes shipping bits really is hard
- 22. Security is often both the joke and the punchline ouch@killbot ~ $ dpkg-‐sig -‐-‐verify puppet_3.7.1-‐1puppetlabs1_all.deb Processing ./puppet_3.7.1-‐1puppetlabs1_all.deb... GOODSIG _gpgbuilder C093A3A56A6E0BEEA2821DD7133957EA11028DF3 1413702159 ouch@killbot ~ $ dpkg-‐sig -‐-‐verify ./puppet_2.7.23-‐1~deb7u3_all.deb Processing ./puppet_2.7.23-‐1~deb7u3_all.deb... ouch@killbot ~ $
- 23. So, so, so many similar-but-different formats
- 24. Let's talk about some popular packaging formats
- 25. .rpm • Managed by the recursively named "RPM Package Manager" & yum • cpio compressed binaries & text files • Post-installation tasks are shell scripts
- 26. .deb • Managed by dpkg & apt, the "Advanced Package Tool" • ar compressed package with two gzipped tarballs & a small text file • Post-installation tasks are shell scripts
- 27. Mac .pkg • Used by Mac OS X, and often delivered in a .dmg (disk image) or a .zip file • xar compressed archive, containing a binary file, two archives, and an XML document • post-installation tasks are still shell scripts
- 28. About all those post-install shell scripts Maybe they're not that safe, but the surface area of this problem is big. That doesn't mean we needed "dash"
- 29. Ruby .gem, Python .egg, and Node .npm • These are library managers with delusions of grandeur • Reuses the "download, decompress, configure, build, install" patterns, which hasn't got much spam in it • Constant compilation is a bummer
- 30. What about... ? #realtalk We only have 45 minutes, and I hope you're going to have some questions for me to evade
- 31. What are some alternatives?
- 32. Source tarballs
- 34. curl | bash
- 36. Full Disclosure • Puppet Labs does use the curl|bash technique as an option for our PE agent installation • If you don't trust your own Puppet Master, who do you trust? • (ALL THE COOL KIDS WERE DOING IT)
- 37. You just wanted Ruby but you got Cthulhu ~ $ curl -‐sSL https://get.rvm.io | bash -‐s -‐-‐ -‐-‐fhtagn G̺̞ ̯͔̮̫̥ ͊̌͂ a͍͕͓̦͈̯̟̋ r̘̰̟e̓̓ ̦ C̋͋ͬt̂̅̓ t͇̻̩̲̬ ͇̪̹͔̾ ̟ḧ́΅ͭ ̩̿ͭ ͖̙̤ ͭl̅ͦ̓ ̝̙̭ ̗ ṷ ů̥͖ ͍͎͍ ̦̟ n̠̣̭ ̞̻̱̳ ̬ ̣̗ ̑ ̖͎ͩ hͯ ͐ ̝̤̊ ̞̭̳͚̞ ̘ s ̓ ͔̣ ̺̝͇ l̃ͪ͐̎̍ a̅͋̏̀ ̜̯͉ ͈͇̲ ̓̑ ̭̻l̂ͬ̽ ͮ ̙͇̼͍ r̭̂̋ͦ ̻̺̭ ̗͙̃ ̻̤̳ ̰̤ i̅̿̌ͫͣͪ ̺̙̽_̻͚ ̤ s ̮͇ e͍̞̚ ̿̌ͮ̍ ̝͕̳́̽ ̩̺ͅ ͉ r f͈̱͓͓ ̦̰̬̗ ̗̝̼ m̞̗͎͍̾̈͊ o͈̩ͪ̈ ̟̤̻͉ ̃ͨͬ́̉ͩ̓ ̰̝e͍͎h '̼̬̤̋̉̽lͪ ͣ ̗̼ͥ̉R ̱͚̏ ̫͙͓̰͔ ̃̂̂ ͕͓̲ y .̚ ̣̫ ̞͓͈ ̼̪̠s͔̹̞̟t͈̘͕ H͎̯ ̙̱a͍̟͍ͅ ̘̼ u͙͓̙̟r̥̹̫͇͎ ͚ ̻ͅ ̣ ̙̹ ͍̮ t ̻̳̮ h ̩̜̣_ e̠_̱̣͔ ̼ ̺͉U s͈̰̣̥ p̻ a͕̗̣̺ k ͇e̤͍̯ ̻̹͓̬ ̹̤̳a̠͍̪ b̩_ ̪_̥͖͎͍ e̪̻̣̣ ̙̼ ͈̬s͇̮̞ ḻ͇ ̥͖̠ ̹̩̖ ̙̲ a̺͈̹̤͈͉ͅ ̣̮͕̙ ̗ h ̠̟ ̰̜̜l̬̹̭ l̺̞ ̩̳̮̩̰͕ͅ ̻r̮̥̦͍͍ ͈̫ e̳̠̙̘ ̱ u̠͇r̮̣͓ ̘̬̰ t͔͚̳̹̰ ̰͖ n̗͍ ̥͕ ̥͉f̜͚r̯͍ͅ o͈̯̦ ̖ ̳͓̦͔ ¯ͭ̔ ̻͙̫̪̪̖͈ ͔̬ ̣̌ ̠̟̱̒ ͍m̻̟ ̭ ̼̠ ͍̣t͖h͔͉̞ e̬̫̦ ̋͂ ̖͇̼ ̊ͤ̓̋̄̐͌̾ ̩̝̮ ͓ d̰̼̞̤͕ ̤̘̣̭͍̖ ̻͈ ̟̭ ͡l̴l h͞ow l͢ f̕o ́̃̍̆̂̇̒ͫ k̲͖̻̻̆͋ͬ̑ ̩͍̭̙ ̥ k̙̣͕͔ ̘̮̤̻̜̳ r̯̰̱̬̭ͅ ḁ͙ ͔͔̺ ̠ ̗ s̻̱͎ ̙̦̝̗͍͎ ̞̪t̫͉̟̻ ͖ ạ̫ r͔̺ ͍.͔̖͚̺̹ ̰̫ Ǹ`ya͠rĺath̢ote͡p̢ ͟s̀`h ̨ a ̷reve̢r̀` ͑̎ ň̽̌ ḯ΅ ̣ ̃̄̇ͪ̂͑ ͉͔̙̤̪̜ h ̉ͦ ͕t͂̔ ͨ͋̅̿ ͔̆ͫ̓ͫ ̫͖̻ e ͫ͌͛ ͦ͆ͭ̽ ̊ͩͩ̇ͣ ̗ͅd̂ ā̇ͤ͋ͭͨ ̗̰ ͙̗̝͕̩̥ ̟͍ ř nͮͯ̑̿͒ e̍͒̅̄ͣ̀ ͅͅ ̪̠̗͕̥ ͋̋ ͙̹͎̺̠ s̊̈̽̊̌ s̈̌ͪ ̱̳ .̄̑̎ ͔̙̣ ̤̰̟̦̥ ͉̉ ͙̬
- 38. curl | bash often assumes • There is no air-gap • Every request is a safe & sane request • That HTTPS is good enough
- 39. curl | bash often forgets • >100% Broadband coverage • Mirrors exist • HTTPS secures transport, not content
- 40. curl | bash totally ignores • The benefits of reusability • The fragility of shell scripts • The fragility of shells
- 41. Security is hard • RVM recently introduced hand-rolled GPG signing* • Thread had 48 comments within a week, almost universally about the implementation • Broke semver, automation, and hearts * https://github.com/wayneeseguin/rvm/issues/3105
- 42. Omnibus
- 43. Isn't that from Chef? • Sure, but so is Test Kitchen • Builds packages while still controlling the entire dependency stack • Lots of love from users with complicated dependency stacks
- 44. Omnibus is one way to skin the entire cat • Abstracts (instead of removes) dependency management • Only builds packages for the platform it's installed on • You're going to want to know Ruby
- 45. FPM
- 46. Effing Package Managers •General purpose swiss-army knife of package building •Works around a lot of the shortcomings of existing package managers •Jordan Sissel is a SAINT (Shout out to #hugops!)
- 47. "Common packaging patterns, a distaste for existing packaging practices, and some hate-driven development yielded FPM! Add some amazing contributions in code, bugs, features, and support from the community and boom we have modern FPM." Jordan Sissel My inbox, Oct 10 2014
- 48. Effing FPM • Swiss army knives are rarely the best tool for a given job • General purpose in this case means a lot (~150ish) of command line flags • Still infinitely better than curl | bash
- 49. Why so many alternatives? What went wrong?
- 50. RPM Packaging can be tough • RPM Spec files are weird • Kind-of M4, kind of Shell, all obtuse • Oh, and kind-of Make; only kind-of • Sort-of competing RPM standards
- 51. Deb Packaging can feels like penance • "debian/" directories are outright hostile to man & beast alike • Debian "Helpers" usually don't • dpatch can use unified diffs (sane) or shell scripts (what?!)
- 52. Conflation of purpose • Some library managers try to install executables, e.g. gem, pip, npm • Remember when I said "delusions of grandeur"? (Google Image Search was kind of useless here)
- 53. But really, I just have a hypothesis! • Developers love solving new problems • Sometimes they confuse their problems for the customer's problems • Maybe packaging isn't a solved problem yet, but it's close
- 54. Where do we go from here?
- 55. Sometimes the only choices you have are bad ones; but you still have to choose.
- 56. TL;DR: this problem is (mostly) solved Stop writing new installers from scratch Give your customers the best packages possible Don't forget Pareto (any number of 80/20 rules)
- 57. Thank you You're wonderful. Thank you for letting me rant at you for as long as you did. mckern@puppetlabs.com @the_mckern
- 58. Questions?