IPS is a transactional, metadata-driven packaging system used by several Solaris variants to manage software installation and updates. Key concepts include FMRIs to uniquely identify packages, package manifests that describe file actions and dependencies, publishers that provide packages, repositories to store and serve packages, the image where packages are installed, and boot environments that provide bootable snapshots of images. Common IPS commands are used to install, update, list, search, and manage packages.
2. What Is IPS?
Image Packaging System, aka "pkg(5)"
Created by Sun for OpenSolaris
Now used by OmniOS, OpenIndiana, Oracle Solaris 11
Transactional, metadata-driven and integrated with ZFS
Network-based, extensive search grammar
Changes-only updates
3. Motivations
Unify packaging and OS patching
Be smf(5)- and ZFS-aware
Verify correct installation
Optimize for the update case
Ease developer burden
Add dependency-based network retrieval
4. IPS: The Good
Every package 100% described by metadata
Updating requires fetching only changed assets
Get a new BE automatically, when needed
Automatic fetching of dependencies
5. IPS: The Not-So-Good
No single-file on-disk format (except archives)
Latency-sensitive
No pre- or post-install scripting*
* This is actually a good thing! Tasks usually scripted are now first-class actions
6. A Few IPS Commands
pkg(1) :: installation and information client
pkgsend(1) :: publication client
pkgrecv(1) :: raw contents transfer utility
pkg.depotd(1M) :: repository server
pkgsign(1) :: cryptographic signing utility
7. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
9. FMRIs in IPS
Publisher name is optional:
pkg://omnios/web/curl Must be preceded by '//' if present
Scheme is also optional:
/web/curl Leading '/' anchors to any publisher root
pkg:/web/curl Note the use of only one '/' after the scheme
web/curl Anything ending in '/web/curl'
curl Anything named 'curl' or ending in '/curl'
//omnios/web/curl Publisher included
10. FMRIs: Version
Strictly numeric comparison, split on punctuation
Comparison is left to right
7.31.0,5.11-0.151006:20130703T175442Z
7.31.0 Component Version ("the software's version")
5.11 Build Version (OS version, aka `uname -r`)
0.151006 Branch Version (distro-specific meaning)
20130703T175442Z Timestamp (ISO 8601)
11. FMRIs: Version
Versions may be included when specifying names:
curl@7.31
curl@*-0.151006
curl@7 Anything 7.x
7.31.x
Any version for branch 0.151006
13. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
14. Package Manifest
Describes a specific version of a package
Collection of actions that deliver files, dirs, links,
dependencies, etc. via attributes
Attributes are key-value pairs
Viewable with `pkg contents -m <name>`
15. Package Manifest
set name=pkg.fmri value=pkg://omnios/web/curl@7.31.0,5.11-0.151006:20130703T175442Z
set name=pkg.summary value="curl - command line tool for transferring data with URL syntax"
set name=pkg.descr value="curl - command line tool for transferring data with URL syntax"
set name=publisher value=sa@omniti.com
dir group=bin mode=0755 owner=root path=usr/bin/amd64
file 3a8938b01cf732fc0b4838218d94508fca75e54c
chash=d923dfc752598ed149a64c873065fc71cbbf83fb
elfarch=i386 elfbits=64 elfhash=aabff399422fb0e74df8ffb4356d7bee97db89a5
group=bin mode=0755 owner=root
path=usr/bin/amd64/curl
pkg.csize=100864 pkg.size=174672
...
link path=usr/lib/amd64/libcurl.so target=libcurl.so.4.3.0
...
depend fmri=library/security/openssl@1.0.1 type=require
depend fmri=library/zlib type=require
depend fmri=web/ca-bundle type=require
16. Manifests: Dependencies
Require :: the referenced package provides essential
functionality; including a version sets a "floor"
Optional :: non-essential, but if installed, must meet
version constraint, if any (same as require)
Exclude :: conflicts; may not be installed with this
package (these are evil, avoid them)
Incorporate :: like optional, but sets "ceiling" as well as
"floor" to the given degree of precision
19. Manifests: Dependencies
Packages containing only incorporate
dependencies are called "incorporations"
!
Used to ensure a compatible set
of installed software
!
Used carefully, they can be very handy:
!
omniti/incorporation/perl-516-incorporation
20. Manifests: Dependencies
$ pkg contents -mr perl-516-incorporation
set name=pkg.fmri
value=pkg://perl.omniti.com/omniti/incorporation/perl-516-incorporation@5.16...
set name=pkg.summary value="Constrains omniti/runtime/perl to version 5.16.x"
set name=pkg.descr value="Constrains omniti/runtime/perl to version 5.16.x"
set name=pkg.human-version value=5.16
set name=publisher value=sa@omniti.com
depend fmri=omniti/runtime/perl@5.16 type=incorporate
Version of omniti/runtime/perl must be 5.16.x
Module dist pkgs have their own versions, but
require the incorporation matching the perl they were
built with
21. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
22. Publisher
An entity that provides packages
Named for products ("omnios")
or domain style ("ms.omniti.com")
One publisher can have multiple URLs
23. Publisher
$ pkg publisher
PUBLISHER TYPE STATUS URI
omnios origin online http://pkg.omniti.com/omnios/release/
ms.omniti.com origin online http://pkg.omniti.com/omniti-ms/
perl.omniti.com origin online http://pkg.omniti.com/omniti-perl/
Publishers are searched in the listed order
List publishers
25. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
26. Repository
Location to which packages are published
Can be used locally (file://)
or remotely (http://) via pkg.depotd(1M)
Created and managed by pkgrepo(1)
27. Repository
# pkgrepo get -s /repo/omniti-ms/
SECTION PROPERTY VALUE
publisher prefix ms.omniti.com
repository version 4
!
# pkgrepo info -s /repo/omniti-ms/
PUBLISHER PACKAGES STATUS UPDATED
ms.omniti.com 602 online 2014-03-23T20:50:49.146202Z
Get repo information
28. Repository
# pkgrepo get -s /repo/omniti-ms/publisher/ms.omniti.com/
SECTION PROPERTY VALUE
feed description ""
feed icon web/_themes/pkg-block-icon.png
feed id ""
feed logo web/_themes/pkg-block-logo.png
feed name package repository feed
feed window 24
publisher alias ""
publisher prefix ""
repository collection_type core
repository description ""
repository detailed_url ""
repository legal_uris ()
repository maintainer ""
repository maintainer_url ""
repository mirrors ()
repository name package repository
repository origins ()
repository refresh_seconds 14400
repository registration_uri ""
repository related_uris ()
repository version 3
Get per-publisher information
32. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
33. Image
Location where packages can be installed
May be rooted at arbitrary point in the filesystem tree
Default image rooted at '/'
Has properties that govern policy; see pkg(1)
35. IPS Concepts
FMRI :: Fault Managed Resource Identifier
Manifest :: describes a specific version of a package
Publisher :: entity that provides one or more packages
Repository :: location for publishing and retrieving pkgs
Image :: location where packages may be installed
Boot Environment :: (BE) bootable instance of an image
36. Boot Environment
Bootable instance of an image
Integrated with ZFS
Can be auto-created according to image policy
Can be manually created
Created and managed by beadm(1M)
40. Use Cases: Install
# dry run, verbose
pkg install -nv foo
!
# latest
pkg install foo
!
# latest available 2.x
pkg install foo@2
!
# exact version
pkg install foo@2.1.2
When "foo"
is not installed
41. Use Cases: Update
# dry run, verbose
pkg update -nv foo
!
# latest available
pkg update foo
!
# stay within 2.x line
pkg update foo@2
!
# downgrade
pkg update foo@1.9
Assuming
"foo 2.1"
is installed
42. Use Cases: List/Info
# all installed packages
pkg list
!
# list installed packages matching "foo"
pkg list foo
!
# list all known versions of foo, installed or not
pkg list -fav foo
!
# detailed information
pkg info foo
!
# same, but remote
pkg info -r foo
43. Use Cases: Inventory
# file/directory paths only
pkg contents foo
!
# raw manifest
pkg contents -m foo
!
# same, but remote
pkg contents -mr foo
!
# list deps
pkg contents -t depend -o fmri
44. Use Cases: Search
Powerful due to package metadata
Local or remote
Expressive grammar
Results sometimes non-obvious
45. Use Cases: Search
pkg_name : action_type : key : token
pkg_name :: the value of pkg.fmri
action_type :: file, dir, link, depend, set, etc.
key :: attribute name within the selected action
token :: attribute value, i.e., "what you're searching for"
46. Use Cases: Search
pkg_name : action_type : key : token
Blank fields implicitly wild-carded
Simple globbing permitted for pkg_name, token
Leading colons optional
`pkg search tmux` is effectively: `pkg search ':::tmux'`
To have success, understand what you're looking for
47. Use Cases: Search
# 'tmux' as any value
pkg search tmux
INDEX ACTION VALUE PACKAGE
basename file usr/bin/tmux pkg:/terminal/tmux@1.6-0.151004
basename file usr/bin/tmux pkg:/terminal/tmux@1.6-0.151002
basename file usr/bin/tmux pkg:/terminal/tmux@1.7-0.151006
pkg.fmri set omnios/terminal/tmux pkg:/terminal/tmux@1.6-0.151004
pkg.fmri set omnios/terminal/tmux pkg:/terminal/tmux@1.6-0.151002
pkg.fmri set omnios/terminal/tmux pkg:/terminal/tmux@1.7-0.151006
48. Use Cases: Search
# same as before, but show only pkg name
pkg search -p tmux
PACKAGE PUBLISHER
pkg:/terminal/tmux@1.6-0.151002 omnios
pkg:/terminal/tmux@1.6-0.151004 omnios
pkg:/terminal/tmux@1.7-0.151006 omnios
49. Use Cases: Search
$ pkg search 'dir::pgsql*'
INDEX ACTION VALUE PACKAGE
...
basename dir opt/pgsql925 pkg:/omniti/database/postgresql-925/ltree@9.2.5-0.151006
...
results from this manifest entry:
dir group=bin mode=0755 owner=root path=opt/pgsql925
This answer:
51. Use Cases: Search
$ pkg search -H -o pkg.name 'depend::web/curl'
developer/versioning/git
developer/versioning/mercurial
entire
incorporation/jeos/omnios-userland
Reverse dependencies
$ pkg search -o pkg.fmri,fmri '*-0.151006:depend:incorporate:web/curl'
PKG.FMRI FMRI
pkg:/incorporation/jeos/omnios-userland@11,5.11-0.151006:20130506T214442Z web/curl@7,5.11-0.151006
pkg:/incorporation/jeos/omnios-userland@11,5.11-0.151006:20130716T202721Z web/curl@7,5.11-0.151006
pkg:/incorporation/jeos/omnios-userland@11,5.11-0.151006:20131030T205312Z web/curl@7,5.11-0.151006
What r151006 packages incorporate on curl,
and at what version?
52. Use Cases: Audit
# check installed state of all pkgs
pkg verify
!
# check state of a single package
pkg verify <pkg>
!
# repair installed state of a package
pkg fix <pkg>
53. Use Cases: Audit
# pkg verify -v curl
PACKAGE STATUS
pkg://omnios/web/curl OK
!
# rm /usr/share/man/man3/libcurl.3
!
# pkg verify -v curl
PACKAGE STATUS
pkg://omnios/web/curl ERROR
file: usr/share/man/man3/libcurl.3
Missing: regular file does not exist
54. Use Cases: Audit
# pkg fix curl
Verifying: pkg://omnios/web/curl ERROR
file: usr/share/man/man3/libcurl.3
Missing: regular file does not exist
Created ZFS snapshot: 2013-10-16-02:07:42
Repairing: pkg://omnios/web/curl
!
DOWNLOAD PKGS FILES XFER (MB)
Completed 1/1 1/1 0.0/0.0
!
PHASE ACTIONS
Update Phase 1/1
!
PHASE ITEMS
Image State Update Phase 2/2
55. Creating IPS Packages
Build software however you wish
Place build product in a proto area
Create manifest
Publish to a repo
IPS does not impose a build framework (think
rpmbuild, debuild)
56. Creating IPS Packages
1. `pkgsend generate /path/to/proto > /tmp/manifest.p5m`
2. Add FMRI, any other 'set' actions to manifest.p5m
3. `pkgsend publish -s <repo_url> -d /path/to/proto
/tmp/manifest.p5m`
pkgsend(1) creates manifests and publishes packages
57. Creating IPS Packages
Adding the 'set' stuff is tedious
May want to make other changes/additions to manifest
This needs to be automated!
Use pkgmogrify(1)
58. Creating IPS Packages
pkgmogrify(1)
Programmatic transformations of manifest contents
Macro replacements
Include other manifests or manifest fragments
Transformation of actions
By convention, we store these directives in a .mog file
beside our build scripts
60. Creating IPS Packages
<transform dir path=opt/riak/data.* -> set owner riak>
!
<transform dir path=opt/riak/data.* -> set group riak>
!
<transform file path=opt/riak/etc/.*.args -> set mode 0644>
!
<transform file path=opt/apache22/libexec/amd64/libphp5.so ->
edit path libphp5.so libphp5.53.so>
!
<transform file path=opt/elasticsearch/config/elasticsearch.yml ->
set preserve true>
!
<transform file path=opt/omni/lib/ruby/gems/1.9/cache.* -> drop>
!
<transform file path=(var|lib)/svc/manifest/.*.xml ->
add restart_fmri svc:/system/manifest-import:default>
pkgmogrify: Transform actions
61. Creating IPS Packages
Tangent: renaming
pkg:/network/iftop
pkg:/omniti/network/iftop
Forgot to follow naming convention
Also useful if upstream name changes
Users may have installed it, can't just abandon it
62. Tangent: renaming
Solution: publish a "rename package"
Transitional package that allows update to new name
Creating IPS Packages
set name=pkg.fmri
value=pkg://ms.omniti.com/network/iftop@1.0.2,5.11-0.151006:20130816T191418Z
set name=pkg.renamed value=true
set name=variant.opensolaris.zone value=global value=nonglobal
depend fmri=pkg://ms.omniti.com/omniti/network/iftop type=require
63. Creating IPS Packages
# pkgrepo create /data/myrepo
# pkgrepo set -s /data/myrepo publisher/prefix=myrepo.example.com
Create a repo with pkgrepo(1)
May now use file:///data/myrepo to publish packages
publisher/prefix sets the default publisher name
64. Creating IPS Packages
$ pkgrecv -s http://pkg.omniti.com/omnios/release/ -d web_curl.p5a -a web/curl
Retrieving packages for publisher omnios ...
Retrieving and evaluating 1 package(s)...
DOWNLOAD PKGS FILES XFER (MB)
Completed 1/1 88/88 1.3/1.3
!
!
ARCHIVE FILES STORE (MB)
web_curl.p5a 158/158 1.5/1.5
!
$ scp web_curl.p5a me@my-other-box:
Create an archive with pkgrecv(1)
# pkg install -g web_curl.p5a web/curl
65. Signing IPS Packages
pkgsign(1) updates the manifest in place on the repo
Adds the 'signature' action
Validates the manifest, which in turn validates its
content
Signed package retains original timestamp
66. Signing IPS Packages
signature <hash of certificate>
algorithm=<signature algorithm>
value=<signature value>
chain="<hashes of certs needed to validate primary certificate>"
version=<pkg version of signature>
Payload & chain :: hashes of certs downloadable from originating repo
Value :: signed hash of manifest's message text
Algorithm :: hash algorithm used, default is rsa-sha256
Version :: pkg(5) version of the signature action
67. Signing IPS Packages
First, publish the unsigned package(s); then:
# pkgsign
-c /path/to/signing.crt
-k /path/to/signing.key
-s <repo_url>
<fmri_list>
Multiple signatures (even from different entities) will not
interfere with each other
Allows different entities to indicate acceptance during
publication process (e.g., dev/QA/release)
68. Questions?
Further reading
Man pages: pkg(5), pkg(1), pkgsend(1), pkgrecv(1),
pkgmogrify(1), pkgrepo(1)
http://omnios.omniti.com/wiki.php/
GeneralAdministration#PackageManagement
http://omnios.omniti.com/media/ipsdevguide.pdf
http://web.archive.org/web/20100105071515/http://
blogs.sun.com/sch/entry/pkg_1_a_no_scripting