This week I started trying to install OmniOS in a Vultr instance. I'm not sure where I first saw Vultr listed but was drawn to it because they offer custom ISO installs. OmniOS isn't supported by most hosting vendors so I would need to install via a custom ISO.
Setting up an account was quick on Vultr, including $5 free credit for opening an account. When creating a new instance you can select the custom ISO after you've added it via URL to your account. They will transfer the ISO to the right datacenter, attach it, then boot up the instance.
The ISO booted fine but installing OmniOS onto the instance didn't work. It turns out that the OmniOS installer doesn't like the way Vultr exposes disks as block devices to the instance. This was mentioned by Dan McDonald in the #omnios channel after he helped me debug. Originally I tweeted about trying to install it and he followed up. He was very helpful and mentioned that the installer is due to be replaced which will work around this issue, but it won't be right away.
It seems just running OmniOS on baremetal is the way to go. I might wind up getting a colo'd box at this point.
I recently came across a static site generator written in Erlang called LambdaPad. I looked around a bit while trying to find a static site generator that would work with Contentful that I would enjoy working with. Most static site generators expect to source documents from the filesystem but LambdaPad allows any source of data you can write in Erlang!
Contentful is a CMS with an API and is free for small use cases. It is easier to use their API as a source then to have other people edit a Git repository in my expected case.
My Github has a branch that can source Contentful entries and provide them to templates. After adding some documentation, examples, and handling Contentful pagination it should be ready for a PR.
... another example of me spending more time on infrastructure instead of a user-facing project which began this tangent!.
After uploading a package to an OmniOS package repository I was unable to find
the package by searching. The package could be installed and local searching
would find it, but the depotd server didn't know how to find the package when
searching. Restarting pkg/server would work around the issue but having to do
that after each publish would get annoying.
There is a command pkgrepo that will refresh the search index remotely!
Running
pkgrepo refresh -s PKGSRVR
is enough to reload the search index.
A previous post showed how to install files. If you wanted to run a service from that package there are a few more steps.
The Service Management Facility provides a way to manage services in OmniOS. If you are running a service you installed from a package, this is the way to do it.
We will need to complete a few steps to package up a service and deploy it with IPS.
Create an SMF manifest that instructs SMF how to run our service
Deploy the SMF manifest
Start the service.
Optionally, the service can be modified to read SMF properties so that it can
be configured through svccfg
A service manifest is an XML documents that contain the information required to run a command as a service. This would normally mean that you have to create a new XML document for each service. Thankfully there is the tool Manifold that can create an manifest with answers to the relevant questions.
Packaging for OmniOS goes over how to create a package using the same build system as is used for building OmniOS. The layout of this repository seems designed for building already written software to be used in OmniOS. If you need to package your own software then this can be more overhead then you are looking for. The tools used by that GitHub repository are included in the default installation of OmniOS and have plenty of documentation on Oracle's site about how to use IPS. It turns out you can start making packages for OmniOS with only a few commands.
This post will cover the tools required to create a package, not necessarily best practices in packaging for OmniOS.
I've created an example repository that can build and upload a package to an IPS package depot if you want to skip ahead.
The packaging commands we will be using are
pkgsend - Generates the package manifest and publishes the package
pkgmogrify - Transforms the package manifest
pkglint - Linter for package manifests
pkgfmt - Formatter for package manifest
pkgrepo - (optional) Refresh the repository search index after upload
We will be packaging a Hello World script stored in hello-world.sh.
#!/usr/bin/bash
echo Hello World!
This file needs an execute bit as well so we will run
chmod +x hello-world.sh
pkgsend will generate a manifest for us if we can build a directory that
mimics the deployed layout. If we put our script in build/usr/bin (and remove
the extension) then run pkgsend generate build we will get a manifest of
files and directories to package.
$ /usr/bin/pkgsend generate build
dir group=bin mode=0755 owner=root path=usr
dir group=bin mode=0755 owner=root path=usr/bin
file usr/bin/hello-world group=bin mode=0755 owner=root path=usr/bin/hello-world
Our manifest so far says we need two directories and a file. This would be
enough of a manifest to start with but can be problematic if the directories
don't line up with the host used to install the package. It would be better to
remove the directories and assume that /usr/bin already exists on the system,
since it really should already be there.
The command pkgmogrify can take a manifest and a transform file and output a
transformed manifest.
A simple transform to do this will be stored in transform.mog
<transform dir path=usr -> drop>
This will drop any directories that include the path usr. If you need are
building a more complex directory structure then using something like
usr/bin$ as the path will only drop the common /usr/bin elements from the
manifest.
For this we will write the manifest to a file the mogrify it to remove the directories.
$ /usr/bin/pkgsend generate build > manifest.pm5.1
$ /usr/bin/pkgmogrify manifest.pm5.1 transform.mog
file usr/bin/hello-world group=bin mode=0755 owner=root path=usr/bin/hello-world
This now has just our script in the manifest. Using pkgmogrify we can easily
script changes to manifests instead of relying on manual changes to clean up a
generated manifest.
We'll write the updated manifest to a new file
$ /usr/bin/pkgmogrify manifest.pm5.1 transform.mog > manifest.pm5.2
We have the manifest for what the package should contain but we still need to describe the package with metadata. We will need to include at least a name, version, description, and summary for the package.
The name and version are contained in an Fault Managed Resource Identifier or FMRI.
I recommend reading the link above about proper format and conventions for
FMRIs but for now we will write metadata.mog to contain
set name=pkg.fmri value=example/hello-world@0.1.0,0.1.0-0.1.0:20160915T211427Z
set name=pkg.description value="Hello World"
set name=pkg.summary value="Hello World shell script"
We can use pkgmogrify to combine our metadata and current manifest file to
make a file manifest used for publishing our package. In this case we use
pkgfmt to format the file as well.
$ /usr/bin/pkgmogrify metadata.mog manifest.pm5.2 | pkgfmt > manifest.pm5.final
The manifest we have now should work for publishing the package. We can verify
using pkglint on the final manifest to check.
$ /usr/bin/pkglint manifest.pm5.final
Lint engine setup...
Starting lint run...
$ echo $?
0
No errors or warnings, wonderful!
We now have a directory structure for the package we would like to create as well as a manifest saying how to install the files. We can publish these components to an IPS package depot with pkgsend
$ pkgsend publish -s PKGSERVER -d build/ manifest.pm5.final
pkg://myrepo.example.com/example/hello-world@0.1.0,0.1.0-0.1.0:20160916T182806Z
PUBLISHED
-s specifies the package server, -d specifies the directory to read, and we pass along the path to our manifest. Our package was then published!
If you are using an HTTP depotd server to publish and see the error pkgsend: Publisher 'default' has no repositories that support the 'open/0' you will
need to disable read-only mode
for the server or publish to a filesystem repository.
The HTTP depotd interface doesn't refresh the search index when a package is published. This can be done with the pkgrepo command.
$ pkgrepo refresh -s PKGSERVER
As part of the process of shaving some yaks today I wound up needing to upgrade my development server to the latest version of OmniOS. I originally installed the LTS version and planned to stay there till the next release. It turns out there isn't much reason not to upgrade to the latest version. You will get needed security updates either way but be able to get around any bugs with OS-related things that have been fixed in the mean time.
The Upgrading to r151014 or
later page had the
needed information and worked quickly. I ran into an issue with the datasets
for my zones causing the problem pkg: Unable to clone the current boot environment when trying to update with pkg. All the zones I care about
are recreated with configuration management so I didn't have a problem
destroying the dataset and recreating them. If it were production I would
have at least snapshotted the needed datasets before destroying them.
For the next release I think I'll update a bit sooner!
When publishing to an IPS depotd server you may see the line
pkgsend: Publisher 'default' has no repositories that support the 'open/0' operation.
If the depotd server will show you a web page but publishing does not work with
pkgsend you may have the server setup in read only mode. svccfg will allow you to change the property with
svccfg -s pkg/server setprop pkg/readonly = false
Don't do this to a server on the internet though, placing an HTTP server in front of depotd will allow you to add authentication. This is otherwise insecure!
While updating Ansible I ran into an issue with an extras module for ZFS and Solaris. A playbook that used to work to set a mount point no longer worked. I was seeing errors that ended in
if int(version) >= 34:\r\nValueError: invalid literal for int() with base 10: '-'\r\n", "msg": "MODULE FAILURE"
An issue was
filed in June
and fixed last month. This change isn't in the latest Ansible 2.1.1.0 which I
was using. For the time being I've added the
extras repository devel
branch as a submodule and used ANSIBLE_LIBRARY=... to get a fixed version.
I've started trying to package some software for OmniOS for personal use. The OmniOS Packaging page in the wiki goes through how to do it using the tools used to build the OS. This is a bit more than I would want to do when publishing software to GitHub. I would rather not rely on a repository used to build the OS just to package one piece of software.
A few months ago I was trying to package a personal project and got most of the way there! So far there is a make target that will package an Erlang release into an IPS package. I think it only got as far as putting the files on disk. I still to add the SMF manifest and fix permissions, but it's much smaller when used to package a single piece of software.