nosh package is a suite of system-level utilities for initializing and running a BSD or Linux system, for managing daemons, for managing terminals, and for managing logging.
It is originally intended for use on BSDs, and to fill the gaps where BSD users don't have the option of
upstart, and want more than what other daemontools family systems provide.
The BSD user sees:
BSD licensing: It's licensed under the MIT Expat Licence, the ISC Licence, and the FreeBSD Licence (in the belief that they are all in essence identical).
no more waiting for
The service management brings things up a lot more quickly.
skills transferrability from the systemd world:
There are whole suites of shim layers to make things familiar for someone coming to BSD from a systemd or upstart Linux background.
Want to run
systemctl start wibble or
initctl status wibble?
You still can, and the command will do the equivalent nosh service management thing.
skills portability from the OpenBSD world:
Want to run
rcctl enable wibble?
You still can, and the command will do the equivalent nosh service management thing.
properly designed dæmon management:
service command does not have the pitfalls of the BSD
It is part of the daemontools family design that dæmons are always started from a known parent process with a known initial execution state, a clean environment that is fully adjustable on a per-dæmon basis, and a known set of open file descriptors.
transparency and interoperability:
Service configuration is expressed with ordinary files, directories, and symbolic links in the filesystem; so is service status.
nosh service bundles and their configurations can be inspected and modified with the likes of the
ln commands, with no need for a running Desktop Bus, and even without the management programs running at all.
Status and control information for running dæmons is in a well-known format that has been stable since the 1990s and that not only can be manipulated with other management toolsets but can also be manipulated from ad hoc tools incorporating the likes of Peter Ruibal's and Andrés J. Díaz's supervise library for Python.
easy ports of Linux packages:
Have a package that comes with a service unit?
It can be run through
convert-systemd-units to produce a native nosh service bundle that will run on a BSD.
Have a package that comes with a service unit and a socket unit?
Such a pair can be converted, too.
notions from UNIXes: It's a fully fledged service and system management package. Like the AIX SRC, system management is separated from service management. Like the Solaris SMF, there are targets/milestones. Terminal login is arranged as proper, fully fledged, services that are integrated with the service management and not bolted on to the side of it.
console-free operation: The system might be a smart telephone, or a server in a datacentre, or even a watch. It's not necessarily convenient to operate it using the system console. It might not even have a physical serial port socket. So kernel (and other) logs go to log directories; the terminal management system provides user-space virtual terminals that can be accessed from an SSH session in a pinch; and mechanisms are not in general designed to demand the use of the console.
This doesn't just hand over the raw toolset and leave the rest to you.
This package runs real BSD machines right now, and has been for some time.
One of the early goals of the package was to write service bundles that did what the 157 scripts in FreeBSD's
See the roadmap for the detailed status of this; but there's enough already done for running working desktop and low-end server machines.
no more waiting for Godot:
launchd on BSD train is never coming and it's long past time to realize that.
It's now 2015, and one can do proper service and system management on BSD right now, with tools that have been succesfully running FreeBSD systems for a couple of years at this point.
And you don't have to learn XML, let alone put it into process #1.
However, it can also be used by Linux users, Debian Hurd users, and others. Conversely, the Linux user sees:
It's not systemd:
This is a point in its own right, for some people.
A rather more sensible point of view is that it's part of maintaining heterogeneity in Linux.
systemd unit files do not lock one in to systemd forevermore, because one can run them through
convert-systemd-units and get nosh service bundles.
And nosh is part of a family of toolsets where mix-and-match composability is a design feature.
But one doesn't have to un-learn all systemd administration habits:
Like the BSD user, one can still go on typing
systemctl start wibble if one wants.
service wibble start.
There is a bunch of System 5 command shims, too.
But it is in the daemontools family:
The fundamental concepts and architecture are well-known; other people's toolsets will interoperate, mix-and-match fashion; and there are widespread third party resources, from tutorials to
run program collections, that can be adapted to it with ease.
Know about running daemontools in a Docker container? The principles are much the same. (Indeed it is simpler. A service bundle for SSH is one of the several hundred service bundles pre-supplied, so you don't have to make one.)
Know about running Ruby under runit?
The ideas are all the same, and the change to the
run script is a simple substitution of
chpst -e, although that's by no means mandatory.
Know about running node under runit or running node under daemontools? The same principles apply; in the latter case the same command names even apply, although one can do the logging in a better way (see the Nosh Guide pages on logging).
Know about running Tomcat under daemontools?
Again, it's just a simple change from
multilog t to
cyclog (since this is the common use case where none of the advanced features of
multilog are used).
Of course, it's not even that if one has
multilog present, taken out of daemontools or daemontools encore; since one can mix and match.
separation of policy and mechanism:
service-manager is "mechanism", providing the raw mechanics of supervising dæmon processes.
svscan) are "policy", determining when, where, and why services are started and stopped.
separation of service management and system management:
The overall state of the system — emergency mode, rescue mode, "multi-user" mode, halted, and so forth — is the domain of the
The states of individual services — stopped, running, starting, stopping, failed — are the domain of the
avoidance of continually regenerated ephemera: Things such as mount@, fsck@, and ttylogin@ services are not stored in non-persistent storage and regenerated on the fly over and over again. Taking a leaf out of the BSD books, they are stored persistently and re-generated when the source information actually changes. See the roadmap for what's in store when it comes to automating this.
properly designed dæmon management: As for BSD users (q.v.).
console-free operation: As for BSD users (q.v.).
transparency and interoperability: As for BSD users (q.v.).
no kernel upgrade treadmill: There are no requirements that one be running the absolute latest kernel and keep on upgrading. nosh doesn't require Desktop Bus in the kernel.
One of the stumbling blocks for daemontools adoption has historically been rounding up
Several people have published collections of
run scripts, such as Gerrit Pape's famous collection of just under 80.
There are (as of version 1.14) some 230 pre-supplied service bundle directories in the
nosh-bundles package, ranging from GNU cron to mongodb, from OSSEC HIDS to RabbitMQ server, from exim4 to dnscache, from bcron to OpenStack.
no central logging bottleneck: The daemontools family way is for individual services to have entirely separate log streams processed by entirely separate log services, which have no access to one another's files and directories and have no need of being the superuser at any point. A service that logs a vitally important line once per week won't have its output of the last few weeks entirely flooded out of the log by a prolific service generating thousands of lines of low importance stuff per minute.
no mandatory logging tool:
There's a whole range of ways to log just to local log files alone, from
dumblog (supplied with freedt) through
cyclog (supplied with nosh) to
s6-log (supplied with s6).
Other people have written further tools if one really does want to log over TCP or UDP to a remote server.
The Nosh Guide has pointers in its "Logging" section.
The features break down into dæmon management, system management, provision of socket-based services, pseudo-terminal and virtual terminal management, and log management.
Its fundamental dæmon management mechanism is that of the daemontools family, which is explained at that hyperlink for the unfamiliar. It comprises several of the augmentations from that family, and further augmentations that are new and unique to nosh. Augmenting that mechanism, or built on top of it, are:
targets/milestones: The extensible system target mechanism is a set of otherwise ordinary service bundles, that "want" or "conflict" with whole sets of other services.
configurable service restart:
To the main
run script are added
restart controls service restart: whether and under what circumstances it happens.
It can also be used for logging/notifying a system operator of service failures, restoring a sane service state after an abort, and even simply inserting configurable delays so that continually failing services don't restart too frequently.
service interdependencies and startup/shutdown orderings:
supervise directories form two subdirectories of what is known as a service bundle, that also contains service interdependency, ordering, and installation information in other subdirectories.
more chain-loading tools:
There are some extra chain-loading tools for manipulating process state, including such things as
a choice of alternative service management policies:
/service directory can be eliminated in favour of a decentralized system that allows one to (for example) place "boot" service bundles in
/etc/sv, "system" service bundles in
/var/sv, and "add-on" service bundles anywhere that one likes.
logging as "first class" services: "Log services" are now merely one case of a general-purpose mechanism. Although one can still employ a one-log-service-per-main-service mechanism, one can also alternatively arrange fan-in of multiple services to a single logging service, and pipelines of three or more services.
Several notions are retained, including:
the filesystem as the database:
Everything involved in services is done with files and directories in the filesystem.
Service interdependences, for example, are expressed with ordinary symbolic links.
systemd, nosh doesn't attempt to be "clever" with the contents of symbolic links, either.
They are used with the conventional filesystem semantics.)
the filesystem as the control/status API:
The service control/status API is the familiar
limited and controlled parsing: Parsing the same things over and over, from text to machine-readable forms, is avoided; parsing is done in its proper place, at configuration time not at run time; and parsers are avoided in security-sensitive areas. There are no parsers in process #1, for example.
/service mechanism can still be used.
(The monitoring process does not use polling every few seconds, however.)
A mechanism for converting systemd unit files (within certain limits that should cover the majority of units, albeit by no means every possible unit) to service bundles is provided.
Several commands have daemontools aliases:
svstat as an alias for
service-control, for examples.
The bundle mechanism falls back to a daemontools-compatible directory structure, and the daemontools-encore augmentations to the
supervise control/status mechanism fall back to a reduced-functionality Bernstein-daemontools-compatible API.
preset command can employ
systemd preset information, FreeBSD/PC-BSD enable/disable information from
/etc/rc.conf.local, and on/off information from
A utility is provided for reading the daemontools-encore status API for a service and returning a message and an exit code suitable for incorporating into Nagios as plug-in that checks daemontools/nosh dæmon states. The utility returns critical errors (to Nagios) about unexpectedly down daemons, and warnings about daemons that aren't staying up for longer than a second or so.
The socket mechanism is compatible with UCSPI services, as well as with services that use
systemd's file descriptor passing protocol.
Taking a leaf out of systemd's (and
inetd's) book, the package augments Bernstein's
There are separate chain-loading programs for listening on sockets (
local-datagram-socket-listen) and for accepting client connections (
local-stream-socket-accept) to then spawn services in response, with configurable concurrency limits.
One can use them in combination or individually.
Thus listening on a socket and accepting client connections can be separated.
Because of this, the package also works with services that expect the "listening" file descriptor rather than the "accepted" file descriptor, which Bernstein's
UCSPI-TCP does not handle.
There is also a
ucspi-socket-rules-check chain loading program for implementing socket access control for both UCSPI-TCP and UCSPI-UNIX.
The target mechanism is similar to that of
systemd (and, to a lesser extent, Solaris SMF's "milestones").
The procedures of enabling, disabling, starting, and stopping services are intentionally fairly similar to those of
systemd, and the standard system targets are also intentionally similar to targets in
The targets themselves have conventional well-understood names such as
emergency target that is started if the system manager is handed the
-b option; which gives an emergency superuser login prompt without attempting to mount filesystems.
rescue target that is started if the system manager is handed the
-s option; which gives an emergency superuser login prompt after attempting to mount filesystems.
There are no run levels.
The service manager can be used subordinate to another system management mechanism, such as
But the package also includes a system manager proper, separate from the service manager, that is intended to be used as process #1 of a BSD or Linux system.
It handles system state including bringing the system up into its normal running state; responding to reboot, halt, and poweroff requests; bringing the system up in "rescue" and "emergency" modes; and fielding things like power notifications and a secure attention key on the system console device.
It also does some of the mandatory system initialization tasks that process #1 is quietly expected to do, such as mounting "API" filesystems, creating "API" device files, and applying bodges to the system clock as early as possible.
The nosh system manager has no dealings in terminals, which are no longer the worry of process #1. Like with the Solaris SMF, terminal login sessions are simply more services, managed by the service manager. Terminal management features include:
With the chain-loading
login-prompt utilities, one can create a simple script to replicate the function of a minimal
getty tailored for virtual terminal use, that chains onto the normal
As a bonus feature, the source package contains a
execlineb script that does exactly that.
In the pre-built service bundles there are
ttylogin@ service bundles for providing TUI login services on kernel virtual terminals (BSD and Linux) and user-space virtual terminals, using these tools in similar chains.
As mentioned, one can still arrange to enable/disable
ttylogin@ service bundles determined by on/off/onifconsole information in the the BSD (and pre-System 5)
/etc/ttys, and still control things that way if desired.
The pre-built packages for virtual terminals indeed do this.
One can run user-space virtual terminals, to augment or even to replace the operating system kernel's virtual terminals. This is a combination of:
console-terminal-emulator utility, that does the actual emulation work.
It aims to be drop-in compatible with the
wsvt25 (NetBSD kernel virtual terminal in "vt100" mode),
linux (Linux kernel virtual terminal), and
xterm (FreeBSD version 9 and later kernel virtual terminal) terminal types.
console-fb-realizer utility, that realizes a virtual terminal via real, physical, framebuffer and input event device hardware.
console-multiplexor utility, that multiplexes one or more virtual terminals onto another one, allowing one to switch between them using keyboard hot-keys.
In combination user-space virtual terminals look like this. For some background on the design, see my white paper in user-space virtual consoles from a decade ago. The design has changed moderately since then, but the ideas and reasoning are the same. See also the white paper on how parts of this system can be integrated with BRLTTY to provide user-space virtual terminals for BRLTTY users that come up before TTY login.
pty-run respectively one can create a tool that can run arbitrary commands attached to the slave side of a nonce pseudo-terminal.
As a bonus feature, the source package contains
execlineb scripts doing exactly this that provide workalikes for Dan Bernstein's never-completed 1999
These are not the only uses of these tools.
login-banner can also be used, for example, to write out a parameterized notice to a service's log file.
The nosh package includes
cyclog, a simple logging dæmon that covers the commonest simple use cases and that is modelled after the program of the same name that used to come with daemontools.
The log file sets that it writes are automatically rotated, timestamped, size-capped both as to total size and as to individual file size, and strictly size capped (in that there's no way, as there is with logrotate-managed logging, for a log file set to even temporarily exceed its size cap).
There are also tools for importing logs into nosh, turning them into streams of log data that can be pushed through to a log service.
syslog-read is a UCSPI tool that can be attached to a local or a UDP datagram socket.
klog-read is a UCSPI tool that can be attached to a FIFO or a local or TCP stream socket.
For extensive details, see their manual pages and the "Logging" section of the Nosh Guide.