There are several reasons why it’s important to look at alternative operating systems, particularly open source ones. Firstly, open source operating systems tend to be more customizable and configurable compared to proprietary operating systems. This allows users to have more control over their system, and to tailor it to their specific needs

Another important reason to look at alternative operating systems is to avoid vendor lock-in. Proprietary operating systems often come bundled with specific hardware or software, which can limit the flexibility of the user.

What is FreeBSD?

FreeBSD is a UNIX operating system that traces its lineage all the way back to the original Bell Labs Research UNIX. Now nearly 29 years old, FreeBSD has a long and storied history as a critical component of the Internet.

FreeBSD was chosen by many of the early mega-sites such as Yahoo and the original Hotmail. FreeBSD also powered cdrom.com—which was, in the 1990s, the world’s most heavily trafficked site.

BSD’s origins in and connections to academia made it many students’ first introduction to UNIX-like operating systems. From this lofty beginning, FreeBSD has continued to innovate and provide high quality infrastructure software.

Today, FreeBSD provides a high performance, robust, and secure operating system with regular releases and long-term support. The FreeBSD Project maintains and releases a complete operating system. It delivers the FreeBSD kernel, its device drivers, userland utilities, and documentation as a single cohesive product. 

Advantage #1 – Integration with OpenZFS

OpenZFS is a robust and powerful filesystem that enables many advanced and innovative features. Unlike operating systems which merely make OpenZFS available as a plugin, FreeBSD has fully integrated OpenZFS as its premiere filesystem. This means that FreeBSD’s OpenZFS is fully supported from the bootstrap code that loads the boot loader, through to the operating system utilities.

FreeBSD has supported booting directly from ZFS for more than ten years. Its boot loader, kernel, and entire root filesystem may all be hosted on a ZFS pool without requiring a separate boot filesystem or initramfs. This support enables one of FreeBSD’s most powerful features, boot environments. 

OpenZFS is both a volume manager, and a filesystem, and provides pooled storage. This means that the system can have an unlimited number of filesystems all sharing the same storage volume and free space. With snapshots and clones it becomes possible to have multiple versions of the root filesystem.

What is a Boot Environment?

OpenZFS snapshots serve as a complete archive of the entire filesystem and its state at a single, atomic point in time. Given this feature, it is obviously best practice to create a snapshot of the system before applying any major changes. This way, if the changes produce any major problems, they can be quickly and reliably “rolled back,” leaving the system in its pre-change condition.

Boot Environments take this concept a step further. Using the bectl utility, a snapshot of the root filesystem is taken, then immediately cloned, creating a boot environment (BE). This clone consumes no additional space since it shares the blocks with the current root filesystem via ZFS’s copy-on-write feature. Now it is safe to apply changes to the system. 

If these changes cause a problem—even a problem severe enough to prevent the system from booting successfully—the operator is able to select this previous version of the root filesystem, the “boot environment,” from the boot loader menu. The system will then boot with the changes undone. 

The boot environment purposely only reverts changes made to the root filesystem, so changes to databases, user files, and other application data are NOT rolled back—only the changes to the operating system itself. If the operator needs to roll back databases and user files as well, that can be done with simple “zfs rollback filesystem@snapshot” commands at the console!

When applied effectively, this scheme can ensure the system is always recoverable, no matter what goes wrong.

The freebsd-update tool, used to upgrade to new versions of FreeBSD, automatically detects when your system is configured with boot environments and will create a “before” environment during each upgrade, ensuring it can be undone.

Boot Environments offer another useful feature, boot once. Using this feature, an alternative boot environment can be selected for the next boot only.

As soon as the boot once flag is observed by the boot loader, it is cleared—the loader itself remembers the flag was set, but all subsequent boots will use the default boot environment. This makes upgrades of remote systems nearly bullet proof. 

Apply the upgrade to a new boot environment, then set it to boot once only (the -t flag to bectl activate). Once the system boots, if successful, then the new boot environment can become the default (bectl activate again, without the -t flag). If the system does not come up for any reason, simple power cycling it will revert to the previously working boot environment.

The bectl utility is powered by libbe, making it easy to create your own tooling and automation on top of its functionality as well. Integrating libbe into your application to manage its datasets and gain the full power of boot environments for even the most unique use cases.

 ZFS integration in FreeBSD

ZFS is integrated in all aspects of FreeBSD. We have already discussed the boot loader, which has full read-only support for all OpenZFS features except native encryption. While some other operating systems can boot from ZFS, they typically require the pool have all modern ZFS features disabled, as they only support the most basic feature flags.

FreeBSD integrated ZFS support directly into its installer in 2014, with the release of FreeBSD 10. This support directly in the installer allows users to install new systems with a ZFS root filesystem, already laid out for boot environments. The installer supports fairly complex pool layouts as well, including multi-way mirrors, multiple mirror sets (similar to RAID10), and all flavours of RAID-Z. This ensures most installations are a common, well supported, and follow best practices with little or no additional configuration required.

ZFS exposes its system-wide tunables on FreeBSD via the common sysctl interface, under the vfs.zfs MIB. This makes it extremely easy to examine and modify the values for every setting from the maximum size of the ARC cache to the queue depth for each I/O type. FreeBSD has a few unique parameters as well: for example, the vfs.zfs.arc_free_target setting controls the interaction between the FreeBSD VM subsystem and ZFS, allowing the ARC to shrink itself when the amount of free memory in the OS drops below a certain threshold.

A set of read-only statistics are also exposed via the sysctl interface, via the kstat.zfs MIB. This interface exposes a rich set of metrics that can be used to monitor the health and performance of ZFS. There are system wide stats about the contents of the ARC broken down into various categories, as well as the per-dataset stats that allow you to determine which datasets are creating the most load on the system.

This combines very well with another feature of FreeBSD, the built-in Prometheus exporter: prometheus_sysctl_exporter. This makes it easy to export all of these ZFS metrics into your monitoring system, along with stats about the rest of the OS.

FreeBSD pioneered the concept of light weight virtualization with containers, via its jail mechanism, introduced in 2000. Modern FreeBSD jails also directly integrate ZFS support. 

This support allows a ZFS dataset to be delegated to a jail, which in turn allows the administrator of the jail to create and manage their own ZFS datasets within the jail, without giving them elevated access to the host system. These in-jail administrators can only see the dataset(s) specifically allocated to them, which makes management of multi-tenant configurations much safer and easier to scale.

Many of the core system tools have also directly integrated ZFS support. This includes some of the most basic administrative tools such as top(1), which provides details about the memory consumption and makeup of the ZFS ARC:

ARC: 17G Total, 3100M MFU, 12G MRU, 13M Anon, 132M Header, 1799M Other
     13G Compressed, 14G Uncompressed, 1.08:1 Ratio

And systat(1) -zarc, which displays details about the ZFS ARC’s cache hit ratios:

                                Rate   Hits Misses | Total Rate   Hits Misses
     arcstats                  : 82%     93     20 |        95%  3713M   189M
     arcstats.demand_data      : 66%     22     11 |        89%   947M   111M
     arcstats.demand_metadata  : 88%     63      8 |        99%  2748M  7872k
     arcstats.prefetch_data    :  0%      0      0 |        14%  7088k 40231k
     arcstats.prefetch_metadata: 88%      8      1 |        27% 11075k 29437k
     zfetchstats               :  0%      0      8 |        13% 57972k   366M
     arcstats.l2               :  0%      0      0 |         0%      0      0
     vdev_cache_stats          :  0%      0      0 |         0%      0      0

Finally, ZFS is wired up to FreeBSD’s devd event system, which provides the capability to automatically respond to ZFS events as they happen. This support is even further extended with zfsd, FreeBSD’s fault management daemon for ZFS. 

The zfsd daemon can automatically replace failed drives with spares, re-online disks that became temporarily disconnected, pre-emptively replace disks that are causing too many failures or slowing the entire pool, and otherwise handle most common ZFS faults.

Advantage #2 – Custom Packaging

FreeBSD has long been renowned for its ports collection—a package building infrastructure that keeps over 30,000 applications available on multiple versions of the OS. FreeBSD ports are the source code which its packages are built from, but users may also directly compile and install code from the ports tree themselves.

By making use of the ports tree—and not just the pre-compiled system packages, from pkg—a FreeBSD user can easily customize the build-time parameters of the application, adding and removing support for additional features and controlling the dependency list.

FreeBSD builds its official, pre-compiled packages using a tool called poudriere (French for powder keg, the name of the previous tool it replaced). 

Poudriere uses a number of interesting techniques—and many of FreeBSD’s core features—to offer a great deal of flexibility. It builds each package in a clean container, with only the base operating system and the declared dependencies installed and no network access allowed. Between each build, the container is restored to a pristine state using a ZFS snapshot, allowing quick turnover without any remnants left behind.

This minimal environment and network isolation ensures that the applications don’t link against other libraries that just happen to be installed on the build system, or otherwise cause undetected issues. Poudriere’s automatic build environment isolation also allows many packages to be compiled concurrently, which is generally a much more effective use of many CPU cores than trying to use the same cores to build packages one at a time. 

Poudriere supports building for multiple versions of FreeBSD, against different checkouts of the ports tree, and supports “package sets” where you can define different configuration options for various use cases. Poudriere also supports cross-building, allowing a single machine to compile packages for multiple architectures. This makes it easy to build all of the different combinations of architecture, FreeBSD version, package versions, and configuration options via a single automated tool running on a single FreeBSD system.

Poudriere also supports mechanisms for package signing, allowing your machines or appliances to install only those packages signed by your trusted cryptographic keys. 

Klara provides customized solutions for package building for use cases across various industries, from traditional application servers and cloud deployments, to air-gapped zero-trust infrastructure, and custom appliances.

Advantage #3 – Fleet Management

One of the most powerful advantages of FreeBSD is the ability for a small team to manage tremendous scale. FreeBSD combines very strong fundamental building blocks with flexible and customizable frameworks to create infrastructure that scales without adding complexity.

Almost two and a half decades of experience with containers, combined with 15 years of integrated ZFS have given FreeBSD a conspicuous lead in making a more manageable, better documented architecture for large scale operations.

Combining this long organizational experience with unusually strong separation of the operating system components from the application software, FreeBSD provides its users a stable base to build upon, while still delivering continuous innovation and cutting edge features.

FreeBSD has always been designed with large infrastructure in mind. Practices such as splitting configuration files into fragments with local overrides and structured configuration with UCL (the Universal Configuration Language) help FreeBSD keep site-wide configuration separate from machine specific configuration, easing automation and making upgrades and replacement much less arduous.

Even a small team can manage a massive infrastructure fleet and provide outsized value to the organization by using ZFS boot environments and any of several fleet orchestration tools to manage and upgrade OS images and containers. For example, Netflix’s OCA server operations team is made up of fewer than 10 people, yet manages many thousands of the servers which stream video to its customers.

Conclusions

FreeBSD is a compelling and cutting-edge operating system that provides a wealth of features and advantages. FreeBSD’s deep OpenZFS integration, completely customizable packaging, and the ability to manage a huge fleet with a small team make it a clear contender for consideration in your next infrastructure build. 

Add in the expert and bespoke FreeBSD support Klara provides, and you have the recipe for high performance, low maintained, long-lived infrastructure.