How to partition your drive using OpenZFS w/ GELI Encryption and continue using the FreeBSD Installer

Author: Jonathan Vasquez <jon@xyinn.org>
Last Updated: 2024-09-22-2300
Tested On: FreeBSD 14.1-RELEASE

Preface

This small guide will show you how I install FreeBSD, still using the FreeBSD Installer, but manually partitioning my system with OpenZFS and GELI Encryption. This guide will use a single drive system and GPT + UEFI. This will be a 14.0-CURRENT install, so I will also be having a physical swap partition that is as big as my RAM in order to support core dumps. A swap file can't be used for this. I also won't be putting swap inside of ZFS since coming from ZFS on Linux, there's been a lot of hard crashes I've experienced in the past (even with optimizations), and lastly I'm not using any disk labels.

Adjust any of the information below according to your setup.

NOTE: If you are testing this in VirtualBox, be aware that your system might not boot due to VirtualBox's weirdness with BIOS/UEFI/GPT setups. In the past (13.0-RELEASE), I noticed that I was only able to install FreeBSD in BIOS mode, but I wasn't able to boot the system successfully afterwards since I received a zio_read: error 5. This error occurred even if I did a regular vanilla installation as well. If I tried to boot and install FreeBSD in EFI mode, it wouldn't boot the install iso at all. I had to install the system in BIOS Mode, and then once setup was complete, I switched back to EFI mode to actually boot the system. You can enable EFI mode by going into the VM settings and checking the Enable EFI (special OSEs Only) box. I'm not surprised given ZFS was primarily made for GPT layout, and MBR (in conjunction with UEFI) may have some issues depending on the hardware, and other variables. There may also be some weirdness with FreeBSD as well though. VirtualBox will most likely throw you into the its UEFI Interactive Shell (and not automatically boot into your UEFI environment). All you need to do is point the shell to your FreeBSD UEFI loader. You'll need to adjust this according to where your FAT32 partition is located:

Shell> FS0:\efi\boot\bootx64.efi

Let the games begin <3

Download and Boot Installation Media

  1. Download FreeBSD, put it on a some device, and boot off of it. I'll be using a 14.0-CURRENT snapshot.
  2. Start up the installer and follow the steps as normal. Once you get to the Partitioning step, select Shell.

Partitioning

Find Drives

# sysctl kern.disks

or

# camcontrol devlist

I'll be using /dev/nvd0 as my main drive.

Wipe Partition Layout

# gpart destroy -F nvd0

Create Partition Layout

# gpart create -s gpt nvd0

# gpart add -t efi -s 512M nvd0
# gpart add -t freebsd-swap -s 32G nvd0
# gpart add -t freebsd-zfs nvd0

Create GELI encrypted device for our zpool

# geli init -g -s 4k nvd0p3
# geli attach nvd0p3

NOTE: The -g for init is critical since that's what will allow the bootloader to ask for the passphrase and allows booting from this encrypted device.

Create our zpool and corresponding datasets

In the following layout, we are keeping it simple and just having one dataset for our /, and one for our /home. This is my preferred layout which I use with my own tools for snapshot cleaning and data replication. It is not a Boot Environments compatible layout. The / dataset will be found by the bootloader by reading the pool's bootfs property (which we will be setting soon), and you can manually update this property if needed. This layout also allows you to mount the entire pool at an alternate location with one command and no errors (example: zpool import -R /mnt tank).

Lastly, your / datasets should not have a mountpoint set, since this can cause issues if multiple / datasets are attempted to be mounted at the same location. For the purposes of the installation, we'll specify a / mountpoint on our tank/os/main dataset so it can automatically mount it in the correct location and allow the installer to complete. Once the installation is complete, we'll unset it.

# zpool create \
-o ashift=12 \
-O compression=lz4 \
-O atime=off \
-m none \
-R /mnt \
tank \
nvd0p3.eli

# zfs create tank/os
# zfs create -o mountpoint=/ tank/os/main
# zfs create -o mountpoint=/usr/home tank/home

NOTE: On FreeBSD, /home is a symlink to /usr/home.

NOTE: I've also tried experimenting with making a tank/root dataset that points to /root so that I can keep the root user's home directory separate as well. However, this causes an error during the FreeBSD installation saying that it can't create the root/.profile file. In order to avoid any unintended effects, I'm not including that dataset here, but feel free to create it manually post install.

Set the ZFS root dataset for booting

# zpool set bootfs=tank/os/main tank

Create and Mount the EFI directory

We'll create and mount our EFI directory so that when the FreeBSD installer extracts the base system, the EFI files will land at this location appropriately. 

# mkdir -p /mnt/boot/efi
# newfs_msdos -F 32 -S 4096 -c 1 /dev/nvd0p1
# mount -t msdosfs /dev/nvd0p1 /mnt/boot/efi

Add fstab entries

We'll add our swap and efi partitions to fstab. The swap will be automatically encrypted per boot since we are specifying the .eli extension on it.

# vi /tmp/bsdinstall_etc/fstab

/dev/nvd0p1        /boot/efi    msdosfs    rw         2    2
/dev/nvd0p2.eli    none         swap       sw,late    0    0

# Optional

# Mount /proc
proc /proc procfs rw 0 0

# Use RAM for /tmp (tmpfs)
tmpfs /tmp tmpfs rw,mode=777 0 0

Return and Continue Installation

Return back to the installer and continue the setup until you reach the Final Configuration step. Select Exit. The next step should be: Manual Configuration. Select Yes so that we can drop back into a chrooted environment of our soon-to-be system. We'll do some last minute configuration needed to ensure a successful boot.

Final Configuration

Enable GELI module loading

# vi /boot/loader.conf

# The following two should already be in here.
cryptodev_load="YES"
zfs_load="YES"

# Add GELI module loading.
geom_eli_load="YES"

# If you are using something lower than FreeBSD 13, you'll also need:
aesni_load="YES"

# These are optional and are settings I use

# Configure boot delay
autoboot_delay="5"

# Enable /dev/diskid and /dev/gptid support
kern.geom.label.disk_ident.enable="1"
kern.geom.label.gptid.enable="1"

Enable ZFS script to load on start up

Enable ZFS in rc.conf so your datasets load correctly.

# vi /etc/rc.conf

zfs_enable="YES"

That's it! Type reboot, and have fun :).

Once you are logged into your system, you'll see everything worked and it will look something like this:

# zfs list

NAME           USED  AVAIL  REFER  MOUNTPOINT
tank           619M  43.0G    96K  none
tank/home       96K  43.0G    96K  /usr/home
tank/os        618M  43.0G    96K  none
tank/os/main   618M  43.0G   618M  none

References