How to partition your drive using OpenZFS w/ GELI Encryption and continue using the FreeBSD Installer

Author: Jonathan Vasquez <jon@xyinn.org>
Last Updated: 2022-04-29-1600
Tested On: FreeBSD 14.0-CURRENT (Framework Laptop @ Batch 6)

Preface

This small guide will show you how I install FreeBSD, still using the FreeBSD Installer, but manually partitioning my system with OpenZFS and GELI Encryption. This guide will use a single drive system and GPT + UEFI. This will be a 14.0-CURRENT install, so I will also be having a physical swap partition that is as big as my RAM in order to support core dumps. A swap file can't be used for this. I also won't be putting swap inside of ZFS since coming from ZFS on Linux, there's been a lot of hard crashes I've experienced in the past (even with optimizations), and lastly I'm not using any disk labels.

Adjust any of the information below according to your setup.

NOTE: If you are testing this in VirtualBox, be aware that your system might not boot due to VirtualBox's weirdness with BIOS/UEFI/GPT setups. In the past (13.0-RELEASE), I noticed that I was only able to install FreeBSD in BIOS mode, but I wasn't able to boot the system successfully afterwards since I received a zio_read: error 5. This error occurred even if I did a regular vanilla installation as well. If I tried to boot and install FreeBSD in EFI mode, it wouldn't boot the install iso at all. I had to install the system in BIOS Mode, and then once setup was complete, I switched back to EFI mode to actually boot the system. You can enable EFI mode by going into the VM settings and checking the Enable EFI (special OSEs Only) box. I'm not surprised given ZFS was primarily made for GPT layout, and MBR (in conjunction with UEFI) may have some issues depending on the hardware, and other variables. There may also be some weirdness with FreeBSD as well though.

Based On: Previous Instructions for Unencrypted Mirrored Setup

Let the games begin <3

Download and Boot Installation Media

  1. Download FreeBSD, put it on a some device, and boot off of it. I'll be using a 14.0-CURRENT snapshot.
  2. Start up the installer and follow the steps as normal. Once you get to the Partitioning step, select Shell.

Partitioning

Find Drives

# sysctl kern.disks

or

# camcontrol devlist

I'll be using /dev/nvd0 as my main drive.

Wipe Partition Layout

# gpart destroy -F nvd0

Create Partition Layout

# gpart create -s gpt nvd0

# gpart add -t efi -s 512M nvd0
# gpart add -t freebsd-swap -s 32G nvd0
# gpart add -t freebsd-zfs nvd0

Create GELI encrypted device for our zpool

# geli init -g -s 4k nvd0p3
# geli attach nvd0p3

NOTE: The -g for init is critical since that's what will allow the bootloader to ask for the passphrase and allows booting from this encrypted device.

Create our zpool and corresponding datasets

# zpool create \
-o ashift=12 \
-O compression=lz4 \
-O atime=off \
-m none \
-R /mnt \
tank \
nvd0p3.eli

# zfs create -o mountpoint=/ tank/os
# zfs create -o mountpoint=/usr/home tank/home

NOTE: On FreeBSD, /home is a symlink to /usr/home.

Create and Mount the EFI directory

We'll create and mount our EFI directory so that when the FreeBSD installer extracts the base system, the EFI files will land at this location appropriately.

# mkdir -p /mnt/boot/efi
# newfs_msdos -F 32 -S 4096 -c 1 /dev/nvd0p1
# mount -t msdosfs /dev/nvd0p1 /mnt/boot/efi

Set the ZFS root dataset for booting

# zpool set bootfs=tank/os tank

Add fstab entries

We'll add our swap and efi partitions to fstab. The swap will be automatically encrypted per boot since we are specifying the .eli extension on it.

# vi /tmp/bsdinstall_etc/fstab

/dev/nvd0p1    /boot/efi    msdosfs    rw    2    2
/dev/nvd0p2.eli    none    swap    sw    0    0

Return and Continue Installation

Return back to the installer and continue the setup until you reach the Final Configuration step. Select Exit. The next step should be: Manual Configuration. Select Yes here so that we can drop back into a chrooted environment of our soon-to-be system. We'll do some last minute configuration needed to ensure a successful boot.

Final Configuration

Enable GELI module loading

# vi /boot/loader.conf

# The following two should already be in here.
cryptodev_load="YES"
zfs_load="YES"

# Add GELI module loading.
geom_eli_load="YES"

# If you are using something lower than FreeBSD 13, you'll also need:
aesni_load="YES"

Enable ZFS script to load on start up

Enable ZFS in rc.conf so your datasets load correctly.

# vi /etc/rc.conf

zfs_enable="YES"

That's it! You should now have a bootable system with your desired layout.

Go ahead and type exit and select Reboot. Enjoy ;).

References