Fast & Stable WiFi with wifibox on FreeBSD
Author: Jonathan Vasquez <jon@xyinn.org>
Last Updated: 2022-09-11-1844
Currently On: FreeBSD 13.1-STABLE (stable/13-n252072-1243360b1a0/GENERIC)
Machine: Framework Laptop (Batch 6). Ordered on November 20, 2021.Preface
So.. I've been using and experimenting with various laptops over the past 4 months, trying to find the right one for FreeBSD, in regards to stability and general usability for normal every day stuff (everything except for gaming. Gaming is reserved for my Razer Laptop on Windows). One thing that has caused a lot of headaches has been the wifi support. All of the laptops I own (Thinkpad X260, Thinkpad X1 Carbon Gen 7, and the Framework Laptop @ Batch 6) are using Intel chips, either via the iwm driver (on the X260), or the iwlwifi driver on the others. However, iwlwifi is highly experimental right now, it will crash your system depending on the state of the card (attempting to assign a static ip, changing it, restarting the wireless interface multiple times, etc). Furthermore, the speeds are all limited to 802.11a/g. Another thing that the driver causes is your machine to crash when you attempt to resume it from sleep, effectively making sleep useless. In this quick guide, we will see how we can easily use wifibox (as a workaround until the native iwlwifi driver on FreeBSD is ready) to achieve full performance and stability.
I'll like to credit John Grafton for his guide on this, which allowed me to understand this better. I recommend you to also check out his post.
What is wifibox?
The TLDR is that wifibox essentially spins up an Alpine Linux VM using FreeBSD's bhyve virtualization technology, and allows you to passthrough your machine's wireless card from the host directly into the guest. To put the cherry on top, there is some "magic" (firewall forwarding rules and natting), that allows the traffic to flow between your host and your guest.
What we are essentially doing here is just delegating all wifi driver responsibilities (including associating with the target access point) to Linux, thus allowing the Linux VM to process all of the networking requests, and then communicate all of that back up to the host (and vice-versa). You'll be able to still communicate with any of your internal LAN services as well :D.
It's insane that you could do something like this, and I love it.. although it's sad we have to do this, but I'm happy the FreeBSD iwlwifi work is progressing and one day we should not need to do this anymore.
Set Up
Installation
Before we begin, let's download and install wifibox before we do any further work (since you may not have internet once we start messing with the network in the meantime):
Since wifibox is already in the ports tree, we also have binary packages available. Installing it should also pull in a few extra dependencies.
# pkg install wifibox# pkg info | grep wifibox
wifibox-1.1.1 Wireless card driver via virtualized Linux
wifibox-alpine-20220712 Wifibox guest based on Alpine Linux
wifibox-core-0.10.0 Wifibox core functionalityBlock Wireless Drivers
Now that we have wifibox installed, we'll want to make sure that we have blocked any wireless drivers that the system is currently using. Since we are going to passthrough the wireless card to the VM, the host system cannot use the wireless card directly. For my case, I know that my system will automatically load the if_iwlwifi module, and that will cause the driver to attach itself to the wireless PCI slot. I've also blocked the if_iwm driver as well which is another Intel driver available for some older cards. You can add the following to your /etc/rc.conf (adjust accordingly), reboot afterwards:
# iwlwifi driver is unstable. Will use wifibox instead.
devmatch_blacklist="if_iwm if_iwlwifi"Find your wireless card's PCI number
Once we are done rebooting, we'll want to find our wireless card's PCI number. This is the number that we will let the VM use later. You can run the following command to find it. Scroll until you either find something related to wireless or WiFi. On my computer the pci number was at 0:20:3.
# pciconf -lv | less
none@pci0:0:20:3: class=0x028000 rev=0x00 hdr=0x00 vendor=0x8086 device=0x02f0 subvendor=0x8086 subdevice=0x0030
vendor = 'Intel Corporation'
device = 'Comet Lake PCH-LP CNVi WiFi'
class = networkNotice that it says none@. This means that there is no driver attached, and that's exactly what we want to see. If something is attached to your wireless card, you'll need to go back to the previous step and find what driver is attached, and block it.
Configuration
Specifying the PCI card
Once that's done, we just need to do some minor configuration and you should be up and running. wifibox uses sane defaults. Once you get something working, feel free to tweak it further :).
First let's tell bhyve where the wireless card is located. You'll want to find the passthru line and change it to the numbers we got before. The format will use slashes instead of colons as a delimiter. So since my numbers were 0:20:3, I'll want to use 0/20/3.
# vim /usr/local/etc/wifibox/bhyve.conf
passthru=0/20/3You can also set the console line to yes, so we can log into the VM and see what's going on in there. Feel free to set it back to no, once you are done experimenting.
Increase the VM memory
We will proactively increase the amount of memory the VM uses. The current default of 45MB defined in the /usr/local/share/wifibox/bhyve.conf file is too low and it seems to override the memory specified in the /usr/local/etc/wifibox/bhyve.conf file. If the memory is too low, it will cause the VM to crash or become unresponsive, which will make our internet connection drop. For now, I've increased my VM memory to 512MB in both of these files to test out this solution. I believe 512MB is a bit high for this purpose but I want to test out the theory first and once confirmed working, I'll lower it to 128MB and see if it still works.
Setting up your wpa_supplicant configuration
You'll want to copy over your /etc/wpa_supplicant.conf if you have one to the /usr/local/etc/wifibox/wpa_supplicant/wpa_supplicant.conf location. This file will be used inside of the VM so that Linux can associate with the access point.
If you don't have one, you can do something like this to create one:
# wpa_passphrase [SSID] [PASSPHRASE] > /etc/wpa_supplicant.confNow copy it over (you can try using a symlink as well - I haven't tested that):
# cp /etc/wpa_supplicant.conf /usr/local/etc/wifibox/wpa_supplicant/Network Configuration in /etc/rc.conf
Before we boot up the VM, let's also add our network configuration to /etc/rc.conf, so that when your machine starts up, it will start the VM automatically and request the IP from the VM. The actual LAN IP will be inside the VM (since that's what's talking to the access point directly). The host will receive an internal IP (In the range of 10.0.0.2 - 10.0.0.254).
wifibox_enable="YES"
ifconfig_wifibox0="SYNCDHCP"
background_dhclient_wifibox0="YES"
defaultroute_delay="0"I haven't gotten a chance to play around and test just using a static IP for this particular interface but for the meantime, since I only want this interface to have 1 possible IP via DHCP, I've restricted the DHCP ranges in /usr/local/etc/wifibox/appliance/udhcpd.conf to only give out 10.1.0.2 as a possible address.
NOTE: If you have something running on those addresses (like I have wireguard on 10.0.0.1), you'll want to adjust the relevant files in the wifibox directory. In my case, I just shifted wifibox to use 10.1.0.0/24 addresses and everything worked. I was still able to communicate with all LAN services on both 192.168.1.0/24 and 10.0.0.0/24 with no issues. I edited the following files to achieve this in the wifibox/appliance directory: interfaces.conf, udhcpd.conf, and uds_passthru.conf.
Also note that devices on your LAN attempting to communicate to a particular port on this machine won't be able to communicate due to the double NAT and firewalling going on. You'll need to adjust the VM firewall appropriately. This is an example of something I did to allow port 22 through, I did notice a performance degradation, and there are some other issues still happening for me regarding this type of connectivity. For now I'm actually using the wireguard IP to communicate with no performance issues, despite the current network setup.
Start up
At this point you pretty much have everything configured. Go ahead and run service wifibox start and wait a few seconds. The VM will start up and then the connection will stabilize. If you run ping, you'll see it initially will fluctuate around 500 ms, but then it should drop down to about 15 ms (depending what you are pinging).
That's it! Enjoy your awesome speeds :).
Other
Connecting to the VM console
If you want to log into the console and play around in there (or debug >_<) you can do the following (make sure the console line is set to yes as described in the bhyve.conf file above):
# wifibox consoleThe username is root and there is no password. Once you want to exit the VM, you'll want to press ~. in sequence. If it isn't working, try pressing [Enter] and try again.
You can see that Linux's dmesg says the following for the iwlwifi specifically:
[ 0.856565] Intel(R) Wireless WiFi driver for Linux
[ 0.856676] iwlwifi 0000:00:06.0: can't derive routing for PCI INT A
[ 0.856678] iwlwifi 0000:00:06.0: PCI INT A: not connected
[ 0.857437] iwlwifi 0000:00:06.0: Failed to set affinity mask for IRQ 41
[ 0.913448] iwlwifi 0000:00:06.0: api flags index 2 larger than supported by driver
[ 0.913455] iwlwifi 0000:00:06.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 89.3.35.37
[ 0.913587] iwlwifi 0000:00:06.0: loaded firmware version 66.f1c864e0.0 QuZ-a0-jf-b0-66.ucode op_mode iwlmvm
[ 0.933782] iwlwifi 0000:00:06.0: Detected Intel(R) Wireless-AC 9560 160MHz, REV=0x354
[ 1.048592] iwlwifi 0000:00:06.0: Detected RF JF, rfid=0x105110
[ 1.105549] iwlwifi 0000:00:06.0: base HW address: f8:e4:e3:eb:35:02ifconfig says this in the VM:
eth0 Link encap:Ethernet HWaddr 00:A0:98:8A:05:71
inet addr:10.1.0.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::2a0:98ff:fe8a:571/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14624 errors:0 dropped:0 overruns:0 frame:0
TX packets:13832 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6303826 (6.0 MiB) TX bytes:6737636 (6.4 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
wlan0 Link encap:Ethernet HWaddr F8:E4:E3:EB:35:02
inet addr:192.168.1.139 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::fae4:e3ff:feeb:3502/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14370 errors:0 dropped:0 overruns:0 frame:0
TX packets:14475 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6770983 (6.4 MiB) TX bytes:6518389 (6.2 MiB)And we see this from the host's ifconfig:
em0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=481049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER,NOMAP>
ether f8:75:a4:ef:9d:81
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wifibox0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:ff:99
inet 10.1.0.2 netmask 0xffffff00 broadcast 10.1.0.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:c0:3f
groups: tap
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 368
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=68009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether a0:ce:c8:d3:ca:0a
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>Sleep / Resume Issues
wifibox does have a workaround for sleep / resume which seems to be working for most people. However, in my case it hasn't been working (tested and reproduced in 13.1-RELEASE and 13.1-STABLE, so it's not a regression). It seems there is a bug in bhyve where the PCI passthrough resource isn't properly released/cleaned up when the host is suspended/resumed. I can sleep and resume the machine fine with no crashes, however in about 1-3 resumes, the wireless card will no longer come back up inside of the VM (it's still visible and works fine on the host), but for whatever reason the wlan0 interface inside of Linux is just completely gone. The only way to "fix" it is to reboot the machine (wifibox restart vmm did not work for me and my devd trigger is working and is ran upon resume). I've opened up this issue at the wifibox repo, opened up a bug in FreeBSD, and emailed the freebsd-virtualization mailing list.
With that said, I was able to figure out a workaround for the above issue. I slightly modified the wifibox provided devd file so that it has a suspend event which stops the entire VM before suspending. Upon resume, the VM will start up again and we will get the IP re-assigned to our interface via DHCP. This adds a small delay between sleep/resumes, but it works well. @pgj has merged in some code that should automate most of this for you, the commit can be seen here.
Below is my modified devd file. You can modify according to your needs:
# cat /usr/local/etc/devd/wifibox.conf
notify 11 {
match "system" "ACPI";
match "subsystem" "Suspend";
action "logger 'Stopping wifibox before suspend' && /usr/local/sbin/wifibox stop && /etc/rc.suspend acpi $notify";
};
notify 11 {
match "system" "ACPI";
match "subsystem" "Resume";
action "/etc/rc.resume acpi $notify && logger 'Starting wifibox after resume and getting IP via DHCP' && /usr/local/sbin/wifibox start && /sbin/dhclient wifibox0";
};