🎛️ The Boot Loader and Kernel Tuning: loader.conf, sysctl and Single-User Mode

Follow a FreeBSD boot from the disk to the login prompt, learn which settings belong in loader.conf and which in sysctl.conf, and use the loader prompt and single-user mode to get out of trouble

▶

The road from power button to login

FreeBSD goes through several stages before you can log in, and tuning means knowing which stage reads which setting:

1. The firmware (the old BIOS, or modern UEFI) finds a disk and runs the first piece of boot code on it.
2. The boot code is small and knows just enough to find the next stage. On a ZFS system it can read ZFS, so it finds the loader inside the pool.
3. The loader is a more capable program with a menu (written in Lua scripts in recent FreeBSD). It reads the loader configuration files, loads the kernel and any modules you asked for, builds the kernel environment (a list of named values) and hands over.
4. The kernel starts, takes its settings from that environment, finds the root file system (Trying to mount root from zfs:zroot/ROOT/default) and starts init.
5. init runs the rc scripts: services, networking, and, once the machine reaches multi-user mode, your /etc/sysctl.conf.

Everything on this page was run for real on FreeBSD 15.0-RELEASE in a virtual machine. The boot output was captured through a virtual serial console, which you will meet in a moment. (Stage descriptions are from the documentation; each thing we show is marked as seen.)

What this machine boots with

root@practice:~ # sysctl machdep.bootmethod
machdep.bootmethod: BIOS
root@practice:~ # gpart show vtbd0
=>      40  33554352  vtbd0  GPT  (16G)
        40      1024      1  freebsd-boot  (512K)
      1064       984         - free -  (492K)
      2048   4194304      2  freebsd-swap  (2.0G)
   4196352  29356032      3  freebsd-zfs  (14G)
  33552384      2008         - free -  (1.0M)
root@practice:~ # zpool get bootfs zroot
NAME   PROPERTY  VALUE               SOURCE
zroot  bootfs    zroot/ROOT/default  local
root@practice:~ # cat /boot/loader.conf
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
zfs_load="YES"
root@practice:~ # ls /boot/loader.conf.d
techcommune.conf
root@practice:~ # ls /boot/defaults /boot/lua | head -9
/boot/defaults:
loader.conf

/boot/lua:
cli.lua
color.lua
config.lua
core.lua
drawer.lua
root@practice:~ # ls /boot/kernel | wc -l
     860

• machdep.bootmethod: BIOS: this virtual machine boots the old way. (The /boot folder also holds the UEFI files, boot1.efi and loader.efi, for machines that boot that way: seen in a listing.)
• gpart show shows the disk. Partition 1, freebsd-boot (512 KB), holds the boot code. Then 2 GB of swap and the 14 GB ZFS partition with the pool.
• zpool get bootfs zroot: the pool has a property bootfs, which names the dataset to boot: zroot/ROOT/default. That is the boot environment from the upgrade page. Changing which environment boots means changing this property, and that is what bectl activate does (from the documentation, not run here).
• /boot/loader.conf: the installer wrote three lines: two settings for disk labels, and zfs_load="YES", which tells the loader to load the ZFS driver so it can mount the root.
• /boot/defaults/loader.conf (199 lines) is the list of every loader setting with its default, mostly commented out. Never edit it. It is the reference: read it to see what exists. Your changes go in /boot/loader.conf or, better, in a file of their own in /boot/loader.conf.d/ (it is techcommune.conf in the listing above: we made it a little later on this page).
• /boot/lua holds the loader's menu scripts, and /boot/kernel holds the kernel and its 860 files (modules and so on).

The kernel environment: kenv

root@practice:~ # kenv | wc -l
     125
root@practice:~ # kenv | head -5
acpi.oem="BOCHS "
acpi.revision="1"
acpi.rsdp="0x000f52b0"
acpi.rsdt="0x7ffe237a"
acpi_dsdt_load="NO"
root@practice:~ # kenv kern.hz
500
root@practice:~ # dmesg | grep -m1 "FreeBSD 15"
FreeBSD 15.0-RELEASE releng/15.0-n280995-7aedc8de6446 GENERIC amd64

The loader builds a list of name="value" pairs, and passes it to the kernel. kenv shows it: 125 entries here, from the ACPI facts the firmware reported to the settings from the loader files. kenv kern.hz reads one value: 500 (set by us, see below). Any setting in loader.conf ends up here, and the kernel reads its tunables from it as it starts. This is why loader settings can change things that cannot be changed once the machine is running.

Three kinds of settings

FreeBSD has a tidy split, and it will tell you which kind you are looking at:

• Writable at run time (sysctl): change it now, no reboot, and it lasts until the next boot.
• Boot-time tunables (loader.conf): only read at boot. At run time they are read only.
• Neither: facts about the hardware, which you simply cannot change.

root@practice:~ # sysctl -d kern.ipc.somaxconn
kern.ipc.somaxconn: Maximum listen socket pending connection accept queue size (compat)
root@practice:~ # sysctl kern.ipc.somaxconn=2048
kern.ipc.somaxconn: 1024 -> 2048
root@practice:~ # sysctl kern.hz=1000
sysctl: oid 'kern.hz' is a read only tunable
sysctl: Tunable values are set in /boot/loader.conf
root@practice:~ # sysctl -W kern.hz
root@practice:~ # sysctl -T kern.hz
kern.hz: 500
root@practice:~ # sysctl -W kern.ipc.somaxconn
kern.ipc.somaxconn: 2048
root@practice:~ # sysctl -T kern.ipc.somaxconn
root@practice:~ # sysctl hw.ncpu=1
sysctl: oid 'hw.ncpu' is read only
root@practice:~ # sysctl -a | wc -l; sysctl -aW | wc -l; sysctl -aT | wc -l
   12028
    1733
    1252

• sysctl -d NAME prints a description of a setting: kern.ipc.somaxconn is the size of the queue of connections waiting to be accepted by a server.
• sysctl kern.ipc.somaxconn=2048 changed it live: 1024 -> 2048 (it was 128, the default, until our /etc/sysctl.conf raised it to 1024: see below). No reboot, effect at once.
• sysctl kern.hz=1000 was refused, and FreeBSD says why in two lines: oid 'kern.hz' is a read only tunable and Tunable values are set in /boot/loader.conf. kern.hz is how many times per second the kernel's clock ticks. It is fixed when the kernel starts, so you can only choose it at boot.
• The two flags make it plain: sysctl -W NAME prints the setting only if it is writable (somaxconn yes, kern.hz no), and sysctl -T NAME only if it is a boot tunable (kern.hz yes, somaxconn no).
• hw.ncpu=1 says only is read only, with no hint about the loader: the number of processors is a fact, not a tunable.
• The counts: 12,028 settings in total, 1,733 writable at run time and 1,252 tunable at boot. Most of the rest are read-only information.

Make a run-time setting permanent: /etc/sysctl.conf

A sysctl change is gone after a reboot. To keep it, put it in /etc/sysctl.conf, which the system reads when it reaches multi-user mode. We chose two settings to show it: the listen queue and the ZFS cache size.

root@practice:~ # sysctl vfs.zfs.arc.max
vfs.zfs.arc.max: 0
root@practice:~ # sysctl kstat.zfs.misc.arcstats.c_max
kstat.zfs.misc.arcstats.c_max: 1314306560
root@practice:~ # sysctl hw.physmem
hw.physmem: 2102890496

ZFS keeps recently used data in memory, the ARC, and by default is willing to use a large share of memory. vfs.zfs.arc.max is 0, which means "decide yourself", and the ARC's ceiling, c_max, came to 1,314,306,560 bytes on this machine with 2,102,890,496 bytes of memory: about 62 percent. On a small machine that also runs services, you may want to cap it. Before changing anything, a safety net, and then the files:

root@practice:~ # bectl create before-tuning
root@practice:~ # bectl list
BE            Active Mountpoint Space Created
before-tuning -      -          8K    2026-10-06 15:04
default       NR     /          952M  2026-10-05 20:58

root@practice:~ # cat > /boot/loader.conf.d/techcommune.conf <<'EOF'
> console="comconsole,vidconsole"
> autoboot_delay="3"
> loader_menu_title="TechCommune Lab"
> kern.hz="500"
> EOF
root@practice:~ # cat /boot/loader.conf.d/techcommune.conf
console="comconsole,vidconsole"
autoboot_delay="3"
loader_menu_title="TechCommune Lab"
kern.hz="500"
root@practice:~ # echo 'kern.ipc.somaxconn=1024' >> /etc/sysctl.conf
root@practice:~ # echo 'vfs.zfs.arc.max=536870912' >> /etc/sysctl.conf
root@practice:~ # grep -v "^#" /etc/sysctl.conf | grep .
vfs.zfs.vdev.min_auto_ashift=12
kern.ipc.somaxconn=1024
vfs.zfs.arc.max=536870912

• bectl create before-tuning: a boot environment as a restore point, for the price of 8K. (The boot files live in the same root dataset as everything else, so the restore point holds the old boot configuration too: our dataset list had no separate /boot.)
• A drop-in file, /boot/loader.conf.d/techcommune.conf, with four boot settings. We will come to them in a moment.
• Two lines appended to /etc/sysctl.conf. (The installer had already put one line in it: vfs.zfs.vdev.min_auto_ashift=12, which tells ZFS how to align disk blocks.)

root@practice:~ # service sysctl restart
root@practice:~ # sysctl kern.ipc.somaxconn vfs.zfs.arc.max kstat.zfs.misc.arcstats.c_max
kern.ipc.somaxconn: 1024
vfs.zfs.arc.max: 536870912
kstat.zfs.misc.arcstats.c_max: 536870912

service sysctl restart re-reads /etc/sysctl.conf without a reboot, so you can test the file straight away: both values are in place, and the ARC's ceiling has followed: c_max is now 536,870,912 (512 MB). One small surprise, seen later on a test machine:

root@practice:~ # sysctl vfs.zfs.arc.max=0
vfs.zfs.arc.max: 536870912 -> 0
root@practice:~ # sysctl kstat.zfs.misc.arcstats.c_max
kstat.zfs.misc.arcstats.c_max: 536870912

setting vfs.zfs.arc.max back to 0 ("decide yourself") did not make the ARC recompute its ceiling: c_max stayed at 512 MB. Some values are only worked out at boot, so after a change like that, a reboot is the way to see the real automatic value.

Boot-time settings: loader.conf.d

The four lines we put in /boot/loader.conf.d/techcommune.conf:

• console="comconsole,vidconsole": send the loader's and the kernel's messages to both the serial port and the screen. A server in a rack usually has no screen; a serial console (or the "serial over LAN" of its management controller) is how you watch it boot.
• autoboot_delay="3": wait 3 seconds at the menu before booting (the default is 10).
• loader_menu_title="TechCommune Lab": the title of the menu, to prove that the file is read.
• kern.hz="500": the kernel clock rate, a boot-only tunable. (The default on this virtual machine is 100.)

We rebooted, with the virtual machine's serial port written to a file, and this is what the machine reported afterwards:

root@practice:~ # uptime
 3:05PM  up 14 secs, 1 user, load averages: 0.37, 0.08, 0.03
root@practice:~ # sysctl kern.hz
kern.hz: 500
root@practice:~ # kenv kern.hz
500
root@practice:~ # sysctl kern.ipc.somaxconn vfs.zfs.arc.max
kern.ipc.somaxconn: 1024
vfs.zfs.arc.max: 536870912
root@practice:~ # sysctl -T kern.hz
kern.hz: 500

uptime says the machine had been up 14 seconds. kern.hz is 500: the loader file was read and the value passed to the kernel (kenv kern.hz shows it in the environment). The two sysctl.conf values are in place, and sysctl -T kern.hz confirms it is a boot tunable.

The loader menu, and the serial console

The captured serial console contained the whole boot, starting with the menu. Redrawn here as plain text (the loader paints it with cursor movements):

TechCommune Lab

  1. Boot Multi user [Enter]
  2. Boot Single user
  3. Escape to loader prompt
  4. Reboot
  5. Cons: ...

  Kernel:   6. kernel (1 of 1)
  Options:  7. Boot Options
            8. Boot Environments

  Autoboot in 3 seconds. [Space] to pause

• Our title is on top: TechCommune Lab. And the countdown began at 3, as we set it.
• 1 boots normally, 2 boots to single-user mode, 3 escapes to the loader prompt, and 4 reboots.
• Under Options: 7. Boot Options (verbose boot, safe mode and so on: from the documentation) and 8. Boot Environments, a menu of the boot environments on the pool, so that you can boot an older one right from the loader. That is the other half of the undo button from the upgrade page: if an update leaves the system unbootable, you choose the old environment here (from the documentation, not run here).
• On the serial console, the kernel's own banner then follows, with the same lines as on a screen: the FreeBSD version, the CPU, the memory (2048 MB), and a line from the loader: Dual Console: Serial Primary, Video Secondary, which is our console= setting at work (it appears in the single-user boot below).

The loader prompt

We caught the menu (a key press pauses the countdown), pressed 3, and arrived at the loader's own prompt, OK. Here you can read and change the kernel environment for this boot only:

OK show autoboot_delay
NO
OK show kern.hz
500
OK show loader_menu_title
TechCommune Lab
OK set kern.hz=250
OK show kern.hz
250
OK show currdev
zfs:zroot/ROOT/default:
OK show kernel
kernel

• show NAME prints a loader variable. autoboot_delay says NO: pausing the countdown told the loader not to continue by itself. kern.hz is 500, from our file, and loader_menu_title is our title. currdev is where the loader will look for the system: zfs:zroot/ROOT/default:, the boot environment.
• set kern.hz=250 changed the value for this boot. Nothing was written to any file.
• Other commands at this prompt: lsdev (list the devices the loader can see), unload and load (choose a different kernel or module), boot -v (a verbose boot) and boot -s (single-user mode). From the documentation, not run here, except boot -s.

Single-user mode

We typed boot -s. The kernel started, found the ZFS root, and stopped before the normal startup:

n acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 16.0.
psm0: model IntelliMouse Explorer, device ID 4
ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
orm0: <ISA Option ROM> at iomem 0xe7800-0xeffff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
attimer0: non-PNP ISA device will be removed from GENERIC in FreeBSD 16.
Timecounters tick every 4.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
usb_needs_explore_all: no devclass
Trying to mount root from zfs:zroot/ROOT/default []...
cd0 at ahcich2 bus 0 scbus2 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device
cd0: Serial Number QM00005
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
Dual Console: Serial Primary, Video Secondary

The last line is the console setting again, and just above it the cd-rom drive from the virtual machine's configuration. The kernel then asked Enter full pathname of shell or RETURN for /bin/sh:. We pressed Enter:

# sysctl kern.hz
kern.hz: 250
root@:~ 
# mount
zroot/ROOT/default on / (zfs, local, noatime, read-only, nfsv4acls)
devfs on /dev (devfs)
root@:~ 
# ps ax | wc -l
      21
root@:~ 
# zpool list -H -o name,health 2>&1
zroot	ONLINE
root@:~ 
# cat /etc/rc.conf | head -3
zfs_enable="YES"
hostname="practice"
ifconfig_vtnet0="DHCP"
root@:~ 

• sysctl kern.hz says 250: the value we set at the loader prompt reached the running kernel, and it beats the 500 in the file. A boot-time override is the quickest way to try a tunable without editing anything.
• mount shows the root file system read-only, so nothing can be broken by accident while you look around. To change files you make it writable first (for ZFS: zfs set readonly=off zroot/ROOT/default, and zfs mount -a: from the documentation, not run here).
• There are only 21 processes: no network, no services, no jails. The machine's name is not even set yet (the prompt reads root@:~). The pool is ONLINE, and /etc/rc.conf is there to be read, and fixed.
• Typing exit leaves single-user mode and continues the boot into multi-user mode:

idge0: Ethernet address: 58:9c:fc:10:bd:50
bridge0: changing name to 'vm-private'
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled
Firewall rules loaded.
Firewall logging enabled.
Creating and/or trimming log files.
Clearing /tmp (X related).
Updating motd:.
Updating /var/run/os-release done.
Starting syslogd.
Mounting late filesystems:.
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Starting background file system checks in 60 seconds.
Starting jails: web.

Tue Oct  6 15:06:33 UTC 2026

FreeBSD/amd64 (practice) (ttyu0)

login:

The jail from the jails page started (Starting jails: web.) and the login prompt appeared on the serial console. After that boot, kern.hz was 250 (and kenv said so), while the file still said 500. The next normal boot used the file, and the value was 500 again.

What single-user mode is for: repairing a system that will not finish booting. A mistake in rc.conf, a broken fstab, a full disk, a forgotten root password. From the documentation, not run here: the root-password reset is zfs set readonly=off zroot/ROOT/default, zfs mount -a, passwd, exit. That also means that anyone with console access can do it, which is why physical and console security matter. The hardening page covers file flags and securelevel, which add protection here. The hardening page covers file flags and securelevel, which add protection here. (FreeBSD can be set to ask for the root password in single-user mode, by marking the console insecure in /etc/ttys: from the documentation, not run here.)

What else belongs where

From the documentation, not run here:

• loader.conf / loader.conf.d: loading modules at boot (vmm_load="YES", as an alternative to kld_list in rc.conf, which we used on the bhyve page), boot_verbose, kern.ipc.nmbclusters and other memory limits, hw. hardware switches, vfs.root.mountfrom.
• sysctl.conf: kern.maxfiles and kern.maxfilesperproc (open files), net.inet.tcp.* (network behaviour), kern.ipc.maxsockbuf, and the ZFS settings vfs.zfs.*.
• rc.conf: which services run, networking, kld_list, jails, and everything else on the other pages.
• Check before you trust a value from a blog post: sysctl -d NAME for the description, and sysctl -W and -T to see how it can be set. A tuning guide for a different release or different hardware may be wrong for yours. Change one thing at a time, measure, and write down what you did.

If a boot setting breaks the boot

• A bad value in a loader file: at the loader menu, press 3 and use unset NAME or set NAME=value to correct it for this boot, then boot, and fix the file afterwards. Or press 8 and boot the previous boot environment, which holds the old configuration.
• Something fails after the kernel starts: boot single-user, make the root writable and fix the file.
• Test boot settings the way we did: with a boot environment as a restore point and a serial console or a virtual machine, never first on a remote production server.

The cheat sheet

• Where settings live: /boot/loader.conf and /boot/loader.conf.d/*.conf (boot time), /etc/sysctl.conf (run time, read at multi-user), /etc/rc.conf (services). Never edit /boot/defaults/loader.conf.
• Look: sysctl NAME, sysctl -d NAME (description), sysctl -W NAME (writable now), sysctl -T NAME (boot tunable), kenv and kenv NAME (the loader's environment).
• Change live: sysctl NAME=VALUE. Make it stay: add NAME=VALUE to /etc/sysctl.conf, then service sysctl restart.
• Boot-only settings: NAME="VALUE" in a file in /boot/loader.conf.d/, then reboot.
• Which environment boots: zpool get bootfs zroot, bectl list, bectl activate NAME.
• At the menu: 1 multi-user, 2 single-user, 3 loader prompt, 4 reboot, 7 boot options, 8 boot environments, Space pauses.
• At the OK prompt: show, set NAME=VALUE, unset NAME, lsdev, boot, boot -s (single-user), boot -v (verbose).
• Single-user: root is read-only (zfs set readonly=off zroot/ROOT/default, zfs mount -a), fix, then exit.
• Serial console: console="comconsole,vidconsole" in a loader file.

← Linux Binary Compatibility: Running Linux Programs on FreeBSD · ↑ FreeBSD Corner · Hardening FreeBSD: A Tested Baseline →