🧊 ZFS in Practice: Pools, Datasets, Snapshots and Self-Healing
Create a pool, set quotas, take snapshots, roll back, clone, copy with send/receive, and watch ZFS repair a damaged disk, all tested on a throwaway pool
Why ZFS gets its own page
On the overview page you met ZFS as the thing underneath FreeBSD's boot environments. It is worth learning for itself, because it replaces several tools you used on Linux (partitioning, LVM, software RAID, the filesystem and part of your backup scheme) with two commands: zpool for the disks and zfs for the data. The ideas you need:
- A pool is built from one or more vdevs (groups of disks, for example a mirror). The pool's job is to hold the data safely.
- Inside a pool you create datasets. A dataset looks like a folder, but it has its own settings (quota, compression, mount point) and its own snapshots. You can make hundreds.
- ZFS is copy-on-write: it never overwrites data in place, and every block carries a checksum. That is what makes snapshots free and lets ZFS notice (and repair) damage.
Practising safely: a pool made of files
You do not need spare disks. ZFS can build a pool out of ordinary files, which is perfect for learning and which we use below. The machine's own system pool, zroot, is never touched. (File-backed pools are for practice only. Real data belongs on real disks.) Everything is run as root on FreeBSD 15.0-RELEASE in a virtual machine.
root@practice:~ # mkdir /var/tmp/zfs-lab root@practice:~ # truncate -s 256m /var/tmp/zfs-lab/disk1 /var/tmp/zfs-lab/disk2 root@practice:~ # ls -lh /var/tmp/zfs-lab total 1.0 KB -rw-r--r-- 1 root wheel 256M Oct 5 22:56 disk1 -rw-r--r-- 1 root wheel 256M Oct 5 22:56 disk2 root@practice:~ # zpool create tank mirror /var/tmp/zfs-lab/disk1 /var/tmp/zfs-lab/disk2
We made two 256 MB files to act as two "disks", and then a mirror, a pool in which each disk holds a full copy of everything:
root@practice:~ # zpool list tank NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT tank 240M 372K 240M - - 4% 0% 1.00x ONLINE - root@practice:~ # zpool status tank pool: tank state: ONLINE config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 0 /var/tmp/zfs-lab/disk2 ONLINE 0 0 0 errors: No known data errors root@practice:~ # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 372K 120M 96K /tank root@practice:~ # df -h /tank Filesystem Size Used Avail Capacity Mounted on tank 120M 96K 120M 0% /tank
- zpool create tank mirror DISK1 DISK2 created the pool, named tank. It printed nothing, which is the Unix way of saying "done".
- zpool list shows the pool: 240 MB, empty, HEALTH ONLINE.
- zpool status is the command you will use most. It draws the layout (the pool, the mirror, the two disks), their states, and three error counters: READ, WRITE and CKSUM (checksum errors). All zeros, and No known data errors.
- The pool mounted itself at /tank, no /etc/fstab entry required. (ZFS also keeps a reserve of free space so a nearly full pool still works. For a toy pool this is a large share, which is why AVAIL shows 120M for a 240M mirror. It is a small percentage on real disks: from the documentation, not run here.)
Datasets, properties and quotas
root@practice:~ # zfs create tank/docs root@practice:~ # zfs create tank/photos root@practice:~ # zfs list -r tank NAME USED AVAIL REFER MOUNTPOINT tank 576K 119M 96K /tank tank/docs 96K 119M 96K /tank/docs tank/photos 96K 119M 96K /tank/photos
zfs create tank/docs made a dataset, and a folder /tank/docs appeared. Each takes only the space its data needs, and all of them share the pool's free space, so there is no partition size to guess in advance.
root@practice:~ # zfs get compression,compressratio,quota tank/docs NAME PROPERTY VALUE SOURCE tank/docs compression on default tank/docs compressratio 1.00x - tank/docs quota none default root@practice:~ # zfs set quota=20m tank/docs root@practice:~ # zfs get quota tank/docs NAME PROPERTY VALUE SOURCE tank/docs quota 20M local root@practice:~ # df -h /tank/docs Filesystem Size Used Avail Capacity Mounted on tank/docs 20M 96K 20M 0% /tank/docs
zfs get reads properties. compression is on by default (the source says default), which means ZFS compresses data as it writes it. quota is none. We then set a quota of 20 MB with zfs set quota=20m tank/docs, the source changed to local (we set it ourselves), and df now shows that dataset as only 20M in size.
What happens if we write too much?
root@practice:~ # dd if=/dev/random of=/tank/docs/big.bin bs=1m count=30 dd: /tank/docs/big.bin: Disc quota exceeded 21+0 records in 20+0 records out 20971520 bytes transferred in 1.406649 secs (14908855 bytes/sec) root@practice:~ # ls -lh /tank/docs total 20 MB -rw-r--r-- 1 root wheel 20M Oct 5 22:57 big.bin root@practice:~ # zfs list -r tank NAME USED AVAIL REFER MOUNTPOINT tank 20.7M 99.3M 104K /tank tank/docs 20.1M 0B 20.1M /tank/docs tank/photos 96K 99.3M 96K /tank/photos root@practice:~ # rm /tank/docs/big.bin
dd copied 20 MB of random data and then stopped with Disc quota exceeded. The dataset was full, while tank/photos still had its 99 MB free: a quota protects the rest of the pool from one runaway dataset. We removed the file again.
Compression, for free
root@practice:~ # yes "TechCommune likes ZFS" | head -c 10m > /tank/photos/log.txt root@practice:~ # sync root@practice:~ # ls -lh /tank/photos/log.txt -rw-r--r-- 1 root wheel 10M Oct 5 22:58 /tank/photos/log.txt root@practice:~ # du -h /tank/photos/log.txt 329K /tank/photos/log.txt root@practice:~ # zfs get compression,compressratio,used,logicalused tank/photos NAME PROPERTY VALUE SOURCE tank/photos compression on default tank/photos compressratio 28.06x - tank/photos used 424K - tank/photos logicalused 10.0M -
We wrote a 10 MB text file full of repeated lines. ls reports its logical size, 10M. But du reports what it uses on disk: 329K. compressratio is 28.06x, and logicalused (10.0M) against used (424K) shows the saving. Real data compresses less than this deliberately repetitive text, but text, logs and databases do compress well, and compression usually makes things faster because less data goes to the disk. (Random data, like the 20 MB we wrote in the quota test above, cannot be compressed at all, and already-compressed files such as JPEG photos save very little.)
Snapshots: the undo button for files
A snapshot is a frozen, read-only picture of a dataset at one moment. It takes no time and no space at first, because ZFS is copy-on-write: it only has to keep the old blocks when the live data changes.
root@practice:~ # echo "version 1" > /tank/docs/report.txt root@practice:~ # echo "keep me" > /tank/docs/notes.txt root@practice:~ # zfs snapshot tank/docs@monday root@practice:~ # zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT tank/docs@monday 0B - 104K - zroot/ROOT/default@2026-10-05-22:14:05-0 4.12M - 853M - zroot/ROOT/default@2026-10-05-22:24:58-0 732K - 1.23G - zroot/ROOT/default@2026-10-05-22:27:47-0 764K - 1.26G - zroot/ROOT/default@2026-10-05-22:31:37-0 1.32M - 1.27G -
We created two files, then zfs snapshot tank/docs@monday. The snapshot name is dataset@name. zfs list -t snapshot lists all snapshots: ours is 0B (nothing has changed since). The other four lines are snapshots made by freebsd-update and bectl during the upgrade page: a boot environment is built from snapshots, so you have been using them already.
Now make a mess, the kind you make by accident:
root@practice:~ # echo "oops, wrong" > /tank/docs/report.txt root@practice:~ # rm /tank/docs/notes.txt root@practice:~ # ls /tank/docs report.txt root@practice:~ # zfs diff tank/docs@monday M /tank/docs/ M /tank/docs/report.txt - /tank/docs/notes.txt
We overwrote report.txt and deleted notes.txt. zfs diff tank/docs@monday compares the snapshot with the live dataset: M is modified, - is removed. Now the rescue:
root@practice:~ # ls /tank/docs/.zfs/snapshot/monday notes.txt report.txt root@practice:~ # cat /tank/docs/.zfs/snapshot/monday/report.txt version 1 root@practice:~ # cp /tank/docs/.zfs/snapshot/monday/notes.txt /tank/docs/ root@practice:~ # ls /tank/docs notes.txt report.txt
Every dataset has a hidden folder, .zfs/snapshot, with one read-only subfolder for each snapshot. We looked at the old files right there, with plain ls and cat, and cp brought back just notes.txt. That is the everyday way to restore one file: copy it out of the snapshot.
And to throw away all changes since the snapshot:
root@practice:~ # zfs rollback tank/docs@monday root@practice:~ # cat /tank/docs/report.txt version 1 root@practice:~ # ls /tank/docs notes.txt report.txt
zfs rollback tank/docs@monday put the whole dataset back as it was: report.txt says version 1 again. Be careful: a rollback destroys everything changed after the snapshot. If there are newer snapshots, ZFS refuses unless you add -r, which destroys them as well (from the documentation, not run here).
Clones and send/receive
A clone is a writable copy of a snapshot that shares all the unchanged blocks, so it costs nothing until you edit it:
root@practice:~ # zfs clone tank/docs@monday tank/docs-test root@practice:~ # echo "experiment" > /tank/docs-test/report.txt root@practice:~ # cat /tank/docs-test/report.txt /tank/docs/report.txt experiment version 1 root@practice:~ # zfs list -r -o name,used,refer,origin tank NAME USED REFER ORIGIN tank 1.05M 104K - tank/docs 160K 104K - tank/docs-test 72K 104K tank/docs@monday tank/photos 96K 96K -
We cloned the snapshot, changed report.txt in the clone only, and the original still says version 1. In zfs list the ORIGIN column shows where the clone came from, and its USED is a mere 72K. Clones are great for testing a change on real data before you do it for real, and then throwing the copy away:
root@practice:~ # zfs destroy tank/docs-test root@practice:~ # zfs send tank/docs@monday | zfs receive tank/docs-copy root@practice:~ # zfs list -r tank NAME USED AVAIL REFER MOUNTPOINT tank 1.11M 119M 104K /tank tank/docs 160K 19.8M 104K /tank/docs tank/docs-copy 104K 119M 104K /tank/docs-copy tank/photos 96K 119M 96K /tank/photos root@practice:~ # cat /tank/docs-copy/report.txt version 1
zfs destroy removed the clone. Next, zfs send turns a snapshot into a stream of data and zfs receive rebuilds it as a new dataset. Here we piped one into the other on the same machine, which made tank/docs-copy with identical content:
root@practice:~ # zfs destroy -r tank/docs-copy root@practice:~ # zfs destroy tank/docs@monday root@practice:~ # zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT zroot/ROOT/default@2026-10-05-22:14:05-0 4.12M - 853M - zroot/ROOT/default@2026-10-05-22:24:58-0 732K - 1.23G - zroot/ROOT/default@2026-10-05-22:27:47-0 764K - 1.26G - zroot/ROOT/default@2026-10-05-22:31:37-0 1.32M - 1.27G -
That is the foundation of ZFS backups. The stream can go anywhere a pipe can go: into a file (zfs send ... > file), to another pool, or to another computer over SSH, and with zfs send -i OLD NEW only the changes between two snapshots are sent, so the second backup is much smaller than the first (both ideas are from the documentation, not run here). We then destroyed the copy and the snapshot to tidy up, and zfs list -t snapshot shows only the system's own snapshots again.
Health: scrub, errors and self-healing
ZFS checksums every block. A scrub reads everything in the pool and checks the checksums, and when it finds a bad block and the pool has a good copy (a mirror, or RAID-Z), it repairs it automatically. To see this for real, we wrote a 40 MB file, noted its fingerprint (sha256), and then deliberately wrecked 150 MB of one disk by writing random data over the middle of its backing file:
root@practice:~ # dd if=/dev/random of=/tank/photos/holiday.bin bs=1m count=40 40+0 records in 40+0 records out 41943040 bytes transferred in 0.444790 secs (94298556 bytes/sec) root@practice:~ # sha256 /tank/photos/holiday.bin SHA256 (/tank/photos/holiday.bin) = 9c05fb6ff771f5063e09bc725726d9db1b1bf2f053fe4b2263e317c0e3b61e17 root@practice:~ # dd if=/dev/random of=/var/tmp/zfs-lab/disk1 bs=1m seek=20 count=150 conv=notrunc 150+0 records in 150+0 records out 157286400 bytes transferred in 0.572895 secs (274546868 bytes/sec) root@practice:~ # zpool status tank pool: tank state: ONLINE config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 0 /var/tmp/zfs-lab/disk2 ONLINE 0 0 0 errors: No known data errors
Look at the first zpool status after the damage: the pool says ONLINE and No known data errors, because ZFS had not looked yet. Corruption on a disk is silent. Then we started the scrub, and:
root@practice:~ # zpool scrub tank root@practice:~ # zpool status tank pool: tank state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P scan: scrub repaired 34.0M in 00:00:00 with 0 errors on Mon Oct 5 23:00:32 2026 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 352 /var/tmp/zfs-lab/disk2 ONLINE 0 0 0 errors: No known data errors root@practice:~ # sha256 /tank/photos/holiday.bin SHA256 (/tank/photos/holiday.bin) = 9c05fb6ff771f5063e09bc725726d9db1b1bf2f053fe4b2263e317c0e3b61e17 root@practice:~ # zpool clear tank root@practice:~ # zpool status tank pool: tank state: ONLINE scan: scrub repaired 34.0M in 00:00:00 with 0 errors on Mon Oct 5 23:00:32 2026 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 0 /var/tmp/zfs-lab/disk2 ONLINE 0 0 0 errors: No known data errors
- scrub repaired 34.0M ... with 0 errors: ZFS found the bad blocks on disk1 and rewrote them from the good copy on disk2.
- The CKSUM column shows 352 checksum errors on disk1. ZFS counts the blocks that failed their checks, which tells you which disk is going bad before you lose anything. The status and action lines in the report tell you what to do.
- And the proof: the file's sha256 is exactly the same as before the damage. The data was never wrong for the reader.
- zpool clear reset the counters, after we had looked at them. (On a real machine, a disk that keeps showing errors should be replaced with zpool replace: from the documentation, not run here.)
Now the other failure: a disk disappearing.
root@practice:~ # zpool offline tank /var/tmp/zfs-lab/disk2 root@practice:~ # zpool status tank pool: tank state: DEGRADED status: One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: scrub repaired 34.0M in 00:00:00 with 0 errors on Mon Oct 5 23:00:32 2026 config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 0 /var/tmp/zfs-lab/disk2 OFFLINE 0 0 0 errors: No known data errors root@practice:~ # echo "written while degraded" > /tank/docs/degraded.txt root@practice:~ # zpool online tank /var/tmp/zfs-lab/disk2 root@practice:~ # zpool status tank pool: tank state: ONLINE scan: resilvered 180K in 00:00:00 with 0 errors on Mon Oct 5 23:01:31 2026 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/zfs-lab/disk1 ONLINE 0 0 0 /var/tmp/zfs-lab/disk2 ONLINE 0 0 0 errors: No known data errors
We took disk2 offline. The pool goes DEGRADED and keeps working on the remaining disk: we wrote a file while it was away. Putting the disk online again triggers a resilver, ZFS copying over what the disk missed: resilvered 180K ... with 0 errors, and the pool is ONLINE once more.
root@practice:~ # zpool history tank | tail -8 2026-10-05.22:59:37 zfs receive tank/docs-copy 2026-10-05.22:59:46 zfs destroy -r tank/docs-copy 2026-10-05.22:59:49 zfs destroy tank/docs@monday 2026-10-05.23:00:32 zpool scrub tank 2026-10-05.23:01:15 zpool clear tank 2026-10-05.23:01:21 zpool offline tank /var/tmp/zfs-lab/disk2 2026-10-05.23:01:31 zpool online tank /var/tmp/zfs-lab/disk2 root@practice:~ # zpool destroy tank root@practice:~ # rm -r /var/tmp/zfs-lab root@practice:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 13.5G 1.83G 11.7G - - 10% 13% 1.00x ONLINE -
zpool history is a log of every ZFS command ever run on the pool, with the time: it is a quiet lifesaver when you ask "who did that, and when?". (zpool destroy and rm at the end removed the practice pool and its two files: the output shows just zroot left.)
⚠️ Real data, real care: zpool destroy and zfs destroy -r delete without asking, and a mirror is not a backup: if you delete a file, both disks delete it. Snapshots help against mistakes, and zfs send to another machine is the backup. And never type disk names from memory on a real server: check them with zpool status first.
We both keep checking everything twice,<br>
and neither of us trusts the hardware.
🧪 Try it yourself
Build your own practice pool (no spare disk needed):
1. Make two 256 MB files with truncate -s 256m in /var/tmp, and a mirror called practice from them. 2. Create a dataset practice/notes, set a quota of 10 MB and write a file bigger than that. 3. Take a snapshot, delete a file, then restore it from .zfs/snapshot. 4. Make a clone of the snapshot, change a file in the clone, and compare it with the original. 5. Use zfs send | zfs receive to make a second dataset, then compare the contents. 6. Take one "disk" offline, check the state in zpool status, then bring it back and watch the resilver. 7. Remove the pool with zpool destroy practice and delete the files.
The cheat sheet
- Pools: zpool create NAME mirror DISK1 DISK2, zpool list, zpool status, zpool history NAME, zpool destroy NAME (careful).
- Datasets: zfs create POOL/NAME, zfs list -r POOL, zfs destroy POOL/NAME.
- Properties: zfs get PROP DATASET, zfs set quota=20m DATASET, zfs set compression=lz4 DATASET.
- Snapshots: zfs snapshot DATASET@NAME, zfs list -t snapshot, zfs diff DATASET@NAME, cp from /MOUNT/.zfs/snapshot/NAME/, zfs rollback DATASET@NAME, zfs destroy DATASET@NAME.
- Clones and copies: zfs clone DATASET@NAME NEW, zfs send DATASET@NAME | zfs receive NEW.
- Health: zpool scrub POOL, zpool status, zpool offline|online POOL DISK, zpool clear POOL, zpool replace POOL OLD NEW.
← Upgrading FreeBSD Safely: freebsd-update, pkg and Boot Environments · ↑ FreeBSD Corner · Jails: Lightweight Containers, the BSD Way →