A how-to manual: zones, services, runtime vs permanent, rich rules, ipsets, port forwarding and policies, logging, panic mode and how not to lock yourself out, all tested on a lab
▶Prefer to watch? This video runs the guide's main commands on a real Ubuntu 24.04 machine, with on-screen captions and no voiceover. Click to play, or watch it on YouTube. It is part of the Networking and Firewalls playlist.
firewalld: zones, services and rich rules
firewall-cmd --list-all
firewalld is the firewall manager that ships with RHEL, Rocky, Alma, Fedora and CentOS Stream, and is available on Debian and Ubuntu. It is a background service (firewalld) that you talk to with firewall-cmd, and it builds the real rules for the kernel's packet filter, nftables, so you do not have to. Instead of ordered lists of rules, you think in zones: "this network card is the untrusted internet, that one is the office, and the office may do these things". It can change the rules while the machine is running without dropping connections, which is why it exists. It is also the tool that locks people out of their own servers with a single forgotten word: --permanent. This guide is how not to be that person.
Every example below ran for real: firewalld 1.3.4 on Rocky Linux 9 in a lab with two virtual clients, one on an "internal" LAN (192.168.50.0/24, interface eth1) and one "outside" (203.0.113.0/24, interface eth2), and real listeners on ports 22, 80, 443, 3306, 8080. "probe" in the examples is a tiny test script: it connects from a client and prints what that client actually experienced (open, refused, or silence). The lab ran as root without systemd, so firewalld was started by hand; on a real server it is a service and you use sudo. Anything not run in the lab is marked "not run in the lab".
What firewalld is, and how it relates to nftables, iptables and ufw
The kernel filters packets with nftables (the successor of iptables). Several tools write nftables rules for you, and you pick ONE per machine:
firewalld zones and named services, a daemon, changes at runtime. Default on the Red Hat family.
ufw "uncomplicated firewall", simpler and flat (no zones). Default on Ubuntu.
nft / iptables the rules themselves, written by hand or by scripts.
Running two managers at once (firewalld and ufw, or firewalld and a hand-written nft script) ends with each one wiping or bypassing the other's rules. Install one, disable the others. On the Red Hat family, firewalld's backend is nftables, as the config file says:
$ grep FirewallBackend /etc/firewalld/firewalld.conf
FirewallBackend=nftables
Container engines (Docker, Podman) add their own rules and zones, and the way they cooperate with firewalld has changed between versions: if you run containers, test that published ports and your zone rules agree (not run in the lab).
Install, start and check
sudo dnf install firewalld RHEL family (usually already installed)
sudo apt install firewalld Debian, Ubuntu (disable ufw first: sudo ufw disable)
sudo systemctl enable --now firewalld start now and at every boot (not run in the lab: it had no systemd)
$ firewall-cmd --state
running
$ firewall-cmd --version
1.3.4
When the daemon is not running, firewall-cmd says so, and the exit status is not zero (useful in scripts):
$ firewall-cmd --state
not running
$ firewall-cmd --add-service=http
FirewallD is not running
$ echo $?
252
Remember what that means: firewalld is the thing that holds the rules. Stop it and the rules go with it. With the daemon stopped, the lab showed ZERO nftables tables: no firewall at all, not "the last settings stay". systemctl stop firewalld on a machine with a public address is an invitation.
The model: zones, interfaces, sources, services and targets
Think of a zone as a trust level with a rule list. Every incoming packet is put into exactly one zone, and that zone's rules decide its fate.
zone a named set of rules: which services and ports are open, plus a default fate for everything else.
service a named bundle of ports ("http" is 80/tcp, "ssh" is 22/tcp), so you say what you mean and not the number.
target what happens to traffic that no rule in the zone matched: the default (reject, with a polite ICMP message), DROP (silence), or ACCEPT (everything allowed).
interface a network card can be bound to a zone: everything arriving on it lands in that zone.
source a source address or network can be bound to a zone too: everything FROM it lands in that zone.
default zone the zone for anything not bound anywhere: the "everyone else" bucket (public, out of the box).
The order in which firewalld decides a packet's zone matters, and the lab confirmed it: a SOURCE match beats an INTERFACE match, which beats the default zone. So the office laptop's address can be given more trust than the LAN card it arrives on, and the rest of the LAN stays as it was.
A first look: the default zones
$ firewall-cmd --get-default-zone
public
$ firewall-cmd --get-zones
block dmz drop external home internal public trusted work
$ firewall-cmd --list-all
public
target: default
icmp-block-inversion: no
interfaces:
sources:
services: cockpit dhcpv6-client ssh
ports:
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Nine zones come predefined. This is what each one actually contains, read from the running firewalld:
drop target DROP nothing allowed, silently dropped (the sender gets no answer)
block target reject nothing allowed, rejected with a message
public services cockpit dhcpv6-client ssh the default, for networks you do not trust
external services ssh for the outside of a router (masquerading is usually switched on here)
dmz services ssh for servers that face the internet but are kept apart from the LAN
work services cockpit dhcpv6-client ssh
home services cockpit dhcpv6-client mdns samba-client ssh
internal services cockpit dhcpv6-client mdns samba-client ssh
trusted target ACCEPT everything allowed. Never put an interface here that faces anyone you do not trust.
Read the first list line again: public already allows "cockpit", the web console on port 9090, on a fresh install of the Red Hat family. If you do not use Cockpit, remove it: firewall-cmd --permanent --remove-service=cockpit, then reload. A default firewall that opens something you did not ask for is still a firewall that opens something.
What a client sees when it asks for a port that is NOT open, the standard (reject) behavior, with port 22 allowed and 80 not:
$ probe c1 22 80
c1 -> 192.168.50.1 port 22 open
c1 -> 192.168.50.1 port 80 No route to host
"No route to host" is not a routing problem. It is firewalld's ICMP "host prohibited" reject, and it is the signature of "a firewall said no". "Connection refused" would mean the machine is reachable and nothing listens on that port: two different messages, two different problems, and a handy first diagnosis.
Runtime and permanent: the one rule to learn
firewalld keeps TWO copies of the configuration: the RUNTIME one (what the kernel enforces right now) and the PERMANENT one (what is loaded at boot and at --reload). A command without --permanent changes only the runtime copy, and it is gone at the next reload or reboot. A command WITH --permanent changes only the stored copy, and nothing happens until the reload. Shown as it happened:
$ firewall-cmd --add-service=http # runtime only
success
$ firewall-cmd --list-services
cockpit dhcpv6-client http ssh
$ probe c1 80
c1 -> 192.168.50.1 port 80 open
$ firewall-cmd --reload # reload = throw the runtime changes away, load the permanent copy
success
$ firewall-cmd --list-services
cockpit dhcpv6-client ssh
$ probe c1 80
c1 -> 192.168.50.1 port 80 No route to host
Now the other way round:
$ firewall-cmd --permanent --add-service=http # stored, but NOT live
success
$ firewall-cmd --list-services
cockpit dhcpv6-client ssh
$ firewall-cmd --permanent --list-services
cockpit dhcpv6-client http ssh
$ probe c1 80
c1 -> 192.168.50.1 port 80 No route to host
$ firewall-cmd --reload
success
$ probe c1 80
c1 -> 192.168.50.1 port 80 open
So there are two good habits. One: make the change at runtime, test it, and when it is right save it with --runtime-to-permanent. Two: for scripts, write with --permanent and finish with --reload.
$ firewall-cmd --add-port=8080/tcp
success
$ firewall-cmd --permanent --list-ports
$ firewall-cmd --runtime-to-permanent
success
$ firewall-cmd --permanent --list-ports
8080/tcp
Careful: --runtime-to-permanent saves EVERYTHING that is different, including the experiment you forgot you made an hour ago. Run firewall-cmd --list-all first and read it.
The same split gives you a safety net, a rule that removes itself. --timeout makes a runtime change temporary:
$ firewall-cmd --add-service=https --timeout=5s
success
$ firewall-cmd --list-services
cockpit dhcpv6-client http https ssh
(six seconds later)
$ firewall-cmd --list-services
cockpit dhcpv6-client http ssh
Timeouts accept s, m and h (5m, 1h). They only work at runtime, and they are the right way to open a port "just for the next ten minutes".
Services and ports
A service is the better way to open something: a name that says what it is, and that follows along if the port list ever changes. The list ships with the package and is large:
$ firewall-cmd --get-services | wc -w
(over 200 names: ssh, http, https, dns, smtp, imaps, mysql, postgresql, nfs, samba, ntp, ...)
$ firewall-cmd --info-service=http
http
ports: 80/tcp
protocols:
source-ports:
modules:
destination:
includes:
helpers:
Everyday use, in the default zone (add --zone=NAME for another one, and --permanent for the stored copy):
firewall-cmd --add-service=http open a service
firewall-cmd --remove-service=http close it
firewall-cmd --list-services what is open as a service
firewall-cmd --add-port=8080/tcp a port that has no service name
firewall-cmd --add-port=5000-5010/udp a range
firewall-cmd --remove-port=8080/tcp
firewall-cmd --list-ports
firewall-cmd --query-service=http "yes" or "no", for scripts
Ports need the protocol and the check is strict. What firewall-cmd says to the usual mistakes:
$ firewall-cmd --add-service=htpp
Error: INVALID_SERVICE: htpp
$ firewall-cmd --add-port=8080
Error: INVALID_PORT: bad port (most likely missing protocol), correct syntax is portid[-portid]/protocol
$ firewall-cmd --add-port=99999/tcp
Error: INVALID_PORT: 99999
$ firewall-cmd --zone=nonsuch --add-service=http
Error: INVALID_ZONE: nonsuch
$ firewall-cmd --add-service=http # when it is already there
Warning: ALREADY_ENABLED: 'http' already in 'public'
success
$ firewall-cmd --permanent --remove-service=https # when it was never there
Warning: NOT_ENABLED: https
success
Warnings are not failures (the command still says success and exits 0). Errors are. In a script, trust the exit status, and read the words.
Zones in practice: interfaces, sources, targets and the default zone
Put each network card in the zone that matches the trust level of what is behind it. In the lab, eth1 is the LAN and eth2 is the outside:
$ firewall-cmd --zone=internal --change-interface=eth1
success
$ firewall-cmd --zone=external --change-interface=eth2
success
$ firewall-cmd --get-active-zones
external
interfaces: eth2
internal
interfaces: eth1
$ firewall-cmd --get-zone-of-interface=eth1
internal
(An interface you never assigned shows "no zone" with --get-zone-of-interface. It is not unprotected: it falls into the default zone.) Like everything else, this is runtime only until you repeat it with --permanent (or --runtime-to-permanent). On a machine where NetworkManager manages the cards, the zone also belongs to the connection (nmcli connection modify NAME connection.zone internal) and NetworkManager will put the interface back where it thinks it belongs: set it there. (Not run in the lab: it had no NetworkManager.)
A zone can also be bound to a SOURCE. Everything from that address uses that zone, whatever interface it arrives on. In the lab, the LAN client 192.168.50.20 is given the trusted zone while the rest of the LAN stays in internal:
$ firewall-cmd --zone=trusted --add-source=192.168.50.20
success
$ firewall-cmd --get-active-zones
external
interfaces: eth2
internal
interfaces: eth1
trusted
sources: 192.168.50.20
$ firewall-cmd --get-zone-of-source=192.168.50.20
trusted
$ probe c1 3306 8080
c1 -> 192.168.50.1 port 3306 open
c1 -> 192.168.50.1 port 8080 open
$ firewall-cmd --zone=trusted --remove-source=192.168.50.20
$ probe c1 3306
c1 -> 192.168.50.1 port 3306 No route to host
Sources can be single addresses or networks (192.168.50.0/24). That is how "my office network may reach the admin ports and the internet may not" is written in one line.
THE TARGET: the default zone behavior for everything no rule matched. drop and block are zones whose whole point is the target. What the client sees, same port, same client:
zone "drop" $ probe c2 22 c2 -> 203.0.113.1 port 22 no answer (timeout)
zone "block" $ probe c2 22 c2 -> 203.0.113.1 port 22 No route to host
Drop is silent: the sender waits for its timeout and cannot even tell the machine exists. It slows down scanners. It also slows down YOU, because a mistake now looks like a hang rather than an error: when you debug, switch to reject first. (Do not overrate drop as protection: the packets are blocked either way.) The per-zone target is set like this, and note the catch, which the lab reported literally:
$ firewall-cmd --zone=external --set-target=DROP
usage: 'firewall-cmd --help' for usage information or see firewall-cmd(1) man page
Option can be used only with --permanent.
$ firewall-cmd --permanent --zone=external --set-target=DROP && firewall-cmd --reload
Values: default (the standard reject), %%REJECT%%, DROP and ACCEPT. Use the word ACCEPT with great care.
The default zone catches everything that has no interface or source of its own:
$ firewall-cmd --get-default-zone
public
$ firewall-cmd --set-default-zone=internal # takes effect at once AND is stored: it needs no --permanent
success
Think twice before changing it: a new interface, a VPN or a USB network card appears and silently lands in whichever zone is the default. A cautious default is a good default.
Rich rules: one host, logging, rate limits
Services and ports apply to everybody who reaches the zone. A rich rule says who, what and what then. The classic case: MySQL (port 3306) must be reachable by one application server and nobody else. In the lab the LAN client has two addresses, .20 (the app server) and .21 (everyone else on the LAN):
$ from 192.168.50.20 3306 ; from 192.168.50.21 3306 # before
from 192.168.50.20 port 3306 No route to host.
from 192.168.50.21 port 3306 No route to host.
$ firewall-cmd --zone=internal --add-rich-rule='rule family="ipv4" source address="192.168.50.20" port port="3306" protocol="tcp" accept'
success
$ firewall-cmd --zone=internal --list-rich-rules
rule family="ipv4" source address="192.168.50.20" port port="3306" protocol="tcp" accept
$ from 192.168.50.20 3306 ; from 192.168.50.21 3306 # after
from 192.168.50.20 port 3306 open
from 192.168.50.21 port 3306 No route to host.
The words of a rich rule, in order: rule, family (ipv4 or ipv6; leave it out and the rule must contain no address), source or destination address, then what it matches (service name=, port port= protocol=, protocol value=), then an optional log, an optional limit, and finally the verdict: accept, reject, drop or mark. Quote the whole rule in single quotes and every value in double quotes, or the shell will eat them.
A rule that rejects, writes a log line (at most three a minute, so an attack cannot fill the disk) and carries a log prefix to search for:
$ firewall-cmd --zone=external --add-rich-rule='rule family="ipv4" source address="203.0.113.50" port port="22" protocol="tcp" log prefix="SSH-DENY " level="warning" limit value="3/m" reject'
success
$ probe c2 22
c2 -> 203.0.113.1 port 22 Connection refused
(the logged packets show up in the kernel log, journalctl -k | grep SSH-DENY; not shown here because a container cannot read the host's kernel log.) Notice the other difference: a rich reject answers with "Connection refused", not "No route to host". The zone's default reject sends ICMP host-prohibited; the rich-rule reject gave a different message. Both are polite; the wording differs.
A rich rule that rejects or drops wins over the plain service that would let the same traffic in: port 22 was open through the ssh service, and the reject above (and the ipset drop below) still shut it for the one source. Remove one with --remove-rich-rule and the exact same text, so keep the text in a file. A limit can also go on an accept rule, for example rule service name="ssh" limit value="10/m" accept, to cap how often it matches and slow a brute-force attempt without blocking real users. (Not run in the lab: the limit on an accept rule.)
Blocking ICMP and blocking whole lists (ipsets)
ICMP types can be blocked per zone. Ping (echo-request) is the one people ask about:
$ ip netns exec c1 ping -c1 192.168.50.1 | grep received # from the LAN client
1 packets transmitted, 1 received, 0% packet loss, time 0ms
$ firewall-cmd --zone=internal --add-icmp-block=echo-request
success
$ ip netns exec c1 ping -c1 192.168.50.1 | grep received
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
$ firewall-cmd --zone=internal --list-icmp-blocks
echo-request
$ firewall-cmd --get-icmptypes | tr ' ' '\n' | head -5
address-unreachable
bad-header
beyond-scope
communication-prohibited
destination-unreachable
Blocking ping is mostly decoration and it makes diagnosing real problems harder. Do NOT block other ICMP types for the fun of it: "fragmentation-needed" and "packet-too-big" are how path MTU discovery works, and blocking them gives you the famous "small things work, big things hang".
For a long list of addresses to block or allow (a blocklist, a partner's networks) use an ipset instead of a hundred rich rules. An ipset must be created PERMANENT and then loaded with a reload; it cannot be created at runtime:
$ firewall-cmd --permanent --new-ipset=blocklist --type=hash:net
success
$ firewall-cmd --reload
success
$ firewall-cmd --permanent --ipset=blocklist --add-entry=203.0.113.0/24
success
$ firewall-cmd --permanent --ipset=blocklist --get-entries
203.0.113.0/24
$ firewall-cmd --permanent --zone=external --add-rich-rule='rule source ipset="blocklist" drop'
success
$ firewall-cmd --reload
success
$ probe c2 22
c2 -> 203.0.113.1 port 22 no answer (timeout)
Remove it again (the same rich rule with --remove-rich-rule, then --delete-ipset=blocklist, then reload) and the client gets in. Entries can be added and removed from an ipset without touching the rules that use it, which is the point: updating a list of ten thousand addresses is one command.
Port forwarding, masquerade and policies
A forward port redirects a port on this machine to another port (or another machine). Here, port 8081 on the LAN side is sent to port 80 on this host:
$ firewall-cmd --zone=internal --add-forward-port=port=8081:proto=tcp:toport=80
success
$ firewall-cmd --zone=internal --list-forward-ports
port=8081:proto=tcp:toport=80:toaddr=
$ probe c1 8081 80
c1 -> 192.168.50.1 port 8081 open
c1 -> 192.168.50.1 port 80 No route to host
Look at the second line: port 80 itself is still closed, and the redirect works anyway. A forward is a separate door. To send traffic to ANOTHER machine, add toaddr=10.0.0.5 and make sure the host forwards packets and masquerades (below).
A router (a firewall in the middle of two networks) needs three things: the kernel must forward packets (net.ipv4.ip_forward=1), firewalld must allow traffic from one zone to the other, and the outside must see the router's address (masquerade, also called source NAT). In firewalld 1.x the "allow traffic from zone A to zone B" part is a POLICY. Without one, the zones are sealed from each other. In the lab the LAN client tries to reach a server on the outside network through the firewall:
$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
$ ip netns exec c1 bash -c 'exec 3<>/dev/tcp/203.0.113.50/9000 && head -c 40 <&3' # from the LAN client, no policy yet
bash: connect: No route to host
$ firewall-cmd --get-policies
allow-host-ipv6
Only a built-in policy exists. Now a policy of our own, from internal to external, that accepts and masquerades:
$ firewall-cmd --permanent --new-policy=lan-to-wan
$ firewall-cmd --permanent --policy=lan-to-wan --add-ingress-zone=internal
$ firewall-cmd --permanent --policy=lan-to-wan --add-egress-zone=external
$ firewall-cmd --permanent --policy=lan-to-wan --set-target=ACCEPT
$ firewall-cmd --permanent --policy=lan-to-wan --add-masquerade
$ firewall-cmd --reload
$ firewall-cmd --info-policy=lan-to-wan
lan-to-wan (active)
priority: -1
target: ACCEPT
ingress-zones: internal
egress-zones: external
services:
ports:
protocols:
masquerade: yes
forward-ports:
source-ports:
icmp-blocks:
rich rules:
$ ip netns exec c1 bash -c 'exec 3<>/dev/tcp/203.0.113.50/9000 && head -c 40 <&3'
you are 203.0.113.1
The last line is the server on the outside, reporting who it thinks connected: 203.0.113.1 is the FIREWALL's outside address, not the LAN client's 192.168.50.20. That is masquerade at work. The built-in policy allow-host-ipv6 is the reason IPv6 neighbor discovery works on a default install; leave it alone. Make the policy as narrow as the job: ingress and egress zones named, and a target of ACCEPT only on the direction you want. (Older guides use "firewall-cmd --zone=external --add-masquerade" for the same purpose; that still works on the zone, and on 1.x the policy is the cleaner way to say what may cross.)
Your own services
When an in-house application listens on its own ports, make it a service instead of a loose list of ports. A service is a small XML file in /etc/firewalld/services/, with the name of the file as the name of the service:
<?xml version="1.0" encoding="utf-8"?>
<service>
<short>My App</short>
<description>The in-house web app: HTTPS on 8443 and a metrics port.</description>
<port protocol="tcp" port="8443"/>
<port protocol="tcp" port="9100"/>
</service>
firewalld does not see a new file until it reloads:
$ firewall-cmd --get-services | tr ' ' '\n' | grep -c myapp
0
$ firewall-cmd --reload
success
$ firewall-cmd --info-service=myapp
myapp
ports: 8443/tcp 9100/tcp
protocols:
source-ports:
modules:
destination:
includes:
helpers:
$ firewall-cmd --permanent --zone=internal --add-service=myapp && firewall-cmd --reload
$ probe c1 8443
c1 -> 192.168.50.1 port 8443 open
Now the zone says "services: ... myapp", so anyone reading the firewall sees what the open port is FOR. A file with a mistake is not loaded, and the reload tells you, in the same words as the checks above:
$ printf '<service><port protocol="tcp" port="99999"/></service>\n' > /etc/firewalld/services/bad.xml
$ firewall-cmd --reload
Error: INVALID_PORT: port number in '99999' is too big
$ firewall-cmd --check-config
Error: INVALID_PORT: port number in '99999' is too big
--check-config validates the stored configuration without touching the running one. Run it before every reload on a machine that matters. A service that talks over many ports, or needs a connection tracking helper (old FTP, SIP), can say so in the same XML (the helper and module elements); copy one from /usr/lib/firewalld/services/ and change it, instead of writing one from scratch. Files in /usr/lib/firewalld are the vendor's defaults and are replaced when firewalld is updated; yours go in /etc/firewalld, which always wins.
Logging denied packets
Rejected and dropped packets are silent by default. To see what the firewall is turning away (a scan, a service you forgot to open, an application that talks to something it should not), turn on the logging of denied packets:
$ firewall-cmd --get-log-denied
off
$ firewall-cmd --set-log-denied=all
success
$ firewall-cmd --get-log-denied
all
Values: all, unicast, broadcast, multicast, off. It adds log rules at the end of the chains, which the lab confirmed in nftables:
$ nft list ruleset | grep -i "log prefix"
meta nfproto ipv6 fib saddr . mark . iif check missing log prefix "rpfilter_DROP: " drop
ct state invalid log prefix "STATE_INVALID_DROP: "
log prefix "FINAL_REJECT: "
The messages land in the kernel log with the prefix FINAL_REJECT: (journalctl -k | grep FINAL_REJECT, or /var/log/messages; the lab could not display them, because a container cannot read the host's kernel log). Each line has the interface, source and destination addresses and ports, so "who is knocking on what" is answered by one grep. On a machine on the internet this is a lot of noise: switch it on to find a problem, then off again with --set-log-denied=off.
Panic mode
Panic mode drops EVERYTHING, in and out, including connections that are already open. It is the "pull the network cable" button for when a machine is compromised and you still have a console:
$ probe c1 22
c1 -> 192.168.50.1 port 22 open
$ firewall-cmd --panic-on
success
$ firewall-cmd --query-panic
yes
$ probe c1 22
c1 -> 192.168.50.1 port 22 no answer (timeout)
$ firewall-cmd --panic-off
success
$ firewall-cmd --query-panic
no
$ probe c1 22
c1 -> 192.168.50.1 port 22 open
Do NOT turn it on over ssh. Your own session dies with everything else, and it stays that way until someone with a console says --panic-off.
What firewalld really built, and where it keeps its files
firewalld is a front end; the real rules are in nftables, in a table of its own, and you can read them (read only: never edit them, firewalld owns that table and rewrites it at will):
$ nft list tables
table inet firewalld
$ nft list ruleset | wc -l
670
$ nft list chain inet firewalld filter_IN_internal_allow
table inet firewalld {
chain filter_IN_internal_allow {
tcp dport 22 accept
ip daddr 224.0.0.251 udp dport 5353 accept
ip6 daddr ff02::fb udp dport 5353 accept
udp dport 137 ct helper set "helper-netbios-ns-udp"
udp dport 137 accept
udp dport 138 accept
ip6 daddr fe80::/64 udp dport 546 accept
tcp dport 9090 accept
tcp dport 8443 accept
tcp dport 9100 accept
}
}
One chain per zone and direction (filter_IN_ZONE_allow holds the plain services), and every line is what a service meant: ssh is "tcp dport 22 accept", mdns is the multicast lines, cockpit is 9090, myapp is 8443 and 9100. Reading this chain is the honest answer to "what does this zone really allow". (670 lines for a handful of zones, which is why nobody writes these by hand.)
The files:
$ ls /etc/firewalld
firewalld.conf firewalld.conf.old helpers icmptypes ipsets lockdown-whitelist.xml policies services zones
$ ls /etc/firewalld/zones
block.xml dmz.xml drop.xml external.xml external.xml.old home.xml internal.xml internal.xml.old public.xml public.xml.old trusted.xml work.xml
$ cat /etc/firewalld/zones/internal.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
<short>Internal</short>
<description>For use on internal networks. You mostly trust the other computers on the networks to not harm your computer. Only selected incoming connections are accepted.</description>
<service name="ssh"/>
<service name="mdns"/>
<service name="samba-client"/>
<service name="dhcpv6-client"/>
<service name="cockpit"/>
<service name="myapp"/>
<interface name="eth1"/>
<forward/>
</zone>
That file IS the permanent configuration of the zone, plain XML, which makes it easy to review, to copy to the next server and to keep in git (/etc/firewalld as a whole, with services, ipsets and policies). The .old files are the previous version that firewalld keeps each time it writes: handy for a quick undo, and harmless to ignore. The main settings are in firewalld.conf:
$ grep -E "DefaultZone|FirewallBackend|LogDenied|Lockdown" /etc/firewalld/firewalld.conf
DefaultZone=public
# Lockdown
Lockdown=no
# LogDenied
LogDenied=off
FirewallBackend=nftables
(Lockdown is a feature that lets only a whitelist of programs change the firewall through D-Bus; not run in the lab.)
Working with the daemon stopped: firewall-offline-cmd
When firewalld is not running, firewall-cmd cannot do anything, not even --permanent:
$ firewall-cmd --permanent --add-service=http
FirewallD is not running
firewall-offline-cmd edits the stored configuration directly, without the daemon. It is for provisioning scripts, kickstarts and images, and for fixing a configuration on a machine whose firewalld would not start:
$ firewall-offline-cmd --zone=public --add-service=https
success
$ firewall-offline-cmd --zone=public --list-services
cockpit dhcpv6-client http https ssh
$ firewall-offline-cmd --get-default-zone
public
The change lands in the zone file and is loaded when firewalld next starts, which the lab confirmed: after starting the daemon, https was in the running zone. Use it ONLY while the daemon is stopped: while it runs, the running daemon owns the files.
Changing a firewall without locking yourself out
You are connected over ssh, and you are about to edit the thing that decides whether ssh works. The habits that keep you in:
Change RUNTIME first, without --permanent. A mistake costs a reload (or a reboot), which restores the stored version. A permanent mistake survives the reboot you were counting on to fix it.
Test from a SECOND session before closing the first one. Open a new ssh connection. If it works, the change is safe to keep.
Use --timeout for anything risky: firewall-cmd --add-port=8080/tcp --timeout=10m. It removes itself, even if you forget.
Allow ssh BEFORE you do anything else, and never put a --zone=drop or a changed default zone on the interface you are connected through without opening ssh in it first.
Keep a console in reach (the hypervisor window, the cloud provider's serial console, the iDRAC): it is the only fix for the one mistake you did not think of.
A cron or at job that runs firewall-cmd --reload in ten minutes is a belt and braces for a runtime experiment: a bad runtime state is wiped by it. (at now + 10 minutes; not run in the lab.)
Save with --runtime-to-permanent only when it works, and read firewall-cmd --list-all before you do.
Run firewall-cmd --check-config before a --reload that follows edited files.
Cheat sheet
firewall-cmd --state is the daemon running
firewall-cmd --reload load the stored copy, discard runtime-only changes
firewall-cmd --runtime-to-permanent save the live state
firewall-cmd --get-default-zone / --set-default-zone=ZONE
firewall-cmd --get-active-zones zones in use, and what is bound to them
firewall-cmd --list-all [--zone=ZONE] everything about a zone
firewall-cmd --list-all-zones every zone (long)
firewall-cmd --zone=Z --change-interface=IF move an interface to a zone (--add-interface to add)
firewall-cmd --zone=Z --add-source=NET bind a network to a zone
firewall-cmd --zone=Z --add-service=S / --remove-service=S
firewall-cmd --zone=Z --add-port=P/tcp / --remove-port=P/tcp
firewall-cmd --zone=Z --add-rich-rule='RULE' / --remove-rich-rule='RULE' / --list-rich-rules
firewall-cmd --zone=Z --add-forward-port=port=P:proto=tcp:toport=Q[:toaddr=IP]
firewall-cmd --zone=Z --add-icmp-block=echo-request
firewall-cmd --permanent --new-ipset=NAME --type=hash:net then --reload, then --ipset=NAME --add-entry=NET
firewall-cmd --permanent --new-policy=NAME zone to zone rules (ingress/egress zones, target, masquerade)
firewall-cmd --permanent --zone=Z --set-target=DROP the zone's default fate (permanent only)
firewall-cmd --info-service=S / --get-services what a service means
firewall-cmd --set-log-denied=all|off / --get-log-denied
firewall-cmd --panic-on / --panic-off / --query-panic
firewall-cmd --check-config validate the stored files
firewall-cmd --permanent ... followed by --reload the scripting pattern
firewall-offline-cmd ... the same, with the daemon stopped
Common problems
I opened the port and it is closed again after a reboot the change was runtime only: use --permanent and --reload, or --runtime-to-permanent
I added --permanent and nothing changed permanent changes are not live until --reload. The first list shows the runtime state, the --permanent list shows the stored one: compare them
The port is open in firewalld but the connection still fails nothing is listening (ss -ltnp shows the real listeners), the application listens on 127.0.0.1 only, the cloud provider's security group blocks it, or it is in a different zone than the interface. "Connection refused" = firewall passed it, nothing listens. "No route to host" = the firewall said no
"No route to host" for a machine on the same network probably not routing: firewalld rejected it with ICMP host-prohibited. Check zone and port: firewall-cmd --get-zone-of-interface=IF; --list-all --zone=ZONE
Connections hang with no error a drop target or a drop rule: silent by design. Switch the target to reject while you debug, then back
Error: INVALID_SERVICE / INVALID_PORT / INVALID_ZONE a typo or a missing protocol (8080/tcp, not 8080); the name after the colon is what it did not recognize
Error: ZONE_CONFLICT: 'eth1' already bound to 'internal' the interface (or source) is already in another zone: use --change-interface, or remove it from the other zone first
Option can be used only with --permanent --set-target and a few others have no runtime form: add --permanent and --reload
FirewallD is not running systemctl status firewalld, journalctl -u firewalld. Use firewall-offline-cmd until it runs again. Remember there is NO firewall in the meantime
A new custom service is not found firewalld only reads /etc/firewalld/services at reload: firewall-cmd --reload. Check the file name ends with .xml and --check-config
A reload fails after editing XML --check-config prints the offending value; fix it, or remove the file you just added
The interface lands in the wrong zone after a reboot NetworkManager owns the zone setting: nmcli connection modify NAME connection.zone ZONE (not run in the lab)
Forwarding or masquerade does not work three things have to be true: net.ipv4.ip_forward=1 (and stored in /etc/sysctl.d), a policy allowing the two zones, and masquerade on the outgoing side. Test one at a time
Traffic between two zones is blocked "for no reason" it is the design: zones are sealed from each other until a policy joins them
Docker containers lose published ports after a reload the container engine's rules are not part of firewalld's configuration and have to be re-created: restart the engine, and test the combination (not run in the lab)
I locked myself out console: firewall-cmd --reload (restores the stored state), or firewall-offline-cmd to fix the stored one, and if all else fails systemctl stop firewalld. Then find the real cause
Two firewalls fighting (ufw, iptables-services, scripts) pick one. systemctl disable --now the others (ufw disable); a pile of leftovers is the usual reason rules "do not stick"
Tips
Name the zone in every command you script (--zone=internal) and do not rely on the default zone.
Change runtime, test from a second session, then save. For anything remote and risky, add --timeout.
Prefer services to bare ports, and give your own applications a service file. The zone listing then explains itself.
Keep the zones narrow: allow what is needed on the interface that faces the people who need it, not "ssh everywhere".
Bind trusted people by SOURCE (an office network, a monitoring host), not by loosening the zone for everyone.
Read --list-all on every machine you inherit. The default zone and the "cockpit" line are the two surprises most often found.
Run --check-config before reloading after hand edits, and keep /etc/firewalld in git.
Turn on --set-log-denied=all when something does not connect, read the lines, and turn it off again.
Test the firewall from outside: nmap -Pn from another machine (see the nmap page) shows what the world sees, not what you think you configured.
Never stop firewalld to "see if that is the problem". Switch the target or open the port instead: the firewall that was stopped is the firewall you will forget to start.