A deep dive into BPF filters: precedence, byte offsets, flags, payloads, IPv6, VLANs and fragments, and the cases where a filter silently matches nothing, all tested against one reference capture
▶Prefer to watch? This video runs the guide's main commands on a real Ubuntu 24.04 machine, with on-screen captions and no voiceover. Click to play, or watch it on YouTube. It is part of the Networking and Firewalls playlist.
tcpdump filters: the BPF expression language
tcpdump -nn -r capture.pcap 'tcp[tcpflags] == tcp-syn'
The Tcpdump quick reference on this site tells you WHICH filters exist: host, port, tcp, a few flag tests. This page is about the filter language itself: how an expression is read, why "icmp or arp and ether broadcast" does not do what it looks like, how to pick fields out of a packet by byte position, and the places where a filter silently matches nothing and gives you no error. Those places (VLAN tags, IPv6, fragments, short snapshot lengths) are where most "tcpdump shows nothing" hours go.
The language is BPF (Berkeley Packet Filter), the same one used by libpcap, Wireshark's capture filters, tshark -f, and many network appliances. tcpdump compiles your expression into a small program and, on Linux, hands it to the kernel, so packets that do not match are never copied to user space. That is why a good filter makes a capture on a busy link possible at all.
Every example below ran for real: tcpdump 4.99.4 with libpcap 1.10.4 on Ubuntu 24.04. The method is the one recommended here: capture ONCE into a file (the "reference capture"), and then ask many questions of it with tcpdump -r FILE FILTER, counting the answers. The reference capture has 149 packets on a lab Ethernet link, produced by a client machine talking to a server: ARP, ping (IPv4 and IPv6), HTTP GET and POST over IPv4 and IPv6, HTTPS with a server name (SNI), DNS over UDP and TCP (including a name that does not exist), a refused connection, a 4000-byte UDP datagram that fragments, VLAN 100 traffic, a ping with a DSCP mark, a ping with TTL 1, and a few crafted scan-style packets (FIN, NULL, XMAS). The count for each filter is the number of packets it matched in that file. Tools used besides tcpdump: tshark, as an independent check of what is really in the file. Anything not run in the lab is marked "not run in the lab".
Capture filters and display filters
Two different languages share the word "filter", and mixing them up is the first source of confusion:
capture filter BPF. Written in tcpdump, tshark -f, and Wireshark's capture dialog. Runs BEFORE the packet is stored, usually in the kernel. Knows addresses, ports, protocol names and raw byte positions. Does NOT understand applications: it has no idea what an HTTP method is, only which bytes would hold one.
display filter Wireshark's own language (http.request.method == "POST", tcp.flags.syn == 1). Runs AFTER capture, on decoded packets, and understands hundreds of protocols. Cannot be used to limit what is captured.
tcpdump has only the first kind. tshark -r FILE -Y EXPRESSION applies the second kind to a saved file, and is a good partner: BPF to capture cheaply, display filters to analyze (see the tshark section). The consequence for a capture: if a filter cannot be written in BPF, capture a little more broadly (a host, a port), save it with -w, and refine later; do not try to capture everything and filter live on a busy link.
The building blocks: type, direction and protocol
A simple filter is made of up to three qualifiers followed by an id:
type host, net, port, portrange (what the id is; if you leave the type out, host is assumed)
direction src, dst, src or dst, src and dst (default: src or dst, so "port 80" is either side)
protocol ether, ip, ip6, arp, tcp, udp, icmp, icmp6 ... (restricts what kind of packet is considered)
The reference capture, as a map of what the common primitives find (the number is how many packets of the 149 matched):
$ tcpdump -nn -r ref.pcap | head -3
reading from file ref.pcap, link-type EN10MB (Ethernet), snapshot length 262144
19:57:03.759333 IP6 fe80::18b8:79ff:fef8:7b54 > ff02::2: ICMP6, router solicitation, length 16
19:57:04.009547 ARP, Request who-has 192.168.60.1 tell 192.168.60.20, length 28
ip 99 IPv4 packets (untagged)
ip6 29 IPv6 packets
arp 4
icmp 10 ICMPv4
icmp6 11 ICMPv6
tcp 94
udp 12
vlan 17 frames with an 802.1Q tag
host 192.168.60.20 103 the client, both directions
ip host 192.168.60.20 99 the client, IPv4 only
host fd00:60::20 26 the client's IPv6 address
src host 192.168.60.20 59
dst host 192.168.60.1 59
net 192.168.60.0/24 103
net fd00:60::/64 26
port 80 51
tcp port 80 51
src port 80 20 the server's replies
dst port 80 31 the requests
udp port 53 8
port 53 or port 22 26
portrange 8000-9999 4
ether broadcast 3
ether multicast 8
Two details are already visible in that list. "host 192.168.60.20" matched 103 packets and "ip host 192.168.60.20" matched 99: the difference is the 4 ARP packets, because host matches ARP as well as IP. And a bare "port 80" is the same 51 as "tcp port 80" here only because nothing else in the file uses port 80; "port 53" matches UDP and TCP together (18), "udp port 53" only the 8 UDP ones. Direction decides the question being asked: "dst port 80" finds the 31 packets going TO a web server, "src port 80" the 20 coming back.
Protocol names that are also keywords need a backslash after proto:
$ tcpdump -nn -r ref.pcap 'ip proto 6'
$ tcpdump -nn -r ref.pcap 'ip proto \tcp'
Both match the same 77 untagged IPv4 TCP packets. (tcp, udp and icmp are reserved words, so inside "proto" they need the backslash; the shell needs the quotes. "ip proto 6" with the number needs neither.)
Combining: and, or, not, parentheses and the precedence trap
The operators are and (&&), or (||), not (!), and parentheses. The rule that surprises everybody: "and" and "or" have EQUAL precedence and are applied from left to right. There is no "and before or" as in most languages. Only "not" binds tighter. The reference capture makes the effect obvious:
icmp 10
arp 4
ether broadcast 3
icmp or arp and ether broadcast 2 read as (icmp or arp) and ether broadcast
icmp or (arp and ether broadcast) 12 what most people mean
(icmp or arp) and ether broadcast 2
not icmp and not arp 135
not (icmp or arp) 135
not icmp or arp 139 read as (not icmp) or arp
The unparenthesized "icmp or arp and ether broadcast" returned 2: the left-to-right reading filtered the pings out. The intended one, "ICMP, or ARP that is a broadcast", returned 12. The fix is simple: ALWAYS use parentheses when mixing and with or. The "not" lines are the usual pair of equal forms (De Morgan: 135 both ways).
Quoting. The shell treats ( ) & | ! > < * [ ] as its own. Always put the whole filter in single quotes:
$ tcpdump -nn -r ref.pcap host 192.168.60.20 and (port 80 or port 443)
bash: -c: line 1: syntax error near unexpected token `('
$ tcpdump -nn -r ref.pcap 'host 192.168.60.20 and (port 80 or port 443)'
Inside single quotes, spaces are optional (tcp[13]==2 and tcp[13] == 2 mean the same). In bash, an unquoted ! also triggers history expansion at the prompt: quote it, or use "not".
Counting what a filter matches
The tool for the rest of this page, and a habit worth adopting: when you are not sure a filter does what you think, run it on a saved file and count:
$ tcpdump -nn -r ref.pcap 'tcp dst port 80' 2>/dev/null | wc -l
31
Redirect stderr (2>/dev/null) to drop the "reading from file" line, and wc -l counts packets (one per line when you do not use -v). Compare the count with what you expect from a second tool (tshark, or the application's own log). A filter that returns 0 is the dangerous result: it might mean "no such traffic", or "my filter cannot see this traffic", and only the second tool tells you which.
Bytes and bits: proto[offset:size]
Beyond the named primitives, BPF can read any byte of any header with proto[offset:size]. proto is a protocol layer (ether, ip, ip6, tcp, udp, icmp), offset counts bytes from the start of THAT header, and size is 1, 2 or 4 (default 1). The result is a number you compare with = == != < <= > >=, and you can do arithmetic on it: & | ^ << >> + - * / and parentheses. Numbers can be decimal or hex (0x1f).
ip[8] one byte: byte 8 of the IP header (the TTL)
ip[2:2] two bytes, starting at byte 2 (the total length)
tcp[13] byte 13 of the TCP header (the flags)
ip[6:2] & 0x1fff the fragment offset: mask two bytes with 0x1fff
ip[1] >> 2 the DSCP: byte 1 shifted two bits right
The sizes that are allowed are 1, 2 and 4, as the program tells you if you try anything else:
$ tcpdump -nn -r ref.pcap 'tcp[0:3] = 1'
tcpdump: data size must be 1, 2, or 4
An offset past the end of the packet does not produce an error, it just never matches (the lab ran "tcp[1000] = 1" and "ip[300] == 1": no message, no packets). That is convenient for payload tests and a trap when the offset is wrong. Names for common fields make it readable: tcpflags (byte 13), icmptype, icmp6type, and constants like tcp-syn, tcp-ack, icmp-echo. Prefer them to magic numbers.
IP header fields: TTL, DSCP, protocol, length
The first byte of the IPv4 header is the version and header length, byte 1 is the DSCP/ECN, bytes 2-3 the total length, bytes 6-7 the flags and fragment offset, byte 8 the TTL, byte 9 the protocol:
$ tcpdump -nn -r ref.pcap 'ip[8] < 5'
19:57:05.053733 IP 192.168.60.20 > 192.168.60.1: ICMP echo request, id 5653, seq 1, length 64
19:57:06.826226 IP 192.168.60.20.40006 > 192.168.60.1.80: Flags [S], seq 0, win 8192, length 0
ip[8] < 5 2 the ping with TTL 1 and the crafted SYN with TTL 3 (the others have 64: ip[8] == 64 matched 97)
ip[1] >> 2 == 46 2 DSCP 46, "Expedited Forwarding": the ping sent with -Q 0xb8 and its reply
ip[1] & 0xfc == 0xb8 2 the same test with a mask instead of a shift
ip[1] != 0 4 anything with a non-zero TOS byte at all (DSCP or ECN)
ip proto 1 10 ICMP, identical to ip[9] == 1
ip proto 17 12 UDP
ip[0] & 0x0f > 5 0 IP header longer than 20 bytes (IP options) : none here
The header length is in the low four bits of byte 0, counted in 32-bit words, which is why TCP and UDP port tests inside the compiler read "x + 14" (the offset of the TCP header depends on this). A query on IP options is a good way to find unusual packets, because normal traffic has none.
Length filters measure the WHOLE FRAME, link header included, unlike ip[2:2], which is the IP total length:
$ tcpdump -nn -e -r ref.pcap 'greater 1400'
... length 1409: 192.168.60.1.443 ...
... length 5235: 192.168.60.1.443 ...
... length 1514: 192.168.60.20.532 ...
... length 1514: 192.168.60.20 > 1 ...
greater 1400 4
len > 1400 4 the same thing
ip[2:2] > 1400 3
less 60 20
A frame of 5235 bytes on an Ethernet link with an MTU of 1500 looks impossible. It is a TLS segment that the network card or kernel assembled (GRO/TSO offloading) before tcpdump saw it: on the capturing machine, "greater 1400" can find frames bigger than the MTU, and a filter like "greater 1400" to look for MTU trouble will report offloaded segments that never traveled that way on the wire. When size matters, capture on a different machine or turn offloading off (ethtool -K IFACE gro off gso off tso off; not run in the lab).
Fragments
A packet bigger than the path MTU is split into fragments. Only the FIRST fragment carries the transport (TCP or UDP) header; the rest are bare IP payload. That has one important consequence for filters: a filter on a port finds only the first fragment. The lab sent a 4000-byte UDP datagram to port 9999 (three fragments):
$ tcpdump -nn -r ref.pcap 'ip[6:2] & 0x3fff != 0'
19:57:06.350943 IP 192.168.60.20.53234 > 192.168.60.1.9999: UDP, length 4000
19:57:06.350962 IP 192.168.60.20 > 192.168.60.1: ip-proto-17
19:57:06.350965 IP 192.168.60.20 > 192.168.60.1: ip-proto-17
udp port 9999 1 only the first fragment has a UDP header
ip[6:2] & 0x3fff != 0 3 any fragment: MF flag set, or offset not zero
ip[6:2] & 0x2000 != 0 2 More Fragments set (first and middle fragments)
ip[6:2] & 0x1fff != 0 2 offset not zero (middle and last fragments)
ip[6] & 0x40 != 0 80 Don't Fragment set (most modern TCP traffic)
host 192.168.60.1 and ip[9] == 17 and ip[6:2] & 0x3fff != 0 3
The two later fragments show up as "ip-proto-17" with no ports, exactly because tcpdump cannot see a UDP header there. The flags field is the top three bits of the two bytes at offset 6 (reserved, DF 0x4000, MF 0x2000) and the rest is the offset in 8-byte units. To follow ONE fragmented datagram, select by the IP identification number in ip[4:2], which all fragments share (here 0xacc7: 3 matches). A capture or a firewall rule on a port that "does not see" the big packets is usually this.
ICMP and ICMPv6 types
icmp[icmptype] reads the type byte, and the names work (the lab used icmp-echo, icmp-echoreply and icmp-unreach; others such as icmp-timxceed exist, not run in the lab):
icmp[icmptype] == icmp-echo 4
icmp[icmptype] == icmp-echoreply 4
icmp[icmptype] == icmp-unreach 2
icmp[icmptype] != icmp-echo and icmp[icmptype] != icmp-echoreply 2 everything that is not ping
$ tcpdump -nn -r ref.pcap 'icmp[icmptype] == icmp-unreach'
19:57:06.192153 IP 192.168.60.1 > 192.168.60.20: ICMP 192.168.60.1 udp port 9998 unreachable, length 38
19:57:06.350990 IP 192.168.60.1 > 192.168.60.20: ICMP 192.168.60.1 udp port 9999 unreachable, length 556
"Destination unreachable" is the one worth watching: it is a firewall or a closed port telling you no. The second line here is the reassembled fragments arriving at a port with nobody listening. For ICMPv6 the field is icmp6type, with names such as icmp6-echo, icmp6-neighborsolicit, icmp6-neighboradvert:
icmp6[icmp6type] == icmp6-echo 3
icmp6[icmp6type] == icmp6-neighborsolicit 2
icmp6[icmp6type] == icmp6-neighboradvert 2
$ tcpdump -nn -r ref.pcap 'icmp6[icmp6type] == icmp6-neighborsolicit' | head -1
19:57:05.060526 IP6 fd00:60::20 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fd00:60::1, length 32
(the IPv6 version of "who has 192.168.60.1" ARP: IPv6 has no ARP.)
TCP flags
The flags are byte 13 of the TCP header: the named bits are tcp-fin, tcp-syn, tcp-rst, tcp-push, tcp-ack and tcp-urg. tcp[tcpflags] reads the byte. The point is the mask: "has this bit" is & != 0, "is exactly this" is ==, and "has these and not those" is & (set) == value:
tcp[tcpflags] & tcp-syn != 0 17 any SYN (SYN and SYN-ACK)
tcp[tcpflags] == tcp-syn 10 SYN, nothing else
tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn 10 SYN set and ACK not set: a new connection attempt
tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack) 7 SYN-ACK: a server accepting
tcp[tcpflags] & tcp-rst != 0 5 resets
tcp[tcpflags] & tcp-fin != 0 12 connection closes
tcp[tcpflags] == 0 1 NULL scan: no flags at all
tcp[tcpflags] & (tcp-fin|tcp-push|tcp-urg) == (tcp-fin|tcp-push|tcp-urg) 1 XMAS scan
tcp[13] == 2 10 the same as == tcp-syn, with the number
Ten connection attempts and only seven SYN-ACKs: the missing three are refused (a closed port), unanswered, or reset. The reset packets say which:
$ tcpdump -nn -r ref.pcap 'tcp[tcpflags] & tcp-rst != 0 and src port 3306'
19:57:06.173451 IP 192.168.60.1.3306 > 192.168.60.20.43542: Flags [R.], seq 0, ack 2802388127, win 0, length 0
19:57:06.793202 IP 192.168.60.1.3306 > 192.168.60.20.40004: Flags [R.], seq 0, ack 1, win 0, length 0
Port 3306 on the server is closed, and it answers every SYN with a reset: "Connection refused" as the application sees it. A SYN with NO reply at all (a firewall DROP) shows up the other way round: a SYN with no matching SYN-ACK or RST. The scan-style packets show how a filter spots them:
$ tcpdump -nn -r ref.pcap 'tcp[tcpflags] == 0'
19:57:06.764767 IP 192.168.60.20.40002 > 192.168.60.1.80: Flags [none], win 8192, length 0
$ tcpdump -nn -r ref.pcap 'tcp[tcpflags] & (tcp-fin|tcp-push|tcp-urg) == (tcp-fin|tcp-push|tcp-urg)'
19:57:06.781638 IP 192.168.60.20.40003 > 192.168.60.1.80: Flags [FPU], seq 0, win 8192, urg 0, length 0
A NULL packet and an all-flags packet are never part of a normal connection. In output, the flag letters are S (SYN), . (ACK), P (PUSH), F (FIN), R (RST), U (URG), so [S.] is SYN-ACK and [R.] is a reset with an acknowledgment. The reminder: the abbreviations in the filter are tcp-syn, not syn. The shortcut "tcp[tcpflags] & syn != 0" is a syntax error (see the error section).
Looking inside the payload: DNS, HTTP, SSH and TLS
BPF cannot parse an application protocol, but if you know where the bytes are, it can read them. Two things you need:
1. Where the payload starts. For UDP it is fixed: the UDP header is 8 bytes, so the data starts at udp[8]. For TCP the header length varies (options), and it is stored in the high four bits of byte 12, in 32-bit words. In bytes, the TCP payload starts at ((tcp[12] & 0xf0) >> 2). So "the first four bytes of the TCP payload" is tcp[((tcp[12:1] & 0xf0) >> 2):4]. Copy that expression; it is the key to every TCP payload filter.
2. The bytes themselves, in hex: ASCII text is easy to turn into a number: "GET " is 47 45 54 20, so 0x47455420; "POST" is 0x504f5354; "HEAD" is 0x48454144; "PUT " is 0x50555420; "HTTP" is 0x48545450 (the start of a response); "SSH-" is 0x5353482d. (GET, POST and SSH- were run in the lab; the others follow from the ASCII table.)
DNS (UDP). The DNS header is 12 bytes at the start of the UDP payload: a 2-byte ID, then 2 bytes of flags (udp[10] and udp[11]), then four counters. The top bit of udp[10] is QR (query or response), the low four bits of udp[11] are the response code, where 0 is NOERROR and 3 is NXDOMAIN:
udp port 53 8
udp port 53 and udp[10] & 0x80 == 0 4 queries
udp port 53 and udp[10] & 0x80 != 0 4 responses
udp port 53 and udp[11] & 0x0f == 3 2 NXDOMAIN
udp src port 53 and udp[11] & 0x0f == 0 2 responses with NOERROR
udp port 53 and udp[12:2] == 1 8 exactly one question in the packet
$ tcpdump -nn -r ref.pcap 'udp src port 53 and udp[11] & 0x0f == 3'
19:57:06.234027 IP 192.168.60.1.53 > 192.168.60.20.50499: 53764 NXDomain 0/0/1 (44)
The two NXDOMAIN responses are the query for the name that does not exist, and, a lab detail, the query for an MX record of www.lab.test: the lab's DNS server (dnsmasq, with the whole zone declared local) also answers "no such name" for a record type it has no data for, where many servers answer NOERROR with an empty answer. To find DNS over TCP, the same payload positions are shifted by the TCP header and by the 2-byte length prefix that DNS-over-TCP adds. (tcp port 53 matched 10 packets here: the handshake and the dig +tcp query; not run in the lab: parsing the DNS flags inside TCP.)
HTTP requests (TCP). The first bytes of the payload are the method:
tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420 1 GET
tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354 1 POST
$ tcpdump -nn -A -r ref.pcap 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354'
19:57:06.129896 IP 192.168.60.20.47448 > 192.168.60.1.80: Flags [P.], seq 1400360895:1400361062, ack 93037020, win 63, ...
E.....@.@..-..<...<..X.PSw.........?.3.....
...H..."POST / HTTP/1.1
Host: 192.168.60.1
User-Agent: curl/8.5.0
Accept: */*
Content-Length: 22
Content-Type: application/x-www-form-urlencoded
user=alice&pass=secret
That is how you spot a form submission with a password in plain text: -A prints the packet as text. Only requests whose first bytes arrive in ONE packet are found (a request split across packets, or sent with a leading space, is not), and HTTPS is invisible this way (see below). The capture holds three GET requests, one over IPv4, one over IPv6 and one inside VLAN 100, and this filter finds only the IPv4 one (tshark finds all three): the IPv6 and VLAN cases are covered in their own sections below.
SSH banner. The server announces itself with "SSH-2.0-..." as the first data it sends:
$ tcpdump -nn -r ref.pcap 'tcp src port 22 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x5353482d'
19:57:06.166921 IP 192.168.60.1.22 > 192.168.60.20.60230: Flags [P.], seq 1431102741:1431102766, ack 2444641045, win 64, ...
Useful to find SSH on a non-standard port: drop the port number and look for the bytes, "tcp[...:4] = 0x5353482d" alone.
TLS handshakes (TCP). A TLS record starts with a type byte (0x16 for a handshake, 0x17 for application data); the handshake message type is five bytes in (0x01 ClientHello, 0x02 ServerHello). Without being able to decrypt anything, that already tells you who is starting HTTPS, and the ClientHello is not encrypted:
tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16 1
tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16 and tcp[((tcp[12:1] & 0xf0) >> 2)+5] = 0x01 1 ClientHello
tcp src port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16 and tcp[((tcp[12:1] & 0xf0) >> 2)+5] = 0x02 1 ServerHello
tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)+1:2] = 0x0301 1 record version field (the legacy value)
tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x17 2 encrypted application data
The server name the client wants (the SNI) is in plain text inside the ClientHello, and "-A" shows it:
$ tcpdump -nn -A -r ref.pcap 'tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16 and tcp[((tcp[12:1] & 0xf0) >> 2)+5] = 0x01' | grep -a -o 'www.lab.test'
www.lab.test
Good for finding which sites a machine contacts, and for troubleshooting a certificate that does not match. There is no filter that matches a specific SNI directly (the position of the field moves with the other extensions), so find ClientHellos in tcpdump, and read the name with grep, or with tshark -Y 'tls.handshake.extensions_server_name == "..."'. Everything after the handshake is encrypted: the filters above see only the first packets.
Snapshot length: the silent killer
The snapshot length (-s) is how many bytes of each packet are kept. The default in current tcpdump is the maximum (262144), which is why the headers of the reference capture say "snapshot length 262144". With an older tcpdump, or with -s set to a small number to save space, the packet is cut, and a filter on a payload byte that was cut off never matches. It also does not complain. The lab asked for the POST request with a snaplen of 64 bytes (the "POST" bytes start at byte 66 of the frame):
$ tcpdump -i eth1 -nn -s 64 -c 1 -w live64.pcap 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354'
0 packets captured
1 packet received by filter
0 packets dropped by kernel
$ curl -d 'user=alice&pass=secret' http://192.168.60.1/ # from the client, while that ran
The request went by, "received by filter" counted it, and NOTHING was captured: the file is empty. (The same filter with the default snaplen finds it, as shown above.) The rule: when a filter looks at the payload, capture with -s 0 (which means "the maximum") or a snaplen at least as large as the highest byte you test, plus the headers. Conversely, once a file was captured with a short snaplen, a payload filter run on it with -r finds nothing for the same reason: the bytes are not in the file. You can check how much a file kept with tshark: frame.len is the packet's real size and frame.cap_len what was saved.
IPv6: why tcp[...] finds nothing
This one is in the manual, in one sentence, and costs people days: in BPF, the upper-layer names tcp, udp and icmp used with a byte offset (tcp[13], udp[10], icmp[0]) apply to IPv4 ONLY. For IPv6 the same expression is quietly false. The reference capture, with tshark's independent count of SYN-only packets as the truth:
tshark: SYN only, IPv4 packets (11 includes 1 in a VLAN) 11
tshark: SYN only, IPv6 packets (includes one with an extension header) 3
tshark: SYN only, in a VLAN 1
tcpdump 'tcp[tcpflags] == tcp-syn' 10 only the untagged IPv4 ones
tcpdump 'ip and tcp[tcpflags] == tcp-syn' 10
tcpdump 'ip6 and tcp[tcpflags] == tcp-syn' 0 IPv6 SYNs: not seen
tcpdump 'ip6 and tcp port 80' 17 but "tcp port 80" (no offsets) DOES work for IPv6
So the plain keywords (tcp, port, host) handle both protocols, and the byte-offset forms do not. The fix for IPv6 is to count the bytes yourself with ip6[...]: the IPv6 header is a fixed 40 bytes, so the TCP header starts at ip6[40], the "next header" field is ip6[6] (6 means TCP), and the TCP flags are at ip6[40+13]:
$ tcpdump -nn -r ref.pcap 'ip6[6] == 6 and ip6[40+13] == 2'
19:57:06.116478 IP6 fd00:60::20.60774 > fd00:60::1.80: Flags [S], seq 3369615732, win 64800, ...
19:57:06.874440 IP6 fd00:60::20.40008 > fd00:60::1.80: Flags [S], seq 0, win 8192, length 0
ip6[6] == 6 17 IPv6 carrying TCP directly
ip6[6] == 6 and ip6[40+13] == 2 2 SYN-only: the missing IPv6 SYNs
ip6[6] == 6 and ip6[40+13] & 0x12 == 0x02 2 SYN set, ACK not set, written with a mask
IPv6 payload tests need the same arithmetic, shifted by 40. The HTTP GET filter for IPv6 (found 1 in the lab, the request the IPv4 filter had missed):
ip6[6] == 6 and ip6[40+2:2] == 80 and ip6[40+((ip6[40+12] & 0xf0) >> 2):4] = 0x47455420
Read it as: the next header is TCP, the destination port (2 bytes at TCP offset 2) is 80, and the first four payload bytes are "GET ". Awkward, but it works, and a script can build it. EXTENSION HEADERS add a second trap. IPv6 can put optional headers (hop-by-hop, routing, fragment, destination options) between the IPv6 header and TCP. The keyword "tcp" and "ip6[6] == 6" only see TCP when it is the immediate next header. The lab sent one SYN with a Destination Options header in front:
$ tcpdump -nn -r ref.pcap 'ip6[6] == 60'
19:57:06.865095 IP6 fd00:60::20 > fd00:60::1: DSTOPT 40007 > 80: Flags [S], seq 0, win 8192, length 0
ip6 and tcp 17 the SYN with the extension header is NOT in this count
ip6[6] == 6 17
ip6[6] == 60 1 next header 60 = destination options
ip6 protochain 6 18 any chain of headers that ends in TCP: includes it
protochain walks the whole chain of extension headers, so "ip6 protochain 6" finds ALL IPv6 TCP, while "tcp" misses a packet that has an extension header: 17 against 18. This matters for security monitoring: an attacker, or just an odd stack, can hide TCP behind an extension header, and a filter written with the plain keyword will never see it. In the same spirit, "ip6 and not tcp and not udp and not icmp6" lists the packets the usual keywords do not know (1 in the lab: that same one). protochain is a libpcap feature that makes the compiled filter longer and slower; use it when completeness matters more than speed.
VLAN tags: the keyword that must come first
A frame with an 802.1Q tag has four extra bytes after the Ethernet addresses, and everything after them is shifted. BPF does not look inside a tag unless you tell it to, with the keyword vlan. Without it, tagged traffic is invisible to protocol filters, not only to the VLAN ones. The reference capture has 17 tagged frames (VLAN 100, the 10.100.0.0/24 network):
tcp 94 untagged only
vlan and tcp 12 the tagged TCP packets
tcp or (vlan and tcp) 106 both: 94 + 12
net 10.100.0.0/24 0 the VLAN's whole network: no packet matches, no error!
vlan and net 10.100.0.0/24 16
vlan 100 17
vlan 100 and tcp port 80 12
tcp port 80 51 and the VLAN's web traffic is not part of it
ether proto 0x8100 17 the tag type: a way to count tagged frames without the keyword
ether[12:2] == 0x8100 17 the same with raw bytes
$ tcpdump -nn -e -r ref.pcap 'vlan 100 and icmp' | head -1
20:... ea:50:55:89:92:75 > 1a:b8:79:f8:7b:54, ethertype 802.1Q (0x8100), length 102: vlan 100, p 0, ethertype IPv4 (0x0800), 10.100.0.20 > 10.100.0.1: ICMP echo request ...
So: "tcp port 80" on a trunk port quietly ignores half the traffic, and the net 10.100.0.0/24 test finds nothing although the host is talking. And now the rule that costs the second hour: the keyword vlan shifts the offsets of everything AFTER it in the expression, and nothing before it. So the order matters:
vlan and ip[8] == 64 14
ip[8] == 64 and vlan 0 (the TTL test was done on the wrong bytes)
vlan and tcp port 80 12
tcp port 80 and vlan 0
Always put vlan FIRST. And be careful with the obvious "either tagged or not" idea: "(vlan or not vlan) and tcp" matched 0 in the lab, far worse than either half, because "not vlan" is evaluated after the shift. If you want both, write the two cases out: "tcp or (vlan and tcp)". For double tags (QinQ), repeat the keyword: vlan and vlan and ... (not run in the lab).
Link-layer filters and -i any
On a normal Ethernet interface the first thing in the packet is the Ethernet header, and the "ether" primitives look at it: ether host, ether src, ether dst, ether broadcast, ether multicast, ether proto 0x0806. They are the right tool for MAC-level questions (a broadcast storm, one device's traffic, ARP):
ether broadcast 3
ether multicast 8
ether proto 0x0806 (ARP) 4
If you capture on "any" instead of one interface, the picture changes. -i any is convenient (you do not have to know which interface the packet takes), but it uses a different link type, "Linux cooked capture" (LINUX_SLL2 here), with a short pseudo-header in place of the Ethernet header. There are no MAC addresses to filter on, and the ether primitives do NOT raise an error, they return nothing:
$ tcpdump -nn -r any.pcap | head -1
reading from file any.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144
ether host 1a:b8:79:f8:7b:54 0 the client's MAC: the packets are there, the filter just cannot see MACs
ether broadcast 0
vlan 0
arp 8 still works: it is a protocol keyword
tcp port 80 75 against 51 on eth1 alone
The 24 extra packets are copies: with -i any, a packet is shown on every interface it crosses, and the VLAN traffic appears on eth1 AND on eth1.100 (without its tag):
$ tcpdump -nn -r any.pcap 'tcp port 80' | awk '{print $2}' | sort | uniq -c
63 eth1
12 eth1.100
So counts from -i any are not comparable with counts from one interface, and a top talkers list will double count. The cooked format has its own bonus, a direction: the keywords inbound and outbound match packets by whether they arrived or left (the reference captures: 94 inbound and 73 outbound), and the output has an In / Out column. On a normal interface the same is done with -Q in or -Q out, or --direction=in, applied in the kernel:
$ tcpdump -i eth1 -nn -Q in 'icmp'
20:03:43.908343 IP 192.168.60.20 > 192.168.60.1: ICMP echo request, id 6542, seq 1, length 64
$ tcpdump -i eth1 -nn -Q out 'icmp'
20:03:43.908360 IP 192.168.60.1 > 192.168.60.20: ICMP echo reply, id 6542, seq 1, length 64
-Q in saw the request arrive, -Q out the reply leave. That is a much cleaner way to ask "does the machine ANSWER" than comparing two captures. (Reading a cooked file also prints "Warning: interface names might be incorrect": harmless.)
How a filter is compiled (-d)
tcpdump translates the expression into BPF instructions, and -d prints them, which makes "what does this filter really do" a question with an exact answer. It needs no capture. The simplest useful example, ip[8] < 5:
$ tcpdump -d 'ip[8] < 5'
(000) ldh [12]
(001) jeq #0x800 jt 2 jf 5
(002) ldb [22]
(003) jge #0x5 jt 5 jf 4
(004) ret #262144
(005) ret #0
Six instructions: load the two bytes at offset 12 (the Ethernet type), jump if it is IPv4 (0x800), load the byte at 22 (14 bytes of Ethernet plus byte 8 of the IP header), jump away if it is 5 or more, otherwise return 262144 (accept, keeping up to that many bytes), else return 0 (reject). That is the whole model: a tiny register machine, a list of jumps, and two exits. Now the real shape of "tcp port 80", which is not small, because it has to handle IPv4 AND IPv6 and fragments:
$ tcpdump -d 'tcp port 80'
(000) ldh [12]
(001) jeq #0x86dd jt 2 jf 8
(002) ldb [20]
(003) jeq #0x6 jt 4 jf 19
(004) ldh [54]
(005) jeq #0x50 jt 18 jf 6
(006) ldh [56]
(007) jeq #0x50 jt 18 jf 19
(008) jeq #0x800 jt 9 jf 19
(009) ldb [23]
(010) jeq #0x6 jt 11 jf 19
(011) ldh [20]
(012) jset #0x1fff jt 19 jf 13
(013) ldxb 4*([14]&0xf)
(014) ldh [x + 14]
(015) jeq #0x50 jt 18 jf 16
(016) ldh [x + 16]
(017) jeq #0x50 jt 18 jf 19
(018) ret #262144
(019) ret #0
The first half (0 to 7) is the IPv6 path with fixed offsets (the ports are at 54 and 56, because 14 + 40 = 54); the second half is IPv4: check protocol, check "not a later fragment" (instruction 12, the jset on 0x1fff: later fragments have no ports), compute the header length with ldxb 4*([14]&0xf), and read the ports from "x + 14" and "x + 16". You can read in it every behavior described above: the filter does not look at VLAN tags (it reads the type at offset 12), fragments are skipped, and the IPv6 path has no extension header handling. The cost of a filter is roughly its instruction count, and the counts grow with how it is written:
ip 5
tcp 13
port 80 25
tcp port 80 21
host 192.168.60.20 15
net 10.0.0.0/8 19
tcp[tcpflags] == tcp-syn 12
tcp port 80 or tcp port 443 or tcp port 22 27
port 80 or port 443 or port 22 or port 53 or port 25 37
ip and tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420 19
Two lessons. "port 80" (25) is more expensive than "tcp port 80" (21): it must handle TCP, UDP and SCTP. And a filter that starts with ip skips the whole IPv6 branch: the last line, which also tests four payload bytes, still compiles to 19 instructions, fewer than plain "tcp port 80". In practice, BPF is fast enough that the count does not matter until you capture at tens of gigabits, but it is the reason a very long list of "or" tests slows a high-rate capture. Other forms: -dd prints the program as C array initializers (for embedding it in your own code), and -ddd prints the instruction count and the instructions as plain numbers (for tools that load a filter):
$ tcpdump -dd 'tcp port 80' | head -3
{ 0x28, 0, 0, 0x0000000c },
{ 0x15, 0, 6, 0x000086dd },
{ 0x30, 0, 0, 0x00000014 },
$ tcpdump -ddd 'tcp port 80' | head -3
20
40 0 0 12
21 0 6 34525
(-d without -i assumes Ethernet and says so: "Warning: assuming Ethernet". With a different link type, for example when capturing on "any", the offsets change, which is exactly why Ethernet filters misbehave on cooked captures. The -y option selects the link type for -d; not run in the lab.)
Files: -r, -w and -F
Three habits make filters easier to get right. First: capture broadly, filter at read time. Run a short, generous capture with -w, then refine with -r, because every pass over the file is free and repeatable:
$ tcpdump -nn -r ref.pcap -w dns.pcap 'port 53'
$ tcpdump -nn -r dns.pcap | wc -l
18
$ tcpdump -nn -r dns.pcap 'udp[10] & 0x80 != 0' | head -2
19:57:06.207456 IP 192.168.60.1.53 > 192.168.60.20.45737: 8701* 1/0/1 A 192.168.60.80 (57)
19:57:06.234027 IP 192.168.60.1.53 > 192.168.60.20.50499: 53764 NXDomain 0/0/1 (44)
The first command read the reference capture, kept only DNS, and wrote the 18 packets to a new file: a subset to send to a colleague (and one that does not contain the passwords that were in the HTTP traffic). Second: for a long or complicated filter, put it in a file and use -F. Comments with # are allowed and the expression can span lines:
$ cat f.bpf
# web or dns, but not from the admin host
(tcp port 80 or tcp port 443 or udp port 53)
and not src host 192.168.60.1
$ tcpdump -nn -r ref.pcap -F f.bpf | wc -l
53
The same filter given inline matched the same 53. A filter file can be kept in version control and shared, which a one-liner in somebody's shell history cannot. Third: read the counters at the end of a live capture. They are in the last lines (on stderr):
$ tcpdump -i eth1 -nn -c 3 'icmp'
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
3 packets captured
6 packets received by filter
0 packets dropped by kernel
"captured" is how many packets tcpdump printed; "received by filter" is how many packets the filter processed in total, which can be MORE than captured, because the kernel keeps counting until tcpdump has actually stopped (here three pings make six ICMP packets, and -c 3 stopped tcpdump after printing three); "dropped by kernel" is the number the system had no room for. The one to watch is dropped: a non-zero number means the capture is incomplete and a filter count is a minimum, never a total. Tighten the filter, write with -w instead of printing, or raise the buffer with -B (KiB). (Not run in the lab: a capture with real drops.)
tcpdump filters and tshark display filters
The reference capture again, asked with tshark's display filters (a different, application-aware language) for what BPF could only approximate:
$ tshark -r ref.pcap -Y 'http.request.method == "POST"'
52 2.370563 192.168.60.20 → 192.168.60.1 HTTP 233 POST / HTTP/1.1 (application/x-www-form-urlencoded)
$ tshark -r ref.pcap -Y 'tls.handshake.extensions_server_name == "www.lab.test"'
77 2.395811 192.168.60.20 → 192.168.60.1 TLSv1 583 Client Hello (SNI=www.lab.test)
$ tshark -r ref.pcap -Y 'dns.flags.rcode == 3'
110 2.474694 192.168.60.1 → 192.168.60.20 DNS 86 Standard query response 0xd204 No such name A nosuch.lab.test OPT
114 2.523745 192.168.60.1 → 192.168.60.20 DNS 83 Standard query response 0x1d11 No such name MX www.lab.test OPT
$ tshark -r ref.pcap -Y 'ip.flags.mf == 1 || ip.frag_offset > 0'
125 2.591610 192.168.60.20 → 192.168.60.1 IPv4 1514 Fragmented IP protocol (proto=UDP 17, off=0, ID=acc7)
126 2.591629 192.168.60.20 → 192.168.60.1 IPv4 1514 Fragmented IP protocol (proto=UDP 17, off=1480, ID=acc7)
127 2.591632 192.168.60.20 → 192.168.60.1 UDP 1082 53234 → 9999 Len=4000
Notice what the display filters did that BPF could not: matched the TLS server name directly, decoded the DNS response code, and looked past VLAN tags and the IP version without being told ("http.request.method == \"GET\"" found all 3 GET requests, over IPv4, IPv6 and VLAN 100, where the BPF payload filter found 1). "tcp.flags.syn==1 && tcp.flags.ack==0" counted 14 packets across IPv4, IPv6 and VLAN, where the tcpdump tcp[...] filter found 10. And tshark knows it is a different language: it refuses a capture filter on a file:
$ tshark -r ref.pcap -f 'port 80'
tshark: Only read filters, not capture filters, can be specified when reading a capture file.
The working split: BPF (tcpdump, or tshark -f) to limit what is captured, display filters (tshark -Y, Wireshark) to analyze what was captured. Use the first when volume or privacy matters (nothing but the traffic you ask for ever touches the disk), the second when you need an application's field.
Error messages
tcpdump's messages are short. The ones produced in the lab, with what causes each:
tcpdump: can't parse filter expression: syntax error tcp port / port 80 and / (port 80 : an unfinished expression
tcpdump: can't parse filter expression: syntax error tcp[tcpflags] & syn != 0 and tcp[tcpflags] == syn : the name is tcp-syn, with the tcp- prefix
tcpdump: unknown host 'nosuch.invalid' host NAME: tcpdump tried to resolve it. Use an address, or the name must resolve
tcpdump: invalid IPv4 address '999.1.1.0' src net 999.1.1.0/24
tcpdump: illegal port number 70000 > 65535 port 70000
tcpdump: unknown ether host 'zz' ether host zz:zz: not a MAC address
tcpdump: unknown ip proto 'nosuch' ip proto nosuch
tcpdump: data size must be 1, 2, or 4 tcp[0:3] = 1
tcpdump: expression rejects all packets icmp6 and ip: a contradiction (a packet cannot be both). A useful message, since it catches a filter that could never match
tcpdump: nosuchif: No such device exists -i nosuchif: the interface name (list them with tcpdump -D or ip -br link)
tcpdump: /nonexistent.pcap: No such file or directory -r with a wrong path
tcpdump: eth1: You don't have permission to perform this capture on that device (socket: Operation not permitted) as a normal user: use sudo, or grant the capabilities
The messages that do NOT exist are the dangerous ones. These are accepted without a word and match nothing, or something other than you expect: "tcp[1000] = 1" (offset beyond any packet), "ip6 and tcp[tcpflags] == tcp-syn" (upper-layer offsets on IPv6), "net 10.100.0.0/24" on a trunk (VLAN tag), "ether host ..." on -i any, "tcp port 80 and vlan" (vlan after the other terms), "(vlan or not vlan) and tcp". Also "portrange 100-50": accepted, and the lab found it matched ports 53 and 80, so reversed ranges are quietly sorted. When the result is zero, suspect the filter before the network.
One shell error deserves a repeat. An unquoted parenthesis never reaches tcpdump:
bash: -c: line 1: syntax error near unexpected token `('
Recipes
All of the following ran against the reference capture or live in the lab, with the counts shown:
New connection attempts to a port: tcp dst port 22 and tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn
Servers accepting (SYN-ACK): tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack) 10 SYNs against 7 SYN-ACKs: three attempts were not accepted
Refused connections (RST from the server's port): tcp[tcpflags] & tcp-rst != 0 and src port 3306 2
Scans with odd flags: tcp[tcpflags] == 0 or tcp[tcpflags] & (tcp-fin|tcp-push|tcp-urg) == (tcp-fin|tcp-push|tcp-urg)
Ports that answered "unreachable": icmp[icmptype] == icmp-unreach 2
DNS failures only: udp src port 53 and udp[11] & 0x0f == 3 NXDOMAIN, 2 in the lab
DNS responses only / queries only: udp port 53 and udp[10] & 0x80 != 0 / == 0 4 and 4
HTTP POSTs (credentials in the clear): tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354
SSH on any port: tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x5353482d
Who starts TLS (ClientHello): tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16 and tcp[((tcp[12:1] & 0xf0) >> 2)+5] = 0x01
Fragments: ip[6:2] & 0x3fff != 0 3
Low-TTL packets (traceroute, loops): ip[8] < 5 2
DSCP-marked traffic (EF): ip[1] >> 2 == 46 2
Is the machine answering? Replies leaving: tcpdump -i eth1 -Q out 'icmp[icmptype] == icmp-echoreply'
IPv6 SYNs (do not use tcp[...]): ip6[6] == 6 and ip6[40+13] == 2 2
IPv6 TCP, hidden headers included: ip6 protochain 6 18
Traffic in a VLAN: vlan 100 and ... (vlan FIRST) 17
Neighbor discovery: icmp6[icmp6type] == icmp6-neighborsolicit or icmp6[icmp6type] == icmp6-neighboradvert
Everything except your own ssh session: not port 22 (and not (vlan and port 22) on a trunk)
Common problems
It shows nothing, but the traffic is there the filter cannot see it: VLAN tag (add vlan first), IPv6 with tcp[...] (use ip6[...]), -i any with an ether filter, an extension header, fragments, or a snaplen too short for a payload test
Count differs from tshark tshark -Y reads all layers; BPF does not. Compare with a display filter that matches the same thing, and find which of the traps above applies
"icmp or arp and ether broadcast" returns too little and/or have equal precedence: parenthesize every mixed expression
net 10.100.0.0/24 finds nothing on a trunk port tagged frames: vlan and net 10.100.0.0/24
tcp port 80 and vlan finds nothing vlan must come first: vlan and tcp port 80
port 9999 shows only one packet of a big datagram fragments: ip[6:2] & 0x3fff != 0 finds the rest
tcpdump says "syntax error" an unfinished expression, or the flag names without the tcp- prefix (tcp-syn, not syn), or unquoted parentheses
"expression rejects all packets" the filter is contradictory (icmp6 and ip). Rewrite it
HTTP filter sees GET but not POST, or the reverse the exact bytes: "GET " has a trailing space (0x47455420), and the request must start in the first data packet
Payload filter on a saved file finds nothing the file was captured with a short snaplen. Recapture with -s 0
Large frames "above the MTU" offloading merges segments before tcpdump sees them: expected on the sending/receiving host; ethtool -K iface gro off for tests
Counts from -i any are doubled packets are listed on every interface they cross (the VLAN sub-interface): use -i with one interface, or add "and not vlan"/interface logic
The capture misses packets under load "dropped by kernel": narrow the filter, write with -w, raise -B, use a faster disk or ring files (-C, -W)
Filter works on one machine, not another different link type (Ethernet, cooked, a tunnel), or an old libpcap: check the first line of the output (link-type) and tcpdump --version
Tips
Capture once into a file, then ask questions with -r and a filter. Counting with wc -l is the cheapest way to check a filter.
Always quote the filter, and always parenthesize when you mix and with or: they have equal precedence.
Put vlan first on trunk ports, and check the link type (the first line) before you trust ether filters.
Remember IPv6: tcp[...] and udp[...] byte offsets do not apply to IPv6. Use ip6[...] with 40, or protochain, and test both protocols.
When a filter reads the payload, capture with -s 0. A short snaplen drops the match without a single warning.
Use the named fields (tcpflags, icmptype, tcp-syn) rather than magic numbers, and keep the long expressions in a -F file with comments.
When in doubt, tcpdump -d shows exactly what the filter compiles to, and a second tool (tshark -Y) shows what is really in the file.
Use BPF to capture less, and display filters to understand more. Never capture more than you need: traffic can contain passwords, and a filtered file is also a safer file to hand to somebody.
Only capture on networks you are authorized to monitor.
