🕸️ Your Own Networks on OpenStack: Subnets, Routers, Ports and Floating IPs (Neutron)
Until now every server lived on the network DevStack made for you. Build your own: a network, a subnet, a router with a leg on the outside, ports with fixed addresses, two NICs in one server. See who can reach whom, what address the outside world sees, and which errors you get when you tear it down in the wrong order. Run for real
💡 How this page was made. Everything shown as output was run for real on 8 October 2026 on the DevStack cloud from the earlier pages (stable/2026.1, one machine, 4 CPUs, 8 GB of RAM, networking by OVN). Commands run as the ordinary demo user, except lines marked devstack-admin (an administrator) and the ovn-nbctl lines (run with sudo on the DevStack machine itself). Guests are tiny CirrOS instances. The ssh commands are shortened, and transcripts say where they are trimmed. All of the networks made on this page were deleted at the end, in the order shown in the last section.
The pieces, in plain words
Neutron, the networking service, gives you a handful of objects. Learn these five and the rest is detail:
- Network: a virtual switch, one flat Ethernet segment. Servers on it can talk to each other directly, and nobody else can hear them.
- Subnet: an IP range (and DHCP settings, gateway, DNS) that belongs to one network. A network can have several, for example one IPv4 and one IPv6.
- Port: one plug on that switch. It has a MAC address and one or more IP addresses. A server's network card is a port.
- Router: connects networks to each other and to the outside. Without one, a network is an island.
- Floating IP: a public address you attach to a server, so the outside can reach it (see the first-instance page).
What is already there
DevStack made two networks and one router. Here is what the demo user starts with:
$ openstack network list -c Name -c Subnets +---------+----------------------------------------------------------------------------+ | Name | Subnets | +---------+----------------------------------------------------------------------------+ | private | 16da1959-835a-471b-a383-1342ec1e84a7, ed48ccc7-edbd-476d-a011-8154439e0eb2 | | public | 0900f602-1ded-41f9-98cf-7c20125f3389, 5779a8f6-7e49-44f2-825b-c7ef21f0cfea | +---------+----------------------------------------------------------------------------+ (trimmed: this DevStack has two subnets on each network, one IPv4 and one IPv6) $ openstack subnet show private-subnet -c cidr -c gateway_ip -c allocation_pools -c dns_nameservers allocation_pools: 10.0.0.2-10.0.0.62 cidr: 10.0.0.0/26 dns_nameservers: 1.1.1.1, 9.9.9.9 gateway_ip: 10.0.0.1 $ openstack router show router1 -c external_gateway_info -c interfaces_info (trimmed) external IPs: 172.24.4.42 and 2001:db8::289, enable_snat: true interfaces: 10.0.0.1 and fd28:9fa8:c0f3::1 (both on network "private") $ OS_CLOUD=devstack-admin openstack network show public -c router:external -c provider:network_type -c mtu mtu: 1500 provider:network_type: flat router:external: true
- private (10.0.0.0/26) is where your servers have lived so far. It belongs to your project.
- public is the outside, marked router:external: true. In this lab it is a flat network, which really means the br-ex bridge on the DevStack machine (172.24.4.0/24). In a real cloud it is the datacenter's public range.
- router1 has one leg on private and one on public. That is what lets servers on private reach the outside at all.
💡 MTU. Networks between servers are tunnelled (this is the geneve type you will see below), so a tenant network has an MTU of 1442, not 1500: 58 bytes go on the tunnel header. If your own guest images misbehave on big packets only, look here first. DHCP hands the right value to the guest, so normally you never notice.
Step 1: a network and a subnet
openstack network create app-net openstack subnet create --network app-net --subnet-range 192.168.50.0/24 --dns-nameserver 1.1.1.1 app-subnet
$ openstack network create app-net -c name -c mtu -c status -c shared -c router:external -f yaml
mtu: 1442
name: app-net
router:external: false
shared: false
status: ACTIVE
$ openstack subnet create --network app-net --subnet-range 192.168.50.0/24 --dns-nameserver 1.1.1.1 app-subnet \
-c name -c cidr -c gateway_ip -c allocation_pools -c dns_nameservers -c enable_dhcp -c ip_version -f yaml
allocation_pools:
- end: 192.168.50.254
start: 192.168.50.2
cidr: 192.168.50.0/24
dns_nameservers:
- 1.1.1.1
enable_dhcp: true
gateway_ip: 192.168.50.1
ip_version: 4
name: app-subnet
$ OS_CLOUD=devstack-admin openstack network show app-net -c provider:network_type -c provider:segmentation_id -f yaml
provider:network_type: geneve
provider:segmentation_id: 59169
$ openstack port list --network app-net -c "Fixed IP Addresses" -c Status
+-----------------------------------------------------------------------------+--------+
| Fixed IP Addresses | Status |
+-----------------------------------------------------------------------------+--------+
| ip_address='192.168.50.2', subnet_id='f6a44b85-0c9d-4526-8918-a568bd0303a3' | DOWN |
+-----------------------------------------------------------------------------+--------+
$ sudo ovn-nbctl ls-list
0c7a5c29-86d1-4dab-8f43-871ace2d0674 (neutron-0c7a5c29-86d1-4dab-8f43-871ace2d0674)
61096247-8232-4bbd-bf97-765cfafeb5c1 (neutron-61096247-8232-4bbd-bf97-765cfafeb5c1)
666227c5-dc27-4729-80d4-3c4a57a30671 (neutron-666227c5-dc27-4729-80d4-3c4a57a30671)
- The subnet is a separate object: the network alone is only the switch, and the addresses, gateway and DHCP settings all come from the subnet. (I did not boot a server on a network without a subnet, so I can't show what that looks like.)
- Gateway 192.168.50.1 and a DHCP pool .2 to .254 were chosen for us. The gateway address is just reserved: there is no router yet.
- DNS: remember page 3, where the guests had no DNS until the subnet said so. Set --dns-nameserver now so the guests resolve names from the first boot. (I did not create a subnet without it on this page.)
- Admin view: the network is geneve, with its own tunnel ID. Tenants never choose this. In ovn-nbctl ls-list there is now one more logical switch, neutron-0c7a5c29-..., matching the network's ID.
- A port appeared by itself: 192.168.50.2, status DOWN. It is the DHCP/metadata port of the network. It stays DOWN because no server is plugged into it, only the infrastructure.
Step 2: a router with a leg on the outside
openstack router create app-router openstack router set --external-gateway public app-router openstack router add subnet app-router app-subnet
$ openstack router create app-router -c name -c status -c external_gateway_info -c routes -f yaml
external_gateway_info: null
name: app-router
routes: []
status: ACTIVE
$ openstack router set --external-gateway public app-router
$ openstack router add subnet app-router app-subnet
$ openstack router show app-router -c status -c external_gateway_info -c interfaces_info -f yaml
external_gateway_info:
enable_snat: true
external_fixed_ips:
- ip_address: 172.24.4.243
subnet_id: 5779a8f6-7e49-44f2-825b-c7ef21f0cfea
- ip_address: 2001:db8::15e
subnet_id: 0900f602-1ded-41f9-98cf-7c20125f3389
network_id: 666227c5-dc27-4729-80d4-3c4a57a30671
interfaces_info:
- ip_address: 192.168.50.1
port_id: fdbde99d-c2da-4c72-a056-1c31782281e8
subnet_id: f6a44b85-0c9d-4526-8918-a568bd0303a3
status: ACTIVE
$ openstack port list --router app-router -c "Fixed IP Addresses" -c Status
+-----------------------------------------------------------------------------+--------+
| Fixed IP Addresses | Status |
+-----------------------------------------------------------------------------+--------+
| ip_address='192.168.50.1', subnet_id='f6a44b85-0c9d-4526-8918-a568bd0303a3' | ACTIVE |
+-----------------------------------------------------------------------------+--------+
- The external gateway is the router's leg on public. It took the addresses 172.24.4.243 (and an IPv6 one) from the public range. Those are the addresses the outside sees when your servers call out (see step 4).
- enable_snat: true: the router rewrites the source of outgoing traffic to its own external address. This is the cloud's version of the NAT box in a home router.
- router add subnet created a port on the subnet, with the gateway address 192.168.50.1. A router is nothing but a set of ports with a routing table.
What Neutron really built on the DevStack machine, in OVN's own tools:
$ sudo ovn-nbctl show (only the app-router part)
router 5d1601be-b437-4f62-95be-d711ec233045 (neutron-fb01c192-e688-4fb8-86c1-7bf86bd43300) (aka app-router)
port lrp-e3817f47-0cba-4244-89a5-87b308db6ed7
mac: "fa:16:3e:58:b8:89"
networks: ["172.24.4.243/24", "2001:db8::15e/64"]
gateway chassis: [b20d8aa6-d887-4256-9d7d-d0d1c847024d]
port lrp-fdbde99d-c2da-4c72-a056-1c31782281e8
mac: "fa:16:3e:f2:3f:2f"
networks: ["192.168.50.1/24"]
nat ceea6432-aaa2-494a-afc3-2b53b107691d
external ip: "172.24.4.243"
logical ip: "192.168.50.0/24"
type: "snat"
$ sudo ovn-nbctl lr-nat-list neutron-fb01c192-e688-4fb8-86c1-7bf86bd43300
TYPE GATEWAY_PORT EXTERNAL_IP EXTERNAL_PORT LOGICAL_IP EXTERNAL_MAC LOGICAL_PORT
snat 172.24.4.243 192.168.50.0/24
$ sudo ovn-nbctl lr-route-list neutron-fb01c192-e688-4fb8-86c1-7bf86bd43300
IPv4 Routes
Route Table <main>:
0.0.0.0/0 172.24.4.1 dst-ip
IPv6 Routes
Route Table <main>:
::/0 2001:db8::2 dst-ip
Two router ports (one outside with a gateway chassis, one inside), one snat rule for 192.168.50.0/24 and a default route to the machine's br-ex address (172.24.4.1). OVN calls these logical routers: there is no Linux network namespace and no iptables for them (ip netns list is empty here). OVN programs flows into Open vSwitch instead. Note that OVN knows the router as neutron- plus its UUID, not as app-router, which cost me one failed command.
Step 3: servers on the new network
openstack server create --image cirros-0.6.3-x86_64-disk --flavor cirros256 --network app-net --key-name mykey --security-group ssh-ping vm-app openstack server create ... --network private ... vm-priv
Both got a floating IP. Inside vm-app:
$ openstack server list -c Name -c Status -c Networks
+---------+--------+---------------------------------------------------------+
| Name | Status | Networks |
+---------+--------+---------------------------------------------------------+
| vm-priv | ACTIVE | private=10.0.0.42, fd28:9fa8:c0f3:0:f816:3eff:fe42:aa4a |
| vm-app | ACTIVE | app-net=192.168.50.234 |
+---------+--------+---------------------------------------------------------+
$ ssh cirros@172.24.4.143 'ip -4 addr show eth0 | grep inet; ip route; cat /etc/resolv.conf; cat /sys/class/net/eth0/mtu'
inet 192.168.50.234/24 brd 192.168.50.255 scope global dynamic noprefixroute eth0
default via 192.168.50.1 dev eth0 src 192.168.50.234 metric 1002
169.254.169.254 via 192.168.50.2 dev eth0 src 192.168.50.234 metric 1002
192.168.50.0/24 dev eth0 scope link src 192.168.50.234 metric 1002
nameserver 1.1.1.1
1442
$ ssh cirros@172.24.4.143 'ping -c2 -W2 192.168.50.1'
64 bytes from 192.168.50.1: icmp_seq=1 ttl=254 time=2.79 ms
64 bytes from 192.168.50.1: icmp_seq=2 ttl=254 time=1.19 ms
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
$ ssh cirros@172.24.4.143 'ping -c2 -W2 10.0.0.42' (vm-priv, on the other network, other router)
2 packets transmitted, 0 received, 100% packet loss, time 1011ms
- DHCP gave the guest everything the subnet says: its address, the router as default route, the DNS server, the MTU (1442) and a route to the metadata service 169.254.169.254 through the network's DHCP port.
- It can reach its own router (192.168.50.1).
- It cannot reach vm-priv. Two networks behind two different routers know nothing about each other. That is the isolation you pay a cloud for, and nothing was configured to get it.
Step 4: what address does the outside see?
A web server on the DevStack machine logs who connects. The servers fetch a page from it:
(on the DevStack machine, a web server that logs who connects:) $ python3 -m http.server 8000 --bind 172.24.4.1 (vm-app and vm-priv fetch http://172.24.4.1:8000/ while both have a floating IP:) 172.24.4.143 - - [09/Oct/2026 00:13:07] "GET / HTTP/1.1" 200 - <- vm-app, its floating IP 172.24.4.187 - - [09/Oct/2026 00:13:07] "GET / HTTP/1.1" 200 - <- vm-priv, its floating IP (floating IP removed from vm-app, it fetches again:) $ openstack server remove floating ip vm-app 172.24.4.143 172.24.4.243 - - [09/Oct/2026 00:19:04] "GET / HTTP/1.1" 200 - <- the router's own address (SNAT) (the two routers, for comparison:) app-router external IP: 172.24.4.243 router1 external IP: 172.24.4.42
- With a floating IP the server's traffic leaves as the floating IP.
- Without one, it leaves as the router's address (172.24.4.243), shared by everything behind that router. Many servers, one face. That is also why nothing outside can start a connection to them: nobody knows which server it would be.
- router1 and app-router have different external addresses. Each router has its own.
Step 5: two networks, one router
Add a second network, db-net (192.168.60.0/24), to the same router:
$ openstack network create db-net -f value -c name db-net $ openstack subnet create --network db-net --subnet-range 192.168.60.0/24 --dns-nameserver 1.1.1.1 db-subnet -f value -c cidr 192.168.60.0/24 $ openstack router add subnet app-router db-subnet (vm-db is then booted on db-net: 192.168.60.134) $ ssh cirros@<vm-app> 'ping -c2 -W2 192.168.60.134; ip route | head -2' 64 bytes from 192.168.60.134: icmp_seq=1 ttl=63 time=4.10 ms 64 bytes from 192.168.60.134: icmp_seq=2 ttl=63 time=2.86 ms 2 packets transmitted, 2 received, 0% packet loss, time 1002ms default via 192.168.50.1 dev eth0 src 192.168.50.234 metric 1002 169.254.169.254 via 192.168.50.2 dev eth0 src 192.168.50.234 metric 1002 $ ssh cirros@<vm-db> 'ping -c2 -W2 192.168.50.234' 64 bytes from 192.168.50.234: icmp_seq=1 ttl=63 time=2.91 ms 64 bytes from 192.168.50.234: icmp_seq=2 ttl=63 time=1.97 ms 2 packets transmitted, 2 received, 0% packet loss, time 1002ms
Now vm-app and vm-db talk to each other. The TTL of 63 (64 minus one) shows they went through the router. No route, no firewall rule and no changes inside the guests. A router with two connected subnets simply routes between them.
⚠️ Security groups still apply. Routing the networks together does not open any ports. A server on db-net is still protected by its own security group, exactly as before. I did not run a test that proves it here (the one connection I tried to a closed port timed out, but a closed port would also do that), so treat this line as the documented behaviour and not as something this page measured.
Step 6: ports with a fixed address
Servers get a random free address by default. When you need one that never changes (a database address, a firewall rule), create the port first and plug the server in:
$ openstack port create --network app-net --fixed-ip subnet=app-subnet,ip-address=192.168.50.100 --security-group ssh-ping web-port \
-c name -c fixed_ips -c mac_address -c status -f yaml
fixed_ips:
- ip_address: 192.168.50.100
subnet_id: f6a44b85-0c9d-4526-8918-a568bd0303a3
mac_address: fa:16:3e:ab:7d:aa
name: web-port
status: DOWN
$ openstack server create --image cirros-0.6.3-x86_64-disk --flavor cirros256 --nic port-id=web-port --key-name mykey --wait vm-fixed
$ openstack server list -c Name -c Networks (trimmed to the new server)
| vm-fixed | app-net=192.168.50.100 |
$ openstack port show web-port -c status -c device_owner -f yaml
device_owner: compute:nova
status: ACTIVE
$ openstack port create --network app-net --fixed-ip subnet=app-subnet,ip-address=192.168.50.100 dup
ConflictException: 409: Client Error for url: http://10.0.2.15/networking/v2.0/ports, IP address 192.168.50.100 already allocated in subnet f6a44b85-0c9d-4526-8918-a568bd0303a3
- --nic port-id=... boots the server on that port. It got 192.168.50.100, the port went from DOWN to ACTIVE and its owner became compute:nova.
- A second port asking for the same address gets a clean 409.
- Ports have their own security groups (--security-group was given to the port, not the server).
💡 Deleting a port that a running server uses. In the cleanup test I ran openstack port delete web-port while vm-fixed was still using it. The command printed nothing, no error, and the port was gone afterwards. I did not look at what happened inside that server, so I cannot tell you what it did. Detach or delete the server first.
Step 7: a server with two network cards
openstack server create ... --network app-net --network db-net ... vm-two
$ openstack server create ... --network app-net --network db-net ... vm-two
$ openstack server show vm-two -c addresses -f value (a floating IP was added to the app-net address)
{'app-net': ['192.168.50.165', '172.24.4.154'], 'db-net': ['192.168.60.152']}
$ ssh cirros@172.24.4.154 'ip -4 -o addr show | grep -v " lo "; ip route'
2: eth0 inet 192.168.50.165/24 brd 192.168.50.255 scope global dynamic noprefixroute eth0
3: eth1 inet 192.168.60.152/24 brd 192.168.60.255 scope global dynamic noprefixroute eth1
default via 192.168.50.1 dev eth0 src 192.168.50.165 metric 1002
default via 192.168.60.1 dev eth1 src 192.168.60.152 metric 1003
169.254.169.254 via 192.168.50.2 dev eth0 src 192.168.50.165 metric 1002
169.254.169.254 via 192.168.60.2 dev eth1 src 192.168.60.152 metric 1003
192.168.50.0/24 dev eth0 scope link src 192.168.50.165 metric 1002
192.168.60.0/24 dev eth1 scope link src 192.168.60.152 metric 1003
CirrOS brought both interfaces up with DHCP. Look at the routing table: two default routes, one per interface, with metrics 1002 and 1003. The first wins, which is fine here. On a real Linux image it is a classic trap: one default route per NIC, and answers leaving by the "wrong" interface. Decide which NIC carries the default route and configure the rest by hand.
Errors you will meet
$ openstack router add subnet app-router private-subnet (private-subnet is already on router1) ConflictException: 409: Client Error for url: http://10.0.2.15/networking/v2.0/routers/fb01c192-.../add_router_interface, IP address 10.0.0.1 already allocated in subnet ed48ccc7-... (a second run, with throwaway names: a-net and b-net both 192.168.50.0/24, a-sub already on router a-rtr) $ openstack router add subnet a-rtr b-sub BadRequestException: 400: ... Bad router request: Cidr 192.168.50.0/24 of subnet 2065b88e-... overlaps with cidr 192.168.50.0/24 of subnet 74f21c51-.... $ openstack network delete a-net Failed to delete network with name or ID 'a-net': ConflictException: 409: ... Unable to complete operation on network 7e2c6368-.... There are one or more ports still in use on the network, id for these ports is: e8690009-.... $ openstack subnet delete a-sub Failed to delete subnet with name or ID 'a-sub': ConflictException: 409: ... Unable to complete operation on subnet 74f21c51-...: One or more ports have an IP allocation from this subnet. $ openstack router delete a-rtr Failed to delete router with name or ID 'a-rtr': ConflictException: 409: ... Router 81c0d963-... has ports still attached: e8690009-... (the URLs and IDs are shortened with "..." in this transcript)
- Same address range on one router. The second subnet is refused with Cidr ... overlaps with cidr .... A router has to tell its networks apart.
- Same range on two different networks, no router involved: allowed. I created 192.168.50.0/24 on db-net while app-net already had it, and Neutron accepted it. Two isolated islands may reuse the same addresses. The trouble starts only on the day you want to connect them.
- private-subnet could not be added to app-router: its gateway address is already used by router1.
- You cannot delete something that is still connected: a network with ports, a subnet with addresses, a router with interfaces. Each error even names the port ID that is in the way.
Tearing it down, in order
It always goes from the edge to the centre:
- servers (they hold ports),
- floating IPs (they stay allocated after a server is deleted, as on the first-instance page; all four were still listed with Fixed IP: None),
- extra ports you made yourself,
- router interfaces (router remove subnet), then the gateway (router unset --external-gateway),
- the router, the subnets, the networks.
$ openstack server delete --wait vm-app vm-priv vm-db vm-fixed vm-two $ openstack floating ip list -c "Floating IP Address" -c "Fixed IP Address" | 172.24.4.154 | None | | 172.24.4.194 | None | | 172.24.4.187 | None | | 172.24.4.143 | None | $ openstack floating ip delete $(openstack floating ip list -f value -c ID) $ openstack router remove subnet app-router app-subnet $ openstack router remove subnet app-router db-subnet $ openstack router unset --external-gateway app-router $ openstack router delete app-router $ openstack subnet delete clash app-subnet db-subnet $ openstack network delete app-net db-net $ openstack network list -c Name | private | | public | $ sudo ovn-nbctl ls-list; sudo ovn-nbctl lr-list (back to the original two switches and one router) 61096247-... (neutron-61096247-...) 666227c5-... (neutron-666227c5-...) cf59f095-... (neutron-15c9a9dd-...)
OVN is back to exactly the two switches and one router it started with, so nothing was left behind.
What this page did not cover
- Provider and VLAN networks, shared networks, RBAC sharing of a network with other projects.
- Static routes on a router, HA routers, distributed floating IPs, port forwarding.
- Load balancers (Octavia), VPNaaS and firewall as a service.
- Security group tests between networks (see the note above), port security and allowed address pairs.
- IPv6 for tenant networks (DevStack's own IPv6 subnets were never used), subnet pools, QoS.
- Real Linux guests with two NICs. Only CirrOS was used.
- Horizon's network topology view (not opened).
The cheat sheet
openstack network create NAME a virtual switch openstack subnet create --network N --subnet-range CIDR --dns-nameserver IP NAME openstack router create NAME a router openstack router set --external-gateway public NAME its leg to the outside (SNAT on) openstack router add subnet ROUTER SUBNET connect a subnet to it openstack port create --network N --fixed-ip subnet=S,ip-address=IP P a plug with a chosen address openstack server create ... --nic port-id=P boot on that port openstack server create ... --network A --network B two network cards openstack floating ip create public ; openstack server add floating ip S IP openstack network list / subnet list / port list / router list what exists openstack port list --router R / --network N / --server S which plugs belong to what openstack router remove subnet R S ; router unset --external-gateway R undo, before deleting sudo ovn-nbctl show (DevStack machine) the OVN view of all of it
Two networks, one router, and suddenly the database can see the web server. Somewhere a security auditor feels a disturbance in the force.
← Persistent Volumes on OpenStack: Cinder, Snapshots, Resize and Boot from Volume · ↑ All tutorials · Automate Your OpenStack Cloud with Terraform: A Whole Network and Server from One File →