WireGuard With Overlapping Client Networks

WireGuard can be a great tool for allowing one company to connect to a few select services in another company’s network. For example, you might host a database (or other similar network service) in your own network that you want to enable clients to connect to securely, without exposing the service to drive-by attacks from the public Internet. Or you might need to access a private service in each of your client’s own networks, for example to make web hooks calls (or calls to other similar network services) from the servers in your network to private servers in your clients’ networks.

There’s no hard limit to the number of different customers you can connect to this way — even with a moderately-sized server, you can easily support thousands of concurrent WireGuard clients (provided that each client on average only sends or receives traffic intermittently, like only a few HTTP requests or DB queries per client per minute) — you generally won’t be limited by anything related to WireGuard in a situation like this, but rather by the server’s own CPU and network capacity to make or serve client requests.

The main trick in this case is dealing with overlaps in the IP address space used by each client. For example, say you have two customers (Customer A and Customer B) who both want you to connect their own private server (Host A for Customer A, and Host B for Customer B) in their own network, both of which have the same IP address of 10.0.0.1.

If you need to make outgoing connections to customers (eg make HTTP requests or DB queries), you need to be able to distinguish between a connection to 10.0.0.1 that should be sent to Host A, versus a connection to 10.0.0.1 that should be sent to Host B. Or if you need to receive incoming connections from customers (eg receive HTTP requests or DB queries) you need to distinguish between a connection from 10.0.0.1 where the response needs to be sent to Host A, versus a connection from 10.0.0.1 where the response needs to be sent to Host B.

Also, each customer may want traffic from your server to appear in their own network as if it originated from a specific address (such as 10.0.0.2), so that they don’t have to deal with the potential for IP address collisions on their side of the connection. A network diagram of this (with your server labeled Host C) might look like the following:

WireGuard Overlapping Client Networks — Figure 1. Two overlapping WireGuard networks

This article will show you how to use the built-in firewall and routing tools on Linux to allow for such connections with overlapping address space (see the WireGuard Containers for Overlapping Networks article for how to do this instead using Docker containers).

First we’ll cover how to handle Inbound Connections (such as HTTP requests or DB queries from our clients to us), then Outbound Connections (HTTP requests or DB queries from us to our clients), and then both Inbound and Outbound connections at the same time (with is really just a straight-forward combination of the first two techniques).

The core techniques that we are going to use in each case are the same:

Use a separate WireGuard interface for each client network.
Use a separate custom routing table for each WireGuard interface.
Use policy routing to determine when to apply each custom routing table.
Use NAT (Network Address Translation) firewall rules to rewrite potentially conflicting client IP addresses to use our own internal address space (over which we have full control, and clients need know nothing about).

Base Configurations

Before we apply these techniques, the base WireGuard configurations we’ll start with will be the following:

On Host A

On Customer A’s private server, Host A, they want to use the following WireGuard config file:

# /etc/wireguard/wg0.conf

# local settings for Host A
[Interface]
PrivateKey = AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEE=
Address = 10.0.0.1

# remote settings for Host C
[Peer]
PublicKey = jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw=
Endpoint = 203.0.113.2:51821
AllowedIPs = 10.0.0.2
PersistentKeepalive = 25

They’ve asked us to provide a public static IP address and port they can use to connect to our WireGuard endpoint on Host C, and we’ve provided them with 203.0.113.2:51821.

Note

At least one side — either the customer or us — must share a static public IP address and port that the other side can use for an Endpoint setting. For all the examples in this article, we’ll share our public IP address (203.0.113.2) and WireGuard listen port for the customer to use as the Endpoint in their config files; but you could instead require that customers provide their public IP address and WireGuard listen port for you to use instead. In that case, you wouldn’t have to provide your own server’s public IP address and WireGuard listen port to the customer.

On Host B

On Customer B’s private server, Host B, they want to use the following WireGuard config file — almost identical to what Customer A wants:

# /etc/wireguard/wg0.conf

# local settings for Host B
[Interface]
PrivateKey = ABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBFA=
Address = 10.0.0.1

# remote settings for Host C
[Peer]
PublicKey = jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw=
Endpoint = 203.0.113.2:51822
AllowedIPs = 10.0.0.2
PersistentKeepalive = 25

Customer B has also asked us to provide them with a public static IP address and port that they can use to connect to our WireGuard endpoint on Host C, and we’ve provided them with 203.0.113.2:51822 (same IP address as for Customer A, but a different port).

Notice that the only differences between the configuration for Host A and Host B is that each uses a different PrivateKey (one that each customer will keep secret, and provide us only with the corresponding public key), and each uses a different port in the Endpoint setting (51821 for Host A, 51822 for Host B).

Since we are going to use a different WireGuard interface for each client on our own server, we must provide each customer with a different WireGuard endpoint port to connect to. (The endpoint port we provide them with should generally match the listen port of the WireGuard interface we use for them on our server, unless we do some additional port mapping for it within our own network.)

To Host A

For our end of the connection to Host A, Customer A wants us to use the WireGuard IP address 10.0.0.2, with a WireGuard configuration that corresponds directly to their own, like so:

# /etc/wireguard/wg-a.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.0.0.2
ListenPort = 51821

# remote settings for Host A
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.1

We’re going to adjust this configuration, however, to use our own chosen WireGuard IP address. These are the core configuration settings that the customer gets to tell us to use:

their WireGuard public key: /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
their WireGuard address: 10.0.0.1
our WireGuard address: 10.0.0.2

But these are the settings we get to decide for ourselves:

our WireGuard private key: CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
our WireGuard public key: jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw= (calculated deterministically from our private key)
our public endpoint IP address: 203.0.133.2
our listen port: 51821
our translated WireGuard address: 10.200.200.1
their translated WireGuard address: 10.100.100.1
a custom routing table for them: 41

We’re going to translate the WireGuard IP address we use to identify their server from 10.0.0.1 to 10.100.100.1, and translate the WireGuard IP address we use for our own server from 10.0.0.2 to 10.200.200.1. Of the settings we decide for ourselves, we only have to share our public key (jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw=), public IP address (203.0.133.2), and listen port (51821) with the customer.

In our base WireGuard config for Customer A, we’ll put the translated address for our server (10.200.200.1) in the Address field — not the address the customer is expecting (10.0.0.2); we will translate it to the customer’s expected address with firewall rules later. We will, however, keep their chosen WireGuard address for their own server (10.0.0.1) in the AllowedIPs setting:

# /etc/wireguard/wg-a.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.200.200.1
ListenPort = 51821

Table = 41
PreUp = ip rule add from 10.200.200.1 table 41 priority 100
PostDown = ip rule del from 10.200.200.1 table 41 priority 100

# remote settings for Host A
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.1

We’ll also include a custom routing table number (41) for the customer in our WireGuard config, as the Table setting. This will ensure that wg-quick doesn’t add any routes for this WireGuard interface to the main routing table of our server; instead, it will add them to table 41.

Finally, we’ll use a PreUp command to add a policy routing rule to configure our server to use this custom routing table to route any traffic we send that originates from our chosen WireGuard address for our server (10.200.200.1). This will ensure that our server will use the correct routing table for traffic with a destination address of 10.0.0.1 when we want 10.0.0.1 to go to Host A.

To Host B

For our end of the connection to Host B, Customer B wants us to use the same IP addresses as Customer A. This is the WireGuard configuration Customer B expects us to use:

# /etc/wireguard/wg-b.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.0.0.2
ListenPort = 51822

# remote settings for Host B
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.1

Notice that this is virtually identical to the WireGuard config Customer A expects us to use, except for our listen port and their public key.

These are the configuration settings that Customer B told us to use:

their WireGuard public key: fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
their WireGuard address: 10.0.0.1
our WireGuard address: 10.0.0.2

But these are the settings we get to decide for Customer B:

our WireGuard private key: CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
our WireGuard public key: jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw=
our public endpoint IP address: 203.0.133.2
our listen port: 51822
our translated WireGuard address: 10.200.200.2
their translated WireGuard address: 10.100.100.2
a custom routing table for them: 42

We can choose to use the same WireGuard key pair and public endpoint IP address as other customers, or we could use different key pairs and endpoint IP addresses for different customers.

Tip

The best practice is to use a separate key pair for each WireGuard interface, even for interfaces on the same machine. In this article, however, we’ll just use the same key pair for both interfaces (for simplicity’s sake, and to illustrate which settings have to be different for different customers, versus which settings can be the same). The risk to using the same key pair for different interfaces is that if the key pair leaks to an adversary (for example, if you later move one interface to a different server, and that other server is then compromised), the security of all interfaces using that same key pair will be compromised.

However, we must choose a different listen port, translated WireGuard addresses, and routing table for each customer. For customer B, we’ll translate their WireGuard address from 10.0.0.1 to 10.100.100.2, and translate our WireGuard address from 10.0.0.2 to 10.200.200.2. We’ll also select a distinct listen port of 51822, and routing table of 42:

# /etc/wireguard/wg-b.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.200.200.2
ListenPort = 51822

Table = 42
PreUp = ip rule add from 10.200.200.2 table 42 priority 100
PostDown = ip rule del from 10.200.200.2 table 42 priority 100

# remote settings for Host B
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.1

Nftables Config

For the firewall configuration of our server, we’ll use the nftables Base Configuration recommended by the WireGuard Nftables configuration guide, but using nftable’s set syntax to include the name and listen port of each WireGuard interface in the wg_iface and wg_port variables.

Also, for the Inbound Connections (and Inbound and Outbound) scenario, where we want to allow customers inbound access to a service on our own server, we’ll also include a rule that enables such access for each of our WireGuard interfaces (which actually will make the nftables configuration more similar to the configuration for “Endpoint B” from the Point to Point section of the WireGuard Nftables configuration guide). For this scenario, we’ll say that this service is listening on TCP port 8080:

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = { "wg-a", "wg-b" }
define wg_port = { 51821, 51822 }

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        meta l4proto { icmp, ipv6-icmp } accept
        # accept all packets that are part of an already-established connection
        ct state vmap { invalid : drop, established : accept, related : accept }
        # drop new connections over rate limit
        ct state new limit rate over 1/second burst 10 packets drop

        # accept all DHCPv6 packets received at a link-local address
        ip6 daddr fe80::/64 udp dport dhcpv6-client accept
        # accept all SSH packets received on a public interface
        iifname $pub_iface tcp dport ssh accept
        # accept all WireGuard packets received on a public interface
        iifname $pub_iface udp dport $wg_port accept

        # (for inbound scenarios only)
        # accept all packets to TCP port 8080 received on a WireGuard interface
        iifname $wg_iface tcp dport 8080 accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
        reject with icmpx type host-unreachable
    }
}
table inet nat {
}

In this base configuration, we’ll also include an empty nat table. In the sections below, we’ll add different chains and rules to this table for each specific scenario.

Tip

Instead of enumerating each individual WireGuard interface name in your nftables configuration, if you name all your WireGuard interfaces with a common prefix, such as wg-abc, wg-def, wg-ghi, etc, you can use simply specify that prefix with an asterisk as the definition for the wg_iface variable:

define wg_iface = "wg-*"

Similarly, instead of enumerating each individual WireGuard listen port in your nftables configuration, if you set aside a discrete UDP port range to use exclusively for WireGuard interfaces, you can simply specify that range as the definition for the wg_port variable:

define wg_port = 51820-51899

However, if you do this for the wg_port variable, we strongly recommend that you either choose a port range outside of the server’s own ephemeral port range (defined by the net.ipv4.ip_local_port_range kernel parameter); or that you configure your server to exclude this range from the ephemeral ports it may choose (by including this range in the net.ipv4.ip_local_reserved_ports kernel parameter). Otherwise, it’s possible that from time-to-time your server may use a port in your chosen WireGuard range (but not currently in use) for some other network service, thereby inadvertently allowing full public access to the service.

Inbound Connections

The first scenario we’ll consider is how to allow inbound access to a private server of ours from customers’ servers, while avoiding address collisions. In this example, we have multiple customers making HTTP requests to the same webserver running at TCP port 8080 on a private server in our network:

Inbound Access With Overlapping Networks — Figure 2. Inbound HTTP requests from two clients with the same WireGuard IP address

To enable inbound access to our server (Host C) from Customer A’s server (Host A), we need to add one DNAT (Destination Network Address Translation) firewall rule to our server, translating the destination IP address of incoming packets from Customer A’s WireGuard interface to use our own private IP address for this interface (10.200.200.1), instead of keeping the IP address the customer chose for us (10.0.0.2).

Similarly, to enable inbound access to our server from Customer B’s server (Host B), we need to add one DNAT firewall rule to translate the destination IP address of incoming packets from Customer B’s WireGuard interface to use our own private IP address for this interface (10.200.200.2), instead of keeping the IP address Customer B chose for us (which was the same as Customer A chose, 10.0.0.2).

These two rules are the minimum needed to satisfy this scenario. However, if these are the only rules we add, from the perspective of the service we want to expose on our server (listening for example at TCP port 8080), connections from Host A and Host B would both appear to originate from the same source IP address of 10.0.0.1. While this technically will work (as the policy routing rule we set up for each host’s traffic via their separate WireGuard interface configurations above can distinguish between packets returning to 10.0.0.1 on Host A versus 10.0.0.1 on Host B), we may want the logging or access control of the service we expose to be able to distinguish between traffic from each of our customers.

Therefore, usually we’d want to also add SNAT (Source Network Address Translation) firewall rules in situations like this, to translate the source IP address of incoming packets from each customer’s chosen address space to the private address space we’ve selected for the customer.

In this scenario, we’d want one SNAT rule to translate the source IP address of incoming packets from Customer A’s WireGuard interface to instead use the private IP address we’ve chosen for them (10.100.100.1); and a second SNAT rule to translate the source IP of incoming packets from Customer B’s WireGuard interface to instead use the private IP address we’ve chosen for Customer B (10.100.100.2).

To implement this with nftables, we’d make the following change to our base Nftables Config file, adding a nat chain with a prerouting hook for our incoming DNAT rules, and a nat chain with an input hook for our incoming SNAT rules:

table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        iifname "wg-a" dnat ip to 10.200.200.1
        iifname "wg-b" dnat ip to 10.200.200.2
    }
    chain input {
        type nat hook input priority 100; policy accept;
        iifname "wg-a" snat ip to 10.100.100.1
        iifname "wg-b" snat ip to 10.100.100.2
    }
}

Tip

When you have lots of rules grouped together in the same chain with the same structure, it’s more efficient to use nftable’s map functionality to express this logic within a single rule, rather than force nftables to sequentially evaluate many individual rules. To take advantage of this optimization, you’d rewrite the above DNAT and SNAT rules like the following:

table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        dnat ip to iifname map {
          "wg-a" : 10.200.200.1,
          "wg-b" : 10.200.200.2,
        }
    }
    chain input {
        type nat hook input priority 100; policy accept;
        snat ip to iifname map {
          "wg-a" : 10.100.100.1,
          "wg-b" : 10.100.100.2,
        }
    }
}

If you’re using iptables, you’d implement the above DNAT and SNAT rules with the following commands:

# iptables -t nat -A PREROUTING -i wg-a -j DNAT --to-destination 10.200.200.1
# iptables -t nat -A PREROUTING -i wg-b -j DNAT --to-destination 10.200.200.2
# iptables -t nat -A INPUT -i wg-a -j SNAT --to-source 10.100.100.1
# iptables -t nat -A INPUT -i wg-b -j SNAT --to-source 10.100.100.2

Tip

If using iptables, you may find it more convenient to run the iptables commands which apply to a particular WireGuard interface via PreUp commands included in the interface’s config file. If so, they would look like this for our wg-a interface config (symmetrically removing rules with PostDown commands that we’ve added with PreUp commands; and using %i to reference the interface’s name):

PreUp = iptables -t nat -A PREROUTING -i %i -j DNAT --to-destination 10.200.200.1
PostDown = iptables -t nat -D PREROUTING -i %i -j DNAT --to-destination 10.200.200.1
PreUp = iptables -t nat -A INPUT -i %i -j SNAT --to-source 10.100.100.1
PostDown = iptables -t nat -D INPUT -i %i -j SNAT --to-source 10.100.100.1

And this for our wg-b interface config:

PreUp = iptables -t nat -A PREROUTING -i %i -j DNAT --to-destination 10.200.200.2
PostDown = iptables -t nat -D PREROUTING -i %i -j DNAT --to-destination 10.200.200.2
PreUp = iptables -t nat -A INPUT -i %i -j SNAT --to-source 10.100.100.2
PostDown = iptables -t nat -D INPUT -i %i -j SNAT --to-source 10.100.100.2

When Customer A makes an HTTP request from Host A to the service listening on TCP port 8080 of our Host C, their initial TCP packet will have a source address of 10.0.0.1 (the address of their WireGuard interface), and a destination address of 10.0.0.2 (the WireGuard address they’re using for our server). When it emerges from the WireGuard tunnel on Host C via the wg-a interface, our nftables rules will rewrite its source address to 10.100.100.1, and its destination address to 10.200.200.1; and will allow it through to the HTTP service listening on port 8080.

When our HTTP service sends back reply TCP packets to Host A in response, the network stack on Host C will automatically flip-flop the source and destination addresses of the reply packets, using a source address of 10.200.200.1, and a destination address of 10.100.100.1. As these packets work their way through the network stack, netfilter‘s connection-tracking system will recognize these packets as part of the same connection for which it had previously applied NAT, and so automatically apply reverse NAT, rewriting their source address to 10.0.0.2, and destination address to 10.0.0.1.

However, it will rewrite these packets in two stages: First it will rewrite the destination address back to 10.0.0.1 (in the “output” stage). After this stage, it will re-apply the system’s routing rules — and the policy routing rule we set in the To Host A section earlier (to use our custom routing table 41) will still match the packet, as it will still have a source address of 10.200.200.1. But now with the packet’s destination address set to 10.0.0.1, the route for 10.0.0.1 in our custom routing table 41 (set up automatically by wg-quick to match the wg-a interface’s AllowedIPs settings) will route 10.0.0.1 out the wg-a interface.

Only after the routing decision has been made will netfilter move on to the stage where it rewrites the source address back to 10.0.0.2 (in the “postrouting” stage). It will rewrite it in time, however, for the packet to be sent back through the WireGuard tunnel with a source address of 10.0.0.2, and destination address of 10.0.0.1 — just as Customer A expects.

When Customer B does the same thing, and makes an HTTP request from Host B to the same HTTP service listening on port 8080 of Host C, their initial TCP packet will have the same source address of 10.0.0.1 and destination address of 10.0.0.2 that the initial packet from Host A had. However, packets from Host B will emerge on Host C from the wg-b interface — therefore our nftables rules will rewrite their addresses to the private addresses we’re using for Customer B: a source address of 10.100.100.2, and destination address of 10.200.200.2.

A similar thing will happen to reply packets for Host B as happened to reply packets for Host A — our HTTP service will initially send reply packets back to Host B with a source address of 10.200.200.2, and a destination address of 10.100.100.2. Netfilter will automatically rewrite the source address to 10.0.0.2, and destination address to 10.0.0.1 — using the same two-stage approach which allows the policy rule we set up in the To Host B section earlier to ensure that these packets are routed out the wg-b interface back to Host B (as the packets will have a source address of 10.200.200.2 and destination address of 10.0.0.1 at the time the routing decision for them is made).

Outbound Connections

The second scenario we’ll consider is how to allow outbound access from our server to private servers on our customers’ networks, while avoiding address collisions. In this example, we will make HTTP requests to multiple webservers running in our customers’ private networks from the same server in our network:

Outbound Access With Overlapping Networks — Figure 3. Outbound HTTP requests to two clients with the same WireGuard IP address

To enable outbound access from our server (Host C) to Customer A’s server (Host A), we need to add one SNAT firewall rule to our server, translating the source IP address of our packets outgoing through Customer A’s WireGuard interface to use the WireGuard IP address the customer chose for us (10.0.0.2), instead of our own private IP address for this interface (10.200.200.1).

Similarly, to enable outbound access from our server to Customer B’s server (Host B), we need to add one SNAT firewall rule to our server, translating the source IP address of our packets outgoing through Customer B’s WireGuard interface to use the WireGuard IP address Customer B chose for us (which was the same as Customer A chose, 10.0.0.2), instead of our own private IP address for this interface (10.200.200.2).

If the application we use to connect outbound to our customer’s servers can select a specific network interface to use by name when setting up the outbound connection, we actually don’t have to do anything more than add those two SNAT rules. For example, with cURL, we can specify wg-a for the --interface flag when connecting to Host A over our wg-a WireGuard interface to ensure the connection is routed via Customer A’s WireGuard tunnel:

$ curl --interface wg-a 10.0.0.1
hello world from customer A

Or specify --interface wg-b to connect to Host B via our wg-b WireGuard interface:

$ curl --interface wg-b 10.0.0.1
hello world from customer B

However, some applications do not allow you to select the specific network interface to use when setting up a network connection; plus often it’s simply more convenient to only have to specify a destination IP address, instead of both interface and IP address together. Therefore, usually we’d want to also add DNAT firewall rules, translating the destination IP address of outgoing packets from our private address space to the address space that customers have chosen for their own servers.

If we do this, we’ll be able to connect to Host A simply by using our private IP address for it:

$ curl 10.100.100.1
hello world from customer A

And the same for Host B:

$ curl 10.100.100.2
hello world from customer B

To make this work, we first need to add a route on our server for each private address (or network) that we want to use to access one of our customer’s hosts (or networks), routing each address/network to the WireGuard interface for that customer.

To route the private IP address we’ve chosen for Customer A’s Host A (10.100.100.1) out our wg-a interface, we’d add the following PostUp command to the [Interface] section of our wg-a config file (/etc/wireguard/wg-a.conf):

PostUp = ip route add 10.100.100.1 dev %i

And to route the private IP address we’ve chosen for Customer B’s Host B (10.100.100.2) out our wg-b interface, we’d add the following PostUp command to the [Interface] section of our wg-b config file (/etc/wireguard/wg-b.conf):

PostUp = ip route add 10.100.100.2 dev %i

Then we can add a DNAT firewall rule to translate the destination IP address of packets outgoing through our wg-a interface to our chosen IP address for Host A (10.100.100.1) to instead use Customer A’s chosen address of 10.0.0.1; and a second DNAT firewall rule to translate the destination IP address of packets outgoing through our wg-b interface to our chosen IP address for Host B (10.100.100.2) to instead use Customer B’s chosen address of 10.0.0.1 (the same address as Customer A chose for Host A).

To implement the firewall rules with nftables, we’d make the following change to our base Nftables Config file, adding a nat chain with an output hook for our outgoing DNAT rules, and a nat chain with a postrouting hook for our outgoing SNAT rules:

table inet nat {
    chain output {
        type nat hook output priority -100; policy accept;
        oifname "wg-a" dnat ip to 10.0.0.1
        oifname "wg-b" dnat ip to 10.0.0.1
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        oifname "wg-a" snat ip to 10.0.0.2
        oifname "wg-b" snat ip to 10.0.0.2
    }
}

Tip

The optimized version of these rules using nftable’s map functionality would be the following:

table inet nat {
    chain output {
        type nat hook output priority -100; policy accept;
        dnat ip to oifname map {
          "wg-a" : 10.0.0.1,
          "wg-b" : 10.0.0.1,
        }
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        snat ip to oifname map {
          "wg-a" : 10.0.0.2,
          "wg-b" : 10.0.0.2,
        }
    }
}

If using iptables, you’d implement the above DNAT and SNAT rules with the following commands:

# iptables -t nat -A OUTPUT -i wg-a -j DNAT --to-destination 10.0.0.1
# iptables -t nat -A OUTPUT -i wg-b -j DNAT --to-destination 10.0.0.1
# iptables -t nat -A POSTROUTING -i wg-a -j SNAT --to-source 10.0.0.2
# iptables -t nat -A POSTROUTING -i wg-b -j SNAT --to-source 10.0.0.2

When we make an HTTP request from Host C to an HTTP service listening on Host A using our private address for Host A of 10.100.100.1, if we don’t specify the network interface to use, the network stack on Host C will apply its routing rules to calculate the appropriate interface. Since we added an explicit route for 10.100.100.1 to use the wg-a interface, the network stack will select wg-a — and set the source address of our request’s packets to match the interface address of 10.200.200.1.

Then, as the packet reaches the “output” stage of netfilter’s processing, it will apply the DNAT rule we set up to match the output interface wg-a, and as a result translate the packet’s destination address to 10.0.0.1. At that point, Host C’s routing rules will be applied again — this time with the packet’s source address set to 10.200.200.1 and destination address set to 10.0.0.1. This source address will match the policy routing rule we set up in the To Host A section earlier, which will direct the routing subsystem to use our custom routing table 41; and this custom routing table will have a match for the destination address of 10.0.0.1 (set up automatically for us by wg-quick based on the AllowedIPs settings of wg-a), keeping it routed out the wg-a interface.

When the packet reaches the “postrouting” stage, netfilter will apply the SNAT rule we set up to match the output interface wg-a, and as a result translate the packet’s source address to 10.0.0.2. After that, the packet will be sent out the wg-a interface through the WireGuard tunnel to Host A. When reply packets from Host A return back through the tunnel, netfilter will automatically apply the reverse NAT, so our HTTP client will receive the replies with a source address matching the destination address it had used originally (10.100.100.1), and a destination address matching the original source address (10.200.200.1).

The same process works similarly when we make an HTTP request from Host C to an HTTP service listening on Host B: We make the request using our private address for Host B, 10.100.100.2; the network stack applies its routing rules to determine that the request should use the wg-b interface with a source address of 10.200.200.2; and netfilter applies DNAT and SNAT to it in stages so that the source address of outgoing packets will be rewritten to 10.0.0.2, and the destination address to 10.0.0.1 — but the source address is not rewritten until after the original source address is used by the policy rule we set up in the To Host B section to confirm the routing decision to route packets out interface wg-b.

Inbound and Outbound

The third scenario we’ll cover is how to avoid address collisions while allowing both inbound access to a private server of ours from customers’ networks, as well outbound access from the same server to other network services in the same customers’ networks. In this example, we want to enable customers to make HTTP requests to our private server at TCP port 8080, but also allow that same private server to query a MySQL database running in each customer’s network:

For this scenario, we need to combine both sets of firewall rules from the above Inbound Connections and Outbound Connections sections. The full nftables config we’d use in that case would be the following:

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = { "wg-a", "wg-b" }
define wg_port = { 51821, 51822 }

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        meta l4proto { icmp, ipv6-icmp } accept
        # accept all packets that are part of an already-established connection
        ct state vmap { invalid : drop, established : accept, related : accept }
        # drop new connections over rate limit
        ct state new limit rate over 1/second burst 10 packets drop

        # accept all DHCPv6 packets received at a link-local address
        ip6 daddr fe80::/64 udp dport dhcpv6-client accept
        # accept all SSH packets received on a public interface
        iifname $pub_iface tcp dport ssh accept
        # accept all WireGuard packets received on a public interface
        iifname $pub_iface udp dport $wg_port accept

        # accept all packets to TCP port 8080 received on a WireGuard interface
        iifname $wg_iface tcp dport 8080 accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
        reject with icmpx type host-unreachable
    }
}
table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        # rewrite destination of incoming client traffic to use our address space
        dnat ip to iifname map {
          "wg-a" : 10.200.200.1,
          "wg-b" : 10.200.200.2,
        }
    }
    chain input {
        type nat hook input priority 100; policy accept;
        # rewrite source of incoming client traffic to use our address space
        snat ip to iifname map {
          "wg-a" : 10.100.100.1,
          "wg-b" : 10.100.100.2,
        }
    }
    chain output {
        type nat hook output priority -100; policy accept;
        # rewrite destination of outgoing client traffic to use client address space
        dnat ip to oifname map {
          "wg-a" : 10.0.0.1,
          "wg-b" : 10.0.0.1,
        }
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        # rewrite source of outgoing client traffic to use client address space
        snat ip to oifname map {
          "wg-a" : 10.0.0.2,
          "wg-b" : 10.0.0.2,
        }
    }
}

Also, we’d need to make sure that we added the additional route for each WireGuard interface needed for outgoing DNAT (as described in the Outbound Connections section above); like this for our WireGuard interface to Customer A:

# /etc/wireguard/wg-a.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.200.200.1
ListenPort = 51821

Table = 41
PreUp = ip rule add from 10.200.200.1 table 41 priority 100
PostDown = ip rule del from 10.200.200.1 table 41 priority 100
PostUp = ip route add 10.100.100.1 dev %i

# remote settings for Host A
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.1

And like this for our WireGuard interface to Customer B:

# /etc/wireguard/wg-b.conf

# local settings for Host C
[Interface]
PrivateKey = CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGA=
Address = 10.200.200.2
ListenPort = 51822

Table = 42
PreUp = ip rule add from 10.200.200.2 table 42 priority 100
PostDown = ip rule del from 10.200.200.2 table 42 priority 100
PostUp = ip route add 10.100.100.2 dev %i

# remote settings for Host B
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.1

With these nftables rules in place for DNAT and SNAT, and the policy routing rules and custom routes set up, both incoming and outgoing connections will work at the same time (each as explained by the Inbound Connections and Outbound Connections sections above).

11/26/2023

by Justin Ludwig