Reply
Highlighted
New Member
Posts: 3
Registered: a week ago
Kudos: 1
Accepted Solution

Site-to-site with WireGuard

I need to set up a camera at a remote site so that a manager at another remote site can view it. There are a bunch of NATs and stuff in the way that I don't control, so I used WireGuard to make a tunnel between two ER-X routers through the main site.

 

Here's a rough map of the IPs. WireGuard is using the 10.8.0.0 subnet. All subnets are /24.

 

Manager    <-> Remote router <-> Main site <-> Remote router <-> Camera
10.10.10.2     10.10.10.1
               10.8.0.2          10.8.0.1      10.8.0.3
                                               10.10.11.1        10.10.11.2

Currently, I have the 10.10.10.x and 10.10.11.x networks both routing properly to the 10.8.0.x. Of course the 10.8.0.x network has no idea how to get back into the remote networks on its own.

 

I do have control over 10.8.0.1. If NAT or routing table changes are needed on the VPN server, that can be done without much difficulty.

 

Can I tell 10.10.10.1, "If you're looking for 10.10.11.0/24, go ask 10.8.0.3" and have 10.8.0.3 actually do the routing? Is this something that can be done purely by routing and NAT rules, or do I need something like OSPF or RIP? Should I abandon WireGuard for something else?

 


Accepted Solutions
Member
Posts: 753
Registered: ‎09-13-2018
Kudos: 144
Solutions: 51

Re: Site-to-site with WireGuard

[ Edited ]

This is what I think your problem is.  On the left and right routers you are not allowing traffic from the opposite end in your allowed-ips, so the traffic is being dropped when it arrives at the other end of the tunnel.

 

configure
set interfaces wireguard wg0 peer yGs... allowed-ips 10.10.11.0/24
set interfaces wireguard wg0 route-allowed-ips false
confirm;save;exit

Note you will need to put the complete public key in, but

The section should look like this when complete

    wireguard wg0 {
        address 10.8.0.2/32
        peer yGs... {
            allowed-ips 10.8.0.0/24
            allowed-ips 10.10.11.0/24
            endpoint <main-site-public-ip>:51820
            persistent-keepalive 15
        }
        private-key YPB...
        route-allowed-ips false
    }

The route-allowed-ips I think is meant to make it automatically add routes, but I prefer (at least until I become more familiar) to explicitly put in the routes.  It is normally used when you want to have all your traffic go through the wireguard tunnel.

 

You will need to make similar changes on other router, but use 10.10.10.0/24 instead of 10.10.11.0/24

 

See this thread Routing Specific Client/Traffic Through VPN (Wireguard), and the posts following it, especially the one by @karog which explains it well.

View solution in original post


All Replies
Member
Posts: 753
Registered: ‎09-13-2018
Kudos: 144
Solutions: 51

Re: Site-to-site with WireGuard

[ Edited ]

Disclaimer: I have never used wireguard, but I have been reading about it, and plan to use.

 

For your specific problem, wireguard should work fine.  You do need to have the "leaf" network ips included in the allowed ips, and you need to have some type of routing so the leaves know to send data to the main router.  For your setup I would use static, since it is the simplest solution, and dynamic would not provide any real benefit for such a simple, loop free network. See this post about the allowed ips.

 

Wireguard is fine. If you have it working between each site an the central site there is no reason to change it to something else that will be at least as complex.

 

Wireguard interfaces (the wg devices) are just standard Layer 3 interfaces with an IP address, and they behave like a standard interface, but they also have address filtering built in (the allowed ips)

 

The next problem is routing.

 

If you are only worried about these sites, static routes are by far the easiest solution.  And that is how you tell a router where to send a packet for a specific address.  Dynamic routing protocols like RIP and OSPF are most useful when there are multiple paths to get between sites, and their goal is to choose the "best" path in any circumstance.  Since you don't have redundant paths, static routes will work as well as dynamic, and are much simpler.

 

If you search for "ip routing tutorial" you can find info about how all this works if you don't know.

 

Before making changes, make backups of your current configs, so there won't be any questions later about the way it was, and you will have a way to fall back to the current configs.

 

On the main site router

 

configure
set protocols static route 10.10.10.0/24 next-hop 10.8.0.2 
set protocols static route 10.10.11.0/24 next-hop 10.8.0.3
commit-confirm

if you still have access, then

confirm

The confirm is to confirm you still have access to the router, and you don't want it to reboot.

 

Then see if you are able to ping 10.10.10.1 and 10.10.11.1 from the main router.

 

If so, then save the change, so it will persist across a reboot.

save;exit

Depending on what the default gateway on the remote routers is (I will assume it is their ISP router), you will also need to put static routes in those routers as well, so they know how to get to the remote router.  Since everything will have to go through the main site, you will need to add this to the left router.   I am basing this on the way it would be with point to point links, if you can directly ping 10.0.0.3 from the management PC, then you could change next-hop to 10.8.0.3 on the left router and 10.8.0.2 on the right router.

 

On left router

configure
set protocols static route 10.10.11.0/24 next-hop 10.8.0.1
commit-confirm

confirm

if it works, then
save;exit

On right router

configure
set protocols static route 10.10.10.0/24 next-hop 10.8.0.1
commit-confirm

confirm

if it works, then
save;exit

 

I would avoid NAT on the wireguard interface, or you are going to need to do port forwarding.  I consider that to be more complicated than adding static routes to the remote leaf networks.

 

If this doesn't work, you will need to post configs, (don't post your wireguard public or private keys) 

 

I assume you have read the stuff on the https://www.wireguard.com/ site?

 

Here's a link to someone that used site-to-site, but not with an interviening routing node 

How to set up a WireGuard Site to Site VPN Between 2 EdgeRouters and it has some extra links at the bottom.

 

And for the edgrouter specifically, there is the many page thread WireGuard for EdgeRouter

New Member
Posts: 3
Registered: a week ago
Kudos: 1

Re: Site-to-site with WireGuard

Thanks for the response. Seems like a good push in the right direction. I understand a lot better about how best to set up the routes.

 

I removed masquerading on wg0 and mucked around a bit until I got static routes to work. Unfortunately, the main site is an Ubuntu box, not a Ubiquiti router. Still, your suggestion regarding the main site routing does work. From the main site, I can ping each of the devices on the "leaf" networks. Correspondingly and after some route setup, devices on the "leaf" networks can also ping all devices on their own networks and on the 10.8 subnet. However, I can't seem to get pings between leaves (i.e.: from 10.10.10.1 to 10.10.11.1).

 

Now things start to get kinda strange. From the main site, everything is business as usual with both pings and HTTP requests to the leaf devices. All devices on the leaf subnets trying to access 10.8 can only ping 10.8.0.1 (except the routers, which properly ping anything on that subnet), and HTTP access to any 10.8 address gets a response from 10.8.0.1. There is no response from anybody on the other leaf subnet. Bottom line is I'm having a hard time locating exactly where the failure is.

 

So you asked for configs. That's probably the most helpful thing I can provide right now. This one is from the 10.10 router. Of course, sensitive information has been redacted or removed. The config on the 10.11 router is almost identical.

 

Spoiler
firewall {
    all-ping enable
    broadcast-ping disable
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    name WAN_IN {
        default-action drop
        description "WAN to internal"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    name WAN_LOCAL {
        default-action drop
        description "WAN to router"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address dhcp
        description Internet
        duplex auto
        firewall {
            in {
                name WAN_IN
            }
            local {
                name WAN_LOCAL
            }
        }
        speed auto
    }
    ethernet eth1 {
        description Local
        duplex auto
        speed auto
    }
    ethernet eth2 {
        description Local
        duplex auto
        speed auto
    }
    ethernet eth3 {
        description Local
        duplex auto
        speed auto
    }
    ethernet eth4 {
        description Local
        duplex auto
        speed auto
    }
    loopback lo {
    }
    switch switch0 {
        address 10.10.10.1/24
        description Local
        mtu 1500
        switch-port {
            interface eth1 {
            }
            interface eth2 {
            }
            interface eth3 {
            }
            interface eth4 {
            }
            vlan-aware disable
        }
    }
    wireguard wg0 {
        address 10.8.0.2/32
        peer yGs... {
            allowed-ips 10.8.0.0/24
            endpoint <main-site-public-ip>:51820
            persistent-keepalive 15
        }
        private-key YPB...
        route-allowed-ips true
    }
}
protocols {
    static {
        route 10.10.11.0/24 {
            next-hop 10.8.0.1 {
            }
        }
    }
}
service {
    dhcp-server {
        disabled false
        hostfile-update disable
        shared-network-name LAN {
            authoritative enable
            subnet 10.10.10.0/24 {
                default-router 10.10.10.1
                dns-server 10.10.10.1
                lease 86400
                start 10.10.10.38 {
                    stop 10.10.10.243
                }
            }
        }
        static-arp disable
        use-dnsmasq disable
    }
    dns {
        forwarding {
            cache-size 150
            listen-on switch0
        }
    }
    gui {
        http-port 80
        https-port 443
        older-ciphers enable
    }
    nat {
        rule 5010 {
            description "masquerade for WAN"
            outbound-interface eth0
            type masquerade
        }
    }
    ssh {
        port 22
        protocol-version v2
    }
    unms {
        disable
    }
}
system {
    login {
        user admin {
            authentication {
                encrypted-password ...
            }
            level admin
        }
    }
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
        server 1.ubnt.pool.ntp.org {
        }
        server 2.ubnt.pool.ntp.org {
        }
        server 3.ubnt.pool.ntp.org {
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone UTC
}

The Wireguard config on the main site VPN server is quite basic.

 

Spoiler
[Interface]
Address = 10.8.0.1/24
SaveConfig = true
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
ListenPort = 51820
PrivateKey = 0D0...

[Peer]
PublicKey = CU3...
AllowedIPs = 10.8.0.2/32, 10.10.10.0/24

[Peer]
PublicKey = RYL...
AllowedIPs = 10.8.0.3/32, 10.10.11.0/24

Finally, the iptables config on the main site VPN server is dirt-simple.

 

Spoiler
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A FORWARD -i wg0 -j ACCEPT

It's probably worth noting that the VPN works fine when routing stuff around the 10.8 subnet.

 

This problem feels like an iptables or routing problem on the main site VPN server or a bug in this build of vyatta-wireguard, although it could definitely be a PEBCAK error.

 

Thanks again.

Member
Posts: 753
Registered: ‎09-13-2018
Kudos: 144
Solutions: 51

Re: Site-to-site with WireGuard

[ Edited ]

This is what I think your problem is.  On the left and right routers you are not allowing traffic from the opposite end in your allowed-ips, so the traffic is being dropped when it arrives at the other end of the tunnel.

 

configure
set interfaces wireguard wg0 peer yGs... allowed-ips 10.10.11.0/24
set interfaces wireguard wg0 route-allowed-ips false
confirm;save;exit

Note you will need to put the complete public key in, but

The section should look like this when complete

    wireguard wg0 {
        address 10.8.0.2/32
        peer yGs... {
            allowed-ips 10.8.0.0/24
            allowed-ips 10.10.11.0/24
            endpoint <main-site-public-ip>:51820
            persistent-keepalive 15
        }
        private-key YPB...
        route-allowed-ips false
    }

The route-allowed-ips I think is meant to make it automatically add routes, but I prefer (at least until I become more familiar) to explicitly put in the routes.  It is normally used when you want to have all your traffic go through the wireguard tunnel.

 

You will need to make similar changes on other router, but use 10.10.10.0/24 instead of 10.10.11.0/24

 

See this thread Routing Specific Client/Traffic Through VPN (Wireguard), and the posts following it, especially the one by @karog which explains it well.

New Member
Posts: 3
Registered: a week ago
Kudos: 1

Re: Site-to-site with WireGuard

First things first. It turns out I had an iptables config (buried in rc.local) that rerouted port 80 and 443 on the main site and didn't show up in iptables -S. Removed those rules and rebooted, now routing works for all services inside the 10.8 subnet. Definitely a silly mistake on my part.

 

Lastly, my issue with routing. Turns out I don't really need static routes. Wireguard does a pretty good job of managing that on its own. Simply adding the destination subnet to the allowed-ips list (on both ends) added the routes that were needed. Now I have remote sites that can be accessed from other remote sites. Now the 10.8, 10.10, and 10.11 subnets are all talking properly. Neat.

 

Thanks for the help!

Reply