Highlighted
Emerging Member
Posts: 87
Registered: ‎09-25-2013
Kudos: 127
Solutions: 6

My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

I have been searching for a way to have multiple tunnels that are terminated to spoke sites with dynamic public ip addresses.

 

After several months of trial and error I have finally come up with a solution on multiple dynamic ospf-gre-ipsec tunnels.

 

So far I have three tunnels working and I'm migrating my 25 other tunnels to this same setup. In the end I will have 28 dynamic ospf-gre-ipsec tunnels all terminating to an EdgeRouter Pro

 

I use a 192.168.254.1 loopback address as the local-ip for ALL of the hub tunnels. Each of the spoke router's use a different loopback address on the same subnet but with a /32 address: Ex: spoke25 loopback and local-ip is 192.168.254.25/32 (you only specify the /32 on the loopback interface, the tunnel interface just wan'ts an ip address)

 

For the tunnel addresses I split the 192.168.200.x network into /30 networks.  So the first tunnel would use address 192.168.200.1/30 on the HUB side and 192.168.200.2/30 on the spoke side.  The next tunnel would use 192.168.200.5/30 on the Hub side and 192.168.200.6/30 on the spoke side.

 

The tunnel is established using the loopback addresses of the routers, ospf uses the 192.168.200.x addresses, and I use the @id and rsa keys for authentication.

 

Here is what I did:

 

Generate a rsa key on the Hub Router: (Can only be done when NOT in configuration mode)  Copy the public key portion and save it to enter on all of the spoke routers

 

 

generate vpn rsa-key

 

Create a loopback address on the Hub Router:

 

 

set interfaces loopback lo address 192.168.254.1/32

 

 

Create your tunnel Interface on the Hub Router:

 

 

set interfaces tunnel tun25 address 192.168.200.21/30
set interfaces tunnel tun25 description 'Remote Site 25'
set interfaces tunnel tun25 encapsulation gre
set interfaces tunnel tun25 ip ospf dead-interval 40
set interfaces tunnel tun25 ip ospf hello-interval 10
set interfaces tunnel tun25 ip ospf network point-to-point
set interfaces tunnel tun25 ip ospf priority 1
set interfaces tunnel tun25 ip ospf retransmit-interval 5
set interfaces tunnel tun25 ip ospf transmit-delay 1
set interfaces tunnel tun25 local-ip 192.168.254.1
set interfaces tunnel tun25 multicast disable
set interfaces tunnel tun25 remote-ip 192.168.254.25
set interfaces tunnel tun25 ttl 255

 

 

Configure ospf:

 

 

#this is the network address for the ip addresses assigned to the tunnel interfaces
set protocols ospf area 0 network 192.168.200.20/30 set protocols ospf parameters abr-type cisco set protocols ospf parameters router-id 255.255.255.255 set protocols ospf redistribute kernel metric 2 set protocols ospf redistribute kernel metric-type 2 set protocols ospf redistribute static metric 2 set protocols ospf redistribute static metric-type 2

 

 

Configure ipsec interface and ike and esp: (This is the same on both hub and spoke routers)

 

 

set vpn ipsec ipsec-interfaces interface eth0
set vpn ipsec esp-group esp-tunnel compression disable set vpn ipsec esp-group esp-tunnel lifetime 3600 set vpn ipsec esp-group esp-tunnel mode tunnel set vpn ipsec esp-group esp-tunnel pfs enable set vpn ipsec esp-group esp-tunnel proposal 1 encryption aes256 set vpn ipsec esp-group esp-tunnel proposal 1 hash sha1 set vpn ipsec ike-group ike-tunnel key-exchange ikev1 set vpn ipsec ike-group ike-tunnel lifetime 28800 set vpn ipsec ike-group ike-tunnel dead-peer-detection action restart
set vpn ipsec ike-group ike-tunnel dead-peer-detection interval 15
set vpn ipsec ike-group ike-tunnel dead-peer-detection timeout 60 set vpn ipsec ike-group ike-tunnel proposal 1 dh-group 2 set vpn ipsec ike-group ike-tunnel proposal 1 encryption aes256 set vpn ipsec ike-group ike-tunnel proposal 1 hash sha1

 

 

Configure Hub ipsec tunnel:

 

 

set vpn ipsec site-to-site peer @SITE25 authentication id @HUBROUTER-TO-SITE25
set vpn ipsec site-to-site peer @SITE25 authentication mode rsa
set vpn ipsec site-to-site peer @SITE25 authentication rsa-key-name SITE25_KEY
set vpn ipsec site-to-site peer @SITE25 connection-type initiate
set vpn ipsec site-to-site peer @SITE25 default-esp-group esp-tunnel
set vpn ipsec site-to-site peer @SITE25 description 'Remote Site 25'
set vpn ipsec site-to-site peer @SITE25 ike-group ike-tunnel
set vpn ipsec site-to-site peer @SITE25 local-address xx.xx.225.218
set vpn ipsec site-to-site peer @SITE25 tunnel 1 allow-nat-networks disable
set vpn ipsec site-to-site peer @SITE25 tunnel 1 allow-public-networks disable
#only encrypt the communication from the gre tunnel's local and remote adresses set vpn ipsec site-to-site peer @SITE25 tunnel 1 local prefix 192.168.254.1/32
set vpn ipsec site-to-site peer @SITE25 tunnel 1 remote prefix 192.168.254.25/32 #Use our local rsa key set vpn rsa-keys local-key file /config/ipsec.d/rsa-keys/localhost.key #Specify the Remote Site's rsa public key set vpn rsa-keys rsa-key-name SITE25_KEY rsa-key ***public key from site25's localhost.key***

Now for the remote site 25:

 

Configure the loopback:

 

 

set interfaces loopback lo address 192.168.254.25/32

 

 

Configure the Tunnel:

 

 

set interfaces tunnel tun0 address 192.168.200.22/30
set interfaces tunnel tun0 description 'Hub Site'
set interfaces tunnel tun0 encapsulation gre
set interfaces tunnel tun0 ip ospf dead-interval 40
set interfaces tunnel tun0 ip ospf hello-interval 10
set interfaces tunnel tun0 ip ospf network point-to-point
set interfaces tunnel tun0 ip ospf priority 1
set interfaces tunnel tun0 ip ospf retransmit-interval 5
set interfaces tunnel tun0 ip ospf transmit-delay 1
set interfaces tunnel tun0 local-ip 192.168.254.25
set interfaces tunnel tun0 multicast disable
set interfaces tunnel tun0 remote-ip 192.168.254.1
set interfaces tunnel tun0 ttl 255

 

 

Configure remote ospf:

 

 

set protocols ospf area 0 area-type normal
set protocols ospf area 0 network 192.168.200.20/30
set protocols ospf area 0 network 192.168.90.0/24
set protocols ospf log-adjacency-changes
set protocols ospf parameters abr-type cisco
set protocols ospf parameters router-id 1.1.1.25

 

 

Configure Remote ike and esp: (use the idential to the Hub)

 

Configure the ipsec tunnel: (HUB public ip is xx.xx.225.218)

 

 

set vpn ipsec site-to-site peer xx.xx.225.218 authentication id @SITE25
set vpn ipsec site-to-site peer xx.xx.225.218 authentication mode rsa
set vpn ipsec site-to-site peer xx.xx.225.218 authentication remote-id @HUBROUTER-TO-SITE25
set vpn ipsec site-to-site peer x.xx.225.218 authentication rsa-key-name HUB_KEY
set vpn ipsec site-to-site peer xx.xx.225.218 connection-type initiate
set vpn ipsec site-to-site peer xx.xx.225.218 default-esp-group esp-tunnel
set vpn ipsec site-to-site peer xx.xx.225.218 ike-group ike-tunnel
set vpn ipsec site-to-site peer xx.xx.225.218 local-address any
set vpn ipsec site-to-site peer xx.xx.225.218 tunnel 1 allow-nat-networks disable
set vpn ipsec site-to-site peer xx.xx.225.218 tunnel 1 allow-public-networks disable
set vpn ipsec site-to-site peer xx.xx.225.218 tunnel 1 local prefix 192.168.254.25/32
set vpn ipsec site-to-site peer xx.xx.225.218 tunnel 1 remote prefix 192.168.254.1/32
set vpn rsa-keys local-key file /config/ipsec.d/rsa-keys/localhost.key
set vpn rsa-keys rsa-key-name HUB_KEY rsa-key ***public key from HUB's localhost.key***

 

 

 I hope this helps others who are in need of this type of setup. I'm thinking that I would use this as a standard even if I have static ip addresses as if one changes for any reason I don't have to change anything on my VPN config!

 

-Eli

 

 

 

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5472
Solutions: 1656
Contributions: 2

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

Great, thanks for sharing your setup details!

Emerging Member
Posts: 54
Registered: ‎08-22-2013
Kudos: 3
Solutions: 1

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

Hi. Thank you for this. I am able to use this and it works at least with ikev1. However, the issue I am having is that I have the tunnel built with WAN interface eth0 and the ERL can see all the remote servers, however my LAN on the ERL eth1 is not able to see those servers. Is there a step (NAT, static route) that is missing on this as I had believe ospf was supposed to take care of this part.

 

EDIT:

OK, I needed to add the subnet (192.168.1.0/24) defined on eth1 to ospf of the "hub":

 

#this is the network address for the ip addresses assigned to the tunnel interfaces
set protocols ospf area 0 network 192.168.200.20/30
set protocols ospf area 0 network 192.168.1.0/24
set protocols ospf parameters abr-type cisco
set protocols ospf parameters router-id 255.255.255.255
set protocols ospf redistribute kernel metric 2
set protocols ospf redistribute kernel metric-type 2
set protocols ospf redistribute static metric 2
set protocols ospf redistribute static metric-type 2

 

 

New Member
Posts: 30
Registered: ‎06-06-2014
Kudos: 2
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

Excellent writeup. A few very minor issues:

   Under the section titled: 

      Configure the ipsec tunnel: (HUB public ip is xx.xx.225.218)

      The 6th line references "esp-transport" but should use "esp-tunnel"

   

  You need to enable ipsec on whichever interface you are using with a line like:

      set vpn ipsec ipsec-interfaces interface eth?

 

  The original poster states that spoke25 loopback would be 192.168.254.25/32 but it is really 192.168.254.26/32 because 192.168.254.1 is the loopback for the hub, making spoke1 192.168.254.2/32, etc.

 

   You will need to create an rsa key on each remote router. If you fail to note the rsa keys, use the command "show vpn ike rsa-keys" to show them

 

 

 

Emerging Member
Posts: 87
Registered: ‎09-25-2013
Kudos: 127
Solutions: 6

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

@wayne47,

 

Thank you for catching the typos.  I have corrected the examples.

 

Regards,

Eli

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

I am trying to get this working, I can see that the IpSec VPN is up and running by doing a "Show VPN log" on each end.

 

From the web GUI I can see each tunnel sending traffice, but nothing is coming in on the tunnel interface.

 

In a working configuration, what IP addresses should you be able to ping from  the command line on each router?

 

 

Veteran Member
Posts: 7,828
Registered: ‎03-24-2016
Kudos: 2040
Solutions: 900

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

1st thing to ping is remote tunnel IP , as configured in:

Spoiler
set interfaces tunnel tun25 address 192.168.200.22/30

Note, these addresses should be in same subnet.

 

You should also be able to ping remote loopback address, as configured inset interfaces loopback lo address 

Spoiler
set interfaces loopback lo address 192.168.254.25/32

However, make sure ping is sourced from local loopback IP, as it will fail otherwise.  Note these loopback addresses don't have to be in same subnet

New Member
Posts: 30
Registered: ‎06-06-2014
Kudos: 2
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

Having run this for over a year now in production, I'm having concerns about continuing to use IPSEC in our envirnmnet. Users complain about slow connections, aborted file transfers and the like. We've traced this to IPSEC adding it's headers, causing IP fragmentation. While it's possible to hack around it by reducing MSS, this is messy and needs to be done on every machine that users want to access remotely.

 

As OpenVPN does not have this, we're considering changing our connections to that. 

 

Any comments, tricks or the like to do this once on the EdgeRouters, rather than each machine? Or is there something else I'm missing?

 

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

I googled "ipsec header overhead" and the suggestion was MSS-adjust size of 1360.

 

Couldn't you simply set the mtu on the GRE tunnel?

 

set interfaces tunnel tunXX mtu 1360

 

That way it would be adjusted in size before it hits the ipsec layer.

 

Or you could set the MTU on the private LAN interfaces on each router.

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

Ok still having problems can not ping the home router from the hub router at the 192.168.200.22 address.

 

Can someone find the minor detail that I missed?

 

My configuration is as follows:

 

Hub Router is EdgeRouter Lite in data center rack with a external static IP of 216.1.1.130, it has an internal LAN of 172.16.0.0/24

 

Home office Router is a EdgeRouter X behind a ComCast router setup with a static IP of 10.10.10.10, it has an remote office LAN of 172.16.10.0/24.

 

Hub router CLI entries for the EdgeRouter Lite:

======================================================================

#
# Create a loopback address on the Hub Router:
#
set interfaces loopback lo address 192.168.254.1/32
commit
#
# Create your tunnel Interface on the Hub Router:
#
set interfaces tunnel tun25 address 192.168.200.21/30
set interfaces tunnel tun25 description 'Remote Site 25'
set interfaces tunnel tun25 mtu 1360
set interfaces tunnel tun25 encapsulation gre
set interfaces tunnel tun25 ip ospf dead-interval 40
set interfaces tunnel tun25 ip ospf hello-interval 10
set interfaces tunnel tun25 ip ospf network point-to-point
set interfaces tunnel tun25 ip ospf priority 1
set interfaces tunnel tun25 ip ospf retransmit-interval 5
set interfaces tunnel tun25 ip ospf transmit-delay 1
set interfaces tunnel tun25 local-ip 192.168.254.1
set interfaces tunnel tun25 remote-ip 192.168.254.25
set interfaces tunnel tun25 multicast disable
set interfaces tunnel tun25 ttl 255
set interfaces tunnel tun25 ip ospf network point-to-point
#
# Configure ospf for the hub router
#
set protocols ospf parameters router-id 172.16.0.1
set protocols ospf log-adjacency-changes detail
set protocols ospf parameters abr-type cisco
set protocols ospf area 0.0.0.0 area-type normal
set protocols ospf area 0.0.0.0 network 192.168.200.20/30
set protocols ospf area 0.0.0.0 network 172.16.0.0/24
#
# Configure ipsec interface and ike and esp: (This is the same on both hub and spoke routers)
#
set vpn ipsec ipsec-interfaces interface eth1
set vpn ipsec esp-group esp-tunnel compression disable
set vpn ipsec esp-group esp-tunnel lifetime 3600
set vpn ipsec esp-group esp-tunnel mode tunnel
set vpn ipsec esp-group esp-tunnel pfs enable
set vpn ipsec esp-group esp-tunnel proposal 1 encryption aes256
set vpn ipsec esp-group esp-tunnel proposal 1 hash sha1
set vpn ipsec ike-group ike-tunnel key-exchange ikev1
set vpn ipsec ike-group ike-tunnel lifetime 28800
set vpn ipsec ike-group ike-tunnel dead-peer-detection action restart
set vpn ipsec ike-group ike-tunnel dead-peer-detection interval 15
set vpn ipsec ike-group ike-tunnel dead-peer-detection timeout 60
set vpn ipsec ike-group ike-tunnel proposal 1 dh-group 2
set vpn ipsec ike-group ike-tunnel proposal 1 encryption aes256
set vpn ipsec ike-group ike-tunnel proposal 1 hash sha1
#
# Configure Hub ipsec tunnel:
#
set vpn ipsec site-to-site peer @SITE25 description 'To Freds Home Router'
set vpn ipsec site-to-site peer @SITE25 authentication id @HUBROUTER-TO-SITE25
set vpn ipsec site-to-site peer @SITE25 authentication remote-id @SITE25-TO-HUBROUTER
set vpn ipsec site-to-site peer @SITE25 authentication mode rsa
set vpn ipsec site-to-site peer @SITE25 authentication rsa-key-name SITE25_KEY
set vpn ipsec site-to-site peer @SITE25 connection-type initiate
set vpn ipsec site-to-site peer @SITE25 default-esp-group esp-tunnel
set vpn ipsec site-to-site peer @SITE25 ike-group ike-tunnel
set vpn ipsec site-to-site peer @SITE25 local-address 216.1.1.130
set vpn ipsec site-to-site peer @SITE25 tunnel 1 allow-nat-networks disable
set vpn ipsec site-to-site peer @SITE25 tunnel 1 allow-public-networks disable
#
# Only encrypt the communication from the gre tunnel's local and remote adresses
#
set vpn ipsec site-to-site peer @SITE25 tunnel 1 local prefix 192.168.254.1/24
set vpn ipsec site-to-site peer @SITE25 tunnel 1 remote prefix 192.168.254.25/24
#
# Configure RSA Keys for ipsec site-to-site
#
set vpn rsa-keys local-key file /config/ipsec.d/rsa-keys/localhost.key
set vpn rsa-keys rsa-key-name SITE25_KEY rsa-key ***public key from site25's localhost.key***
#
# Extra ospf commands on the hub
#
set protocols ospf redistribute kernel metric 2
set protocols ospf redistribute kernel metric-type 2
set protocols ospf redistribute static metric 2
set protocols ospf redistribute static metric-type 2

commit

save

======================================================================

 

Home router CLI entries for the EdgeRouter-X:

 

======================================================================

#
# Create a loopback address on the Remote site 25
#
set interfaces loopback lo address 192.168.254.25/32
commit
#
# Create your tunnel Interface on the remote Router:
#
set interfaces tunnel tun0 address 192.168.200.22/30
set interfaces tunnel tun0 description 'HUB GRE Tunnel'
set interfaces tunnel tun0 mtu 1360
set interfaces tunnel tun0 encapsulation gre
set interfaces tunnel tun0 ip ospf dead-interval 40
set interfaces tunnel tun0 ip ospf hello-interval 10
set interfaces tunnel tun0 ip ospf network point-to-point
set interfaces tunnel tun0 ip ospf priority 1
set interfaces tunnel tun0 ip ospf retransmit-interval 5
set interfaces tunnel tun0 ip ospf transmit-delay 1
set interfaces tunnel tun0 local-ip 192.168.254.25
set interfaces tunnel tun0 remote-ip 192.168.254.1
set interfaces tunnel tun0 multicast disable
set interfaces tunnel tun0 ttl 255
set interfaces tunnel tun0 ip ospf network point-to-point
#
# Configure home ospf:
#
set protocols ospf parameters router-id 172.16.10.1
set protocols ospf log-adjacency-changes detail
set protocols ospf parameters abr-type cisco
set protocols ospf area 0.0.0.0 area-type normal
set protocols ospf area 0.0.0.0 network 192.168.200.20/30
set protocols ospf area 0.0.0.0 network 172.16.10.0/24
#
# Configure ipsec interface and ike and esp: (This is the same on both hub and spoke routers)
#
set vpn ipsec ipsec-interfaces interface eth1
set vpn ipsec esp-group esp-tunnel compression disable
set vpn ipsec esp-group esp-tunnel lifetime 3600
set vpn ipsec esp-group esp-tunnel mode tunnel
set vpn ipsec esp-group esp-tunnel pfs enable
set vpn ipsec esp-group esp-tunnel proposal 1 encryption aes256
set vpn ipsec esp-group esp-tunnel proposal 1 hash sha1
set vpn ipsec ike-group ike-tunnel key-exchange ikev1
set vpn ipsec ike-group ike-tunnel lifetime 28800
set vpn ipsec ike-group ike-tunnel dead-peer-detection action restart
set vpn ipsec ike-group ike-tunnel dead-peer-detection interval 15
set vpn ipsec ike-group ike-tunnel dead-peer-detection timeout 60
set vpn ipsec ike-group ike-tunnel proposal 1 dh-group 2
set vpn ipsec ike-group ike-tunnel proposal 1 encryption aes256
set vpn ipsec ike-group ike-tunnel proposal 1 hash sha1
#
# Configure the ipsec tunnel: (HUB public ip is 216.1.1.130)
#
set vpn ipsec site-to-site peer 216.1.1.130 description 'To hub router'
set vpn ipsec site-to-site peer 216.1.1.130 authentication id @SITE25-TO-HUBROUTER
set vpn ipsec site-to-site peer 216.1.1.130 authentication remote-id @HUBROUTER-TO-SITE25
set vpn ipsec site-to-site peer 216.1.1.130 authentication mode rsa
set vpn ipsec site-to-site peer 216.1.1.130 authentication rsa-key-name HUB_KEY
set vpn ipsec site-to-site peer 216.1.1.130 connection-type initiate
set vpn ipsec site-to-site peer 216.1.1.130 default-esp-group esp-tunnel
set vpn ipsec site-to-site peer 216.1.1.130 ike-group ike-tunnel
set vpn ipsec site-to-site peer 216.1.1.130 local-address any
set vpn ipsec site-to-site peer 216.1.1.130 tunnel 1 allow-nat-networks disable
set vpn ipsec site-to-site peer 216.1.1.130 tunnel 1 allow-public-networks disable
#
# Only encrypt the communication from the gre tunnel's local and remote adresses
#
set vpn ipsec site-to-site peer 216.1.1.130 tunnel 1 local prefix 192.168.254.25/32
set vpn ipsec site-to-site peer 216.1.1.130 tunnel 1 remote prefix 192.168.254.1/32
#
# Configure RSA Keys for ipsec site-to-site
#
set vpn rsa-keys local-key file /config/ipsec.d/rsa-keys/localhost.key
set vpn rsa-keys rsa-key-name HUB_KEY rsa-key ***public key from HUB's localhost.key***

======================================================================

 

Test Commands issued from the hub router:

======================================================================

ubnt@RackRtr:~$ show vpn log
Dec 28 10:41:39 00[DMN] Starting IKE charon daemon (strongSwan 5.2.2, Linux 3.10.20-UBNT, mips64)
Dec 28 10:41:57 12[IKE] <1> 24.1.1.66 is initiating a Main Mode IKE_SA
Dec 28 10:41:58 10[IKE] <peer-SITE25-tunnel-1|1> IKE_SA peer-SITE25-tunnel-1[1] established between 16.1.1.130[HUBROUTER-TO-SIT
E25]...24.1.1.66[SITE25-TO-HUBROUTER]
Dec 28 10:41:59 04[IKE] <peer-SITE25-tunnel-1|1> CHILD_SA peer-SITE25-tunnel-1{1} established with SPIs cc466b20_i c852cb72_o and TS
192.168.254.1/32 === 192.168.254.25/32
Dec 28 10:44:32 05[KNL] 10.255.254.0 appeared on ppp0
Dec 28 10:44:32 15[KNL] 10.255.254.0 disappeared from ppp0
Dec 28 10:44:32 01[KNL] 10.255.254.0 appeared on ppp0
Dec 28 10:44:32 14[KNL] interface pptp0 activated

 


ubnt@RackRtr:~$ ping 192.168.200.21
PING 192.168.200.21 (192.168.200.21) 56(84) bytes of data.
64 bytes from 192.168.200.21: icmp_req=1 ttl=64 time=0.319 ms
64 bytes from 192.168.200.21: icmp_req=2 ttl=64 time=0.252 ms
^C
--- 192.168.200.21 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.252/0.285/0.319/0.037 ms


ubnt@RackRtr:~$ ping 192.168.200.22
PING 192.168.200.22 (192.168.200.22) 56(84) bytes of data.
^C
--- 192.168.200.22 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4004ms

======================================================================

 

Test Commands issued from the home router:

======================================================================

ubnt@hmrtr:~$ show vpn log
Dec 28 10:41:16 00[DMN] Starting IKE charon daemon (strongSwan 5.2.2, Linux 3.10.14-UBNT, mips)
Dec 28 10:41:26 09[KNL] creating acquire job for policy 192.168.254.25/32[gre] === 192.168.254.1/32[gre] with reqid {1}
Dec 28 10:41:26 10[IKE] <peer-216.1.1.130-tunnel-1|1> initiating Main Mode IKE_SA peer-216.1.1.130-tunnel-1[1] to 216.176.18
0.130
Dec 28 10:41:27 13[IKE] <peer-216.1.1.130-tunnel-1|1> IKE_SA peer-216.1.1.130-tunnel-1[1] established between 10.10.10.10[SI
TE25-TO-HUBROUTER]...216.1.1.130[HUBROUTER-TO-SITE25]
Dec 28 10:41:28 14[IKE] <peer-216.1.1.130-tunnel-1|1> CHILD_SA peer-216.1.1.130-tunnel-1{1} established with SPIs c852cb72_i
cc466b20_o and TS 192.168.254.25/32 === 192.168.254.1/32


ubnt@hmrtr:~$ ping 192.168.200.22
PING 192.168.200.22 (192.168.200.22) 56(84) bytes of data.
64 bytes from 192.168.200.22: icmp_req=1 ttl=64 time=0.424 ms
64 bytes from 192.168.200.22: icmp_req=2 ttl=64 time=0.268 ms
^C
--- 192.168.200.22 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.268/0.346/0.424/0.078 ms

 

ubnt@hmrtr:~$ ping 192.168.200.21
PING 192.168.200.21 (192.168.200.21) 56(84) bytes of data.
^C
--- 192.168.200.21 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3022ms

 

======================================================================

 

 

Veteran Member
Posts: 7,828
Registered: ‎03-24-2016
Kudos: 2040
Solutions: 900

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

Any VPN technology adds extra header, checksum and the likes.

For TCP , most elegant way to tell endpoints to lower packet size is mss-clamp

Besides setting MTU on tunnel (if it already hasn't) use below , and set value to mtu minus 40

 

Spoiler
set firewall options mss-clamp interface-type tun
set firewall options mss-clamp mss 1358

 

New Member
Posts: 30
Registered: ‎06-06-2014
Kudos: 2
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

[ Edited ]

Setting the MTU that low will cause fragmentation. 

New Member
Posts: 30
Registered: ‎06-06-2014
Kudos: 2
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

I'd look at doing following:

Paragraph 2:

set interfaces tunnel tun25 remote-ip 192.168.254.2  (not 25)

Paragraph 3:

set protocols ospf area 0.0.0.0 network 192.168.200.0/24

Paragraph 4. I do not have this line:

set vpn ipsec ike-group ike-tunnel proposal 1 dh-group 2

Paragraph 6: I use /32, not /24

set vpn ipsec site-to-site peer @SITE25 tunnel 1 local prefix 

Paragraph 7 I use: 

set vpn rsa-key-name SITE1_KEY rsa-key "the key"

Veteran Member
Posts: 7,828
Registered: ‎03-24-2016
Kudos: 2040
Solutions: 900

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

@wayne47wrote:

Setting the MTU that low will cause fragmentation. 

Read up on mss-clamp.  It's an elegant way to force local and remote endpoints into sending smaller TCP packets.

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

Yes, I read wayne47 investigation to imply that the ipsec payload packet and headers should not be fragmented.

So that was why I thought that the upstream gre layer should have a limit, so that each ipsec packet could be transmitted as one entity.
Emerging Member
Posts: 87
Registered: ‎09-25-2013
Kudos: 127
Solutions: 6

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

I had the same issues that turned out to be a longstanding bug  where the 2 cores would process packets out of order and often drop the last fragment of a fragmented packet causing the whole packet to be dropped.  While the symptoms are mostly in UDP, I saw an improvment in everything once this was resolved.

 

They recently fixed it in 1.10 alpha 1.  I have confirmed that many of my issues were tied to this bug, including my VPN issues.

 

I'd give it a try.  You'll have to update both ends though.

 

-Eli

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

Ok, I finally have this running!!!

 

I image that 

 

 

set service nat rule 5010 outbound-interface eth1
set service nat rule 5010 type masquerade
set service nat rule 5010 log disable
set service nat rule 5010 description 'masquerade for WAN'

 

I was unable to establish a ipsec connection between the sites.

 

However, adding the following command to each side; allowed ipsec to start working.

 

set vpn ipsec site-to-site peer tun0 force-encapsulation enable

and on the other end:
set vpn ipsec site-to-site peer @SITE25 force-encapsulation enable

 

 

set service nat rule 5010 source address 172.16.0.0/24

and on the other end:

set service nat rule 5010 source address 172.16.10.0/24

 

 

 

 

 

 

 

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

OK, I just brought up my second spoke router off of the hub and it has fully connectivity to the hub LAN.

 

Should the spokes be able to talk to each other via the hub?

 

When I ping for the first time; from spoke to spoke it seems that the first packet gets acknowledged and then the remainder of the packets are lost.

 

New Member
Posts: 35
Registered: ‎09-12-2017
Solutions: 3

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

OK, my central hub is EdgeRouter Lite and it appears to be a problem with the hardware offloading for forwarding.

 

When that functionality is disabled; everything works as expected.

 

So, it appears that the first packet is handled in software and then it programs the hardware; which then falls on it's face as it can't handle inbound and outbound on the same ethernet connection.

 

Emerging Member
Posts: 90
Registered: ‎11-06-2014
Kudos: 7
Solutions: 1

Re: My HowTo on multiple dynamic ospf-gre-ipsec tunnels

I've been deploying the 1.10.1 version to my ER units and would like to get OSPF built between them.  Could someone post a 'show configuration commands' dump of both the "HUB" and "Spoke" units?