Reply
Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

On the other site it worked. Please help me out of this nightmare.

 

ChristianSchmid@ubnt-h:~$ sudo ipsec up peer-wachenheim2.is-very-good.org-tunnel-1
initiating Main Mode IKE_SA peer-wachenheim2.is-very-good.org-tunnel-1[10] to 95.88.58.182
generating ID_PROT request 0 [ SA V V V V ]
sending packet: from 95.89.93.87[500] to 95.88.58.182[500] (156 bytes)
received packet: from 95.88.58.182[500] to 95.89.93.87[500] (136 bytes)
parsed ID_PROT response 0 [ SA V V V ]
received XAuth vendor ID
received DPD vendor ID
received NAT-T (RFC 3947) vendor ID
generating ID_PROT request 0 [ KE No NAT-D NAT-D ]
sending packet: from 95.89.93.87[500] to 95.88.58.182[500] (372 bytes)
received packet: from 95.88.58.182[500] to 95.89.93.87[500] (372 bytes)
parsed ID_PROT response 0 [ KE No NAT-D NAT-D ]
local host is behind NAT, sending keep alives
generating ID_PROT request 0 [ ID HASH ]
sending packet: from 95.89.93.87[4500] to 95.88.58.182[4500] (76 bytes)
received packet: from 95.88.58.182[4500] to 95.89.93.87[4500] (76 bytes)
parsed ID_PROT response 0 [ ID HASH ]
IKE_SA peer-wachenheim2.is-very-good.org-tunnel-1[10] established between 95.89.93.87[95.89.93.87]...95.88.58.182[95.88.58.182]
scheduling reauthentication in 27940s
maximum IKE_SA lifetime 28480s
generating QUICK_MODE request 2251626977 [ HASH SA No KE ID ID ]
sending packet: from 95.89.93.87[4500] to 95.88.58.182[4500] (444 bytes)
received packet: from 95.88.58.182[4500] to 95.89.93.87[4500] (444 bytes)
parsed QUICK_MODE response 2251626977 [ HASH SA No KE ID ID ]
connection 'peer-wachenheim2.is-very-good.org-tunnel-1' established successfully
ChristianSchmid@ubnt-h:~$

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

Tunnel is down again, please help me out of this nightmare. What should i do to prevent the VPN from eth0 ????? eth0 is connected to DSL-Router with VOIP-Functions...

 

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.88.58.254, eth1
S       0.0.0.0/0 [210/0] via 192.168.2.1, eth0
C    *> 95.88.58.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
C    *> 192.168.0.0/24 is directly connected, eth2
C    *> 192.168.2.0/24 is directly connected, eth0

 

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.89.93.254, eth1
S       0.0.0.0/0 [210/0] via 172.16.10.1, eth0
C    *> 95.89.93.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
C    *> 172.16.10.0/24 is directly connected, eth0
C    *> 192.168.60.0/24 is directly connected, eth2

 

ChristianSchmid@ubnt-h:~$ sudo ipsec up peer-wachenheim2.is-very-good.org-tunnel                 -1
generating QUICK_MODE request 1168126798 [ HASH SA No KE ID ID ]
sending packet: from 95.89.93.87[4500] to 80.153.226.37[4500] (444 bytes)
received packet: from 80.153.226.37[4500] to 95.89.93.87[4500] (444 bytes)
parsed QUICK_MODE response 1168126798 [ HASH SA No KE ID ID ]
CHILD_SA peer-wachenheim2.is-very-good.org-tunnel-1{2} established with SPIs ccb                 e893a_i c3aff866_o and TS 192.168.60.0/24 === 192.168.0.0/24
connection 'peer-wachenheim2.is-very-good.org-tunnel-1' established successfully

 

Again the wrong interface: 80.153 is DSL-Box on eth0 !!!!!!!

 

ChristianSchmid@ubnt-h:~$ restart vpn
Clearing IPsec process...


ChristianSchmid@ubnt-h:~$ sudo ipsec up peer-wachenheim2.is-very-good.org-tunnel-1
initiating Main Mode IKE_SA peer-wachenheim2.is-very-good.org-tunnel-1[2] to 95.88.58.182
generating ID_PROT request 0 [ SA V V V V ]
sending packet: from 95.89.93.87[500] to 95.88.58.182[500] (156 bytes)
received packet: from 95.88.58.182[500] to 95.89.93.87[500] (136 bytes)
parsed ID_PROT response 0 [ SA V V V ]
received XAuth vendor ID
received DPD vendor ID
received NAT-T (RFC 3947) vendor ID
generating ID_PROT request 0 [ KE No NAT-D NAT-D ]
sending packet: from 95.89.93.87[500] to 95.88.58.182[500] (372 bytes)
received packet: from 95.88.58.182[500] to 95.89.93.87[500] (372 bytes)
parsed ID_PROT response 0 [ KE No NAT-D NAT-D ]
local host is behind NAT, sending keep alives
generating ID_PROT request 0 [ ID HASH ]
sending packet: from 95.89.93.87[4500] to 95.88.58.182[4500] (76 bytes)
received packet: from 95.88.58.182[4500] to 95.89.93.87[4500] (76 bytes)
parsed ID_PROT response 0 [ ID HASH ]
IKE_SA peer-wachenheim2.is-very-good.org-tunnel-1[2] established between 95.89.93.87[95.89.93.87]...95.88.58.182[95.88.58.182]
scheduling reauthentication in 28133s
maximum IKE_SA lifetime 28673s
generating QUICK_MODE request 3845453112 [ HASH SA No KE ID ID ]
sending packet: from 95.89.93.87[4500] to 95.88.58.182[4500] (444 bytes)
received packet: from 95.88.58.182[4500] to 95.89.93.87[4500] (444 bytes)
parsed QUICK_MODE response 3845453112 [ HASH SA No KE ID ID ]
CHILD_SA peer-wachenheim2.is-very-good.org-tunnel-1{1} established with SPIs cc641a66_i c4fb8c66_o and TS 192.168.60.0/24 === 192.168.0.0/24
connection 'peer-wachenheim2.is-very-good.org-tunnel-1' established successfully

 

 

Ubiquiti Employee
Posts: 2,296
Registered: ‎05-08-2017
Kudos: 416
Solutions: 343

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

 I suggest adding these authentication IDs to the EdgeRouters:

 

#wachenheim
configure set vpn ipsec site-to-site peer hassloch2.is-very-good.org authentication id @wachenheim2.is-very-good.org set vpn ipsec site-to-site peer hassloch2.is-very-good.org authentication remote-id @hassloch2.is-very-good.org commit ; save
#hassloch configure set vpn ipsec site-to-site peer wachenheim2.is-very-good.org authentication id @hassloch2.is-very-good.org set vpn ipsec site-to-site peer wachenheim2.is-very-good.org authentication remote-id @wachenheim2.is-very-good.org commit ; sa

 

The issue may also be related to the dynamic DNS address not updating correctly. Is there anything in the logs that indicate that either hassloch or wachenheim is updating to the wrong IP address?

show log | no-more

 

You can verify the current Dynamic DNS status with:

show dns dynamic status

 

We can also choose to ignore the default route for the main routing table on eth0 completely, the commands are:

 

configure
set interfaces ethernet eth0 dhcp-options default-route no-update
commit ; save

 

Ben


Ben Pin - EdgeMAX Support

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

Many thanks for your advices. The IP on the CableModems on eth1 are nearly static, they change even after power loss or when the ISP is running an update. Do i have to reboot the ER4 after running your settings?

Ubiquiti Employee
Posts: 2,296
Registered: ‎05-08-2017
Kudos: 416
Solutions: 343

Re: IPsec Site-to-Site down after a few minutes

A restart of the IPsec process and a renew of the DHCP lease should be sufficient.

sudo ipsec restart

renew dhcp interface eth0

 

Ben


Ben Pin - EdgeMAX Support

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

Tunnel is down again. remote 'wachenheim2.is-very-good.org' @ 80.153.226.37 is again the wrong ip / interface eth0 instead of eth1. What can i do?

 

ChristianSchmid@ubnt-h:~$ show vpn ipsec sa
peer-wachenheim2.is-very-good.org-tunnel-1: #6, ESTABLISHED, IKEv1, ed9239f5cf2d       3327:86b97c0f55d1c1ab
  local  'hassloch2.is-very-good.org' @ 95.89.93.87
  remote 'wachenheim2.is-very-good.org' @ 80.153.226.37
  AES_CBC-128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048
  established 14469s ago, reauth in 13701s
  peer-wachenheim2.is-very-good.org-tunnel-1: #1, INSTALLED, TUNNEL, ESP:AES_CBC       -128/HMAC_SHA1_96/MODP_2048
    installed 1928 ago, rekeying in 650s, expires in 1673s
    in  c1c895ba, 170143 bytes,  1373 packets,     0s ago
    out c7fa4e63, 111326 bytes,  1380 packets,     0s ago
    local  192.168.60.0/24
    remote 192.168.0.0/24

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

@UBNT-benpin wrote:

The issue may also be related to the dynamic DNS address not updating correctly. Is there anything in the logs that indicate that either hassloch or wachenheim is updating to the wrong IP address?

show log | no-more

 

You can verify the current Dynamic DNS status with:

show dns dynamic status

 

Ben


Christian,

 

Did you ever check this?  If the dyndns update is exiting via the VDSL eth0 interface, and your dyndns provider always uses the ip address it receives the update request from, that is going to cause a problem, since the dyndns name will then be associated with the VDSL address instead of the Cable connection ip address.

 

 

    dns {
        dynamic {
            interface eth1 {
                service dyndns {
                    host-name wachenheim2.is-very-good.org
                    login fiftys333
                    password ****************
                }
            }
        }
        forwarding {
            cache-size 150
            listen-on eth2
        }
    }

see this issue: Forcing DynamicDNS Updates to go through the same interface

 

I think only one of your interfaces on each host is using dyndns (the cable connections).  See @16again's last post in the above thread for what I think is most likely to solve your problem.  Put a static host route in the main routing table to your dyndns provider, forcing any connection to the dyndns update to use the cable connection. The problem with this is the default gateway is being received via dhcp, and I don't know how to determine the next-hop address received from the DHCP offer gateway address (I know how on cisco, see below)

 

As a short term work around, since you state that your cable modem address is "nearly static", and perhaps always has the same gateway address, you could look at your routing table to determine the next hop being used by eth1, and then use a command like 

set protocols static route <dyndns-update-address>/32 next-hop <next-hop associated with eth1>

Perhaps @waterside@smyers119@16again or @redfive know how to do in EdgeOS what Cisco allows in IOS

 

From Cisco's How to Configure DHCP Statically Configured Routes Using a DHCP Gateway

ip route prefix mask {ip-address |
interface-type interface-number [ip-address]}
dhcp [distance]

Example:
Router(config)# ip route 209.165.200.225 255.255.255.255 ether1 dhcp
Router(config)# ip route 209.165.200.226 255.255.255.255 ether2 dhcp 20

 Cisco doc that describes feature How to Configure DHCP Statically Configured Routes Using a DHCP Gateway

 

Jon

 

p.s. @smyers119 said 

This is going to be a load-balancing issue.

I think he is correct, although it may be because dyndns is being updated via the incorrect interface.

 

 

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

I get an error:

 

ChristianSchmid@ubnt:~$ configure
[edit]
ChristianSchmid@ubnt# set protocols static route 162.88.175.4/32 next-hop 95.88.58.254
[edit]
ChristianSchmid@ubnt# commit
[ protocols static route 162.88.175.4/32 next-hop 95.88.58.254 ]
no ip static 162.88.175.4/32 95.88.58.254 fall-over bfd

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

I wasn't expecting that, but found this:  

https://community.ubnt.com/t5/EdgeRouter/Cannot-add-anymore-static-routes-in-v1-10-1-Error-no-ip-sta...

 

which says the error can be ignored.  

verify that the route made it into the routing table, then save the config

 

# run show ip route | grep static

# commit ; save

# exit

 

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

Okay, on Wachenheim,  show ip route shows:

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.88.58.254, eth1
C    *> 95.88.58.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
S    *> 162.88.175.4/32 [1/0] via 95.88.58.254, eth1
C    *> 192.168.0.0/24 is directly connected, eth2
C    *> 192.168.2.0/24 is directly connected, eth0

 

But on Hassloch (we moved the LAN interface yesterday from eth2 to eth3, because we have now two Ubiquiti US-48-500W Switches with fiber connections) it shows:

 

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.89.93.254, eth1
C    *> 95.89.93.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
S    *> 162.88.175.4/32 [1/0] via 88.134.221.102 (recursive via 95.89.93.254 )
C    *> 172.16.10.0/24 is directly connected, eth0
C    *> 192.168.60.0/24 is directly connected, eth3

 

 

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

@ChristianSchmid wrote:

S    *> 162.88.175.4/32 [1/0] via 88.134.221.102 (recursive via 95.89.93.254 )


what set protocol command did you enter?

 

also from $ prompt

 

show interfaces
Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

Wachenheim:
set protocols static route 162.88.175.4/32 next-hop 95.88.58.254

 

ChristianSchmid@ubnt:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         192.168.2.22/24                   u/u  Telekom FB
eth1         95.88.58.182/24                   u/u  Kabel D
eth2         192.168.0.1/24                    u/u  LAN
eth3         -                                 A/D
lo           127.0.0.1/8                       u/u
             ::1/128

 

 

Hassloch:
set protocols static route 162.88.175.4/32 next-hop 88.134.221.102

 

ChristianSchmid@ubnt-h:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         172.16.10.11/24                   u/u  Telekom Gate
eth1         95.89.93.87/24                    u/u  Kabel D
eth2         -                                 A/D
eth3         192.168.60.1/24                   u/u  LAN
lo           127.0.0.1/8                       u/u
             ::1/128

 

 

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

 


@ChristianSchmid wrote:

Hassloch:
set protocols static route 162.88.175.4/32 next-hop 88.134.221.102

 

ChristianSchmid@ubnt-h:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         172.16.10.11/24                   u/u  Telekom Gate
eth1         95.89.93.87/24                    u/u  Kabel D
eth2         -                                 A/D
eth3         192.168.60.1/24                   u/u  LAN
lo           127.0.0.1/8                       u/u
             ::1/128

 

 


where did you get the 88.134.221.102 from ?  

 

Here's what I said

 

set protocols static route <dyndns-update-address>/32 next-hop <next-hop associated with eth1>

and what I meant by that was the address from the routing table associated with the default route 0.0.0.0/0

 

 

 

But on Hassloch (we moved the LAN interface yesterday from eth2 to eth3, because we have now two Ubiquiti US-48-500W Switches with fiber connections) it shows:

 

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.89.93.254, eth1
C    *> 95.89.93.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
S    *> 162.88.175.4/32 [1/0] via 88.134.221.102 (recursive via 95.89.93.254 )
C    *> 172.16.10.0/24 is directly connected, eth0
C    *> 192.168.60.0/24 is directly connected, eth3

 

Notice that network 95.89.93.0/24 is directly connected to eth1, your DHCP client address provided by Kabel is 95.89.93.87/24 and the gateway it provided is 95.89.93.254

 

Please remove that wrong route and replace with host route to 95.89.93.254

 

$ configure
delete protocols static route 162.88.175.4/32 next-hop 88.134.221.102
set protocols static route 162.88.175.4/32 next-hop 95.89.93.254
commit ; save ; exit

 

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

Sorry for my mistake, route on Hassloch is now:

 

IP Route Table for VRF "default"
S    *> 0.0.0.0/0 [100/0] via 95.89.93.254, eth1
C    *> 95.89.93.0/24 is directly connected, eth1
C    *> 127.0.0.0/8 is directly connected, lo
S    *> 162.88.175.4/32 [1/0] via 95.89.93.254, eth1
C    *> 172.16.10.0/24 is directly connected, eth0
C    *> 192.168.60.0/24 is directly connected, eth3

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

are tunnels currenlty up?   if not bring them up.  Then wait, hopefully the problem is resolved, but it may take a few day to know for sure.

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

The downside of this, it that it the Kabel connection ever gives you an address in another network with a different gateway, it is going to break the dyndns update.

 

I don't know how to do what the cisco command I referenced does.  It may be possible to do with a script, but that's something I haven't done, and there may be a supported way.  May @UBNT-benpin knows of a way.

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

Maybe the next firmware has a feature that dyndns updates on a wan interface are only made trough the the next hop of the intterface.

I hope that Kabel does not change their routings for the next time...

 

Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

does your dyndns provider keep a log you can see that tell when the last few ip address changes were made?  That would be interesting to see, because it would confirm that either it was being set incorrectly, or if the problem still hasn't been discovered.

 

The other possibility is that there is a pc client inside your LAN that is updating the dyndns.  If that is the case it could still update the dyndns via the VDSL interface, and the next time the tunnel negociated, it could try to use the eth0 interface again.

 

Worth checking.

Emerging Member
Posts: 65
Registered: ‎10-25-2017
Kudos: 1

Re: IPsec Site-to-Site down after a few minutes

[ Edited ]

Here are the logs, the ip is not changing. We never use a pc for dyndns-updates,it's always done by router.

 

2018-11-19 12:01:29hassloch2.is-very-good.org95.89.93.87system=dyndnsddclient/3.8.3 (using SSL)nochg 95.89.93.87
2018-11-19 11:34:39hassloch2.is-very-good.org95.89.93.87system=dyndnsddclient/3.8.3 (using SSL)nochg 95.89.93.87
2018-11-18 10:48:18hassloch2.is-very-good.org95.89.93.87system=dyndnsddclient/3.8.3 (using SSL)good 95.89.93.87

 

2018-11-18 11:03:37wachenheim2.is-very-good.org95.88.58.182system=dyndnsddclient/3.8.3 (using SSL)good 95.88.58.182
Emerging Member
Posts: 440
Registered: ‎09-13-2018
Kudos: 69
Solutions: 26

Re: IPsec Site-to-Site down after a few minutes

So you are saying they were not changing before the last change?

 

If that is the case it doesn't explain the last failure you reported on the 19th.

 

good luck, I don't have any more ideas.

Reply