New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Usg pro 4 / site to site vpn unable to access gateway

This isn’t a big show stopper, I’m just curious than anything.  I have a site to site vpn setup and resources on the azure side can access everything on PREM except the usg itself.  If I try to ssh to the usg or hit it in a browser I get connection denied.  Anyone have a clue how to fix this, or is it expected behavior?

Ubiquiti Employee
Posts: 1,482
Registered: ‎02-28-2017
Kudos: 480
Solutions: 148

Re: Usg pro 4 / site to site vpn unable to access gateway

Connections should be allowed to the USG over the VPN by default. If you SSH to the USG and type:
configure
show vpn

Is your auto-firewall-nat-exclude enabled? (top line)
If so, let's look at a pcap:
sudo tcpdump -npi <interface> host x.x.x.x and port 22

Where <interface> is the LAN interface you're trying to connect to over the VPN, and x.x.x.x is the private IP you're sourcing from behind Azure.
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Thank you for the reply!  I'm a Microsoft employee, and loving the ability to keep this site-to-site VPN to Azure running BTW!  I really like this unifi gateway.  I've only had it for about 3 weeks, but I love it.  

 

auto-firewall-nat-exclude is enabled per "show vpn".  

 

sudo tcpdump -npi eth0 host 10.1.0.6 port 22

 

^^

is giving me a syntax error.  I believe eth0 is the lan1 interface, according to my config when I dumped it.  I assume I am just formatting the command wrong? 

 

 

New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Just running this sudo tcpdump -npi eth0 host 10.1.0.6, and then trying to SSH from 10.1.0.6 (Azure linux VM) to 192.168.1.1 (my gateway IP) shows nothing.  If I try to ssh to 192.168.1.2 from 10.1.0.6, then I actually see in the tcpdump where its trying to connect (but won't, because 192.168.1.2 doesnt have anything on it right now).  

New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Jaffe - Any thoughts on my last reply? 

Ubiquiti Employee
Posts: 1,482
Registered: ‎02-28-2017
Kudos: 480
Solutions: 148

Re: Usg pro 4 / site to site vpn unable to access gateway

With the command:

sudo tcpdump -npi eth0 host 10.1.0.6 port 22


It actually needs to be the literal way I wrote it in my initial reply, the "and" operator needs to be included:

sudo tcpdump -npi eth0 host 10.1.0.6 and port 22

It's a common mistake and easy to miss, no worries there!

And I just tested this as well, you won't see anything on the LAN interface if the destination address you're trying to hit belongs to the USG's interface... that's because the packets aren't technically "entering" or "leaving" the LAN interface with USG destined traffic. Do you have VTI enabled on this VPN? You can check in the USG with:

show interfaces

And it should show something like:
vti0 10.255.254.1/32 u/u

If so, you can tcpdump on the VTI0 (most likely it'll be VTI64 interface since it's a "manual" VPN and not "auto") like this:

sudo tcpdump -npi vti64 host 10.1.0.6 and port 22

And check the results. If vti isn't enabled there's not a way you can sniff the traffic on the USG itself, where the USG LAN address IP is the source or destination of the traffic. You'd want to sniff with either wireshark or tcpdump on the source of what you're trying to SSH from.

Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

[ Edited ]

Hi Brandon - Thank you so much!  

 

When I run "show interfaces" all I get is eth0, eth1, eth2, eth3, and lo.  If I have someone on the remote VPN I see the l2tp interface.  I am not seeing the site-to-site interface, and I assume that is because I have dynamic routing disabled, because that was the only way I could get my 192.168.1.x LAN AND my 192.168.2.x remote VPN users able to access resources.  Instead of dynamic routing, I have the following in my advanced config JSON file:

 

   "vpn":{
      "ipsec":{
         "site-to-site":{
            "peer":{
               "104.215.123.192":{
                  "tunnel":{
                     "10":{
                        "allow-nat-networks":"disable",
                        "allow-public-networks":"disable",
                        "esp-group":"ESP_104.215.123.192",
                        "local":{
                           "prefix":"192.168.2.0/24"
                        },
                        "remote":{
                           "prefix":"10.1.0.0/24"
                        }
                     }
                  }
               }
            }
         }
      }
   }

 

So - I assume I need to go the wireshark route here to try and see whats happening?  Like I said, from Azure I can access all of my other on-prem resources ,just not the USG.  At this point it is just an "I want to know why" thing, versus a major need Man Happy  

New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Hi Brandon - So I did some digging here.  I set up a packet capture on the Azure side for one of my VMs.  I then set up a tcpdump on my USG at home with the following command "sudo tcpdump -n src 10.1.0.4 and not dst 192.168.1.53".  Basically I'm capturing everything from the VM in Azure (10.1.0.4) that is not my laptop just to eliminate some noise.  From that Azure VM I can hit other IPs on my home network just fine - example I can view 192.168.1.12 and I see that coming through on the tcpdump on the USG, as well as the packet capture in Azure.  When I try to hit 192.168.1.1 however, which is the IP of my USG at home, I see the SYN packet in Azure, I see several retransmissions, but I never see any ACK.  On the USG side my tcpdump in this scenario never shows anything.  If I am reading this properly, for whatever reason Azure is not sending packets to 192.168.1.1 over the VPN.  I cannot for the life of me figure out why, so I will open a support ticket with Microsoft (I work at Microsoft, so this will be easy).  On the Azure local network gateway I have specified 192.168.1.0/24 as the remote IP range.  I cannot for the life of me understand why it isn't handling 192.168.1.1 properly, but is handling everything else properly. 

Ubiquiti Employee
Posts: 1,482
Registered: ‎02-28-2017
Kudos: 480
Solutions: 148

Re: Usg pro 4 / site to site vpn unable to access gateway

You won't see anything on the USG's tcpdump because there's no interface to capture on without VTI. You'll see the packets show up in tcpdump if you're testing clients behind the USG, because the USG has to foward that packet out the LAN interface, and a reply enters the LAN interface... but if the destination address is the USG itself, the only thing the packet can "enter" and "leave" is a VTI interface, or in your case, the WAN interface... but you can't capture the inner-tunnel (non-encrypted) traffic on the WAN interface. That's why VTI is helpful.

Can you show me the Azure packet capture? TCP retransmissions are common with asymmetric routing, I wonder if the path *to* the USG isn't the same path the USG is sending the reply packets.
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

I fixed this.  Needed a firewall rule on WAN LOCAL to allow access from my azure network to the USG.  This is probably soemthing that shoudl get documented by ya'll somewhere.  

Ubiquiti Employee
Posts: 1,482
Registered: ‎02-28-2017
Kudos: 480
Solutions: 148

Re: Usg pro 4 / site to site vpn unable to access gateway

It is documented in a forum post by myself:
https://community.ubnt.com/t5/UniFi-Routing-Switching/Block-VPN-traffic-into-LAN/td-p/2028004/page/2

The reason it's not on a KB article is because the VPN you have setup is very specific and this applies to probably less than 1% of users, this is because you have to manually disable "auto-firewall-nat-exclude", as well as use a VPN without VTI enabled.

But judging by your json file, you didn't manually disable the auto-firewall, which means it still provisions the following rules in the iptables chain:
Chain UBNT_VPN_IPSEC_FW_HOOK (1 references)
pkts bytes target prot opt in out source destination
3164 181K ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 500,4500
195 32392 ACCEPT esp -- * * 0.0.0.0/0 0.0.0.0/0

The second one (esp) is what accepts the traffic from the remote VPN traffic destined to the USG's LAN interface address and subnet.
So it makes no sense that adding a rule on WAN_LOCAL actually fixed this issue. Can you delete that rule and try to SSH again to the USG LAN address?
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Confirmed.  WHen I remove the rule I created I can no longer access the USG.  Put the rule back in, and I can access it.  

Ubiquiti Employee
Posts: 1,482
Registered: ‎02-28-2017
Kudos: 480
Solutions: 148

Re: Usg pro 4 / site to site vpn unable to access gateway

I just tested this and it worked the way you said it... when I posted on that forum post I linked, I didn't account for the address of the USG itself.
However, this doesn't make sense, and *shouldn't* work this way.

In the future we plan on adding a VPN firewall ruleset where you can easily filter the ingress VPN traffic destined to the USG, or the hosts behind it on the LAN.
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Adding that rule was a Hail Mary.  I didn’t expect it to work.  This has been fun to play with though.  I just replaced 2 of my 3 home access points with your ap-hp access points, and the whole configuration was so easy.  Ill be replacing the third soon. 

New Member
Posts: 32
Registered: ‎06-26-2018
Kudos: 4

Re: Usg pro 4 / site to site vpn unable to access gateway

Just so you know, the reason that I disabled vti is because that was the only way I could get my remote user vpn connections to be able to see the azure resources.  That’s where you see the 192.168.2.0/24 tunnel route I added.  If there’s a better way to make that happen I’m open to changing it.