07-09-2018 01:10 PM
This isn’t a big show stopper, I’m just curious than anything. I have a site to site vpn setup and resources on the azure side can access everything on PREM except the usg itself. If I try to ssh to the usg or hit it in a browser I get connection denied. Anyone have a clue how to fix this, or is it expected behavior?
07-09-2018 03:20 PM
Is your auto-firewall-nat-exclude enabled? (top line)
If so, let's look at a pcap:
sudo tcpdump -npi <interface> host x.x.x.x and port 22
Where <interface> is the LAN interface you're trying to connect to over the VPN, and x.x.x.x is the private IP you're sourcing from behind Azure.
07-09-2018 04:51 PM
Thank you for the reply! I'm a Microsoft employee, and loving the ability to keep this site-to-site VPN to Azure running BTW! I really like this unifi gateway. I've only had it for about 3 weeks, but I love it.
auto-firewall-nat-exclude is enabled per "show vpn".
sudo tcpdump -npi eth0 host 10.1.0.6 port 22
is giving me a syntax error. I believe eth0 is the lan1 interface, according to my config when I dumped it. I assume I am just formatting the command wrong?
07-09-2018 04:58 PM
Just running this sudo tcpdump -npi eth0 host 10.1.0.6, and then trying to SSH from 10.1.0.6 (Azure linux VM) to 192.168.1.1 (my gateway IP) shows nothing. If I try to ssh to 192.168.1.2 from 10.1.0.6, then I actually see in the tcpdump where its trying to connect (but won't, because 192.168.1.2 doesnt have anything on it right now).
07-25-2018 09:52 AM
With the command:
sudo tcpdump -npi eth0 host 10.1.0.6 port 22
It actually needs to be the literal way I wrote it in my initial reply, the "and" operator needs to be included:
sudo tcpdump -npi eth0 host 10.1.0.6 and port 22
It's a common mistake and easy to miss, no worries there!
And I just tested this as well, you won't see anything on the LAN interface if the destination address you're trying to hit belongs to the USG's interface... that's because the packets aren't technically "entering" or "leaving" the LAN interface with USG destined traffic. Do you have VTI enabled on this VPN? You can check in the USG with:
And it should show something like:
vti0 10.255.254.1/32 u/u
If so, you can tcpdump on the VTI0 (most likely it'll be VTI64 interface since it's a "manual" VPN and not "auto") like this:
sudo tcpdump -npi vti64 host 10.1.0.6 and port 22
And check the results. If vti isn't enabled there's not a way you can sniff the traffic on the USG itself, where the USG LAN address IP is the source or destination of the traffic. You'd want to sniff with either wireshark or tcpdump on the source of what you're trying to SSH from.
07-25-2018 03:06 PM - edited 07-25-2018 03:08 PM
Hi Brandon - Thank you so much!
When I run "show interfaces" all I get is eth0, eth1, eth2, eth3, and lo. If I have someone on the remote VPN I see the l2tp interface. I am not seeing the site-to-site interface, and I assume that is because I have dynamic routing disabled, because that was the only way I could get my 192.168.1.x LAN AND my 192.168.2.x remote VPN users able to access resources. Instead of dynamic routing, I have the following in my advanced config JSON file:
So - I assume I need to go the wireshark route here to try and see whats happening? Like I said, from Azure I can access all of my other on-prem resources ,just not the USG. At this point it is just an "I want to know why" thing, versus a major need
07-26-2018 11:53 AM
Hi Brandon - So I did some digging here. I set up a packet capture on the Azure side for one of my VMs. I then set up a tcpdump on my USG at home with the following command "sudo tcpdump -n src 10.1.0.4 and not dst 192.168.1.53". Basically I'm capturing everything from the VM in Azure (10.1.0.4) that is not my laptop just to eliminate some noise. From that Azure VM I can hit other IPs on my home network just fine - example I can view 192.168.1.12 and I see that coming through on the tcpdump on the USG, as well as the packet capture in Azure. When I try to hit 192.168.1.1 however, which is the IP of my USG at home, I see the SYN packet in Azure, I see several retransmissions, but I never see any ACK. On the USG side my tcpdump in this scenario never shows anything. If I am reading this properly, for whatever reason Azure is not sending packets to 192.168.1.1 over the VPN. I cannot for the life of me figure out why, so I will open a support ticket with Microsoft (I work at Microsoft, so this will be easy). On the Azure local network gateway I have specified 192.168.1.0/24 as the remote IP range. I cannot for the life of me understand why it isn't handling 192.168.1.1 properly, but is handling everything else properly.
07-27-2018 08:25 AM
Can you show me the Azure packet capture? TCP retransmissions are common with asymmetric routing, I wonder if the path *to* the USG isn't the same path the USG is sending the reply packets.
07-27-2018 08:29 AM
I fixed this. Needed a firewall rule on WAN LOCAL to allow access from my azure network to the USG. This is probably soemthing that shoudl get documented by ya'll somewhere.
07-27-2018 12:03 PM
The reason it's not on a KB article is because the VPN you have setup is very specific and this applies to probably less than 1% of users, this is because you have to manually disable "auto-firewall-nat-exclude", as well as use a VPN without VTI enabled.
But judging by your json file, you didn't manually disable the auto-firewall, which means it still provisions the following rules in the iptables chain:
Chain UBNT_VPN_IPSEC_FW_HOOK (1 references)
pkts bytes target prot opt in out source destination
3164 181K ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 500,4500
195 32392 ACCEPT esp -- * * 0.0.0.0/0 0.0.0.0/0
The second one (esp) is what accepts the traffic from the remote VPN traffic destined to the USG's LAN interface address and subnet.
So it makes no sense that adding a rule on WAN_LOCAL actually fixed this issue. Can you delete that rule and try to SSH again to the USG LAN address?
07-27-2018 02:12 PM
However, this doesn't make sense, and *shouldn't* work this way.
In the future we plan on adding a VPN firewall ruleset where you can easily filter the ingress VPN traffic destined to the USG, or the hosts behind it on the LAN.
07-27-2018 02:21 PM
Adding that rule was a Hail Mary. I didn’t expect it to work. This has been fun to play with though. I just replaced 2 of my 3 home access points with your ap-hp access points, and the whole configuration was so easy. Ill be replacing the third soon.
07-27-2018 02:35 PM
Just so you know, the reason that I disabled vti is because that was the only way I could get my remote user vpn connections to be able to see the azure resources. That’s where you see the 192.168.2.0/24 tunnel route I added. If there’s a better way to make that happen I’m open to changing it.