Member
Posts: 136
Registered: ‎11-23-2016
Kudos: 24

Re: DNS resolution of local clients (DHCP)

There really is no excuse for this.   They know what the fix is, it is just changing a couple of lines in one of the perl scripts.  A solution has been posted for a month or more.  I have no idea why they can't just make the change.   The solution that was posted works, however, as soon as you update the firmware/controller, you have to reapply the fix.

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

Wait - @chaycock - there is a workaround posted for this?  Do you have a link to it?  I swear I've searched for hours and have not found it.  You would be my official hero of the week.

Member
Posts: 136
Registered: ‎11-23-2016
Kudos: 24

Re: DNS resolution of local clients (DHCP)

I wish I could remember where I saw it, but unfortunately I cannot.  Also, there was a problem where when the DHCP assignments were being made, the hostname that got written to the hosts file was actually the mac address and not the host name.  I remember seeing a fix for that as well, but again, you had to re-apply it after every firmware update.

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

No worries! At this point, I'm just going back to my EdgeRouter, since it actually works.  I'll miss the DPI and single interface, but can live with it until the USG actually works.

Member
Posts: 136
Registered: ‎11-23-2016
Kudos: 24

Re: DNS resolution of local clients (DHCP)

> I'll miss the DPI

 

I'm not really sure that you will miss anything....from what I have been reading lately, the DPI on USG is all screwed up and cannot really be trusted.

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)


@jchappell wrote:

Prior to 5.3.8, all you needed was something akin to this in the "service" block of config.gateway.json:

 

  "dhcp-server": {
        "shared-network-name": {
            "LAN_192.168.10.0-24": {
                "subnet": {
                    "192.168.10.0/24": {
                        "domain-name": "domain.suffix.lan"
                    }
                }
            }
        }
    },

This is also all that was necessary to make it work in EdgeOS on my ERL, so it was pretty consistent between the two.


And that's still all you need. 

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)


@jchappell wrote:

Here we are, yet another controller release (5.4.9 > 5.4.11), and this is still broken.

 

@UBNT-cmb - any chance you could at least toss us an update?


Same as I noted before (elsewhere if not in this thread), that's been added to 5.5.x controller, and should be UI-exposed in the next 5.5.x release version. It didn't get in before the 5.4.x feature freeze. 

Emerging Member
Posts: 47
Registered: ‎11-19-2016
Kudos: 55

Re: DNS resolution of local clients (DHCP)

Just quoting you @UBNT-cmb from October, 2016:

 

"It was merged in this morning. The next 5.4.x, 5.3.x and 5.2.x releases will have it. "

 

Which didn't happen.  So you can understand why we're disappointed and a bit skeptical that it will be "UI-exposed in the next 5.5.x release version".  You stated you submitted it in early October 2016 after it "was mistakenly deleted".  You guys froze 5.4.x over 6 months ago?  That seems odd.  We've also had at least 3 releases SINCE you stated that, none of which have had it.  

 

We have also reported that the fix listed USED to work, but no longer does.  Are you stating that is the supported work-around for this problem?

New Member
Posts: 21
Registered: ‎07-07-2016

Re: DNS resolution of local clients (DHCP)

When is 5.5 supposed to GA?
Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)


@robpickering wrote:

Just quoting you @UBNT-cmb from October, 2016:

 

"It was merged in this morning. The next 5.4.x, 5.3.x and 5.2.x releases will have it. "

 

Which didn't happen.  


No, that did happen exactly as I described it. That was in reference to what this thread originally started as, which was disabling DNS registration of DHCP hostnames. You're pulling in a completely unrelated thing with it, which I've never said anything about other than it being added in 5.5.x. 

 

The problem people have from there in some cases, is Windows 10 and some other OSes will only do LLMNR lookups on non-FQDNs if you don't have a default domain specified. The config.gateway.json in place there does work to assign default domain. Nothing changed to make that config work or not work any differently than ever before. 

 

If it's not working, troubleshoot why - clients getting the default domain? /etc/hosts getting updated on USG? Clients actually pointed to USG for DNS? Can you dig/nslookup those hostnames? 

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)

[ Edited ]

@chaycock wrote:

There really is no excuse for this.   They know what the fix is, it is just changing a couple of lines in one of the perl scripts.  A solution has been posted for a month or more.  I have no idea why they can't just make the change.   The solution that was posted works, however, as soon as you update the firmware/controller, you have to reapply the fix.


No idea what you're referring to, deploying that configuration has nothing to do with the Perl and it's not a firmware-side issue, it just doesn't configure that. Maybe that's in reference to something unrelated I haven't seen, what solution posted are you talking about? 

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

[ Edited ]

So, I switched back to my EdgeRouter Lite earlier today (for a few other reasons - notably, I wanted to experiment with IPv6, which is easier to fiddle with on the ERL right now), but before that, I did do a fair bit of educating myself on how this works under the hood and dug into the issue some.  

 

Turns out, a few of my local devices (notably, a Windows 10 pc, an Ubuntu 16.10 pc, and an ancient Brother printer) were actually getting inserted into /etc/hosts correctly, and were being resolved.

 

Since all of my devices get the domain suffix/search domain sent down properly via DHCP, it narrowed the list of devices that weren't being inserted into /etc/hosts to my iOS and macOS devices.  Upon inspecting /var/log/messages, I found an error being thrown by `on-dhcp-event.sh` while trying to update /etc/hosts for those Apple devices.  They turned out to be failing this regex test in the script starting at L26:

if [[ ! "$client_name" =~ ^[0-9a-zA-Z_-]{1,255}$ ]]; then
    logger -s -t on-dhcp-event "Invalid client name: $client_name"
    exit 1
fi

It appears that the process that calls this script at lease time is recording the ClientName as the full FQDN: e.g. `macbook.local.domain.lan` and `iPad.local.domain.lan` whereas my Windows and Linux devices were picked up and recorded in the leases file with a client name of `raspi` or `windowsboxen1`.  This resulted in `Invalid client name: macbook.local.domain.lan`, etc... in /var/log/messages every time these devices renewed their lease.

 

I didn't get the chance to dig further into the issue than this just yet, but it was very consistent, all of my Apple devices, including two iPads (one on 10.2.1, one on 9.something), two iPhones (on 10.2.1), and a Macbook and Mac Mini (both on 10.12.3) all failed this test, and were being recorded in their respective lease blocks with their full FQDN for client name.

 

To be extra clear here, all of these Apple devices are not being inserted into /etc/hosts at all due to this, as best I can tell, whereas the Windows and Linux devices were being inserted properly (both short name and FQDN in /etc/hosts)

 

I think the reason I had never noticed this before is that 99% of my FQDN-based local lookups are to devices that didn't register, where most of my lookups against the Windows machine just use a netbios lookup.

 

My next step was going to just see about making the regex match a client name or an FQDN, and see if that would get it back up and running.  Based on what the rest of the script is doing, it looks like it might just work with that change alone.

 

I'm going to see if I can give it a shot tomorrow, but maybe that at least helps narrow things down some, @UBNT-cmb.  Appreciate your stopping by the thread.

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

[ Edited ]

As an aside, this also may be broken on EdgeOS similarly if this script is shared as-is, but I have not tested it in this depth as I just enabled dnsmasq as the DHCP service on 1.9.1 instead (which is working fine).

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)

Good troubleshooting @jchappell. That same change is in EdgeRouter as well. That's there for security reasons, and to prevent potential breakage caused by throwing invalid hostnames in /etc/hosts. That regex checks what is valid as a non-FQDN hostname. 

 

Having seen some of the situations that are triggering that excluded log, I think we ought to:

1) Either allow periods, or trim the client-hostname to everything up to the first period. 

2) Trim out spaces and register the hostname that way (seems plenty of devices have a space, and weren't registered as expected previously, and not at all now). 

 

Whether to strip out other invalid characters to obtain a valid hostname to register is another question, might do that as well. 

 

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

You're right, @UBNT-cmb, a couple of devices do seem to be using a client name with a space (like my Nintendo 3DS).  I would think either stripping out the spaces or doing a blanket replacement of all spaces with a hyphen would both be a totally valid solution, as long as it's applied consistently.  I've not seen any attempts with other invalid characters on my network, but it wouldn't surprise me that there are things out there that will try it.  The devices I have seem to just blanket substitute any invalid character other than space with an empty string, and a space with a hyphen.

 

My first approach for point #1 was going to be to set up the regex with a couple of capturing groups.  The first being everything up to the first period (which should be the raw client name), and then the second group being everything after that period, which should be the domain the device is expecting to be registered with.  Then a simple test of whether capture group 2 == $domain as a sanity check to make sure the device and the dhcp service agree on the domain name as a final sanity check.  Not positive that last check is necessary, but it sounds safe at least.

Emerging Member
Posts: 47
Registered: ‎11-19-2016
Kudos: 55

Re: DNS resolution of local clients (DHCP)

Apologies for length...

 

First, I want to thank @UBNT-cmb for re-engaging on this forum topic.  It's obviously an issue that a lot of folks are dealing with and we just want to figure out how to resolve it.  Having someone official from UBNT posting gives your customers a sense that you're engaged in a resolution.  Being transparent with that resolution drives understanding and loyalty from your customers.  Nothing but good will happen if you're engaged, we may not like the answers, but at least we're getting answers and that drives understanding.

 

Second, I want to thank @jchappell for that great troubleshooting!  I'm guessing you've made significant strides to figuring out this issue and assiting UBNT with a solution.

 

I want to address this statement from @UBNT-cmb:

"No, that did happen exactly as I described it. That was in reference to what this thread originally started as, which was disabling DNS registration of DHCP hostnames. You're pulling in a completely unrelated thing with it, which I've never said anything about other than it being added in 5.5.x. "

 

They are related, not in your mind perhaps, but in the minds of your customers.  Having local DNS working properly is the issue (the thread title is "DNS resolution of local clients (DHCP)".  Hosts receive their IP addresses from the DHCP Server, they then are recorded in the DNS (and/or Hosts file) so that other hosts may find them (by appending a default domain if necessary).  If any of that process fails to operate properly, the whole process is seen to be broken because the end result (DNS resolution) fails.  So while this thread may have started with DHCP host registration, the whole point of that process is so that they can be found via FQDN and/or Host Name DNS resolution.  This thread started with you stating, "This was wrongly disabled by default. I submitted a change to restore the previous behavior, so the next controller release should have that on again".  This was in answer to @grahamrb asking why DNS Resolution was no longer working...the assumption (incorrectly) was that whatever "this" was that you were referring to would in fact fix the issue of DNS resolution.  It didn't.  Hence the length of this thread.

 

The other thread addressing this issue (Proper DNS resolution of local hostnames) is:

https://community.ubnt.com/t5/UniFi-Routing-Switching/UniFi-USG-local-DNS-not-resolving-local-hostna...

 

So, thanks to the troubleshooting of our other customer we seem to be finding a root cause and possibly a permanent fix.  

 

When I look at the /etc/hosts file on my USG, I see a mix of things:

  1. Several hosts are listed with just their MAC address as the Hostname (see below)
  2. Several hosts are listed with a "Hostname"
  3. For the hosts listed with a "Hostname" they are listed with the original name discovered and NOT the hostname for which I've changed them to using the USG interface (e.g. "iPhone-7-Plus" instead of what's defined on the client Alias as "iphone-7plus"

For those hosts that are just MAC addresses instead of any hostname, I don't see a pattern.  Some examples are: HP OfficeJet printer, Apple TV, UBNT CloudKey, NextSAN Transporter, Mac mini, etc.  None of which have spaces in their local names...

 

The only hosts that seem to be consistently correct are my Linux systems.

I have no Windows systems on my network at all.

 

For ALL of the hosts in the /etc/hosts file, DNS resolution from a MacBook Pro is successful for the shortname (I can even use the MAC addresses). Unfortunately, none of the names registered are the names I've set them to, so it makes actually using the names impossible, because I don't know how they've registered...this may in fact be part of the issue.  The expectation is that if I've set an Alias, that the Alias will be the registered name, but that doesn't appear to be happening.

 

I'm also missing a large number of hosts in the /etc/hosts file.  

 

This brings me to another behavior I'm seeing...

 

I've long noticed that the "Clients" interface in the UniFi UI Controller seems to "lose" hosts.  I have close to a hundred hosts (76 based on the /config/dhcpd.leases file), but I only see 33 in my Clients interface.  When comparing my clients Interface to the /etc/hosts file, they seem to closely match.  It's almost like the host is disappearing from the Clients interface and in turn is then removed from the /etc/hosts file...does that make sense?  I'd prefer to have every host EVER seen appear in the Clients file, even if it hasn't been heard from in a long time (at a minimum if it's in the dhcpd.leases file, it should remain in DNS).

 

Hope this helps clarify things...

 

-Rob

 

 

 

 

 

New Member
Posts: 17
Registered: ‎05-09-2016
Kudos: 4
Solutions: 1

Re: DNS resolution of local clients (DHCP)

[ Edited ]

I spent a little time this morning working on a solution that addresses both of @UBNT-cmb's points (both sanitizing invalid hostnames and fixing scenarios where the clientname in the DHCP lease is an FQDN already)

 

Beginning at L18 of `/opt/vyatta/sbin/on-dhcp-event.sh`, modify it as follows:

 

action=$1
base_client_name=$2
client_ip=$3
client_mac=$4
domain=$5
file=/etc/hosts
changes=0

# First, prepare the client_name variable to guarantee it's only got valid
# characters.
# This prepares the client_name as follows:
#   1) Deletes whitespace.
#   2) Deletes non alphanumeric, non-underscore/hyphen, or non-period chars.
# Example: 'gam\!~ma box.local.foo.net' becomes 'gammabox.local.foo.net'
clean_client_name=`echo $base_client_name | tr -c -ds '[:alnum:]_.-'`

# Split the client_name into an array at each period, in case the client sent
# an FQDN.
client_array=(${clean_client_name//./ })

# Finally, assign the actual client name from the array.
client_name=${client_array[0]}

if [[ ! "$client_name" =~ ^[0-9a-zA-Z_-]{1,255}$ ]]; then
    logger -s -t on-dhcp-event "Invalid client name: $client_name"
    exit 1
fi

The full script, including the change can be found here: https://gist.github.com/jchappell82/c5c9e0fa7170fd244f8d51737e2e5ce5

 

I went with `tr` instead of `sed` or another RegEx style matcher for sanitizing the client_name, because it's far simpler than the RegExes that I was coming up with in my insufficiently caffeinated state this morning.

 

Once applied, events that used to fail, such as:

 

Feb 12 12:02:10 HomeUSG on-dhcp-event: Invalid client name: Jons-iPad.local.my.domain
Feb 12 12:02:10 HomeUSG dhcpd: execute: /opt/vyatta/sbin/on-dhcp-event.sh exit status 256
Feb 12 12:06:53 HomeUSG dhcpd: data: host_decl_name: not available
Feb 12 12:06:53 HomeUSG on-dhcp-event: Invalid client name: Jons-AppleWatch.local.my.domain
Feb 12 12:06:53 HomeUSG dhcpd: execute: /opt/vyatta/sbin/on-dhcp-event.sh exit status 256
Feb 12 12:13:36 HomeUSG dhcpd: data: host_decl_name: not available
Feb 12 12:13:36 HomeUSG on-dhcp-event: Invalid client name: Nintendo Wii U.local.my.domain
Feb 12 12:13:36 HomeUSG dhcpd: execute: /opt/vyatta/sbin/on-dhcp-event.sh exit status 256
Feb 12 12:14:17 HomeUSG dhcpd: data: host_decl_name: not available
Feb 12 12:14:17 HomeUSG on-dhcp-event: Invalid client name: EPSON4D3509.local.my.domain
Feb 12 12:14:17 HomeUSG dhcpd: execute: /opt/vyatta/sbin/on-dhcp-event.sh exit status 256

now result in correctly written values in /etc/hosts:

 

192.168.X.X	 Jons-iPhone.local.my.domain	 #on-dhcp-event 2c:33:61:xx:xx:xx
192.168.X.X	 gamma.local.my.domain	 #on-dhcp-event ac:bc:32:xx:xx:xx
192.168.X.X	 Jons-AppleWatch.local.my.domain	 #on-dhcp-event f4:f:24:xx:xx:xx
192.168.X.X	 Nintendo3DS.local.my.domain	 #on-dhcp-event 40:f4:7:xx:xx:xx
192.168.X.X	 NintendoWiiU.local.my.domain	 #on-dhcp-event b8:ae:6e:xx:xx:xx
192.168.X.X	 Jons-iPad.local.my.domain	 #on-dhcp-event 84:89:ad:xx:xx:xx
192.168.X.X	 EPSON4D3509.local.my.domain	 #on-dhcp-event b0:e8:92:xx:xx:xx

Additionally, I have confirmed that devices that were previously working (like the Windows 10 desktop and Linux PCs) continue to insert correct entries into /etc/hosts as well.

 

With all this said and done, dig/nslookup now return valid records.

❯ dig gamma.local.my.domain

; <<>> DiG 9.8.3-P1 <<>> gamma.local.my.domain
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40103
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;gamma.local.my.domain.	IN	A

;; ANSWER SECTION:
gamma.local.my.domain. 0	IN	A	192.168.X.X

;; Query time: 6 msec
;; SERVER: 192.168.X.X#53(192.168.X.X)
;; WHEN: Sun Feb 12 12:44:28 2017
;; MSG SIZE  rcvd: 62

 

 

 

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)


@robpickering wrote:

I've long noticed that the "Clients" interface in the UniFi UI Controller seems to "lose" hosts.  I have close to a hundred hosts (76 based on the /config/dhcpd.leases file), but I only see 33 in my Clients interface.  When comparing my clients Interface to the /etc/hosts file, they seem to closely match.  It's almost like the host is disappearing from the Clients interface and in turn is then removed from the /etc/hosts file...does that make sense?  I'd prefer to have every host EVER seen appear in the Clients file, even if it hasn't been heard from in a long time (at a minimum if it's in the dhcpd.leases file, it should remain in DNS).

 


The clients list in the controller only displays recently-active clients. Inactive clients can be seen under Insights. I'm not sure exactly the reason for the separation of those, but we at least need a more obvious pointer from Clients to Known Clients under Insights for the inactive ones.

 

When a DHCP client releases its lease, or the lease expires, its hostname is removed from the hosts file. That's best, as it's technically correct to no longer resolve a host that doesn't have a lease, and otherwise hosts would grow endlessly.

 

The current alternative if you want them to stay in DNS longer is to use a longer lease length. Soon another alternative will be to define it as a fixed IP with an alias in the controller, which will then be permanently registered in DNS on USG. That one didn't make 5.5.4 but is scheduled for 5.5.5. That's probably the ideal answer for hosts you want to resolve forever. 

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)

[ Edited ]

Thanks for that, and confirming, @jchappell. We'd already changed it to allow periods. Stripping the invalid characters is a good addition, doing some testing in various circumstances to confirm. 

 

I think we should be fine leaving the FQDNs in /etc/hosts, don't think I'll trim that. 

Ubiquiti Employee
Posts: 5,167
Registered: ‎08-08-2016
Kudos: 5645
Solutions: 355

Re: DNS resolution of local clients (DHCP)

I missed pulling this over for the last USG release, but it will make the next one later this week. 

 

I added the cleanse part there @jchappell, and allowed periods. Thanks again for your help!