Comcast IPv6 issues when hwnat enabled on ER-X

[Moscato]

Does anyone have ipv6 working with offload enabled, applied, and QoS disabled, anywhere?

Not just Comcast.

[Ekkoville]

As I've stated earlier, I have offload enabled and IPV6 working with Comcast Blast internet. I have a Netgear CM1000 DOCSIS 3.1 Modem. No QoS either.

[simu]

Admittedly, I'm on the other side of the world (read: Switzerland), but on my setup IPv6 PD with a /48 is working fine with hwnat enabled and QoS disabled on an ER-X SFP.

[Moscato]

Okay. Can anyone have working offload on docsis 3.0, with qos disabled, in North America

[trs3]

Yes. Most people. See all the previous posts, including the original. Only some people are having problems.

[Moscato]

Is there any known pattern?

I have:
Comcast
Docsis 3.0
Motorolla surfboard 6141
25/5 plan
illinois

It fails

[sjjenkins]

@UBNT-sandisn Any further word? Are you guys getting in contact with the users here who can reproduce the issue and provide you equipment info and logs? Are you sending them test firmwares to see if the issue can be improved? This seems to be happening with enough regularity that user error is unlikely. I'd help test myself, but I'm not experiencing the issue.

[Moscato]

I can have it fail after running the basic wizard with ipv6 working, and then turning on offload and rebooting.
When it comes back up, it fails to get an ipv6 address.

That pretty much rules out user error.

[unknownzer]

If anyone feels comfortable posting ipv6 neighbor logs along with route table on failed setups. This would probably help, also please check you kernel log for failure on the offload module. If the module is to blame you will see something I the realms of a stack Trace and kernel oops. Various system logs can be viewed inside /var/log/

[trs3]

Here's a tantalizing hint from a Comcast customer using a Cisco router:

"Found 1 issue after HOURS of support from Cisco - the Hop Limit field in the IPv6 header for DHCPv6 packets from Xfinity was set to 0 so all of the DHCPv6 advertisements were being (correctly) dropped by the [Cisco] ASA. More hours on the phone trying to find the right comcast "engineer" to fix it. Finally receiving a /128 address on my outside interface."

That would be location-dependent, I would think. Could hwnat possibly affect the hop count?

Other users in the same forum *http://forums.xfinity.com/t5/Your-Home-Network/bd-p/YHN) report various problems with prefix delegation that are also location-dependent.

[unknownzer]

A hop limit of 0 is normal for pd but not normal packets. Hop limits are effected by routing agents, but not transparent ones. The behavior for 0 hop is similar to no forward.

However I ask is this issue relating to hwnat.

[BSKDNE]

I recently had to enable hwnat on my ER-X to get Comcast gigabit speeds, but subsequently I've been getting frequent drops in connection last 5-15 minutes where it isn't able to get an IP address. 

I apologize in advance because my network skills are quite lacking. I'm assuming my issues line up with this thread discussion?

Will the ERLite-3 resolve this issue with ER-X? 

My modem is SB8200

[sjjenkins]

---

@BSKDNE wrote:

I recently had to enable hwnat on my ER-X to get Comcast gigabit speeds, but subsequently I've been getting frequent drops in connection last 5-15 minutes where it isn't able to get an IP address. 

I apologize in advance because my network skills are quite lacking. I'm assuming my issues line up with this thread discussion?

Will the ERLite-3 resolve this issue with ER-X? 

My modem is SB8200

---

Hi @BSKDNE. As far as I know, the issue is limited to the ER-X, but I haven't been able to reproduct the problem on a Seattle Comcast residential connection. Again, as far as I know, the ERL hasn't been reported to have this issue, as it manages HWNAT in a slightly different manner.

[ooferomen]

---

@BSKDNE wrote:

I recently had to enable hwnat on my ER-X to get Comcast gigabit speeds, but subsequently I've been getting frequent drops in connection last 5-15 minutes where it isn't able to get an IP address. 

I apologize in advance because my network skills are quite lacking. I'm assuming my issues line up with this thread discussion?

Will the ERLite-3 resolve this issue with ER-X? 

My modem is SB8200

---

no this thread is about ipv6 not working at all with hwnat enabled for some people

[BSKDNE]

---

@sjjenkins wrote:

---

@BSKDNE wrote:

I recently had to enable hwnat on my ER-X to get Comcast gigabit speeds, but subsequently I've been getting frequent drops in connection last 5-15 minutes where it isn't able to get an IP address. 

I apologize in advance because my network skills are quite lacking. I'm assuming my issues line up with this thread discussion?

Will the ERLite-3 resolve this issue with ER-X? 

My modem is SB8200

---

Hi @BSKDNE. As far as I know, the issue is limited to the ER-X, but I haven't been able to reproduct the problem on a Seattle Comcast residential connection. Again, as far as I know, the ERL hasn't been reported to have this issue, as it manages HWNAT in a slightly different manner.

---

@sjjenkins@ooferomen

I apologize I meant to say loss in IP address numerous times an hour, lasting 5-10 minutes. In hindsight, I don't think I ever grab an IP address again unless i do a reset of the router. I noticed this only after enabling hwnat. I was using this router with it disabled with comcast blast pro (250 mbps) perfectly fine.

I directly hooked up the modem to the computer, and no issues since then.

What I also noticed was that my gigabit compatible VPN is throttled when connect to ER-X (hwnat enabled). I get 900 mbps -> 100 mbps. Directly connecting to modem, i get negligible losses with VPN.

Last, how do I disable hwnat? I want to connect through ER-X and see if VPN speeds are impacted with hwnat disabled. I tried:

set system offload hwnat disable

This results in something like "configuration node already exists" and does not disable it when I show offload.

[sjjenkins]

You can try manually editing the /config/config.boot file to disable it, then either reboot or "load / commit / save."

Another option would be to use the delete command via the CLI, then try the set command again:

https://help.ubnt.com/hc/en-us/articles/204960094-EdgeRouter-Configuration-Mode

[BSKDNE]

---

@sjjenkins wrote:

You can try manually editing the /config/config.boot file to disable it, then either reboot or "load / commit / save."

Another option would be to use the delete command via the CLI, then try the set command again:

https://help.ubnt.com/hc/en-us/articles/204960094-EdgeRouter-Configuration-Mode

---

@sjjenkins i forgot to reboot, that's why. Doh! But I noticed regardless of hwnat offload, my VPN speeds are about 10% of the speeds I get from comcast. 

hwnat offload enabled: 900 mbps -> 100 mbps (vpn)

hwnat offload disabled: 500 mbps -> 60 mbps (vpn)

Do you know if the ERL will fix the VPN speed issue? I know you mentioned that it handles hwnat offloading differently as well..

[MasterIPSee]

Hello All,

This is my first post here on the Ubiquity networks forums. I want to report I am having this exact issue with my ERX and comcast internet connection.  When I have hwnat enabled on my ERX dhcpv6-pd does not work, when it's disabled it works without issue. 

When offloading is enabled with an valid existing lease, IPv6 forwarding continues for a short perioid of time until the lease expires. If I clear the lease before enabling offloading, a new lease is never aquired. 

I am currently running on an ERPOE5 as it's not facing the same issue with IPv6 and IPv4 offloading enabled. However, there is a driver issue on that platform that is causing UDP forwarding issues so were currently stuck inbetween a rock and a hard place. 

As soon as I have a switch which suports port mirroring and some time on my hands I will get some packet caputes and upload all of the needfuls.

-Scott

[kifl]

Well, apparently this problem is now affecting me...  I have Time Warner Cable Spectrum MAXX internet (300/20) using a Arris SB6183 in the Los Angeles metro area.  I had been running EdgeOS 1.9.0 with hwnat enabled since last October when my modem got the D30CM-OSPREY-1.5.2.3-GA-01-NOSH firmware with no problems.  About a week ago we had an outage.  It was a fairly short one in the middle of the night, so it might've been some sort of planned maintainance.  When the internet came back, my router was no longer getting an IPv6 address (it never got one, it didn't loose it after a while as some people reported).  After many reboots, fiddling with the config, and upgrading to 1.9.7+hotfix.1, I finally gave up and connected a laptop directly to the modem. As soon as the modem finished rebooting, the laptop instantly had an IPv6 address.  So I came here to post about it this evening, and saw this post stickied at the top.  Turned off hwnat, and lo and behold, IPv6 is back!  Fortunately I don't actually need hwnat for my internet speed, but that doesn't help others who do.

My config is just the standard "Basic Setup" wizard with a single VLAN for a guest network added (requesting a /56 with a /64 going to the LAN and a /64 going to the Guest network), some port-forwards set up, and a dynamic dns host setup.  Nothing too fancy.  Unfortunately, I don't have a device that I can put in to do a packet capture, but if there is anything else I can do to help debug the issue, let me know.  But I would guess that it was some change that they made to their equipment during the outage that made it no longer work with the hwnat enabled since I hadn't touched my ER-X's config in ages.

[dbirkhead]

I think the post talking about Hop Limit on DHCPv6 packets was on to something.   I have observed (both with Charter and Comcast) in two different geographies that when HWNAT is disabled the Hop Limit is at 1 from upstream.  When it is enabled I still see Hop Limit of 1 but it is not being seen or processed by the ERX.

This leads me to think that something in the offload process is decrementing HLIM before it is given to the processor which then dutifully discards it since it is now zero.

Obviously this is a theory, but am hoping that UBNT might investigate this as a possibility.