Reply
New Member
Posts: 5
Registered: ‎04-01-2015

ER-6P Hardware Offload Not Working (1.10.8)

I have been struggling for the past few weeks on figuring out why my ER-6P is not supporting hardware offloading:

 

Relevant Configuration:

set system offload hwnat disable
set system offload ipv4 forwarding enable
set system offload ipv4 vlan enable
set system offload ipv6 forwarding enable

 

Firewall:~$ show ubnt offload statistics 

 Statistics
========================

RX packets:                        0    bytes:                     0
TX packets:                        0    bytes:                     0
Bypass packets:                    0    bytes:                     0
Bad L4 checksum:                   0    bytes:                     0

Protocol        RX packets      RX bytes                TX packets      TX bytes

ipv4            0                 0                   0                 0
ipv6            0                 0                   0                 0
pppoe           0                 0                   0                 0
vlan            0                 0                   0                 0

 Forwarding cache size (IPv4)
=============================

table_size (buckets)                  16384
table size (bytes)                    2097152
flows_max (bytes)                     9830400

 Flow cache table size (IPv6)
=============================

table_size (buckets)                  8192
table size (bytes)                    1048576
flows_max (bytes)                     2883584

 Flow timers
=============================

cycles                                156629928388415
clock_rate                            1000000000
HZ                                    100
timer_ticks                           15631859
new_flow_interval (timer_ticks)       1200
old_flow_interval (timer_ticks)       400

 Low-level IPv4 flow dynamics
=============================

ipv4_flow_found                       0
    ipv4_flow_found_expired           0
    ipv4_flow_found_old_random_bypass 0
    ipv4_flow_found_action_bypass     0

ipv4_flow_not_found                   0

 IPv4 flow creation dynamics
=============================

ipv4_create_flow_found                            0
ipv4_create_flow_found_replaced                   0
ipv4_create_flow_not_found                        0
ipv4_create_flow_not_found_replaced_expired       0
ipv4_create_flow_not_found_replaced_non_expired   0

 Low-level IPv6 flow dynamics
=============================

ipv6_flow_found                       0
    ipv6_flow_found_expired           0
    ipv6_flow_found_old_random_bypass 0
    ipv6_flow_found_action_bypass     0

ipv6_flow_not_found                   0

 IPv6 flow creation dynamics
=============================

ipv6_create_flow_found                            0
ipv6_create_flow_found_replaced                   0
ipv6_create_flow_not_found                        0
ipv6_create_flow_not_found_replaced_expired       0
ipv6_create_flow_not_found_replaced_non_expired   0

 Flow cache flushes
=============================

ipv4_flushes                          0
ipv6_flushes                          0

Firewall:~$ show ubnt offload flows 

------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Cavium ip_fwd cache: (IPv4)
================================================================================================================================================================================================|
| Src        | Src   | Dst        | Dst   | Proto | PPPoE | VLAN |    Flow Info     |
| Addr       | Port  | Addr       | Port  |       | Sess  | Tag  |------------------------------------------------------------------------------------------------------------------------------------|
|            |       |            |       |       |       |      |  Src Addr   | Src port  | Dst Addr   | Dst port   | Proto | PPPOE Sess ID | VLAN Tag | TOS  | nfmark | Pkts Rcvd | Pkts Send |
============================================================================================================================================================================================================================================|

Does anyone have this working on a ER-6P?  I'm not using anything like QoS, IPSec, or features that I'm aware of that would disable offloading.

New Member
Posts: 32
Registered: ‎10-14-2018
Kudos: 3

Re: ER-6P Hardware Offload Not Working (1.10.8)

[ Edited ]

It might just be a bug with the statistics command. I noticed that when I ran the command for the first time it showed a bunch of zeros. But the subsequent invocations of the same command do give non-zero counters.

 

Spoiler
ubnt@ubnt:~$ show version
Version:      v1.10.8
Build ID:     5142457
Build on:     11/20/18 18:10
Copyright:    2012-2018 Ubiquiti Networks, Inc.
HW model:     EdgeRouter 6P
HW S/N:       xxxxxxxxxxxxxx
Uptime:       19:58:14 up 2 days, 22:25,  1 user,  load average: 0.00, 0.03, 0.05


ubnt@ubnt:~$ show ubnt offload

IP offload module   : loaded
IPv4
  forwarding: enabled
  vlan      : enabled
  pppoe     : disabled
  gre       : disabled
IPv6
  forwarding: enabled
  vlan      : enabled
  pppoe     : disabled

IPSec offload module: loaded

Traffic Analysis    :
  export    : enabled
  dpi       : enabled
    version       : 1.422

ubnt@ubnt:~$ show ubnt offload statistics

 Statistics
========================

RX packets:                    90931    bytes:             101068333
TX packets:                    83694    bytes:             101236662
Bypass packets:                 7237    bytes:               1002943
Bad L4 checksum:                   0    bytes:                     0

Protocol        RX packets      RX bytes                TX packets      TX bytes

ipv4            8106           2158406                8274           2226940
ipv6            82443          98883603               82443          99999333
pppoe           0                 0                   0                 0
vlan            382             26324                 214             13332

 Forwarding cache size (IPv4)
=============================

table_size (buckets)                  16384
table size (bytes)                    2097152
flows_max (bytes)                     9830400

 Flow cache table size (IPv6)
=============================

table_size (buckets)                  8192
table size (bytes)                    1048576
flows_max (bytes)                     2883584

 Flow timers
=============================

cycles                                253361022768905
clock_rate                            1000000000
HZ                                    100
timer_ticks                           25304261
new_flow_interval (timer_ticks)       1200
old_flow_interval (timer_ticks)       400

 Low-level IPv4 flow dynamics
=============================

ipv4_flow_found                       4841
    ipv4_flow_found_expired           454
    ipv4_flow_found_old_random_bypass 1
    ipv4_flow_found_action_bypass     0

ipv4_flow_not_found                   3647

 IPv4 flow creation dynamics
=============================

ipv4_create_flow_found                            503
ipv4_create_flow_found_replaced                   428
ipv4_create_flow_not_found                        373
ipv4_create_flow_not_found_replaced_expired       373
ipv4_create_flow_not_found_replaced_non_expired   0

 Low-level IPv6 flow dynamics
=============================

ipv6_flow_found                       80903
    ipv6_flow_found_expired           1159
    ipv6_flow_found_old_random_bypass 49
    ipv6_flow_found_action_bypass     0

ipv6_flow_not_found                   894

 IPv6 flow creation dynamics
=============================

ipv6_create_flow_found                            1276
ipv6_create_flow_found_replaced                   1121
ipv6_create_flow_not_found                        280
ipv6_create_flow_not_found_replaced_expired       280
ipv6_create_flow_not_found_replaced_non_expired   0

 Flow cache flushes
=============================

ipv4_flushes                          0
ipv6_flushes                          0

New Member
Posts: 5
Registered: ‎04-01-2015

Re: ER-6P Hardware Offload Not Working (1.10.8)

[ Edited ]

First of all, thank you fanghui for replying to this thread!  It's not always conveyed in forums the appreciaition for people taking the time to responed.

 

Would you please reply with your particular config?  I don't get the same result.

 

meggert@Firewall:~$   show ubnt offload 

IP offload module   : loaded
IPv4
  forwarding: disabled
  vlan      : disabled
  pppoe     : disabled
  gre       : disabled
IPv6
  forwarding: enabled
  vlan      : disabled
  pppoe     : disabled

IPSec offload module: loaded

Traffic Analysis    :
  export    : enabled
  dpi       : enabled
    version       : 1.422

meggert@Firewall:~$ show configuration commands | match offload
set system offload hwnat disable
set system offload ipv4 forwarding enable
set system offload ipv4 vlan enable
set system offload ipv6 forwarding enable

 This is even after a reboot of the router :-(

New Member
Posts: 32
Registered: ‎10-14-2018
Kudos: 3

Re: ER-6P Hardware Offload Not Working (1.10.8)

Sure. My config is a bit long and I am sure most of the stuff is irrelevant. Redacted version posted below. I also omitted some information like static DHCP mapping, forwarded ports, etc. I cannot imagine those would affect offloading.

 

Spoiler
firewall {
    all-ping enable
    broadcast-ping disable
    ipv6-name ALLOW_ALLv6 {
        default-action accept
    }
    ipv6-name DROP_AND_LOGv6 {
        default-action drop
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    ipv6-name GUESTv6_LAN {
        default-action drop
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 30 {
            action accept
            description "Allow SSH"
            destination {
                port 22
            }
            protocol tcp
        }
    }
    ipv6-name GUESTv6_LOCAL {
        default-action drop
        description "Guest inbound traffic to the router"
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related sessions"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 30 {
            action accept
            description "Allow IPv6 icmp"
            protocol ipv6-icmp
        }
    }
    ipv6-name WANv6_IN {
        default-action drop
        description "WAN inbound traffic forwarded to LAN"
        rule 10 {
            action accept
            description "Allow established/related sessions"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    ipv6-name WANv6_LOCAL {
        default-action drop
        description "WAN inbound traffic to the router"
        rule 10 {
            action accept
            description "Allow established/related sessions"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 30 {
            action accept
            description "Allow IPv6 icmp"
            protocol ipv6-icmp
        }
        rule 40 {
            action accept
            description "allow dhcpv6"
            destination {
                port 546
            }
            protocol udp
            source {
                port 547
            }
        }
    }
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    name ALLOW_ALL {
        default-action accept
    }
    name DROP_AND_LOG {
        default-action drop
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    name GUEST_LAN {
        default-action drop
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 30 {
            action accept
            description "Allow SSH"
            destination {
                port 22
            }
            protocol tcp
        }
    }
    name GUEST_LOCAL {
        default-action drop
        description "Guest inbound traffic to the router"
        enable-default-log
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 30 {
            action accept
            description "Allow DNS"
            destination {
                port 53
            }
            protocol tcp_udp
        }
        rule 40 {
            action accept
            description "Allow DHCP"
            destination {
                port 67,68
            }
            protocol tcp_udp
        }
        rule 50 {
            action accept
            description "Allow ICMP"
            protocol icmp
        }
    }
    name WAN_IN {
        default-action drop
        description "WAN to internal"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    name WAN_LOCAL {
        default-action drop
        description "WAN to router"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address dhcp
        description Internet
        dhcpv6-pd {
            no-dns
            pd 0 {
                interface eth1 {
                    host-address ::1
                    no-dns
                    prefix-id :1
                    service slaac
                }
                interface eth2 {
                    host-address ::1
                    no-dns
                    prefix-id :2
                    service slaac
                }
                interface eth3 {
                    host-address ::1
                    no-dns
                    prefix-id :3
                    service slaac
                }
                interface eth4 {
                    host-address ::1
                    no-dns
                    prefix-id :4
                    service slaac
                }
                interface eth5 {
                    host-address ::1
                    no-dns
                    prefix-id :0
                    service slaac
                }
                interface eth5.2 {
                    host-address ::1
                    no-dns
                    prefix-id :b
                    service slaac
                }
                interface eth5.3 {
                    host-address ::1
                    no-dns
                    prefix-id :c
                    service slaac
                }
                prefix-length /60
            }
            rapid-commit enable
        }
        duplex auto
        speed auto
    }
    ethernet eth1 {
        address 192.168.4.1/24
        description "LAN 1"
        duplex auto
        speed auto
    }
    ethernet eth2 {
        address 192.168.5.1/24
        description "LAN 2"
        duplex auto
        speed auto
    }
    ethernet eth3 {
        address 192.168.6.1/24
        description "LAN 3"
        duplex auto
        speed auto
    }
    ethernet eth4 {
        address 192.168.7.1/24
        description "LAN 4"
        duplex auto
        speed auto
    }
    ethernet eth5 {
        address 192.168.3.1/24
        description "LAN 0"
        duplex auto
        speed auto
        vif 2 {
            address 192.168.11.1/24
            description "GUEST VLAN"
        }
        vif 3 {
            address 192.168.12.1/24
            description "IOT VLAN"
        }
    }
    loopback lo {
    }
}
port-forward {
    auto-firewall enable
    hairpin-nat enable
    lan-interface eth1
    lan-interface eth2
    lan-interface eth3
    lan-interface eth4
    lan-interface eth5
    wan-interface eth0
}
service {
    dhcp-server {
        disabled false
        hostfile-update disable
        shared-network-name LAN0 {
            authoritative disable
            subnet 192.168.3.0/24 {
                default-router 192.168.3.1
                dns-server 192.168.3.1
                lease 86400
                start 192.168.3.100 {
                    stop 192.168.3.150
                }
            }
        }
        shared-network-name LAN0.2 {
            authoritative disable
            subnet 192.168.11.0/24 {
                default-router 192.168.11.1
                dns-server 192.168.11.1
                lease 86400
                start 192.168.11.2 {
                    stop 192.168.11.200
                }
            }
        }
        shared-network-name LAN0.3 {
            authoritative disable
            subnet 192.168.12.0/24 {
                default-router 192.168.12.1
                dns-server 192.168.12.1
                lease 86400
                start 192.168.12.2 {
                    stop 192.168.12.200
                }
            }
        }
        shared-network-name LAN1 {
            authoritative disable
            subnet 192.168.4.0/24 {
                default-router 192.168.4.1
                dns-server 192.168.4.1
                lease 86400
                start 192.168.4.100 {
                    stop 192.168.4.150
                }
            }
        }
        shared-network-name LAN2 {
            authoritative disable
            subnet 192.168.5.0/24 {
                default-router 192.168.5.1
                dns-server 192.168.5.1
                lease 86400
                start 192.168.5.100 {
                    stop 192.168.5.150
                }
            }
        }
        shared-network-name LAN3 {
            authoritative disable
            subnet 192.168.6.0/24 {
                default-router 192.168.6.1
                dns-server 192.168.6.1
                lease 86400
                start 192.168.6.100 {
                    stop 192.168.6.150
                }
            }
        }
        shared-network-name LAN4 {
            authoritative disable
            subnet 192.168.7.0/24 {
                default-router 192.168.7.1
                dns-server 192.168.7.1
                lease 86400
                start 192.168.7.100 {
                    stop 192.168.7.150
                }
            }
        }
        static-arp disable
        use-dnsmasq enable
    }
    dns {
        forwarding {
            cache-size 3000
            listen-on eth1
            listen-on eth2
            listen-on eth5
            listen-on eth3
            listen-on eth4
            listen-on eth5.2
            listen-on eth5.3
        }
    }
    gui {
        http-port 80
        https-port 443
        older-ciphers enable
    }
    nat {
        rule 5010 {
            description "masquerade for WAN"
            outbound-interface eth0
            type masquerade
        }
    }
    ssh {
        port 22
        protocol-version v2
    }
    ubnt-discover {
        disable
    }
    unms {
        disable
    }
    upnp2 {
        acl {
            rule 100 {
                action allow
                external-port 1024-65535
                local-port 0-65535
                subnet <REDACTED>/32
            }
            rule 9000 {
                action deny
                description "Deny all"
                external-port 1024-65535
                local-port 0-65535
                subnet 0.0.0.0/0
            }
        }
        listen-on eth5
        nat-pmp enable
        secure-mode enable
        wan eth0
    }
}
system {
    host-name ubnt
    login {
        user ubnt {
            authentication {
                encrypted-password <REDACTED>
                public-keys putty {
                    key <REDACTED>
                    type ssh-rsa
                }
            }
            level admin
        }
    }
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
        server 1.ubnt.pool.ntp.org {
        }
        server 2.ubnt.pool.ntp.org {
        }
        server 3.ubnt.pool.ntp.org {
        }
    }
    offload {
        hwnat disable
        ipv4 {
            vlan enable
        }
        ipv6 {
            forwarding enable
            vlan enable
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone America/Los_Angeles
    traffic-analysis {
        dpi enable
        export enable
    }
}
zone-policy {
    zone guest {
        default-action drop
        from iot {
            firewall {
                ipv6-name DROP_AND_LOGv6
                name DROP_AND_LOG
            }
        }
        from lan {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from local {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from wan {
            firewall {
                ipv6-name WANv6_IN
                name WAN_IN
            }
        }
        interface eth5.2
    }
    zone iot {
        default-action drop
        from guest {
            firewall {
                ipv6-name DROP_AND_LOGv6
                name DROP_AND_LOG
            }
        }
        from lan {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from local {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from wan {
            firewall {
                ipv6-name WANv6_IN
                name WAN_IN
            }
        }
        interface eth1
        interface eth2
        interface eth3
        interface eth4
        interface eth5.3
    }
    zone lan {
        default-action drop
        from guest {
            firewall {
                ipv6-name GUESTv6_LAN
                name GUEST_LAN
            }
        }
        from iot {
            firewall {
                ipv6-name DROP_AND_LOGv6
                name DROP_AND_LOG
            }
        }
        from local {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from wan {
            firewall {
                ipv6-name WANv6_IN
                name WAN_IN
            }
        }
        interface eth5
    }
    zone local {
        default-action drop
        from guest {
            firewall {
                ipv6-name GUESTv6_LOCAL
                name GUEST_LOCAL
            }
        }
        from iot {
            firewall {
                ipv6-name GUESTv6_LOCAL
                name GUEST_LOCAL
            }
        }
        from lan {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from wan {
            firewall {
                ipv6-name WANv6_LOCAL
                name WAN_LOCAL
            }
        }
        local-zone
    }
    zone wan {
        default-action drop
        from guest {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from iot {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from lan {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        from local {
            firewall {
                ipv6-name ALLOW_ALLv6
                name ALLOW_ALL
            }
        }
        interface eth0
    }
}

Senior Member
Posts: 3,079
Registered: ‎08-06-2015
Kudos: 1305
Solutions: 176

Re: ER-6P Hardware Offload Not Working (1.10.8)


@fanghui wrote:

It might just be a bug with the statistics command. I noticed that when I ran the command for the first time it showed a bunch of zeros. But the subsequent invocations of the same command do give non-zero counters.

 

Yes this is normal and expected.  Stats collection is not actually enabled by default after a reboot.  The first time you run 'show ubnt offload statistics' it will enable collection but will show all zeros as nothing had been collected yet.

 

Each time you reboot if you want stats you'll need to rerun that command.

Highlighted
New Member
Posts: 32
Registered: ‎10-14-2018
Kudos: 3

Re: ER-6P Hardware Offload Not Working (1.10.8)


@waterside wrote:

Yes this is normal and expected.  Stats collection is not actually enabled by default after a reboot.  The first time you run 'show ubnt offload statistics' it will enable collection but will show all zeros as nothing had been collected yet.

 

Each time you reboot if you want stats you'll need to rerun that command.


 That to me is a bizarre design choice but good to know. Thanks!

 

I presume the rationale behind this is that statistics collection consumes resources, so it's not enabled until asked for. If that is the case is there a CLI to turn it off again, without having to reboot the router?

New Member
Posts: 5
Registered: ‎04-01-2015

Re: ER-6P Hardware Offload Not Working (1.10.8)

So, it appears that the issue with my setup may be different than what other are experiencing.  It doesn't appear that offload is working for anything IPv4.  It's my understanding that certain features enabled on the router will disable this from running, does anyone know where that is documented?

 

See how my output differs from @fanghui output from the same command:

Firewall:~$ show ubnt offload 

IP offload module   : loaded
IPv4
  forwarding: disabled
  vlan      : disabled
  pppoe     : disabled
  gre       : disabled
IPv6
  forwarding: enabled
  vlan      : disabled
  pppoe     : disabled

IPSec offload module: loaded

Traffic Analysis    :
  export    : enabled
  dpi       : enabled
    version       : 1.422
Regular Member
Posts: 716
Registered: ‎06-17-2015
Kudos: 146
Solutions: 43

Re: ER-6P Hardware Offload Not Working (1.10.8)

@Scrounger    If you have booted your router recently, post the output of: cat /var/log/vyatta/vyatta-commit.log

 

offloading llink

 

 

Regular Member
Posts: 302
Registered: ‎02-12-2013
Kudos: 81
Solutions: 23

Re: ER-6P Hardware Offload Not Working (1.10.8)

@Scrounger If you have Netflow enabled, then it will disable IPv4 offloading - just saying, since you didn't post your config, so don't know if that applies.
New Member
Posts: 5
Registered: ‎04-01-2015

Re: ER-6P Hardware Offload Not Working (1.10.8)

Yep, good call, I thought I removed all of the flow-accounting config, missed part of it.
Reply