Reply
Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6
Accepted Solution

BUG: PPPoE client pppoe dialing before MTU set on eth0

I have found an issue with using pppoe client on the Edge Router Lite with using Baby Jumbo Frames RFC4638.

Some testing has been done to find the problem but im pretty sure this is the problem that needs fixing:

This is the config that I have for pppoe config:

interfaces {
    ethernet eth0 {
        description WAN
        duplex auto
        firewall {
            in {
                name WAN-LAN
            }
            local {
                name WAN-ME
            }
        }
        mtu 1508
        pppoe 0 {
            default-route auto
            firewall {
                in {
                    name WAN-LAN
                }
                local {
                    name WAN-ME
                }
            }
            mtu 1500
            name-server auto
            password ****************
            user-id username@isp.co.uk
        }
        speed auto

 So you can see above that my config is all ok for getting a full 1500 MTU on pppoe.

When the Edge Router boots up looking into the logs, it seems that the pppoe0 interface is dailed before the increased MTU is set on eth0.  This means that the pppoe0 interface can only come up at 1492 as the default MTU for eth0 is set to 1500. See logs below:

Jun  1 10:00:07 stables-router kernel: eth0: 1000 Mbps Full duplex, port 0
Jun  1 10:00:07 stables-router kernel: eth1: 1000 Mbps Full duplex, port 1
Jun  1 10:00:07 stables-router kernel: eth2: 100 Mbps Full duplex, port 2
Jun  1 10:00:08 stables-router zebra[497]: Zebra 0.99.20.1 starting: vty@0
Jun  1 10:00:09 stables-router kernel: ip_set: protocol 6
Jun  1 10:00:31 stables-router pppd[1048]: pppd 2.4.4 started by root, uid 0
Jun  1 10:00:31 stables-router pppd[1048]: Connected to 00:30:88:00:00:06 via interface eth0
Jun  1 10:00:31 stables-router pppd[1048]: Connect: ppp0 <--> eth0Jun  1 10:00:31 stables-router zebra[497]: interface ppp0 index 5 <POINTOPOINT,NOARP,MULTICAST> added.
Jun  1 10:00:31 stables-router pppd[1048]: CHAP authentication succeeded
Jun  1 10:00:31 stables-router pppd[1048]: peer from calling number 00:30:88:00:00:06 authorized
Jun  1 10:00:31 stables-router zebra[497]: warning: PtP interface ppp0 with addr xxx.xxx.xxx.xxx/32 needs a peer address
Jun  1 10:00:31 stables-router zebra[497]: interface index 5 was renamed from ppp0 to pppoe0
Jun  1 10:00:32 stables-router zebra[497]: interface pppoe0 index 5 changed <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>.
Jun  1 10:00:32 stables-router pppd[1048]: local  IP address xxx.xxx.xxx.xxx
Jun  1 10:00:32 stables-router pppd[1048]: remote IP address 212.87.69.130
Jun  1 10:00:32 stables-router pppd[1048]: primary   DNS address 212.87.64.7
Jun  1 10:00:32 stables-router pppd[1048]: secondary DNS address 212.87.64.10
Jun  1 10:00:32 stables-router zebra[497]: interface eth0 mtu changed from 1500 to 1508

 

Somehow the pppd needs to stop dialing the pppoe0 session until the MTU has been changed on the eth0 interface so that the pppoe0 interface can come up with a 1500 byte session.  I dont want to use MSS clamping as 1500 MTU is fully supported in the UK.

Can this be done and changed for the next release of the software?

If i issue a disconnect interface pppoe0 and then a connect interface pppoe0 this resolves the issue

Is there a way to make this happen after the router has finished booting via a script until we can get a fix inplace?

If you need any further info then please let me know.

Thanks

Rich


Accepted Solutions
Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5465
Solutions: 1656
Contributions: 2

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@lasersailing wrote:

That works nicely Man Happy  pppoe0 now initiates at full 1500 MTU initially.  Happy days.


Could you also try another approach: edit the file "/opt/vyatta/share/vyatta-cfg/templates/interfaces/ethernet/node.tag/pppoe/node.def" on the router and add a line "priority: 400" after the line "tag:". If that also works we could make that change in the next release. Thanks!

View solution in original post


All Replies
Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5465
Solutions: 1656
Contributions: 2

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Yeah this boot ordering issue has been discussed before and we need to look into and address it. Thanks for testing and reporting this.

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Is there any possibility of looking to get the fixed in the next release?  Im suprised it wasnt fixed in the 1.5 release a few weeks ago?

Certanily in the UK PPPoE is being used much more now as FTTC is taking off in large numbers and it will support 1500 MTU so this is a bit of a pain.

I have come up with a temporary workaround to fix this but its not pretty!

root@router:/# cat /config/scripts/post-config.d/reconnect-pppoe0
#!/bin/bash
run=/opt/vyatta/bin/vyatta-op-cmd-wrapper
sleep 180
$run disconnect interface pppoe0
$run connect interface pppoe0

So basically after 3 minutes of booting it will disconnect and re-connect the pppoe interface.  This should be long enough to allow the FTTC Modem to get sync with the DSLAM in the cab before.

Maybe someone has a better suggestion?

Is there a timescale for when this may be fixed?

Thanks

Rich

Regular Member
Posts: 337
Registered: ‎06-08-2013
Kudos: 166
Solutions: 19

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Rich,

If you don't mind me asking, who's your ISP? I've still not got 1500Byte frames working on a BT Consumer line.

Cheers,

Matt

Veteran Member
Posts: 5,417
Registered: ‎03-12-2011
Kudos: 2711
Solutions: 128

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@lasersailing wrote:

root@router:/# cat /config/scripts/post-config.d/reconnect-pppoe0 #!/bin/bash run=/opt/vyatta/bin/vyatta-op-cmd-wrapper sleep 180 $run disconnect interface pppoe0 $run connect interface pppoe0

 


Hmm while I haven't run into this particular issue for whatever reason (perhaps my ISP's RADIUS server is slower than yours - next time I restart my router I should have a look and see if I can see what the exact timings are, all I know is it connects with a full 1500 byte MTU after a reboot without issues), it seems like a more reliable hack would be something like:

root@router:/# cat /config/scripts/pre-config.d/eth0-mtu
#!/bin/bash
ip link set dev eth0 mtu 1508

Note that's pre-config not post-config. This should ensure that the MTU of your eth interface is 1508 before any of the vyatta stuff gets done, which of course means it should happen before PPPoE comes up too.

This assumes that the vyatta config setting the MTU to 1508 when it's already 1508 will result in a no-op and not have any side-effects. I haven't tested it but it seems like it'd be less prone to failure than cycling the PPPoE interface!

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Matt

I work for an ISP called Timico as a Network Engineer, so I have my line through them.  Our LNS and radius platform is extremley fast and so it is responding very quickly.

Are you running ADSL or FTTC?  If ADSL what modem are you using?

Rich

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

NVX

Ill try that and report back later on..

Rich

Regular Member
Posts: 337
Registered: ‎06-08-2013
Kudos: 166
Solutions: 19

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Rich,

I'm on BT FTTC. I'm hoping to find someone on the same service who's got 1500Byte frames working :-)

Cheers,

Matt

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Matt

I presume you are on BT Infinity then? Presuming that you dont use the BT home hub and just have an Openreach mdoem.

If so which model of the Openreach modem do you have?  They either use ECI or Huawei

Differences can be seen here:

http://wiki.aa.org.uk/FTTC_Modem

I have 1500 MTU working on 21CN ADSL, now i understand the MTU issues fully within the EdgeRouter im now going to move onto the FTTC.  Should be pretty easy.

 

Rich 

Regular Member
Posts: 337
Registered: ‎06-08-2013
Kudos: 166
Solutions: 19

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Yeah, BT Infinity, with the ECI modem plugging into an ERL.

I've seen a number of people that have got it working using ISPs such as yours with an LNS based BTWholesale product but BT Infinity (and EE) use the BTW PTA product where the BRAS is within the BT network and I've not seen anyone who has 1500Byte frames working in this setup.

I'm going to dive into things a bit deeper on Thursday (hopefully) to find out exactly what's happening, so posts like yours are helpful as they give me things to look for when debugging :-) 

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

[ Edited ]

NVX

That works nicely Man Happy  pppoe0 now initiates at full 1500 MTU initially.  Happy days.

Jun  1 10:00:07 stables-router kernel: eth0: 1000 Mbps Full duplex, port 0
Jun  1 10:00:07 stables-router kernel: eth1: 1000 Mbps Full duplex, port 1
Jun  1 10:00:07 stables-router kernel: eth2: 100 Mbps Full duplex, port 2
Jun  1 10:00:08 stables-router zebra[497]: Zebra 0.99.20.1 starting: vty@0
Jun  1 10:00:09 stables-router kernel: ip_set: protocol 6
Jun  1 10:00:09 stables-router zebra[497]: interface eth0 mtu changed from 1500 to 1508
Jun 1 10:00:31 stables-router pppd[1053]: pppd 2.4.4 started by root, uid 0 Jun 1 10:00:31 stables-router pppd[1053]: Connected to 00:30:88:00:00:06 via interface eth0 Jun 1 10:00:31 stables-router pppd[1053]: Connect: ppp0 <--> eth0
Jun 1 10:00:31 stables-router zebra[497]: interface ppp0 index 5 <POINTOPOINT,NOARP,MULTICAST> added. Jun 1 10:00:31 stables-router pppd[1053]: CHAP authentication succeeded Jun 1 10:00:31 stables-router pppd[1053]: peer from calling number 00:30:88:00:00:06 authorized Jun 1 10:00:31 stables-router zebra[497]: warning: PtP interface ppp0 with addr xxx.xxx.xxx.xxx/32 needs a peer address Jun 1 10:00:31 stables-router zebra[497]: interface index 5 was renamed from ppp0 to pppoe0 Jun 1 10:00:32 stables-router zebra[497]: interface pppoe0 index 5 changed <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>.

 Thanks for that suggestion, much clearner until ubnt can get the code sorted.

Now onto FTTC 1500 MTU.....

Rich

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Matt

Just out of interest is your edgerouter connected up to your FTTC modem at 1Gb or has it negiotated at 100Mbps?

Rich

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5465
Solutions: 1656
Contributions: 2

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@lasersailing wrote:

That works nicely Man Happy  pppoe0 now initiates at full 1500 MTU initially.  Happy days.


Could you also try another approach: edit the file "/opt/vyatta/share/vyatta-cfg/templates/interfaces/ethernet/node.tag/pppoe/node.def" on the router and add a line "priority: 400" after the line "tag:". If that also works we could make that change in the next release. Thanks!

Highlighted
Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

ancheng

Yep that also works a treat:

Jun  1 10:00:07 stables-router kernel: Interface 0 has 3 ports (RGMII)
Jun  1 10:00:07 stables-router kernel: eth0: 1000 Mbps Full duplex, port 0
Jun  1 10:00:07 stables-router kernel: eth1: 1000 Mbps Full duplex, port 1
Jun  1 10:00:07 stables-router kernel: eth2: 100 Mbps Full duplex, port 2
Jun  1 10:00:08 stables-router zebra[497]: Zebra 0.99.20.1 starting: vty@0
Jun  1 10:00:09 stables-router kernel: ip_set: protocol 6
Jun  1 10:00:30 stables-router zebra[497]: interface eth0 mtu changed from 1500 to 1508
Jun  1 10:00:32 stables-router ntpd[1037]: ntpd 4.2.6p2@1.2194 Thu Nov 22 02:26:00 UTC 2012 (1)
Jun  1 10:00:32 stables-router ntpd[1038]: proto: precision = 60.518 usec
Jun  1 10:00:34 stables-router ntpd_intres[1040]: host name not found: 0.ubnt.pool.ntp.org
Jun  1 10:00:34 stables-router ntpd_intres[1040]: host name not found: 1.ubnt.pool.ntp.org
Jun  1 10:00:34 stables-router ntpd_intres[1040]: host name not found: 2.ubnt.pool.ntp.org
Jun  1 10:00:34 stables-router ntpd_intres[1040]: host name not found: 3.ubnt.pool.ntp.org
Jun  1 10:00:37 stables-router pppd[1185]: pppd 2.4.4 started by root, uid 0
Jun  1 10:00:37 stables-router pppd[1185]: Connected to 00:30:88:00:00:06 via interface eth0
Jun  1 10:00:37 stables-router pppd[1185]: Connect: ppp0 <--> eth0
Jun  1 10:00:37 stables-router zebra[497]: interface ppp0 index 5 <POINTOPOINT,NOARP,MULTICAST> added.
Jun  1 10:00:38 stables-router ntpd[1038]: ntpd exiting on signal 15
Jun  1 10:00:38 stables-router pppd[1185]: CHAP authentication succeeded
Jun  1 10:00:38 stables-router pppd[1185]: peer from calling number 00:30:88:00:00:06 authorized
Jun  1 10:00:38 stables-router zebra[497]: warning: PtP interface ppp0 with addr xxx.xxx.xxx.xxx/32 needs a peer address
Jun  1 10:00:38 stables-router zebra[497]: interface index 5 was renamed from ppp0 to pppoe0
Jun  1 10:00:38 stables-router zebra[497]: interface pppoe0 index 5 changed <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>.

 NTP seems a bit keen to get a response from the timeservers even though no connectivity is up!

Which solution is better and cleaner?

Thanks

Rich

New Member
Posts: 6
Registered: ‎07-15-2014
Kudos: 3
Solutions: 1

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@Mephi wrote:

I've seen a number of people that have got it working using ISPs such as yours with an LNS based BTWholesale product but BT Infinity (and EE) use the BTW PTA product where the BRAS is within the BT network and I've not seen anyone who has 1500Byte frames working in this setup.


RFC 4638 states the following:

5.1 MRU Negotiations

Since Ethernet (without jumbo frames) has a maximum payload size of 1500 octets, the PPPoE header is 6 octets, and the PPP Protocol ID is 2 octets, the Maximum-Receive-Unit (MRU) option MUST NOT be negotiated to a size larger than 1492, unless both the PPPoE client and server have indicated the ability to support a larger MRU in the PPPoE Discovery Stage.

Since the ability to support a larger MRU requires the exchange of the "PPP-Max-Payload" tag, and EdgeOS doesn't currently support this, BT are quite correct in their behaviour.

That forcing it to 1500 works for some other ISPs is due to those ISPs not correctly implementing the RFC and forcing it to 1500 could have unforeseen circumstances.

The only real solution is to wait unti EdgeOS correctly supports the "PPP-Max-Payload" tag.  OpenWrt has only just got this capability too.

Emerging Member
Posts: 43
Registered: ‎04-14-2014
Kudos: 6

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

Middling

Yes you are right, we run our own LNS's so control the MTU ourselves and its being negioated properly.  I can see this from looking at our core LNS's to see the MTU that has been negioated.  This means that as you say, its ISP dependant.

RFC 4638 Information

Some pointers regarding RFC 4638, PPPoE and Linux-based routers.

  1. The wan interface needs to be set to an MTU of 1508 on the assumption that there needs to be room for the 1500 mtu PPP package on the way to the modem (1500 may work - untested though).
  2. You will require the unreleased ppp 2.4.6. In testing, ppp 2.4.5 with the rp-pppoe plugin updated to the latest version seems to work, although a recompile will be necessary. This can be fetched from ppp's website / git.
  3. Using ppp's debug option will allow you to see the rp-pppoe plugin negotiating a PPP-Max-Payload option with the BRAS via a PPPoE option. If this isn't seen, then your MTU is forced back to 1492.

Ensure you're using kernel 2.6.34+ or 2.6.36.3+

Edge Router is using:

ii  ppp                                                   2.4.5-4                                               Point-to-Point Protocol (PPP) - daemon

and:

Linux stables-router 3.4.27-UBNT #1 SMP Thu Jun 19 18:21:34 PDT 2014 mips64 GNU/Linux

 So all is ok there and supported.

Rich

 

Regular Member
Posts: 337
Registered: ‎06-08-2013
Kudos: 166
Solutions: 19

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


Middling wrote: Since the ability to support a larger MRU requires the exchange of the "PPP-Max-Payload" tag, and EdgeOS doesn't currently support this, BT are quite correct in their behaviour.

The only real solution is to wait unti EdgeOS correctly supports the "PPP-Max-Payload" tag.  OpenWrt has only just got this capability too.


I'm not sure that's the case, I took a packet capture of my PPPOE session initialisation and I see a PPP-Max-Payload: 05dc listed for both the PADI from the ERL and the PADO from the BRAS.

 

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5465
Solutions: 1656
Contributions: 2

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@lasersailing wrote:

Which solution is better and cleaner?


Either way should work, but we probably should use the priority approach since using a "pre-config" script to fix our own configuration seems a bit ugly Icon Lol  Thanks for testing!

New Member
Posts: 6
Registered: ‎07-15-2014
Kudos: 3
Solutions: 1

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0


@Mephi wrote:

I'm not sure that's the case, I took a packet capture of my PPPOE session initialisation and I see a PPP-Max-Payload: 05dc listed for both the PADI from the ERL and the PADO from the BRAS.


 Interesting.  I thought EdgeOS was using an old Vyatta version of PPP that didn't support RFC 4638 for PPPoE connections.  Can you try running PPP with debug options so that the PPP-Max-Payload negotiation is logged.

I don't yet have an Edgerouter (this was one of the few issues holding me back so, if RFC 4638 actually works, i may have to push the purchase up my list.  So many projects, not enough money ^_^), but i do have an RFC 4638 1500 MTU connection running using OpenWrt Barrier Breaker with my ISP, Zen.

I had to set the PPPoE MTU to 1508, same as underlying ethernet interface, for it to work.  Perhaps you could try that and enable debug options and report back?

Regular Member
Posts: 337
Registered: ‎06-08-2013
Kudos: 166
Solutions: 19

Re: BUG: PPPoE client pppoe dialing before MTU set on eth0

I'm out at IETF for the week, so I can't make changes for a while. I want to get to the bottom of my RFC4836 issues so I'll have a go when I get back.

I never tried setting the pppoe interface to 1508, I'll add that to my list :-)
Reply