Reply
New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

@UBNT-afomins

 

>> I've attached a log file of the debug output requested for the loss of BGP routes to the AWS VPN.
>> Feb 12 08:17:32 EdgeRouter ubnt-protocols-cfg[1849]: /usr/bin/vtysh-set -c configure terminal -c router bgp 65001 -c neighbor 169.254.12.57 timers 30 30 failed: 10752
>Looks like "bgpd" configuration failed. Maybe timer values are incorrect? Please post full BGP configuration.

 

 


Note that the neighbor 169.254.12.57 in the log file above turned out to be a leftover from a prior VPN setup. I deleted it but am still getting a similar error:

 

Feb 14 01:08:06 EdgeRouter BGP[1796]: BGP-6: BGPd 1.2.0 starting: vty@0, bgp@179
Feb 14 09:09:22 EdgeRouter ubnt-protocols-cfg[1834]: /usr/bin/vtysh-set -c configure terminal -c router bgp 65001 -c neighbor 169.254.13.25 timers 10 30 failed: 10752

 

Here's the BGP configuration:

 

protocols {
   bgp 65001 {
      neighbor 169.254.13.25 {
         remote-as 7224
         soft-reconfiguration {
            inbound
         }
         timers {
            holdtime 30
            keepalive 10
         }
      }
      neighbor 169.254.15.209 {
         remote-as 7224
         soft-reconfiguration {
            inbound
         }
         timers {
            holdtime 30
            keepalive 10
         }
      }
      network 192.168.1.0/24 {
      }
   }
   }
}

New Member
Posts: 10
Registered: ‎01-09-2014
Kudos: 5

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@n0dyjeffwrote:

I upgraded my Edgeroute Lite-3 this morning from 1.9.7+hotfix 4 and lost both BGP routes to the AWS VPN. This persisted through several reboots of the ERL3. After reverting back to the 1.9.7+hotfix 4 version, the BGP routes came back. Anyone else experiencing issues in this area?


I had the exact issue. If I did "configure" then "show protocols", my BGP configuration was missing. If I tried to add it again I got a message saying:

[ protocols bgp 65000 ]
The same object already exists
Error configuring routing subsystem. See log for more detailed information

Commit failed

If I looked in /config/config.boot, the "protocols bgp 65000" node was in there. Deleting it didn't help.

I've reverted to 1.9.7+hotfix 4. Hopefully it gets fixed soon!

New Member
Posts: 2
Registered: ‎07-06-2017

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

[ Edited ]

@UBNT-afominsthanks I can upgrade now

New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@jeaguilarwrote:

I had the exact issue. If I did "configure" then "show protocols", my BGP configuration was missing. If I tried to add it again I got a message saying:

[ protocols bgp 65000 ]
The same object already exists
Error configuring routing subsystem. See log for more detailed information

Commit failed

If I looked in /config/config.boot, the "protocols bgp 65000" node was in there. Deleting it didn't help.

I've reverted to 1.9.7+hotfix 4. Hopefully it gets fixed soon!


I'm also missing the BGP configuration when "show configuration" in V1.10.0. The BGP configuration I posted is the configuration from V.1.09, which presumably is being carried over to V1.10 but just isn't showing for some reason.

New Member
Posts: 10
Registered: ‎01-09-2014
Kudos: 5

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@n0dyjeffwrote:

I'm also missing the BGP configuration when "show configuration" in V1.10.0. The BGP configuration I posted is the configuration from V.1.09, which presumably is being carried over to V1.10 but just isn't showing for some reason.


Workaround!

I reverted to 1.9.7+hotfix 4 and deleted my BGP configuration. Then I upgraded to 1.10.0 and tried to add my BGP configuration back.

set protocols bgp 65000 neighbor 169.254.44.101 remote-as 7224
set protocols bgp 65000 neighbor 169.254.44.101 soft-reconfiguration inbound 
set protocols bgp 65000 neighbor 169.254.44.101 timers holdtime 30      
set protocols bgp 65000 neighbor 169.254.44.101 timers keepalive 10

That resulted in a failure:

    [ protocols bgp 65000 ]
    Starting routing daemon: bgpd.

    [ protocols bgp 65000 ]
    Warning: Configured holdtime is set below the default value
    Error configuring routing subsystem. See log for more detailed information

    Commit failed

The log indicated the following error:

    ubnt-protocols-cfg[3892]: /usr/bin/vtysh-set -c configure terminal -c router bgp 65000 -c neighbor 169.254.44.101 soft-reconfiguration inbound failed: 10752

So I tried with the minimum configuration:

set protocols bgp 65000 neighbor 169.254.44.101 remote-as 7224

No "timers holdtime 30", "timers keepalive 10", or "soft-reconfiguration inbound", and, voila! This time BGP came back up. It looks like an issue specifically with "soft-reconfiguration inbound".

New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@UBNT-afomins
@jeaguilarwrote:

No "timers holdtime 30", "timers keepalive 10", or "soft-reconfiguration inbound", and, voila! This time BGP came back up. It looks like an issue specifically with "soft-reconfiguration inbound".


I can exonerate the "soft-reconfiguration inbound". I followed your workaround by deleting both the "timers holdtime 30" and "timers keepalive 10" items from the configuration, but left the "soft-reconfiguration inbound" in place (from the 1.09 configuration). Then loaded V1.10.0 and the VPN came up. In summary, here's the BGP configuration that works:

 

 bgp 65001 {
     neighbor 169.254.13.25 {
         remote-as 7224
         soft-reconfiguration {
             inbound
         }
     }
     neighbor 169.254.15.209 {
         remote-as 7224
         soft-reconfiguration {
             inbound
         }
     }
     network 192.168.1.0/24 {
     }
 }

And here's the configuration that DOESN'T work:

bgp 65001 {
   neighbor 169.254.13.25 {
      remote-as 7224
      soft-reconfiguration {
         inbound
      }
      timers {
         holdtime 30
         keepalive 10
      }
   }
   neighbor 169.254.15.209 {
      remote-as 7224
      soft-reconfiguration {
         inbound
      }
      timers {
         holdtime 30
         keepalive 10
      }
   }
   network 192.168.1.0/24 {
   }
}
New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@n0dyjeffwrote:

@UBNT-afomins
@jeaguilarwrote:

No "timers holdtime 30", "timers keepalive 10", or "soft-reconfiguration inbound", and, voila! This time BGP came back up. It looks like an issue specifically with "soft-reconfiguration inbound".



As a workaround to removing the timers from specific neighbors, I've found that V1.10.0 will accept global timer configurations. The following BGP config also works:

 bgp 65001 {
     neighbor 169.254.13.25 {
         remote-as 7224
         soft-reconfiguration {
             inbound
         }
     }
     neighbor 169.254.15.209 {
         remote-as 7224
         soft-reconfiguration {
             inbound
         }
     }
     network 192.168.1.0/24 {
     }
     timers {
         holdtime 30
         keepalive 10
     }
 }

 

Member
Posts: 190
Registered: ‎04-28-2015
Kudos: 105
Solutions: 1

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

@UBNT-afomins,

 

I believe the issue may be with the BGP keepalive/hold timers.

 

Upon migrating from a 1.9.7 built directly to 1.10 on an ER-Pro8, I had one router fully eject the "protocols bgp" tree from the configuration.  Upon looking at the boot-up commit log, there was a warning message that the hold timer was less than the default, and the config engine seems to have treated that as a fatal error and just ejected the protocols bgp and child clauses.

 

Removing the timers from the config allowed the configuration to load properly upon reset of the router.

 

 

New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@matthardemanwrote:

@UBNT-afomins,

 

I believe the issue may be with the BGP keepalive/hold timers.

 

Upon migrating from a 1.9.7 built directly to 1.10 on an ER-Pro8, I had one router fully eject the "protocols bgp" tree from the configuration.  Upon looking at the boot-up commit log, there was a warning message that the hold timer was less than the default, and the config engine seems to have treated that as a fatal error and just ejected the protocols bgp and child clauses.

 

Removing the timers from the config allowed the configuration to load properly upon reset of the router.

 

 


Agreed. Specifically, the error occurs with the hold timer. The keepalive timer can be configured for neighbors without issue, but the command 

set protocols bgp 65001 neighbor 169.254.15.209 timers holdtime 30

causes the warning message and the commit fails. A workaround is to configure the hold timer globally.

 

Member
Posts: 186
Registered: ‎12-11-2013
Kudos: 213
Solutions: 7

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

[ Edited ]

@briellewrote:

 

Maybe its because I've been a network admin since the mid 90s, but...  I beg to differ about rotuting protocols having nothing to do with security.  I'll leave it up to your imagination what an unauthenticated routing protocol that has historically been used to cause havoc and on networks when misconfigured or left wide open has to do with security.  And yes, I'm aware of RIPv2 and its md5 auth, just I'm sure you are aware of the dangers of any of the routing protocols when not properly secured even with authentication.

Regardless, it was a valid question regardless of what undertones you may have thought I had behind it.


 

It's a bit arrogant to assume you have more experience or knowledge than others, isn't it?  I have nothing to prove here.  My only concern is to make sure UBNT gives the issue attention.

 

Your question wasn't valid, because it was responding to a bug report with a suggestion that the feature is a security risk and should be avoided as justification.  Simply put, that is not helpful.

 

To be more specific on why your assertion is incorrect, other routing protocols suffer from the same "security" concerns if not properly configured.  Properly configuring in this case involves things like disable RIP on interfaces that aren't links between routers, which are in turn secured by other means.  The exact feature of RIP in question is one used to safeguard against routing loops.  Another mechanisim is to restrict what prefixes are permitted to be recieved as well as advertised.  Any network engineer who's been working on networks "since the 90's" should be aware of all this, and further, having any understanding of BGP or OSPF should laugh at the idea of either being secure.

 

I'm confused at what point you're trying to make.

Member
Posts: 186
Registered: ‎12-11-2013
Kudos: 213
Solutions: 7

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

@UBNT-afomins when can we expect a fix for the issue(s) below?  Will 1.10 be pulled back from release?



@awhenry41wrote:

@UBNT-afomins  We have discovered what, for us, is a critical flaw in the 1.10 firmware, and anyone using RIP on an Edgemax device should beware of this.  The 1.10 firmware does not properly parse the RIP split-horizon poison-reverse configuration command.  This results in any router using this configuration to fail after the 1.10 upgrade, as uplink interfaces go unconfigured, leaving the devices unreachable.  Furthermore (though less serious), the router will accept but cannot commit a configuration including RIP split-horizon poison-reverse.  Here are the results of such an attempt:

 set interfaces ethernet eth6 vif 2 ip rip split-horizon poison-reverse
[edit]
# commit
[ interfaces ethernet eth6 vif 2 ip rip split-horizon poison-reverse ]
Params not configured

Commit failed

Here is the output of the vyatta-commit.log file on a router where an upgrade to 1.10 was attempted:

[ policy ]
Starting routing daemon: ripd ripngd ospfd ospf6d bgpd.

[ interfaces ethernet eth6 vif 3 ip rip split-horizon poison-reverse ]
Params not configured

[ interfaces ethernet eth6 vif 2 ip rip split-horizon poison-reverse ]
Params not configured

[ system ntp ]
Stopping NTP server: ntpd.
Starting NTP server: ntpd.

[ system ip arp base-reachable-time 30 ]
sysctl: cannot stat /proc/sys/net/ipv4/neigh/eth6/2/base_reachable_time_ms: No such file or directory
sysctl: cannot stat /proc/sys/net/ipv4/neigh/eth6/3/base_reachable_time_ms: No such file or directory

[ system ip arp stale-time 60 ]
sysctl: cannot stat /proc/sys/net/ipv4/neigh/eth6/2/gc_stale_time: No such file or directory
sysctl: cannot stat /proc/sys/net/ipv4/neigh/eth6/3/gc_stale_time: No such file or directory

[ service ssh ]
The SSH service will be started after commit. Check /var/log/messages.

[ protocols rip passive-interface default ]
Warning: default value is deprecated

[ service dhcp-relay ]
Stopping dhcrelay:  OK
Starting dhcrelay:  OK

Commit failed

So, beware if you use RIP at all, and if you specifically make use of RIP split-horizon poison-reverse *DO NOT UPGRADE TO 1.10*.  1.9.7 does not suffer from this issue.

 

This is trivial to reproduce.  Hoping for a quick fix.

 

Thank you.


 

New Member
Posts: 2
Registered: a week ago

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

[ Edited ]

My EdgeRouter X SFP seems to have been bricked after upgrade. Upgraded from 1.9.7+hotfix4 via the GUI. GUI are not reachable anymore after reboot.

 

Did a factory reset but could still not reach or ping the router. The connected eth port LED is however blinking...

 

Is there any solution for this or do i need to send it to repair?

Ubiquiti Employee
Posts: 919
Registered: ‎07-20-2015
Kudos: 797
Solutions: 70

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

[ Edited ]

@doc_karl
> upgrading from 1.9.1 to this version on the edgepoint R6 also removed all the physical interfaces from my switch group - I needed to go in a reenable them.
> No once I re-add the physical interfaces to the switch group within 1.10 they seem to persist - but during the upgrade process from 1.9.1 they disappeared...
That's weird because I was not able to reproduce this issue after upgrading from 1.9.1 to 1.10.0. Please post your switch0 configuration.

 

@ryanm
> 1.9.7hf4 works fine. That's what I downgraded back to. I just upgraded it again to get you the screenshots.
Ok, now I understand what you mean, however I can not reproduce this yet because I do not have EP-R6 in my lab (it should arrive next week).
I tested switch0 VLAN configuration on ER-X-SFP (it is based on same hardware as EP-R6) and it works fine.

Can you please try configuring v1.10.0 switch0 via CLI like so:

set interfaces switch switch0 address 192.168.2.1/24
set interfaces switch switch0 mtu 1500
set interfaces switch switch0 switch-port interface eth1 vlan pvid 14
set interfaces switch switch0 switch-port interface eth2 vlan pvid 1041
set interfaces switch switch0 switch-port interface eth2 vlan vid 14
set interfaces switch switch0 switch-port vlan-aware enable


Does it work on EP-R6 v1.10.0?

 

@n0dyjeff
> Feb 12 08:17:32 EdgeRouter ubnt-protocols-cfg[1849]: /usr/bin/vtysh-set -c configure terminal -c router bgp 65001 -c neighbor 169.254.12.57 timers 30 30 failed: 10752
> timers {
> holdtime 30
> keepalive 10
> }
@jeaguilar
> I had the exact issue. If I did "configure" then "show protocols", my BGP configuration was missing.

Root cause of the failure is that your holdtime is less than default value in routing daemon. This did not cause any issues in prior f/w versions because we used to ignore all warning originating from routing daemon and used default values instead.
To fix this issue you should set holdtime >= 90:

set protocols bgp 65001 neighbor 169.254.13.25 timers holdtime 90
set protocols bgp 65001 neighbor 169.254.15.209 timers holdtime 90


@Frixzon
> My EdgeRouter X SFP seems to have been bricked after upgrade
> Did a factory reset but could still not reach or ping the router.
You can try connecting ER-X-SFP via ssh-recovery from neighbor device (your laptop or any Linux router in same network):
https://community.ubnt.com/t5/EdgeMAX-Beta/new-feature-explaining-ssh-recovery-service-in-v1-10-0-al...

New Member
Posts: 17
Registered: ‎05-30-2016
Kudos: 1

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

And again lost a part of the config, pptpc not comming up, if I click disable.

 

 The specified configuration node is not valid
interfaces pptp-client pptpc0 disable
 
So they are in the config but not completly, this issue plays each upgrade, off course no feedback of what's missing.
 
======
 
So I go in cli config is complete, disconnect interface ... and connect interface ... and solved.  Strange
New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@UBNT-afominswrote:

Root cause of the failure is that your holdtime is less than default value in routing daemon. This did not cause any issues in prior f/w versions because we used to ignore all warning originating from routing daemon and used default values instead.
To fix this issue you should set holdtime >= 90:

set protocols bgp 65001 neighbor 169.254.13.25 timers holdtime 90
set protocols bgp 65001 neighbor 169.254.15.209 timers holdtime 90

 


Understood and thanks for the research and response. However, I should note that the holdtime figure of 30 comes from the AWS specified configuration for Vyatta devices using their VPN / VPC setup. It seems likely that others using the AWS VPN will encounter this issue in the future, unless either AWS changes their recommended configuration or something changes on the EdgeMAX side. Here is the relevant section of the AWS configuration recommendations:

! #4: Border Gateway Protocol (BGP) Configuration
!
! BGP is used within the tunnel to exchange prefixes between the
! Virtual Private Gateway and your Customer Gateway. The Virtual Private Gateway
! will announce the prefix corresponding to your VPC.
!
! Your Customer Gateway may announce a default route (0.0.0.0/0),
! which can be done with the 'network' statement.
!
! The BGP timers are adjusted to provide more rapid detection of outages.
!
! The local BGP Autonomous System Number (ASN) (65001) is configured
! as part of your Customer Gateway. If the ASN must be changed, the
! Customer Gateway and VPN Connection will need to be recreated with AWS.
!

set protocols bgp 65001 neighbor 169.254.15.209 remote-as '7224'
set protocols bgp 65001 neighbor 169.254.15.209 soft-reconfiguration 'inbound'
set protocols bgp 65001 neighbor 169.254.15.209 timers holdtime '30'
set protocols bgp 65001 neighbor 169.254.15.209 timers keepalive '10'

! To advertise additional prefixes to Amazon VPC, replace the 0.0.0.0/0 from the
! the following line with the prefix you wish to advertise. Make sure the prefix is present
! in the routing table of the device with a valid next-hop.

set protocols bgp 65001 network 0.0.0.0/0
New Member
Posts: 32
Registered: ‎10-21-2014
Kudos: 3

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


  • Failover load-balancing stops working after reconfiguration (it works fine when configuring load-balancing for the first time or after reboot). Workaround - reset ubnt-util daemon after reconfiguring load-balancing:


No Thanks...

 

Why is this released with this "known issue"?

 

New Member
Posts: 22
Registered: ‎02-08-2016
Kudos: 11
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!


@n0dyjeffwrote:

@UBNT-afominswrote:

Root cause of the failure is that your holdtime is less than default value in routing daemon. This did not cause any issues in prior f/w versions because we used to ignore all warning originating from routing daemon and used default values instead.
To fix this issue you should set holdtime >= 90:

set protocols bgp 65001 neighbor 169.254.13.25 timers holdtime 90
set protocols bgp 65001 neighbor 169.254.15.209 timers holdtime 90

 


 

I should also mention that UBNT's own instructions for connecting to AWS VPN call out a holdtime of 30. 

11. Define the BGP neighbors and peering options.

set protocols bgp 65000 timers holdtime 30
set protocols bgp 65000 timers keepalive 10
set protocols bgp 65000 network 192.168.1.0/24
New Member
Posts: 29
Registered: ‎12-21-2016
Kudos: 7
Solutions: 1

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

@UBNT-afomins

> I will upload 1.10.0 and GPL code to download.ubnt.com next week

 

 

Can you please upload the GPL code.

Member
Posts: 104
Registered: ‎07-30-2014
Kudos: 19

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

I usually wait when new firmware is released but after reading everyone's responses I decided to make the leap.  Initial impressions are favorable.  Speedtest was perfect with bufferbloat and jitter reduced.  Before the upgrade the browser was running sluggish.  After the upgrade from v1.9 to v1.10.0 the responsiveness is markably improved.  Be sure to remove old firmware before uploaded the new image:

 

SSH (Putty works) to the router IP and directly after log in use the following CLI Commands:

show  system image <- if you only have one image marked "running" then exit.
delete system image <- enter cmd like this and it will prompt to delete the old image.

exit SSH Session.
via GUI: now upload the new firmware and apply.

New Member
Posts: 27
Registered: 3 weeks ago
Kudos: 2
Solutions: 1

Re: EdgeMAX EdgeRouter software version v1.10.0 has been released!

Actually it looks like it just got released today:

https://www.ubnt.com/download/edgemax
Reply