Reply
New Member
Posts: 3
Registered: ‎06-24-2015
Kudos: 2

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

Thanks, this resolved the issue with default routes not populating in tables 201 and 202 when using LB with dynamic IPs.

There is still an issue with lb-local-metric-change. It doesn't change the metric until ubnt-util is restarted with kill -kill `pidof ubnt-util`
Ubiquiti Employee
Posts: 996
Registered: ‎07-20-2015
Kudos: 899
Solutions: 71

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@xpoinsar
> But now it's completely refusing to work. Even factory reset doesn't unblock it.
> 14:01:35 crit ubnt kern kernel EXT3-fs (sda2): error in ext3_orphan_add: Journal has aborted>
> 2018-04-04 14:01:35 crit ubnt kern kernel EXT3-fs (sda2): error in ext3_free_blocks_sb: Journal has aborted
> 2018-04-04 14:01:35 crit ubnt kern kernel EXT3-fs (sda2): error: remounting filesystem read-only
> 2018-04-04 14:01:35 crit ubnt kern kernel EXT3-fs (sda2): error: ext3_journal_start_sb: Detected aborted journal
> 2018-04-04 14:01:35 err ubnt kern kernel Aborting journal on device sda2.
> 2018-04-04 14:01:35 alert ubnt kern kernel journal_bmap: journal block not found at offset 2198 on sda2
Looks like filesystem corruption or bad flash. What's ER model?

 

@ggiersdorf
> Just a point of reference after I upgraded my Edge Router X to 1.10.1 I found I had major speed issues
> I have a gig fibre connection and NO MATTER what I did I couldn't get more than 527mb
> I eventually downgraded but was still unable to get full bandwidth

  1. Have been able to get 1Gbps with previous EdgeOS firmware version? Which version?
  2. Maybe WAN interface is configured in hal-duplex mode? Please show output of "sudo ethtool ethX" (change ethX to your WAN interface)
New Member
Posts: 10
Registered: ‎09-06-2016

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@UBNT-afomins

Its an ER-Lite model.

Emerging Member
Posts: 69
Registered: ‎12-18-2017
Kudos: 8
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!


@UBNT-afomins wrote:

To @tubstr, @starckIT, @PVKK, @chjohans, @flamber, @cewald, @nob0dy, @paganini, @root313 and everyone else who is experienceing problems with LB in 1.10.1 - please run following one-liner in shell to fix this issue in 1.10.1:

 

curl -O https://dl.ubnt.com/firmwares/edgemax/afomins/fix-lb-regression-in-1-10-1/ubnt-add-connected.pl && chmod +x ubnt-add-connected.pl && sudo cp ubnt-add-connected.pl /usr/sbin/

 

This one-liner will download and install patched `ubnt-add-connected.pl` script (SHA256 - 05ef0fc712cfa3df907b43db618429d14ae4400c05dfdb711cf9b9531a73ad9c)

 

This regression will be fixed in 1.10.2


 

Thank @UBNT-afomins for your thoughtfulness.

 

ATM I'm running the LB on 1.10.1 with these "tweaks" (we already discussed)

 

set load-balance group G lb-local disable
set load-balance group G lb-local-metric-change disable
set interfaces ethernet eth4 pppoe 4 default-route none
set interfaces ethernet eth3 pppoe 3 default-route none
set protocols static interface-route 0.0.0.0/0 next-hop-interface pppoe4 distance 1
set protocols static interface-route 0.0.0.0/0 next-hop-interface pppoe3 distance 2

Should I revert them before applying your .pl script, or doesn't it matter?

 

BTW, if I can take the liberty to askl, what do you think - broadly speaking - about the command

 

ip route add default scope global nexthop dev pppoe3 nexthop dev pppoe4 

about which I asked you to chime in here ? Would have it been a somewhat decent workaround?

 

And in case, should that command make anything different / better than the above quoted settings

 

set protocols static interface-route 0.0.0.0/0 next-hop-interface pppoe4 distance 1
set protocols static interface-route 0.0.0.0/0 next-hop-interface pppoe3 distance 2

or not?

 

Well, thanks in advance for your understanding in the matter and best regards!

Regular Member
Posts: 696
Registered: ‎11-10-2012
Kudos: 81
Solutions: 16

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

I do not even think LB applies to me, as i do not know what it is.

I just had a fairly simple router setup/mode, other than some CRM settings.

 

I'm hoping 1.10.1  did not leave some residual issues, as my internet now seems much slower, but it could be unrelated.

 

I only had hotfix3 to revert back to after my router just went offline, can someone refer me to hotfix4 file?

New Member
Posts: 3
Registered: ‎03-20-2018
Kudos: 1

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

[ Edited ]

@UBNT-afomins- You mentioned here that the dnsmasq bug would be patched/patch would be merged in v1.10.1 but it doesn't look like it's been fixed in the release notes?

 

Is it just missing from the notes or next version?

Member
Posts: 279
Registered: ‎10-16-2016
Kudos: 21
Solutions: 6

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

Hey guys,

 

my ER-X got totally unresponsive this night after 4 days running fine:

 

https://community.ubnt.com/t5/EdgeRouter/ER-X-hangs-up-tonight-Dont-know-why/m-p/2308183#M203984

 

I used the 10.0.1 version provided as beta before a couple of weeks here ... after reboot it came up and was fine so far.

But its very complicated to get thhe keys for the location for me Man Sad

 

So is there a possibility to do a watchdog or somethink like that ?

Emerging Member
Posts: 52
Registered: ‎06-06-2016
Kudos: 7
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@UBNT-afomins

 

since we need the link to be up, i cant test your hotfix. we are back to 1.9.7 h4. got really trouble at work the last weeks cause the link went down every day.

is there a ETA for 1.10.2 ? ..this will be the next time i give it a try

Regular Member
Posts: 413
Registered: ‎12-25-2013
Kudos: 360
Solutions: 11

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@UBNT-afomins

 

If he deletes it, then the url-filtering databases will have to be re-created and this is a terribly slow operation on EdgeRouter.

 

Much better options would be if:

 

 - this was kept outside of config tree so it doesn't have to be copied

 - this was kept inside of config tree but moved upon upgrade instead of copied

 - Ubiquity looked into terrible low performance of blacklist database creation and fixed it so it can be deleted and re-created without waiting for 2 hours

 

Senior Member
Posts: 3,606
Registered: ‎05-15-2014
Kudos: 1253
Solutions: 254

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@xpoinsar ER-L has internal USB stick for it's storage. It's easy to replace it. The long thread how to replace is HERE however simplified methods evolved as explained HERE or HERE

New Member
Posts: 10
Registered: ‎09-06-2016

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@UBNT-afomins

 

Managed to capture error with serial console cable :

Waiting 15sec before mounting root device...
usb 1-1: new high-speed USB device number 2 using OcteonUSB
usb-storage 1-1:1.0: USB Mass Storage device detected
scsi0 : usb-storage 1-1:1.0
scsi 0:0:0:0: Direct-Access              USB DISK 2.0     PMAP PQ: 0 ANSI: 6
sd 0:0:0:0: [sda] 7831552 512-byte logical blocks: (4.00 GB/3.73 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] No Caching mode page found
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] No Caching mode page found
sd 0:0:0:0: [sda] Assuming drive cache: write through
 sda: sda1 sda2
sd 0:0:0:0: [sda] No Caching mode page found
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Attached SCSI removable disk
JBD: recovery failed
EXT3-fs (sda2): error loading journal
VFS: Cannot open root device "sda2" or unknown-block(8,2): error -17
Please append a correct "root=" boot option; here are the available partitions:
1f00             512 mtdblock0  (driver?)
1f01             512 mtdblock1  (driver?)
1f02              64 mtdblock2  (driver?)
0800         3915776 sda  driver: sd
  0801          145408 sda1 00980900-01
  0802         1709056 sda2 00980900-02
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,2)

What can I do now ?

Senior Member
Posts: 3,606
Registered: ‎05-15-2014
Kudos: 1253
Solutions: 254

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@xpoinsar replace the USB stick for new one. See my post above yours on how-to.

New Member
Posts: 10
Registered: ‎09-06-2016

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

I managed to reinstall everything.

Everything is back except squiguard filtering category that is a source of issues for the moment.

Ubiquiti Employee
Posts: 996
Registered: ‎07-20-2015
Kudos: 899
Solutions: 71

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@Donnie-Brasco
> Should I revert them before applying your .pl script, or doesn't it matter?

Your dual-WAN config with fine,you can leave it and apply patch.

 

> BTW, if I can take the liberty to askl, what do you think - broadly speaking - about the command
> ip route add default scope global nexthop dev pppoe3 nexthop dev pppoe4
> Would have it been a somewhat decent workaround?
This command is not needed because you already have default gateway via pppoe3 and pppoe4 in your config

 

@Blinc
> I only had hotfix3 to revert back to after my router just went offline, can someone refer me to hotfix4 file?
All firmware images including (1.9.7+hotfix.4) are available here -> https://www.ubnt.com/download/edgemax

 

@T-Dawg
> You mentioned here that the dnsmasq bug would be patched/patch would be merged in v1.10.1 but it doesn't look like it's been fixed in the release notes?
> Is it just missing from the notes or next version?
It's my fault - I forgot to merge this fix when building "v1.10.1" firmware. Please provided workaround until this is propely fixed in "v1.10.2"

 

@nob0dy
> is there a ETA for 1.10.2
> since we need the link to be up, i cant test your hotfix.
We plan to release 1.10.2 next week and until then you can test workaround described here -> https://community.ubnt.com/t5/EdgeRouter/EdgeMAX-EdgeRouter-software-version-v1-10-1-has-been-releas...

 

@levicki
> If he deletes it, then the url-filtering databases will have to be re-created and this is a terribly slow operation on EdgeRouter.
> Much better options would be if:
> - this was kept outside of config tree so it doesn't have to be copied
> - this was kept inside of config tree but moved upon upgrade instead of copied
Here's workaround for this:

  1. Move url-filtering from /config - sudo mv /config/url-filtering ~
  2. Upgrade & reboot
  3. Restore url-filtering from old firmware storage - sudo cp -fR /root.dev/w.o/home/ubnt/url-filtering /config
  4. Reboot again 
Regular Member
Posts: 413
Registered: ‎12-25-2013
Kudos: 360
Solutions: 11

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

[ Edited ]

@BranoB

 

Also worth noting is that when USB flash fails it is sometimes caused by the power supply which is about to fail soon so it is worth checking whether supply is OK.

Ubiquiti Employee
Posts: 996
Registered: ‎07-20-2015
Kudos: 899
Solutions: 71

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

@erxowner
> I was setting up dnssec using v1.10.1 and I noticed that one of the options are wrong (options no-resolve) for dnsmasq
> I think options no-resolve should instead be options no-resolv.
Check following help message for "service dns forwarding options":

ubnt@walter# set service dns forwarding options
Possible completions:
<text> Additional options for dns forwarding. You must
       use the syntax of dnsmasq.conf in this text-field. Using this
       without proper knowledge may result in a crashed dnsmasq daemon.
       Check system log to look for errors.

According to this help message it is up to admin to specify correct option syntax.
Where did you get "options no-resolve" from? Was this mentioned in one of our articles?

 

New Member
Posts: 8
Registered: ‎02-28-2016
Solutions: 1

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

[ Edited ]

 


@UBNT-afomins wrote:

@flamber
 

To @tubstr, @starckIT, @PVKK, @chjohans, @flamber, @cewald, @nob0dy, @paganini, @root313 and everyone else who is experienceing problems with LB in 1.10.1 - please run following one-liner in shell to fix this issue in 1.10.1:

 

curl -O https://dl.ubnt.com/firmwares/edgemax/afomins/fix-lb-regression-in-1-10-1/ubnt-add-connected.pl && chmod +x ubnt-add-connected.pl && sudo cp ubnt-add-connected.pl /usr/sbin/

 

This one-liner will download and install patched `ubnt-add-connected.pl` script (SHA256 - 05ef0fc712cfa3df907b43db618429d14ae4400c05dfdb711cf9b9531a73ad9c)

 

This regression will be fixed in 1.10.2


 

Perhaps the latest firmware has been updated to be less tollerant of odd configurations like mine that have logic defying elements used to work around Edge OS limitations, but the quick fix doesn't seem to work for my load balancing issue.

 

I have a "fake" load balancer setup just to mark packets for me (see explanation here) and when it's in place during boot, route table 201 is missing the default route over eth0 and the load-balance never seems to correct itself.  If I remove the load-balance config after boot and put it back in place the route tables show up with a default route and everything works as usual.

 

Load-balance config:

load-balance
 group WAN_VPN {
     interface eth0 {
         weight 100
     }
     lb-local enable
     lb-local-metric-change enable
}

Broken route after boot:

show ip route table 201

0.0.0.0/24 dev vtun1  scope link
blackhole default  metric 256
10.8.8.0/24 dev vtun1  scope link
127.0.0.0/8 dev lo  scope link
192.168.10.0/24 dev switch0  scope link
192.168.20.0/24 dev switch0.7  scope link
192.168.30.0/24 dev switch0.8  scope link
192.168.40.0/24 dev vtun0  scope link

Working route after load-balance removal and reconfiguration (using the same configuration shown at the top of this post):

show ip route table 201
0.0.0.0/24 dev vtun1 scope link default via <my public ip address> dev eth0 blackhole default metric 256 10.8.8.0/24 dev vtun1 scope link <my public ip address>/22 dev eth0 scope link 127.0.0.0/8 dev lo scope link 192.168.10.0/24 dev switch0 scope link 192.168.20.0/24 dev switch0.7 scope link 192.168.30.0/24 dev switch0.8 scope link 192.168.40.0/24 dev vtun0 scope link

Is there any hope this will be fixed in the next release as well?

 

Ubiquiti Employee
Posts: 996
Registered: ‎07-20-2015
Kudos: 899
Solutions: 71

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

[ Edited ]

@david_woodward
> Perhaps the latest firmware has been updated to be less tollerant of odd configurations like mine that have logic defying elements used to work around Edge OS limitations,
> but the quick fix doesn't seem to work for my load balancing issue
I would tike to say that you are using LoadBalancing in very unusual way, but even this weird configfuration works fine on my test enviroment and I can see default gateway in 201 table after boot.

  1. Please show output "show load-balance status" and "show load-balance watchdog" when default gateway is missing from table 201
  2. Does running "sudo kill -kill `pidof ubnt-util`" help?
  3. Where is WAN_VPN is attached to? Which interface?
  4. Did your LB configuration worked fine with previous f/w version? Which one?
New Member
Posts: 8
Registered: ‎02-28-2016
Solutions: 1

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

[ Edited ]

 

Thanks for the quick reply.  I'll play with it some more this weekend and get back with you.  But, here are the answers I can provide from the top of my head.

 

This config is currently working on an ERPoE-5 running 1.9.7+hotfix.4.  I'm attempting to run more or less the same config on a new ER-X running 1.10.1 (which will eventually replace the ERPoE-5).

 

The WAN_VPN lb group has "eth0" under it as shown in my previous post, but the WAN_VPN group is not referenced elsewhere in the config at all (ie. not associated with any "firewall modify" rules).  It appears that just having the LB group defined is enough to mark incoming packets coming over eth0 as shown in the "iptables -t mangle -vnL" output below:

 

.
.
.
Chain FORWARD (policy ACCEPT 314K packets, 129M bytes)
pkts bytes target prot opt in out source destination
4922K 5265M UBNT_WLBI_WAN_VPN all -- * * 0.0.0.0/0 0.0.0.0/0
.
.
.
Chain UBNT_WLBI_WAN_VPN (1 references)
pkts bytes target prot opt in out source destination
314K 129M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore mask 0x7f800000
337 16560 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 mark match ! 0x0/0x7f800000
1 60 MARK all -- eth0 * 0.0.0.0/0 0.0.0.0/0 state NEW MARK xset 0x64800000/0x7f800000
1 60 CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match ! 0x0/0x7f800000 CONNMARK save mask 0x7f800000
314K 129M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
.
.
.

  

The ER-X is offline right now and I'll play with it again this weekend, but if I recall correctly, the lb status command showed that the link was "up", but "inactive".  I tried adding a watchdog ping to it to see if it would become active and I think I remember seeing something in the output that indicated the watchdog service was unable to ping my destination (not surprised with a blackhole default), and it remained inactive.

 

Thanks again for the quick response.

Emerging Member
Posts: 64
Registered: ‎05-14-2016
Kudos: 101
Solutions: 2

Re: EdgeMAX EdgeRouter software version v1.10.1 has been released!

The patch for load-balance isn't working for me. I can fail over but it doesn't want to fail back.

 

If I do this:

 

sudo kill -kill `pidof ubnt-util`

My load-balance transition script fires, logs that primary is going active, and 'show ip route' shows the primary as selected, but nothing actually works going out the primary and it falls back to the secondary 30-seconds later.

 

I'm running with lb-local and lb-local-metric-change enabled. Have zero issues transitioning back on 1.10.

Reply