Reply
Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@TrayDay

Thanks for bringing this to my attention, I hadn't noticed the certificate had failed to renew.

 

Weird, do you have access to a console cable? I'd need logs to find out what's wrong.

New Member
Posts: 4
Registered: ‎12-04-2017

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

[ Edited ]

@Lochnair

Unfortuantely, I do not have a console cable. I had just received my ERL two days ago and wanted to try this out. Glad to know I wasn't missing something on my end. Somewhat of a relief. But I do apologize for not having a console cable.

Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@TrayDay

In that case, have you upgraded the firmware after receiving it? The kernel tree I'm using is from 1.9.7, so if you're still on an old one, you'll likely to have issues.

New Member
Posts: 4
Registered: ‎12-04-2017

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

[ Edited ]

@Lochnair

 

Thanks Lochnair, That's exactly what the problem was. SFE and Cake installed when reboot, the ERL did not HOSE on me. Yes, It was the firmware. I had downloaded the beta firmware 10.alpha since I had read that UDP dropped packed was pretty much gone on it. So I wanted to try it out with cake and sfe. 

New Member
Posts: 5
Registered: ‎11-30-2016

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

[ Edited ]

Could I help test this out for the ER-X? I have a few of them lying around.

 

 

I think I understand that I'll need to putty into it with the adapter and load the E50-SFE uimage via tftp from my server.

 

Do I then download/extract the sfe module, update dependencies, reboot, and then load SFE?

 

If there are any special ER-X related commands different from the ones in the first post or other ones you had to track down, can you please share them?

 

Any help is appreciated greatly @Lochnair

New Member
Posts: 17
Registered: ‎01-09-2017
Kudos: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@boushidosan @Lochnair

 

I'm also interested in this.

 

I don't have a serial adapter for my ER-X.

Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@boushidosan

That looks right. When it comes to U-Boot commands I can't really remember. Aside from what you can find in this thread, @Arie is probably the best source of information, seeing as he was the one to first figure out how to TFTP-boot on it.

Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@pwndave

If you don't have a serial adapter, you'll have to dd the kernel image to the right MTD partition manually. It's not hard, but if it goes wrong you're in for a world of pain, especially considering you don't have a serial adapter.

Emerging Member
Posts: 42
Registered: ‎03-10-2015
Kudos: 13

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

Here are the very rough instructions on how to install on an ER-X. You'll need:

- USB serial cable (I use a raspberry pi TTL cable)

- A TFTP server (I use OS/X's built-in)

 

1. Make sure your ER-X is on 1.9.7 hotfix 4

2. Power off, hook up serial cable ( https://wiki.openwrt.org/_media/media/ubiquiti/edgerouter-x-sfp2.jpg?cache=&w=900&h=675&tok=d5f495 ). Power on and make sure it works.(57600 8N1). Let it boot normally.

3. Place the uImage ( https://dl.lochnair.net/UBNT/E50-SFE/uImage ) in the root folder of your TFTP server

3. SCP the sfe-modules.tar.bz2 ( https://dl.lochnair.net/UBNT/E50-SFE/sfe-modules.tar.bz2 ) over to the home directory of your admin user.

4. Shell in and extract (as root): tar -xf sfe-modules.tar.bz2 -C /

5. Run depmod -a

6. Power off and on

7. In serial console, choose option 1 (Load system code to SDRAM via TFTP.)

8. Enter a valid IP for the router, the IP of the TFTP server and the name of the image we want to load, uImage.

9. Watch it boot, it it's succesful, you can power cycle and choose option 2 instead, which writes the uImage.

10. Shell in and run as root: modprobe fast-classifier

 

I've just tested SFE again on my ER-X but the results were no better compared to stock. Not sure why, results on ER-X+LEDE were much better.

New Member
Posts: 17
Registered: ‎01-09-2017
Kudos: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

[ Edited ]

@Arie @Lochnair

 

Thank you both for your input. I have ordered a usb ttl adapter kit in order to continue this further. I bricked my ER-X attempting something crazy.

 

My failed method that no one should ever try again:

 

1. Download 1.9.7+hf4 tar.

2. Extract to directory

3. Delete vmlinux.tmp file

4. Copy uImage file to this folder (I did this because i noticed that the vmlinux.tmp file and the uImage file had a similar structure and size)

5. Rename uImage to vmlinux.tmp

6. Calculate vmlinux.tmp's MD5

7. Replace the contents of vmlinux.tmp.md5 with the new MD5.

8. Tar all the original file names together with a new file name.

9. Do "add system image http://localfileserver/handcrafted-tar-file.tar"

10. Reboot.

11. Cry a salty river.

 

Note: I falsley assumed that the spare system image could be used to recover from a bad flash. Now I understand that the spare image is only switched to from within a runnable system.

New Member
Posts: 17
Registered: ‎01-09-2017
Kudos: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@Arie @Lochnair

 

I fully recovered my ERX using my USB TTL kit and have begun messing with this again.

 

Now, using @Arie's last post as guidance, I am now running my ERX on 1.9.7hf4 + SFE (uImage + sfe-modules + modprobe fast-classifier) + CAKE. M y cake bandwidth parameters have been bumped to 1gbit in both directions.

 

The rest of my network is connected to the "WAN" side of my ERX configuration and my main computer is hooked up to the LAN side. I performed an SMB transfer (with ports forwarded) and was only getting about 180-190Mbit ingress  performance. Halfway through the transfer test, I performed a "modprobe -r fast-classifier" (to disable SFE) and my transfer speed jumped up to 205-210Mbit ingress. Both sides of the transfer were using SSD's as the source and destination.

 

Any input on these disappointing results?

Emerging Member
Posts: 42
Registered: ‎03-10-2015
Kudos: 13

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

I had similar results. Since you have the USB TTL cable, you could go all the way and try running LEDE with SFE patches.

New Member
Posts: 17
Registered: ‎01-09-2017
Kudos: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@Arie In the future, I may consider it, if the speed of my internet connection requires it. However, at present, the current processing power is sufficient.

 

I was hoping that maybe SFE would provide a drop in CPU usage at my usage level. Not that that would have made a difference. I didn't need the extra CPU power for anything.

 

It's good to know that this hardware really provides a lot of options.

New Member
Posts: 4
Registered: ‎06-04-2017

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

Is it possible to install LEDE and SFE patches with the USB TTL cable?  If so, are there simple instructions for a beginner somewhere?

 

If not, are there simple instructions somewhere for the install of both using the USB TTL cable?  Thank you for your help. 

Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@Coltslaughter

The installation instructions on the OpenWrt Wiki should still be correct. To get SFE you need to apply dissent1's PR to the source tree before building the image.

 

The only difference to the instructions on the blog they're linking to, is applying the patch after cloning:

curl -L https://github.com/lede-project/source/pull/1269.patch | git am

Make sure that you enable the SFE options in menuconfig before building.

New Member
Posts: 4
Registered: ‎06-04-2017

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

Thanks so much!

 

Are there any benchmarks on the difference of Cake+SFE vs LEDE+SFE vs the native EdgeOS QOS on the ER-X?  The problem I am trying to solve is that with QOS enabled on my ER-X, my throughput drops from about 950/950 to 250/250, but without it I get bufferbloat.  I am trying to decide on what my best route is to have a QOS that works well and not completely waste the gigabit internet I am paying for.  I also dont want to overcomplicate things, if at all possible.  Do you, or anyone, have a recommendation on how I should proceed?    

Emerging Member
Posts: 102
Registered: ‎07-25-2016
Kudos: 131

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@Coltslaughter I don't know what benchmarks have been done with SFE, but I'm almost certain that with the ER-X hardware you're not going to be able to rate limit to 950 Mbps and control the queue no matter what. You're lucky if you're getting 250 Mbps symmetric.

 

If you're considering new hardware, one suggestion at a reasonable price is a PCEngines APU2 (https://www.pcengines.ch/apu2.htm). I did some AQM testing with these: http://www.drhleny.cz/bufferbloat/cake/round2/. Potentially, an APU2 could be used as a middlebox for AQM and the ER-X could do routing and firewalling with its nicer UI, but I have not set one up this way.

 

In my tests the APU2s are doing both traffic generation and AQM, which they wouldn't be doing when used as a router or middlebox (only AQM). They're able to generate Gbit traffic and do 950Mbit Cake on egress at the same time, but not Cake on egress and ingress at 950Mbit + generate traffic at the same time. They may be able to do 950/950Mbit if they're not also generating traffic (might be able to test that if you want), but I'm not sure you even need Cake on ingress in your case anyway.

 

If you do go for one of these, avoid the apu1 since they have Realtek NICs, and note that only the apu2c4 has the Intel I210AT NIC. The apu2c0 and apu2c2 come with an Intel i211AT, which is also good and better than the Realteks, but has 2 transmit and receive queues instead of 4, and fewer available hardware offloads, which may or may not matter to you. I haven't tested the difference, and in my case for a $26 difference I didn't want to find out.

 

Sorry that most of this post wasn't directly related to the EdgeRouter. I also have an ER-X and love its functionality and price, just its CPU has limits to be aware of when it comes to AQM at Gbit speeds...

 

Emerging Member
Posts: 66
Registered: ‎04-15-2017
Kudos: 14

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@LochnairI noticed the new SFE Kernel and modules for the 1.10.0 firmware, have you tested the one for the ERL? Are there any differences we should be aware of before trying it out? I noticed there are 3 files this time, the third one looks like it maybe what you used  to build the Kernel, correct me if I'm wrong.

 

 

Thanks for your work on SFE and CAKE they have been fabioulous on the ERL.

Member
Posts: 230
Registered: ‎11-01-2015
Kudos: 97
Solutions: 5

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

@JoeyJoe

I have not. Honestly even I don't know. I applied the SFE patches on the new branch, and that's all I've done so far. The builds are run automatically by a GitHub hook. I assume it's the ksrc archives you mean? They're used for building out-of-tree modules, like CAKE and WireGuard.

 

I'm not at liberty to test on my ERL right now, due to it functioning as my home router at the moment. I might be able to swap it out with the ER-X over the next few days.

Emerging Member
Posts: 66
Registered: ‎04-15-2017
Kudos: 14

Re: Qualcomm Fast Path (SFE) for the EdgeRouters

[ Edited ]

@Lochnair

 

I'll give it a try in the next few weeks when I have some time, I have a spare USB drive I can use to try it out and if things don't work well I can swap back to a known working USB dirve.

 

Yes it was the ksrc I was talking about.

 

 

 

 

Reply