- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Sticky This Topic
- Bookmark
- Subscribe
- Printer Friendly Page
EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-15-2013
04:39 AM
- last edited on
07-21-2015
05:17 PM
by
UBNT-stig
EDIT by UBNT-stig: NOTE this rescue kit will only works on ER-Lite and ER-PoE.
This forum topic is the home of the EdgeMax Rescue Kit—a network bootable rescue image for EdgeOS-based routers.
Basically, it's a minimal Linux kernel with recovery tools embedded in initramfs. You connect to your router via serial, boot it via TFTP and get into a minimal rescue system.
The following scripts for automated recovery procedures are included:
- emrk-factory-reset — removes the config.boot, optionally making a backup copy inside the config dir so you can access it easily after reboot.
- emrk-remove-user-data — removes any data from writable dirs, getting the system back to clean EdgeOS (all config data, user installed packages on so on are removed). It also removes copies of user data left after resets, as pressing reset button doesn't really remove the data but renames the old writable dir to some random name and creates a new one. Useful if you are going to give away your ERL to an untrusted third party.
- emrk-reinstall — downloads release tarball from URL and reinstalls EdgeOS from scratch .
NOTE: emrk-remove-user-data and emrk-reinstall are highly disruprive! The former will remove everything but the stock EdgeOS image. The latter will remove and re-create all the partitions, so all your data, if you still have any, will be lost irrecoverably. Use it only if there is no hope to recover your installation in a less disruptive way.
I did my best to get it to work, but I can't guarantee there are no bugs, so use at your own risk.
How to use
Download the latest version of the EMRK image from http://packages.vyos.net/tools/emrk/ (or use mirror director link: http://mirror.vyos.net/tools/emrk/)
Install a TFTP server. On UNIX systems some TFTP implementation is always available from ports/repos, Windows users may use this one: http://tftpd32.jounin.net/
Copy the image to your TFTP server directory.
It's better to set up a DHCP server. If you are using ISC DHCPD, use these options in your subnet declaration:
option tftp-server-name "x.x.x.x"; # Your TFTP server address next-server x.x.x.x; # Your TFTP server address option bootfile-name "emrk-0.9c.bin";
filename "emrk-0.9c.bin
Connect to your router via serial. Power on or reboot the router and, before EdgeOS kernel loads, press any key (I usually keep pressing some key repeatedly from the start). This will get you to the bootloader console where you can boot your router manually.
Connect router eth0 interface to your network.
If you use a DHCP server, issue the following command in U-Boot CLI:
dhcp;tftpboot;bootoctlinux $loadaddr
If you don't have a DHCP server, you may configure network manually:
set ipaddr x.x.x.x (where x.x.x.x is desired router IP address) set netmask 255.255.255.0 (or whatever mask you want) set serverip y.y.y.y (where y.y.y.y is your TFTP server address) set bootfile <EMRK image file name> Optionally, set gatewayip z.z.z.z (where z.z.z.z is your default gateway)
tftpboot
bootoctlinux $loadaddr
If everything goes right, after bootoctlinux command you will see the kernel booting and will be asked if you want to continue after a scary disclaimer. Answer "yes" (case insensitive) or "no".
Next you will be asked if you want to use DHCP or configure the network interface statically.
To reinstall EdgeOS from scratch, you will need to download the tarball and put it on your HTTP/FTP/SSH server.
Here is an example reinstall log (kernel boot messages omitted).
Loading EMRK 0.9a Mounting filesystems Bringing up eth0 Checking boot partition Boot partition looks intact Attempting to mount boot partition Boot partition successfully mounted Looking for kernel file Found a kernel Checking kernel MD5 sum file Found kernel MD5 sum file Checking kernel MD5 sum Kernel MD5 sum is correct Checking root partition Root partition looks intact Attempting to mount root partition kjournald starting. Commit interval 5 seconds EXT3 FS on sda2, internal journal EXT3-fs: mounted filesystem with writeback data mode. Root partition successfully mounted Looking for system image file Found a system image file Checking system image MD5 sum file Found system image MD5 sum file Checking system image MD5 sum System image MD5 sum is correct ********************************************** Welcome to EdgeMax Rescue Kit! This tool is distributed under the terms of GNU General Public License and other licenses Brought to you by SO3 Group WARNING: This tool is not developed, officially supported or endorsed by Ubiquiti Networks! Using it may lead to destroying your router configuration or operating system Ubiquiti Networks support will not help you with using it or fixing consequences of using it. This tool itself is distributed without any warranty and authors are not liable for any damage it may cause By using this tool you agree you are doing it at your own risk and understand what you are doing ********************************************* Enter 'Yes' to proceed, 'No' to reboot yes or no: yes Do you want to configure network via DHCP? yes or no: yes udhcpc (v1.17.1) started Sending discover... Sending select for 10.91.19.175... Lease of 10.91.19.175 obtained, lease time 86400 /usr/share/udhcpc/default.script: Resetting default routes route: SIOCDELRT: No such process /usr/share/udhcpc/default.script: Adding DNS 10.91.19.1 EMRK provides some scripts for automated recovery procedures: emrk-factory-reset -- reset config to factory default emrk-remove-user-data -- remove all the user data including config and everything emrk-reinstall -- reinstall EdgeOS from scratch (wipes any user data too) Enter 'reboot' to reboot your router BusyBox v1.17.1 (Debian 1:1.17.1-8) built-in shell (ash) Enter 'help' for a list of built-in commands. /bin/ash: can't access tty; job control turned off EMRK> EMRK>emrk-reinstall WARNING: This script will reinstall EdgeOS from scratch If you have any usable data on your router storage, it will be irrecoverably destroyed! Do you want to continue? yes or no: yes Unmounting boot partition Unmounting root partition Re-creating partition table Creating boot partition Formatting boot partition mkfs.vfat 3.0.9 (31 Jan 2010) Creating root partition Formatting root partition Mounting boot parition Mounting root partition kjournald starting. Commit interval 5 seconds EXT3 FS on sda2, internal journal EXT3-fs: mounted filesystem with writeback data mode. Enter EdgeOS image url: ftp://10.91.19.5/pub/ER-e100.v1.2.0.4574253.tar % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 68.3M 100 68.3M 0 0 8319k 0 0:00:08 0:00:08 --:--:-- 6318k Unpacking EdgeOS release image Verifying EdgeOS kernel Copying EdgeOS kernel to boot partition Verifying EdgeOS system image Copying EdgeOS system image to root partition Creating EdgeOS writable data directory Cleaning up Installation finished Please reboot your router
If you want to do things manually, there are busybox with symlinks to its applets (cat/grep/rm/cp/mv/tar/etc., ping, traceroute, nc etc.), curl, parted, and mkfs.ext3/mkfs.vfat.
I used it on my on ERL when I replaced the USB stick with a 8GB one yesterday. Hope you find it useful, any bug reports or suggestions are welcome.
Here are the scripts it uses: https://github.com/vyos/emrk
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-15-2013 09:14 AM
Thanks for the effort and for including the warning!
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-15-2013 02:37 PM - edited 07-15-2013 06:28 PM
This is pretty cool. One of my test ERL's has been bricked for a while I hadn't gotten around to running our factory install script, so I gave your build a try instead. It worked and rebooted fine.
One thing you may want to add is the version file since the normal upgrade script looks for that. Without that the next upgrade failed for me:
ubnt@ubnt:~$ add system image upgrade-v1.2.0dev.4578432.130715.0102.tar Checking upgrade image... Done Preparing to upgrade... Done Copying upgrade image... /usr/bin/ubnt-upgrade: line 449: [: ==: unary operator expected Done Removing old image... Done Checking upgrade image... Done cp: can't stat '/root.dev/version': No such file or directory Finishing upgrade... Failed to move current image ubnt@ubnt:~$
After the failed upgrade, I was left with an upbootable system. So I used the rescue imagine again to recover and then did a "sudo touch /root.dev/version" so that the next upgrade would work. I suppose we should be the upgrade script more robost so that it doesn't leave you with an unbootable system if the upgrade goes sideways.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-15-2013 06:18 PM - edited 07-15-2013 10:46 PM
I might have to try this. My 8GB upgrade is not going so well. After the first dd the router booted, then after a reboot the router refused to boot from the drive ever again. Even after zeroing it out and starting dd over...
[EDIT]
This script didnt work for my 8GB drive. I'm starting to think either something is corrupted on it or the router just doesnt like the drive. I'm going to RMA the drive.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 07:40 AM
Yeah, good point, I overlooked it. Just copy the version file to the root partition?
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 07:43 AM
linuxwarz: Hhm, interesting. What messages do you get from U-Boot when you connect via serial after running the reinstall script?
Could you also boot to the rescue image once more and post "ls /mnt/root" and "ls /mnt/boot"?
Some say there are incomptible USB sticks and it may be the case, but I want to make sure if it was not bug in the script.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 10:43 AM
All of your answers are in the attached file. I wonder why I have DHCP issues in the bootloader too... your script works fine. I use the DHCP in the tftpd32 application.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 01:20 PM
Interesting. To me it looks like the ERL doesn't like the flash drive, but the fact parted and mkfs didn't throw any errors confuses me. You may look at the bin/emrk-reinstall script from the github link above and reproduce the steps manually, maybe it will give some useful information.
Also I updated the image, fixed the bug with missing version file. http://baturin.org/files/edgeos/emrk-0.9b.bin
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 01:51 PM
Thanks for this it looks like It's unbricked my ERL
I also ran the command to fix the missing file, so hopefully an update won't re-brick it.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-16-2013 11:51 PM
With 0.9b the version file should get copied correctly already, so in the future it should not be a trouble. Would be nice if someone tried to install 1.1.0 and upgrade to 1.2.0 for verification. (:
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-17-2013 01:27 AM
notbad.jpg
Will have to remember this for if/when I brick an EdgeMAX (so far I've been lucky! )
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-17-2013 06:05 PM
This makes me want to brick one of mine to test!!! I have been lucky with 80+ in the network and none failed yet minus the one that found lightning and let its smoke out
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
07-23-2013 08:27 PM - edited 07-23-2013 08:30 PM
@Magician wrote:This makes me want to brick one of mine to test!!! I have been lucky with 80+ in the network and none failed yet minus the one that found lightning and let its smoke out
I've got a fix for that. https://www.sparkfun.com/products/retired/10622
http://youtu.be/Ex8oY24zycQ?t=8m42s
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
10-01-2013 01:10 PM
Thanks for the great tutorial!
Just wanted to say that I had no luck with TFTPD on windows 8 x64, it would start the connection but then never transfer any data and time out.
But Open TFTP server worked first try to serve the file: http://sourceforge.net/projects/tftp-server/
Because I didn't want to setup my own FTP/HTTP server for the final upload of tarball I just used the direct link to the current beta image: http://dl.ubnt.com/firmwares/edgemax/v1.3.0/ER-e100.v1.3.0beta1.4599951.tar
Now it works again
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
10-01-2013 01:20 PM
Hey guys,
I've heard from Stig yesterday some folks couldn't download the image.
We were having multiple problems with hardware, L3 switch, and DoS attacks yesterday. Now we resolved that, EMRK images and other stuff we host should be available again.
Sorry for the inconvenience.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
10-01-2013 01:36 PM
I am running version "emrk-0.9a.bin" instead of the newest "emrk-0.9c.bin"
Are there any big changes that I should know about between them?
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
10-01-2013 01:43 PM
@hilo90mhz wrote:
I am running version "emrk-0.9a.bin" instead of the newest "emrk-0.9c.bin"
Are there any big changes that I should know about between them?
There was an issue with the version file missing which would cause a problem the NEXT time you upgrade. It should be ok if you do:
sudo touch /root.dev/version
Not sure what changed in version "c".
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
10-01-2013 02:00 PM
I should start keeping proper changelogs I guess.
Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
02-18-2014 08:57 AM
I have a ER-Lite that failed the firmware upgrade from 1.3 to 1.4. From the console port, it appears to boot but there are a ton on SQUASHFS errors that fly by about read failures. These errors even occur after defaulting the unit.
So I've tried running the rescue kit to reinstall the image but I'm running into an issue at the final boot step. The emrk-0.9c.bin transfers fine from my TFTP server. But when I type bootoctlinux $loadaddr, the boot process hangs at a point. The attached pic shows where it hangs.
Any thoughts?
Will

Re: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content
02-18-2014 11:56 AM
If it's stopping there, it could be a hardware issue, so please contact support@ubnt.com for help with verification and RMA if necessary.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Sticky This Topic
- Bookmark
- Subscribe
- Printer Friendly Page