Reply
Highlighted
Veteran Member
Posts: 4,026
Registered: ‎05-15-2014
Kudos: 1470
Solutions: 274

EdgeOS file system layout and firmware images

[ Edited ]

The various ER router models have various physical flash memory components on top of which is a standardized file system layout across all models. This article is focusing on the latter layout only.

 

Each ER system has two firmware images that you can switch between. There's the 'current default boot image' and 2nd non-running image often called 'other' or 'secondary' or 'backup'  or 'previous' image.

 

When the system boots, it boots into the current default boot image. During the firmware upgrade process, following high-level steps are executed (not necessary in this order).

  • The secondary image is deleted
  • The new firmware is un-packed and installed as secondary image
  • Entire content of /config/ folder is copied over to secondary image
  • The secondary image is marked as default boot image
  • and the system is rebooted.

After successful upgrade, you've booted to your new default firmware image and your old firmware image is now secondary image in exact state as it was before upgrade. If you find out the new firmware has issues or you need to quickly revert to previous firmware for whatever reason, all you have to do is set your secondary image as default (remember this was the image before upgrade) and reboot.

There are following commands to manipulate images:

show system image will show you information about installed firmware images

set system image default-boot will make the secondary image the default boot image (and your current running image will become secondary image) ...reboot is needed to activate the image

delete system image will delete the secondary image

add system image <url or file path> will install new image. You can provide URL directly to UBNT web page and the system will download it prior installing it. 

 

The system partition where images and data are stored (regardless of the ER model) is mounted in /root.dev

It' content is as follows

 

# ls -al /root.dev/
total 155472
drwxr-xr-x    5 root     root           784 May  2 23:38 .
drwxr-xr-x   33 root     root           888 May  8 21:31 ..
-rw-r--r--    1 root     root      79593472 Apr 26 14:42 squashfs.img
-rw-r--r--    1 root     root            33 Apr 26 14:42 squashfs.img.md5
-rw-r--r--    1 root     root      79593472 Apr 17 11:24 squashfs.o
-rw-r--r--    1 root     root            33 Apr 17 11:24 squashfs.o.md5
-rw-r--r--    1 root     root            42 Apr 26 14:29 version
-rw-r--r--    1 root     root            42 Apr 17 11:07 version.o
drwxr-xr-x   12 root     root           888 May  8 21:31 w
drwxr-xr-x   11 root     root           824 Apr 20 15:09 w.o
drwxr-xr-x    2 www-data root           224 Dec 31  2014 www

 

/root.dev/squashfs.img is your current firmware in SquashFS format. During the boot this image mounted into (root) mount point. This is however read-only portion of the firmware (imagine it as bottom layer or background layer). Then on top of the read-only firmware content of /root.dev/w  is 'overlayed' ..this is the read/write component (imagine this as top transparent layer). Within this writable layer you'll see the /config folder where all the config is stored.

During boot process, content of /config/config.boot is interpreted by EdgeOS that in turn creates various configs for all the linux backend components (typically the /etc/ config files i.e. dnsmasq, strongswan, zebra and many others). 

 

The /root.dev/squashfs.o and  /root.dev/w.o is the 'other' or secondary image read-only and read-write content respectively. When you change default boot images the .o are swapped for non .o files.

 

So in a running image your config is in /config/config.boot file which is the same file as /root.dev/w/config/config.boot file.

In a similar fashion, you can access your secondary image config in /root.dev/w.o/config/ (no need to reboot to other image if you just want to check your previous config or get some files from secondary image, all is in /root.dev/w.o/ )

 

Lastly about special files and folders.

As I mentioned, all configuration in /etc and other folders is generated by EdgeOS during boot or firmware image installation. As such if you modify any of the files manually there are good chances your changes won't survive next reboot or firmware upgrade.

 

/config/ path is special. During firmware upgrade entire content of /config/ is copied over to secondary image thus everything under config will "survive" firmware upgrade and reboot.

 

Under /config there are few special files and folders, I'm going only mention two important:

/config/config.boot this is your main config generated via GUI or CLI commands

/config/scripts/post-config.d/ any executable file in this folder will be executed once after system boots up (after config.boot is executed) ..these are your custom boot scripts

/config/scripts/pre-config.d/ scripts to be executed on boot prior config.boot is executed

/config/scripts/firstboot.d/ contains script run only on first boot after firmware upgrade

/config/data/firstboot/install-packages/ optional for .deb packages that would be automatically installed on first boot after firmware upgrade

 

Some other important folders are:

/etc/ where most of the config files for system components resides

/var/run/ stores run-time information such as system pids and dhcp leases files ...and much more

/var/log/ is where logs are kept (during running session, see note below on tmpfs)

/etc/ubnt/firstboot created after during first boot after firmware upgrade ...if non existent this is first boot after upgrade

/opt/vyatta/etc/config.boot.default default factory config (if you need to reset to defaults copy this file over your config)

 

When you do df note which file systems are mounted on tmpfs. These are in-memory file systems that definitely won't survive reboot or power outage. Note /tmp and /var/log is in memory. This is to protect flash memory from degradation on constant writing.

 

Lastly, some ER models have limited storage and here are folders to check (and clean) when you get error: file system full or no more space lef on device.

/var/cache/apt/

/var/lib/apt/ ...if you have apt sources configured these folders tend to grow and eventually you'll run out of space

/config/url-filtering/ ...if you have huge content filter rules configured, keep eyes on this folder size too

 

 

 

 

 

Veteran Member
Posts: 7,428
Registered: ‎03-24-2016
Kudos: 1920
Solutions: 849

Re: EdgeOS file system layout and firmware images

Great post !

 

I'd like to emphasize /config folder is only copied on firmware upgrade. If you boot into new firmware, make config changes, and revert back to old firmware......changes are gone and you end up with old config.

 

/root.dev/squashfs.img i...... During the boot this image is uncompressed

I doubt entire entire image image is decompressed, this would chew up almost all ER-X RAM.

Are  requested blocks/files  decompressed while accessed (=on the fly) ?

 

 

Veteran Member
Posts: 4,026
Registered: ‎05-15-2014
Kudos: 1470
Solutions: 274

Re: EdgeOS file system layout and firmware images

[ Edited ]

Good catch, thank you, the image is just mounted (that's the nature of squashfs you can mount it as is) ...I've updated the article.

Reply