HowTo use sfs and changes folder with a full Linux install

Moderator: Forum moderators

Post Reply
User avatar
wiak
Posts: 3627
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 994 times
Contact:

HowTo use sfs and changes folder with a full Linux install

Post by wiak »

REPOSTING this old early FirstRib-related info of early 2019 from the old forum since I find it useful background info more readily at hand. The whole of the original old forum thread remains also relevant. Find whole thread at: https://oldforum.puppylinux.com/viewtop ... 2#p1025462

Note that this HowTo is intended to take away some of the mystery of how Linux distros can use overlayfs mount option (or aufs can be used with minor tweak to what follows) to implement load-sfs and changes/persistence facilities. In particular I wish to show how such facility can be used with a full install despite some myths going around that such facility is only useful for frugal installs.

Moreover, this HowTo is purposively written to show the actual commands required - I strongly believe that first principles are more important to understanding than a polished implementation, which this isn't! Rather this HowTo just provides a simple example, which hopefully aids actual understanding. It is intended to show you how simple the 'layering' methodology really is (independent distro makers aren't actually performing magic - they just happen to have learned these and other tricks), though I certainly won't be explaining the individual steps (please google and use command man pages for that).

Having said all that, I'm not saying this is for beginners, and since most users on Puppy are using frugal installs and aufs rather than full installs and overlayfs, this howto will maybe only be of interest to a few in its current form. Easy as I say to modify for aufs and/or frugal install situations as long as you understand the principles of operation illustrated in this example (or make a test full install of Void Linux and try it... as shown!).

Scripters can of course easily then implement all of this with a user friendly gui frontend... I'm using overlayfs, but can be easily changed to use aufs instead (I will show the minor alterations for that later along with same for frugal installs, though both these alterations are trivial really. I used overlayfs because it has been officially adopted for Linux kernel and aufs wasn't available by default on the system I used). In practice, a distro maker would likely implement these commands in the boot scripts (be it inside some init script or inside initrd/initramfs followed by a switch_root) but I hope to here show that these facilities can be used simply and independently of such grand schemes, or in addition to them (not that appropriately modifying initramfs, for example, is actually very difficult anyway - I might cover that later but want to avoid any confusing extra complexity for now).

The following shows how to take a full install of a Linux distribution and by means of an overlayfs of three layers (lower, middle, upper) be able to do the following:

1. Temporarily load and use an sfs file without needing to extract it.
2. Create a writable changes folder without affecting the underlying root filesystem (so that any changes made can be rolled back afterwards).
3. Successfully achieve the above two items without having to modify any part of the boot system (such as initramfs - though for more permanent use, that could be done later) and with no need to reboot the system to use the loaded sfs (which can easily be unloaded afterwards etc...).

The following should work with any full installed Linux (and alternatively could be arranged to work with frugal installs though I won't go into that possibility at the moment). As it happens I was using a full install of 64bit Void Linux which I had installed to the /dev/sda7 ext4 formatted partition I have on my computer. The sfs file I used in my tests was from tahrpup64 repo: gimp&mypaint-x86_64.sfs; though I renamed it to replace the annoying & with an underscore instead.

The directory 'lower' has the underlying root filesystem of Void Linux mounted readonly (via -o bind,ro) to it.
The directory 'middle' has the gimp_mypaint sfs loop mounted to it (readonly).
The directory 'upper' is writable and ends up containing any changes made whilst running the resulting merged filesystem in a chroot.

To get everything going, I simply cut and pasted the following into a terminal on my full-installed Void Linux system though any similar full-installed Linux system would do (assuming it has overlayfs included by kernel etc):

Code: Select all

mkdir -p /overlaystuff
cd /overlaystuff
mkdir -p lower middle upper work merged
mount -o bind,ro / lower
mount /root/Downloads/gimp_mypaint-x86_64.sfs middle
mount -t overlay -o lowerdir=middle:lower,upperdir=upper,workdir=work none merged
mount --bind /proc merged/proc
mount --bind /sys merged/sys
mount --bind /dev merged/dev
mount --bind /tmp merged/tmp
mount -t devpts devpts merged/dev/pts
chroot merged

Note that prior to the command 'chroot merged', you may in some circumstances need to copy /etc/resolv.conf to merged/etc/resolv.conf but that is not necessary in my above setup since resolv.conf in overlayfs layer 'lower' is simply a readonly mirror of the original in underlying void root filesystem.

After the above, you can, for example, then start gimp (from the loaded sfs) by entering:

gimp &

End result of above is that gimp from the sfs is running successfully on the system. Once finished you can close gimp and enter 'exit' in the terminal to exit from the chroot and then clean up the mounts with:

Code: Select all

umount -l merged/proc && umount -l merged/sys && umount -l merged/dev/pts && umount -l merged/dev && umount -l merged/tmp && umount -l merged && umount -l lower && umount -l middle

followed, if you wish, by:
cd /
and remove the /overlaystuff directory to start afresh.

Note that it is also trivial to backup that 'upper' (changes folder) for re-use, or make the whole process a bit more automatic and user friendly with some scripting with yad and/or gtkdialog or whatever...

As I said, it is pretty much trivial to also open up initramfs and implement the above in there along with a switch_root so that the overlay system becomes the actual root filesystem in practice. However, I'm not wanting that additional complexity at the moment since I want this HowTo to show how simple using an overlayfs for these 'tricks' is, without actually needing any initramfs or init modifications
...
wiak

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

JusGellin
Posts: 178
Joined: Fri Jan 19, 2024 11:12 pm
Has thanked: 10 times
Been thanked: 23 times

Re: HowTo use sfs and changes folder with a full Linux install

Post by JusGellin »

Thanks for this instruction. I'm still very new to all this, but enjoy picking up bits of it.

I'm curious about the overlays.
In your example you show:

mount -t overlay -o lowerdir=middle:lower,upperdir=upper,workdir=work none merged

Does the mount -t overlay... mean the setup is for overlayfs?
The reason I ask is when I look at the bootinit.log for BookwormPup64_10.0.6, it shows the same for its mounts.
I thought that puppy linux was just using aufs. So is BookwormPup then using overlayfs?
Am I looking at this correctly for Bookworm? I don't know what the log would show for other pups. If they were using aufs would it show
mount -t aufs...?
Thanks

User avatar
wiak
Posts: 3627
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 994 times
Contact:

Re: HowTo use sfs and changes folder with a full Linux install

Post by wiak »

JusGellin wrote: Fri Apr 26, 2024 11:24 pm

Does the mount -t overlay... mean the setup is for overlayfs?
The reason I ask is when I look at the bootinit.log for BookwormPup64_10.0.6, it shows the same for its mounts.
I thought that puppy linux was just using aufs. So is BookwormPup then using overlayfs?

The first post is about loading añ sfs using overlay filesystem on top of a full installed Linux distro. The example was done using a full install of Void Linux, but same could be done using for example a full install of Linux Mint. It was not done with a Puppy Linux frugal install, but I did use an sfs that was originally created for use with Puppy. Traditional Pups indeed use aufs. Kennel Linux firstrib initrd bazed distros have always used overlay fs in preference to aufs because overlay fs is standard in mainstream kernel builds so doesnt need an aufs patched kernel.

I often myself use mainstream full installs, but use overlay fs kennel linux 'tricks' with them to get similar save on demand persistence flexibility as that of frugal installed KL distros. Overall, I dont restrict my setups to any one distro, but seek general purpose flexibility. I support KL distros mainly, but dont tie myself in any kind of fanbase scenario. Linux is Linux to me; no particular distro allegiance from me - just Linux-related technology more generally, but using this forum mainly for discussions/collaborations/contibuted systems and howtos.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

JusGellin
Posts: 178
Joined: Fri Jan 19, 2024 11:12 pm
Has thanked: 10 times
Been thanked: 23 times

Re: HowTo use sfs and changes folder with a full Linux install

Post by JusGellin »

Thanks for helping me understand better. I can see the title of this topic does say full Linux install
It looks like you have the knowledge to to use the tools (even puppy linux) to make Linux work like you want -- very impressive! Overlays are very powerful.

Just for my beginner knowledge - since I see the BookwormPup log indicating the overlay mounting shows mount -t overlay..., it does mean BookwormPup is using overlayfs instead of aufs - right?
Thanks

JusGellin
Posts: 178
Joined: Fri Jan 19, 2024 11:12 pm
Has thanked: 10 times
Been thanked: 23 times

Re: HowTo use sfs and changes folder with a full Linux install

Post by JusGellin »

I started up a different pup - FossaPup64CE and indeed found that the boot log indicates it is using aufs because it shows mount -t aufs.... It's probably not that important on my level, but it's just good to know. Doing this is helping me to try things to learn more.

I just checked the Bookwormpup64 area - It indicated it is using overlayfs instead of aufs

User avatar
rockedge
Site Admin
Posts: 5711
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1990 times
Been thanked: 2097 times
Contact:

Re: HowTo use sfs and changes folder with a full Linux install

Post by rockedge »

t does mean BookwormPup is using overlayfs instead of aufs - right?

Yes. Now on some of the most recent Puppy variants it is possible to indicate using aufs or overlay specified on the kernel command line in the boot stanza's.

Post Reply

Return to “How-To”