pupmode=67 - an indirect save mode, a bit like PUPMODE=13, but overlayfs compatible.

[gyrog]

Please try it on a frugal install of a recent Puppy, on a Linux partition.

Required:
FrugalPup v20 or later, although best with FrugalPup v35.
Booting with "grub2" to accomodate a second "initrd" file.

Download files from https://www.mediafire.com/folder/j2u10wl713fky/pm66.

1. Download 'local-initrd.gz':
For Puppies released this year (2021) download 'local-initrd-woof.gz'.
For older Puppies download 'local-initrd-old.gz'. (This one works for me on Puppies as old as Xenialpup.)
Rename the downloaded file to 'local-initrd.gz' and place it in the install directory of the test Puppy.

2. Download 'local-pm67.sfs' and 'BOOT_SPECS' and place in the install directory.

3. Rename 'local-pm67.sfs' to be an 'ldrv_...sfs':
If your "zdrv" is 'zdrv_upuphh+d_21.04.sfs' rename 'local-pm67.sfs' to 'ldrv_upuphh+d_21.04.sfs'.
If your install directory is on a Linux partition you can use a symbolic link instead,

Code: Select all

ln -sf local-pm67.sfs ldrv_upuphh+d_21.04.sfs

4. Modify 'grub.cfg' to boot the test Puppy using 'local-initrd.gz':
If you have FrugalPup v35, you can generate a replacement boot-entry for the test Puppy with either "FrugalPup->Boot" or 'bootentry'.
Other wise you need to change the "initrd" line from:

Code: Select all

initrd /puppy/upuphh+d/initrd.gz

to

Code: Select all

initrd /puppy/upuphh+d/initrd.gz /puppy/upuphh+d/local-initrd.gz

5. Reboot.
The new 'init' script in 'local-initrd.gz' plus 'BOOT_SPECS' will enable the 'ldrv_...sfs' to be loaded.
Then run the CLI utility 'setup-pm67' ("setup-pm67 -h" for help), and reboot.
Select "SAVE".

6. '/etc/rc.d/PUPSTATE' should indicate that "PUPMODE=67".

Return to PUPMODE=12 using an existing "${DISTRO_FILE_PREFIX}save", by renaming the archive file so the 'init' script won't find it.

To return to PUPMODE=12 and keep the changes made in PUPMODE=67, run the CLI script 'savefolder67to12' ("savefolder67to12 -h" for help).
If the save directory is '/pups/upuphh+d' then

Code: Select all

savefolder67to12 /pups/upuphh+d

will produce '/pups/upuphh+d/upuphh+dsave.new'.
Rename '/pups/upuphh+d/upuphh+dsave.new' to '/pups/upuphh+d/upuphh+dsave', rename '/pups/upuphh+d/upuphh+dPM667save.tar.gz' to '/pups/upuphh+d/Xupuphh+dPM667save.tar.gz' and reboot.

How does it work?
It's a bit like PUPMODE=13, the RW layer is a tmpfs in RAM, and the top RO layer is a savefolder.
But unlike PUPMODE=13 this, savefolder called "${DISTRO_FILE_PREFIX}PM67save", remains truly RO.

Like PUPMODE=66, on shutdown the user is prompted to "SAVE" or not, if "SAVE" is selected the contents of the RW layer are saved into a "${DISTRO_FILE_PREFIX}PM667save.tar.gz" archive file,
then on next boot the contents of the archive are extracted into a tmpfs in RAM, before it is added to the stack.
Also before it is added to the stack, the savefolder is populated by moving all ordinary files from the tmpfs into it,
and then a new archive is produced because the RW layer in RAM is a lot smaller without all the ordinary files.

Why do it?
Because, while it is part of the stack, the savefolder is never modified, only read, even on shutdown.
This makes it compatible with using an overlayfs stack, whose RO layers are not allowed to be changed.
But it still ends up containing the bulk of the files that end up in the save layer, e.g. files from installed ".pet" files, etc...,
so the archive remains small, even after installing many ".pet" files, unlike PUPMODE=66.

The statement "moving all ordinary files" is significant;
a) the "moving" is what makes the archive smaller,
b) "all ordinary files" means that the savefolder contains only "ordinary" files, "whiteout" files and symbolic links remain in the RW layer in RAM, so theoretically in an overlayfs environment the savefolder could exist on non-Linux partitions.

RW="Read-Write"
RO="Read-Only"

Note: This is mainly a "proof of concept" at the moment.
Obviously 'setup-pm67' should be integrated into "Boot Manager" for a production environment.

[gyrog]

I've uploaded new versions of the files, see previous post for file details.

Changes:

1. Bugfix in 'init' script so it handles deleted directories correctly.

2. A little bit of "insurance" in 'rc.shutdown':
The partition containing the savefolder is remounted as "read-only" after any save archive is written.

[dancytron]

Can you ever merge the whiteout files with the save folder and actually delete things or do deleted things stay in the save folder forever?

Or do I not understand what is going on at all and I should go back to sleep?

[gyrog]

Can you ever merge the whiteout files with the save folder and actually delete things or do deleted things stay in the save folder forever?

Or do I not understand what is going on at all and I should go back to sleep?

The "merge" in 'init' is a bit more sophisticated than I probably indicated,
during the copy, it checks all "whiteout" files,
if the corresponding real file exists in any ".sfs" file, then it is required, and ignored,
if the corresponding real file exists only in the savefolder, then both the "whiteout" file and the real file are deleted,
otherwise it's an "orphan" "whiteout" file that "deletes" nothing, so it is deleted.

So if you create a new file, it will be copied into the savefolder during next boot.
If you then delete that file, it will be deleted from the savefolder during next boot.

I hope that clarifies things.

[gyrog]

More background to the pupmode=67 "merge" during 'init':
at the point in 'init' where this is done,
the stack mounted as '/pup_new' contains all the ".sfs" files that will be loaded via 'init',
but the RW directory in RAM and the savefolder itself, are just directories, not yet connected to the stack,
so they can be manipulated as much as we like, without caring what software is being used to build the stack.

[dancytron]

thanks, that makes sense.