BookwormPup64 10.0.8

[MHHP]

Hello!

In order to compile the kernel, I had to download the "libelf-dev"-package.

How about including this package in the devx?

/MHHP

[MHHP]

And fortunately for me, I've only used Pupmode13 since I found out about it.

I wonder how many Puppy users use PUPMODE 12 or 66 and not 13, and I wonder how many Puppy users don't boot from a flash drive. IMO PUPMODE 13 is the best feature of Puppy

Any Puppy built before https://github.com/puppylinux-woof-CE/w ... 4a19038574 (March 2023) will default to PUPMODE 12 (!) when you boot from a flash drive without specifying pmedia=bootflash, which is not the default in the ISO. Bookworm Pup64 has the fix for this and will default to PUPMODE 13 if booting from a flash drive when pmedia=usbflash or unspecified (the default in the ISO), so I believe that very few users use it with PUPMODE 12 and actually notice the limited support for SFS loading with overlay.

I have always used pupmode 12, since my start with puppies (in 2007, with pup2.16), and I still do. Although I have moved from a 2fs-save-file to a save-file-directory...

Will you look in to the sfs_load pupmode 12 issue shortly?

Best regards/
MHHP

upmode

[dimkr]

Will you look in to the sfs_load pupmode 12 issue shortly?

This is not an issue, but more like an artificial limitation.

It is possible to simulate sfs_load under PUPMODE 12 using symlinks, but:
1. Loading and unloading will be slow because the symlinks are written to disk, not RAM
2. After improper shutdown (like power loss), there will be leftover symlinks, and they will be broken or point to the wrong place
3. Something will need to clean up invalid symlinks to protect against (2), making (1) much worse but also slowing down boot and/or shutdown

To avoid these issues and put users in risk of data loss or broken system after reboot, I decided not to support PUPMODE 12 in sfs_load.overlay.

However, I'm open to suggestions: if anyone has a good proposal for a way to simulate dynamic loading under PUPMODE 12 that doesn't suffer from these issues, let's do it.

[retiredt00]

Dear dimkr
The symlink SFS loading was utilized years ago for full installs and there was the recommendation "remount and properly un-mount the SFS and the symlinks will go".
So an idea is sfs_load.overlay to write a file (say /var/temp_sfs_loaded) with the full path(s) and name(s) of transiently loaded SFSs so if the file is still there after an improper or forgotten shutdown, rc.startup or a script in /root/Startup will load and unload the SFS9s) to clean up symlinks.
This could be done after finish booting without a problem, unless the SFS overwrites real files, which may be a bad idea to begin with.
Will not affect boot times in usual/clean boots and maybe provide a message for the delay if clean up is necessary

On a different but related point, SFSs for apt-based puppies would be useful το have a mechanism to communicate the new packages to apt/dpkg.
This implies that some instructions should be given on how to do it when building the SFS from debian packages, and include a mechanism in SFS_load.overlay το utilize this information.
It certainly looks too involved for the SFSs currently circulating in the forum but it can make a far more robust SFS use.
Such an approach would be useful for the "dog" family too and I guess even fatdog could translate this info into their package manger system.

[dimkr]

So an idea is sfs_load.overlay to write a file (say /var/temp_sfs_loaded) with the full path(s) and name(s) of transiently loaded SFSs so if the file is still there after an improper or forgotten shutdown, rc.startup or a script in /root/Startup will load and unload the SFS9s) to clean up symlinks.

This list can run out of sync with reality (for example, power failure after symlinks were created, but before the list was fully flushed to disk). The only reliable cleanup routine I can think of is slow search for dangling symlinks, in the entire save layer, but it's slow.

On a different but related point, SFSs for apt-based puppies would be useful το have a mechanism to communicate the new packages to apt/dpkg.

I'm strongly against this, because the idea of dynamically loaded modules is incompatible with the idea of a package manager that tracks dependencies. There's no clean way to make SFSs place nicely with apt, especially if you don't want two copies of packages that come from SFSs (so the system doesn't break once a SFS is unloaded).

If a SFS contains package x, apt is configured to treat x as already installed and the user installs package y that depends on x, then the SFS with x is unloaded, the dependency tree is broken (y is installed although x isn't). The user needs to remove y (to make apt happy), then install y again, this time let apt also install the x dependency first. Then, if the SFS is loaded again, you have two copies of x, one inside the SFS and one in the save layer, and you can't remove either if you want the freedom to unload that SFS.

[dancytron]

I'm strongly against this, because the idea of dynamically loaded modules is incompatible with the idea of a package manager that tracks dependencies. There's no clean way to make SFSs place nicely with apt, especially if you don't want two copies of packages that come from SFSs (so the system doesn't break once a SFS is unloaded).

Does the adrv sfs file or the other extra sfs files have dpkg information in them or just the main .sfs file and the changes folder?

FWIW, I totally agree with you. In Debian Dog, you are perfectly free to create sfs files with or without the dpkg info and load them in any order you want. If you do it wrong, then it breaks in spectacular ways.

[dimkr]

Does the adrv sfs file or the other extra sfs files have dpkg information in them or just the main .sfs file and the changes folder?

bdrv contains apt and stores apt's initial state. If adrv was built by woof-CE (by adding a package to DISTRO_PKGS_SPECS, then using ADRV_INC), apt should treat the packages in adrv as installed. Otherwise, it doesn't know about them.

[dancytron]

Does the adrv sfs file or the other extra sfs files have dpkg information in them or just the main .sfs file and the changes folder?

bdrv contains apt and stores apt's initial state. If adrv was built by woof-CE (by adding a package to DISTRO_PKGS_SPECS, then using ADRV_INC), apt should treat the packages in adrv as installed. Otherwise, it doesn't know about them.

Thanks.

People will need to be careful not to use the adrv with the apt data in it and then trying to boot without it using the same save folder/file. I'm pretty sure that would break it.

[retiredt00]

This list can run out of sync with reality (for example, power failure after symlinks were created, but before the list was fully flushed to disk).

I'm not sure why this can not be solved after re-loading and removing the SFS.
If you have a power failure and you reboot in a system with broken links assuming that the SFS does not clobber vital files you can reload the SFS, overwriting existing symlinks, since you know they are broken, remove the SFS and you are all cleaned up.
If some rare catastrophic scenario takes place that can not be solved by the above I can only argue that it can also happen in pupmode 13 too (if one saves periodically for example).
I actually think that any system can be messed up if a power failure happens during package installation/removal. So I would not worry that much about these rare cases.

I'm strongly against this

That makes sense. Apologies that I was not clear enough but I was referring primarily to SFSs that are loaded at boot and integrate in the overlayfs.
But even if are integrated during dynamic loading and apt is run at this point, if sfs_load.overlay "knows" that apt was used it can call apt/dpkg look for missing packages after the SFS is removed (though this might be too much)

[dimkr]

snapmergepuppy.overlay ignores the symlinks when saving under PUPMODE 13, so the problem of leftover symlinks is only relevant to 12 (if the restriction to 5 and 13 is removed). Therefore, by design, SFS loading under 13 is safe against power loss, unlike 12.

sfs_load.overlay doesn't replace existing files, so loading again to get rid of broken links is not a solution (and, maybe you no longer have that SFS that left the symlinks).

[rockedge]

In the KL series of distros using SFS-Load it is important that the xbps package manager does not track anything with loaded SFS's. In KL's case an SFS loaded on the fly is meant to be a temporary installation and therefore handled differently than package manager installed, more permanent packages. The temporary, for the current session, is exactly that.... a temporary component that because of it's installation method of being mounted and symlinked into the system, does not remain in the final merged file system after a reboot/shutdown. Small scripts run at system start and during system shutdown, checking the merged rootfs for any symlinks targeting the earlier mounted SFS package that were orphaned because of an unclean unloading of the SFS package and removes them.

This is why there is no mechanism to share any package information between SFS-Load and the xbps package manager in KLV's. In the Puppy Linux AUFS scheme the SFS's are merged differently and can remain persistent which is an AUFS advantage but also in Puppy's there is a distinction between the temporary SFS package and those hard installed and the PPM and Pkg do not "know" anything about loaded SFS's

[retiredt00]

snapmergepuppy.overlay ignores the symlinks when saving under PUPMODE 13,

So a user can not generate any persistent symlinks in a pupmode 13 installation?
Is that a limitation of overlayfs or a design choice?

[rockedge]

Is that a limitation of overlayfs or a design choice?

The SFS's are dynamically mounted and the symlink targets may be entirely different after a system restart, rendering them orphaned

[dimkr]

So a user can not generate any persistent symlinks in a pupmode 13 installation?

No, you can create persistent symlinks.

Under PUPMODE 13, snapmergepuppy.overlay doesn't persist symlinks to files from dynamically-loaded SFSs (because they will be broken after you unload the SFS or reboot, so there's no reason to save them).

[dimkr]

https://github.com/puppylinux-woof-CE/woof-CE/pull/4131

I started working on a set of improvements in sfs_load.overlay:

• Support for PUPMODE 12 with basic protection against broken symlinks after power loss (@retiredt00's idea with cleaning followed by flush to disk on both startup and shutdown - should be fast if there's no cleanup to do)

• Support for -q (quiet operation - loading or unloading without confirmation) and -c (no splash at the top of the screen)

• Support for the sfs_load -q +/path/to/this.sfs -/path/to/that.sfs syntax to silently load if not loaded or unload if loaded

• (No changes for existing users with PUPMODE 5 or 13: everything is backward-compatible, and only PUPMODE 12 needs special handling)

• (Now sfs_load.overlay is 298 lines compared to 178 lines before these new features, or compared to 2527 lines in sfs_load, so it covers more common use cases without too much complexity)

Feedback and help with testing of all use cases under PUPMODEs 5, 12, 13 and 66 is welcome

(use at your own risk, I did only some basic tests and I'm sure there will be issues)

[mikeslr]

The most practical solution to the PupMode 12 issues is to re-write the installation codes used by frugalpup-installer, grub2config and grub4dos. These default to PupMode12 when installed to a hard-drive. 68 percent responding to the Poll use PupMode13 and for another 21 percent the default doesn't matter as they prefer PupMode 5. Only 11% of those responding to the Poll use PupMode12. https://www.forum.puppylinux.com/viewtopic.php?t=9384, The installers should default to PupMode 13 as that will cover the needs of 89% of Puppy users.

While the Poll sample is small IMHO it is strongly indicative because (a) it overwhelmingly suggests that PupMode 12 is disfavored and (b) those who responded are those most familiar with Puppy and its options. Those most likely to get caught up short by PupMode 12's limitations are 'newbies' who (a) don't know there are options and use whatever Mode is the default, (b) don't respond to polls and most importantly (c) are very unlikely to read about the PupMode12 limitations when using Overlays and the means by which they can be over-come. And add the following: The ability to SFS-Load and Un-Load 'on the fly' used to be much more important when computers had much less RAM and Storage space. Today, that's rarely the case and with the availability of AppImages and Portables the SFS format for running applications can AFAICT be completely avoided.

To run PupMode 13 from a hard-drive used to require two changes: (1) edit of the Linux/Kernel line of the boot argument to pmedia=ataflash and (2) the selection of Save Session to 0/zero + 'Ask at Shutdown'. The recent Puppys I've used wrote (2) by default, but allowed the User to change it. While the mechanics and code may be different, if the default were PupMode 13 but the User was still able to edit Save Session Interval, wouldn't that simple adjustment satisfy the needs of those who actually want PupMode 12?

Unless I'm mistaken --a Big IF-- revising installation codes is just a matter of editing some Bash-Scripts. Once edited, newly created Puppys will have them. Those Puppys already 'in the Wild' can 'upgrade' by simply installing a pet.

[dancytron]

The most practical solution to the PupMode 12 issues is to re-write the installation codes used by frugalpup-installer, grub2config and grub4dos. These default to PupMode12 when installed to a hard-drive. 68 percent responding to the Poll use PupMode13 and for another 21 percent the default doesn't matter as they prefer PupMode 5. Only 11% of those responding to the Poll use PupMode12. https://www.forum.puppylinux.com/viewtopic.php?t=9384, The installers should default to PupMode 13 as that will cover the needs of 89% of Puppy users.

While the Poll sample is small IMHO it is strongly indicative because (a) it overwhelmingly suggests that PupMode 12 is disfavored and (b) those who responded are those most familiar with Puppy and its options. Those most likely to get caught up short by PupMode 12's limitations are 'newbies' who (a) don't know there are options and use whatever Mode is the default, (b) don't respond to polls and most importantly (c) are very unlikely to read about the PupMode12 limitations when using Overlays and the means by which they can be over-come. And add the following: The ability to SFS-Load and Un-Load 'on the fly' used to be much more important when computers had much less RAM and Storage space. Today, that's rarely the case and with the availability of AppImages and Portables the SFS format for running applications can AFAICT be completely avoided.

To run PupMode 13 from a hard-drive used to require two changes: (1) edit of the Linux/Kernel line of the boot argument to pmedia=ataflash and (2) the selection of Save Session to 0/zero + 'Ask at Shutdown'. The recent Puppys I've used wrote (2) by default, but allowed the User to change it. While the mechanics and code may be different, if the default were PupMode 13 but the User was still able to edit Save Session Interval, wouldn't that simple adjustment satisfy the needs of those who actually want PupMode 12?

Unless I'm mistaken --a Big IF-- revising installation codes is just a matter of editing some Bash-Scripts. Once edited, newly created Puppys will have them. Those Puppys already 'in the Wild' can 'upgrade' by simply installing a pet.

It's editing/creating the grub.cfg, menu.1st etc menu entries, right?

Why not give them all the choices and have a boot menu entry for each pupmode (12, 13 and clean boot with no save file) every time they boot up?

[wizard]

@mikeslr

These default to PupMode12 when installed to a hard-drive. 68 percent responding to the Poll use PupMode13 and for another 21 percent the default doesn't matter as they prefer PupMode 5. Only 11% of those responding to the Poll use PupMode12. https://www.forum.puppylinux.com/viewtopic.php?t=9384, The installers should default to PupMode 13 as that will cover the needs of 89% of Puppy users.

With a total of only 28 votes in the poll, I don't think we can draw any accurate conclusions.

Thanks
wizard

[dimkr]

With a total of only 28 votes in the poll, I don't think we can draw any accurate conclusions.

And, the wording is probably unclear to many users. Asking what kind of media people install Puppy on (flash drive, external hard drive, ...) is one thing, and asking which PUPMODE they use is another. The use of a technical term or an implementation detail distorts the poll results, because it increases the number of voters who know what PUPMODE they use because they use the system in non-standard ways (like those who use PUPMODE x but build their own adrv with changes to shutdownconfig and rc.shutdown) and pushes away new users.

(I still believe the majority of Puppy users install it on a flash drive and use pmedia=, pmedia=cd or pmedia=usbflash, therefore they use PUPMODE 13)

[retiredt00]

https://github.com/puppylinux-woof-CE/woof-CE/pull/4131

Dear dimkr
Since is not so simple to install the changes in an existing puppy from the link you provided I made a pet with those 3 files.
Although is small enough to fit in the forum given its transient life, till the next vanilalla/bookworm pup, the sfs_overlay_12.pet file is again uploaded in wetransfer here https://we.tl/t-D4n5DFYOkN so will be gone in a week.

[dimkr]

Since is not so simple to install the changes in an existing puppy from the link you provided I made a pet with those 3 files.

And it's not easy to implement these changes, or test them, but I do it anyway, and do it for free.

[stevie pup]

With a total of only 28 votes in the poll, I don't think we can draw any accurate conclusions.

And, the wording is probably unclear to many users. Asking what kind of media people install Puppy on (flash drive, external hard drive, ...) is one thing, and asking which PUPMODE they use is another. The use of a technical term or an implementation detail distorts the poll results, because it increases the number of voters who know what PUPMODE they use because they use the system in non-standard ways (like those who use PUPMODE x but build their own adrv with changes to shutdownconfig and rc.shutdown) and pushes away new users.

That'll be me then! The reason I haven't voted in the poll is because there wasn't an option for "don't know". I run Puppy's either from USB stick or frugal install on portable hard drive. I leave everything as defaults, as messing around with adrv, sfs, whatever, would be beyond me.

Therefore I haven't got a clue which "Pupmode" I'm using.

[rockedge]

And it's not easy to implement these changes, or test them, but I do it anyway, and do it for free.

me too....and?

[retiredt00]

but I do it anyway, and do it for free.

dear dimkr
I'm not sure how to interpret this comment.
Is this a call for (well deserved) praises for your work on puppylinux or en expression of (unjustified*) frustration for my "not so simple to install" statement?
Hopefully this first.

* I said that is "not simple" because I have seen users having no idea how to use a patch or download the full file from the pull link. So I just wanted to justify my very simple pet offering.
Had nothing to do with your specific post or work.

[rockedge]

Or grab SFS-Load from DebianDog and adapt it for Bookworm Pup64 with a "thank you fredx181". It works .......I tested it.

[dimkr]

I'm not sure how to interpret this comment.

If you want improvements in sfs_load, help develop and test it. And when somebody volunteers to implement ideas or requests of others, don't complain.

[dancytron]

Or grab SFS-Load from DebianDog and adapt it for Bookworm Pup64 with a "thank you fredx181". It works .......I tested it.

I wonder how much of the DD stuff just works or close to it.

Apt2sfs, editsfs, and maybe even the kernel generator theoretically should.

[rockedge]

I wonder how much of the DD stuff just works or close to it.

Apt2sfs, editsfs, and maybe even the kernel generator theoretically should.

I think most should work OOTB or close to it.

[d-pupp]

I agree with @stevie pup. I have been using Puppy for a long time and had to do some searching and look up what pup-mode I was using. I do a frugal install to an internal hard drive.
This is somewhere on the forum but to save searching this code worked for me.

Code: Select all

cat /etc/rc.d/PUPSTATE | grep PUPMODE

[peebee]

Or

Menu -> System -> Pup-SysInfo: Sys-Specs -> Profiles -> Quickview

Distro State:
PUPMODE Mode 12 (normal install, auto-save)