s15pup64-ovrly - overlayfs-only pup

[ozsouth]

s15pup64-ovrly is a minimalist pup, based on peebee's s15pup64-22.12 (made with his permission).
Is overlay-only & useful for folk to check out overlayfs. This is the pup I mostly use.

Version 5 (408mb small bugfix) 05apr24. Based on peebee's 9 Dec 2023 release.
I have also updated firefox and increased compression (medium-high).
Has Firefox-ESR, 6.1 Kernel & small-medium Firmware. bdrv loads just before adrv.

v5 here: https://www.mediafire.com/file/04i3sctl ... 5.iso/file

(No delta, as would be over 240mb). If you are happy with v4t, update not essential.

SFS can now be installed/uninstalled in pupmode 5 ONLY (meaning no savefile/savefolder present).
They are not saved on exit. Can either be unloaded on exit & not included in any savefile or
savefolder created then, OR you can leave them loaded & reboot (will not be offered a save option).
A list of currently Installed SFS is available via the Filesystem menu.

Once a savefile/savefolder exists, can queue sfs to load on next (& future) boots. Once booted with
sfs loaded this way, can unqueue to unload on subsequent boot (would then be removed).
To be available, sfs used for queueing must be on the same partition as the savefile/savefolder, as
that partition will be mounted at /mnt/home. It should be in same folder as puppy .sfs, or a copy
of the queued sfs will be made in that folder, and NOT removed if later unqueued. This could impact
storage if limited (i.e. usb stick). After unqueueing & reboot, config files remain.
A list of currently Queued SFS is available via the Filesystem menu.

(NOTE: The previous v4t bugfix stopped most problematic installing/queueing operations.
You cannot install or queue devx, kbuild or puppy system sfs files - will likely cause corruption.
You can rename first if you absolutely must, then load - but I say don't do it.
Some bad choices can't be stopped without removing queueing completely. Be careful.)

Otherwise, sfs are mounted view-only. They could be renamed b, a, or y drv, & will then load on reboot.
A list of currently Mounted Files is available via the Filesystem menu.

This pup is for Frugal Install (= Standard Puppy Install) ONLY. Use at own risk.
I included a warning on exit about a generic puppy shortcoming - any files in main / folder will
NOT be saved on first shutdown/reboot, but once a savefile/savefolder is created, will be saved
on future exits. Subfolders are not affected.

Savefile/savefolder options are available upon reboot/shutdown. Iso is CD legacy/efi boot.
Has a 35mb fdrv (firmware) & FirefoxESR 115.8.0 (with updater - method below) as adrv;
both easily replaceable.

Has 6.1.73 64bit overlay-only non-usrmerge kernel. Sources & Headers in forum Kernels section.
wl wireless driver in forum Drivers section. (Note many broadcom adaptors covered by fdrv).

NOTE: for successful Compiling, must have a separate frugal install with no savefile/savefolder,
& then devx named as devx_s15pup64_22.12.sfs, sources named as bdrv_s15pup64_22.12.sfs, in the
same folder as puppy_s15pup64_22.12.sfs , so both load on boot. Headers could be (as needed)
loaded as ydrv_s15pup64_22.12.sfs . You CANNOT install/queue a devx to a running system.

devx for s15pup64_22.12-231209 here: https://www.mediafire.com/file/s1a70r54 ... 9.sfs/file

This pup has a good range of basic software. In included FirefoxESR, zoom works via https://app.zoom.us/wc/ ,
google meet & talky.io work, as do my subscription sites. Other .sfs/.pets available are (use at own risk):

anydesk v6.2.0 .pet here: https://www.mediafire.com/file/y92uamo0 ... o.pet/file
anydesk v6.2.0 .sfs here: https://www.mediafire.com/file/nyc7ln2v ... o.sfs/file
Puts entry in Network menu. Note: correct locale & timezone must be set in puppy. On first use, may need to quit & restart & wait.

libreoffice 6.4 .pet here: https://www.mediafire.com/file/np6teh69 ... o.pet/file
libreoffice 6.4 .sfs here: https://www.mediafire.com/file/gk8un6fv ... o.sfs/file
Puts entry in Document menu. Is set to US - to change this, go to:
tools/options/language settings/languages - edit locale & western settings.

masterpdf4 .pet here: https://www.mediafire.com/file/1xhdxyc4 ... o.pet/file
masterpdf4 .sfs here: https://www.mediafire.com/file/bi383zrr ... o.sfs/file
Puts entry in Document menu.

flashgames .pet here: https://www.mediafire.com/file/frv53gu4 ... o.pet/file
flashgames .sfs here: https://www.mediafire.com/file/tvbw1lf4 ... o.sfs/file
Puts entry in Fun menu; now v31; 8 games in /apps folder.

Method for updating Firefox-ESR:
FirefoxESR (adrv) is designed to run as spot. To run as root, edit /usr/local/bin/firefox64esr , & remove 'run-as-spot'.
Updater (in Internet menu) is for 'run-as-spot' only - updates should be offered automatically if running as root.
Updater method (for run-as-spot ONLY): To update Firefox, close it, & click Update Firefox (in Internet Menu).
This temporarily runs Firefox as root - necessary to allow update.
Then click Menu (3 bars on top right) then click Help, then click About Firefox. Will update if possible &
then restart needed. Then close Firefox, then open it from icon - updated version runs, as user spot.

[greengeek]

Keen to give this a try.
Working with iso 1a so far.
1) Have not been able to work out how to switch off tap-to-click yet...
2) Opening a hard drive to view contents seems verrrrrrrrrrrry slow first time (similar to Fossa9.6). Takes a long time to work through the icons and let me open directories.

Otherwise seems quite quick.
Trialling the latest Midori on it so far (not really a Firefox fan) and posting from it.

Still don't really grasp why aufs is verboten nowadays - but if we have to move forward then we have to move forward...
(Is multicore synching the biggest issue with aufs??)

(Booted fine and running on Toshiba Satellite Pro S500 core i5)

[amethyst]

Opening a hard drive to view contents seems verrrrrrrrrrrry slow first time

Re-compressing your Puppy files with gzip should speed things up.

[ozsouth]

@greengeek - nothing wrong with aufs when working, but for the last 4 days, we have been unable to compile aufs kernels, due (we now know) to a kernel commit which broke aufs. Our one & only aufs dev is currently very busy & could not devote time to fixing it, but a non-puppy (porteus linux) aufs user found the problem & temporarily fixed it by reverting to a previous file. A permanent fix will hopefully arrive soon. First indication we had of a problem is that we had an obvious kernel panic. Some errors are less obvious & may go undetected for some time, causing who knows what. This means aufs has a significant element of risk for those wanting the latest kernel. Chromebooks use a kernel about 12 behind current, by which time errors will have been reported by early adopters. I would say 3 kernels behind (2-3 weeks) would be relatively safe.
5.4, 5.10 & 5.15 kernels are no longer patched in aufs. Eventually they will break. It is possible to find & fix errors, but difficult.
Overlayfs (the general linux default) is unaffected by this, as kernel commits are tested against it (& not aufs). Hence my simple puppy remaster - to give puppians a 'taste' of basic overlayfs operation, as one day aufs may not be viable. Biggest single issue with THIS pup is that FULL 'sfs-load on the fly & unload' is unavailable, but pets etc & queueing mitigate this.

Re speed: Interesting. I don't notice that on my 8yo i3 laptop. I did set older ntfs-3g driver, which may be slow on ntfs partitions.

• Maybe a 5.10 kernel would improve this. If you use that, have to ensure s15pup64initmodules.txt not present in working folder (containing sfs's).

For tap-to-click disable, flsynclient is supposed to set it, but didn't work for me, but xinput (in a terminal) did:

xinput [gets touchpad device number. Mine was id=11]

xinput list-props 11 [gets Tapping Enabled process number. Mine was (341)]

xinput set-prop 11 341 0 [turns tapping off - last number 1 turns on - NOTE: all numbers are MY settings]

You can try making a script of the last command & put in /root/Startup (I assume your id & process number won't change).

[rockedge]

Biggest single issue is that sfs-load on the fly & unload' is unavailable

Have you tried out the SFS-Load we are using in KLV-Airedale and KLV-Spectr that @fredx181 assembled for DebianDog?

It has worked for KLV since it's inclusion. This SFS-Load is for overlay filesystems and uses a symlink approach. There are in KLV scripts that check and remove and clean up any dangling symlinks during the boot cycle if the system shuts down before the SFS is unloaded. Of course in Kennel Linux as in DebianDog SFS files can be given a 2 digit prefix and those will load automatically during boot.

So perhaps there is the possibility of a SFS-Load-on-the-fly mechanism in this s15pup64-ovrly

[HerrBert]

Thank you for the invitation to try out overlayfs based kernels.
I've been playing with this for several hours this afternoon.

The approach of symlinking everything sounds very weird - however, if it works for you, enjoy

Since we are/will_be no longer able to load a sfs on the fly, i have a question:

I've tried some appimages, and it seems that overlay driven Puppies have the ability to load sfs in the form of appimage files, which is a little bit confusing to me In my understanding, appimages are still squashed file system files loaded on the fly...

I took my homemade min-blabla(version).sfs, extracted it to another application directory and turned it into an appimage. I had to fiddle with the local storage files when running as spot, but at least i can run it from a sfs turned into an appimage on a non-aufs-puppy-linux.

What am i missing?

[dimkr]

Since we are/will_be no longer able to load a sfs on the fly

It's already implemented in woof-CE (since September '23), any newly built Puppy without aufs or with the punionfs=overlay boot code (to force use of overlay even if aufs is supported) supports sfs_load with overlay out of the box.

[ozsouth]

@rockedge
@dimkr
I do understand that newer (Sept '23 on) pups can have SFS-LOAD options in overlayfs, but v1 is based on a May '23 pup, as an intro to overlayfs. I did this so as to NOT rely on symlinks or garbage collections, with a view to having a savefolder/savefile. I edited earlier posts to stress THIS pup doesn't have SFS-LOAD. I may do an overlayfs pup based on a newer pup in the new year.

@HerrBert
My (limited) understanding of Appimages (which I usually turn into pets) is that they install similar to pets & are not unloaded like sfs on-the-fly.
Glad you found a solution in this case.

[ozsouth]

.

[ozsouth]

I tried loading sfs's in s15pup64-22.12-231209, using overlayfs.
Interesting. Some loaded & ran immediately & others didn't. I discovered that in sfs's, relative-symlinked libs can't be used in the installed program's working directory - have to rename lib as old symlink name; i.e. if you have libQtSvg.so.4 as a relative symlink to libQtSvg.so.4.8.5 in the working directory, need to delete symlink & rename libQtSvg.so.4.8.5 to libQtSvg.so.4.
Also, entries in /usr/local/bin loaded via sfs after boot won't overwrite originals; i.e. if you load libreoffice sfs which includes a new defaultwordprocessor script, abiword will still start if you run /usr/local/bin/defaultwordprocessor . If those things are addressed in your sfs, loading & unloading works.
I will work on something next month. Meanwhile, the current offering is working well without sfs loading & unloading.

[Dry Falls]

I don't know about overlay-sfs_load but the legacy sfs_load runs fix menus and ldconfig on installation. Slackware's ldconfig will sort out the symlinks. Might also run update-desktop-database. Just like aufs, it sounds like sfs-load-overlay mounts to a lower level in the hierarchy so one wouldn't expect the new files to overwrite existing files. You'll need an install script for each misbehaving sfs module (like is done in fatdog) for that.

[dimkr]

sounds like sfs-load-overlay mounts to a lower level in the hierarchy

sfs_load.overlay doesn't add another layer to the SFS stack because overlay doesn't allow this. Instead, it uses symlinks in /initrd/pup_rw to simulate dynamic loading.

Also, entries in /usr/local/bin loaded via sfs after boot won't overwrite originals

This is an known, intentional limitation. A SFS loaded by sfs_load.overlay can't "hide" files below it, simply because it's not part of the SFS stack. The only way for sfs_load.overlay to replace a file with a symlink is to delete the file, which is clearly not a good idea (because that would generate lots of I/O and can lead to data loss).

(SFSs loaded at boot time don't have this limitation, because they're added to the stack normally.)

Some loaded & ran immediately & others didn't.

Can you share some problematic SFSs so I can fix the bug in sfs_load (if there is one)? I tested a large variety of SFSs with the latest sfs_load.overlay in woof-CE (last modified Oct 22) and it works great even for SFSs that contain symlinks.

(The symlink created by sfs_load.overlay under /initrd/pup_rw is absolute and not relative but this shouldn't matter because it still points to the correct file, so you get the same file when you dereference the symlink.)

I don't know about overlay-sfs_load but the legacy sfs_load runs fix menus and ldconfig

So does sfs_load.overlay, see https://github.com/puppylinux-woof-CE/w ... erlay#L186

[ozsouth]

@dimkr - masterpdf4 sfs linked below. The relative-symlinked libQt files in the working directory cause failure to execute with the new sfs_load. Works fine when run on bootup in 230513 'taster' pup as is, and on s15pup64_22.12-231209 version if you delete symlink & rename lib as old symlink name in working directory. Used my 5.10.200 overlay-only kernel (see forum Kernels section). Worst case scenario is careful crafting of sfs's needed.

https://www.mediafire.com/file/x63xfsba ... 2.sfs/file

[ozsouth]

Version 3 of this is coming soon. Based on s15pup64_22.12-231209, with peebee's permission. I'm waiting for the latest Firefox to arrive & testing more in the meantime. This one allows installing/uninstalling sfs on-the-fly in pupmode 5 only (meaning no savefile/folder present). Once a savefile/folder is present, sfs can be queued for loading upon reboot & available until (if ever) they are unqueued & a reboot. Other than that sfs are view only, but can be renamed as bdrv, adrv, or ydrv & load upon reboot. Bdrv now loads after fdrv. SFS actions are initiated by clicking on file (should be on same partition as savefile/folder) in ROX-Filer. I seem to have fixed my symlink issue.
I wanted a straightforward system without multiple messages or issues from bad user choices.

I have an issue with compiling. Seems one must have a separate frugal install for this, loading devx as bdrv (essential), & sources as adrv (desirable), upon bootup. Otherwise a 'cannot run C compiled programs' error halts compiling. It does seem that as long as the devx is loaded upon boot, headers could be installed/uninstalled on-the-fly if needed (sources could too, but takes a few minutes to unload). Devx, sources & headers could be queued/unqueued if desired, but I'm unsure of long-term impact of that.
Bookwormpup64 has got round this by loading (upon boot) a cut-down compiling sfs module - kbuild.

[ozsouth]

V3 released - see 1st post.

[ozsouth]

.

[Clarity]

Hello @ozsouth
Downloaded ISO file and booted directly to desktop via QEMU. Screenshot shows initial checks for items are shown. Excellent performance in initial testing. Very nice!

Code: Select all

qemu-system-x86_64 -name "OZsouth's s15pup64 v3a via Ventoy USB in QEMU" -enable-kvm -m 2G -smp 2 -device AC97 -net nic -net user -rtc base=localtime -vga std -cdrom /root/Downloads/s15pup64-22.12-231209-ovrly-3a.iso

Initial usage checks shown

s15Pup64 v3a.jpg (50.19 KiB) Viewed 971 times

[ozsouth]

@Clarity - glad that works well for you. Sorry, v3a had unwanted menu entry (but not executable, so cosmetic .jwmrc fix only).

NOTE: I made another small update, to v3b (fixed .jwmrc) - see 1st post. 3 options: new iso, delta & 'fixer' .pet.

[Clarity]

Thanks @ozsouth. Boots v3b ISO file fine in QEMU; its stable and performs well.

s15Pup64 v3b.jpg (113.94 KiB) Viewed 944 times

If possible
2 utilities helpful in quick command searches (tldr) and complete system performance in a low-resource screen (btop) would be a welcomed addition, OOTB.

@ozsouth BTW: those 2 utilities are NOT in the base s15Pup64 from @peebee . My mentioning the 2 is merely a mention. They has no operational impact on your current PUP. Those are simple useful utilities.

[Dry Falls]

Hi @ozsouth. I'm having a problem with the sfs_load.overlay gui. Right click is fine and cli as well, but the gui has no eg. yes/no buttons. Just opens the window with installed files highlighted but no action available. extrasfsfind (Quick-sfs-Load) does the trick I'd have expected from sfs_load.overlay gui if sfs_load contains this line

Code: Select all

[ "$PUNIONFS" = "overlay" ] && . /usr/sbin/sfs_load.overlay && exit 1

. May want to look into it.

df

[ozsouth]

@Dry Falls - good one. I removed sfs_load & filemnt from the menu, as I envisaged all operations as being run in ROX-Filer on the sfs itself (as my first post says). I disabled load/unload once a savefile/savefolder exists, as unloading was very unreliable. I wanted to offer queue/unqueue then reboot for that approximate function. I always favour reliability over extra functions.
I did not expect folk to click on the executables, so made no provision for that.
I should have realised folk who had used sfs_load more than me could use it as gui. If your workaround works for you (& due to the difficulty I had coding & as my popup messages follow my posted protocol), I'm inclined to suggest to folk generally, to left-click sfs files directly from ROX-Filer.

[ozsouth]

Version 3d (iso & delta) released. See 1st post. Locks out gui sfs load - I attempted to fix that in v3c, but decided it needed more work.

[Clarity]

Downloading v3d now.

Question
@peebee has stabilized for this month's release at v240208.

• Is there a chance that his latest will provide the ability for your efforts that you are abandoning to be successful?

Just a thought...not a request for you to change/revisit.

If you do an update, I will test.

[ozsouth]

@Clarity - this function works ok in aufs, but less so (in my view) for overlayfs. I was planning to possibly work on a march 2024 s15pup. In the interim I'll see if I can improve on my earlier attempts, including looking at other s15pups.
As our aufs man is back on deck, the urgency is less.

[Clarity]

+1

I will test pristine ISO file boots, tomorrow; on bare-metal, as well, this time around.

[ozsouth]

@Clarity , @Dry Falls - check your pm's re- possible 3e.

[Dry Falls]

Thanks @ozsouth. Got the pm and looking forward to testing. A few questions first. I think it's a woofce thing but why is sbin copied to /pup_new/initrd? The init script also tries to place /bin there too but a trailing slash throws an error ("bin is a directory"). I commented those lines in the init script and have experienced no ill-effects.
Also, I'm a bit confused by the precedence of the sfs files -- what Dimkr objected to when I referred to it as layering . The Readme suggests adrv, bdrv and ydrv take precedence over the base file and possibly the save file. This means any duplicate files will overwrite those. Is this correct?

I've been trying this in lighthouse too by renaming your init to init-overlay and redirecting to that file at the top of the lighthouse init if it finds the overlay bootparameter. So far it's working real well.

Thanks for giving us the chance and encouragement to experiment with this.

df

[ozsouth]

@Dry Falls - interesting - that sbin & bin action must be woof-ce (or a tributary) - I didn't touch that. Might be fixed in current woof-ce.
B, A, Y drvs load after Puppy, F & Z drvs, so can overwrite items in them - I was sure this was standard for all puppies. Savefile follows after that, so it overwrites everything. I use the standard boot parameter punionfs, but an overlay-only kernel loads overlayfs regardless.

As an aside, I'm finding queueing sfs good, as I can queue the likes of anydesk & unqueue it when there is an update, then queue the new version. Only rider is that it really must be in same folder as Puppy sfs. I'm up to 3g in my development, which is a fixed & streamlined 3d, refining the questions asked.
3e was a sideline, using a gui queueing list - still needs explanation of options though - highlighting/unhighlighting isn't obvious.

My plan is to release 3g as version 4 once the next firefox lands. Then I'll continue on with 3e development, which may eventually get released (feedback will help).

[dimkr]

The init script also tries to place /bin there too but a trailing slash throws an error ("bin is a directory"). I commented those lines in the init script and have experienced no ill-effects.

Should be fixed by https://github.com/puppylinux-woof-CE/woof-CE/pull/3824, if and when it gets merged.

The Readme suggests adrv, bdrv and ydrv take precedence over the base file and possibly the save file.

The README is old, I don't know who wrote it and when but it has some inaccuracies. The order, from top to bottom, is: save > adrv > ydrv > bdrv > puppy_* > [...] > fdrv > zdrv, i.e. adrv can override a file in any lower layer, and save always wins because it's the top layer.

(But don't confuse this with sfs_load.overlay: it doesn't change the SFS stack, because overlay doesn't allow this: it only adds symlinks in the save layer until the SFS is "unloaded" + stale symlinks are cleaned up on shutdown and boot)

[ozsouth]

@Dry Falls wrote:

I think it's a woofce thing but why is sbin copied to /pup_new/initrd? The init script also tries to place /bin there too but a trailing slash throws an error ("bin is a directory"). I commented those lines in the init script and have experienced no ill-effects.

I presume you mean lines 1499 , 1503 & 1504:

Code: Select all

1499 cp -a /init* /sbin /README.txt /pup_new/initrd/

1503 [ -f /bin/cryptsetup ] && cp -af /bin/cryptsetup /pup_new/initrd/bin/ 
1504 [ -f /sbin/cryptsetup ] && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/

As @dimkr has concurred above, I could edit /sbin out of 1499.
1503-1504 don't seem to do anything unless folder is made (if cryptsetup exists), so could comment those out.

Also, I added 'last' to end of line 93 in README.txt:
* So, if this is not a first boot, init has to setup any save layer and insert it into the stack, last.