s15pup64-ovrly - closed

[ozsouth]

01march25 - Not continuing with this - my Sovly (simple overlay) pup is my focus now.
At end of this post is the basic bdrv, which converts (all?) 2024 s15pup64 pups to overlay-only.

There are also some SFS app links below - could be used in Sovly pup (use at own risk).

anydesk v6.2.0 .sfs here: https://www.mediafire.com/file/nyc7ln2v ... o.sfs/file
Puts entry in Network menu. Note: correct locale & timezone must be set in puppy. On first use, may need to quit & restart & wait.

libreoffice 6.4 .sfs here: https://www.mediafire.com/file/gk8un6fv ... o.sfs/file
Puts entry in Document menu. Is set to US - to change this, go to:
tools/options/language settings/languages - edit locale & western settings.

masterpdf4 .sfs here: https://www.mediafire.com/file/bi383zrr ... o.sfs/file
Puts entry in Document menu.

flashgames .sfs here: https://www.mediafire.com/file/tvbw1lf4 ... o.sfs/file
Puts entry in Fun menu; now v31; 8 games in /apps folder.

######################################################################

Conversion sfs below - may not work in future if big changes made to s15pup64 & I don't intend to test further.
.

[greengeek]

Keen to give this a try.
Working with iso 1a so far.
1) Have not been able to work out how to switch off tap-to-click yet...
2) Opening a hard drive to view contents seems verrrrrrrrrrrry slow first time (similar to Fossa9.6). Takes a long time to work through the icons and let me open directories.

Otherwise seems quite quick.
Trialling the latest Midori on it so far (not really a Firefox fan) and posting from it.

Still don't really grasp why aufs is verboten nowadays - but if we have to move forward then we have to move forward...
(Is multicore synching the biggest issue with aufs??)

(Booted fine and running on Toshiba Satellite Pro S500 core i5)

[amethyst]

Opening a hard drive to view contents seems verrrrrrrrrrrry slow first time

Re-compressing your Puppy files with gzip should speed things up.

[ozsouth]

@greengeek - nothing wrong with aufs when working, but for the last 4 days, we have been unable to compile aufs kernels, due (we now know) to a kernel commit which broke aufs. Our one & only aufs dev is currently very busy & could not devote time to fixing it, but a non-puppy (porteus linux) aufs user found the problem & temporarily fixed it by reverting to a previous file. A permanent fix will hopefully arrive soon. First indication we had of a problem is that we had an obvious kernel panic. Some errors are less obvious & may go undetected for some time, causing who knows what. This means aufs has a significant element of risk for those wanting the latest kernel. Chromebooks use a kernel about 12 behind current, by which time errors will have been reported by early adopters. I would say 3 kernels behind (2-3 weeks) would be relatively safe.
5.4, 5.10 & 5.15 kernels are no longer patched in aufs. Eventually they will break. It is possible to find & fix errors, but difficult.
Overlayfs (the general linux default) is unaffected by this, as kernel commits are tested against it (& not aufs). Hence my simple puppy remaster - to give puppians a 'taste' of basic overlayfs operation, as one day aufs may not be viable. Biggest single issue with THIS pup is that FULL 'sfs-load on the fly & unload' is unavailable, but pets etc & queueing mitigate this.

Re speed: Interesting. I don't notice that on my 8yo i3 laptop. I did set older ntfs-3g driver, which may be slow on ntfs partitions.

• Maybe a 5.10 kernel would improve this. If you use that, have to ensure s15pup64initmodules.txt not present in working folder (containing sfs's).

For tap-to-click disable, flsynclient is supposed to set it, but didn't work for me, but xinput (in a terminal) did:

xinput [gets touchpad device number. Mine was id=11]

xinput list-props 11 [gets Tapping Enabled process number. Mine was (341)]

xinput set-prop 11 341 0 [turns tapping off - last number 1 turns on - NOTE: all numbers are MY settings]

You can try making a script of the last command & put in /root/Startup (I assume your id & process number won't change).

[rockedge]

Biggest single issue is that sfs-load on the fly & unload' is unavailable

Have you tried out the SFS-Load we are using in KLV-Airedale and KLV-Spectr that @fredx181 assembled for DebianDog?

It has worked for KLV since it's inclusion. This SFS-Load is for overlay filesystems and uses a symlink approach. There are in KLV scripts that check and remove and clean up any dangling symlinks during the boot cycle if the system shuts down before the SFS is unloaded. Of course in Kennel Linux as in DebianDog SFS files can be given a 2 digit prefix and those will load automatically during boot.

So perhaps there is the possibility of a SFS-Load-on-the-fly mechanism in this s15pup64-ovrly

[HerrBert]

Thank you for the invitation to try out overlayfs based kernels.
I've been playing with this for several hours this afternoon.

The approach of symlinking everything sounds very weird - however, if it works for you, enjoy

Since we are/will_be no longer able to load a sfs on the fly, i have a question:

I've tried some appimages, and it seems that overlay driven Puppies have the ability to load sfs in the form of appimage files, which is a little bit confusing to me In my understanding, appimages are still squashed file system files loaded on the fly...

I took my homemade min-blabla(version).sfs, extracted it to another application directory and turned it into an appimage. I had to fiddle with the local storage files when running as spot, but at least i can run it from a sfs turned into an appimage on a non-aufs-puppy-linux.

What am i missing?

[dimkr]

Since we are/will_be no longer able to load a sfs on the fly

It's already implemented in woof-CE (since September '23), any newly built Puppy without aufs or with the punionfs=overlay boot code (to force use of overlay even if aufs is supported) supports sfs_load with overlay out of the box.

[ozsouth]

@rockedge
@dimkr
I do understand that newer (Sept '23 on) pups can have SFS-LOAD options in overlayfs, but v1 is based on a May '23 pup, as an intro to overlayfs. I did this so as to NOT rely on symlinks or garbage collections, with a view to having a savefolder/savefile. I edited earlier posts to stress THIS pup doesn't have SFS-LOAD. I may do an overlayfs pup based on a newer pup in the new year.

@HerrBert
My (limited) understanding of Appimages (which I usually turn into pets) is that they install similar to pets & are not unloaded like sfs on-the-fly.
Glad you found a solution in this case.

[ozsouth]

.

[ozsouth]

I tried loading sfs's in s15pup64-22.12-231209, using overlayfs.
Interesting. Some loaded & ran immediately & others didn't. I discovered that in sfs's, relative-symlinked libs can't be used in the installed program's working directory - have to rename lib as old symlink name; i.e. if you have libQtSvg.so.4 as a relative symlink to libQtSvg.so.4.8.5 in the working directory, need to delete symlink & rename libQtSvg.so.4.8.5 to libQtSvg.so.4.
Also, entries in /usr/local/bin loaded via sfs after boot won't overwrite originals; i.e. if you load libreoffice sfs which includes a new defaultwordprocessor script, abiword will still start if you run /usr/local/bin/defaultwordprocessor . If those things are addressed in your sfs, loading & unloading works.
I will work on something next month. Meanwhile, the current offering is working well without sfs loading & unloading.

[dimkr]

sounds like sfs-load-overlay mounts to a lower level in the hierarchy

sfs_load.overlay doesn't add another layer to the SFS stack because overlay doesn't allow this. Instead, it uses symlinks in /initrd/pup_rw to simulate dynamic loading.

Also, entries in /usr/local/bin loaded via sfs after boot won't overwrite originals

This is an known, intentional limitation. A SFS loaded by sfs_load.overlay can't "hide" files below it, simply because it's not part of the SFS stack. The only way for sfs_load.overlay to replace a file with a symlink is to delete the file, which is clearly not a good idea (because that would generate lots of I/O and can lead to data loss).

(SFSs loaded at boot time don't have this limitation, because they're added to the stack normally.)

Some loaded & ran immediately & others didn't.

Can you share some problematic SFSs so I can fix the bug in sfs_load (if there is one)? I tested a large variety of SFSs with the latest sfs_load.overlay in woof-CE (last modified Oct 22) and it works great even for SFSs that contain symlinks.

(The symlink created by sfs_load.overlay under /initrd/pup_rw is absolute and not relative but this shouldn't matter because it still points to the correct file, so you get the same file when you dereference the symlink.)

I don't know about overlay-sfs_load but the legacy sfs_load runs fix menus and ldconfig

So does sfs_load.overlay, see https://github.com/puppylinux-woof-CE/w ... erlay#L186

[ozsouth]

@dimkr - masterpdf4 sfs linked below. The relative-symlinked libQt files in the working directory cause failure to execute with the new sfs_load. Works fine when run on bootup in 230513 'taster' pup as is, and on s15pup64_22.12-231209 version if you delete symlink & rename lib as old symlink name in working directory. Used my 5.10.200 overlay-only kernel (see forum Kernels section). Worst case scenario is careful crafting of sfs's needed.

https://www.mediafire.com/file/x63xfsba ... 2.sfs/file

[ozsouth]

Version 3 of this is coming soon. Based on s15pup64_22.12-231209, with peebee's permission. I'm waiting for the latest Firefox to arrive & testing more in the meantime. This one allows installing/uninstalling sfs on-the-fly in pupmode 5 only (meaning no savefile/folder present). Once a savefile/folder is present, sfs can be queued for loading upon reboot & available until (if ever) they are unqueued & a reboot. Other than that sfs are view only, but can be renamed as bdrv, adrv, or ydrv & load upon reboot. Bdrv now loads after fdrv. SFS actions are initiated by clicking on file (should be on same partition as savefile/folder) in ROX-Filer. I seem to have fixed my symlink issue.
I wanted a straightforward system without multiple messages or issues from bad user choices.

I have an issue with compiling. Seems one must have a separate frugal install for this, loading devx as bdrv (essential), & sources as adrv (desirable), upon bootup. Otherwise a 'cannot run C compiled programs' error halts compiling. It does seem that as long as the devx is loaded upon boot, headers could be installed/uninstalled on-the-fly if needed (sources could too, but takes a few minutes to unload). Devx, sources & headers could be queued/unqueued if desired, but I'm unsure of long-term impact of that.
Bookwormpup64 has got round this by loading (upon boot) a cut-down compiling sfs module - kbuild.

[ozsouth]

V3 released - see 1st post.

[ozsouth]

.

[Clarity]

Hello @ozsouth
Downloaded ISO file and booted directly to desktop via QEMU. Screenshot shows initial checks for items are shown. Excellent performance in initial testing. Very nice!

Code: Select all

qemu-system-x86_64 -name "OZsouth's s15pup64 v3a via Ventoy USB in QEMU" -enable-kvm -m 2G -smp 2 -device AC97 -net nic -net user -rtc base=localtime -vga std -cdrom /root/Downloads/s15pup64-22.12-231209-ovrly-3a.iso

Initial usage checks shown

s15Pup64 v3a.jpg (50.19 KiB) Viewed 4620 times

[ozsouth]

@Clarity - glad that works well for you. Sorry, v3a had unwanted menu entry (but not executable, so cosmetic .jwmrc fix only).

NOTE: I made another small update, to v3b (fixed .jwmrc) - see 1st post. 3 options: new iso, delta & 'fixer' .pet.

[Clarity]

Thanks @ozsouth. Boots v3b ISO file fine in QEMU; its stable and performs well.

s15Pup64 v3b.jpg (113.94 KiB) Viewed 4593 times

If possible
2 utilities helpful in quick command searches (tldr) and complete system performance in a low-resource screen (btop) would be a welcomed addition, OOTB.

@ozsouth BTW: those 2 utilities are NOT in the base s15Pup64 from @peebee . My mentioning the 2 is merely a mention. They has no operational impact on your current PUP. Those are simple useful utilities.

[ozsouth]

@Dry Falls - good one. I removed sfs_load & filemnt from the menu, as I envisaged all operations as being run in ROX-Filer on the sfs itself (as my first post says). I disabled load/unload once a savefile/savefolder exists, as unloading was very unreliable. I wanted to offer queue/unqueue then reboot for that approximate function. I always favour reliability over extra functions.
I did not expect folk to click on the executables, so made no provision for that.
I should have realised folk who had used sfs_load more than me could use it as gui. If your workaround works for you (& due to the difficulty I had coding & as my popup messages follow my posted protocol), I'm inclined to suggest to folk generally, to left-click sfs files directly from ROX-Filer.

[ozsouth]

Version 3d (iso & delta) released. See 1st post. Locks out gui sfs load - I attempted to fix that in v3c, but decided it needed more work.

[Clarity]

Downloading v3d now.

Question
@peebee has stabilized for this month's release at v240208.

• Is there a chance that his latest will provide the ability for your efforts that you are abandoning to be successful?

Just a thought...not a request for you to change/revisit.

If you do an update, I will test.

[ozsouth]

@Clarity - this function works ok in aufs, but less so (in my view) for overlayfs. I was planning to possibly work on a march 2024 s15pup. In the interim I'll see if I can improve on my earlier attempts, including looking at other s15pups.
As our aufs man is back on deck, the urgency is less.

[Clarity]

+1

I will test pristine ISO file boots, tomorrow; on bare-metal, as well, this time around.

[ozsouth]

@Clarity , @Dry Falls - check your pm's re- possible 3e.

[ozsouth]

@Dry Falls - interesting - that sbin & bin action must be woof-ce (or a tributary) - I didn't touch that. Might be fixed in current woof-ce.
B, A, Y drvs load after Puppy, F & Z drvs, so can overwrite items in them - I was sure this was standard for all puppies. Savefile follows after that, so it overwrites everything. I use the standard boot parameter punionfs, but an overlay-only kernel loads overlayfs regardless.

As an aside, I'm finding queueing sfs good, as I can queue the likes of anydesk & unqueue it when there is an update, then queue the new version. Only rider is that it really must be in same folder as Puppy sfs. I'm up to 3g in my development, which is a fixed & streamlined 3d, refining the questions asked.
3e was a sideline, using a gui queueing list - still needs explanation of options though - highlighting/unhighlighting isn't obvious.

My plan is to release 3g as version 4 once the next firefox lands. Then I'll continue on with 3e development, which may eventually get released (feedback will help).

[dimkr]

The init script also tries to place /bin there too but a trailing slash throws an error ("bin is a directory"). I commented those lines in the init script and have experienced no ill-effects.

Should be fixed by https://github.com/puppylinux-woof-CE/woof-CE/pull/3824, if and when it gets merged.

The Readme suggests adrv, bdrv and ydrv take precedence over the base file and possibly the save file.

The README is old, I don't know who wrote it and when but it has some inaccuracies. The order, from top to bottom, is: save > adrv > ydrv > bdrv > puppy_* > [...] > fdrv > zdrv, i.e. adrv can override a file in any lower layer, and save always wins because it's the top layer.

(But don't confuse this with sfs_load.overlay: it doesn't change the SFS stack, because overlay doesn't allow this: it only adds symlinks in the save layer until the SFS is "unloaded" + stale symlinks are cleaned up on shutdown and boot)

[ozsouth]

@Dry Falls wrote:

I think it's a woofce thing but why is sbin copied to /pup_new/initrd? The init script also tries to place /bin there too but a trailing slash throws an error ("bin is a directory"). I commented those lines in the init script and have experienced no ill-effects.

I presume you mean lines 1499 , 1503 & 1504:

Code: Select all

1499 cp -a /init* /sbin /README.txt /pup_new/initrd/

1503 [ -f /bin/cryptsetup ] && cp -af /bin/cryptsetup /pup_new/initrd/bin/ 
1504 [ -f /sbin/cryptsetup ] && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/

As @dimkr has concurred above, I could edit /sbin out of 1499.
1503-1504 don't seem to do anything unless folder is made (if cryptsetup exists), so could comment those out.

Also, I added 'last' to end of line 93 in README.txt:
* So, if this is not a first boot, init has to setup any save layer and insert it into the stack, last.

[ozsouth]

It appears editing /sbin out of line 1499 is correct. New version: 1499 cp -a /init* /README.txt /pup_new/initrd/

However, what about lines 1503-1504?
1503 [ -f /bin/cryptsetup ] && cp -af /bin/cryptsetup /pup_new/initrd/bin/
1504 [ -f /sbin/cryptsetup ] && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/

Does cryptsetup NEED to be copied to /pup_new/initrd?

Rizalmart's github request had me thinking 'yes' & making /bin or /sbin folder (as needed) in /pup_new/initrd/ , thus:
1503 [ -f /bin/cryptsetup ] && mkdir /pup_new/initrd/bin && cp -af /bin/cryptsetup /pup_new/initrd/bin/ # mkdir bin
1504 [ -f /sbin/cryptsetup ] && mkdir /pup_new/initrd/sbin && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/ # mkdir sbin

Earlier comments had me thinking 'no' & commenting them out.

@Dry Falls @dimkr

ALL options (including doing nothing OR commenting out all 3 lines) appear to work, but I've no idea which is best to implement.

[ozsouth]

@Dry Falls - thanks for that. 1 issue - crypsetup is in /sbin - see pic below. ldd says 'not a dynamic executable', whereas /initrd/bin version says 'static', as you said. I note that the initrd version was not installed in previous iterations of this pup, as the lines 1503-1504 didn't do anything. Now what to do? Maybe remove /sbin from line 1499 & comment out lines 1503-1504 to avoid a clash?

LATER: I did as I suggested above & made a savefile with luks - which uses cryptsetup - is fine.

cryptsetup.jpg (40.02 KiB) Viewed 4708 times

[ozsouth]

I just checked over several puppies - pretty much every puppy released in the last 2 years has in initrd.gz, (in the last 30 or so lines of init), these 6 lines (can view init file in /initrd):

Code: Select all

cp -a /init* /sbin /README.txt /pup_new/initrd/
chmod -x /pup_new/initrd/init
dmesg > /tmp/dmesg.txt

[ -f /bin/cryptsetup ] && cp -af /bin/cryptsetup /pup_new/initrd/bin/
[ -f /sbin/cryptsetup ] && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/

This is obviously due to woof-ce. Some of it doesn't work. My rework, replacing the blank 4th line & editing the 5th line, will (I think) enact the code's intention:

Code: Select all

cp -a /init* /sbin /README.txt /pup_new/initrd/
chmod -x /pup_new/initrd/init
dmesg > /tmp/dmesg.txt
cp -f /etc/rc.d/functions_x /pup_new/initrd/sbin/ #copy real file over dead link
[ -f /bin/cryptsetup ] && mkdir /pup_new/initrd/bin && cp -af /bin/cryptsetup /pup_new/initrd/bin/ #must create dir
[ -f /sbin/cryptsetup ] && cp -af /sbin/cryptsetup /pup_new/initrd/sbin/

In the original, the value of /sbin in the first line is questionable after boot, since it's contents shouldn't thereafter execute. Ditto the last line. The 2nd last line does not appear to do anything. Earlier, I tried taking /sbin out of first line & commenting out last 2 lines (of original) & ran it for a day - seems fine, BUT

as last week's Vanilla Dpup (10.0.47) has the 6 lines in first code box above, I feel that if @dimkr hasn't changed them, I don't feel like being (& making those who download my pup) pioneers in this.

So, I'll wait a couple of days for more comments, then release v4.