Puppy Linux Discussion Forum

I'll have a look. I've pulled my #4 and My #3a is my best attempt yet. At some point I seem to just go in circles. If yours works consistently and keeps people out of trouble I'd say it's a go. Get back to you tomorrow probably.

Thanks, Jim

Winner!

I looked at your installpkgC2.sh, ran it a bit, and used your code to find the errors in my cut4. That runs too but the function setup is clumsy.
Decided that yours was simpler and better. The near duplicated symlinked/non-symlinkedcode doesn't add much size.

Then at 3:24, an aha. What if in your code, all of the installs used the install first to wkdir and then only adjust directories if the symlinked system was detected? like:

Adjust_Directories () {
 if [ -L /bin ]; then	#Marv
   if [ -d ${WKDIR}/bin ] ; then cp -fr ${WKDIR}/bin  ${WKDIR}/usr; rm -fr ${WKDIR}/bin; fi
   if [ -d ${WKDIR}/lib ] ; then cp -fr ${WKDIR}/lib  ${WKDIR}/usr; rm -fr ${WKDIR}/lib; fi
   if [ -d ${WKDIR}/lib32 ] ; then cp -fr ${WKDIR}/lib32  ${WKDIR}/usr; rm -fr ${WKDIR}/lib32; fi
   if [ -d ${WKDIR}/lib64 ] ; then cp -fr ${WKDIR}/lib64  ${WKDIR}/usr; rm -fr ${WKDIR}/lib64; fi
   if [ -d ${WKDIR}/sbin ] ; then cp -fr ${WKDIR}/sbin ${WKDIR}/usr; rm -fr ${WKDIR}/sbin; fi
 fi  #Marv

Then the duplicate normal code in that if else structure isn't needed.

Trimmed and run with a few pets. Sample attached with Marvmods tagging changes. Feel free to use or toss.

Feed the fire and sleep now.

Jim

AM Update: Doing it this (trimmed) way does eliminate the option to save directly to the savefile or a puppy layer. Neither is a possibility as they stand for the new symlinked path pups so that would only apply to older pups and to hardware with very limited memory and no swap. Keeping the extra code as per your scheme keeps this possibility. Is it ever used?? Laying an unknown pkg directly on an AUFS or overlayfs layer or on a cherished savefile gives me the willies anyway. Just my thoughts.

Marv wrote: ↑Sun Mar 05, 2023 11:01 am

Winner!
Decided that yours was simpler and better.

Looks really good Jim, even simpler.

However another problem, I realized that symlinks created from /usr/lib and /usr/lib64 when installing non-debian packages were left dangling if a package was removed because they're not included in /var/packages/libXXX.files. Was able to correct this with code at the end of /usr/local/petget/removemodes.sh but it just wipes out all dangling symlinks which might cause problems? Trying to do it in Adjust_Directories but its fighting me. I think I'll create another function and try placing it where it will work.

I saw those dangling symlinks at some point. Good luck undangling them.

I did some more looking vis-a-vis the Direct install modes and the AUFS mode is disabled and the default is to tmpfs, so basically the same memory use as your install, with the option being there for a user to choose to install directly to savefile. I'll wager no-one ever has selected that and hope no-one ever will. I've never even seen it.

A little experiment: I picked a fairly largish pet for me, a rather piggish HPLIP, extracted it to /tmp on my circa 2012 laptop with 4GB RAM, not an excessive amount these days. It weighed in around 10MB extracted and bumped my tmpfs usage from 11% to 12%.

I really think the trimmed version of your install.sh is OK (with or without the vestigial Direct install code).

Jim
.

@jrb fredx181 has worked on solutions to the dangling symlinks in DebianDog and KLV-Airedale for cleanup functions after loading SFS on the fly on layered Overlayfs systems and have shut down or rebooted while the SFS is loaded.

The mechanism is experimental that does a SFS package load on a running system using only symlinks, which is much different than SFS load on the fly on AUFS systems.

In case of the SFS not being unloaded and leaving symlinks all around, there is a routine that cleans them up at the next system start up. Maybe something there might be useful.

Marv wrote: ↑Sun Mar 05, 2023 4:52 pm

I really think the trimmed version of your install.sh is OK (with or without the vestigial Direct install code).

Looks like you were right, the more I played with it the worse it got!

rockedge wrote: ↑Sun Mar 05, 2023 6:48 pm

@jrb fredx181 has worked on solutions to the dangling symlinks in DebianDog and KLV-Airedale for cleanup functions after loading SFS on the fly on layered Overlayfs systems and have shut down or rebooted while the SFS is loaded.

The mechanism is experimental that does a SFS package load on a running system using only symlinks, which is much different than SFS load on the fly on AUFS systems.

In case of the SFS not being unloaded and leaving symlinks all around, there is a routine that cleans them up at the next system start up. Maybe something there might be useful.

Thanks rockedge, I actually have a fix that works. It just really annoyed me that those symlinks weren't showing up in the /var/packages/libXXX.files so they could be removed normally. I've had to modify /usr/local/petget/removemodes.sh a small bit, but it works. So, after a bit more testing tonight I'll do another build. "Perfect is the enemy of good."

Have you tried Jammy64pup yet?

Cheers, J

Inch by inch, or perhaps micron by micron.

It's happening is highly unlikely but something like this

wkdir_memcheck () {
  USE=`df --output='pcent'  /tmp/petget_proc/wkdir | grep -o '[0-9]*'`
  if [ "$USE" -ge "90" ]; then    #or so, Marv
    . /usr/lib/gtkdialog/box_splash -timeout 2 -fontsize large -text "Temporary memory full, aborting install, consider setting up a swap" > /dev/null 2>&1 &
    exit
  fi
}  

could be added and called just before Adjust_Directories perhaps.

If you seriously consider the option of fixing PPM instead of adding apt and Synaptic, consider submitting your fixes to https://github.com/puppylinux-woof-CE/woof-CE. Otherwise, they will be lost just like many other manual changes that went into "official" Puppy releases.

dimkr wrote: ↑Mon Mar 06, 2023 8:28 am

If you seriously consider the option of fixing PPM instead of adding apt and Synaptic, consider submitting your fixes to https://github.com/puppylinux-woof-CE/woof-CE. Otherwise, they will be lost just like many other manual changes that went into "official" Puppy releases.

I have got PPM working and have tried it in several different Puppies, both X86 and X86_64, Debian and non-Debian. Still working on understanding the old code and polishing it up. With a bit more testing I will submit. Latest version in new release this morning.

Marv wrote: ↑Sun Mar 05, 2023 9:26 pm

Inch by inch, or perhaps micron by micron.

It's happening is highly unlikely but something like this

Code: Select all

wkdir_memcheck () {
  USE=`df --output='pcent'  /tmp/petget_proc/wkdir | grep -o '[0-9]*'`
  if [ "$USE" -ge "90" ]; then    #or so, Marv
    . /usr/lib/gtkdialog/box_splash -timeout 2 -fontsize large -text "Temporary memory full, aborting install, consider setting up a swap" > /dev/null 2>&1 &
    exit
  fi
}  

could be added and called just before Adjust_Directories perhaps.

Will try this out Marv, but I think I better get a new beta release out so people don't think I've given up. PPM is behaving pretty well now but there's a few things I want to better understand before calling it finished.

OK, New ISO at page 1 post 1

Added, PupControl, PupSysinfo, load-monitor, xfdiff-cut. PPM working well, more testing needed. Uninstall those fix.pets before using, or better yet create new Pupsave.

Try it and let me know what you think. I think it's pretty close to ready, knock on wood. (I think I said that before, still true.)

Cheers, J

Hi @jrb,

you need to adjust the corresponding md5sum for your latest *B3 on page 1 - the hash you gave belongs to the *B2 version.

Many thanks for your effort to provide a jammypup64.

peace

one wrote: ↑Mon Mar 06, 2023 3:05 pm

Hi @jrb,

you need to adjust the corresponding md5sum for your latest *B3 on page 1 - the hash you gave belongs to the *B2 version.

Many thanks for your effort to provide a jammypup64.

peace

Could have sworn I changed it. Oh well, I'm easily confused.

Thanks @one and you're very welcome.

@jrb: I thought A2 version was good, but this B3 is much better, haven't found anything to suggest yet....

All good here with B3 so far. Scrubbed my ydrv prior to first boot with it. Nice to see Pup-Sysinfo and PupControl on board. All my portables, shortcuts, themes, PCManFM and Sylpheed OK. Tested all 3 printers installed under earlier isos and they're fine. I'll check and see if Claws Mail has imap support now and if so give it a spin. Posting from B3 with the Brave portable now.

I've run it with the stock kernel and with a 'structure corrected' AUFS/overlay 6.1.13. I have @ozsouth 6.1.14 set up to go but haven't swapped it in yet.

My only complaint is that it didn't repair my laptops ditzy BIOS for me -or shovel my slush- Some PPM use coming up when I tackle an EDID workaround for that BIOS problem, but I already have a fair bit of PPM use with pretty much what's in B3 and it does seem solid now.

Thanks, Jim

@jrb I still haven't checked which fixes made it into B3 but latest woof-CE has some important fixes (some reported and fixed for @radky's F96), and I know some didn't make it into B3:

https://github.com/puppylinux-woof-CE/w ... 366c3c254a
https://github.com/puppylinux-woof-CE/w ... fb388a246a
https://github.com/puppylinux-woof-CE/w ... 507812bcd6
https://github.com/puppylinux-woof-CE/w ... 1309e7e647
https://github.com/puppylinux-woof-CE/w ... bbdf3b34be
https://github.com/puppylinux-woof-CE/w ... ac2e86e20e
https://github.com/puppylinux-woof-CE/w ... 2abe723aa5

Plus, you can grab an up-to-date 5.15.x kernel built from the Ubuntu 22.04 kernel source (like the one you're using one, if I understand correctly, but up-to-date) at https://github.com/puppylinux-woof-CE/w ... kernel.yml. This a monster kernel with a huge variety of drivers and currently it has both aufs and overlay.

We're definitely on the right track!

"B3" frugal install on ext4 HDD partition, on core2 HP Elitebook.
Some observations:
Pup Control > Desktop Wallpaper is not active/working

First shutdown creating an encrypted save-file hangs/fails > alt+ctrl+backspace > exits to console prompt > reboot > hangs > alt+ctrl+del > un-mounts 'stray filesystem' , 'Terminated' > console prompt > reboot > hangs, making hard shutdown with power button necessary.
On next boot the save-file gets loaded but the password is not recognized, using 'c' (instead of password) goes to a console prompt.
Attempt to create an encrypted save-file abandoned.

First shutdown creating an unencrypted (normal) save-file hangs > alt+ctrl+del > console prompt > reboot > hangs , making hard shutdown with power button necessary.

Next boot the (normal) save-file gets loaded correctly.

Attached is the mk2fs.log from one of my failed attempts to create a working encrypted save-file. The encrypted save-file cannot be mounted.

Posting from 'Another Jammy64pup', with Pale Moon as browser.
Regards
proebler

dimkr wrote: ↑Tue Mar 07, 2023 6:06 am

@jrb I still haven't checked which fixes made it into B3 but latest woof-CE has some important fixes (some reported and fixed for @radky's F96), and I know some didn't make it into B3:

https://github.com/puppylinux-woof-CE/w ... 366c3c254a
https://github.com/puppylinux-woof-CE/w ... fb388a246a
https://github.com/puppylinux-woof-CE/w ... 507812bcd6
https://github.com/puppylinux-woof-CE/w ... 1309e7e647
https://github.com/puppylinux-woof-CE/w ... bbdf3b34be
https://github.com/puppylinux-woof-CE/w ... ac2e86e20e
https://github.com/puppylinux-woof-CE/w ... 2abe723aa5

Plus, you can grab an up-to-date 5.15.x kernel built from the Ubuntu 22.04 kernel source (like the one you're using one, if I understand correctly, but up-to-date) at https://github.com/puppylinux-woof-CE/w ... kernel.yml. This a monster kernel with a huge variety of drivers and currently it has both aufs and overlay.

We're definitely on the right track!

Will do. BTW: Making progress on /usr/local/petget/installpkg.sh. Can get .tgz's installed with symlinks to /usr/lib/x86_64-linux-gnu included in /var/packages/XXX.files

proebler wrote: ↑Tue Mar 07, 2023 9:06 am

"B3" frugal install on ext4 HDD partition, on core2 HP Elitebook.
Some observations:
Pup Control > Desktop Wallpaper is not active/working

First shutdown creating an encrypted save-file hangs/fails > alt+ctrl+backspace > exits to console prompt > reboot > hangs > alt+ctrl+del > un-mounts 'stray filesystem' , 'Terminated' > console prompt > reboot > hangs, making hard shutdown with power button necessary.
On next boot the save-file gets loaded but the password is not recognized, using 'c' (instead of password) goes to a console prompt.
Attempt to create an encrypted save-file abandoned.

First shutdown creating an unencrypted (normal) save-file hangs > alt+ctrl+del > console prompt > reboot > hangs , making hard shutdown with power button necessary.

Next boot the (normal) save-file gets loaded correctly.

Attached is the mk2fs.log from one of my failed attempts to create a working encrypted save-file. The encrypted save-file cannot be mounted.

Posting from 'Another Jammy64pup', with Pale Moon as browser.
Regards
proebler

Good report, Thanks. I noticed that several of the scripts, including wallpaper, in /usr/local/PupControl/defaultapps are empty. You can fix the wallpaper script simply by typing in "wallpaper". Will be in next build. Maybe you could see what else can be fixed? These are scripts so just clicking on them should open something.

Cheers, J

Edit: Oh yeah, I'll have a look at the encryption problem. No clue at present.

proebler wrote: ↑Tue Mar 07, 2023 9:06 am

"B3" frugal install on ext4 HDD partition, on core2 HP Elitebook.
Some observations:

First shutdown creating an unencrypted (normal) save-file hangs > alt+ctrl+del > console prompt > reboot > hangs , making hard shutdown with power button necessary.

Next boot the (normal) save-file gets loaded correctly.

Regards
proebler

This sounds just like the shutdown problem that people are chasing in F96-CE_2. There it seems to be hardware related, and possibly related to changes in rc.shutdown and shutdownconfig between CE_1 and CE_2 (my take only). I tried to replicate the failure on my second generation i5 based laptop circa 2012 in f96-CE_2 and couldn't. Frugal install to a Samsung EVO 850 SSD.

In jammypup B3, I also failed to replicate it in the following boots:

1: Pristine, stock kernel, savefile created was a 64MB EXT2 unencrypted.
2: As above but with the 6.1.13 kernel swapped in.
3:Using my ydrv and the 6.1.13 kernel. I carry slightly modified rc.shutdown and shutdownconfig to allow toggling savefile mode vs nosavefile mode without changing the pupmode. They are based on the scripts in the current S15 and LxPupSc64, dated 12/1/22. As discussed elsewhere, sync is blocking in all current kernels but I am of the opinion that when dealing with newer flash and SSD devices with significant caches on board it 'isn't' and a 'sleep 0.1' after the final syncs in rc.shutdown gives cleaner error free saves to the savefile. I have a fsck check at the end of my rc.shutdown as a final sanity check and it generally confirms that. Got my knuckles slapped for posting that

No failures in any of the cases above.

At any rate no problem with savefile use on my hardware in B3. I never encrypt so won't jump into that.

@gyrog is the WoofCE specialist on PUP's init/shutdown. Maybe he can shed some light.

proebler wrote: ↑Tue Mar 07, 2023 9:06 am

First shutdown creating an encrypted save-file hangs/fails > alt+ctrl+backspace > exits to console prompt > reboot > hangs > alt+ctrl+del > un-mounts 'stray filesystem' , 'Terminated' > console prompt > reboot > hangs, making hard shutdown with power button necessary.
On next boot the save-file gets loaded but the password is not recognized, using 'c' (instead of password) goes to a console prompt.
Attempt to create an encrypted save-file abandoned.

First shutdown creating an unencrypted (normal) save-file hangs > alt+ctrl+del > console prompt > reboot > hangs , making hard shutdown with power button necessary.

Next boot the (normal) save-file gets loaded correctly.

I duplicated your results. But while waiting for the savefile to be created I looked in TaskManager and noticed that /usr/sbin/cryptsetup was stalled, using 0% cpu. I checked in Bionic64 and Fossa64-9.5 and they both used cryptsetup-1.7.5-musl_static_x86_64.pet which is available from PPM. So I killed cryptsetup in TaskManager which allowed me to reboot without saving. After rebooting I installed cryptsetup-1.7.5-musl_static_x86_64.pet and was able to create an encrypted savefile which worked for me when I rebooted. Will be in next build.

If you want to use encrypted savefile in the meantime, use PPM to download, not install, cryptsetup-1.7.5-musl_static_x86_64.pet. Follow the instructions, it doesn't seem happy about replacing already installed packages, and then click on it in /root. It will install and you should be good to go.

I didn't have any problem creating unencrypted savefile. Makes it hard to investigate, hopefully someone out there has the answer.

Good Luck, J

Claws Mail IMAP support is enabled now in B3. I was able to set up and use my gmail supported UMN account, not always the easiest go. I'll use it along side Sylpheed for now and adjust/get used to its look and feel.

My 2 cents says the non-encrypted savefile problem lies in rc.shutdown. It has changed considerably since fossapup 9.5, and a bit between F96-CE_1 and the one in upup B3 and F96-CE_2. The most recent changes have to do with storing asound.conf -seems innocuous- and shutting down connmand -a possibility- It's been quite a while but at one point we saw hangs due to network process shutdown order before in it. I have them all and have looked at the diffs but have no hardware where the failure occurs to test on.

Cheers,

I agree with Marv, that the hanging which I experienced has to do with the hardware.
I made an install to a FAT USB partition, crated a (normal) save-file and no hanging occurred during shutdown or reboot on an i3 machine.
On the core2 Elitebook the shutdown or reboot always hangs.

Another try to create an encrypted save-file, running shutdownconfig from terminal gives the following:

root@puppypc4519:~$ shutdownconfig
sda6,ext4,/vpup/JRB/upupsave.2fs
dc: can't open '67363660': No such file or directory
dc: can't open '1024': No such file or directory
dc: can't open '1280': No such file or directory
dc: can't open '2048': No such file or directory
dc: can't open '4096': No such file or directory
65536+0 records in
65536+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 0.275926 s, 243 MB/s
Creating a ext4 filesystem in upupsave_luks-xyz.4fs...

The file gets created but is no good.
Regards
proebler

proebler wrote: ↑Wed Mar 08, 2023 7:33 am

I agree with Marv, that the hanging which I experienced has to do with the hardware.
I made an install to a FAT USB partition, crated a (normal) save-file and no hanging occurred during shutdown or reboot on an i3 machine.
On the core2 Elitebook the shutdown or reboot always hangs.

Another try to create an encrypted save-file, running shutdownconfig from terminal gives the following:

root@puppypc4519:~$ shutdownconfig
sda6,ext4,/vpup/JRB/upupsave.2fs
dc: can't open '67363660': No such file or directory
dc: can't open '1024': No such file or directory
dc: can't open '1280': No such file or directory
dc: can't open '2048': No such file or directory
dc: can't open '4096': No such file or directory
65536+0 records in
65536+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 0.275926 s, 243 MB/s
Creating a ext4 filesystem in upupsave_luks-xyz.4fs...

The file gets created but is no good.
Regards
proebler

Let's give Marv's 2 cents a try. Here are the rc.shutdown files from both Bionic64 and Fossa64. Install one and try again, then repeat.

Edit: Just checking, did you install cryptsetup-1.7.5-musl_static_x86_64.pet before creating upupsave_luks-xyz.4fs?

@Marv - You know I was all worried about symlinking libs into /usr/lib/x86_64-linux-gnu and then removing dangling symlinks if the package was removed? It suddenly came to me, DOH!, that all I had to do was change LD_LIBRARY_PATH to:

/lib:/usr/lib:/root/my-applications/lib:/usr/local/lib/lib64:/usr/lib64

No symlinking required. Will rewrite /usr/local/petget/installpkg.sh accordingly.

Thank you jrb.

Changing to cryptsetup-1.7.5-musl_static_x86_64 works.
LUKS-encrypted save-files can then be created.

No luck with changing rc.shutdown however.
I had tried that before. The system always ends up at a command prompt that does not accept any commands.

The USB I am using has a number of puppies available, I tried the reboot sequence with Bionic64, BusterPup_8, SCpup32, Vanila Dpup_9.2.0 and CloudPup64 (Fossa). For all of them I get correct shutdown/reboot.
... so I guess the problem is with Jammy64pup on this particular hardware:
..................................................................
Computer Vendor Hewlett-Packard
Product Name HP EliteBook 6930p
Version F.16
Family 103C_5336AN
...............................................................
BIOS Vendor Hewlett-Packard
Version 68PCU Ver. F.16
Release Date 12/08/2009
Runtime Size 64 kB
ROM Size 1536 kB
Firmware 87.37
..............................................................
CPU Specifications:
Processor Name Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz
Socket Designation Intel(R) Genuine processor
Family Other
Manufacturer Intel(R) Corporation
Signature Type 0, Family 6, Model 23, Stepping 10
Voltage 1.6 V
External Clock 266 MHz
BogoMips 5053.73
Min/Max Speed 800/2534 MHz
Current Speed Core 0:2533 MHz, 1:1600 MHz
Core Count 2
64-bit capable Yes
Frequency governor ondemand
Freq. scaling driver acpi-cpufreq
.........................................................................

Regards
proebler

proebler wrote: ↑Thu Mar 09, 2023 10:35 am

No luck with changing rc.shutdown however.
I had tried that before. The system always ends up at a command prompt that does not accept any commands.

The USB I am using has a number of puppies available, I tried the reboot sequence with Bionic64, BusterPup_8, SCpup32, Vanila Dpup_9.2.0 and CloudPup64 (Fossa). For all of them I get correct shutdown/reboot.
... so I guess the problem is with Jammy64pup on this particular hardware:

The next release will be using @dimkr 's latest 5.15.78 kernel, maybe that will help?

Cheers, J

Edit: Did you try upup-22.04-jrb-A3.iso? It used peebee's 5.15.80 kernel with aufs and should be compatible with the zdrv's from the other puppies you mentioned. Just rename them to zdrv_upup_22.04.sfs and copy over their vmlinuz as well. Might be worth a try.

proebler wrote: ↑Thu Mar 09, 2023 10:35 am

Thank you jrb.

No luck with changing rc.shutdown however.
I had tried that before. The system always ends up at a command prompt that does not accept any commands.

Regards
proebler

Oh, that'n is a real stinker.

Try two quick things in stock jammypup if you will. 1: Prior to shutdown, use the task manager to kill connmand and connman-ui-gtk.bin.
and 2: Use the task manager to kill bluex, blueman, python3 and a related daemon roughly o??d. I am not running any bluez so I can't check the spelling of that daemon right now but it is related to bluez, sorry.

jrb wrote: ↑Thu Mar 09, 2023 12:57 pm

The next release will be using @dimkr 's latest 5.15.78 kernel, maybe that will help?

This kernel is old and has several known issues. https://github.com/puppylinux-woof-CE/w ... kernel.yml should be up to date and nearly identical to the Ubuntu 22.04 kernel.