Puppy Linux Discussion Forum

I am looking to integrate the Void Linux's kernels firmware collection into the kernels made for Kennel Linux variants and ultimatley be able to produce a fdrv SFS for use with usrmerge and non-usrmerge Puppy Linux systems.

The different firmware groups (Intel, AMD for example) have separate packages for Void Linux so very hardware specific firmware collection can be used, but with a collection covering a lot of different devices in Void Linux can be around 880 M in size!

In woof-CE kernel-kit there seems to be a mechanism to filter the firmware and create smaller fdrv SFS of around 68-100 M in size. This filtering seems to based on the size of the firmware files but not sure yet on how this is actually achieved.

The plan is to take the Void Linux /usr/lib/firmware and trim it down to a smaller size fdrv SFS. A balanced fdrv SFS that can be used by huge kernels found in Puppy Linux and kernels huge or Void Linux type used in KLV's.

Not much yet discovered but here is something that perhaps gets things started.

lsmod | cut -d ' ' -f 1 | tr '_' '?' | xargs -I % find /lib/modules/ -name "%.*" | xargs -I % modinfo % | awk '$1 == "firmware:" { print $2}'

seems to do this:

• Lists loaded modules (keep in mind any builtin kernel module that requires a firmware is left-out here)

• extract first column, and replace '_' in names with single-char wildcard, as often filenames will instead use '-'

• find said module-file, and hand in to modinfo to derive a list of firmware blobs

While looking around I ran across a firmware package for a Linux kernel for drivers for automobile devices......
https://cbs.centos.org/koji/buildinfo?buildID=38935
Goes to show what OS is running in the car's computers.

The so-called 'trimming' in firmware_picker.sh is very problematic. Some drivers don't use a hardcoded, constant name like x.bin that appears in a firmware: line in modinfo output, but $something.bin (and format x or y into this template at runtime). firmware_picker.sh doesn't handle these cases correctly because it looks for exact file names inside drivers, but this runtime string formatting is very common in WiFi and GPU drivers, way more common than it used to be. Therefore, this trimming operation omits many firmware binaries and produces small fdrv with limited hardware support.

This is why I switched to a fdrv built from Debian's firmware packages (same set as Debian's live images) in my builds - it's bigger, probably has unused firmware files that can be removed, but I no longer hear complaints about missing firmware. I don't see a good way to fix firmware_picker.sh

@dimkr I did an experimental filtering of the Void Linux firmware packages using the kernel-kit firmware_picker.sh which reduced the overall size of 846 M as SFS to a 136 M fdrv SFS.

I simply put the pieces that firmware_picker.sh is looking for in the correct locations but using the Void Linux firmware package(s). First I used the FirstRib utilities to chroot the KLV-Airedale rootfs (uncompressed), then updated/upgraded the rootfs using xbps-install. This particular firstrib_rootfs I installed the xbps-install -Sy linux-firmware which places the firmware in a usrmerge position /usr/lib/firmware. Took the freshly created /usr/lib/firmware and moved it's content to /mnt/sdb1/woof-out_x86_64_x86_64_ubuntu_focal64/linux-firmware.

Then I copied from the uncompressed 00modules-6.8.8_1, which is created by FirstRib scripts that extract the vmlinuz and builds the 00modules from the firstrib_rootfs, the /usr/lib/modules to /mnt/sdb1/woof-out_x86_64_x86_64_ubuntu_focal64/kernel-kit/output/6.8.8_1/lib/modules. This setup now will produce a non-usrmerge version of an fdrv uncompressed -> /mnt/sdb1/woof-out_x86_64_x86_64_ubuntu_focal64/kernel-kit/zfirmware_workdir.

Now I ran the firmware_picker.sh script which suprisingly actual produced the directory /mnt/sdb1/woof-out_x86_64_x86_64_ubuntu_focal64/kernel-kit/zfirmware_workdir with which I produced the fdrv_fossapup64_9.6.sfs using PackIt and xz compression to build the SFS.

I tested the resulting fdrv_fossapup64_9.6.sfs 136 M in a F96-CE_4 and it the system booted and works including the eth0 and eth1 network cards.

Probably not the best fdrv but this can easily be made to a usrmerge or non-usermerge fdrv SFS.

So experiment worked....how well is not known at this time

Hi !
I don`t know if this is somehow useful in your cases ..........just want to remind on the so called "Zdrv_Cutter" (Topic in the old Forum).

Remove unneeded modules and firmware - Zdrv_Cutter

.
https://oldforum.puppylinux.com/viewtopic.php?t=51552

How to use Zdrv-cutter?

https://oldforum.puppylinux.com/viewtopic.php?t=84335

Thanks, backi, for rediscovering technosaurus's zdrv_cutter. I wonder to what extent it still works. I remembered it, but couldn't find it and asked about it a couple of times.

If I'm not mistaken it's objective was to produce a zdrv.sfs containing only those drivers and firmware required by the user's computer. So not what rockedge needs; but something which every Puppy user may want.

When technosaurus wrote it Puppys had not yet separated drivers and firmware into the different file-systems, zdrv.sfs and fdrv.sfs. Current structuring of Puppys may be different that the ones he wrote it for. And 'user-merge' may present additional problems.

Still, technosaurus' publications may be a valuable starting point from which a current application could be developed. Unfortunately, the coding employed is far above my paygrade.

If you want to write a new 'fdrv trimmer' that's better than kernel-kit's one, I recommend testing on a >= Skylake laptop, especially with an AMD or NVIDIA GPU. The missing firmware is often SOF (audio) firmware that's required since Skylake AFAIK, WiFi firmware (>=802.11ac) or GPU firmware.

Hello!

I've found some years earilier in a Gentoo linux forum thread, or bug report page that if we simply extracting linux firmware package is not enough. There is a WHENCE database file and copy-firmware.sh which is intended to generate symbolic links etc. over the previously mentioned database file. But if we would like to use that shell script on modern pups or under older ones we need to consider python3, rdfind, and realpath-FULL existence which are required to run the script correctly. The realpath link over the busybox binary won't work. As well another experience if we trim the generated firmware product of this method then we need to consider too that, may some hardwares won't work either. Will the output sfs around 500MB? Unfortunately yes, with xz compression as well, but at this point we need to rethink do we need complete hardware drivement or not. That's it...

syntax on (ext) linux filesystem:

./copy-firmware.sh ../../targetdir/usr/lib/firmware/

After we extracted the newest of these packages:

https://git.kernel.org/pub/scm/linux/ke ... mware.git/

(copy-firmware.sh is in the extracted firmware directory from the gzipped package next to the extracted raw firmware binaries.)

Here realpath is a symbolic link to realpath-FULL and the .BAK is the renamed one from which pointed to busybox. I packed it to an ydrv that is why it is marked as missing.

I hope it helps!

If you want to write a new 'fdrv trimmer' that's better than kernel-kit's one, I recommend testing on a >= Skylake laptop, especially with an AMD or NVIDIA GPU. The missing firmware is often SOF (audio) firmware that's required since Skylake AFAIK, WiFi firmware (>=802.11ac) or GPU firmware.

Unfortunately I don't have access to any of those machines.

I was able with Dr.Frankenstein methods put together a usable usrmerge and non-usrmerge 01firmware SFSand fdrv SFS running the woof-CE kernel-kit firmware_picker.sh against the linux-firmware directory from that the script downloads. This action reduced the 2.2 G firmware down to 380+ M which came to around 130 M as an SFS.

The non-usrmerge fdrv_fossapup64_9.6.sfs I am testing on a F96-CE_5-alpha3 which is in turn based on a woof-CE built Fossapup64, and so far on the machines I am booting this distro with this fdrv on has had the hardware working. Only the pulseaudio is not 100% and can't connect with pavucontrol but that was happening with other kernel and fdvr combinations as well.

I have tried a couple of runs using copy-firmware.sh using the WHENCE and WHENCE.ubuntu lists but did not get a successful result

mikeslr wrote: ↑Mon May 06, 2024 4:39 pm

Thanks, backi, for rediscovering technosaurus's zdrv_cutter. I wonder to what extent it still works. I remembered it, but couldn't find it and asked about it a couple of times.

If I'm not mistaken it's objective was to produce a zdrv.sfs containing only those drivers and firmware required by the user's computer. So not what rockedge needs; but something which every Puppy user may want.

When technosaurus wrote it Puppys had not yet separated drivers and firmware into the different file-systems, zdrv.sfs and fdrv.sfs. Current structuring of Puppys may be different that the ones he wrote it for. And 'user-merge' may present additional problems.

Still, technosaurus' publications may be a valuable starting point from which a current application could be developed. Unfortunately, the coding employed is far above my paygrade.

I just used technosaurus's zdrv_cutter-0.3 on Vanilladpup_9.3.14 on an eeepc 1005HA
The zdrv....sfs dropped from 46mb to 8mb
Rebooted to normal desktop with internet, sound and bluetooth all working.