KLV-Airedale-RT with full Real Time Kernel 6.1.38-rt13

[geo_c]

But the good news from my remaster fail is that the RT-kernel is OKAY! As far as we can tell...

[rockedge]

@geo_c have you ever tried working with the "pseudo full install" which is a trick I'll use to create the system then squash from there.

It might be a nice way to set up exactly the way you want. In this method you uncompressed and copy the contents of 07KLV-airedale_rootfs.sfs into a directory called /upper_changes
then remove the SFS and add a /directory to replace the sfs 07Dummy_rootfs. In this method everything is directly written to the rootfs file system in /upper_changes and all the configurations will remain persistent.

Then squash /upper_changes back into the rootfs SFS's and you can achieve the remaster this way. I often us the "pseudo full install" with Kennel Linux's.

Might be something you'd be interested in further exploring.

[geo_c]

@geo_c have you ever tried working with the "pseudo full install" which is a trick I'll use to create the system then squash from there.

It might be a nice way to set up exactly the way you want. In this method you uncompressed and copy the contents of 07KLV-airedale_rootfs.sfs into a directory called /upper_changes
then remove the SFS and add a /directory to replace the sfs 07Dummy_rootfs. In this method everything is directly written to the rootfs file system in /upper_changes and all the configurations will remain persistent.

Then squash /upper_changes back into the rootfs SFS's and you can achieve the remaster this way. I often us the "pseudo full install" with Kennel Linux's.

Might be something you'd be interested in further exploring.

Let's see if I have this straight.

Let's say I want to 'build" a system, I first unsquash the fresh 07KLV-airedale_rootfs.sfs and copy it's contents to /upper_changes, leaving a dummy directory called 07Dummy_rootfs, that way when I install applications, updates and configurations, it writes over the rootfs rather than whiting out files. Everything is in the single directory. When I'm satisfied with the results and want an immutable rootfs, just squash /upper_changes and rename it 07KLV-airedale_rootfs.sfs.

That sounds like something worth doing.

Maybe not as complicated as building a system using firstrib, which still feels a wee bit over my head as of yet. But I want to take the dive on that also.

[rockedge]

Let's see if I have this straight.

Yes. You got the gist of it.

[wiak]

Maybe not as complicated as building a system using firstrib, which still feels a wee bit over my head as of yet. But I want to take the dive on that also.

Well, it is part of 'firstrib' actually; it is the FR initrd that allows that pseudo install methodology. However I know what you mean - you were talking about the build script (build_firstrib_rootfs.sh), which is actually really easy to use too (I've just built about ten distros in the last hour during tests; if it was difficult I'd spend days doing that, but it is easy as pie once you try it and understand). For example, can be as easy as this:

Fetch the latest build_firstrib_rootfs.sh script (see new PM I sent you today - it changed again), and simply try a Fedora Rawhide build for fun by putting build_firstrib_rootfs.sh script into an empty directory where you want the build and making that executable. Then simple run the command (no f_plug required for this simple noX disto to begin with):

./build_firstrib_rootfs.sh fedora default amd64

Sit back and wait a few minutes for that build to complete and produce a firstrib_rootfs directory for you.

Then, rename that to 07firstrib_rootfs (no need to squashfs it), and

put a copy of FR skeleton initrd.gz in the same directory, and

a vmlinuz, 00modules.sfs and 01firmware.sfs from some other recent KL-build (assuming huge-kernel type as used in traditional Puppy Linux distros too).

REMEMBER: In firstrib-based/KL distros the names of the numbered files don't matter except for the two-digit number part, so you can use the likes of 00modules.sfs or 00zdrvXXX.sfs or simply 00.sfs (or uncompressed directory containing modules simply named as 00). Same goes for 07firstrib_rootfs/ can be squashed (but only if you wish) and renamed anything you like, whether uncompressed as a dir or as a sfs file, as long as name starts with 07 if that is the layer you want it at). Of course you must have two objects with same 2-digit start of filename/dir or confusion would reign...! (In fact to disable the use of a layer file or dir, I commonly simply rename with a D in front of the name, or an N for No... as long as first character not a digit it won't be used).

Finally, copy in executable script wd_grubconfig and run that (./wd_grubconfig) to get correct grub menu stanza(s) that you can boot the system with (user root passwd root).

[wiak]

@geo_c

Alternatively, per my immediately above post, can do the same thing to make a wee Ubuntu Jammy commandline starter distro. Again no f_plug required for bootable with root/root login per above post. Takes longer to build since also makes a tiny Void Linux host system that it leaves behind as firstrib_rootfsDBTSTRP (you can use or delete that one at the end according to your preference...). Here is the Ubuntu Jammy (no f_plug required) build command:

./build_firstrib_rootfs.sh ubuntu jammy amd64

Also you can optionally use mksquashfs on the end result if you so wish.

For example, the fedora default firstrib_rootfs turned out as 231MiB uncompressed. With a reasonably heavy gzip mksquashfs compression it reduced to 103MiB, but truly, for fast distro use that uses less RAM/CPU in practice, zstd is indeed definitely the way to go (doesn't matter it isn't quite the smallest - that is pretty much irrelevant). So in test I used:

mksquashfs firstrib_rootfs/ 07firstrib_rootfs.sfs -comp zstd -Xcompression-level 19

with end result being 98 MiB in size. But no need to compress with FirstRib otherwise anyway. Works fine with uncompressed dir numbered layers or a mix of that and numbered sfs addons. Usually I also use zstd compression level 19 for my own home builds (or uncompressed often too), but still experimenting between 15 to 19, but 19 does indeed work well enough. Compressed can actually end up a bit more responsive than uncompressed on some machines because the likes of zstd decompression is quicker than Input/Output of large uncompressed files on some slower media-based systems.

[wiak]

EDIT: One thing I forgot to mention. Thunar mounts the nvme SSD drive partition (/dev/nvme0n1p8) I'm building on as noexec,nodev. Debootstrap refuses to build if drive not mounted exec, dev, so prior to building a Debian-based distro on my machine I simply open terminal and remount the already mounted partition with command:

mount -i -o remount,exec,dev /dev/nvme0n1p8

You can always check how your partitions are mounted simply be entering the command: mount

OK, aside from major issue reports, that is me finished doing any major dev work for a few more months at least...

[geo_c]

Thanks @wiak,

I will eventually be trying out a build or two to get the feel for it. Everything I'm reading tells me that basically I will get a command line system, or desktop if added to a plug file (or maybe in the main firstrib script) and that sounds totally doable.

But when it comes to building a distro that rises to the level of a daily driver, where I'm concerned I'll fall short is in the area of things like installing CUPS, pulseaudio and that kind of stuff.

I remember before KLV-airedale had cups installed, I tried doing it myself, and I couldn't get it to run no matter how much I read about configuring it. Eventually @rockedge came to the rescue and built a new beta with CUPS running OOTB.

I could probably get a decent system going using the simple network tools like nmtui, but maybe you see where I start to feel in a little over my head. Of course one only learns by doing in the long run.

Last night, I did one more experiment with long collected KLV-airedale upper_changes that I had remastered. Instead of using the remaster, I copied the uncompressed collected XXupper_changes from my archive, which had worked well updating the rootfs versions through to sr2.

I had that whole system archived on USB in it's exact state from a working hard drive install. Booted it, checked to see that it was all working properly, then did the latest Void Update, rebooted and got the same Power Manager failure. So that confirmed that indeed it wasn't @fredx181's KLV-remaster script that was likely the culprit, but instead something to do with dropping in newer 07KLV-root.sfs's underneath so many XXupper_changes, I had started with RC3, dropped in RC6, then RC13, and finally sr2.

The void update broke the power manager going all the way back to RC13. So the problem actually occurred way earlier than KLV-Airedale-RT.

So I decided to start over and use @rockedge's suggestion of the psuedo full install, because I really like the RT-kernel. And of course that is working properly after running the latest void update, which again updated dbus libraries.

The takeaway is that I like the builds you, @Sofiya, and @rockedge make available, but I can't just expect to update on a previous XXupper_changes without running into potential problems.

And that's why being able to have a build of my own loaded with my favorite applications starts to look appealing.

[geo_c]

I'm working on this fresh psuedo full install of KLV-Airedale-RT. So I wanted to get my printer working. Installed hplip Turned off the firewall, ran the cups browser interface, doesn't see the printer. After several tries, decided to install the hp-toolbox, gui utilities for hp printers, and ran it from the desktop file. Nothing happened. So I looked in the desktop file, grabbed the command line: hp-toolbox and ran it from a terminal which returned this:

As far as I can tell it failed to open because of a dbus error.

The only thing I installed on this system is xfe and hplip, I ran a system update last night which updated dbus-libs.

So I don't know. Maybe there are issues at play here.

EDIT: There were a few packages for python3 -dbus and Qt5-dbus involved. I've got to wonder if those packages are behind the curve, because I noticed in OctoXbps that hplip is orphaned. Perhaps it's just that package that isn't working anymore.

[rcrsn51]

The hplip-print-scan-klv-3.19.8_2.x86_64.xbps package I made last year for the KLV project still works with this version.

[rockedge]

@geo_c

The hplip-print-scan-klv-3.19.8_2.x86_64.xbps package I made last year for the KLV project still works with this version.

I have had good success with using this package. Definitely give it a try. I am testing on as many different system variations as I have available to track down this dbus issue.

So how is a fresh install of KLV RT working after the rolling updates? Are the problems appearing with dbus after the system update on fresh installs with a new upper_changes?

Trying to determine if the layers of upper_changes combined in the remaster are the cause or do we have a global problem?

[geo_c]

The hplip-print-scan-klv-3.19.8_2.x86_64.xbps package I made last year for the KLV project still works with this version.

I'll check that out as I soon as I locate it on the forum.

@geo_c
So how is a fresh install of KLV RT working after the rolling updates? Are the problems appearing with dbus after the system update on fresh installs with a new upper_changes?

No problem so far, I'm using the psuedo full install now, and the update didn't break the power manager. Everything else appears to be working normally. I just finished installing Ardour, and that's the final baseline package for me. I have all my portables linked and running, Rox configs and icons, all the stuff I normally throw into a system, just a few more tweaks.

Also I uninstalled the Void repo hplip packages. Seemed to be a smooth uninstall. Don't think it broke anything.

In fact, I spent a couple hours today, and I have this KLV-Airedale-RT just about to the state that I had it with those 6 months of XXupper_changes.

And of course all in the one upper_changes directory. So I'm soon ready to resquash.

I might just not install hplip in the 07rootfs squash, instead I can try those out on a proper upper_changes after squashing, that way I can always roll back, when it's time to update that package in the future.

I don't see that KLV-Airdale-RT is missing anything. Seems full featured and fast.

[rockedge]

The hplip-print-scan-klv-3.19.8_2.x86_64.xbps package I made last year for the KLV project still works with this version.

I'll check that out as I soon as I locate it on the forum.

Here it is hplip-print-scan-klv-3.19.8_2.x86_64.xbps

[geo_c]

I'm ready to re-squash this canine-linux:

[rockedge]

Those look really nice

I use PackIt sometimes to squash but most of the time from a terminal ->

Code: Select all

mksquashfs upper_changes 07KLV_rootfs-zstd.sfs  -b 1048576 -comp zstd -noappend -Xcompression-level 9

mksquashfs upper_changes 07KLV-airedale_rootfs.sfs  -b 1048576 -comp xz -Xdict-size 100% -noappend

mksquashfs upper_changes 07KLV-airedale_rootfs.sfs  -b 1048576 -comp gz -Xdict-size 100% -noappend

[geo_c]

Those look really nice

I use PackIt sometimes to squash but most of the time from a terminal ->

That's super helpful, I was just about to ask a question about which way to squash it.

[geo_c]

Wow!

It appears to have worked. I used xz compression.

I'll poke around and see how everything is running. This is exciting. Why didn't I understand the psuedo full install before now?

[rockedge]

@geo_c before you squash,

1. look in /upper_changes/var/cache/xbps and /upper_changes/var/cache/fontconfig then delete any files in them to ensure they are empty.

2. check /upper_changes/mnt and ensure the directory is empty.

3. check /upper_changes/root/.bash_history and empty it and save.

Also possibly remove any Mozilla cache and or profile for Firefox or just don't use it at all before the squash

These steps could/will save a lot of megabytes. Now squash xz

[geo_c]

@geo_c before you squash,

So I cut down about 400MB by clearing the xbps cache and the browser caches, did as you said and erased the bash history, and the total size is 1.1G with ardour and a lot of plugins, rox icons, etc...

This method solves a lot of problems for me.

With a real time kernel and working copy of Ardour, it's a fine distro for audio.

Now it's personalized, clean, and squashed, so I don't really have to worry about much in the way of breaking it.

Also it's now possible to do what wiak described awhile back, that is I can install hplip first thing, and if it works, shutdown and squash those upper_changes into a read-only layer. effectively having an application sfs to add and remove from the system.

I just have to remember how to install a local xbps package.

Is it simply: xbps-install /path-to/my-package.xbps ?

[rockedge]

you can use OctoXBPS to install local packages or there is the right-click menu in Thunar or using inst-xbps on the command line using the script @fredx181 set up. There is an extra step to register the package in an index for xbps, then use xbps-install with a option switch.....a bit cumbersome so Fred wrote a script that takes care of the whole process.

[geo_c]

you can use OctoXBPS to install local packages or there is the right-click menu in Thunar or using inst-xbps on the command line using the script @fredx181 set up. There is an extra step to register the package in an index for xbps, then use xbps-install with a option switch.....a bit cumbersome so Fred wrote a script that takes care of the whole process.

Well, I installed that local package using OctoXbps, and it installed successfully, but still not seeing the printer.

I hate CUPS. Even when I had printers installed on fosspup9.5, I still had to reinstall them periodically, for some reason they would just stop working.

[wiak]

Crap, it's my remastered file system. Well, I was so happy with it.

I guess it's back to the drawing board on upper_changes.

I would say that it is always a matter of luck if an old (series) of upper_changes worked with a new KLV release. Logically, a new release will have newest versions of packages and maybe some additional packages that weren't used in previous release. The effect of that is that dpkg/apt package manager database will be completely different than what was in earlier pristine KLV.

The upper_changes will overwrite and same dir/files of course, but still these upper_changes addons can't know about anything different in the new KLV release - hence, one way or the other the dpkg/apt database will inevitably not be correct. System may appear to work (maybe always) but it certainly 'broken' in that usage really. So, sadly, no, a series of upper_changes files are only really useful/valid if you keep the same underlying pristine KLV version.

As for CUPS... sometimes that has worked perfectly on some distros for some time, but then again sometimes I have failed to get it to work at all. I also hate it - best chance overall, as I see it, is when upstream distro have taken careful effort to ensure printing works such that I can trust the CUPS installation + configs they provide. Unfortunately, even then I run into problems. My current latest full install of Linux Mint does print fine most of the time... but sometimes the system simply fails to find the printer. Previous Zorin CUPS seemed to work fine for me, but only via wifi (usb connection usually claimed issues and took ages printing, if at all). I therefore agree with you overall about CUPS - I pretty much hate it for these issues I've often experienced with it. rsrsn51's slim CUPS package for KLV-Airedale did also work fine for me when I tried it long time back using an old HP deskjet 2540P though (pretty old rubbish printer, I think, but does support Airprint or whatever it is called).

[geo_c]

@wiak As far as I can tell CUPS is working properly on KLV-Airedale-RT, it may just be the hplip that's not working, but as I referenced earlier, all that configuring cups is a headache for me, and I'm somewhat lost, although I do remember getting a network printer configured once by using network printer ip address rather than automatic network discovery.

On another note, as I built this psuedo full install and squashed it, I made careful notes of everything I did, so that maybe I can create a build script one day for various distros, where I boot into a clean install run the script, do a few manual configs in the desktop and squash. Also using first rib, I'll know exactly what packages to include, etc...

Everytime I update upper_changes from here on out I'll have a new entry in the log which looks like this:

[rockedge]

Printing with CUPS has always been the adventure. I've had the same experience that @wiak talks about. Sometimes works well and even though setting up the same printer on the same distro on another machine similar to the first and regardless of the level of effort, fail to get printing to work. No rhyme or reason as to why not.

Can be disheartening. Though so far I have had success configuring a Canon MG2500 and a HP F4500 via USB connection. But not so with network connections and can't pinpoint why not. F96-CE_4 I have connected to network printers through the LAN/WLAN fairly regularly and successfully via USB.

The network printer though is run through a Puppy Linux Tahr-6.0.5 set up with a very old CUPS and has an avahi daemon going. It makes a Canon MG2500 act like an AirPrint capable printer. Works well for printing from Android or iPhone/iPad and the various desktop/laptop/server computers.

But so far have had spotty results with KLV connecting to it. According to the debug error.log the reasons look fixable.

[geo_c]

Printing with CUPS has always been the adventure. But so far have had spotty results with KLV connecting to it. According to the debug error.log the reasons look fixable.

So to follow up on the KLV printing:

I have had fantastic results printing in KLV-airedale all the way to sr2. In fact it prints so much faster than fossapup that it's mind boggling.

I'm working on two machines, and on this machine I installed (edit)cups hplip from the void repo because it's a more recent version.

It appears to be installed correctly because all the drivers appear in the cups dialogs, but I simply can't seem to connect to the printer.

What could be different from sr2, that maybe I could find and configure?

[rockedge]

@geo_c ahhh so with sr1 you printed without a real problem, but sr2 seems to have the exact same setup, but will not connect to the printer.

The Gods taunting us. We'll need to dig deeper and compare.

[geo_c]

@geo_c ahhh so with sr1 you printed without a real problem, but sr2 seems to have the exact same setup, but will not connect to the printer.

The Gods taunting us. We'll need to dig deeper and compare.

Well not exactly, sr2 with the old kernel printed great. It's this version that's not seeing the network printer. So as I understood, KLV-airedale-RT is just sr2 with a different kernel, of course there have been some big system updates since then.

I liked the standard sr2, and I have a machine that doesn't like the real time kernel as much, so I was thinking of doing a psuedo full install of sr2 and doing the same thing I did with RT-13.

Whether a fresh, system-updated sr2 with a new install of hplip will see my network printer or not remains to be seen.

EDIT Come to think of it RT-13 using my collected XXupper_changes was seeing the printer just fine also. And printing fast. So I don't know what's different about starting with the fresh RT-13 files and new upper_changes.

The only thing I did in the old collected system was install hplip from the void repo and run it. It worked.

[geo_c]

...Come to think of it RT-13 using my collected XXupper_changes was seeing the printer just fine also. And printing fast. So I don't know what's different about starting with the fresh RT-13 files and new upper_changes.

The only thing I did in the old collected system was install hplip from the void repo and run it. It worked.

Makes me think that something that was setup in my old XXupper_changes, like permissions or ports, which is not present on a fresh install, the old XXupper_changes might have written something that isn't in the RT rootfs.

EDIT: Now I remember:

I had remastered the sr2-rootfs with my old XXupper_changes, so I only dropped the RT-kernel and modules, inits, etc into the mix, that's why it kept seeing my printer. When I finally converted to the KLV-Airedale-RT-rootfs from a clean install with fresh upper_changes and hplip, the printer stopped showing up.

[williwaw]

I've just built about ten distros in the last hour during tests............
For example, can be as easy as this:

Fetch the latest build_firstrib_rootfs.sh script (see new PM I sent you today - it changed again),
@wiak where can I find the updated script, have you posted it somewhere ?

and simply try a Fedora Rawhide build for fun by putting build_firstrib_rootfs.sh script into an empty directory where you want the build and making that executable. Then simple run the command (no f_plug required for this simple noX disto to begin with):

./build_firstrib_rootfs.sh fedora default amd64 is this arguement the iso or perhaps a directory of the extracted iso or perhaps the / filesystem of an installed fedora on another partition? or is the script run from a running target distro FS as with the method viewtopic.php?t=9256

Sit back and wait a few minutes for that build to complete and produce a firstrib_rootfs directory for you.

Then, rename that to 07firstrib_rootfs (no need to squashfs it), and

put a copy of FR skeleton initrd.gz in the same directory, and

a vmlinuz, 00modules.sfs and 01firmware.sfs from some other recent KL-build (assuming huge-kernel type as used in traditional Puppy Linux distros too).

REMEMBER: In firstrib-based/KL distros the names of the numbered files don't matter except for the two-digit number part, so you can use the likes of 00modules.sfs or 00zdrvXXX.sfs or simply 00.sfs (or uncompressed directory containing modules simply named as 00). Same goes for 07firstrib_rootfs/ can be squashed (but only if you wish) and renamed anything you like, whether uncompressed as a dir or as a sfs file, as long as name starts with 07 if that is the layer you want it at). Of course you must have two objects with same 2-digit start of filename/dir or confusion would reign...! (In fact to disable the use of a layer file or dir, I commonly simply rename with a D in front of the name, or an N for No... as long as first character not a digit it won't be used).

Finally, copy in executable script wd_grubconfig and run that (./wd_grubconfig) to get correct grub menu stanza(s) that you can boot the system with (user root passwd root).

Are there any limitations on choices for target OS's? Can I find the script to do any OS as described above in the viewtopic.php?t=9337 thread?
Tx

[wiak]

Are there any limitations on choices for target OS's? Can I find the script to do any OS as described above in the viewtopic.php?t=9337 thread?
Tx

It is the same build_firstrib_rootfs.sh script as in that above link; by itself it builds very basic (commandline only) root filesystems for any of Void Linux, Arch Linux, Debian, Devuan, Ubuntu, or Fedora Rawhide.

You can download that build script per above link via:

Code: Select all

wget -c https://gitlab.com/firstrib/firstrib/-/raw/master/latest/build_system/build_firstrib_rootfs.sh && chmod +x build_firstrib_rootfs.sh

and make the script executable (chmod +x) prior to using it per below.

The build root filesystem (by default called firstrib_rootfs), which is just a directory the above build script will automatically create containing a minimal linux filesystem of the chosen distro, can then be booted directly with FirstRib initrd.gz (also downloadable per above thread link) along with a Puppy-huge-type kernel, modules sfs, and firmware sfs (the modules and firmware addons have filenames starting with digits 00 and 01 respectively, which tells FR initrd.gz required overlay layers for these special addons).

Code: Select all

wget -c https://gitlab.com/firstrib/firstrib/-/raw/master/latest/build_system/initrd-latest.gz -O initrd.gz  # FR skeleton initrd.gz

To build a more complete distro firstrib_rootfs for any of these distro types you need to create a text file f_00 plugin that mainly just contains package manager install commands for the likes of Xorg or whatever and adds users and sets passwords and whatever you put in it.
@rockedge makes most of the KL-Void Linux builds so he can help you with how to make suitable f_00 plug for building these (he has some posts on the forum about howto make these already, with example f_ plugin shell-script code). For Arch Linux KL builds, Sofiya does most of that nowadays (I used to, but too busy these days). For Debian-based distros, well, no-one has been writing extra f_00 plugs for expanding the basic debootstrap builds of these yet. Obviously any DebianDog maker knows a lot about expanding the basic minimal debootstrap creation since DebianDog distros also start from a debootstrap minimal build and then add to that via apt package managere. Generally it is pretty easy though - for Debian-based distros just a matter of installing Xorg say, and XFCE and the distro will pretty much work... and with official reliable upstream package management as well as sfs addon layers support from the overlayfs-based FR initrd.gz (or however it is compressed).

Ability to build Void Linux based distros was how FirstRib build scripts started and it is possible to make really small distros via Void Linux because that doesn't start with any debootstrap full commandline-based root filesystem, but instead simply starts with busybox and official Void Linux package manager (xbps). Nevertheless, it is surprisingly easy to write a build f_ plugin for Void Linux distros too that will provide Xorg or Wayland plus, say XFCE or GNOME (or some other Desktop Manager or environment) just via a few xbps install commands in the required plugin, but with Void Linux builds you really have very fine-grained control of what is the end build result.

In summary all FirstRib builds are created using a command of the following form in an empty build directory:

Code: Select all

./build_firstrib_rootfs.sh <distro-type> <distro-variant> <amd64|i686> f_00_plugin_name

# though nowadays I think only Void Linux provides for i686 (32bit) builds? If the distro doesn't have a 'variant' just type 'default' (anything will do there in that case actually).

For example:

Code: Select all

./build_firstrib_rootfs.sh ubuntu jammy amd64 f_plugin_named_whatever

or for more detail simply get help with the command via:

Code: Select all

./build_firstrib_rootfs.sh --help

I recommend simply building that recent Fedora Rawhide and reading the simple f_00 plugin and new addons_f.plug scripts to see how the whole process works: viewtopic.php?p=95981#p95981