Page 1 of 1

Airedale remaster compression

Posted: Sat Jul 08, 2023 5:22 pm
by geo_c

I'm going to remaster my 10GB of numbered upper_changes using the built in script from rc13.

Which compression would you use? I'm not thinking about xz, I'm trying to decide between gzip, lz4, and zstd.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 5:47 pm
by geo_c

Also, I'm wondering about the numbering, and if the firmware is included in the remaster.

I ask because the remaster script is numbering the .sfs as 01, and says it's the root file system. But does this include everything loaded at the time, including the firmware?

I'm guessing it does.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 5:53 pm
by geo_c

Well I aborted my first attempt because it pushed the CPU up to 95 degrees. This machine runs around 50-65 degrees normally.

cpu speeds were maxed.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 6:30 pm
by dancytron

Lz4 causes a lot less stress on the cpu than xz. edit: Fred says they are about the same. He's probably right..

On my old laptop that ran hot, I'd usually set the CPU on powersave when I did an xz compressed Debian Dog remaster.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 6:39 pm
by rockedge

@geo_c We'll have to revisit the KLV-remaster script(s). The default action is to copy the entire system as it is currently running. Meaning everything gets written into the remaster version including the loaded SFS's. The remaster tool renumbers the remaster SFS.
There is a way to remaster just the root file system (07KLV-airedale_rootfs.sfs) I beleive.

@fredx181 adapted his remaster program from DebianDog to work in KLV. I have not extensively tested the tool yet. Perhaps Fred can chime in or I can track down the relevant posts concerning the remaster tool and how it works and share the link.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 6:45 pm
by geo_c
dancytron wrote: Sat Jul 08, 2023 6:30 pm

Lz4 causes a lot less stress on the cpu than xz.

On my old laptop that ran hot, I'd usually set the CPU on powersave when I did an xz compressed Debian Dog remaster.

I was in fact using LZ4 compression.

I'm not sure where a cpu scaling tool is in KLV, not in the menus. I'll have to look into it.


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 7:17 pm
by Sofiya
geo_c wrote: Sat Jul 08, 2023 6:45 pm
dancytron wrote: Sat Jul 08, 2023 6:30 pm

Lz4 causes a lot less stress on the cpu than xz.

On my old laptop that ran hot, I'd usually set the CPU on powersave when I did an xz compressed Debian Dog remaster.

I was in fact using LZ4 compression.

I'm not sure where a cpu scaling tool is KLV, not in the menus. I'll have to look into it.

I can scale from the console, I don't know if it works in KLV

Code: Select all

echo "conservative" >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor  
echo "conservative" >/sys/devices/system/cpu/cpu1/cpufreq/scaling_governor 
echo "conservative" >/sys/devices/system/cpu/cpu2/cpufreq/scaling_governor 
echo "conservative" >/sys/devices/system/cpu/cpu3/cpufreq/scaling_governor  

Code: Select all

1. CPUFreq Governor
2. Governors In the Linux Kernel

2.1  performance
2.2  powersave
2.3  ondemand
2.4  conservative
2.5  schedutil

Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 7:40 pm
by williwaw

Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 8:03 pm
by dancytron

Here's a link to @fredx181 and I talking about the remaster program and excluding things with the config file.

I haven't had a chance to do anything with it since then.

https://forum.puppylinux.com/viewtopic. ... 683#p80683


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 8:22 pm
by fredx181
rockedge wrote:

There is a way to remaster just the root file system (07KLV-airedale_rootfs.sfs) I beleive.

Yes, see here:
https://forum.puppylinux.com/viewtopic. ... 929#p77929
The trick is to add

Code: Select all

usr/lib/modules/*
usr/lib/firmware/*

to the exclude config.
edit: at least it doesn't add the kernel modules and firmware to the remaster, but will add the files from inside 10gtkdialogGTK3_filemnt64.sfs (but no big deal IMO as it's very small).
Then it will still output to 01KLV-airedale_rootfs-remaster.sfs though (or whatever you choose as name), so, when finished, rename it to 07KLV-airedale_rootfs.sfs (replace it, but recommended to make backup first).
Don't know about CPU stress, I think it's about the same using LZ4 or XZ (both very high).


Re: Airedale remaster compression

Posted: Sat Jul 08, 2023 9:17 pm
by dancytron
fredx181 wrote: Sat Jul 08, 2023 8:22 pm
rockedge wrote:

There is a way to remaster just the root file system (07KLV-airedale_rootfs.sfs) I beleive.

Yes, see here:
https://forum.puppylinux.com/viewtopic. ... 929#p77929
The trick is to add

Code: Select all

usr/lib/modules/*
usr/lib/firmware/*

to the exclude config.
edit: at least it doesn't add the kernel modules and firmware to the remaster, but will add the files from inside 10gtkdialogGTK3_filemnt64.sfs (but no big deal IMO as it's very small).
Then it will still output to 01KLV-airedale_rootfs-remaster.sfs though (or whatever you choose as name), so, when finished, rename it to 07KLV-airedale_rootfs.sfs (replace it, but recommended to make backup first).
Don't know about CPU stress, I think it's about the same using LZ4 or XZ (both very high).

It's been a while, but iirc I looked at the contents of "10gtkdialogGTK3_filemnt64.sfs" and decided there was nothing that needed to be upgradable in there.


Re: Airedale remaster compression

Posted: Sun Jul 09, 2023 3:59 pm
by fredx181
dancytron wrote: Sat Jul 08, 2023 9:17 pm
fredx181 wrote: Sat Jul 08, 2023 8:22 pm
rockedge wrote:

There is a way to remaster just the root file system (07KLV-airedale_rootfs.sfs) I beleive.

Yes, see here:
https://forum.puppylinux.com/viewtopic. ... 929#p77929
The trick is to add

Code: Select all

usr/lib/modules/*
usr/lib/firmware/*

to the exclude config.
edit: at least it doesn't add the kernel modules and firmware to the remaster, but will add the files from inside 10gtkdialogGTK3_filemnt64.sfs (but no big deal IMO as it's very small).
Then it will still output to 01KLV-airedale_rootfs-remaster.sfs though (or whatever you choose as name), so, when finished, rename it to 07KLV-airedale_rootfs.sfs (replace it, but recommended to make backup first).
Don't know about CPU stress, I think it's about the same using LZ4 or XZ (both very high).

It's been a while, but iirc I looked at the contents of "10gtkdialogGTK3_filemnt64.sfs" and decided there was nothing that needed to be upgradable in there.

Yes, @rockedge wouldn't it be better if the files from "10gtkdialogGTK3_filemnt64.sfs" are part of the main rootsfs rather than separate SFS ? More simple setup that way IMO.
(but perhaps there's good reason not to, don't know)


Re: Airedale remaster compression

Posted: Sun Jul 09, 2023 4:21 pm
by rockedge

@fredx181 It could be merged into the rootfs.

At the time we left it out so when working on the mechanisms that 10gtkdialogGTK3_filemnt64.sfs supports, it would be easier to repeatedly, in short succession, change something and squash it quickly. So we could test out changes rapidly without having to deal directly with the KLV rootfs each time. There were many of those times.

Now though that it is hammered out and we haven't added anything or changed anything in it I think it could be easily added to /usr/local/bin

Looks like we could produce a sr2 that of course has a newer kernel and is completely upgraded and merge the 10gtkdialogGTK3_filemnt64.sfs into the rootfs.

I could probably do that today.