Best way to compress mksquashfs?

[Sofiya]

example on a 279,7 МиБ file

Code: Select all

mksquashfs [source folder] [target file.sfs]  -comp xz -b 512k

final result 45,5 МиБ
-----------------------------------------------------------------------------------------------------------------------------------
example on a 279,7 МиБ file

Code: Select all

mksquashfs [source folder] [target file.sfs]  -b 1048576 -comp xz -Xdict-size 100%

final result 44,5 МиБ

as you can see the difference is small.
if there is a better way write about it here

[rockedge]

I add one more option: -noappend

Code: Select all

mksquashfs upper_changes 07KLV-airedale_rootfs.sfs  -b 1048576 -comp xz -Xdict-size 100% -noappend

[Sofiya]

-noappend
what function does it perform ?

[amethyst]

example on a 279,7 МиБ file

Code: Select all

mksquashfs [source folder] [target file.sfs]  -comp xz -b 512k

final result 45,5 МиБ
-----------------------------------------------------------------------------------------------------------------------------------
example on a 279,7 МиБ file

Code: Select all

mksquashfs [source folder] [target file.sfs]  -b 1048576 -comp xz -Xdict-size 100%

final result 44,5 МиБ

as you can see the difference is small.
if there is a better way write about it here

Change the bytes size to 1048576 in your first example or make the size 512k in your second example. You need to compare apple with apples. Your source folder is also too small in order to spot a significant difference. If you take a source folder of about 800MB, the difference in size will be about 40MB comparing 1MB xz compression with 128K compression for example. Not really all that significant. Given the extremely slow compression time of 1MB size xz compression, not really worth it. So, 128K size compression remains a good default for xz compression however xz compression (and decompression) is slow in general and with size not really an issue with large storing media, not really a good option for compression these days. But...always nice to see the max compression that can be achieved (not always the most practical though as mentioned).

[mikewalsh]

@Sofiya :-

If you're investigating ways of packing things as small as possible, you might also be interested in UPX:-

https://upx.github.io/

This is primarily for executables, although it'll work equally well for Appimages once they've been created, because the entire thing acts just LIKE one big executable. I've used it from time to time, and it works well.

There's versions available for 32-bit AND 64-bit, and they're statically-compiled, so can be used independently in any OS you like. It's easy enough to use it on executables in any package first, THEN you can pack the whole app whatever way you want.....

I first mentioned it on the old Forum, a couple of months before it went down for good:-

https://oldforum.puppylinux.com/viewtopic.php?t=118636

It's well worth a look.

Mike.

[dimkr]

-Xbcj x86 will reduce size a little.

The "best" combination of mksquashfs options depends on your use case - I use -comp zstd -Xcompression-level 19 because size is only ~10% bigger but decompression is much faster, but if you want minimum size even if this means extra RAM, CPU and time needed during decompression - go with xz.

[amethyst]

I'll go with gzip for lowest CPU usage, fast (enough) compression and decompression and decent compression rate. Also most compatible with any other devices and most distributions (like older Puppys for example). Gzip also very easy to use with simple, general commands. Wouldn't bother with zstd.

[dimkr]

zstd decompression can be much faster than gzip, depending on compression level and the data you're compressing. For binary files, it tends to be a win-win situation (better compression and faster decompression): with many big files, zstd level 3 beats gzip level 9 in size, compression speed, decompression speed and compressed size.

[amethyst]

Compatibility is key for me. Gzip is an excellent "allrounder". Not as resource hungry either. A bit like people still using and preferring mp3 and mp4 ( and h264 codec instead of h265 codec) formats. Works everywhere well enough.

[amethyst]

For fast compression I use gzip 16K block size. I use this for my adrv (which I use instead of a save file). 128K for the rest.

[Sofiya]

Based on this, you can simply use the Packit program

[rockedge]

-noappend
what function does it perform ?

-noappend if the destination file/device already contains a squashed file system, overwrite it, rather than append the new data to an existing file system

[rockedge]

KLV-Airedale-sr3 components compression comparison:
With gzip (512k)->

Screenshot(42).jpg (16.85 KiB) Viewed 1536 times

compressed with xz (1M) ->

Screenshot(41).jpg (22.61 KiB) Viewed 1536 times

This pushes the ISO well over 1 gigabyte which presents a problem with isohybrid function reliability and I do not want to exceed 950 mb for the ISO size.
Or perhaps a switch to the same methods as BarryK is using with image .img files. I find .img a pain to work with and find it limits installation methods.

[Sofiya]

to supplement and update in SFS
Found a use for this .. "maybe of course stupidity but oh well"
right-click on the sfs file click in the Update sfs window
unmount sfs add what we need, then click OK "in the Xdialog window" and it is mounted in a new sfs, rename " mynew.sfs " to the desired name.

in /usr/local/bin/update_sfs

Code: Select all

#!/bin/sh

unsquashfs -d utmp *.sfs
Xdialog --msgbox "Edit then click OK" 5 30
mksquashfs utmp mynew.sfs -b 1048576 -comp xz -Xdict-size 100% -noappend
sleep 1
rm -r utmp

in /root/.local/share/file-manager/actions/Update sfs ( for PCManFM )

Code: Select all

[Desktop Entry]
Type=Action
Tooltip=Update sfs
Name=Update sfs
Profiles=profile-zero;
Icon=/usr/share/pixmaps/sfs.png
[X-Action-Profile profile-zero]
MimeType=inode/directory
Exec=xterm -e update_sfs
Name=Default profile

[Sofiya]

KLV-Airedale-sr3 components compression comparison:
With gzip (512k)->
Screenshot(42).jpg

compressed with xz (1M) ->
Screenshot(41).jpg

essential difference

[amethyst]

XZ is the best for max compression by far but as slow as molasses at compressing and decompressing. Only really good for distribution purposes so you can pack for the smallest size possible for downloads. In all other scenarios it's a poor choice.

[amethyst]

KLV-Airedale-sr3 components compression comparison:
With gzip (512k)->
Screenshot(42).jpg

compressed with xz (1M) ->
Screenshot(41).jpg

essential difference

Would make a slight difference if you also compress the gzip with 1M block size but XZ will still win by far as far as compression size is concerned. It will take at least 10 times longer to do the operation though with XZ compression.

[rockedge]

Nice SFS edit tools with nicOS-Utility-Suite or a favorite from @fredx181 edit-sfs-1.2_0.noarch.xbps

I usually use the manual method of using Rox to mount an SFS then create a new directory somewhere and copy over the contents of the mounted SFS. Then use PackIt or command line to squash it again.

[Sofiya]

KLV-Airedale-sr3 components compression comparison:
With gzip (512k)->
Screenshot(42).jpg

compressed with xz (1M) ->
Screenshot(41).jpg

essential difference

Would make a slight difference if you also compress the gzip with 1M block size but XZ will still win by far as far as compression size is concerned. It will take at least 10 times longer to do the operation though with XZ compression.

Code: Select all

 mksquashfs KLA-OT2basesettings 10KLA-OT2basesettings.sfs  -b 1048576 -comp gzip -Xcompression-level 9 -noappend 

compression time is very fast.
10 MIB difference

[Sofiya]

That's all figured out, the utility from @fredx181 went with a bang with a small script correction
something similar to my creation, only more functional.

[Sofiya]

Prettier with gxmessage than with Xdialog viewtopic.php?p=95757#p95757

Code: Select all

#!/bin/sh

unsquashfs -d utmp *.sfs
gxmessage -title "$(gettext 'Update SFS')" -bg yellow -fn "regular 14" -wrap -center -button "$(gettext 'OK')" -default "$(gettext 'OK')"   " $(gettext 'Edit then click OK')"
mksquashfs utmp mynew.sfs -b 1048576 -comp xz -Xdict-size 100% -noappend
sleep 1
rm -r utmp

[rockedge]

That's all figured out, the utility from @fredx181 went with a bang with a small script correction
something similar to my creation, only more functional.

Can you please share the modifications to the script that you made? the xbps package as is doesn't work correctly in F96-CE_4 when I install it manually.
I'm interested in seeing what you corrected for KLA

[Sofiya]

That's all figured out, the utility from @fredx181 went with a bang with a small script correction
something similar to my creation, only more functional.

Can you please share the modifications to the script that you made? the xbps package as is doesn't work correctly in F96-CE_4 when I install it manually.
I'm interested in seeing what you corrected for KLA

For KLA, the script is native xbps package, I only have the strings translated, the buttons are in my language