PaDS 1.1.7 - convert .deb, .pet, .sfs, .tazpkg, .txz, .tar.gz to .sfs (Update/Fix: 2020-10-04)

[taersh]

Because you don't use a savefile ... (I think you mentioned this in another thread)

Yes, that's true.
But in the beginning I used save files.
At these times I booted from USB flash drives.
I think I'm sure, the pup_ro1 was empty as well.

But other's may proof me wrong?

[one]

I just started upupef2004+4 and looked in /initrd/pup_ro1. It is a symbolic link to the savefolder and populated (NOT empty).

peace

[JASpup]

I believe pup_ro1 is the savefile. pup_rw layer updates pup_ro1 when we execute save2flash in PUPMODE 13.

My 1st attempt to make an adrv was from pup_rw. It did not work. I tried again in pup_ro1 and it works.

adrv are from mksquashfs, not PaDS, but the output file format is the same. This is a tangent subtopic (i.e., mksquashfs makes the PUPMODE 5 creating need for .sfs apps from PaDS).

If you have a pupsave you like, you can probably go .2fs -> .sfs and rename the file [system]adrv.sfs.

It is not a fool-proof process, but it works.

Taersh you use a different method for PUPMODE 5?

[gyrog]

If different .sfs files contain different versions of the same file, order makes a difference, in the aufs stack, and when combining them.

How does PaDS manager ordering?

[taersh]

.deb is extracted (using undeb_file) into the build directory of the new .sfs
.tar.gz & .gz is extracted (using tar -zxf) into the build directory of the new .sfs
.pet is extracted (using tar -xzvf) into the build directory of the new .sfs
.sfs is copied (using cp --preserve=all --remove-destination -f -r) into the build directory of the new .sfs
.tazpkg is extracted (using Extract-tazpkg) into the build directory of the new .sfs
.txz is extracted (using tar -Jxvf) into the build directory of the new .sfs

So, from alphabetical order of files existing files will be overwritten.
Extracted file from Z-Example overwrites existing file from previous A-Example.

[gyrog]

@taersh,
Thanks for the detailed reply.

Hmmm..., that's a slight difficulty if there is 'x.txt' in both the 'a.sfs' and the 'z.sfs', and you want the version that is in 'a.sfs'.
I wss hoping that there might be some facility to manually select the order, in case the alphabetic order of the filenames does not give the desired result.

Actually you could do it for .sfs files by renaming them, i.e. prepending a number.
But that might not work for .pet files. I know that if you rename a .pet and then click on it to install it, the install fails because the external filename doesn't match the internal directory name.
Then again on reading your response more closely, since you extract the files out of a .pet with tar, renaming .pet files might not be a problem.

I'll try to find time to give it a go.

[gyrog]

Yes, you can rename .sfs files as much as you like, i.e. prepend a number to force order.
But if you rename a .pet file by prepending a number, it goes crazy, on one occasion the .sfs was empty, on another it contained the whole directory it was supposed to save the .sfs to, and wrote the contents of the .pet as a directory.

Don't rename .pet file, and all seems to work as expected.

[taersh]

This is because the .pet file is a renamed .tar.gz file which is created from a .tar file which is created from the source directory.

[gyrog]

Ordering, it's all too obvious.
There's no need to rename files to produce a desired ordering.
The files in the right side list seem to be processed in the order they appear in the list, top to bottom.
The files in the one at the bottom take precedence.

So, imagine that the contents of these files are being "pushed" to an aufs stack, i.e. always added to the top.
Then the contents of each file will "clobber" any file with the same path and name, that is already in the stack.

The long and short of it is, if order matters because some of the container files have different versions of the same file, make sure that the right side list has the container files listed in reverse order, i.e. the container file that has the version of the file you want to use, must be below any other container files that have the same file.

I just did a little test with 2 sfs files containing the same text file, but the contents of each of these text files was different, so I could easily tell which sfs file took precedence.

[wizard]

dpup stretch 7.5rc5
-downloaded dvdstyler and dependencies from ppm (30 .deb files) and installed
-dvdstyler working

-boot stretch pfix=ram and create new save file, reboot
-installed PaDS-1.1.7.pet
-run pads and create DVDstyler_x86.sfs from the 30 .deb files
-load sfs on the fly
-dvdstyler does not open
-run dvdstyler in terminal, get following error

dvdstyler: error while loading shared libraries: libwx_gtk2u_html-3.0.so.0: cannot open shared object file: No such file or directory

-run pfind in system files for libwx_gtk2u_html-3.0.so.0, not found
-unpack DVDstyler_x86.sfs on another partition and file is there in /usr/lib/i386-linux-gnu/libwx_gtk2u_html-3.0.so.0
-open rox and navigate to usr/lib/i386-linux-gnu and it is a symlink to ./, looks like lots of files in the sfs did not load

Did I make a mistake somewhere?

Thanks
wizard

[taersh]

What PaDS option did you use,

- right-click action
- GUI

??

In DPup Stretch base .sfs

/usr/lib/i386-linux-gnu

is it a symbolic link or a real directory?

[wizard]

What PaDS option did you use,

- right-click action
- GUI

??

In DPup Stretch base .sfs

/usr/lib/i386-linux-gnu

is it a symbolic link or a real directory?

used GUI
its a link

Got it working as follows:
-in the installed version, found the missing files in /usr/lib
-unpacked DVDstyler_x86.sfs on another partition and moved the files from /usr/lib/i386-linux-gnu/ to /usr/lib
-rebuilt the DVDstyler_x86.sfs

The new sfs loads and runs. Curious why pads didn't put them in the correct directory..

Thanks
wizard

[taersh]

So, inside base .sfs it's a symbolic link and inside the DVD Styler .sfs it's a directory?
If so, the libs couldn't be found as the .sfs is below the layer of the base .sfs.
Though, PaDS should create the .sfs dependent on what it finds inside the system - related to the i386-linux-gnu being a directory or symbolic link.

Could you repeat this, but building the .sfs via right-click action (Combine to SFS) from all files stored into a single directory?

[taersh]

I downloaded DPup Stretch 7.5 .iso with 4.19 kernel.
It refuses to boot on my machine.
A few seconds after message Loading Kernel Modules... the screen turns black and nothing happens after that.
Tried it four times, no success.
Unable to do testings on DPup Stretch...
Please, try building the .sfs via right-click action (Combine to SFS) from all files stored into a single directory

[wizard]

So, inside base .sfs it's a symbolic link and inside the DVD Styler .sfs it's a directory?

YES

Created DVDstyler with the right click method and it works. Odd that the GUI fails.

I've only had booting problems with stretch 7.5 on my Dell E5510 in a virtual box 6 machine, it stalls at "loading kernel modules"

Thanks
wizard

[taersh]

Created DVDstyler with the right click method and it works. Odd that the GUI fails.

Ok, thanks.
Need to examine the GUI part of PaDS.
It's the part that had issues from the beginning.

[JASpup]

Troubleshooting:

What are the three empty boxes? For the third, what is "Category:" besides the Menu Category?
If I click the save icon in the middle right, the fields clear.

If I click save icon and Finished, the .desktop launcher is large and comprehensive, but it still won't run from the menu, probably because it is not including the --no-sandbox switch.

[JASpup]

A separate strange event:

I have two versions of this .sfs.

One was created in Xenial with only two packages required:
-rwxrwxrwx 1 root root 52541164 Mar 28 2018 chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_i386.deb
-rwxrwxrwx 1 root root 1044048 Mar 28 2018 chromium-codecs-ffmpeg-extra_65.0.3325.181-0ubuntu0.14.04.1_i386.deb

The Tahr version has 3 MORE dependencies (many other dependencies were installed):
-rwxrwxrwx 1 root root 52541164 Mar 28 2018 chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_i386.deb
-rwxrwxrwx 1 root root 1044048 Mar 28 2018 chromium-codecs-ffmpeg-extra_65.0.3325.181-0ubuntu0.14.04.1_i386.deb

+

-rwxrwxrwx 1 root root 1922936 Jul 1 2015 libgtk-3-0_3.10.8-0ubuntu1.6_i386.deb
-rwxrwxrwx 1 root root 166948 Jul 1 2015 libgtk-3-common_3.10.8-0ubuntu1.6_all.deb
-rwxrwxrwx 1 root root 88548 Oct 8 2018 libxkbcommon0_0.4.1-0ubuntu1.1_i386.deb

Yet the Xenial version is bigger:
-rwxrwxrwx 1 root root 78077952 May 18 19:45 32Chromium65-Tahr_core.sfs
-rwxrwxrwx 1 root root 82653184 Aug 21 10:40 32Chromium65-Xenial.sfs

[JASpup]

simultaneous .pet maker not working, but it does create a md5 file of zero bytes.

-rw-r--r-- 1 root root 0 Aug 21 12:02 32Chromium65-Tahr_FULL.pet-md5.txt

This is all version 1.1.4.

[jfw01]

fossaPup32
# ls -ld /usr/lib/i386-linux-gnu/
drwxr-xr-x 53 root root 60 Feb 8 11:11 /usr/lib/i386-linux-gnu/

[Booting off a CD with save files in a homebrew sfs on a flash drive. If there's an official plan for sfs (or other write-once) savefiles, I'm open to being navigated to a topic that covers it. Mine is unavailable in bionicPup32, because it depends on CONFIG_SHWH=Y in the kernel build.]

When I install:

Code: Select all

libxdo3_3.20160805.1-4_i386.deb
xdotool_3.20160805.1-4_i386.deb
PaDS-1.1.7.pet

I get a working PaDS.

When I:
- gather the same files into a temporary directory (eg /tmp/jfw01)
- start PaDS from the bottom-left menu
- navigate to that directory, using a child directory (eg /tmp/jfw01/tmp) as temporary space
- select the same 3 files
- create .sfs
I get:
- an error in /root/.cache/lxsession/LXDE/run.log:
/usr/local/pads-gui/padsgui: line 159: cd: PaDS-1.1.7: No such file or directory
- and an sfs with xdotool and without PaDS; specifically, no /usr/local/bin/pads_gui

Eliding some detail, installing it on a clean boot does not produce a menu entry or a usable PaDS.

Hacking in some debug,

Code: Select all

# diff -u padsgui.orig padsgui
--- padsgui.orig	2022-02-08 12:38:08.031028349 +1300
+++ padsgui	2022-02-08 12:33:47.587026918 +1300
@@ -156,6 +156,11 @@
 		esac
 		if [[ "$CHECKPET" != "tazpkg" && "$CHECKPET" != "sfs"  && "$CHECKPET" != *\.gz ]]; then
 			if [ "$doit" = "true" ]; then
+			    echo ==============DEBUG========
+			    pwd
+			    ls -lad *
+			    echo "PETDIR = '$PETDIR'"
+			    echo ==============================
 				cd $PETDIR
 				rm -f pet.specs
 				yes n 2>/dev/null | cp -ai * $DIR/$NAME 2>/dev/null
# 

the same code is run multiple times, and seems unpromising only with the .pet; this from run.log:

Code: Select all

==============DEBUG========
/tmp/jfw01/tmp
-rwxr-xr-x 1 root root  34892 Feb  8 12:34 libxdo3_3.20160805.1-4_i386.deb
drwxr-xr-x 3 root root     60 Jun 12  2018 libxdo3_3.20160805.1-4_i386.deb.files
-rw-r--r-- 1 root root 139264 Feb  8 12:12 PaDS-1.1.7-01.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:12 PaDS-1.1.7-01.sfs-md5.txt
-rw-r--r-- 1 root root   4096 Feb  8 12:20 PaDS-1.1.7-02.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:20 PaDS-1.1.7-02.sfs-md5.txt
-rw-r--r-- 1 root root 274432 Feb  8 12:23 pads-1.1.7-03.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:23 pads-1.1.7-03.sfs-md5.txt
drwxr-xr-x 2 root root     40 Feb  8 12:34 PaDS-1.1.7-04
PETDIR = 'libxdo3_3.20160805.1-4_i386.deb.files'
==============================
==============DEBUG========
/tmp/jfw01/tmp
-rw-r--r-- 1 root root 139264 Feb  8 12:12 PaDS-1.1.7-01.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:12 PaDS-1.1.7-01.sfs-md5.txt
-rw-r--r-- 1 root root   4096 Feb  8 12:20 PaDS-1.1.7-02.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:20 PaDS-1.1.7-02.sfs-md5.txt
-rw-r--r-- 1 root root 274432 Feb  8 12:23 pads-1.1.7-03.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:23 pads-1.1.7-03.sfs-md5.txt
drwxr-xr-x 3 root root     60 Feb  8 12:34 PaDS-1.1.7-04
-rwxr-xr-x 1 root root  56424 Feb  8 12:34 xdotool_3.20160805.1-4_i386.deb
drwxr-xr-x 3 root root     60 Jun 12  2018 xdotool_3.20160805.1-4_i386.deb.files
PETDIR = 'xdotool_3.20160805.1-4_i386.deb.files'
==============================
==============DEBUG========
/tmp/jfw01/tmp
-rw-r--r-- 1 root root 139264 Feb  8 12:12 PaDS-1.1.7-01.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:12 PaDS-1.1.7-01.sfs-md5.txt
-rw-r--r-- 1 root root   4096 Feb  8 12:20 PaDS-1.1.7-02.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:20 PaDS-1.1.7-02.sfs-md5.txt
-rw-r--r-- 1 root root 274432 Feb  8 12:23 pads-1.1.7-03.sfs
-rw-r--r-- 1 root root     67 Feb  8 12:23 pads-1.1.7-03.sfs-md5.txt
drwxr-xr-x 3 root root     60 Feb  8 12:34 PaDS-1.1.7-04
-rwxr-xr-x 1 root root  46720 Feb  8 12:34 PaDS-1.1.7.pet
PETDIR = 'PaDS-1.1.7'
==============================
/usr/local/pads-gui/padsgui: line 164: cd: PaDS-1.1.7: No such file or directory

I can continue searching for myself, for the code for which this is cleanup, but I suspect that other people know this code much better. Please may I have next steps.

[JASpup]

@jfw01 have you tried an older pup?

Sometimes I find that easier than wrestling with incompatibility if it's a distro-specific problem.

[jfw01]

Further to https://www.forum.puppylinux.com/viewto ... 349#p49349:

@JASpup, thanks; I think it's a PaDS(++) specific problem.

.pet files are being defined here:
https://wikka.puppylinux.com/PETs?redirect=no
as tar files with gzip compression, with a checksum appended, with no punctuation separating the checksum from the compressed content.

Some unknown process is producing .pet files (including PaDS) which are tar files with xz compression.

When PaDS processes those files:
- it includes the contents of the working directory into the sfs
- it excludes the contents of the conflicting .pet

Please find attached a listing of an sfs, showing the first problem.

rc2-ls.txt

Unpatched PaDS; difference between two successive created sfs's.

(2.02 KiB) Downloaded 60 times

Please find attached two patches to make PaDS 1.1.7 stop on this kind of error and accept xz compressed pet files.

03-error-diff.txt

Patch to /usr/local/pads_gui/error because PaDS now uses more tools.

(641 Bytes) Downloaded 66 times

09-padsgui-diff.txt

To /usr/local/pads_gui/padsgui to stop on errors from unpack failure, and to accept .tar.xz .pet files.

(1.52 KiB) Downloaded 60 times

The resulting PaDS is not entirely reassuring. I would attach the diff between two PaDS*.sfs, one made by the hand-patched version, and one made by the .sfs made from the hand-patched version. I think that I am going to hope that I fiddled while I was tired, and that these differences are within the acceptable range.

Please may I have help with the following questions?

1) Is there a definitive set of code for manipulating .pet files that I should invoke instead of rolling my own?

2) Where is the definitive public copy of PaDS, and what is the right way to submit a pull request?

3) Is there an appetite to have a conversation about the file format of .pet files? eg what range of compression should be supported; whether md5 is a good way of verifying authenticity; whether there should be a defined way of removing the checksum, so that further extraction can be expected to complete without error. If there is not a definitive set of .pet manipulating code, then that is a likely outcome of this branch, and I propose to put it into /etc/rc.d/functions_pet

Regards,
James.

[JASpup]

@jfw01 you are writing .xz created .pet files create faulty output?

My .sfs are usually all .deb, still hit-or-miss.

These hiccups highlight complexity.

PaDS works, though it demands specific conditions.

[amethyst]

For best results do this merging operation manually especially if the order of copying over the files is important because the wrong order could lead to an unsatisfactory result. I just use UExtract to extract every package and then copy the files over in the correct order before making the new sfs. I don't think one can choose in which order the packages are to merge (unless you rename the packages with a numeral pre-fix before using the application I suppose) with this application but I may be wrong (I don't use it).

[jfw01]

@JASpup,
my PaDS came from the .pet link here: https://www.forum.puppylinux.com/viewto ... 6355#p6355
at the beginning of this topic. Yes, my view is that .xz compression triggered the fault. I don't know what tool(s) created the .pet.

Do I assume that @taersh reads this thread? If not, what do you think the etiquette is around PMing them?

[TerryH]

@JASpup,
my PaDS came from the .pet link here: https://www.forum.puppylinux.com/viewto ... 6355#p6355
at the beginning of this topic. Yes, my view is that .xz compression triggered the fault. I don't know what tool(s) created the .pet.

Do I assume that @taersh reads this thread? If not, what do you think the etiquette is around PMing them?

taersh hasn't posted or possibly visited the forum since mid September 2021, he may see email notification, but possibly may not respond.

[JASpup]

Do I assume that @taersh reads this thread? If not, what do you think the etiquette is around PMing them?

The etiquette is The Golden Rule.

[mikeslr]

ditto what TerryH wrote.
Since the publication of PaDS 1.1.7 the only problem I've encountered is when trying to use a pet which wasn't properly created. When dir2pet is run it creates a pet.specs file. If you later make changes and run dir2pet again without having deleted the 'old' pet.specs file it will be written to the new pet. Reading that pet.specs file PaDS is unable to deconstruct the pet and fails.

@ jfw01. There are three thing you can do. If you're lazy or lack expertise like me, now that you know PaDS has a problem with xz compression:
(1) when you encounter that problem you can use UEXtract to deconstruct the problem application; then dir2pet or dir2sfs to repackage it before using it as a source under PaDS.
(2) analyze the code taersh used in creating PaDS, modify the code and post it as a 'fork' acknowledging taersh's role in creating the application.
(3) There are other applications available on the Forum for combining packages. But until now PaDS has been the easiest for me to use. But you can explore the others.

(2) is beyond my capabilities. But as a result of recent developments, the creation of a new or forked application to combine packages would be greatly appreciated. Among those recent developments are: (a) both Voidpup and KLV-xxx are based on Void linux, use packages in xbps format and may require taking into consideration of /media rather than /mnt folders, perhaps other structural concerns; (b) I have a vague recollection that debian and ubuntu are changing their packaging format; (c) maybe Slackware as well.

[jfw01]

Hi @mikeslr ,

Thank you for the survey of options.

(2) is half-done, for my particular corner of the symptom space, with the code in patches attached to this thread. I think that the right way to "post it as a 'fork' acknowledging taersh's role", if the definitive public copy is in something like github, is to have that platform fork it, then make the change and issue a pull request.

So I'm asking for help with finding the definitive public copy.

If I can get an actual delivery out of this process, then I am open to being mentioned into future issues (a),(b) and (c), to find out what I can achieve with them.

[mikeslr]

...
So I'm asking for help with finding the definitive public copy.
...

Being essentially a bash-script only 45.63 KiB, well within the size limitations of this Forum, PaDS 1.1.7 has only been published AFAIK on this Forum. Similar conditions obtained with regard to its predecessors, except that they were published on this Forum's predecessor. Prior to his passing, that forum was maintained by John Murgha and is now archived, http://oldforum.puppylinux.com/.

I don't have versions earlier than 1.1.4 and Search engines were unsuccessful in finding download links. The pinistall scripts of both 1.1.7 and 1.1.4 begin with the following:

#------------------------------------------------------------------------------
# PaDS PostInstall Script
# Some Actions to execute after installing the .pet file
# 2018-07-08 RSH for Puppy Linux [Emphasis supplied]
#------------------------------------------------------------------------------

Although he has not been active recently, RSH is a member of this forum now going under the handle taersh. Best to contact him directly.
In the absence of a response within a reasonable time, I would think under the circumstances, any fork providing attribution and expressing the same intention could be maintained anywhere by anyone.

[Sorry I can't be more definitive. Intellectual property law was not one of the specialties I practiced. The general 'Rule' of Law is to honor a party's intention so long as that does not violate 'the public interest'; that in the event of ambiguity or conflict to consider what 'the reasonable man' would understand or do; and that the 'free flow of information and knowledge' is 'in the public interest'.
Intellectual Property Law was 'forked' to grant rights in the implementations and expressions of ideas so as to encourage the publication of those implementations and expressions by providing an economic incentive to do so. Essentially the rights granted were to permit the party holding them to prohibit the republication without receiving fair compensation. But that's where the fork gets into messy details enabling specialists to make a living arguing about them.

I could detail several factors I think are relevant to the current situation but as I said I can't definitively hazard an opinion as to their significance. But my best guess is that under the circumstances, PaDS is to be considered a gift to Puppy Linux* and anyone using it in any way for the benefit of Puppy Linux has the right to do so.]

-=-=-=-=-
* Puppy Linux is an ambiguous term. A reasonable interpretation includes the Forum together with any and all who join it or make use of its content.