Puppy Linux Discussion Forum

I successfully installed Pkg-cli into KLV-Airedale-beta17 running in a QEMU VM, then used Puppy's Pkg-cli to update the Puppy repos and then installed a PET into KLV-Airedale directly by using the Pkg-cli GUI supplied in Puppy Linux.

The next tests will be with pkg doing some more intensive PET package management. xdeb has been the utitltiy to install .deb files but experimentation with pkg to install other various package formats. Eventually I might try to write a script that will sync the package management systems.

And here is the urxvtControl PET running (rxvt-unicode was installed with xbps-install GUI OctoXBPS).

That's great rockedge. A very interesting development done so quickly. It is also very encouraging for Pkg development and efforts to sync with native package managers.

I vote for this PKG2 to become THE Package Manager in KLV as well as any DOGs that find it appealing. This technology will mature.

If so, if will need aPKG2 User Guide with simple examples to support user understanding to avoid constant questions.

@Clarity Since KLV is based on Void Linux, which is not Ubuntu or Debian at all and Void Linux uses runit and the excellent XBPS package manager, I think it would take major coding modification to have Pkg2 as KLV-Airedale's sole package manager.

rockedge wrote: Fri Aug 19, 2022 7:54 pm

@Clarity Since KLV is based on Void Linux, which is not Ubuntu or Debian at all and Void Linux uses runit and the excellent XBPS package manager, I think it would take major coding modification to have Pkg2 as KLV-Airedale's sole package manager.

Yes, runit is a big advantage on KLV-airedale compared to sysVinit, and xbps is a very fast and efficient package manager for all packages from Void (managing all the runit related pre and post scripts too) so it only makes sense that that is number one package manager for this distro. However, great to have Pkg2 as alternative extra since it potentially opens up possibility to use packages from other distros including Puppy repos and is as rockedge suggested a neat solution for installation of the many simple dotpet packages around. Yes, there are limitations and risks that are worth bearing in mind if you don't want to mess up your distro; in particular the move most distros have made to putting most all bins in /usr/bin and most all libs in /usr/bin, whereas many old dotpets might sometimes use /bin and /lib so will break... For future devs of dotpets though it would be good to use the likes of /usr/local hierarchy or perhaps /opt such that it becomes more easily possible to avoid clashes between xbps (Pkg manager main) and Pkg2 (package manager alien/alternative). Despite the lack of 'purity' and risks involved I think it is a great flexible situation to have such an arrangement and in true Puppy-style in terms of 'freedom/anarchy/user-chosen-risks' - we don't want our hands held too much here and if we do break the system ever, we learn from it, and can easily build it again anyway...

I'm also really interested in the idea of having a standalone package manager. In fact I originally developed firstrib/weedog to use Void's xbps precisely because a standalone static version of that could be used with busybox - and build what you like from that combination - (the WDL initrd being originally designed to boot it) - and most of that Void-based development was taken on and over by rockedge (I became busy with Arch linux primarily, as well as maintaining the build scripts, including the WDL initrd, and adding the weedogit script as an additional boot-most-any-distro/root-filesystem facility).

But if Pkg2 can, as it seems, also be used as a standalone package manager then that opens up another avenue for building simple WeeDogs via busybox + Pkg2 and again, via the well-established WeeDogLinux build_firstrib via one (or more if you wish) simple text file plugin(s), a method to build a distro of any complexity anyone wants. i.e. its great for KLV-Airedale, which is Void xbps based to also have Pkg2 functionality, but it also becomes similarly possible to build a KL distro in similar fashion that starts out with Pkg2 as it only or primary package manager.

I expect some core programs might be missing in that busybox/Pkg2-only scenario, but I imagine that matter could be addressed (for example, I remember hearing that original pkg depended on core-utils, which depends on other libs, whereas a change to make it work via busybox alone would be very beneficial for the scenario I am imagining). The end result is a very simple build distro system that (just like now for KLV-Airedale) uses only two simple scripts (and no complex skeleton root filesystem full of many additional secondary scripts needing to be downloaded too) plus a single build plugin (list of commands in text); i.e. one single script (with list of extra build commands provided by single text file plugin) builds the root filesystem (that is build_firstrib_rootfs) and the second script builds the initrd for that (being build_weedog_initrd).

Though as has been shown elsewhere on the forum the functionality of that second script can instead be done via a skeleton build of WDL initrd and (optionally) assembled automatically by weedogit.sh (which was created originally as a convenient way to try out frugal installs of other distros, but can also use any firstrib_rootfs build as its root filesystem component).

We have ended up with a very very simple build system that does not involve scripts called multiple other scripts or needing to download complex-designed skeleton root-filesystem as a pre-requisite. The two build scripts are the whole thing, so pretty easy to read/understand and learn to use them via a few build plugin exemplars - no gurus required, though of course the more you do such builds the more expert you become at it. It really is a bit like using LEGO and leaves the overall design up to the user/creator of the plugins - end result is you can really design any distro you like and/or simply steal the root-filesystem from elsewhere , such as mainline distros described at Distrowatch (and then, optionally, add the special KLV flexible sfs/frugal-install load/save-utilities being developed for KLV-Airedale team to any one of them - also allows you to modify any such mainline distro root filesystem, if you have the skill and wish, and then remaster it to any shape and form you might wish - and if you script the modifications you don't have to worry about restricted licensing issues regarding distribution of ready-made isos - just supply the build plugin script...).

Hi just some thoughts.

The problem with having two package managers is that 'the left-hand doesn't know what the right hand is doing'.
I solved that problem under Puppys by planning in advance and creating 'alphabet drives' i.e. advr.sfs, ydrv.sfs. The nicOS-utility-Suite's Save2SFS makes it easy to create and update those. But PaDS can also be used.

Planning in advance is required when using Save2SFS as it will incorporate into an adrv or ydrv the contents of the SaveFile/Folder and applications/files in RAM not yet Saved. Such advanced planning is not always possible. [Booting pfix=ram is the inconvenient work-around].

The 'Main' Package manager doesn't need to know the contents of an 'alphabet drive' as it can't modify one anyway. As long as the application which creates an 'alphabet drive' can handle an 'alien packaging technique' any alien package can provide the source files for an 'alphabet drive'; and the Main Package Manager doesn't have to have that capability.

The problem with standard Puppys is that their initrd only looks for and will only use a limited number of 'alphabet' sfs, the ones I mentioned and fdrv.sfs and zdrv.sfs. I think editing initrd removes that limitation. Someone --taersh?-- had some posts about accomplishing that by editing initrd. Quickpup64 certainly doesn't have that limitation. On boot-up it copies bdrv.sfs, gdrv.sfs and xdrv.sfs into RAM.

Overlays doesn't have that limitation. But IIRC, overlays can't use load-on-the-fly. This is not a problem as a requirement of 'alphabet' SFSes is that they be loaded at boot-up. Again IIRC, overlays establish priority thru a naming convention: higher numbers have lower priority.

So, suppose that when aufs is to be used an odvr.sfs [O - for other] was recognized by initrd? And suppose that pkg-cli --in addition to the other choices it offered-- could optionally package applications as an odrv.sfs* [or when overlays are used as a user-selected high-number SFS]? An odrv.sfs can be assigned low priority to avoid its content conflicting with other components of the operating system.

---=---
* A modification to Save2SFS could add updating odrv.sfs. Or perhaps that capability could also be added to pkg-cli.

mikeslr wrote: Sat Aug 20, 2022 3:57 pm

The 'Main' Package manager doesn't need to know the contents of an 'alphabet drive' as it can't modify one anyway. As long as the application which creates an 'alphabet drive' can handle an 'alien packaging technique' any alien package can provide the source files for an 'alphabet drive'; and the Main Package Manager doesn't have to have that capability.
...
So, suppose that when aufs is to be used an odvr.sfs [O - for other] was recognized by initrd? And suppose that pkg-cli --in addition to the other choices it offered-- could optionally package applications as an odrv.sfs* [or when overlays are used as a user-selected high-number SFS]? An odrv.sfs can be assigned low priority to avoid its content conflicting with other components of the operating system.

Actually, unless you code some way such that the 'Main' package manager does not know the contents of any addon drive then it DOES know about it because all such filesystems become part of the merged overlay whether using aufs or overlayfs. If a package ends up in a usual bin directory, for example, such as /usr/bin, and merged in as part of the layer structure, then that can certainly cause problems with a package manager trying to install app to same place (or already having that app recorded in its package management database as installed).

Also it doesn't matter whether it is aufs or overlayfs, aside from the load on the fly mechanism, that overlayfs doesn't itself support, the merging is very similar:

mikeslr wrote: Sat Aug 20, 2022 3:57 pm

Again IIRC, overlays establish priority thru a naming convention: higher numbers have lower priority.

In fact neither aufs nor overlayfs themselves have any alphabetic nor numeric ordering of layers method of prioritising the layer order. That prioritising is entirely decided by the designer of the initrd/init (usually shell script) code. In Puppy traditional design, the initrd, as you say uses a limited set of alphabetically named addon sfs layers (in a pre-defined order by the initrd/init script designer) and some Puppy members have extended the initrd/init script to allow for more alphabetically ordered layers, but that has nothing, per se, itself to do with aufs. Aufs just handles the layers in the order it is told to by the initrd/init code.

For KLV-Airdale, the initrd/init shell script uses the functionality of overlayfs (but also KLV-Air does implement sfs load-on-the-fly albeit via a sym-links method, and outside of the initrd, since overlayfs itself can't provide that). It is the design/script-code of the initrd/init used in KLV-Air that carefully checks for sfs layer addons by looking at the first two characters of their filenames and only loads those that are two-digit numeric and does so in numeric order; like I say, nothing to do with overlayfs itself. In fact the initrd in KLV airedale can use uncompressed directories or sfs files as the addon layers as long as their directory names have first two characters as numeric between 00 and 99 (actually 99 such layer addons are allowed for since 00 is a special one for loading/sharing modules between the initrd and the main filesystem). Directories themselves are not usually read-only, but as far as overlayfs merging is concerned they are treated as such - in practice it is possible to change their contents via writing to them directly, but the immediate result of doing so is not defined in overlayfs standard, though any such changes would certainly come into effect on reboot.

I had to make some more modifications to Pkg-master components for smoother operation.

1. replace all instances of #!/bin/ash in Pkg-cli scripts with #!/bin/bash
2. due too an Unary operator expected in Bash error in line 187 of the /usr/local/bin/pkg main script change ->

# set $DIRECTSAVEPATH (where we want to install pkgs)
if [ $PUPMODE -eq 3 ] || [ $PUPMODE -eq 7 ] || [ $PUPMODE -eq 13 ];then
  DIRECTSAVEPATH="/initrd${SAVE_LAYER}" #SAVE_LAYER is in /etc/rc.d/PUPSTATE.
elif [ "$PUPMODE" = "2" ]; then
  DIRECTSAVEPATH=""
fi

To this ->

# set $DIRECTSAVEPATH (where we want to install pkgs)
if [[ $PUPMODE -eq 3 ]] || [[ $PUPMODE -eq 7 ]] || [[ $PUPMODE -eq 13 ]];then
  DIRECTSAVEPATH="/initrd${SAVE_LAYER}" #SAVE_LAYER is in /etc/rc.d/PUPSTATE.
elif [[ "$PUPMODE" = "2" ]]; then
  DIRECTSAVEPATH=""
fi

The main difference between [ and ] and [[ and ]] is that word splitting and pathname expansion are not performed on the latter. For example, if you have * within [ and ], it will be expanded to the files in the $PWD whereas the same * inside [[ and ]] won't. And most likely you intend to use the literal *. For more details see bash manpage.

rockedge wrote: Sun Aug 21, 2022 4:50 am

I had to make some more modifications to Pkg-master components for smoother operation.

1. replace all instances of #!/bin/ash in Pkg-cli scripts with #!/bin/bash

Unfortunately that signals Pkg needs modified so that it works correctly with ash if you want it to be more universally useful - otherwise it is not very standalone in the sense it needs bash which itself has lots of dependencies. Best to make it work with busybox ash.

@wiak good point about using #!/bin/ash for more all around usage.

I think the fixes I did can be translated into ash

rockedge wrote: Sun Aug 28, 2022 5:24 pm

@wiak good point about using #!/bin/ash for more all around usage.

I think the fixes I did can be translated into ash

That could then prove very useful.

Intrigued to try this, but fell at early stage. I followed guidance in the README.md for installation.
./installer.sh revealed the following.

root# ./installer.sh
cp: cannot overwrite non-directory '/sbin' with directory 'sbin/'
cp: cannot overwrite non-directory '/usr/sbin' with directory 'usr/sbin'
Pkg was NOT installed!

./installer.sh: line 21: can't create /root/.packages/pkg-1.9.23-noarch.files: nonexistent directory
./installer.sh: line 30: can't open /usr/sbin/pkg: no such file
Setting up Pkg...

./installer.sh: line 43: pkg: not found

./installer.sh: line 46: pkg: not found

I guess I need some basic help, if someone has time.

Remember this is totally experimental!

I installed pkg manually by downloading it from GitHub extracting it and placing the components by hand

Experiments lead the way to future uses