So the basic idea here is to create a hardlink but define the actual file content later. My application here is syncing package managers. For instance, I know from the DISTRO_PKGS_SPECS file that libc6 is including in the glibc package, which has a file list found in /var/packages/builtin_files. What I don't know is the file list for libc6...or at least the portion of files from this package that are built-in-to puppy.
However, I can create a fake package list for libc6 that points to glibc via symlinks and hardlinks and then by overwriting the symlink later on, with the actual file list. This will automatically update the file which is a hardlinked to this symlink. Skeptical? Here is a demonstration:
Code: Select all
$#Make our testing directory and cd into it.
$ cd /
$ mkdir test
$ cd test
$ ls #Verify this directory is empty
$ "echo a" > a #Let "a" be a builtin file list.
bash: echo a: command not found
$ echo a > a
$ ln -s a b #Let "b" be a symlink to our builtin file list
$ ls
a b
$ ln b c #Let "c" be file list for a compatible distro equivalent to part of file list "b"
$ echo b > b "#Overwrite the symlink"
$ cat c "When the symlink is overwritten so is the hard link.
b
$#The overwritten symlink for instance could contain the actual file list for "c".
Application #1 - update glibc with dpkg (two package manager scenario)
So say for instance we want two package managers on our system (i.e. ppm and dpkg) and we want to use dpkg to update glibc. Prior to doing this we need make sure for the actual distro package equivalents to the glibc package are replaced with the real file list for the component package. This is so that dpkg knows which files to remove after updating glibc because multiple versions of glibc can break the system. In DISTRO_PKGS_SPECS, the glibc package is broken down as follows for upupGG+D
Code: Select all
yes|glibc|libc-bin,libc6,libcrypt1,libc6-dev,tzdata|exe,dev,doc,nls
Application #2 - update glibc with pkg
Now supposedly dpkg has no issue upgrading a distro by updating the component packages. That said, when the top layer is ram PUPMODE=13, puppy has a potentially safer way to update these core libs. In this PUPMODE, the changes are made to the save layer then all layers are re-evaluated. The running system doesn't actually see these changes until the layers are re-evaluated so this should protect the system from breaking in the middle of the installation process.
Since "pkg" is easier to maintain/modify than dpkg, we could write package in a way to update these builtin file packages without having to update the various sub-components of these packages defined in the compatible distros. This is actually preferable because this way we won't re-evaluate the layers until all of the subcomponent packages are updated.
Notes on applications #1 and #2
Since having multiple versions of various components of glibc can break a system due to binary incompatibility issues (, unless done in a carefully segregated way), it will be necessary to uninstall the old package when installing the new package. For this to work with dpkg, it might be necessary to write a "CONTROL" file that contains an uninstall script. This could be as simple as removing the files based on the package list.
Application #3 building an AppImage
trister, posed the question of how to use ldd to include all the dependent libs in an AppImage. This idea is kind of a minimal system build like is used in PuPnGo (more info). We would likely do something like this, if we wanted the AppImage to work in chroot mode. For this scenario, in some cases we might want more than just the dependent libs but also other files from the packages where these dependent libs reside.
I propose a two step build process. In the first step we generate the specification for the AppImage. The specification, would include the file lists for each package but broken out into the essential parts as identified by ldd and the extra files which may or may not be need. There would also be a list of which of these package lists to include as part of the build process. The script would have some options to specify which of these file lists are commented out in the include file. Some criteria for including (or excluding) a package might be if given package (or lib) is essential per ldd or whether or not it is it included in the base_sfs.
After creating the build specification, the user edits it to suit their needs, and tries building the AppImage. If it works as hoped they keep it, otherwise they update the specification and try again.
See Also / Notes
- In scenario #1 and perhaps indirectly in scenario #2 we create a file list for the component packages to those defined in DISTRO_PKGS_SPECS. One way, we can do this is via web-scripting via wget and sed. I discuss how to do this in the following thread:
Scrape Debian Package File List w/ wget & sed
- See the thread "Create Portable AppImage" for instructions on how to create an AppImage
- The ppm, blocks you from installing the distro specific component packages for these combined woofCE component packages (e.g. glibc). This is to prevent you from breaking your system. You can get around this using "pkg" (see post), which is a command line alternative to the ppm (Puppy package manager).
- I'm working on allowing duel package managers in puppy (ppm + dpkg), for some preliminary work, see the threads:
* I sort of installed APT, dpkg and pkg on upupGG+D
* Create dpkg status file from ppm metadata
Notes on Modulararity
Debian breaks un software into more separate packages than some distributions. There are pros and cons to this from a modularity perspective. In puppy users often like the ability to quickly load or unload packages via sfs files. For such applications it is typically desirable to include more components in a sfs than a single debian package. That said the maxium number of aufs layers is 127 and this is related to the maximum number of arguments of a linux system call. So in theory sfs could be broken down into a lot of sub-sfs files but even if we don't exceed the maximum number of layers with this approach there could be some overhead. The core components of puppy (e.g. glibc) as defined by unique lines DISTRO_PKGS_SPECS are of better size for the purposes of modularity from a layered file system perspective. That said, puppy further combines these pacackes into a few layers such as the base_sfs layer.
If we wanted a terminal only version of puppy we might move the glibc component of the base_sfs into a seperate layer and only load it as needed.
Partial Remastering
The puppy remaster approach is typically used to generate a new version of puppy. However, to streamline this process, others have built scripts to incorporate changes into either a new layer (e.g. a adrv) or into the base sfs. One could define a base_sfs remaster tool, which creates a new base_sfs that only includes changes related to libs defined in DISTRO_PKGS_SPECS. Such a script would replace the base_sfs file and clean out any whiteout files that block said changes.
Puppy cleans out the whiteout files when booted with pfx=clean (see post). This is typically done when one upgrades the base_sfs file. For remastering see: "nicOS-Utility-Suite".
