Some questions regarding .pets and .sfs files

[FloraMae]

I'm a little curious if people here with more experience and knowledge than I have could maybe answer some questions I have regarding .pet packages and .sfs files.

1) What are the main advantages and disadvantages to using each? Let's say someone wanted to package up a browser for a puppy, which would be a better format to use? Why? Why not the other?

2) Would there be any benefit to making a .sfs file for something small like a browser or utility program?

3) Is there a limit to how many .sfs files a puppy can have?

4) What sort of things should be kept in mind if packaging a program?

5) Is there any easy way to test a program on a lot of different puppy versions or is this just a "load it on each one to test"?

6) If I wanted to package a program to run as a non-root user, how best would this be accomplished for most compatibility?

7) Any advice in general? Any general info?

From what I can gather, it "seems" that .pet packages would work best for small individual programs and .sfs files for larger programs or sets of programs. However, there is so much I don't know.

[mikewalsh]

@FloraMae :-

In general, your final assumption is correct. Small apps/utilities are usually easier as a .pet; they won't occupy much space in the 'save'. Larger apps are often better as an SFS; however, things like browsers - which update frequently - are NOT. Because an SFS is a 'read-only' file, you can update your browser in RAM, yet at the next boot the SFS package will overwrite everything with the older version again......because SFS files take precedence in the layering 'stack'.

This is why we developed the 'portable' browser concept for Puppy. It keeps large items like browsers - along with their associated cache that can quickly balloon to enormous proportions! - completely outside the 'save', yet allows them to update themselves as normal.

There IS a 'limit' to the number of SFS packages that can be loaded at any one time, though newer Pups have a larger number of 'slots' available. I'm not 'au-fait' with what, exactly, controls this.

For testing, there's no option but to try the item in question in each individual Puppy, due to the different versions of the glibc, libraries, kernels, etc.

I'll leave it there for now. Others can help to answer some of your more specific queries.

Mike.

[FloraMae]

@FloraMae :-

In general, your final assumption is correct. Small apps/utilities are usually easier as a .pet; they won't occupy much space in the 'save'. Larger apps are often better as an SFS; however, things like browsers - which update frequently - are NOT. Because an SFS is a 'read-only' file, you can update your browser in RAM, yet at the next boot the SFS package will overwrite everything with the older version again......because SFS files take precedence in the layering 'stack'.

This is why we developed the 'portable' browser concept for Puppy. It keeps large items like browsers - along with their associated cache that can quickly balloon to enormous proportions! - completely outside the 'save', yet allows them to update themselves as normal.

There IS a 'limit' to the number of SFS packages that can be loaded at any one time, though newer Pups have a larger number of 'slots' available. I'm not 'au-fait' with what, exactly, controls this.

For testing, there's no option but to try the item in question in each individual Puppy, due to the different versions of the glibc, libraries, kernels, etc.

I'll leave it there for now. Others can help to answer some of your more specific queries.

Mike.

I wasn't aware of the read-only limitation of sfs. Good to know.

I am curious about this "portable browser concept". Any howto articles or other places of detailed information that you know of?

[Jasper]

@FloraMae

For me the difference between Pet and SFS installations is simply down to security, RAM requirements and deciding between day to day use/occasional use. (BTW I am using a small Pupsave file on a flash drive)

A SFS browser can be used for online banking.

[FloraMae]

@FloraMae

For me the difference between Pet and SFS installations is simply down to security, RAM requirements and deciding between day to day use/occasional use. (BTW I am using a small Pupsave file on a flash drive)

A SFS browser can be used for online banking.

Good point, an sfs browser and the read only limitation could be used to be sure of integrity of the browser files.

[mikewalsh]

@FloraMae :-

As far as the 'portable' browsers are concerned, these IS no official 'documentation' as such, because it wasn't an 'official' development. We hashed the whole concept of these together between us, here on the Puppy Forum. It was - if you like - a joint community development; input came from many different members, and there was never any one specific thread dedicated to the development process.

It would take an age to track down every thread where these things were mentioned and/or discussed. I'm afraid it's something you need to accept; they're a reality, they work, and most members don't worry too much about HOW they work.....so long as they DO. The idea is simple; the app and all config files are self-contained within a single directory, and executed via a launcher script that calls everything from within that directory. This includes any additional libs that may be necessary; we don't include dependencies for the X-desktop, since these are common to pretty much all Puppies.

The initial mentions of the concept probably date back around 15 yrs to the heyday of the old Forum, from HoerMirAuf, IIRC. More recently, @fredx181 probably 'kicked-off' a revival with his 'portable' Firefox builds. I took great interest in this, since my final year with Windows XP was very different to the many previous annual reinstalls. I'd recently discovered the PortableApps.com website, and had populated XP entirely with portable apps instead. It worked beautifully, and was the smoothest install I'd ever done!

Ever since moving to Puppy 10 years ago, I'd wanted to emulate this portable app ability. Fred had 'pointed the way', so to speak, and I basically took the bit between my teeth and ran with it. I'm still building apps this way even now.

I put together a list of all the threads I've started where I've introduced various portable apps, with links to each one. You can find it here:-

viewtopic.php?t=5104

Mike.

[FloraMae]

@FloraMae :-

As far as the 'portable' browsers are concerned, these IS no official 'documentation' as such, because it wasn't an 'official' development. We hashed the whole concept of these together between us, here on the Puppy Forum. It was - if you like - a joint community development; input came from many different members, and there was never any one specific thread dedicated to the development process.

It would take an age to track down every thread where these things were mentioned and/or discussed. I'm afraid it's something you need to accept; they're a reality, they work, and most members don't worry too much about HOW they work.....so long as they DO. The idea is simple; the app and all config files are self-contained within a single directory, and executed via a launcher script that calls everything from within that directory. This includes any additional libs that may be necessary; we don't include dependencies for the X-desktop, since these are common to pretty much all Puppies.

The initial mentions of the concept probably date back around 15 yrs to the heyday of the old Forum, from HoerMirAuf, IIRC. More recently, @fredx181 probably 'kicked-off' a revival with his 'portable' Firefox builds. I took great interest in this, since my final year with Windows XP was very different to the many previous annual reinstalls. I'd recently discovered the PortableApps.com website, and had populated XP entirely with portable apps instead. It worked beautifully, and was the smoothest install I'd ever done!

Ever since moving to Puppy 10 years ago, I'd wanted to emulate this portable app ability. Fred had 'pointed the way', so to speak, and I basically took the bit between my teeth and ran with it. I'm still building apps this way even now.

I put together a list of all the threads I've started where I've introduced various portable apps, with links to each one. You can find it here:-

viewtopic.php?t=5104

Mike.

Earlier I downloaded every "portable" browser on the forum I could find and extracted them all and looked them over. The concept seems fairly straightforward but my first "try" (LibreWolf) failed with a segmentation fault and I probably need more sleep before I can work through that. But, it all gave me some ideas I want to try once I get a better grasp of how these are working and setup.

[Jasper]

@FloraMae

As I am using LibreWolf as a SFS.

Once you have got the initial boot screen and chosen your settings and saved them.

The Profile is a hidden folder in your root directory

./librewolf

This should be approximately 50mb or thereabouts.

This can be moved to another location and 'sym-linked' back to root.

When you unload the SFS history/junk etc is not retained.

Just your profile settings remain.

[bigpup]

pet and sfs are different ways to package software.

A pet package is a Puppy packaging method to provide software to install into the Puppy OS.
Download the pet package.
Left click on it to install it.
Puppy uses a save file/folder that holds anything that is installed.

An SFS package of a program.
Is a way to package a program, but not actually install it to use it.
The sfs package is loaded or unloaded into the Puppy file system,
when loaded it is layered into the operating file system. It works as if it was actually installed.
When unloaded it is no longer layered into the operating file system. So no longer there to use.
This is done with SFs load program.

It also allows the SFS packages to be stored outside of the save. usually in /mnt/home

Note:
The different SFS files that makeup a Puppy version are not the same thing as a SFS packaged program.
Those SFS's are (Linux file system inside a file) parts of the Puppy Linux file system, when layered together form the complete operating file system.

WELCOME TO LINUX SOFTWARE!

Linux software has to be compiled for the specific Linux OS it is going to be used in.
Puppy is a specific Linux OS.
But program pet packages and program SFS packages still need to be compiled for specific versions of Puppy Linux.
good thing a lot of Puppy versions are close to using the same core files a program needs to be able to run.
But more an issue with pet packages.

The idea of the portable packages is about providing everything in it that will be needed.
So what Puppy version it was compiled for is not going to be an issue.
It should work in any Puppy version.

SFS packages can have in them maybe some needed file or added program that will be needed, but not be in some specific Puppy version.

But With Linux software packaging.
Nothing ever seems to be 100% perfect all the time.

[Jasper]

Each offering of Puppy may use a different Glibc.

https://abi-laboratory.pro/?view=timeline&l=glibc

So, it helps to point out to members which OS you compiled/built your application.

[mikewalsh]

I concur with @Jasper , above. Kernels aren't usually the issue; what invariably throws a great big spanner in the works is the version of the glibc in use (the General 'C' library, against which everything in the distro is compiled).....over time, as new distro builds appear, the glibc tends to incrementally move to a newer version.

Example; back in the days of Racy/Wary/Lucid Puppy, we were on 2.10, or 2.10.1. When I joined in 2014, the newly-released Tahrpup was on glibc 2.19. Xenialpup is 2.23, Bionicpup is 2.27, Fossapup is 2.31. I believe the current newest Pups are around 2.36/7 ATM. Stuff built for an older glibc will frequently run under a newer one, since there is a high degree of backwards compatibility here. It won't work the other way round, though!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some years ago, @watchdog pioneered a method for including a self-contained 'build' of the glibc with certain browser packages, so that a new browser could be run, natively, inside an elderly Puppy. I've successfully used this with many 'zilla-based browsers. I've yet to have any success getting this to work with Chromium-based 'clones'.

I keep trying, though.....

Mike.

[bigpup]

Welcome to Linux Software!

[greengeek]

From what I can gather, it "seems" that .pet packages would work best for small individual programs and .sfs files for larger programs or sets of programs.

In my usage the size does not matter. There is one significant difference between pets and sfs - that relates to how you have decided to save your data.

Most people have a "savefile" or "savefolder". The reason for this is that it allows them to save data from each session - eg if they add a program (such as a new word processor) it "sticks" in their operating system and is available next time they boot.

BUT - what if that program turns out to be faulty? What if it causes a segmentation fault? What is the point of having locked that program into your savefile?

So - I do not have a savefile. I want to test lots of browsers, word processors, music players etc etc - but I do not want all of these tests to become permanent in my Puppy.

Therefore - for me - pets are better because they load into memory when I test them, and then disappear permanently when I power off or reboot. Automatically gone. Whereas .sfs files get "stuck" and stop my Puppy from shutting down.

(This is a very specific use case and maybe you will never want to run without savefile. But if you do - then it is easier to use pets rather than sfs.)

Also - if you build a pet you can include a "pinstall.sh" file that is a script that runs specific actions at install time. I don't think this can be done quite the same way with sfs files.

[mikeslr]

@FloraMae

As I am using LibreWolf as a SFS.

Once you have got the initial boot screen and chosen your settings and saved them.

The Profile is a hidden folder in your root directory

./librewolf

This should be approximately 50mb or thereabouts.

This can be moved to another location and 'sym-linked' back to root.

When you unload the SFS history/junk etc is not retained.

Just your profile settings remain.

No. On shut-down/reboot RAM is cleared including all the contents of the hidden profile folder and hidden cache folder in /root. Moving them and symlinking them back, however, would have redirected cache and profile to other locations outside of RAM -- or why bother moving them? Shutdown does nothing to remove cache from the folder outside of RAM. In the absence of such symlinks, if a Save is executed, the then contents of the cache and profile folders will be written to the SaveFile/Folder.
Usually it is desired that the contents of profile (bookmarks, addons, settings) be managed dynamically; but web-cache not be permitted to accumulate. You can locate web-browsers within an READ-ONLY module such as adrv and operate under PupMode5 without a SaveFile/Folder. Then, everything will be in RAM, including the entire LibreWolf application (about 70Mbs). But changing your profile or updating your web-browser would require the entire rebuilding of that module.
Which is why I prefer portables and use addons to manage cache.

[mikeslr]

Getting back to your OP, there are two factors I think important: (1) Floorp web-browser was forked from firefox but retains firefox's structure and system for managing files; (2) Puppys are not its only target audience.

Very few LinuxOS can manage SFSes. While other LinuxOSes can use portables --I recently constructed a firefox portable to use in Kubuntu-- the niceties Mikewalsh and others built in are either not needed or do not translate. Puppys operate 'in RAM' either by copying file-systems into RAM (such as Puppys adrv.sfs) or by mounting file-systems in RAM (such as LibreWolf.sfs). Other Linuxes constantly read-from and write-to storage. So it isn't necessary to limit what accumulates in RAM. Under other Linuxes you operate as a limited User without access to the entire file-system. Portables could not be located in /opt --even if I as Users provided the Admininstrator's password. Nor could they operate from /mnt/ or /media. Kubuntu permitted one folder under /home/mike where a portable could be located. I don't know where other Linuxes may restrict them. Mike's MenuAdd doesn't work. To include access to a portable via a Menu Listing may require unique specifications of the path in each Linux of /usr/share/applications/...desktop's Exec= argument.

A pet is just the designation given to a 'tar'ed folder so that Puppy Package Manager knows to extract it and how to deploy its contents. Deconstructing the pet jasper created reveals:

floorp-structure.png (95.68 KiB) Viewed 203 times

The entirety of floorp is located in /opt except for an icon in /usr/share/pixmaps and a desktop file in /usr/share/applications whose Exec= argument "Exec=/opt/floorp/floorp" specifies the path to the executable.

I would suggest that if desired the pet format be offered to Puppy. But a tar.gz or tar.xz is almost as good. A User under Puppy can locate the 'untarred' folder anywhere and create menu entries to it. To create an SFS all that is necessary is to repeat what jasper may have done: move the untarred folder into opt within a specifically named folder --in his case floorp-11..., add a /usr folder with its contents to that, then right-click it and select dir2sfs rather than dir2pet.
If you examine the firefox portable Mikewalsh publishes, you'll find within it a folder named firefox whose content is just the extracted firefox.tar.gz. The other components of that portable are a DATA folder containing a pixmap and a desktop file which the MenuAdd script will copy to the desired locations. Mike's script also symlinks the LAUNCH script to /usr/bin, renaming that to firefox (the name called by Exec=argument). The LAUNCH script is a wrapper for the actual executable, but adds two things: (1) the instruction to use the libs in the 'extralibs' folder and to (2) locate firefox's profile and web-cache within the portable's folder. The 'extralibs' folder contains mostly libraries which are needed for firefox to generate sound. Not having run floorp, I don't know if it will need them.