Proposed Utility Update: Resize 'Save' File

[ozboomer]

Hi, all...

----------------------------------------------------------------------------------------------------
2023.09.26: 1.0.0 @ozboomer .pet packaging
- minor code mods to reflect 'historical changes' through Puppy variants
- arbitrary adoption of '1.0.0' version number
- simplified .pet packaging (but works Ok with PPM)

resizepfile-1.0.0.pet

(3.91 KiB) Downloaded 88 times

----------------------------------------------------------------------------------------------------
2023.09.23: initial .pet packaging thanks to @bigpup
- code updates to show save file size change after proposed change is made; including 'freememapplet' (tray) icons (more or less - no blinking)
- sorry, it seems a post can only have 3 attachments(?) See the original post and file: https://forum.puppylinux.com/viewtopic. ... 326#p99326

----------------------------------------------------------------------------------------------------

I've always used Puppy with a frugal installation built on a 'save' file, as I find this the simplest means for me to replicate a 'snapshot' of my system and it makes for easier backups and recovery.

Consequently, the function found at: Menu > Utility > Resize personal storage file (otherwise known as /usr/sbin/resizepfile.sh) has always been a useful utility for me. However, I have found it lacked a certain amount of helpful information when I'd been using it.

To that end, I've updated it somewhat so that it presents some extra information on the GUI that changes as you move through the options. The GUI now looks like the following:

resizepfile.sh.png (65.9 KiB) Viewed 2324 times

...and I've attached a short video that shows it in use; once focus is assigned to the drop-down list, I just use the up and down arrow keys to cycle through the options... and as the amount added to the 'save' file changes, the icons change in the same way that they do in the 'freememapplet_tray' program.

I basically wanted to find out if folks would like to try it out... and see if it might be a suitable update to the 'real' utility. If it works out Ok, I'd then look to make the utility available to those 'better suited to the task than I' (!) to work it through github, etc... but that's down the track a ways I think.

So, what is going to be a suitable way to let folks try it out, as it's a (12k) shell script, so it only needs to be dropped into a directory somewhere and run.

Do I drop the source in here... (try to) make a pet out of it... or what is the best way to proceed?

BTW, I've been building it on Fossapup64 9.5 but it seems to work Ok on Slacko 6.3.2 and Xenial64 7.5... and I'm working my way through trying it on Bionic, Tahrpup and a few other puppies.

Edit: Making .pet available here as well as in the discussion...

[fredx181]

Looks very good, I'd say just share it in some way, as script attachment (with instructions) or as a .pet, so that people can try it.

[ozboomer]

Looks very good, I'd say just share it in some way, as script attachment (with instructions) or as a .pet, so that people can try it.

Okie, then...

For simplicity's sake, I'll just attach the source here. I tried to mainly add to the source that I found in fossapup64:: /usr/sbin/resizepfile.sh... although I tweaked a few things in the dialog and elsewhere; I'm never great with gtkdialog.. but this has been one Heck! of a learning exercise... particularly when I thought the change would be 'minor'

Instructions are simple-ish...

• Download the resizepfile.txt file to somewhere convenient... or to a location mentioned in PATH if you want to run it from anywhere.

  Code: Select all

  root# set | grep ^PATH=
  PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/root/my-applications/bin:/usr/games:/usr/lib/qt5/bin:/usr/lib/x86_64-linux-gnu/qt5/bin
  root#
  

• Rename the file and make it executable:

  Code: Select all

  root# mv resizepfile.txt resizepfile
  root# chmod 755 resizepfile
  root#
  

• Now, just run it:

  Code: Select all

  root# ./resizepfile
  root#
  

O'course, you can do all the same things via rox... I'm just an 'older person' so do most everything from the CLI

Once the script is running, it works much the same as it always has... but the icons and 'disk usage bars' in the lower half of the GUI update as the size increase value is changed via the drop-down list. As mentioned earlier, I find it helpful to just assign focus to the list and then use the arrow keys to cycle through the list.

The icons here follow the same convention as is used in the system tray 'free space' icon; see the C source for
freememapplet_tray located at https://distro.ibiblio.org/puppylinux/s ... .6.tar.bz2 or mirror equivalents.

I think it's helpful that I've made the first option in the list a zero change in size, as this lets you see the current state of the icon(s) before you start making real changes; I found it interesting to see how slowly/quickly you need to change the size of the additional space before the icons change colour - if 'riding the colour icon' is important to you

Anyway, please give it a try and see how it goes. As the original script has been around in a lot of puppies now, it was pretty robust; I just hope it's still as strong now...

Fanx! for any thoughts or discoveries you might make

Edit: Fixed-up typos (Dang!)

[fredx181]

Rename the file and make it executable:

root# mv renamepfile.txt renamepfile
root# chmod 755 renamepfile
root#

Now, just run it:

root# ./renamepfile

Typo ? (I think it should be resizepfile).

[ozboomer]

Rename the file and make it executable:

Now, just run it:

root# ./renamepfile

Typo ? (I think it should be resizepfile).

Quite right you are. Fixed.. Thanks.

[bigpup]

Here is a pet package I made of your version of resizpfile.sh

I made it a simple pet that only installs your version replacing the old one.

It installes in /usr/sbin/

The menu entry is already there in most Puppy versions. So did not provide anything for it.
The menu entry should run it after installing the pet package.

All of my Puppy installs are using save folders, so All I tested was that it runs. As expected, I get: you do not need to resize you are using a save folder.
.

resizepfile.pet

(3.7 KiB) Downloaded 61 times

.
Note:
ozboomer,
You can attach it to your first post if you want it easily found.
As a forum moderator, I can too, but this is your topic, and you should control it.

[ozboomer]

Here is a pet package I made of your version of resizpfile.sh

I made it a simple pet that only installs your version replacing the old one.

It installes in /usr/sbin/

The menu entry is already there in most Puppy versions. So did not provide anything for it.
The menu entry should run it after installing the pet package.

Thanks for sorting that out... and giving me a nudge to explore the .pet packaging a bit more.

I don't (yet) understand how it can roll it back but it does it... so it seems it's 'good-ho' to just install my updated version over the existing version.

One thing I did note is that the different Puppies may have different versions (at least according to file sizes), that I haven't investigated fully yet, viz:-

distro-versions.png (71.97 KiB) Viewed 2118 times

...but obviously, there is some consistency depending on the 'lineage' of the Puppies involved.

ozboomer,
You can attach it to your first post if you want it easily found.
As a forum moderator, I can too, but this is your topic, and you should control it.

I'll re-add the file to the first post... but I'll probably do a specific topic for the rewritten tool eventually, once I sort out the version issues, how to document the changes 'in-code'... and will probably rewrite it again; my coding style is much more verbose than most... but that largely comes from my VAX/VMS history, I think

More to come down the track, once I've done some more reviewing and folks have had a chance to try it out some...

[bigpup]

One thing I did note is that the different Puppies may have different versions (at least according to file sizes)

This program has been in Puppy from the very beginning of ability to make a save file.

So it has had a few tweaks done to it over the years.
Similar to what you have done.

So, the size of it has changed, but not what it basically does.

[ozboomer]

Out of interest (and for completeness), does Puppy (generally) follow an 'official' semantic versioning convention ( see https://semver.org/ )? Are there any actual 'standards' as such for what we develop for Puppy... or are we actually more 'laid back' than that?

I can see it would be pretty important for things like /etc/init and /etc/rc.d/rc.shutdown, etc... but where does it sit for things like resizepfile.sh ?

Like I've said elsewhere, I really have no ('active & current') clue about 'git' and other VCS... but maybe there are pointers to some info that would help me out...? I note (in Fossapup64 anyway), CLI git is available through the 'devx' SFS... and the PPM says we have gitg and git-cola as 'simpler' ways into 'the way of git'...

Trying not to bend my brain too much...

[mikewalsh]

@ozboomer :-

Out of interest (and for completeness), does Puppy (generally) follow an 'official' semantic versioning convention ( see https://semver.org/ )? Are there any actual 'standards' as such for what we develop for Puppy... or are we actually more 'laid back' than that?

Umm.....I guess you mean like, say, Ubuntu with the 5-yr LTS releases? Where they keep releasing "point" updates every few months for the first couple of years or so.....that kind of thing?

I'm no 'expert' on this subject, though even I employ incremental version numbers, especially in recent years where, instead of just re-packaging the work of others, I'm now actually building the odd utility/application of my own from scratch. With the Puppies, I suspect it varies from one "dev" to another. Certainly, some release upgraded builds during the development phase FAR more frequently than others; dimkr, for instance.....not in so many words, of course!.....tends to release updated, re-numbered builds far more frequently because I think he feels it helps to track down issues and get bugfixes sorted far more easily with a detailed 'plan of action'. Also, he tends to do things the 'official' way - mostly, I guess, because he works in the industry for a living, he's a highly-organised individual, and is simply used to doing things in a properly-structured manner.

Rockedge is another who tends to release frequent upgrades.....in his case, with the 'Kennel Linux' series.

Highly unlike my own very 'scattershot' approach to developing/working on personal projects; I never document anything, and any 'plan of action' is purely in my own head (no 'notes' even). I'm quite sure nobody in this field would EVER employ ME, because I'm way too disorganised, and have never been in the position of needing to make sure others can follow my train of thought and can see where I'm going with it.....

(*shrug...*)

Mike.

[mikewalsh]

I don't (yet) understand how it can roll it back but it does it... so it seems it's 'good-ho' to just install my updated version over the existing version.

Well, think about it. You install your updated version over the top of the original, and that updated build of

Code: Select all

reseizepfile.sh

.....ends up in the save-file. Through the magic Puppy employs using 'whiteout' files, etc, Puppy now only sees this new version, OK? That's all well & good, and is what you want. BUT;

If, as & when you 'uninstall' this new version - for whatever reason - you will then find the older, original version in its place once more. Why? Simple.....because the original is still an integral part of the Puppy 'base' SFS. Without a 'whiteout' file to mask it, and a new version to replace it, Puppy henceforth 'sees' the original version again.....in essence, you've 'rolled back'.

Mike.

[ozboomer]

Oooo, multi-threaded responses... FanX! @mikewalsh

At the risk of opening a major Pandora's Box (that should probably be done elsewhere, I expect)...

[... git and versioning ...]

Acshually, I understand that we're in a bit of a hybrid place... and I appreciate that we can go either way, with rigid or flexible version (non-)control. It's just that I 'migrated' fairly swiftly from (Civil to Software) engineering... and had worked with both 'types' of (version control) discipline in terms of software development, for system management, databases and applications... and I was just wanting to confirm that either way was 'Ok' for our 'small'(!?) Puppy developments.

Even though it's not a 'requirement', GNU Coding Standards suggest that programs should support common options such as "--help" to display usage information and "--version" to display the program's version; this is partly why I asked if Puppy had any such agreed-to 'standard' for what we produce.

[... PPM and 'Puppy's way' (of layering) ...]

Ya.. but it's the 'in betweening' that (can) catch you.. (I think) you can only have the Puppy '.sfs' ('baked-in') version or the last installed version installed though PPM (effectively, into the layer provided by the 'save' file). So, you have to manage the versioning outside of PPM... or you start making the applications more complicated (by embedding dependency/version testing, etc - wot a can of worms). The available 'roll-back' is sort-of limited that way.

I think this is why the .pet 'standard' (almost) 'requires' a version number on the .pet file, so PPM can do its checking (for not installing versions that are already installed), etc. I'm pretty-much talking through my hat here... but I still think the history/versions in the 'internals' (of shell scripts, for example) need to be properly maintained (and not necessarily 'rigidly' via a VCS) or at least do the magic with the '---help' and '--version' options mentioned above, so folks who want to work on existing applications can more easily determine if they're modifying the latest in-use version.

As always, YMMV.....

[wiak]

... and the PPM says we have gitg and git-cola as 'simpler' ways into 'the way of git'...

You'd be kidding yourself if you believed that. Unless you understand a bit of git commandline I doubt these gui frontends would make any sense at all. Personally I hate them - much easier the simple commandlines I personally use. Not that git is particularly simple either way. It is amazingly useful though.

[bigpup]

Software development in Puppy Linux, is basically what ever way you want to do it, as long as something useful is produced.

But a little common sense comes in there somewhere.

If you are releasing a program give it some kind of version number.
As it gets bug fixed, tweaked, improved, bump up the version number, so people know it is the latest version.
Use this version number for the download file, pet, SFS package, etc.... of the program.
Now the program itself, when it is installed, usually is just a name.
Example this resizepfile.sh script file.

As far as seeing a version number for a specific installed program, script, SFS package, it is up to whoever produced it, how to do that.
A lot of the big programs have the version number in their program window tittle, that shows at the top of the programs operating display window.
Some have a about, you can click on, to show info about the program.

Some are basic script files and if info is not in the top of the script they probably did not provide any info about version of it.
But hopefully the download pet, file, etc.... has something about version.

most of the bigger Puppy programs do have a help to click on when running the program.
Some give you help information as you use the program.
At this step in the program. Here is some help info about doing this step.
Some have, place the cursor over an input slot, action button, etc...., and info pops up about using it.

Example:

Screenshot.png (100.73 KiB) Viewed 1991 times

.
Here is the pet I posted with a version number, simply added to the name of the pet package the number.
The resizepfile.sh file inside is just named resizepfile.sh.

resizepfile 0.1.pet

(3.7 KiB) Downloaded 64 times

[mikeslr]

I know nothing about git. Look to others for advice in that respect. But I think I understand the various systems of organization Puppys use and would disagree with MikeWalsh's response.

Puppys operate 'in RAM'. See, viewtopic.php?p=55827#p55827 and viewtopic.php?p=63818#p63818 for a technical explanation. The following is my layman's understanding:
On boot-up, each Puppy's initrd instructs Puppy what file-systems to copy into RAM . Once copied into RAM, the content of a file-system is indexed/cataloged. Puppy's SaveFile or SaveFolder is not copied into RAM. Rather, it is just mounted and its content indexed/cataloged.
Initrd's instructions establish the priorities the various file-systems will have in RAM. As a general Rule, Puppy Devs locate essential applications in the 'core/base' file-system which usually bears the name of that Puppy in this format: for Xenialpup64, puppy_xenialpup64_7.5.sfs, for fossapup64_9.5, puppy_fossapup64_9.5.sfs. This core/base file-system contains the Puppys essential applications: applications which 'everyone' may want and/or 'defines' the essence of that Puppy. A Puppy ISO may also include an adrv.sfs --for example-- adrv_xenialpup64_7.5.sfs-- and/or a ydrv.sfs --for example-- ydrv_fossapup64_9.5.sfs. There are tools to create and/or modify them. These contain 'additonal' applications. Puppy will boot to desktop without them. However, if present their content has priority over the content of the 'core' file-system if there is a conflict. 'Merge in RAM' works like graphic overlays: the index will point to the content having the highest priority, lower priority content is ignored.
An adrv.sfs has priority over a ydrv.sfs. Having even higher priority is the content of the SaveFile/Folder, with the ‘current content in RAM’ having the highest priority. When you install a package –pet, deb or other-- its content is dispersed in your file-system-in-RAM. When you change a setting, you’ve made that change in RAM. You operating system in RAM may not recognize that change until you Menu>Exit>Restart Graphical Server AKA ‘X’ which causes Puppy to re-catalog/index its current content. Some packages trigger a re-catalog. Those changes, however, do not become a permanent part of your operating system until a Save is executed.

That’s why bigpup was able to construct a pet that did not include the files necessary to create a menu entry. The pet, installed, has a higher priority than the original so its content would be used. But the original’s files relating to the menu entry are still present and --there being no files having higher priority-- would be used.

The pet –originally only in RAM-- was written to your SaveFile/Folder when a Save was executed. That did not remove those files located in puppy_xxx_#xx.sfs, merely re-directed the index away from them. A SaveFile/Folder is READ-WRITE. When you uninstalled the pet, the index once again pointed to the content in the copy of those files in RAM of the puppy_xxx_#xx.sfs.

White-outs come into play when you use Menu>Setup>Remove Builtins, or delete a file in RAM that the copying of a file-system on Storage placed there. Those files are still in a READ-ONLY file-system on Storage. What happens is a file bearing the ending ‘.wh’ is written in RAM adjacent to the effected file or folder. .wh tells Puppy to ignore this file or folder. Remastering copies files from RAM, but follows the instructions and doesn’t copy files/folders which are to be ignored. Amethyst’s Save2SFS tool [in viewtopic.php?p=12983#p12983] when creating an adrv.sfs or ydrv.sfs writes both the files and the white-outs. This can lead to problems which amethyst has addressed, see viewtopic.php?p=73301#p73301.

When a package is installed its contents are copied from their locations in the package to the corresponding location in RAM (later preserved by Save or Save2SFS in the SaveFile/Folder or adrv or ydrv). Those locations are not entirely random. To a large extent Puppys follows Linux’s general Rules as to where files belong. See for example, https://i.pinimg.com/originals/f1/56/6a ... 2e817b.png and the discussion at https://tecadmin.net/linux-file-system/

Short version: executables are placed in /bin or /sbin folders; libraries=dependencies are placed in /lib folders; /opt can be used for anything including entire applications with their own file-systems but is ‘not on-the-path’: the operating system will not look there unless told to do so by an executable in a bin file or the Exec=argument of a desktop file. Desktop files (which are used to generate menu entries) are almost always located at /usr/share/applications; but I’ve occasionally seen them in Puppys at /usr/local/. Generally Linux distinguishes between bin and sbin reserving the latter for executables relating to hardware. Puppy, however, is more relaxed.

If you open a /usr/share/applications...desktop file in a text editor you’ll see a line beginning with icon=. Icons/pixmaps can be placed anywhere; but you may have to provide the ‘full path’ to where. Linux generally will look in both the /usr/share/icons and /usr/share/pixmaps folder when no path is specified. But Puppys will only look in /usr/share/pixmaps. You’ll also see an Exec= argument. Again without specifying the path, both Puppys and Linux in general will look in all the /bin and /sbin folders.

These file systems also have priorities. The Top level /bin and /sbin folders –these are where the creators of a distro locate binaries-- and /lib folder (for dependencies)-- have lower priority than /usr/lib, /usr/bin and /usr/sbin. These are where under Linux in general the files installed by Users are to be located. I think the files ‘hanging from’ /usr/local have even higher priority; as does –under Puppys-- files located in /root/my-applications. Those folders there are unique to Puppys.

Things got complicated when 64-bit operating systems became possible. Slackware added /lib64 and /lib32 folders; while debian & Ubuntu added three folders bearing the name /x86_64-linux-gnu. As a result, applications that make use of /x86_64-linux-gnu have to be ‘translated’ if they are to be used in Slacko Puppys; while for some reason --perhaps relating to the instructions in initrd-- applications making use of /lib64 folders can sometimes be used OOTB in non-Slacko Puppys.

Things got even more complicated when Ubuntu and debian –but not Slackware-- adopted the ‘user-merge’ Rule. My understanding of it is that it restricts use of the top level /bin, /sbin and /lib folders to the creators of a distribution. In this situation, the Dev of a Puppy is NOT a creator since Puppy is ‘derived’ from its binary compatible distro via woof.

My explorations suggest that at least to some extent, binaries and libraries which the creator of a pet or SFS might have previously located in /bin, /sbin and /lib if placed in /root/my-applications/bin or .../lib will still result in a functional application.

Another ‘wrinkle’: Other Linuxes use a folder named ‘Home’ for each User to store his/her settings and customizations. A Puppy User runs as Root and his/her Home folder is /root. So until recently Puppys did not have or need a folder named Home. However, in order to make use of google-chrome and chromium-clones, it was found that the easiest way was to create a top-level Home folder, and limit its permissions to those granted Spot.

[wiak]

Things got even more complicated when Ubuntu and debian –but not Slackware-- adopted the ‘user-merge’ Rule. My understanding of it is that it restricts use of the top level /bin, /sbin and /lib folders to the creators of a distribution.

It is actually quite simple (a simplification really) and not in itself restrictive at all.

https://debian-handbook.info/browse/sta ... archy.html

Note that many modern distributions, Debian included, are shipping /bin, /sbin and /lib as symlinks to the corresponding directories below /usr so that all programs and libraries are available in a single tree. It makes it easier to protect the integrity of the system files, and to share those system files among multiple containers, etc.

The problem traditional Puppy has is two-fold.

First is due to it being a distro that involves merging filesystems. You can't merge sfs addons that use actual /bin, /lib dirs on top of underlying root filesystems where these are just all symlinks to /usr hierarchy. All new stuff coming from upstream repos is organised with view that /bin, /lib and so on are just symlinks.

Second is that security has been very important since UNIX was designed. Standards are created as a simplification to be followed or expect countless things not to work. /home/user_name is the standard for user home directories. Putting it anywhere else is just asking for trouble, meaning incompatibility with upstream repos.

Perhaps in original Puppy days, these issues (and avoiding PAM/mulit-user support, as part of way to make tiny distro just with one user root) didn't matter since Puppy used its own repos - but now it relies on upstream Debian or whoever, and important apps really aren't designed to run as user root.

Unfortunately there have been a few who have proudly boasted about Puppy doing things 'differently', as if protecting their favourite distro from criticism, when really it would have been wise to advocate replacing the non-standard 'mistakes' with that which works correctly with upsteam downloads out-of-the-box. Unfortunately such differences were generally a design error that inevitably comes back to haunt. The ridiculous situation has, however, had the positive community effect that it has given forum members plenty to do over the years - hacking workarounds endlessly to make these non-standard matters less broken. Such a waste of human hours unfortunately, but fortunately more recent Pup design is reducing many of these original incompatibilities.

Whilst the location or even existence of a home directory for a user isn't forced in the Filesystem Hierarchy Standard, matters that are should be adhered to if such simply huge-time-wasting issues are to be avoided.

[ozboomer]

New version (see top of topic).

Minor updates, works Ok with PPM.