How often do you experience system crashes?

[bigpup]

Do something that may need to use swap.

While it is in operation.

In a terminal run this command:

Code: Select all

vmstat 1

You need to check the swap column where si means “swap in”, and so means “swap out.” If the numbers are high, it means a lot of swapping activity which is an indicator of low memory issues. If you see swap usage by free but little active swapping, tweaking swappiness might be due.

When streaming using Firefox.
Do you have it set to use hardware acceleration when available?
That is suppose to use the GPU instead of the CPU when streaming videos.

[amethyst]

Do something that may need to use swap.

While it is in operation.

In a terminal run this command:

Code: Select all

vmstat 1

You need to check the swap column where si means “swap in”, and so means “swap out.” If the numbers are high, it means a lot of swapping activity which is an indicator of low memory issues. If you see swap usage by free but little active swapping, tweaking swappiness might be due.

When streaming using Firefox.
Do you have it set to use hardware acceleration when available?
That is suppose to use the GPU instead of the CPU when streaming videos.

It uses the swap but very little thereof. Swap is working but ineffectively as already stated. Where's the hardware acceleration setting?

[greengeek]

Why? My swap file is hardly EVER used (and when it is used it's just a couple of mega bytes). I have long been sceptical of the way Puppy uses the swap file, I think it's ineffective or not working correctly.

Yes i think you are correct. I don't know if it's a puppy issue or Linux in general - but i agree that the swap partition is not used correctly. (At least when save file is in use)
However - Puppy's behaviour is different when there is no savefile in use - under those conditions Puppy grabs about a third of the swap partition as a form of pseudo RAM. (If the swap partition is available during the boot phase)

Having the extra pseudoRAM allows you to do things that would not normally fit into 2GB of RAM.

I could not do video procesing until I added a swap partition of 30GB. Just saying - it's worth a try.
(Won't help those who have a save file - but I think you avoid save files...)

[amethyst]

I have a swap file and no save file. Nothing is being "grabbed".

[dimkr]

@greengeek I think you're talking about zram, and that's not how it works. If you have a recent Puppy, the init script creates a zram block device and puts swap on it, if no swap partition is found. This pseudo swap partition sits in RAM and it's compressed, so for every 1 MB of RAM "wasted" on this swap space, you gain more than 1 MB of usable virtual memory that's slower to access compared to RAM, but probably faster than swap on your disk.

The init script also sets /proc/sys/vm/swappiness to 5, which keeps this swap space unused until the system is really running out of RAM, so there's no upfront RAM penalty for having this in-memory swap, and it will improve system stability if you run into OOM conditions frequently.

Regarding the differences between 32 bit and 64 bit Puppy, they don't really matter. A 64-bit application will need more memory, because pointers are bigger. Some applications are affected more than others (Python, browsers, etc') but the difference is not big. An equivalent 64 bit Puppy, where the package selection and package versions are identical to your 32 bit one, won't make a computer more prone to crashing, unless it suffers from faulty RAM (and a 32 bit OS prevents it from accessing the faulty memory after the first 4 GB of RAM), overheating (for example, when GPU acceleration is supported only in a 64 bit OS) or some misconfiguration (high swappiness or not enough swap space to prevent OOM). And, a 64 bit OS can improve stability because applications that try to use more than 4 GB of RAM don't just crash.

(And make sure you have the latest microcode for your CPU, if you use an old Intel. Some Intel CPUs ship with known bugs that make them freeze or crash when software uses this or that instruction. Maybe you have one of those, and the problematic instructions are x86_64 ones that never get called in a 32 bit OS. 'Official' Puppy releases ship without microcode, so you're using the "factory" microcode.)

[amethyst]

I've decided to stick with just one Puppy and running the same browser all the time. Seems to be better for stability. Currently using Bionic 64 and the latest Firefox. The amount of crashes could be because of all the testing I have been doing the past days. Booting different Puppys, kernels, different browsers, etc. Just give it time to settle. I've noticed with this laptop, booting different Puppys with different kernels in a short period could mess up some hardware configuration and lead to problems.

[bigpup]

Where's the hardware acceleration setting?

In Firefox it is in the settings -> Performance.
Uncheck use recommended performance settings
Then it will show the option for Use hardware acceleration when available.

If not using a save.
Puppy is using a ramdisk for the operating save.
So that is using up RAM space.

I also wonder what the graphics hardware is set for how much RAM is used for graphics.

I do notice when streaming videos from say YouTube.
Setting the video quality can affect play.
Too high and play is very slow or choppy on low ability graphics hardware.

[amethyst]

Puppy is using a ramdisk for the operating save.

Well, 50% of your installed RAM is actually set aside for file storage if you operate without a save file (which will just sit there unused if you don't save files to the running system). It makes sense then to rather operate with a save file to avoid this situation. I'm just going to create and use a small save file for system configuration purposes.

[dimkr]

Up to 50%, not 50%. This space is not preallocated.

[amethyst]

Up to 50%, not 50%. This space is not preallocated.

Oh, okay. Thanks for clearing that up. So the "balance" not used will be available for normal, system operation? If this is the case, I don't have to use a save file then. I do notice however with Bionic that about 100MB of this space is used at bootup (booting pfix=nocopy). I haven't seen this with older Puppys, why is this?

[dimkr]

I do notice however with Bionic that about 100MB of this space is used at bootup (booting pfix=nocopy). I haven't seen this with older Puppys, why is this?

You'll need to look at /initrd/pup_rw and find which files occupy this ramdisk. ncdu is a good tool for that.

[greengeek]

@greengeek I think you're talking about zram, and that's not how it works.

Hi dimkr - firstly thanks for weighing in on this - you are a top dev and I respect your opinion highly.
However - in this case my comments are not to do with zram but are instead relevant to a specific "swap" behaviour seen on older puppies and unfortunately very few people seem to understand the relevance of the way swap works in one particular instance.
(I can't speak for modern puppies - they may have removed the init code that impacts what I am about to discuss)

I believe Amethyst does not use save files - therefore my comments will apply to him.

In short - Puppy's originally used "swap" partitions in a couple of different ways (when booted without save files !!)

The first way is the standard way - ie RAM pages can be swapped in and out of RAM to give all code some CPU time in order to avoid system lockup. (In my opinion this does not work properly on Puppy. In fact this ability seems pretty useless in Puppies and Amethyst's comments reflect the common experience).

But there is a second way that Puppy uses swap - as long as a "large swap partition" is available during the boot process.
This second way is what I think of as Puppy's "secret" method of extending RAM. (This is only available when booting without savefile!!!)

I (and I believe Amethyst also) use adrvs or ydrvs (etc) as a means of introducing personalisations and customisations instead of using savefiles. Therefore the method I am about to discuss is one way of avoiding system lockups that are caused by insufficient RAM or insufficient kernel buffers.

I know you are busy - and I know Puppy has moved on from some of the older init processes - but I think it is important that someone of your stature and experience is made aware that this technique does in fact extend the abilities (and reliability) of many different Puppy's used on many different machines. I believe there are still many Puppians like me who use 64bit machines with only 2-4GB of RAM. Such users can benefit from this technique.

There are many really good older Puppies (eg Tahr and Xenial and many derivatives) that can still run really well even with low RAM.

Could you perhaps review my technique here and I think it is my opinion that you would find it is quite different to the zram you referred to:
viewtopic.php?t=3239
(Anyone can follow what I wrote there and test it for themselves)

When Puppy is booted without savefile - and with a large swap partition available during the boot phase - Puppy will grab about one third of that large swap partition and add it onto the normal RAM total - to give you a much larger effective RAM. Puppy calls this total "Personal Storage Space". (It is treated as real RAM).
I know this sounds bogus but it is TRUE.
Obviously this artificial swapRAM is slower than real RAM - but even so it transforms the Puppy into much more than it could be when relying only on low totals of actual RAM. It does not increase speed but definitely increases functionality and overcomes most system crashes and lockups.

This trick is invaluable for people like me who make do with machines with 3GB or less of RAM.

Here are some other posts referring to the same ability:

Maik murks from 2011:
https://oldforum.puppylinux.com/viewtop ... 07#p588707

Another of my explanantions how to test this:
https://oldforum.puppylinux.com/viewtop ... 34#p996234

My discussion about normal swap use (and "swappiness") and their limited value:
viewtopic.php?p=50591#p50591

This technique really DOES help to make many Puppies stable in low RAM environments.

Maybe this technique is lost in recent Pups?
Maybe zram tries but fails to achieve this in recent Pups?

Either way - anyone who wants great performance from older pups (especially but not necessarily 32bit) would do well to try this Puppy quirk.

[amethyst]

The question is why Puppy does not work so well with a save file, definitely not as well as Windows. I can't recall ever running out of RAM using Windows. As for a swap partition - never used one. So you create it with GParted and then it just works automatically after next boot (or do you need to do something to enable it)?

[bigpup]

Again the boot process will find and use swap, if there is a swap partition or a swap file, there to find.

[greengeek]

As for a swap partition - never used one. So you create it with GParted and then it just works automatically after next boot (or do you need to do something to enable it)?

Yes - using Gparted is the best way I know of to create a swap partition. (You don't need to enable it in any way. Just make sure it is available before you boot and then the normal Puppy boot process does it all...)
In it's simplest form this can be done with a usb stick - use Gparted to format the entire USB stick as "linux swap". (Plug the usb "swap stick" in before booting and it will be found during boot)
However - you probably know that usb sticks have limited write speed and limited maximum number of writes - so not the best swap option.

A better way is to build a swap partition into your hard drive when first setting up Puppy. (not so easy later as you would have to shrink the existing partitions...)

But there are two other ways that can be pretty useful:

1) Use an external SSD instead of a usb stick. (You can get SSDs that connect via a special cable to a usb port. Works pretty well if you have a fast usb port)

2) Plug in a "usb3 express card" (if you are running a laptop) and plug the SSD or usb3 stick into that. (Again - this must all be plugged before booting - but usb3 express cards are very fast and handle fast usb sticks and/or SSD drives very well). This is handy if you have an express port that you don't normally use...

But if you can start fresh and use the top 20GB or 30GB of your hard drive as a SWAP partition it will make a huge difference to (almost) any puppy that is running without savefile.

[amethyst]

Again the boot process will find and use swap, if there is a swap partition or a swap file, there to find.

Again - You need to switch the swap file on if you don't have a save file/folder. I'm not using a save file nor save folder, will the system pick up and activate the swap partition automatically without running a save file/folder?

[amethyst]

As for a swap partition - never used one. So you create it with GParted and then it just works automatically after next boot (or do you need to do something to enable it)?

Yes - using Gparted is the best way I know of to create a swap partition. (You don't need to enable it in any way. Just make sure it is available before you boot and then the normal Puppy boot process does it all...)
In it's simplest form this can be done with a usb stick - use Gparted to format the entire USB stick as "linux swap". (Plug the usb "swap stick" in before booting and it will be found during boot)
However - you probably know that usb sticks have limited write speed and limited maximum number of writes - so not the best swap option.

A better way is to build a swap partition into your hard drive when first setting up Puppy. (not so easy later as you would have to shrink the existing partitions...)

But there are two other ways that can be pretty useful:

1) Use an external SSD instead of a usb stick. (You can get SSDs that connect via a special cable to a usb port. Works pretty well if you have a fast usb port)

2) Plug in a "usb3 express card" (if you are running a laptop) and plug the SSD or usb3 stick into that. (Again - this must all be plugged before booting - but usb3 express cards are very fast and handle fast usb sticks and/or SSD drives very well). This is handy if you have an express port that you don't normally use...

But if you can start fresh and use the top 20GB or 30GB of your hard drive as a SWAP partition it will make a huge difference to (almost) any puppy that is running without savefile.

I'll set it up when I experience crashes again..until then......

[greengeek]

Again - You need to switch the swap file on if you don't have a save file/folder. I'm not using a save file nor save folder, will the system pick up and activate the swap partition automatically without running a save file/folder?

(I am only aware of Puppy's behaviour when a swap partition is in use. I did once try a swap file briefly but I don't consider that to be the same thing as a swap partition and would avoid it)
The system will definitely pick up a swap partition automatically when running without save file or save folder.

[amethyst]

Again - You need to switch the swap file on if you don't have a save file/folder. I'm not using a save file nor save folder, will the system pick up and activate the swap partition automatically without running a save file/folder?

(I am only aware of Puppy's behaviour when a swap partition is in use. I did once try a swap file briefly but I don't consider that to be the same thing as a swap partition and would avoid it)
The system will definitely pick up a swap partition automatically when running without save file or save folder.

Thanks, not so with a swap file, you need to activate it manually (the system picks it up but doesn't activate it so it's useless) if you don't have a save file/folder.

[dimkr]

Maybe this technique is lost in recent Pups?
Maybe zram tries but fails to achieve this in recent Pups?

Puppy's init script hasn't changed a lot throughout the years.

It activates swap on the swap partition it finds (https://github.com/puppylinux-woof-CE/w ... ysinit#L71), otherwise tries to use /mnt/home/pupswap.swp or /pupswap.swp (if you created it using the "boot manager" thing - I think it's a terrible idea), otherwise it creates a zram block device (https://github.com/puppylinux-woof-CE/w ... sinit#L104, https://github.com/puppylinux-woof-CE/w ... ysinit#L61) and puts swap on it.

Then, the init script expands the ramdisk (/initrd/pup_rw) size limit (https://github.com/puppylinux-woof-CE/w ... sinit#L274) to 1/2 or 3/4 of RAM size, depending on PUPMODE and union file system used. This is what you see in the "free space" tray icon.

If the amount of RAM + swap is limited to 4 GB (so only a small portion of your swap partition is in use), you're probably using a 32 bit kernel with PAE disabled. In addition, the init script puts 5 in /proc/sys/vm/swappiness, which means that regardless of the swap partition size, the kernel will avoid using it unless really running low on RAM.

AFAIK nothing here is relatively new, except the fallback to zram if no swap partition/file is found.

If you have a machine with very little RAM but also a very slow disk, maybe you can try to create small but fast zram swap and use a slow but big swap partition, and give zram higher priority. But you'll need to modify the init script a bit to make the zram part unconditional, which isn't hard.

(Personally, I don't use swap partitions and let the init script fall back to zram. My computers have very old spinning disks, or old SATA SSDs that fail every once in a while but somehow they return from the dead. The system freezes and crashes when disk returns I/O errors or corrupts swap. However, everything is stable and I hardly feel any performance drop when I run a browser on my <= 2 GB RAM laptops and open many tabs, because zram is so much faster than swap on a slow disk. However, with a super slow single-core CPU and a fast disk, a swap partition can be faster - I have one computer like that.)

[greengeek]

Puppy's init script hasn't changed a lot throughout the years.

It activates swap on the swap partition it finds (https://github.com/puppylinux-woof-CE/w ... ysinit#L71),... otherwise it creates a zram block device (https://github.com/puppylinux-woof-CE/w ... sinit#L104, https://github.com/puppylinux-woof-CE/w ... ysinit#L61) and puts swap on it.

Then, the init script expands the ramdisk (/initrd/pup_rw) size limit (https://github.com/puppylinux-woof-CE/w ... sinit#L274) to 1/2 or 3/4 of RAM size, depending on PUPMODE and union file system used. This is what you see in the "free space" tray icon.

Thanks for the links. They all seem to suggest that this code has not been changed since 2016 - is that correct?? Very useful to know, thanks.

[dimkr]

@greengeek This shows when each line was born:

https://github.com/puppylinux-woof-CE/w ... rc.sysinit

(But sometimes it's stupid, a line moved from another location in the same file counts as a deleted line plus an added line)

And this is the changelog:

https://github.com/puppylinux-woof-CE/w ... rc.sysinit

[greengeek]

@greengeek This shows when each line was born:
https://github.com/puppylinux-woof-CE/w ... rc.sysinit

Hmmm, that doesn't match with the changelog notes at the beginning of each file. Makes it look like the zram thing is modified 3 years ago - although the script notes suggest no change since 2016. (I could be misinterpreting...)

[dimkr]

Hmmm, that doesn't match with the changelog notes at the beginning of each file. Makes it look like the zram thing is modified 3 years ago - although the script notes suggest no change since 2016. (I could be misinterpreting...)

Yes, ignore the weird changelog at the top. I don't understand those who document changes using comments and use a version control system like git. If you describe and document the changes using your version control system, you don't need to describe them inside the code too, and this internal changelog is almost guaranteed to run out of sync with changes made to the code, because the "source of truth" regarding what was changed and when is not the code itself.

[ozsouth]

@amethyst - I never have crashes with s15pups. I think your 32bitters run well due to their small footprint.
On that score, you may wish to try my fossa64-med, in remasters section. At 272mb, is easy on ram.

[amethyst]

@amethyst - I never have crashes with s15pups. I think your 32bitters run well due to their small footprint.
On that score, you may wish to try my fossa64-med, in remasters section. At 272mb, is easy on ram.

Spend a bit of time with Bionic 64 ang got it running quite nicely. Some if these streaming sites run much better on a 64-bit system actually. I did download Fossa 64 the other day. Runs well but I prefer Bionic 64.

[amethyst]

After lots of testing I've finally settled with what I know works very well on this machine, ie. Bionic 32-bit and a slightly older Firefox (version 89) with ozsouth's 5.4 kernel. The older Firefox makes a huge difference on this machine. Added advantage is that I can run my older 32-bit Wine applications without having to make compatibility changes on a 64-bit system. In the very rare situations I require a 64-bit browser for DRM streaming, I have Bionic 64 at hand. So for those with slightly challenged machines, I can recommend this setup. A very good streaming site that works well with this setup and is resource friendly is: https://movieshd.watch/. Also, I can recommend the following addons for Firefox: uBlock Origin, Agent switcher (run the youtube site in mobile mode using the Android Tablet switch) and Image Block X.