Puppy Linux Discussion Forum

[Edit- Although there might be some info in this post that may, or may not, be of use, the post is all over the place. I wasn't thinking clearly. It might be best to skip this post and avoid some suffering.]

Hi. I don't want to sound like Chicken Little, but it seems best to mention that there may be boot and partition access issues in some situations, or for some users, with the upcoming (?) e2fsprogs (?), with respect to ext4 filesystems.
As you can tell by the uncertainty, and the question marks, I don't really know what I am writing about. Further, although a rough scan of posts was done, a search was not undertaken, so I don't know if I am trying to post about something that is old news and already well discussed ( if so, I apologise).

However, it seems best to mention it, so users who might be affected can be prepared.

What does this post stem from? You may be aware that Fatdog64-814 and 900Alpha have recently been released. The ext4 filesystems created using the GParted provided in these releases will have the metadata_csum and metadata_csum_seed attributes (together with others, of course). Naturally, these changes will not be limited to just Fatdog64. They are, or will be, increasingly more wide spread. I don't know if this might already be the case with some Pups, Dogs, KLVs, etc..

Where kernel support is not available, the new ext4 filesystems will not be able to be mounted. Which is good for data protection, of course. This could, however, impact things for example, if a new partition were to be created, with a new ext4 filesystem, that was intended to be used as a shared data, or boot partition.

Kernel support for these attributes has been in place for a long time. However, this is a Puppy forum, and a long time has a different meaning here, or so I think .

To give a very rough idea of the bounds, a test was undertaken using an ext4 partition that was created using the default GParted in FD900alpha, and a "multiboot" USB drive based on one of the USB-Boot images in the Fatdog64 ISO.

The partition could be mounted by Slacko32-7.0, Slacko64-7.0, BionicPup32-8.0, BionicPup64-8.0 and, of course, later Pups.
Slacko32-6.9.9.9 could not mount the partition.
It follows that Slacko64-6.3.2 and Tahr32-605CE also could not, they were too early.

I apologise for being so long winded, but there might also be considerations with respect to using older bootloaders to boot Pups etc. on the new ext4 filesystems. For example, an attempt to boot FD900Alpha on a new ext4 partition using MX21.3 failed. Although the kernel in MX 21.3 supported the attributes, Grub2 could not boot it whilst the metadata_csum_seed was present. When this result was posted Jamesbond suggested that this was likely to affect the bootloaders in FD813 as well, and that those in Fatdog64-814 had been updated to accommodate this. I would imagine that the same has happened, or would also be happening, in the wider spheres of the Pups, Dogs, KLV etc..

If, like myself, you are sometimes using larger distros as your "boot lead", so to speak, then it could be helpful to check and plan, maybe. I guess I should also mention here that the Grub2 Os-Prober is also, apparently, being disabled. It already is in Debian 12.0.0. [EDIT-or so I had thought].

Ext3, f2fs etc. filesystems are, of course, not affected by this.

It is not the end of the world. I guess if you are not mixing things you may not even be affected by it. However, it seems best to have warning so that you can plan if you need to.

Apologies for yet another waffling tome. Doubly so if I have just been wasting your time. Thanks.
[Edited to correct a misinterpretation that I had, amongst many]

@baldronicus :-

It's not something that ever affects me, nor is it ever likely to. In all the years I've run Puppy, I've always used ext3. It works for me, and has been extremely stable.

However; it's the default FS for most mainstream distros, and is increasingly used with various newer Puppies.....especially, I think, if default installation schemes are employed? (Could be wrong on this point). But whatever, it's definitely a point of which the community will need to be aware.

Thanks for bringing it to our attention. Cheers!

Link to discussion in the FatDog Forums...

Mike.

Let me see if I can get to the essence here.

Basically the way I understand your explanation, this is a kernel support issue, so if an OS is using a later kernel, ext4 partitions for booting, etc should not be a problem?

All the recent pups and Kennel Linux I run are exclusively on ext4 partitions. Some drives are setup with the bootloader on ext4, others fat32.

Hi. Thankyou @mikewalsh (also for providing the link), @bigpup and @geo_c .
Please accept my apologies for your, and maybe others, suffering, in reading the first post. I'm not the best one to be posting about this since I don't understand a lot, and still don't really have a clear idea in my head about the ramifications (although @jamesbond 's explanations in the thread that Mikewalsh kindly linked to, together with this one https://forum.puppylinux.com/viewtopic. ... 758#p93758 have helped a lot).

Here's an attempt to be a bit clearer, although, even if it is not stated, please prepend "If I have the right idea/understanding" behind every line, as it is very likely that I don't.

If my understanding is correct, the Ext4 filesystem attributes, metadata_csum, and metadata_csum_seed, are now being (or have been ) fully activated (if that is the correct term) by the upstream e2fsprogs. Newer distro releases are likely to be incorporating this in the tools used for generating, reading and writing new ext4 filesystems.

If the back-end tools for a file manager (for example), or a bootloader, don't have support for all of the attributes of a filesystem, then the filesystem will not be mounted (this, of course, protects the integrity of the filesystem and the data on it).

Of course, if it can't be mounted, then you cannot access any files that might be on it, and a bootloader can't boot from it.

Although kernel support for the new attributes has been available for a long time, this is not the whole story. There are also considerations regarding the level of support in the userspace tools (I think) and bootloaders. (Note that I might be misinterpreting the information that Jamesbond has kindly provided). For example, the filesystem in MX Linux 21.3 had the metadata_csum attribute, but not the metadata_csum_seed one. Even though the kernel and tools had full support for both of the attributes and could mount the newer ext4 filesystems, the bootloader did not have support for the metadata_csum_seed attribute and could not boot from one of those filesystems.

We have different set-ups and approaches, and if we have more than one machine we might be using different approaches on different machines. How this impacts each person/system is going to vary. The important thing is that we are aware of it and can plan for it.

One consideration is that the only filesystem that this impacts is ext4. Further it only impacts access to ext4 filesystems that have been created using newer tools, that will add the above attributes to the filesystem.

As Mikewalsh has pointed out, he is only using ext3 and is not likely to be affected by this.

If you are not intending to use any new releases (e.g. you are happy with your current set-up) then the problem doesn't arise either. But it doesn't hurt to be aware of it, in case things change.

If you are only using tools that create "earlier" ext4 filesystems (e.g. you only use the GParted from a Puppy which doesn't apply the attributes by default) then you may not be affected. Newer Puppies and the like, could, I think, be put on the earlier filesystem, and should work (that had been tried with FD900Alpha, booting via the Grub2 in MX21.3, and it worked fine. I think it would be the same for Puppies etc..).

It could be that you are prepared to modify any "newer" ext4 filesystem that might be created, using tune2fs.

If you only run with "the new" and just use the bootloaders and tools that come with any releases that bring this in, then you probably don't have to worry either, as everything should be "in sync" and supported.

If you don't have data partitions that are shared between different distros, or releases of distros, then maybe you can contain things, as each distro will only be accessing it's own filesystem. The boot question arises here, of course. If you only boot from the newest installation then the bootloader should be able to access it's files and should be able to boot distros on "old" and "new" ext4 filesystems. It may not work the other way around (although there may be the variable of Grub2 Os-Prober if you are using a larger distro. However, my understanding is that Os-Prober is being disabled in newer releases).

If you are mixing old and new, using shared data partitions etc., then you might need to plan things.

These considerations are, of course, not exhaustive (and I wouldn't really rely on my interpretations).

Another tome, but I'm hoping it's a bit better than the mess of the first post. Thanks.

baldronicus wrote: ↑Thu Jul 13, 2023 3:02 am

. Newer Puppies and the like, could, I think, be put on the earlier filesystem, and should work (that had been tried with FD900Alpha, booting via the Grub2 in MX21.3, and it worked fine. I think it would be the same for Puppies etc..).

Is there any evidence that new Puppies are making ext4 partitions with this attribute?

For example, Gparted in the Starter Kit does not.

rcrsn51 wrote: ↑Thu Jul 13, 2023 1:43 pm

Is there any evidence that new Puppies are making ext4 partitions with this attribute?
For example, Gparted in the Starter Kit does not.

Puppies using e2fsprogs from 1.47.0 onwards, according to the release notes, released in Feb 2023. I'm not sure if there are any active Puppies with this version, but they will eventually arrive.

This feature __can__ be turned off by editing /etc/mke2fs.conf (something that I wasn't aware of when I spoke with baldronicus initially), however, I think baldronicus point is that there is no guarantee that it will be turned off by the distros. For example, the (once upon a time troublesome) 64bit-inode feature was once enabled in this way by the upstream e2fsprogs, and Gparted, rather than depending on distro developers to turn it off, put the task to themselves and __explicitily__ disabled 64bit-inode features when formatting ext4. Most probably because most distros didn't turn off the said feature.

However, I don't think Gparted is going to repeat it with this feature, hence, baldronicus advanced warning.

Okay, so I have a use case scenario:

My laptops boot and run from a single ext4 partition, and the boot loader was installed with grub4dos in fossapup64_9.5 and the partition formatted with the gparted version from 2020.

Among the OS's booting from these partitions are KLV-airedale, which is basically a frugal install of Void Linux, a rolling release, in which I do system wide updates.

Would a potential problem be that the older gparted formatted partition might be incompatible with newer attibutes that might be written by newer software in KLV-airedale?

and/or

Does a potential conflict stem from the possibility that using a newer version of gparted, say one updated in KLV-airedale may render an older grub4dos unable to read the partition? For instance if I formatted a new drive using gparted in KLV-airedale, then tried to run grub4doss in fossapup to install the bootloader.

From the linked post above I believe it is the latter that is the area of concern.

geo_c wrote: ↑Thu Jul 13, 2023 3:02 pm

Would a potential problem be that the older gparted formatted partition might be incompatible with newer attibutes that might be written by newer software in KLV-airedale?

No. Existing partitions are not affected, so they will continue to work, and boot, as usual. The offending feature will only be applied when you first created/format the filesystem.

Does a potential conflict stem from the possibility that using a newer version of gparted (to create a new partition), say one updated in KLV-airedale may render an older grub4dos unable to read the (newly created) partition? For instance if I formatted a new drive using gparted in KLV-airedale, then tried to run grub4doss in fossapup to install the bootloader.

Yes. I added the words in bold to make it clearer.

From the linked post above I believe it is the latter that is the area of concern.

Correct.

geo_c wrote: ↑Thu Jul 13, 2023 3:02 pm

Does a potential conflict stem from the possibility that using a newer version of gparted, say one updated in KLV-airedale may render an older grub4dos unable to read the partition? For instance if I formatted a new drive using gparted in KLV-airedale, then tried to run grub4doss in fossapup to install the bootloader.

Boot up KLV and look in /etc/mke2fs.conf. What attributes are enabled by default for ext4? Are any of these problematic csum attributes listed?

@jamesbond: If some future Puppy is built from a parent distro that has the problem ext4 attributes enabled by default AND that breaks a current Puppy bootloader when new ext4 partitions are created, can the problem be solved just by deleting the attribute from /etc/mke2fs.conf?

rcrsn51 wrote: ↑Thu Jul 13, 2023 3:28 pm

Boot up KLV and look in /etc/mke2fs.conf. What attributes are enabled by default for ext4? Are any of these problematic csum attributes listed?

Look out for these:
metadata_csum_seed is definitely problematic.
metadata_csum might be problematic if the bootloader is older.

@jamesbond: If some future Puppy is built from a parent distro that has the problem ext4 attributes enabled by default AND that breaks a current Puppy bootloader when new ext4 partitions are created, can the problem be solved just by deleting the attribute from /etc/mke2fs.conf?

Editing /etc/mke2fs.conf helps future partitions (when they are created), but not the ones already created before mk2fs.conf is changed.

To "fix" those, you can do run this: tune2fs -O ^metadata_csum_seed /dev/disk (you can similarly turn of metadata_csum as well in the same way). The filesystem must be unmounted, and often times you need to e2fsck them first. Of course, for this to work, the tune2fs needs to come from e2fsprogs version that already understand both. I didn't do extensive checks, but e2fsprogs 1.43.9 from 2018 already "knows" these features and can turn them off, so any Puppies within the last 5 years should be able to recover.

Thanks.

How did this problem arise in FatDog in the first place? Because you compiled e2fsprogs with a full set of default ext4 attributes?

rcrsn51 wrote: ↑Thu Jul 13, 2023 4:28 pm

How did this problem arise in FatDog in the first place? Because you compiled e2fsprogs with a full set of default ext4 attributes?

No special compilation options. It's plain vanilla build. The changes come from upstream since version e2fsprogs 1.47.0 onwards, as per the release notes I've linked for you in the previous post. Fatdog is probably the first one (among all other Puppies) which uses this version of e2fsprogs.

The Starter Kit has e2fsprogs 1.47.0 but NOT the metadata_csum_seed attribute. I'm guessing that the Debian 12 maintainers modified their /etc/mke2fs.conf.

I looked at VoidPup64-22.02-230701 and it DOES have metadata_csum_seed. I used it to make an ext4 partition which would NOT boot from GRUB2.

However, Grub4dos appears to be immune to the problem.

rcrsn51 wrote: ↑Fri Jul 14, 2023 8:02 am

The Starter Kit has e2fsprogs 1.47.0 but NOT the metadata_csum_seed attribute. I'm guessing that the Debian 12 maintainers modified their /etc/mke2fs.conf.

The default settings change was reverted in Debian 12 - see https://metadata.ftp-master.debian.org/ ... _changelog.

dimkr wrote: ↑Fri Jul 14, 2023 8:02 am

The default settings change was reverted in Debian 12 - see https://metadata.ftp-master.debian.org/ ... _changelog.

Thanks. If I manually add metadata_csum_seed using tune2fs, the partition becomes unbootable.