aufs future

[peebee]

o news
- linux-v6.0 is released and aufs5.x-rcN branch supports it.
- note that core file produced by linux-v6.0 is broken. the bug is fixed
just after linux-v6.0 is released.
- git repositories for aufs6 will appear in a week or two.

J. R. Okajima

[peebee]

"J. R. Okajima":
> - git repositories for aufs6 will appear in a week or two.

I was going to create new repositores for aufs6 on github, but I
couldn't. aufs5-linux.git on github is a fork of torvalds/linux.git,
and github doesn't allow me to create a new fork of the repository which
I already made a fork.
Obviously I could make a brand new empty aufs6-linux.git repo, but in
this case I had to push all git-objects in mainline. That is
unrealistic.
So I decided to rename aufs5-linux.git and aufs5-standalone.git on
github to aufs-linux.git and aufs-standalone.git respectively. And they
will have the branches for aufs6 series.

If you have a clone (or a fork) of aufs5 repositories, please rename
them to non-versioned name. I guess .git/config is the place.

[jamesbond]

Great news
The news of aufs demise is greatly exaggerated

[ozsouth]

So I decided to rename aufs5-linux.git and aufs5-standalone.git on
github to aufs-linux.git and aufs-standalone.git respectively. And they
will have the branches for aufs6 series.

That breaks my old kernel-kit 2018 builds.sh & buildu.sh. Anyone using them (I think I've sent copies to 4 folk),
will need to manually change aufs5-standalone to aufs-standalone in line 193 to reflect change.

AND - my scripts don't allow for 6.x kernel download - I think adding |6.* to line 180, then
aufs_6 line (like aufs5 line 193) after current line 193, & ksubdir_6 entries after current lines 197 & 303 fix it.

Also, you'll now get an error & just 2 choices for firmware - I choose option 1 (copy in my own), then don't copy
in when asked near end, but use an fdrv.

*** OTHER OPTION - scrap it & use more modern methods (OR I could use my overlay-only kit with 6.x option).

[dimkr]

@ozsouth Why from 2018 and not the latest?

[rockedge]

AUFS is cool. I like AUFS better than Overlayfs. But as of late been having trouble getting AUFS patches from Git since it's always asking for my Git user name and token then refusing access. So after X tries I switched to overlayfs and/or been grabbing Puppy kernels from dimkr's workflow artifacts.

[ozsouth]

@dimkr - I had a working system, so didn't see the need to change. I've looked at peebee's current-kernel-kit output. I've made the updates I suggested earlier & compiled 5.15.74 aufs/overlay - is OK. Then made 6.0.3 overlay only - is OK. One day (soon?) I won't be able to fix this, or my output will fail. Then I'll modernise, if I think I can. The debian kernel conversion idea interests me too.
@rockedge - having played with overlayfs options, I can see good features in aufs, but am still somewhat concerned about it's future. That kernel kit I sent you today now has the fixes I detailed earlier.

[peebee]

o news
- aufs5-linux.git and aufs5-standalone.git on github are renamed to
aufs-linux.git and aufs-standalone.git respectively.
- for each repo, aufs6.0 branch is released.

J. R. Okajima

[peebee]

- Master branch in github.com/aufs-linux.git is updated to
linux-v6.1-rc2.
- aufs6.x-rcN branch is for linux-v6.1-rc2.

J. R. Okajima

----------------------------------------
- aufs6-linux.git#aufs6.0
aufs: remove unnecessary debug code

- aufs6-linux.git#aufs6.x-rcN
Addition to above,
aufs: for v6.1-rc1, O_TMPFILE
aufs: for v6.1-rc1, the type of filldir_t
aufs: version 6.x-rcN

- aufs6-standalone.git
ditto

[peebee]

J. R. Okajima has released aufs-6.1

[peebee]

J. R. Okajima has released aufs-6.2

[peebee]

BarryK has posted some thoughts on overlayfs............
https://bkhome.org/news/202304/overlayf ... hetic.html

[ozsouth]

In s15pup64-22.12+3, just performed the operation Barry had trouble with - no issues. Note, I don't use the extra overlayfs settings he did (just overlayfs=y). Possibly, EasyOS is significantly different from s15pup - I haven't tried it for some time.
The only operational shortcoming I have is that sfs-load is OK for viewing, but if loading is attempted, will mount sfs files on /mnt (requiring manual unmounting) & not make files available to the filesystem. I have rewritten this to not allow mounting (with a warning), & extra sfs's have to be loaded on bootup (see viewtopic.php?p=75624#p75624).
I also rewrote init to allow b,c,d,e drives to aid this shortcoming (see viewtopic.php?p=74198#p74198).
The stray containers issue has no real impact. I can use savefolders or savefiles without issue.

[amethyst]

extra sfs's have to be loaded on bootup

I would say that is a massive disadvantage.

[ozsouth]

@amethyst - yes, for some folk that is definitely a show-stopper, which is why I point that out constantly, but also made a mitigation. Without that b,c,d,e mitigation, folk would basically rely on portables, appimages, .pets etc. That is definitely not for folk who want to load a mass of sfs's. My main point was that I'm not having the disaster Barry experienced.

[BarryK]

In EasyOS, the layers are:

RW: ext2 f.s. in /dev/zram1
RO: ext4 f.s.
RO: easy.sfs mounted on a folder

So I guess, perhaps, overlayfs doesn't like the compressed ram.

The init script creates a ext2 f.s. in the zram, then some folders in it, and mounts one of the folders as the RW layer.

I absolutely must use zram, it gives so much extra ram space to work with.

[dimkr]

extra sfs's have to be loaded on bootup

I would say that is a massive disadvantage.

Limited support for SFS loading after boot time exists since https://github.com/puppylinux-woof-CE/woof-CE/pull/3723. It supports only PUPMODE 5 and 13 (not 12), because it relies on symlinks in the RAM layer.

(Theoretically, it's possible to add support for PUPMODE 12 as well, but there must be some "garbage collection" of broken symlinks from the save layer to dynamically-loaded SFSs, after power failure or unclean shutdown.)

[rockedge]

but there must be some "garbage collection" of broken symlinks from the save layer to dynamically-loaded SFSs, after power failure or unclean shutdown.

In KLV-Airedale we use a script that runs at boot that cleans up dangling symlinks which so far has worked reliably though not thoroughly tested for all of the potential scenarios of crashed, or shut down systems with SFS's loaded on the fly. So it is not fully understood or known what can go wrong.

The SFS's loaded at boot behave and are handled differently.

[Grey]

some thoughts on overlayfs............

Well, my opinion and the opinion of Russian developer(s) have already been voiced: viewtopic.php?p=79993#p79993

[ozsouth]

J.R.O. has released aufs-6.3

[peebee]

J. R. Okajima has announced that he will not be providing any further updates to aufs-5.10 and aufs-5.15.

This does not mean that aufs builds of 5.10 and 5.15 will stop working - only that if upstream changes to kernels 5.10 and 5.15 invalidate the aufs patches he won't develop further updates. Depending on the incompatibility we may be able to create local updates as was done in the past for 5.4.

He will concentrate on 6.1 and later versions for the future. The latest aufs release is 6.3 but 6.4 and 6.5 should appear shortly.

[peebee]

aufs-6.4 and aufs-6.5 have been released by J. R. Okajima:
https://github.com/sfjro/aufs-standalone/branches

[peebee]

There seems to be an aufs problem with kernel 6.6.4
https://github.com/sfjro/aufs-standalone/issues/32

[dimkr]

Currently I am busy and have no time for aufs.

More and more red flags

[ozsouth]

I tried a suggestion by sfjro on github (although not specific to my issue) - modprobe.blacklist=LSM . Still kernel panic. As this could be caused by a kernel change, later today I'll build a 6.1.65 kernel (released same day as 6.6.4) to see if we have a downstream issue with aufs.

[rockedge]

@ozsouth I have also successfully built 6.6.4 and also seeing a kernel panic during the root switch to the AUFS file system. Used the latest kernel-kit on this build.

Using your 2018 kernel-kit I just did a 6.6.0-rt15 patched with AUFS 6.6 and the full real time patch for 6.6.0-rt15, which boots successfully on F96-CE_4, KLV-Airedale-RT and KLV-Spectr-RT

[ozsouth]

I successfully made a 6.1.65 kernel - briefly tested - ok, so no downstream issue, but 6.6.4 (onwards?) remains problematic with aufs. sfjro (our aufs champion, who does this for free & has a busy life) has tried to make a patch for another 6.6.4 issue & will get to our issue eventually, so Puppy 6.6 aufs kernels will have to stay at 6.6.3 in the interim. Fortunately, 6.1 aufs & all overlayfs kernels are not affected.

[ozsouth]

Seems like a solution is in the pipeline - a kernel commit to /fs/stat.c broke aufs, & aufs6.6.4 will appear soon. Meanwhile, stat.c from 6.6.3 is to be used.

[ozsouth]

I successfully made a 6.6.5 aufs/overlayfs kernel using stat.c from 6.6.3. Seems fine, but I won't release it publicly. Will wait for aufs6.6.4 release & maybe make a kernel in a week or 2.

[ozsouth]

More discussion on github re- aufs 6.6.4+ failure:

The Porteus dev who discovered the /fs/stat.c problem has found that "udba=none" in their initrd mount code also fixes the kernel panic. Most (all?) Puppies have udba=reval twice in the init script. I checked that bookwormpup64_10.0.3 & s15pup64-22.12+4 do.
Posting this from the latter - boots 6.6.4 with the change - no obvious issues, but not sure what side-effects this change may cause.

From the geekstuff aufs examples:

The following are possible values for udba:

Code: Select all

udba=none – With this option, the AuFS will be faster, but may show incorrect data, if the user created any files/directories without going through the AuFS.
udba=reval – With this option, the AuFS will re-lookup the branches and update it. So any changes done on any directories within the branch, will be reflected in “/common”.
udba=notify – With this option, the AuFS will register for inotify for all the directories in the branches. This affects the performance of AuFS.

Upon reading the above, don't think I'll pursue it, even though I did create a directory and a file with no errors.