aufs future

Moderator: Forum moderators

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

LateAdopter wrote: Mon Oct 12, 2020 9:31 am Hello peebee
It looks like AUFS is in-tree for debian and ubuntu kernel git, so presumably the code gets updated automatically when it is rebased on mainline. There doesn't seem to be any noticeable delay since they built 5.8 in July.

Have you ever tried building a Puppy kernel from the Debian or Ubuntu kernel git rather than mainline? At least you know it's been properly tested.
No...........
Can't see how they can have any later version than the AUFS main git???
And do their kernels include AUFS? I can't see why they should as they don't need layering? and they'd be more likely to use overlayfs if they did....
Do you have links?

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

LateAdopter
Posts: 109
Joined: Sat Aug 15, 2020 5:10 pm
Been thanked: 17 times

Re: aufs future

Post by LateAdopter »

Hello peebee
Debian/Ubuntu fork the software they use and maintain it themselves. That includes the kernel. They can cherrypick commits and add their own.
Here are some links for Ubuntu:
https://kernel.ubuntu.com/git/ubuntu/unstable.git/
If you look in the master-5.8 tree it has root/fs/aufs and was finished 27th July
If you look in master it doesn't have aufs.

https://git.launchpad.net/~ubuntu-kerne ... aster-next
If you look here the latest 5.8 kernel for groovy has aufs

https://kernel.ubuntu.com/~kernel-ppa/config/
If you look at typical configs they have aufs=m

But Ubuntu don't do rolling release so they are unlikely to do a 5.9 branch.

Debian do rolling release for sid which is currently equivalent of about 5.8.14 and they are likely to do a 5.9. I can't make sense of their repository that uses gitlab. I think their kernels have aufs too, but I can't prove it. As far as I can decode it, they removed aufs at 5.9-rc4 and haven't put it back yet. Their source is currently at 5.9-rc8, I think.
I expect they will update to 5.9 stable soon.
User avatar
Duprate
Posts: 296
Joined: Sat Aug 22, 2020 8:14 pm
Location: Southern Brazil
Has thanked: 139 times
Been thanked: 97 times

Re: aufs future

Post by Duprate »

"Peebee wrote:
Kernel 5.9 has been released (not yet identified as LTS)
These altered VERY UNOFFICIAL files allow 5.9 to be apparently built successfully ...
Running with it now: Launch of
kernel: 5.9.0 -lxpup64
Compilation date: Mon 12 October 08:54:17 BST 2020 "

I'm testing your kernel on FatDog64 811 ... it works ... Firefox, OK ... Chromium, Chrome, Vivaldi, crash ... Wine, OK ... Wine + games, crash ... Unfortunately, for my knowledge, I can only test ...
Attachments
kernel.jpg
kernel.jpg (127.37 KiB) Viewed 3891 times
User avatar
Duprate
Posts: 296
Joined: Sat Aug 22, 2020 8:14 pm
Location: Southern Brazil
Has thanked: 139 times
Been thanked: 97 times

Re: aufs future

Post by Duprate »

Complementing the tests of the Kernel 5.9.0-lxpup64, which worked well: LibreOffice 6.3.3, Gimp 2.10.7, Firefox 81.0.2, Seamonkey 2.53.3, ClamAV-0.103.0, all of the ROXapps type;
Wine-3.3-V.2.1 + 32bit-fd64_811.sfs, Windows AppsPortables (The system freezes on heavier games).
What does not work (freezing) everything that is based on Chrome.

It's not a lack of memory, the PC has 8 GB. For an experimental kernel, the result is great! I like the peebee works. I am always using and testing your kernels.
User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

Duprate wrote: Sun Oct 18, 2020 12:59 am What does not work (freezing) everything that is based on Chrome.
No problems here with both chrome64 and chromium64 on fossapup64 with kernel 5.9.1
...
screenshot1.png
screenshot1.png (247.28 KiB) Viewed 3841 times
...
screenshot2.png
screenshot2.png (248.57 KiB) Viewed 3841 times

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

User avatar
Duprate
Posts: 296
Joined: Sat Aug 22, 2020 8:14 pm
Location: Southern Brazil
Has thanked: 139 times
Been thanked: 97 times

Re: aufs future

Post by Duprate »

It's okay if it doesn't work perfectly on FatDog64. Kernel-5.9.1-lxpup64, you did it for LxPupSc and, without the support of the developer, J. R. Okajima. We live in a difficult time. You did a great job!
I also tested the 5.9.1 kernel with Debian live 10.6, and the result was similar to FatDog64. In three attempts, in one Vivaldi took but it worked.
I will also test with FossaPup64 ...
Attachments
kernel-5.9.1.jpg
kernel-5.9.1.jpg (106.75 KiB) Viewed 3804 times
User avatar
fredx181
Posts: 2436
Joined: Tue Dec 03, 2019 1:49 pm
Location: holland
Has thanked: 250 times
Been thanked: 923 times
Contact:

Re: aufs future

Post by fredx181 »

peebee wrote: Mon Oct 12, 2020 6:14 am Kernel 5.9 has been released (not yet identified as LTS)
https://www.kernel.org/

These VERY UNOFFICIAL aufs changed files allow 5.9 to be built apparently successfully...
https://u.pcloud.link/publink/show?code ... 2kPYwrBWQk

Running with it now:
Kernel Release: 5.9.0-lxpup64
Build Date: Mon Oct 12 08:54:17 BST 2020
...
Thanks a lot for the changed files, successfully built kernel 5.9.1 with working aufs for DebianDog.
Normally I use existing Debian kernel (linux-image-*), build single aufs module (using dkms) and add it to the kernel, but for 5.9 it doesn't work anymore because Debian didn't apply the aufs patches for it (probably due to the on hold development of aufs, as they wouldn't work anyway), so had to build the kernel myself (which was a good learning experience for me :) ).
Shared the 5.9.1 kernel "packages" here: viewtopic.php?p=8266#p8266

Fred
User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

5.10 assistance required........

https://www.zdnet.com/article/linux-5-1 ... rity-bugs/ : "Torvalds drew attention to the removal of an addressing tool, called set_fs()"

this causes problems in https://github.com/sfjro/aufs5-standalo ... ufs/xino.c
e.g. "fs/aufs/xino.c:894:2: error: implicit declaration of function ‘set_fs’; did you mean ‘sget_fc’?"

Is there a simple replacement for set_fs and get_fs in xino.c ??

https://github.com/sfjro/aufs5-standalo ... -734782410

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

User avatar
fredx181
Posts: 2436
Joined: Tue Dec 03, 2019 1:49 pm
Location: holland
Has thanked: 250 times
Been thanked: 923 times
Contact:

Re: aufs future

Post by fredx181 »

Looks like aufs development will continue (hopefully soon):
https://github.com/sfjro/aufs5-standalo ... -734830521

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

Good news - an unofficial aufs patch for the next LTS Kernel 5.10 has been produced by J. R. Okajima....
see:
https://github.com/sfjro/aufs5-standalo ... -738004389

Last edited by peebee on Tue Dec 15, 2020 9:24 pm, edited 1 time in total.

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

User avatar
fredx181
Posts: 2436
Joined: Tue Dec 03, 2019 1:49 pm
Location: holland
Has thanked: 250 times
Been thanked: 923 times
Contact:

Re: aufs future

Post by fredx181 »

Thanks peebee ! That's great.

User avatar
Duprate
Posts: 296
Joined: Sat Aug 22, 2020 8:14 pm
Location: Southern Brazil
Has thanked: 139 times
Been thanked: 97 times

Re: aufs future

Post by Duprate »

Hi! It is certainly good news! We have used aufs for years, it is great, we got used to the facilities that aufs provides. Mr. J.R.Okajima, has been striving to maintain (alone?) this project for all these years and in return for his effort, Linus Torvalds refused to include it in the Kernel.
So, the future of aufs, may just be an over-life.
Overlayfs has been part of the kernel since version 3.18 (Debian Stretch). I'm using DebianDog Sid from fredx181, which uses Overlayfs and I'm really enjoying it. A very complete system, with the facilities that are the traditional mark of Puppy systems. I can compile the kernel from official sources (Kernel.org).
However, to compile a kernel with aufs support, I depend on a patch.
Okay, the aufs-5.10-changed-files.tar.xz patch has been released. What do I do with the pach?
I'm sure that other users like me also want the freedom to compile their own kernel and not just depend on the distro repository, which is not always up to date. Don't be offended, whoever uses Linux has the desire to learn ...
If we didn't want to evolve, we would use Windows ...

We should continue to use aufs but also give Overlayfs a chance!

(Sorry for any misunderstanding, I'm using Google Translate)

ozsouth
Posts: 1285
Joined: Sun Jul 12, 2020 2:38 am
Location: S.E. Australia
Has thanked: 193 times
Been thanked: 548 times

Re: aufs future

Post by ozsouth »

I think sfjro would be better off forgetting interim kernels (like 5.5 thru 5.9) and just concentrating on LTS (like 5.4 & 5.10).
Would save a lot of time & effort on his part & not interfere too much with puppy's progress.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

Good news........

Hello all,

I am going back to aufs development, and a new release will come early
in next year (aufs5.{8,9,10}). Also I am going to stop supporting
aufs4.19--aufs5.3 branches.
My new development base version is aufs5.4.

J. R. Okajima

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

Subject aufs5 GIT release (v5.{8,9,10})
From J. R. Okajima
Date Today 16:21
I'm back.
Thanks for your patience and sorry for you inconvenient.
Also I appriciate PB who has made his aufs patch for v5.{8,9,10}.
I hope I can keep my development pace as I had.

o news
- As previously announced, aufs4.19--aufs5.3 has ended.
The new development base is aufs5.4.3. Not aufs5.4, because NFSv3 ACL
is broken in linux-v5.4. (aufs5.4 is still supported)
- Since v5.8, proc_mounts.patch in aufs5-standalone.git became unnecessary.
- linux-v5.10 has removed an internal function 'set_fs()' (on some
architectures), and aufs had to follow that change. Also aufs removed
its f_op->read() and ->write() functions.

J. R. Okajima

----------------------------------------
- aufs5-linux.git#aufs5.4..aufs5.7
aufs: minor, pseudo keyword 'fallthrough'
aufs: minor, MAINTAINERS
aufs: cosmetic, fix a comment

- aufs5-linux.git#aufs5.8
Addition to above,
aufs: for v5.8-rc1, a_ops->readahead

- aufs5-linux.git#aufs5.9
Addition to above,
aufs: for linux-v5.9-rc1, arg of ->handle_event()

- aufs5-linux.git#aufs5.x-rcN
Addition to above,
aufs: linux-v5.10-rc1, no more f_op->read() and ->write()
aufs: linux-v5.10-rc1, no more set_fs()
for aufs: linux-v5.10-rc1, no more vfs_(read|write)f_t

- aufs5-standalone.git
Addition to above,
aufs: unnecessary proc_mounts.patch for linux-v5.8 and later

- aufs-util.git#aufs5.8..aufs5.x-rcN
Revert "minor, make-var ProcMounts_Times"
Revert "minor, check AUFS_SUPER_MAGIC"
Revert "au_proc_getmntent: optimize reading, retry"

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

LateAdopter
Posts: 109
Joined: Sat Aug 15, 2020 5:10 pm
Been thanked: 17 times

Re: aufs future

Post by LateAdopter »

Hello peebee
Thanks for tracking this...
The version of kernel-kit that I use needs the new aufs-util 5.8 branch added to funcs.sh manually. I don't know scripting so I can't say whether the new 5.x-rcN would also work, but I'm unlikely to need that.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

LateAdopter wrote: Mon Jan 04, 2021 9:33 am

The version of kernel-kit that I use needs the new aufs-util 5.8 branch added to funcs.sh manually.

Thanks - I've added aufs-util 5.8 to Woof-CE.

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

ozsouth
Posts: 1285
Joined: Sun Jul 12, 2020 2:38 am
Location: S.E. Australia
Has thanked: 193 times
Been thanked: 548 times

Re: aufs future

Post by ozsouth »

Thanks Peebee - have now built 5.8.18 64bit - works well - new AMD drivers help with my e2-9000e pc.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

Hopefully temporary problem......

I have raised a github issue as the patch files are missing in the 5.10 branch:
https://github.com/sfjro/aufs5-standalone/issues/3

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

User avatar
rockedge
Site Admin
Posts: 5494
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1784 times
Been thanked: 1961 times
Contact:

Re: aufs future

Post by rockedge »

Yes I thought something was missing. I kept running across a missing patch file while attempting to compile a 5.10.1 real time kernel with full virtualization support enabled. Though worth a mention is the real time patches encountered errors and did not complete either.

Only tried using the kernel kit.

gyrog
Posts: 583
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 170 times
Contact:

Re: aufs future

Post by gyrog »

Given the "Subject" of this topic, I decided to reply.

Can we have an overlayfs based Puppy instead of an aufs based Puppy?
Yes,
I've been running such a beast for a couple of years now.
My current daily workhorse is XenialPup 7.5 from 'mio22u_up_xenialpup_7.5.sfs', i.e. using overlayfs for it's stack.

BUT,
there are significant differences between overlayfs and aufs, these mean some things have to change.

1. Once an overlayfs stack has been mounted, it's structure can't be changed.
Accordinng to the kernel documentation, the result is undetermined.
(This is the most annoying, because I seem to end up rebooting a lot, but maybe that's due to the stuff I do,
if you spend all your time running ordinay applications, maybe it wouldn't matter so much.)

'sfs_load' can't append an extra layer to the stack, or remove a layer from the stack.
'sfs-load' is reduced to simply managing the list of extra sfs's that get built into the stack at boot.
(remounting the stack with a differrent set of layers changes nothing.)

'snapmergepuppy' cannot work at all, so the current "odd" pupmodes can't work.

'init' can't add layers one by one, the stack has to be created in one go,
although you can have sub-stacks, so my current overlayfs puppy has a read-only stack containing only the sfs files,
the running stack has only 2 layers, the RW layer and the sfs stack.
So the 'init' script has to be significantly rewritten as a minimum.

2. An overlayfs RW layer requires 2 directories "on the same filesystem". The directory containing the layer, and a "work" directory.
An aufs RW layer requires only the directory containing the layer.

The current 'savefile' mechanism uses the mount-point of the savefile as the single directory to add to the stack.
There remains an issue as to where to create the "work" directory when using a savefile as an RW layer for an overlayfs stack.
I never found a solution that would reliably shutdown cleanly.

But of course there's more than 1 way to skin a cat.
For instance, in my overlayfs Puppy, I can run my pupmode=12 in a "save query" way, i.e. on shutdown it will ask if I want to save or not, and won't if I select "no".
When running this way, the 'init' script creates an archive of the RW layer directory if there is no archive;
If there is an archive it clobbers the RW directory, with the archive contents;
At shutdown, if I ask for a save the archive is deleted, if I ask for no save, the archive is not touched and all changes to the RW directory are clobbered at the next boot.
(From ny browsing of the forum, some folk run pupmode=13 so they can have "save query", but pupmode=13 going doesn't mean "save query" has to go.)

So, we should stick with aufs as long as we can.
But, the end of aufs, does not have to mean the end of Puppy, an overlayfs based Puppy is doable.

gyrog

gyrog
Posts: 583
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 170 times
Contact:

Re: aufs future

Post by gyrog »

If we are to have a Puppy without "aufs", then it's a Puppy without 'sfs_load' as we know it, i.e. being able to dynamically add a large application to Puppy, inclduing integration into the Puppy menu, try it, and then keep it, or remove it from Puppy and it's menu.

I have established 1 possible way of doing the same thing without 'sfs_load', i.e. use "portable" applications, e.g. ".AppImage" files,
plus a utility like my "PortableActivate" to add the "portable" applications to the Puppy menu, (and as a CLI program in the $PATH).
And it does this with a very minimal footprint on the Puppy "save".

Oh, it works just as well in my "overlays" Puppy, as it does in a "normal" "aufs" Puppy.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

aufs-5.13 available

Post by peebee »

o news
- linux-v5.13 is released, and aufs-5.13 follows it.

no new commits

J. R. Okajima

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

gyrog
Posts: 583
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 170 times
Contact:

Re: aufs future

Post by gyrog »

@peebee Good news.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

Re: aufs future

Post by peebee »

o news
- for linux-v5.14-rc1, replace kmalloc_index() by __kmalloc_index()
aufs5.x-rcN branch supports linux-v5.14-rcN, and it simply follows the
changes in mainline.

My current development base version is linux-v5.4 which is marked as
"longterm" on wwe.kernel.org. When linux-v5.14 is released, I will move
my development base version to linux-v5.10 which is also longterm.
In other words, aufs-5.4..aufs5.9 will end.

J. R. Okajima

----------------------------------------
- aufs5-linux.git#aufs5.4..aufs5.13
nothing

- aufs5-linux.git#aufs5.x-rcN
aufs: for linux-v5.14-rc1, replace kmalloc_index() by __kmalloc_index()

- aufs5-standalone.git
ditto

- aufs-util.git
nothing

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

ozsouth
Posts: 1285
Joined: Sun Jul 12, 2020 2:38 am
Location: S.E. Australia
Has thanked: 193 times
Been thanked: 548 times

Re: aufs future

Post by ozsouth »

I assume he means development/updates will end, as K 5.0 - 5.3 aufs is currently still available. Could then still build 5.4, although I'm now using 5.10.50.

User avatar
peebee
Posts: 1429
Joined: Mon Jul 13, 2020 10:54 am
Location: Worcestershire, UK
Has thanked: 143 times
Been thanked: 564 times
Contact:

aufs-5.14 released

Post by peebee »

o news
- linux-v5.14 is released, and new branch "aufs5.14" too

o misc.
- minor, update the copyright year
I just simply forgot it.

As I announced on 19 Jul, aufs5.4..aufs5.9 end. They are not supported
anymore. My new development base version 5.10.

J. R. Okajima

----------------------------------------
- aufs5-linux.git
aufs: minor, update the copyright year

- aufs5-standalone.git
ditto

- aufs-util.git
nothing

Builder of LxPups, SPups, UPup32s, VoidPups; LXDE, LXQt, Xfce addons; Chromium, Firefox etc. sfs; & Kernels

linuxunix
Posts: 99
Joined: Fri Aug 14, 2020 4:34 am
Has thanked: 4 times
Been thanked: 2 times

Re: aufs future

Post by linuxunix »

https://sourceforge.net/projects/lxpup/ ... 2/download

huge-5.14.2

Virtualbox to download there?

User avatar
rockedge
Site Admin
Posts: 5494
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1784 times
Been thanked: 1961 times
Contact:

Re: aufs future

Post by rockedge »

I just built a huge 5.14.1. Have not tested it yet. Perhaps I will patch it for and re compile for a full real time kernel

Post Reply

Return to “Kernels”