What is 'usrmerge'?

[Chelsea80]

What is usrmerge?

What is different from non-usrmerge?

@peebee's NoblePup32

This is a USRMERGE build so only usrmerge compatible .sfs can be installed.

How can you tell if an .sfs is usrmerge or not?

Is it better to use a usrmerge OS?

Thanks.

[TerryH]

In usrmerge the directories /bin, /lib, /lib64, /sbin and also /usr/lib64 and /usr/sbin are symlinks to /usr/bin and /usr/lib. so a directory that would in a non-usrmerge that would exist in /lib, i.e. /lib/firmware is in /usr/lib, the entry which is in /lib is a symlink to /usr/lib/firmware.

The only way to tell the difference is to look at the structure within the .sfs. The naming convention did not change to identify the difference.

I haven't tried it, but I think you could possibly use a usrmerge sfs in a non-usrmerge installation, but not the other way.

The only reason it might be considered better is due to usrmerge being a newer release.

[Chelsea80]

@TerryH

Thanks for your reply.

In usrmerge the directories /bin, /lib, lib64, /sbin and also /usr/lib64 and /usr/sbin are symlinks to /usr/bin and /usr/lib. so a directory that would in a non-usrmerge that would exist in /lib, i.e /lib/firmware is in /usr/lib, the entry which is in /lib is a symlink to /usr/lib/firmware.

I have been trying to, slowly, work out your explanation. So it seems to me (I'm no techy) that the usrmerge calls the non-usermerge that calls what is needed. If so, why the duplication. If not, is this a faster or safer way to get to the same end-point? Or am I 'barking' up the wrong tree?

The only way to tell the difference is to look at the structure within the .sfs. The naming convention did not change to identify the difference.

So a 'basic user' like me could screw things up by not knowing if an .sfs is or is not compatable.

I haven't tried it, but I think you could possibly use a usrmerge sfs in a non-usrmerge installation, but not the other way.

It might be worthwhile if not already done a full and guided explanation was put somewhere on the Forum for those that haven't a clue (meaning me).

In the meantime, again, thanks for your reply.

[Chelsea80]

@TerryH

Follow up:

Downloaded NoblePup32 (usrmerge), installed to a USB Stick 4GB with UNetbootin. Created Save File 1GB.

Then:

Applications > Internet > Get Web Browser > Firefox

I am posting from it now.

I haven't tried it, but I think you could possibly use a usrmerge sfs in a non-usrmerge installation, but not the other way.

So unless the Firefox.sfs is a usrmerge .sfs then it looks like it's a two way street.

Except that @peebee says a usrmerge .sfs must be used

Confused? You bet I am.

[ozsouth]

@Chelsea80 - To hopefully simplify, the general key factor is - Does the SFS contain ANY of /bin , /lib , /lib32 , /lib64 , /sbin folders?
(You can click on SFS in File manager & check). If yes, sfs cannot be used in usrmerge. If no, it can. My Firefox.sfs use /opt & /usr/local/bin & thus can be used in either. This is the downside of deriving puppies from upstream sources - we have no control over some policies.
Currently, Jammypup, Bookwormpup, Vanilladpup & Noblepup are our usrmerge pups.

[Chelsea80]

@ozsouth

Thanks for joining in.

So, had a look in Firefox.sfs and it shows:

Screenshot 1.jpg (13.76 KiB) Viewed 2836 times

And

Screenshot 2.jpg (14.5 KiB) Viewed 2836 times

To hopefully simplify, the general key factor is - Does the SFS contain ANY of /bin , /lib , /lib32 , /lib64 , /sbin folders?
(You can click on SFS in File manager & check). If yes, sfs cannot be used in usrmerge. If no, it can.

As you can see the Firefox.sfs does contain bin and lib folders. This means Firefox shouldn't work, right? But I am posting from it now.

You and @TerryH have both tried to help me, thanks for that .

This is getting way beyond my understanding so I am going to call it a day and try not to be so inquisitive

I'll stick to being a key pusher, I know my place in life.

[ozsouth]

@Chelsea80 - the bin & lib in your example are under the /usr folder, so that is fine for usrmerge.
What is not fine for usrmerge, is if they are in the main primary folder ( / ).

[Chelsea80]

@ozsouth

the bin & lib in your example are under the /usr folder, so that is fine for usrmerge.
What is not fine for usrmerge, is if they are in the main primary folder ( / )

This is what is in the ( / )

Screenshot.png (45.54 KiB) Viewed 2820 times

.
I see bin lib. and sbin. Knowledge and experience is not on my side.

[peebee]

Hi @Chelsea80

Sorry if my warning has caused concern!

If you attempt to install an incompatible .sfs then there are checks in place which will warn you and stop the .sfs being loaded.

So you don't need to worry unless your favourite .sfs fails to load.....

We have to make this change because all the "big boys" (but not Slackware as yet) have decided this is the way to go.....
https://wiki.debian.org/UsrMerge
https://www.freedesktop.org/wiki/Softwa ... eUsrMerge/
https://fedoraproject.org/wiki/Features/UsrMove

[ozsouth]

@Chelsea80 - ah - I re-read your initial post - you were ENQUIRING about Noblepup, not actually running it, whilst you use bionicpup (non-usrmerge). Your Firefox SFS (above) should work in both.

[Chelsea80]

@peebee

Thank you for the clarification. More happier knowing a fail safe system is in place.

So in effect 'we' are being forced to follow the

"big boys"

whether 'we' like it or not.

Not withstanding that progress is inevitable in all aspects of life, it seems a shame that the original concept of Puppy is being eroded.

Surely the developers must be pulling there hair out with having to make yet another modification.

Speaking as a user only:

There is so much talent, knowledge and experience in our community that I wonder why 'we' just can't go our own sweet way in everything.

Do we really need to play catch-up all the time?

OK, I might be talking out of my a*s* here but again I say, with the abundance of skill that is available on this Forum .....

I thought Windoz was bad enough with it's never ending forced up-dates.

I expect to be hammered for this post but my declared interest is that I am really happy with Puppy, if not then I would follow the herd.

[peebee]

Do we really need to play catch-up all the time?

It is not possible to build a non-usrmerge Pup using the usrmerge components from one of the usrmerge distros..... so we either have to go usrmerge or stop building Pups from those distros (i.e. Debian and Ubuntu in particular).

[mikewalsh]

@Chelsea80 :-

It boils down to one thing. Everybody else in the Linux eco-sphere has decided this is the way to go (once again, meekly following RedHat's decision). Because of this, in a few years time, you won't be able to download one piece of open-source software from anywhere that isn't written to work with usrmerge.

A few years after that, you won't even see the term 'usrmerge' used any more.....because it will have become "the norm".

So, by deciding to try and stick with the 'old' way of doing things, we would put ourselves in a position where we would need to re-write every piece of software before the community could use it. That's crazy; can you now see WHY we're simply following everybody else?

If we don't, we'll be generating an awful lot of unnecessary work.....and making life far harder than it needs to be.

They call this progress, y'know? Sad.....but true.

(There's a small core of developers at Red Hat that seem as though they would like nothing better than to turn Linux into a clone of Windows. They really want to get rid of all Linux distros - leaving just RedHat - OR, force everybody else to knuckle under and combine into one 'Linux' distro under RedHat's overall control. All these daft decisions in recent years; PulseAudio, systemd, PipeWire, etc, etc, are just small steps towards the culmination of the eventual grand master plan....)

Personally, I hope it never happens.....although a certain amount of rationalisation is always going to be necessary for the benefit of all.

Mike.

[dimkr]

The vast majority of packages that contain executables place them in /usr/bin, and not /bin, /sbin or /usr/sbin.

This is why most major distros unified /bin and /usr/bin by moving the few files there to /usr/bin and making /bin a link to /usr/bin. This simplifies the file system layout, improves compatibility between distros (if /bin and /usr/bin have the same contents because one points to the other, an application can use either /bin/grep or /usr/bin/grep without some distros having to patch it) and improves performance (because only one directory needs to be scanned when searching for an executable). I've never seen a good argument for the traditional layout, with separate /bin and /usr/bin, except the advantage of backward compatibility with very few packages.

You'll hit compatibility problems in a usrmerge system only if you try to load a SFS that places files under /bin, /sbin, /lib, etc', rather than their /usr counterparts. They should be rare or just 'borrowed' from an old Puppy that's incompatible for other reasons as well.

(In addition, the distinction between /usr/bin and /usr/sbin has no practical uses these days and it's nothing but an historical implementation detail - just like the / vs. /usr split, so Fedora plans to unify /usr/bin and /usr/sbin.)

The /usr/lib vs. /usr/lib64 vs. /usr/lib/x86_64-linux-gnu split is a different matter: Debian and derivatives put libraries under /usr/lib/x86_64-linux-gnu instead of the other two so one Debian installation can contain libraries that target multiple architectures: for example, a 64-bit Debian with a 32-bit Wine that uses 32-bit libraries from /usr/lib/i686-linux-gnu. When the libraries for each architecture sit in a different directory, they don't conflict. Sometimes, people who whine about usrmerge and how it breaks their old packages actually talk about this thing (which doesn't break packages unless you have a misconfigured ld.so.conf) and complain about the cosmetic impact on file paths.

[ozsouth]

@mikewalsh - for those who don't like the idea of usrmerge, there is always Slackware - but for puppy, that necessitates building/rebuilding compliant packages, which will only become more difficult as time goes on as folk here like modern packages. Slackware has a fairly limited set of packages, even with alien bob's contributions. I've managed to convert a (very) few of your portables for s15pup64, but dependency hunting is a pain.
I'm also hoping that /usr/sbin -> /usr/bin will be the end of major structural changes.

Also, though s15pup may not be popular amongst regular forum contributors, look at the download stats - 295 downloads in 8 days.
.

[Chelsea80]

@peebee @mikewalsh @dimkr

Ok, thank you for your posts and from your information I have been enlightened somewhat.

I did say

I might be talking out of my a*s*

So the inevitable will eventually happen, also throw in the mix 64bit .

Then I guess that all the lovely 'old' Puppys will not be worked on any more, shame really.

Just one more thing for me to do. Put the soap-box back in the cupboard.

[rockedge]

@Chelsea80 A good visual comparision between a KLV-Airedale (usrmerge) and F96-CE_4 non-usrmerge:

KLV-Airedale with usrmerge

Screenshot(2).jpg (24.1 KiB) Viewed 2708 times

F96-CE_4 with non-usrmerge

Screenshot(1).jpg (21.41 KiB) Viewed 2708 times

Most important is that in non-usrmerge the kernel modules are in /lib/modules and in usrmerge in /usr/lib/modules and the same pattern for the kernel firmware.

[dimkr]

So the inevitable will eventually happen,

What do you mean by "inevitable"? Why describe a relatively harmless change in the file system layout as a big disaster?

also throw in the mix 64bit .

It is possible to build a 32-bit Puppy with the usrmerge layout, but expect problems (but not because of usrmerge). Nobody is testing 32-bit kernels anymore, 32-bit kernels lack many security features and mitigations for known issues, many things are broken and the entire graphics stack seems to drop support for very old (<2008? not sure) GPUs. If your CPU is 64-bit capable, the difference in resource consumption should be negligible and that's pretty much the only reason not to switch to a 64-bit Puppy.

[fredx181]

...
Then I guess that all the lovely 'old' Puppys will not be worked on any more, shame really.
...

I wouldn't know why not, but perhaps I'm missing something.
edit: I mean that these older Puppys don't depend on the newest developments IMO.

Btw, I'm a bit disappointed with implementing the usrmerged system in (newer) Puppy so soon while it still should be possible to build a working non-usrmerged system.
(for example; Devuan still supports --no-usrmerge when building a system with debootstrap, (e.g. with mklive-daedalus), no problems for me running it too, e.g. no problems installing packages with apt).
But ok, probably there comes a time that no-usrmerge will become really problematic, but not yet IMO.

[mikewalsh]

@Chelsea80 :-

The trouble is that tech is a very fast-moving landscape. As soon as new tech appears - whether hardware or software - there's a hard-core of individuals out there who immediately find ways to convert it / compromise it for malicious uses. So this is why it makes sense, really, to always stick with the very newest OSs - or at least, long-term support releases that still receive patches & security fixes to mitigate all this crap......because dev teams always tend to concentrate on fixes for the newest stuff, since this is what the highest proportion of computer users tend to want all the time.

I, too, like older tech. It sticks in my craw that older gear/software has to be abandoned because of these twats that are utterly convinced they have a God-given right to line their nests at everybody else's expense, but unfortunately, this is the way the tech landscape will always be. We have to work with it as best we can.

Sod's law, mate.

Mike.

[Marv]

...
Then I guess that all the lovely 'old' Puppys will not be worked on any more, shame really.
...

I -a pretty old dog- use, support, and work on both structures pretty much at the same time in my kennel. Structure adjusting kernels is simple and @jrb did some nice work on structure adjusting scripts, both for kernels and for legacy pets, in his jammypup64 work. There has always been some library structure variability in the 64b OS, slackware vs ubuntu for example. Unfortunate there, but not too hard to keep straight.

Guess I'd say don't ignore change, but also don't write off the well polished legacy pups.

My 2 cents worth,

[fredx181]

The trouble is that tech is a very fast-moving landscape. As soon as new tech appears - whether hardware or software - there's a hard-core of individuals out there who immediately find ways to convert it / compromise it for malicious uses. So this is why it makes sense, really, to always stick with the very newest OSs - or at least, long-term support releases that still receive patches & security fixes to mitigate all this crap......because dev teams always tend to concentrate on fixes for the newest stuff, since this is what the highest proportion of computer users tend to want all the time.

I, too, like older tech. It sticks in my craw that older gear/software has to be abandoned because of these twats that are utterly convinced they have a God-given right to line their nests at everybody else's expense, but unfortunately, this is the way the tech landscape will always be. We have to work with it as best we can.

Sod's law, mate.

Ok, but Puppy succeeded well so far in avoiding the use of systemd, so possibly it can also avoid also the other "mainstream" pushed developments (well.. yes, probably not easy, I guess specially if depending on stuff from Ubuntu or Debian).
EDIT: Example also is GTK2, can we still use it in the future ? Well... probably not, it's replaced now/soon by GTK3/GTK4 ... WTF ! I want my GTK2 !!! (sorry for the rant )

[mikewalsh]

@fredx181 :-

^^^

Touché, mon ami; touché....

Mike.

[dimkr]

But ok, probably there comes a time that no-usrmerge will become really problematic, but not yet IMO.

It's already unsupported in Debian testing, you get a more and more broken Puppy as packages adopt DEP17 to prepare for mandatory usrmerge in Debian 13. The right time to add usrmerge support in woof-CE and adjust the few incompatible packages is now.

I think that building Puppy with usrmerge, that drops compatibility with very few packages, is much better than having no Puppy release now, or when Debian 13 is out. Same with X.Org, better migrate away from GTK+ 2 and other X.Org-only things before they're gone for good.

[fredx181]

But ok, probably there comes a time that no-usrmerge will become really problematic, but not yet IMO.

It's already unsupported in Debian testing, you get a more and more broken Puppy as packages adopt DEP17 to prepare for mandatory usrmerge in Debian 13. The right time to add usrmerge support in woof-CE and adjust the few incompatible packages is now.

I think that building Puppy with usrmerge, that drops compatibility with very few packages, is much better than having no Puppy release now, or when Debian 13 is out. Same with X.Org, better migrate away from GTK+ 2 and other X.Org-only things before they're gone for good.

Alright, great of course that you look forwards into the future, however with "not yet" I just meant to say that about Debian 12, as BookwormPup is based on.

[bigpup]

Seems to me the usrmerge file system is setup to work with older programs and code.

It has a bunch of older ways of using directories, symbolically linked to the newer directory locations.

Isn't this just about changing the location of stuff in the Linux file system?

But using directory symbolic links for the old directory locations that point to the new directory locations?

So if some program code is looking for something that was in the old directory location, it will be symbolically linked to find it in the new directory location?

Seems that would be what happens if you install an older program.

It is trying to install something in the older directory location, but that location is now only symbolically linked to the new directory location.
That older directory no longer a usable directory. It is only a symbolic link to the actual directory being used by the file system.
So it is actually put in the new directory location.
You can not install into a symbolic link of a directory.
The directory symbolic link just points to where the actual directory is.

Puppy has been doing this for years, using directory symbolic links to point to the actually used directory locations in the Puppy file system.
Had to do that, when the Puppy file system layout, was way different, from the normally setup Linux file system.

[dimkr]

But using directory symbolic links for the old directory locations that point to the new directory locations?

Yes. Some things (e.g. build systems) hardcode paths like /usr/bin/grep and some pre-usrmerge distros had to do ugly things like a /usr/bin/grep->/bin/grep symlink, so they don't need to patch all the things that assume /usr/bin/grep (or the other way around, each distro decided what to place in /bin instead of /usr/bin). When /bin is a symlink to /usr/bin, a script can assume either /bin/grep or /usr/bin/grep, so usrmerge increases compatibility between distros, or between applications and distros.

usrmerge breaks only one thing, and that's SFSs that contain /bin, /sbin, etc', because of the way aufs and overlay work (small price to pay compared to all the things that start to work or work more efficiently thanks to usrmerge). If they see a directory at /bin in a layer above the main SFS, /bin becomes a directory even if it's a symlink in the main SFS. Therefore, the contents of /bin and /usr/bin are different, and applications that make wrong assumptions (like grep in /bin) start to break.

[Marv]

The problem comes in when a non-usrmerge pet or SFS overwrites the OS simlink with a physical directory and breaks that symlink -and the OS- @jrb and I pretty much had that ironed out in jammypup64 by E1 (upup-22.04-jrb-E1.iso). The main work/change was in /usr/local/petget/install.sh there, attached as an example of what can be done only. Those modifications caught and corrected or rejected pretty much everything we threw at it at that time. The two worlds can co-exist.

dimkr posted the explanation while I was writing