How it Works; Overview

Moderator: Forum moderators

Post Reply
User avatar
wiak
Posts: 3627
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 994 times
Contact:

How it Works; Overview

Post by wiak »

The following is as accurate as I could make it, but may have some errors, slight inaccuracies, including grammar issues. Please don't edit it in any way, however, since even a comma or semi-colon can make a big difference to ambiguity; I prefer to fix such matters myself to avoid loss of intended technical meaning. Thank you.

In two posts I want to explain in sufficiently correct detail the simple how and why FirstRib boot mechanisms work with most any distro filesystem, and not just specially contructed ones that are initrd dependent.
This first part, hopefully explains in its ten steps, why much of the confusion and claims regarding difference between full and frugal are simply a MYTH. The kernel doesn't care, and you can construct distros that freely mix technologies some claim are special to so-called 'frugal installs' (via, for example, an initrd, such as the FirstRib initrd, that has been carefully designed to always, best as I can, take into consideration the lack of correctness of various myths). These claims of filesystem 'difference' between some proponents of so-called 'frugal installation' tend, I feel, to obscure understanding how Linux works and the resulting lack of understanding tends to frighten away some perfectly capable users from becoming involved in some distro building and development. Distro development/building/creating is easier (and potentially much more flexible) than they end up believing, but not when they have been convinced that a frugal installed distro is a complicated different kind of Linux. It isn't.

Funnily enough, someone, who was trying to insult me, angrily claimed that I was a distro 'builder' and not a 'developer', but the claim (insinuating some kind of inferiority) to me showed a lack of understanding. At most, all of us involved in distro creation are distro builders - we develop these builds. The likes of Linus Torvalds, who design and develop core system code, such as the C sources internal to the kernel, have C code related extreme system-core technical knowledge that is not at all required for creating/building/developing new distros here on this forum. All we need is knowledge of how the overall system works in terms of bolting the various Torvald's-type designed components and subsystems together. Much indeed like LEGO.

Like everyone else, who tend to be called 'developers' on this forum, I simply know enough to actively assemble distros together and create a FirstRib build system that is less limited in terms of layers and filesystem types (compressed or uncompressed) than many an other design. I am indeed nothing more and nothing less than a distro builder, and thus like all other distro creators on this forum; I am not an artist, genius, or anyone special IMO.

You could easily, I'm sure, become a distro creator too, or involved directly in that process, if you wished or had time for it, with but perhaps (if required) a little extra knowledge that dispels some of the popular myths often posted elsewhere.

PART 1:

1. A Linux system is stored as an organized filesystem whose arrangement can be thought of as having a hierarchy with the root directory '/' at the top.
(Confusingly there is a directory actually called 'root', but that is something different; that is simply the home directory of user root).

2. The Linux filesystem unifies all otherwise separate parts of itself (such as partitions, physical devices such as hard drives and usb sticks) into that single directory structure with / at the top.

3. Most computers only really allow one program to run on them at any one time, but fast scheduling gives the useful illusion of many program process all running individually in parallel.

4. On boot, the computer uses a boot loader program, such as grub, to find the location of the linux kernel (in practice, now often named vmlinuz) and if used, an initial RAM filesystem, nowadays simply often named initrd.

5. The linux kernel then starts running and uses the initrd loaded into RAM that grub has located (via grub.cfg) for it. However in order to schedule start up any other part of the system, the kernel needs to have suitable driver programs that know how to read the storage devices (or it won't, for example, be able to load any programs from the externally stored main filesystem parts, which it needs to be able to 'mount' as part of the overall filesystem).

That boot-needed media-finding driver code is sometimes built into the kernel (a 'huge' style of kernel) or supplied as separate module driver programs (boot-media-related driver modules being then needed inside the initial-loaded initrd). If the kernel can't find the required driver/module/firmware code it will be unable to boot the Linux system so you will get a Kernel Panic (i.e. the system will freeze/crash). A kernel panic will also commonly result if the initial boot init cannot at the end of its work find the real root filesystem init that is most usually stored in main filesystem at location /sbin/init.

6. Assuming the kernel finds the required boot-time driver code, it then looks for the first program to run, which it expects at pathname /init or /linuxrc within the RAM-loaded initrd filesystem. It is important to understand that any program the computer system runs must first be loaded into RAM for its code to be executed (including the kernel itself).

7. A major job of that initial boot-time init is to unify all the physical hard drives and partitions and so on into a single directory structure. And this fact leads to something very important you should understand:

It DOES NOT matter whether that unified Linux single directory structure (overall Linux filesystem) was assembled as a full install or as a frugal install. By that stage, all the kernel cares about is that it can see all parts of the resulting filesystem.

THAT is why FirstRib initrd can boot pretty much any pre-built Linux distro root filesystem, be that provided as a "full install", or as a "frugal install" in the forum of multiple smaller filesystems (compressed or uncompressed) that the FR initrd program init needs to first glue together...

8. Once the required single directory structure is provided or assembled, the early init program (if used), typically does what is called a 'switch root' to where the top of the overall Linux filesystem is stored, and starts up the actual first process to replace itself as Linux Process ID number 0. Depending on the design, that PID0 process is usually begun with a program also called 'init' (but a different one to that earlier init; this second 'init' program can be stored anywhere on the overall Linux filesystem, but is typically held in directory /sbin). However people have designed different versions of that second main init program and given them different names, so it might be called 'init', or 'systemd', or 'runit', or some other name devised by someone else for helping the kernel to control the systems processes. The likes of sysV init, systemd, and runit, are referred to as 'init systems'; they all do pretty much the same job so doesn't 'really' matter which one you use (except that some work more efficiently overall than others if they are better designs...). My advice is: Ignore the hate, use whatever works best for you in practice; I tend to go with the flow (i.e. use whatever upstream distros recommend).

9. Nowadays, it is quite common for most all of the required single directory structure to be installed into a single partition. However, it was ALWAYS more traditional to store some parts of the required overall linux filesystem in separate filesystems, usually in the form of separate partitions, or could include filesystems stored in the form of a compressed files (one example being squashed filesystems, or in img files, or as network accessible drives/partitions/filesystems). Having the filesystem stored in pieces in that matter is nothing to do with whether the description full install or frugal install is being used - in either case, the component parts can be glued together via a process called 'mounting'. As long as the final result of the 'overall mounting process' results in a unified single directory structure the kernel can use. It matters not to the linux kernel whether we call the installation full or frugal. To a large extent, the distinction is a MYTH. It was typical, for example, in even the earliest UNIX, and also Linux, distributions that the directory /usr, for example was stored in read-only form on a separate disk partition, or even remotely on a different system.

In other words, Linux filesystem parts are typically stored on separate devices or in separate partitions or in compressed filesystems, for example, and part of the boot process includes fusing/joining/glue-ing all these parts together into a unified single directory structure (hierarchy with / at the top) that the kernel can use. It does not matter whether some sort of union filesystem is used for that assembly or not (e.g. some kind of union filesystem layer approach); once the structure is assembled the kernel couldn't care less how it was glued together - the concept of full install or frugal install is in that sense irrelevant.

10. In fact, in practice, a distro builder can use a mixture of so-called 'full install' techniques and frugal install techniques to glue the overall linux filesystem together (FirstRib initrd was indeed designed with that lack of difference between full and frugal install kept carefully in mind); i.e. can have different parts of the filesystem stored in separate partitions (in full install type parts) and simply use mountpoints and the mount command to join them into the system (in fact, ALL mechanisms, full or frugal, use 'mount' to include filesystem parts). Some of the filesystem parts can be stored on different devices (e.g. removable or network drives) or as compressed filesystems (e.g. squashed filesystem files or img files) and, again these can simply be mounted into the overall filesystem. And/or some only, or all, of these filesystem parts can be fused together via some union filesystem, merged like layers in a graphics image (using the likes of unionfs, aufs, overlay, or fuse) and the result of that merge then itself simply mounted onto the overall structure. May seem at first like magic, but in reality it is purlely logical and straightforward.

SUMMARY:

That is: FR initrd was carefully designed with that recognition of there effectively being no such thing as a full install or a frugal install, except that the latter term is used to indicate some form of layering is being employed; the kernel couldn't care less (traditional Linux filesystems commonly used read-only filesystem parts too, albeit usually in the form of separate partitions for separate parts of the filesystem); instead there are simply various techniques that can be used to store filesystem parts separately and then glue them together... Once assembled, you can then boot it. Hence, almost any distro can be booted by FR initrd, run exactly as normal, but be enhanced by the specially devised management of compressed or uncompresses layers capabilities (i.e. 'frugal install' capabilities) FR initrd can give them.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

Post Reply

Return to “KL-Dev_Work”