Basic Linux distro design 'secrets' in 11 steps
Posted: Thu Sep 03, 2020 9:53 am
viewtopic.php?p=2088#p2088
Intro: Lots of possible boot mechanisms: For example: old ramdisk initrd method; newer (since kernel 2.6) initramfs method. Also, you can use what is called ramfs, which cannot use swap so can't be 'freed' from memory (RAM), or more likely nowadays, you can use tmpfs, which can, for example, use swap. So to simplify boot description let's look at most use case nowadays, which is as follows:
1. In Puppy forum systems, initrd is normally an external compressed file (and it's actually an initramfs rather than older initrd block system type, despite filename reflecting legacy initrd type).
2. The boot loader (e.g. grub...) is told where to find the compressed kernel, and needs to be able to mount that location and uncompress the kernel into RAM.
3. The kernel may well not have external storage drivers built in and so can't access external modules or the external initramfs at this stage, so it is the boot loader that has to load that into RAM somewhere during boot too (I 'think' it is left in compressed state in RAM at this stage). As far as I understand it (and it works...) the boot loader passes to the kernel, prior to the kernel starting, the RAM location of where it has put the initramfs file.
4. Usually nowadays, kernel uses tmpfs (created in RAM by the kernel) for its initial root filesystem (which in kernel terms is called rootfs). It needs to get uncompressed initramfs into there. So since on boot it finds its internal rootfs empty of file "init" (i.e. since 'usually' we aren't using a kernel that has an embedded initramfs) it uncompresses that compressed initramfs that the boot loader stored somewhere in RAM.
5. Now the kernel has an initial root filesystem to use... It looks in there for the file "init" (i.e. /init), which can be any binary file the initramfs creator wanted that to be, but usually it is a shell script that the initramfs creator (such as me for WeeDog Linux) has written, which, like all shell scripts, contains a first line such as #!/bin/sh.
6. That initial rootfs root filesystem is pretty small, and often just contains a few empty directories (such as /dev, /proc, /bin and so on) and commonly has a tiny statically built version of busybox binary (in say /bin), which contains an ash implementation of a shell script interpreter, which is enough to interpret the contents of that "init" shell script file. One other thing that relatively small root filesystem does need to include, however, is kernel-compatible modules that the kernel needs to access external storage since the big root filesystem is stored externally.
7. The main job of the "init" shell-script in that initial root filesystem (which was loaded into kernel rootfs) is really to mount the partition that contains the actual (big) root filesystem you really want the Linux kernel to use. In frugal install type systems, such as WeeDog Linux, DebianDogs, and Puppy, that external big root filesystem is usually stored as a to-be-mounted sfs file (or several such to-be-mounted into different layers sfs files).
8. Also in Puppy similar systems that early "init" also sets up the filesystem layers type system - (such as aufs, or, kernel-official overlayfs) so the final used filesystem is the combination of all the merged layers (a similar concept to layers in graphics editing programs, but for different files/directories layers instead of image layers). These layers get mounted/merged to a single directory 'mountpoint' that has been created (with mkdir) on that initial rootfs. The path to that mountpoint can be anything really - for no particular reason, in WeeDog, I generally use mountpoint /mnt/layers/merged and overlayfs (Puppy uses some other location for sure, and mainline Puppy and DebianDogs use aufs rather than overlayfs for their filesystem "layers", and very different init scripts altogether to the one I wrote for WeeDog).
9. Assuming, busybox, one of the last commands in "init" uses busybox version of command switch_root to change what Linux sees as / (the top directory of the final root filesystem hierarchy). At this stage, the tmpfs small rootfs gets freed-up/emptied out of RAM, so doesn't waste RAM space, so even if initial rootfs loaded initramfs was quite large, that is irrelevant in the final running system. For example, in WeeDog Linux "init" shell script I use the command:
(where merged is relative to /mnt/layers, meaning its actually /mnt/layers/merged in absolute terms...)
10. Note that switch_root not only changes the Linux kernel perspective of where / is, it also runs the program /sbin/init (which itself can be any binary program or yet another shell script; it can for example be the binary program for SysVinit, or it could be systemd init, or runit, or something else the distro developer chose to use...).
11. Note the 'exec' in command "exec switch_root merged /sbin/init". The original small initramfs "init" was the first process to run (so had Process ID 1). The command exec causes the larger /sbin/init to become the new PID 1, which then forks all the programs you want to run in userspace thereafter (anything you like... for example... startx, which can result in JWM or Openbox or whatever being started and so on according the the instructions and configurations you then choose for your distro creation...).
Being definitely biased, I would claim that WeeDog Linux (no matter the repo 'flavour' - Arch, Ubuntu, Debian, Devuan, SliTaz) has one of the smallest, most CPU/RAM-efficient initial "init" scripts around: maybe a quarter the lines of shell script code of most others, so perhaps a bit easier to understand, though not 'easy' since it does a lot... and thus has tons of the kinds of frugal install functionality you might expect or need plus some extras (such as using either or both sfs files and uncompressed directories as filesystem layers). However, all the Puppy forum discussed distros have their strong points and are all to be recommended.
NOTE: I was originally just responding to above quote in this thread, but I realize this post of mine isn't particularly about booting from isos, so I've moved it to its own howto thread here.
Basic Linux distro design 'secrets' in 11 steps, by wiak...williams2 wrote: Sat Aug 08, 2020 10:37 pmAll the bootloader does is copy vmlinuz and initrd.gz to ram.
It can do other things, like hide partitions, but mostly all that any bootloader does, is to copy vmlinuz and initrd.gz to ram and start vmlinuz executing.
Intro: Lots of possible boot mechanisms: For example: old ramdisk initrd method; newer (since kernel 2.6) initramfs method. Also, you can use what is called ramfs, which cannot use swap so can't be 'freed' from memory (RAM), or more likely nowadays, you can use tmpfs, which can, for example, use swap. So to simplify boot description let's look at most use case nowadays, which is as follows:
1. In Puppy forum systems, initrd is normally an external compressed file (and it's actually an initramfs rather than older initrd block system type, despite filename reflecting legacy initrd type).
2. The boot loader (e.g. grub...) is told where to find the compressed kernel, and needs to be able to mount that location and uncompress the kernel into RAM.
3. The kernel may well not have external storage drivers built in and so can't access external modules or the external initramfs at this stage, so it is the boot loader that has to load that into RAM somewhere during boot too (I 'think' it is left in compressed state in RAM at this stage). As far as I understand it (and it works...) the boot loader passes to the kernel, prior to the kernel starting, the RAM location of where it has put the initramfs file.
4. Usually nowadays, kernel uses tmpfs (created in RAM by the kernel) for its initial root filesystem (which in kernel terms is called rootfs). It needs to get uncompressed initramfs into there. So since on boot it finds its internal rootfs empty of file "init" (i.e. since 'usually' we aren't using a kernel that has an embedded initramfs) it uncompresses that compressed initramfs that the boot loader stored somewhere in RAM.
5. Now the kernel has an initial root filesystem to use... It looks in there for the file "init" (i.e. /init), which can be any binary file the initramfs creator wanted that to be, but usually it is a shell script that the initramfs creator (such as me for WeeDog Linux) has written, which, like all shell scripts, contains a first line such as #!/bin/sh.
6. That initial rootfs root filesystem is pretty small, and often just contains a few empty directories (such as /dev, /proc, /bin and so on) and commonly has a tiny statically built version of busybox binary (in say /bin), which contains an ash implementation of a shell script interpreter, which is enough to interpret the contents of that "init" shell script file. One other thing that relatively small root filesystem does need to include, however, is kernel-compatible modules that the kernel needs to access external storage since the big root filesystem is stored externally.
7. The main job of the "init" shell-script in that initial root filesystem (which was loaded into kernel rootfs) is really to mount the partition that contains the actual (big) root filesystem you really want the Linux kernel to use. In frugal install type systems, such as WeeDog Linux, DebianDogs, and Puppy, that external big root filesystem is usually stored as a to-be-mounted sfs file (or several such to-be-mounted into different layers sfs files).
8. Also in Puppy similar systems that early "init" also sets up the filesystem layers type system - (such as aufs, or, kernel-official overlayfs) so the final used filesystem is the combination of all the merged layers (a similar concept to layers in graphics editing programs, but for different files/directories layers instead of image layers). These layers get mounted/merged to a single directory 'mountpoint' that has been created (with mkdir) on that initial rootfs. The path to that mountpoint can be anything really - for no particular reason, in WeeDog, I generally use mountpoint /mnt/layers/merged and overlayfs (Puppy uses some other location for sure, and mainline Puppy and DebianDogs use aufs rather than overlayfs for their filesystem "layers", and very different init scripts altogether to the one I wrote for WeeDog).
9. Assuming, busybox, one of the last commands in "init" uses busybox version of command switch_root to change what Linux sees as / (the top directory of the final root filesystem hierarchy). At this stage, the tmpfs small rootfs gets freed-up/emptied out of RAM, so doesn't waste RAM space, so even if initial rootfs loaded initramfs was quite large, that is irrelevant in the final running system. For example, in WeeDog Linux "init" shell script I use the command:
Code: Select all
exec switch_root merged /sbin/init10. Note that switch_root not only changes the Linux kernel perspective of where / is, it also runs the program /sbin/init (which itself can be any binary program or yet another shell script; it can for example be the binary program for SysVinit, or it could be systemd init, or runit, or something else the distro developer chose to use...).
11. Note the 'exec' in command "exec switch_root merged /sbin/init". The original small initramfs "init" was the first process to run (so had Process ID 1). The command exec causes the larger /sbin/init to become the new PID 1, which then forks all the programs you want to run in userspace thereafter (anything you like... for example... startx, which can result in JWM or Openbox or whatever being started and so on according the the instructions and configurations you then choose for your distro creation...).
Being definitely biased, I would claim that WeeDog Linux (no matter the repo 'flavour' - Arch, Ubuntu, Debian, Devuan, SliTaz) has one of the smallest, most CPU/RAM-efficient initial "init" scripts around: maybe a quarter the lines of shell script code of most others, so perhaps a bit easier to understand, though not 'easy' since it does a lot... and thus has tons of the kinds of frugal install functionality you might expect or need plus some extras (such as using either or both sfs files and uncompressed directories as filesystem layers). However, all the Puppy forum discussed distros have their strong points and are all to be recommended.
NOTE: I was originally just responding to above quote in this thread, but I realize this post of mine isn't particularly about booting from isos, so I've moved it to its own howto thread here.