Bullseye build script

[fredx181]

Now I tried clicking Ok in the dialog with the mouse pointer (I guess you did that, right?)

Did you ? If so, I'm almost sure that's the problem, please try again from FossaPup and press "Enter" after choosing keyboard-layout.
EDIT: Or try new mklive-bullseye64, see next post.

when I run the build from DebianDog, there's no problem

Witch One? there are many dogs

I did from BusterDog: https://debiandog.github.io/doglinux/zz03busterdog.html

[fredx181]

*** mklive-bullseye update ***

Change is that the keyboard-layout dialog is now using "whiptail" rather than "dialog".
This should avoid a build failure in some cases, when clicking with the mouse cursor on the keyboard-layout dialog.
Using whiptail, mouse clicking simply doesn't do anything, so the only way to continue is to press "Enter".

Script replaced at first post and appimages replaced here:
https://debiandog.github.io/MakeLive/Re ... yedog.html

Fred

[Luke2021]

Did you ? If so, I'm almost sure that's the problem, please try again from FossaPup and press "Enter" after

No, only keyboard. i will try updated script.

[Luke2021]

turned error exit status 255
Errors were encountered while processing:
keyboard-configuration
E: Sub-process /usr/bin/dpkg returned an error code (1)
FAILED

[fredx181]

Pity! Thanks anyway for testing, now I think I'm really out of ideas, still a mystery for me. (except to try from one of the 'Dog' OS's, no need for using the appimage, just the bare script attached at post #1 should work).

[spotted]

@ fredx
Do you know how to turn off whatever and turn on lightdm or gdm3 so I can install a downloaded MATE desktop. This is on rcrsn51's bullseye starter-kit.
I had a go at lightdm and mate install but when rebooting got the lightdm greeter ok but it needed login and password that the starter-kit dont have, and every other thing on the screen grayed out. Could not go anywhere, had to pull the plug. I know how to use synaptic to install lightdm and mate desktop but I dont know how to kick it in. Thanks

[rcrsn51]

... needed login and password that the starter-kit dont have,

The Starter Kit has a root:root user built-in, but you can easily add your own users from the System menu app.

If you read the examples for installing browsers like Chrome, you will see how they run from a "guest" user.

[fredx181]

@ fredx
Do you know how to turn off whatever and turn on lightdm or gdm3 so I can install a downloaded MATE desktop. This is on rcrsn51's bullseye starter-kit.
I had a go at lightdm and mate install but when rebooting got the lightdm greeter ok but it needed login and password that the starter-kit dont have, and every other thing on the screen grayed out. Could not go anywhere, had to pull the plug. I know how to use synaptic to install lightdm and mate desktop but I dont know how to kick it in. Thanks

Hi spotted, any reason why you need a login manager (e.g. gdm3, lightdm) ? And do you prefer to login as normal user instead of root ?
Btw, for the Starter kit, root's password = root
I tested on the Starter Kit, by installing "mate-desktop-environment".
Then: Exit X, at console typed "startx" and Mate Desktop appeared.
But... somehow no icons showing on the Desktop, I've read on the web that you need mate-tweak for that, did install it and configured to show icons on the Desktop but still didn't work for me.
So made a simple script:

Code: Select all

#!/bin/bash
killall caja &
wait $! 
sleep 3
caja -n --force-desktop &
wait $!
disown

Made it executable and added to ~/Startup, after restarting X or reboot, the Desktop icons showed for me.
EDIT: Some further testing tells me that above problem happens only when logged in as root, when logged in as a normal user there's no workaround needed.

Tell me if you need a login-manager and I can help further (autologin from /etc/inittab needs to be disabled then), gdm3 is not a good choice IMO, because installing it ruins the "no-systemd" setup, lightdm can be a PITA to set up login as root, I'd recommend "slim" (small and simple).

EDIT: To disable autologin from /etc/inittab, do in terminal:
cp -af /etc/inittab-noauto /etc/inittab
Or: If you want autologin in /etc/inittab changed to another user, say that you first created user "spotted", then change:
1:2345:respawn:/bin/login -f root tty6 </dev/tty1 >/dev/tty1 2>&1
to:
1:2345:respawn:/bin/login -f spotted tty6 </dev/tty1 >/dev/tty1 2>&1

Fred

[spotted]

@ hi fredx
any reason why you need a login manager (e.g. gdm3, lightdm)
Thats what I had to do with a big-boy install on my pi4, didnt even think that I could just install 'mate-desktop-environment'
and startx.
Thats too easy, surely there must be a harder way!
I thought I would have to turn of pcmanfm and openbox firstly.
Anyhow fun and games, I thought I had saved all the mate deb's but I lost them when I had to pull the plug on the lightdm greeter all on the greeter was all greyed out. I also installed oxygen curses with the mate desktop and wanted to see what they looked like so restarted X and lost all the mate, lightdm, gdm3 and gnome desktop deb's when I pulled the plug. Give me a day or two, I shall read up your post. I have to wait until 2.00 in the morning to download mate again, I have a slow ISP, bandwith oversold. Thanks

[fredx181]

Hi spotted, here's "StarterKit+Mate-remaster-amd64.iso": https://github.com/DebianDog/MakeLive/r ... -amd64.iso
It's a remaster of the Starter Kit with added Mate Desktop (and some tweaking done).
As I said earlier Mate + root login doesn't work properly, so "puppy" is the default (automatic) login user.

any reason why you need a login manager (e.g. gdm3, lightdm)
Thats what I had to do with a big-boy install on my pi4, didnt even think that I could just install 'mate-desktop-environment'
and startx.
Thats too easy, surely there must be a harder way!
I thought I would have to turn of pcmanfm and openbox firstly.

To explain: The SESSION_MANAGER variable points to "mate-session" after installing Mate. From ~/.xsession:
SESSION_MANAGER=/usr/bin/x-session-manager
So, openbox is in fact "turned off" automatically this way.

Specifications:
Default (automatic) login: user puppy with pasword: puppy
Password root = root
Use "sudo" for administrative tasks, e.g. sudo apt install ...
FYI, I configured it so that when using sudo, there's no need to enter password (as it's sort of standard for a "live" system, official Ubuntu or Debian live systems work the same).

EDIT: Note that not all applications work correctly when running from Menu, for example: peasyclock needs to run as superuser, so run: sudo peasyclock from terminal.

Fred

[spotted]

Hi Fred,

Downloading your iso now, a whopping av. 40kb/s at 4 am in the morning. During the day 10-15 kb/s, virus-19 lockdown, no kids at school all on their phones.

Got up 3 in the morning yesterday and installed mate, had to learn how to make an executable, all I knew was to make a file in startup called new file then from properties click make executable and it worked. So if it worked being called new file it should work being called fredx-follie and it does. Some times the icons dont appear on the desktop after reboot but clicking on fredx-follie brings them up. I installed every mate- in synaptic except earthquake and hud but I cant get volume control, temperature into the sys tray or sound in smplayer, its defeated me.

Downloaded 20% of the iso so far got another 2h 10min to go, leaving the laptop running and going back to bed.

Checked download at 7.00 am and it was finished but only 181 mb. bad download, so another effort late tonight.

[fredx181]

*** Bullseye-Starterkit + Mate Desktop ISO ***

Earlier shared here (configured automatic default login as user "puppy"): viewtopic.php?p=30344#p30344
Back then couldn't get root login to work properly on Mate, but now found the 'secret' how to handle it.
StarterKit+Mate-remaster-amd64-2.iso:
https://github.com/DebianDog/MakeLive/r ... md64-2.iso Size: 343MB
Login details:
Default login (automatic) as root, password = root
Or login as user puppy, password = puppy
Kernel: 5.10.0-7-amd64

@spotted FYI, as you reported (and I experienced too) the workaround script that I suggested for root login (to be added to /root/Startup), doesn't always work.
Instead of the script, adding to /root/.xsession: caja -n --force-desktop & works OK, so then becomes:

Code: Select all

#!/bin/sh

#setxkbmap -option terminate:ctrl_alt_bksp &
sleep 2
/usr/local/bin/start-up &
caja -n --force-desktop &
export SESSION_MANAGER=/usr/bin/x-session-manager
exec $SESSION_MANAGER

Fred

[spotted]

@ fredx
I will download your latest remaster early hours tomorrow morning.
Have to churn through 2.5 gb of data by about 6.00am before my weekly 4 gb expires, it dont roll over. ( Lots of crazy russian drivers and mr bean)

[fredx181]

@ fredx
I will download your latest remaster early hours tomorrow morning.
Have to churn through 2.5 gb of data by about 6.00am before my weekly 4 gb expires, it dont roll over. ( Lots of crazy russian drivers and mr bean)

If downloading is so problematic, instead of the remaster ISO, why not just use the Mate setup that you already have (as I understood) with the modification of /root/.xsession (as I wrote above) ?

[spotted]

Too late fredx, Coming from your latest, glad I got it. Palemoon in 'guest' a-la-la rcrsn51. I see I dont have to sudo jerk caja or authenticate into gparted, gufw or synaptic, good move. I did not see any firewall show up as installed in synaptic so I installed gufw. Will there be any problems if a firewall is already installed, a conflict.
Only thing left is SOUND
edit, SOUND now working in smplayer.

[recobayu]

Hi Fred,
I don't see the mate desktop option on this mklive-bullseye64 -gui. But I look on github, the mate desktop was added. How to use that? Thank you

[fredx181]

*** mklive-bullseye update ***

Change: Added Mate Desktop to DE choices.
Script updated at first post and appimages updated see here: https://debiandog.github.io/MakeLive/Re ... yedog.html

@recobayu Earlier on, removed the Mate Desktop from DE choices because I couldn't figure out how to make it run properly "as root", solved now.
(for more info, see above on this page)

mklive-bullseye-de_config.png (89.48 KiB) Viewed 2654 times

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Mate Desktop minimal

mate.png (73.95 KiB) Viewed 2654 times

Fred

[fredx181]

Bullseye is now the new "stable" release, Buster has become "oldstable".
https://www.debian.org/News/2021/20210814

Fred

[trister]

Hello fred
I've been trying Bullseye script (DDog option no additions libs during install).
It loads MUCH faster than Busterdog!
Also, the wine appimage "trick" I posted in the wine forums works out of the box (=all my windows stuff worked without changing anything else)

I have 2 main issues though:
1. I can't put/load more than 65 squashfs fromlive folder (This existed in older debians but you posted me a fix which it worked)
2. I did a frugal install with EXIT command and a changed.dat file. Apt2sfs thinks this is a full install (so I can't run the normal apt2sfs only the apt2sfs-fullinst). Did I miss something?

Also, inside changes.dat everything is saved inside a folder named "upperdir" not in root (like all the other Dogs/puppies) . Does this relate with issue #2 or is this the new normal structure of the changes.dat?

Thanks again for your amazing work !!!

[fredx181]

Hello trister,

It loads MUCH faster than Busterdog!

If you mean that it boots faster, yes, probably because of the newer kernel.

I have 2 main issues though:
1. I can't put/load more than 65 squashfs fromlive folder (This existed in older debians but you posted me a fix which it worked)

Yes, I've decreased the number of loop devices again because the "overlay" system uses more RAM at initial boot, and found that less loop devices decreased the RAM usage.
If you want you can change that by modifying initrd1.xz:

Extract initrd1.xz from standing in the 'live' dir:

Code: Select all

mkdir newinitrd        # make dir where the extracted files go
cd newinitrd         # go inside newinitrd directory
xz -dc ../initrd1.xz | cpio -i        # extract  ../initrd1.xz

Open inside newinitrd "linuxrc" with texteditor, find the line "let y=x+20" (line 419) and change 20 to e.g. 255
let y=x+255

Create new initrd1.xz (note: will overwrite existing in live dir):

Code: Select all

cd newinitrd   # or open terminal in newinitrd dir.
find . -print | cpio -o -H newc 2>/dev/null | xz -f --extreme --check=crc32 > ../initrd1.xz

EDIT: But I'm not sure if loading so many .squashfs at boot works well with the new 'overlay' system, so I'm curious if it works for you.
EDIT2: I tested again booting with 255 loop devices loaded, and initial RAM usage is a lot more (for me around 210MB, with the default 20 loop devices it's 160MB)

2. I did a frugal install with EXIT command and a changed.dat file. Apt2sfs thinks this is a full install (so I can't run the normal apt2sfs only the apt2sfs-fullinst). Did I miss something?

Mmm... apt2sfs works OK for me (package version 1.1.3), can you give details how you used apt2sfs and what exact error you got ? (so I can try to reproduce)

Also, inside changes.dat everything is saved inside a folder named "upperdir" not in root (like all the other Dogs/puppies) . Does this relate with issue #2 or is this the new normal structure of the changes.dat?

Yes, changes are in "upperdir" which is standard for the 'overlay' system (major change in Bullseye compared to Busterdog is that 'aufs' is replaced by 'overlay')

Fred

[trister]

If you want you can change that by modifying initrd1.xz:

Extract initrd1.xz from standing in the 'live' dir:

Code: Select all

mkdir newinitrd        # make dir where the extracted files go
cd newinitrd         # go inside newinitrd directory
xz -dc ../initrd1.xz | cpio -i        # extract  ../initrd1.xz

Open inside newinitrd "linuxrc" with texteditor, find the line "let y=x+20" (line 419) and change 20 to e.g. 255
let y=x+255

Create new initrd1.xz (note: will overwrite existing in live dir):

Code: Select all

cd newinitrd   # or open terminal in newinitrd dir.
find . -print | cpio -o -H newc 2>/dev/null | xz -f --extreme --check=crc32 > ../initrd1.xz

EDIT: But I'm not sure if loading so many .squashfs at boot works well with the new 'overlay' system, so I'm curious if it works for you.
EDIT2: I tested again booting with 255 loop devices loaded, and initial RAM usage is a lot more (for me around 210MB, with the default 20 loop devices it's 160MB)

Tried this (put x+200) but I still get this error when I put more squqashfs files (I have 4GB ram and with almost the same and much more squashfss BusterDog runs without issues):
The error appears after the LAST squashfsfile is loaded and boot stops with the prompt : /#

Code: Select all

[b]mount: mounting overlay on /union failed: Out of memory
sh:can't access tty: job control turned off[/b]

If I remove some squashfs (and keep arroung 64) booting works again.

[fredx181]

Tried this (put x+200) but I still get this error when I put more squqashfs files (I have 4GB ram and with almost the same and much more squashfss BusterDog runs without issues):
The error appears after the LAST squashfsfile is loaded and boot stops with the prompt : /#

Code: Select all

mount: mounting overlay on /union failed: Out of memory
sh:can't access tty: job control turned off

If I remove some squashfs (and keep arroung 64) booting works again.

Pity... could be a limitation of 'overlay', the number of .squashfs modules that can be loaded, but not sure.
One workaround maybe is to use a slightly older kernel (5.8) that I have (and initrd1.xz) with aufs enabled, tell me if you are interested.
(or compile your own kernel with aufs enabled, I can probably help then to create initrd1.xz for it)

[trister]

...
mount: mounting overlay on /union failed: Out of memory
sh:can't access tty: job control turned off
...

Pity... could be a limitation of 'overlay', the number of .squashfs modules that can be loaded, but not sure.
One workaround maybe is to use a slightly older kernel (5.8) that I have (and initrd1.xz) with aufs enabled, tell me if you are interested.
(or compile your own kernel with aufs enabled, I can probably help then to create initrd1.xz for it)

Thanks for the older kernel offer but my plan is to try and use latest versions at this point.

Tried on a clean frugal install (no changes.dat) with only the 01-filesystem and 80 very small dummy squashfs files and same problem occured. So I guess you are correct with the limitation (unless there is another parameter that can increase that). Is this a kernel parameter about overlay (that can be solved with a new compile)?

A workarround might be to somehow keep different folders with squashfs files and load them on demand (I saw somewhere a script that did that -but can't remember where) but I'm not sure if I run again on this limitation

Also, I've ported many of my apps to appimage (but can't do this with all sfs).

[dancytron]

Maybe combine you sfs files into fewer larger sfs files? You could do it with the edit-sfs app or at the console.

[wiak]

Pity... could be a limitation of 'overlay', the number of .squashfs modules that can be loaded, but not sure.

I was curious since, as you know, WeeDog has always used overlayfs for its frugal install features and I have scripted it to allow up to 99 layers by default (actually 100 layers if I included layer 00), but I've never tested how many layers can be loaded by it in practice. Don't have enough energy in me to modify its initrd to allow for more (though it is only half a dozen lines of code need changed to theoretically allow for up to 1000 layers... though not sure what use that would be in practice).

Anyway, (and this is in WDL_Arch64 not Bullseye but maybe the results useful to you) I used the following one liner to create 99 small sfs modules NNtest.sfs (each of which contain one file whose name is fileNN i.e. file02 up to file99):

Code: Select all

for i in `seq -w 02 99`;do mkdir -p $i; touch ${i}/file${i};mksquashfs $i ${i}test.sfs; rm -rf $i; done

The resulting numbered files are all layered to appear under / directory of the merged overlay.

Following successful booting with that arrangement in WeeDog_Arch64 I can now confirm that at least up to 99 sfs modules (and no doubt more) can be successfully overlaid, so overlayfs is not the limiting factor you seem to be experiencing.

As far as RAM usage is concerned, please find below the free report on immediate booting for the case of one sfs file only (01firstrib_rootfs) followed by rebooting with all of 01 up to 99 sfs modules included:

Code: Select all

Report from free utility

With only one (01firstrib_rootfs) squashfs module in overlay

[root@bootstrap ~]# free
               total        used        free      shared  buff/cache   available
Mem:         3855104      183184     3058060       37912      613860     3409888
Swap:              0           0           0

With total of 99 squashfs modules in overlay:

[root@bootstrap ~]# free
               total        used        free      shared  buff/cache   available
Mem:         3855104      293660     2936992       37800      624452     3295204
Swap:              0           0           0

So, at a glance, seems to be only using about 1MB per additional sfs module mounted as a layer; approx: (293MB - 183MB)/99.

I've attached a few images of pcmanfm illustrating the experiment somewhat.

Loaded fast and without issue by the way, so using large numbers of numbered sfs modules (at least with WeeDog) seems to be more practical than I actually imagined it would be. Now I wonder what the actual limit to the number is - that experiment will have to wait however...

Cheers,

wiak

[trister]

...
(though it is only half a dozen lines of code need changed to theoretically allow for up to 1000 layers... though not sure what use that would be in practice).
...

Thank for the tests.

Check the state of one of my custon OS -usage of many layers in practice (in my main home OS I can use up to 120+ SFSs):
https://pastebin.com/bb4YCniY
The idea is that I can interchange/update different versions of same app without issue and the system works like in a "go-back" mode. Everytime I reboot everything is the same.Also, the pupils can experiment as much as they want and after reboot everything is restored.

This is the only way for me to handle the many PCs I use every year so, that many SFS are a must for me.

So, is there an easy way that more layers can be supported by Bullseye; I don't want a custom modification that won't work in next fred's update (so I guess recompiling kernel is not good).

...
Anyway, (and this is in WDL_Arch64 not Bullseye but maybe the results useful to you) I used the following one liner to create 99 small sfs modules NNtest.sfs (each of which contain one file whose name is fileNN i.e. file02 up to file99):

Code: Select all

for i in `seq -w 02 99`;do mkdir -p $i; touch ${i}/file${i};mksquashfs $i ${i}test.sfs; rm -rf $i; done

The resulting numbered files are all layered to appear under / directory of the merged overlay.
...

This was also the second test I did but it displayed the 'out of memory' error (with 4GB or ram it was sure not my PCs problem)

@fredx181
In my first post I meant that Bullseye loads squashfs during boot MUCH faster than BusterDog.

[fredx181]

Pity... could be a limitation of 'overlay', the number of .squashfs modules that can be loaded, but not sure.

I was curious since, as you know, WeeDog has always used overlayfs for its frugal install features and I have scripted it to allow up to 99 layers by default (actually 100 layers if I included layer 00), but I've never tested how many layers can be loaded by it in practice. Don't have enough energy in me to modify its initrd to allow for more (though it is only half a dozen lines of code need changed to theoretically allow for up to 1000 layers... though not sure what use that would be in practice).

Anyway, (and this is in WDL_Arch64 not Bullseye but maybe the results useful to you) I used the following one liner to create 99 small sfs modules NNtest.sfs (each of which contain one file whose name is fileNN i.e. file02 up to file99):

Code: Select all

for i in `seq -w 02 99`;do mkdir -p $i; touch ${i}/file${i};mksquashfs $i ${i}test.sfs; rm -rf $i; done

The resulting numbered files are all layered to appear under / directory of the merged overlay.
....

Many thanks William, nice way to be able to test loading 100 modules (changed .sfs to .squashfs).
On Bullseye, tested booting with the 100 .squashfs and went without errors for me, so indeed it can not be a limitation of 'overlay':

2021-08-23-150725_897x641_scrot_597x427.png (189.61 KiB) Viewed 2236 times

@trister

This was also the second test I did but it displayed the 'out of memory' error (with 4GB or ram it was sure not my PCs problem)

Strange why it works for me and not for you (btw, I have only 2GB RAM), you probably have a different setup, my frugal install is on ext4, using savefolder, bootloader is grub4dos.
EDIT: And probably I use different Desktop env: the tint2 setup made with mklive-bullseye (but I don't expect that it would make any difference).

[trister]

...

Many thanks William, nice way to be able to test loading 100 modules (changed .sfs to .squashfs).
On Bullseye, tested booting with the 100 .squashfs and went without errors for me, so indeed it can not be a limitation of 'overlay':
2021-08-23-150725_897x641_scrot_597x427.png
...
Strange why it works for me and not for you (btw, I have only 2GB RAM), you probably have a different setup, my frugal install is on ext4, using savefolder, bootloader is grub4dos.
EDIT: And probably I use different Desktop env: the tint2 setup made with mklive-bullseye (but I don't expect that it would make any difference).

I used DebianDog option. So it's something to do with my installation.
Can you upload your initrd1.xz to check if it works for me?
Thanks

[fredx181]

...

Many thanks William, nice way to be able to test loading 100 modules (changed .sfs to .squashfs).
On Bullseye, tested booting with the 100 .squashfs and went without errors for me, so indeed it can not be a limitation of 'overlay':
2021-08-23-150725_897x641_scrot_597x427.png
...
Strange why it works for me and not for you (btw, I have only 2GB RAM), you probably have a different setup, my frugal install is on ext4, using savefolder, bootloader is grub4dos.
EDIT: And probably I use different Desktop env: the tint2 setup made with mklive-bullseye (but I don't expect that it would make any difference).

I used DebianDog option. So it's something to do with my installation.
Can you upload your initrd1.xz to check if it works for me?
Thanks

You mean with "DebianDog" the DDog Desktop choice (with openbox_xfce) ?
The initrd1.xz for sure is the same, no matter which Desktop choice.
But anyway I'd be happy to upload my initrd1.xz, it's for kernel 5.10.0-8-amd64, is that OK ?

[trister]

You mean with "DebianDog" the DDog Desktop choice (with openbox_xfce) ?
The initrd1.xz for sure is the same, no matter which Desktop choice.
But anyway I'd be happy to upload my initrd1.xz, it's for kernel 5.10.0-8-amd64, is that OK ?

ok thanks