Debian Dog for kirkwood - embeded armel[Test Project]

[fcopurico]

I started asking for help to fred181 in BusterDog + build system (no-systemd). I see I may be gettinh off toplic there. To prevent hijack I am opening this thread.
Here is the link where I started: https://www.forum.puppylinux.com/viewto ... 361#p49361

Objective:
As must Puppies, Busterdog has some advantages that I want to apply to a small Debian server on a Goflex Home unit.
OS is Read Only + a Read/Write savefile. This is a MOST have in a device that can be easily break. To recover on a crash we only need to delete the savefile and replace with backup. And in case it it working from NAND writing to NAND only at the end of session will improve life expectancy of the NAND. My own obective is having only basic debian os. But is we succed this method cam be used to install fred181/rcrsn51 BusterDog or BullseteDog.

Here is where I start.

Some unmodified official Debian Images:

Code: Select all

https://ftp.debian.org/debian/dists/bullseye/main/installer-armel/current/images/kirkwood/netboot/

Here are similar projects, where we can gather info.
Openwrt

Code: Select all

https://openwrt.org/toh/seagate/goflexhome

I believe for kernel all should start with

Code: Select all

make kirkwood_defconfig

And here some one posted similar kernel config, what he use for similar device

Code: Select all

https://judepereira.com/blog/wp-content/uploads/config.txt

I do not see anything about 64 bit. So I will assume is a 32 bit kernel. As I said. I have not started my studies.

Here an old script, dockstar.debian-wheezy.sh. I been told it does not work today. But is a good starting point.

Code: Select all

http://projects.doozan.com/debian/kirkwood.debian-wheezy.sh
http://projects.doozan.com/debian/dockstar.debian-squeeze.sh

That is all I had read. But I do not started to build anything. I will start by installing OpenWrt in NAND to be use as System Recovery method. And I plan to install Debian on USB. Latter in Sata HDD.

[fredx181]

Hi @fcopurico ,
I did some investigation and found a way to make a Buster arm64 (or armel) rootfs with (qemu-)debootstrap on Busterdog64.
Tell me if this goes in the direction you want (arm64 is ok?, if not change the ARCH variable on top perhaps ?)
--------------------------------------------------------------------------
EDIT: Can you say what's exactly your goal ? looking at https://www.debian.org/ports/arm/index.en.html armel or ...?
As I said I know close to nothing about ARM.
--------------------------------------------------------------------------
The below script needs to run from a Linux partition, e.g. sda2, ext4 formatted and will create chroot in debian-arm64 directory.
Just a concept. Works ok for me from Busterdog64, installs some basic packages and the kernel.
If this or perhaps a version with your edits is OK then we can see later how to add porteus-boot.
EDIT: Tried also ARCH=armel and debootstrap went ok, but the kernel not installing, found out later that it should be "linux-image-marvell" or "linux-image-rpi"
(edited code a little, EDIT: also later, removed line 17 as it was wrong )

Code: Select all

#!/bin/bash

WORKDIR=debian-arm64
# WORKDIR=debian-armel  # edit; uncomment for armel
export ARCH=arm64
# export ARCH=armel  # edit; uncomment for armel

apt update    # may be required
apt install binfmt-support qemu-user-static debootstrap

# perhaps need to start service binfmt-support
service binfmt-support start

mkdir -p $WORKDIR/chroot && cd $WORKDIR 

qemu-debootstrap --arch=$ARCH --no-merged-usr --include=apt-transport-https,ca-certificates,apt-utils --keyring /usr/share/keyrings/debian-archive-keyring.gpg --variant=minbase --exclude=debfoster buster chroot http://ftp.debian.org/debian

echo "if preferred add contrib and non-free in chroot/etc/sources.list"
echo "make it: deb http://ftp.debian.org/debian buster main contrib non-free"
read -sp "Press ENTER to continue . . . "

# mount binds ...
mount --bind /proc chroot/proc
#mount --bind /tmp chroot/tmp
mount --bind /dev chroot/dev
mount --bind /sys chroot/sys
mount -t devpts devpts chroot/dev/pts
echo -en "`cat /etc/resolv.conf`" > chroot/etc/resolv.conf

############ Start running in chroot ############
chroot_in () {
export HOME=/root
export LC_ALL=C
echo "APT::Install-Recommends "false"; APT::Install-Suggests "false";" > /etc/apt/apt.conf
echo "Acquire::Check-Valid-Until "0";" >> /etc/apt/apt.conf

# install some basic packages, edit as desired
apt update  # may be required
apt-get install whiptail keyboard-configuration sysvinit-core elogind xz-utils cryptsetup cryptsetup-bin gnupg dirmngr apt-utils wget elogind libelogind0 libpam-elogind udev --yes

dbus-uuidgen > /var/lib/dbus/machine-id
echo "live" > /etc/hostname   # change 'live' to what's preferred
echo "127.0.0.1	 localhost" > /etc/hosts
echo "127.0.1.1	 live" >> /etc/hosts    # change 'live' to what's preferred
mkdir /live
mkdir -p /opt/bin

[ $ARCH = armel ] && ARCH=marvell     # edited for armel (Kirkwood)
# [ $ARCH = armel ] && ARCH=rpi    # edited for armel, uncomment for "linux-image-rpi" (Raspberry Pi)
# install kernel
apt install linux-image-$ARCH     # edit: for armel make it "linux-image-marvell" or "linux-image-rpi"

# install more as desired ...
# .....
}
export -f chroot_in

# run chroot_in function
chroot chroot /bin/bash -c chroot_in

# UNMOUNT at the end !!
umount chroot/dev/pts
umount chroot/sys
umount chroot/proc
umount chroot/dev

[fcopurico]

Yes, what I like is that we seems to be on same page. And Thanks ahead for giving it a try.
As I said I was going to work next week. But last night I did spent quite a few hours on the Internet. I did conclude, that for this thread purpose, quemu is the way to go as maybe others my like to join. I will be looking and trying your posted script next. No done yet.

fredx181, for testing boots in quemu, I guess any ARM base will do. 23 or 64 bit. As it will be just a proof of concept. From the kernel config in

Code: Select all

https://judepereira.com/blog/wp-content/uploads/config.txt

I can determine it is 32 bit armel. There is no mention of 64bit os in the config.

As I got your attention, I am trying to start this week. Last night I try to install Openwrt in NAND. I need this to have an easy Recovery System in place. I soft bricked my unit in my attempt. I should have it working in some hours. My idea is to test boot script from USB. Today I was going to try a few of the method sugested in a few tutorials I found for quemu. I guess I need to start your scrypt 1rst.

My point, give me a few hours to recover and test. Hopefully what you just posted is what I need. THANK YOU

Note:
Is the thread in the right Place? Board index> Dog House> DebianDogs

[fcopurico]

Ahh I forgort
EDIT: Can you say what's exactly your goal ?

I had this Goflex Home, I replace it with new drive. And I am giving it another purpose.
Short story. This units worked as good as you can expect, 5-10 years I do not recalled. But the manufacturer put a kill switch on it. It made it dependable on a Providers Server. You could not change it settings without hoking up to the Internet. More BS just the usual, they still your password and your files. So to kill the units the manufacturer just removed the server. And many owners end up with a paper wait. There for a need for a need for a different internal OS. Simplest I guess Openwrt or many have premade Debian images. So there is no real need for an alternative.

I am a BusterDog user, I see its goodness I I thought I could use same advantages in the box. Most important is: RO OS + RW savefile. In general with BusterDog, If it breaks repairing will be as simple as removing the savefile. It will be a simple Linux Server, no need for full Busterdog installation. It has no Video output. What cat it do? It just a server, from Email handling, uTorrent, Personal Clowd, you can name it. It is a Low power device with a 1-4 TB Sata disk.

Normally I would no be as honest but as I was having fun, this is my story. 1rst Debian debootstrap build for Android. 1rst stage on PC second stage inside Android phone. Then I build them with Openwrk SDK. Recently just use Debian MultyArch capabilities. lately I build 2nd stage in qemu. But to my surprise you had show me a new method.

Funny I follow your script manually. Like a step by step tutorial. When I saw binfmt-support 1rts impression was: I do not need that. Wao I was wrong, That is a nice method. Instead of building it inside qemu. It uses qemu as an interpreter. What a surprise?

I have to do some personal issues during the day but hope to test the build rootfs today. As always THANKS

[fredx181]

Hi fcopurico,

Update: --- post content removed ---
Sorry, posted a setup here yesterday, including creating porteus-boot initrd1.xz, but wasn't thinking straight.
The initrd1.xz NOT suitable for ARM, it contains 32-bit bins and libs (i386) (e.g busybox), didn't realize that earlier.

EDIT: Did find a way now to include ARM busybox etc... in the porteus-boot initrd1.xz but I have no idea TBH , can initrd be used to boot on ARM?
Reading some about booting ARM based OS on the web I get the impression that it may need a different approach.

[fcopurico]

fredx181
As I was expecting, I could not work much on the weekend. Two good things happens. You introduce me with a new botstrap method with binfmt-support and you seems to post what I need with elogind. I have not been able to test it, but I believe you had covered 75% of what I needed from you.

Yes I desired porteus-boot. But that is not a most have. I have not try your new method in Bullseye. Or maybe just maybe Debian builds in live/persistence, is enough. Hope to find out this week.

0) This is Done: I build your armell Debian with your script, but not tested.

My plan for tonight.
1) Build a PC AMD64 image. Run it on my PC. Just to know what to expect.
2) Armel produced kernel, transformed to uBoot expected kernels
vmlinuz-4.19.0-18-marvell ===> uImage
initrd.img-4.19.0-18-marvell ==> uInitrd

They should look like this ones on Debian

Code: Select all

http://ftp.debian.org/debian/dists/stable/main/installer-armel/current/images/kirkwood/netboot/marvell/sheevaplug/uImage
http://ftp.debian.org/debian/dists/stable/main/installer-armel/current/images/kirkwood/netboot/marvell/sheevaplug/uInitrd

Note: those are not for Goflex Home, Debian do not offer them. But sheevaplug should be close enoght.

Needed transform commands are some what like:

Code: Select all

INITRD_IMAGE="initrd.img-4.19.0-18-marvell"
LINUX_TLD="vmlinuz-4.19.0-18-marvell"
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n $INITRD_IMAGE -d $INITRD_IMAGE uInitrd
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n $LINUX_TLD -d zImage.fdt uImage

But I will expect that something is missing.

I guess Debian build debs we could always use are:

Code: Select all

linux-image-4.19.0-18-marvell_4.19.208-1_armel.deb
linux-image-marvell_4.19+105+deb10u13_armel.deb 
linux-headers-4.19.0-18-marvell

3) Once I know uBoot kernels work on my device, I will try the full minimal OS build with your script.

4) Alternative method: Test the OS build with you script using my old uBoot kernels. This may be a simple way to test ahead the validity of the scrypt.

Lets see what I can do tonight.

[fcopurico]

I had waist all night trying just to build for my PC and run in AMD64. Well adding package live-boot and doing mksquashfs chroot filesystem.squashfs
And I have a 1GB for saving at /persitence
My grub4dos is:

Code: Select all

title Debianlive (live)
  find --set-root uuid () ##########
  kernel /live/vmlinuz boot=live toram=filesystem.squashfs persistence
  initrd /live/initrd.img

There is no error, system boot. At the end I hit [Enter] and system ask for login:
I type root and root password. But it does not let me login.

I did also added passwd to the scrypt. But results stay the same.

I even try to log in as a read only single user. By Adding init=/bin/bash to kernel line. This time I can enter to # without logging.
I try

Code: Select all

mount -n -o remount, rw /
passwd

But it did not help neither. I guess persistence file at is not loading.

Then boot again as emergency single user. Then tryied to save a test file.
Then rebooted in My usual Busterdog. Mounted the \persitence, and as expected the test file I save was not really save.

So toram=filesystem.squashfs do mount correctly to ram. But dmesg show no record of persitence.

Sorry did not have time to work for the Armel. I will try that next. I hope to here form you with a hit. See, if persitence work, that will meet my basic need while we get an answer if porteous could be use.

I see you tomorrow.

[fredx181]

Hi fcopurico,

Short reply for now:
For persistence to work I'd suggest (well, what works for me):
- required live-boot installed (anyway before initramfs is created in the chroot (or recreate initramfs later))
- have a file named "persistence.conf" inside the persistence savefile with content: / union

Why you can't login as root I don't know, did it output as "succesful" when you did set the passwd in chroot ?

PS. Wise to test first with amd64. If you come to stage of armel testing, tell me and I will try to come up with some.

[fredx181]

Hi fcopurico,

Did similar as you and works ok for me , logging in as root and persistence works too
Edited the bare script (from second post) for amd64 and added passwd and added live-boot to the install section:
EDIT: added later, install Devuan elogind (and libelogind0, libpam-elogind) as the Debian elogind cannot be trusted IMO (sooner or later Debian elogind will be replaced by systemd, depending on which packages install)

Code: Select all

#!/bin/bash

WORKDIR=debian-amd64
# WORKDIR=debian-armel  # edit; uncomment for armel
export ARCH=amd64
# export ARCH=armel  # edit; uncomment for armel

apt update    # may be required
apt install binfmt-support qemu-user-static debootstrap

# perhaps need to start service binfmt-support
service binfmt-support start

mkdir -p $WORKDIR/chroot && cd $WORKDIR 

qemu-debootstrap --arch=$ARCH --no-merged-usr --include=apt-transport-https,ca-certificates,apt-utils --keyring /usr/share/keyrings/debian-archive-keyring.gpg --variant=minbase --exclude=debfoster buster chroot http://ftp.debian.org/debian

echo "if preferred add contrib and non-free in chroot/etc/sources.list"
echo "make it: deb http://ftp.debian.org/debian buster main contrib non-free"
read -sp "Press ENTER to continue . . . "

# mount binds ...
mount --bind /proc chroot/proc
#mount --bind /tmp chroot/tmp
mount --bind /dev chroot/dev
mount --bind /sys chroot/sys
mount -t devpts devpts chroot/dev/pts
echo -en "`cat /etc/resolv.conf`" > chroot/etc/resolv.conf

############ Start running in chroot ############
chroot_in () {
export HOME=/root
export LC_ALL=C
echo "APT::Install-Recommends "false"; APT::Install-Suggests "false";" > /etc/apt/apt.conf
echo "Acquire::Check-Valid-Until "0";" >> /etc/apt/apt.conf

passwd   # edited 

# install some basic packages, edit as desired
apt update  # may be required
apt-get install whiptail keyboard-configuration sysvinit-core xz-utils cryptsetup cryptsetup-bin gnupg dirmngr apt-utils wget udev libcap2 dbus live-boot live-tools --yes

#### Install elogind from Devuan ####
echo "Install elogind from Devuan" 
echo "(Debian elogind can be problematic as it conflicts with policykit-1):"
echo "From: https://pkgmaster.devuan.org/devuan/pool/main/e/elogind/"
wget --no-check-certificate "https://pkgmaster.devuan.org/devuan/pool/main/e/elogind/elogind_241.4-2_$ARCH.deb" "https://pkgmaster.devuan.org/devuan/pool/main/e/elogind/libelogind0_241.4-2_$ARCH.deb" "https://pkgmaster.devuan.org/devuan/pool/main/e/elogind/libpam-elogind_241.4-2_$ARCH.deb"
# wget --no-check-certificate "https://github.com/doglinux/busterdog/releases/download/v1.0/elogind_241.4-2_armel.deb" "https://github.com/doglinux/busterdog/releases/download/v1.0/libelogind0_241.4-2_armel.deb" "https://github.com/doglinux/busterdog/releases/download/v1.0/libpam-elogind_241.4-2_armel.deb"
dpkg -i *.deb
apt -f install   # force installing dependencies (if required)
# rm -f *.deb # uncomment to remove the elogind debs

dbus-uuidgen > /var/lib/dbus/machine-id
echo "live" > /etc/hostname   # change 'live' to what's preferred
echo "127.0.0.1	 localhost" > /etc/hosts
echo "127.0.1.1	 live" >> /etc/hosts    # change 'live' to what's preferred
mkdir /live
mkdir -p /opt/bin

[ $ARCH = armel ] && ARCH=marvell     # edited for armel (Kirkwood)
# [ $ARCH = armel ] && ARCH=rpi    # edited for armel, uncomment for "linux-image-rpi" (Raspberry Pi)
# install kernel
apt install linux-image-$ARCH     # edit: for armel make it "linux-image-marvell" or "linux-image-rpi"

# install more as desired ...
# .....
}
export -f chroot_in

# run chroot_in function
chroot chroot /bin/bash -c chroot_in

# UNMOUNT at the end !!
umount chroot/dev/pts
umount chroot/sys
umount chroot/proc
umount chroot/dev

Finally I made it into a live folder containing the required files and put the live folder in "btest" folder:
My menu.lst entry:

Code: Select all

title btest
 uuid 12e1e33f-47a6-4979-b13a-404e60f746b9
 kernel /btest/live/vmlinuz boot=live live-media-path=/btest/live persistence
 initrd /btest/live/initrd.img

My persistence file (included "persistence.conf") is on sda7, also "btest' is on sda7
Booted OK, could login, but needed to press enter first a few times, it's a bit inconvenient.

Once logged in I typed losetup -a (for to see if persistence is loaded), output:

Code: Select all

/dev/loop1: [2055]:264 (/run/live/persistence/sda7/persistence)
/dev/loop0: [2055]:3014661 (/run/live/persistence/sda7/btest/live/filesystem.squashfs)

Made a few changes and found them later in the mounted persistence file (from Busterdog) so it works.

[fcopurico]

I burn a day just understanding the uBoot (uInitrd & uImage). I guess I was confuse, as I thought I needed a file name intranfs. Now I guess I was wrong that initranfs is just a name some one give to uInitrd. I have not tested this theory, but that is what (-n) seems to be use for. The the point is that I waisted one day of work.

So I added some lines to your script to build uBoot's uInitrd & uImage. And the filesystem.squashfs. And this time I tried on USB. I place my persistence in 3 places hopping it gets pickup. In \, \live\ and \root\. For some reason kernel just get lost, It just hangs.

So I decided to booted in yesterday files on PC's HD again. Without noticing I left the USB attached. Boy this is crazy, the old nonworking login, start working. So I logged. And the persistence file pickup was from the USB!!

So in this weird behaviuor. What I see is happening is that if the persistence file is not pickup then the login fails. Weird no? Persistence is empty with only the Flag persistence.conf. So I removed USB and there I fail again to login.

I will be ignoring this PC stuff. I will concentrate in ARM with your new script. I am going to rest some hours and try again latter. But from your comments I have all the help I needed. You prove it works. I only need to tested in my Arm device. This has been a nice learning experience. I never thought that devian persistence was was to use. Well once I get a hold of its secretes. I see you use live-media-path=/btest/live persistence

As always THANKs for your support.

[fredx181]

I place my persistence in 3 places hopping it gets pickup. In \, \live\ and \root\.

If persistence file is on the root (/) of a partition, it should be found automatically (if specified 'persistence')
Otherwise need to specify persistence-path=
From https://manpages.debian.org/buster/live ... .7.en.html :

persistence-path=PATH
live-boot will look for persistency files in the root directory of a partition, with this parameter, the path can be configured so that you can have multiple directories on the same partition to store persistency files.

Strange that you cannot login, what exacly happens when you try ? And did you try from another tty, e.g. tty2 ?
EDIT: Perhaps setting up auto-login helps, in /etc/inittab, find the line:
1:2345:respawn:/sbin/getty 38400 tty1
And replace with:

Code: Select all

# 1:2345:respawn:/sbin/getty 38400 tty1  # comment out existing
1:2345:respawn:/bin/login -f root tty6 </dev/tty1 >/dev/tty1 2>&1   # for auto-login

[fcopurico]

fredx181
I know you are a good guy. Trying to help me anyway you can. And in fact you done most of what I needed from you. porteus-boot is not needed for now, If have persistence. For this project one save file is enough. You show me qemu-user-static as an interpreter, this is fun. And your last script with devuan-elogind is a hit.

The good news is that I have running the device with our Debian in the USB. Well as a regular Linux file system structure for now. squashfs+persitence should not be a mayor thing.

But to be honest with time I am starting to get frustrated with Linux and the developers. In the past the more experience mean faster we work. But today people had lost their minds. In this case developers. They keep changing things. So experience worth nothing, as to do every little thing we need to spend hours in the web. Always trying to find out why what it used to work now it does not. Any little change become a mayor project. This is ridiculous.

I do not know, how to explain easily. Let me try. There used to be many kernels deb, one per CPU. It seems they had move to a more general Kernel. The CPU/Board specifics are place in a dtb container. But Debian had posted only a very limited dtb. In my case I am luck that projects.doozan.com had my kirkwood-goflexhome.dtb.

I had my device working with what doozan.com offer. So my device can Auto boot from USB, Sata or NAND. In that order. If I do not have USB, then it boot from Sata if not from NAND. But I wanted to build this RO filesystem + RW save file. Because I will be placing this unit far from my reach. So my answer was, if it breaks I just delete my savefile replace with a backup. Easy to do from far away. And even I can create an auto correction scheme. Backing up automatic every day. If HDD fail to boot then the NAND can replace the backup. I bet you see my point.

Boy I did not knew this was so difficult. In general doozan.com quick answer is just use their kernels. But I need mine build my way. There is no easy explanation how is done. Well to make it simple I had spent the last few days trying to compile/boot in hundreds of different ways. In general uBoot rejects out initrd claiming

Loading init Ramdisk from Legacy Image at 01100000 ...
Image Name: initramfs-4.19.0-18-marvell
Created: 2022-02-17 20:06:22 UTC
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 15665801 Bytes = 14.9 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... Bad Data CRC
Ramdisk image is corrupt or invalid

So after many searches I found a way to stop uBoot and manually send commands to boot. Then the error became more clear. Done, until I was able to boot. But I was force to kill the USB to HDD to NAND thing. Witch is the whole point. So I guess I will need a few more days in this story to get my thing as I want it to be.

Lets resume. In general your script works 100% in AMD. In general it does too in armel. I did have a few small changes. But as I have not been able to have it run smoothly in my device I will work it a few more days. Hopefully you see me back in a few days, with my response on what I did. Notice that your scrip is not the issue. Instead is how we manipulate the generic linux-image-4.19.0-18-marvell.deb to create expected uBoot files for a dedicated CPU+Board. In this case GoflexHome.

[spotted]

Hello fcopurico, are you still around. I have an odroidN2+ ARM board gathering dust because of that sudo jerk always interfering with what I want to do. Can you do a write up starting with what bullseye image to use. Thanks