Strange error during assembly ( KL-Arch ) It's decided - Closed

[Sofiya]

Strange error during assembly ---- on Arch

Note:
KLA-OT2baseCE

the package database is not synchronized
the build fails

Code: Select all

:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)
:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)

[wiak]

Strange error during assembly ---- on Arch

Note:
KLA-Manjaro-Openbox-CE-1.1 - is going to

the package database is not synchronized
the build fails

Code: Select all

:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)
:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)

Could you provide more details please? What build command are you using and what is the plugfile involved if any?
Ah, it is Manjaro, so presumably you are doing a firstribit type of build? i.e. extracting sfs files out of official Manjaro and so on?

[Sofiya]

Strange error during assembly ---- on Arch

Note:
KLA-Manjaro-Openbox-CE-1.1 - is going to

the package database is not synchronized
the build fails

Code: Select all

:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)
:: Synchronizing package databases...
error: failed to synchronize all databases (unexpected error)

Could you provide more details please? What build command are you using and what is the plugfile involved if any?
Ah, it is Manjaro, so presumably you are doing a firstribit type of build? i.e. extracting sfs files out of official Manjaro and so on?

Manjaro -- Manjaro has nothing to do with it and is not related in any way

KLA-OT2baseCE
./build_firstrib_rootfs.sh arch default amd64 f_00_Arch_amd64-openboxBASE_jgmenu_600r5.plug

https://forum.puppylinux.com/viewtopic.php?t=9996

[Xyz]

I couldn't update Kla Hyprland either, this solved the problem for me.:

https://arcolinuxforum.com/viewtopic.php?t=3699

[Sofiya]

I couldn't update Kla Hyprland either, this solved the problem for me.:

https://arcolinuxforum.com/viewtopic.php?t=3699

I mean update ( full build from build script )

[wiak]

I'll try a build tonight if I can

[Sofiya]

I'll try a build tonight if I can

It seems to me that (Arch) as always overdid it with (Pacman) a ( pacman ) this is the heart of Arch

[Sofiya]

In general, as I said, something has changed in "pamac"

if during build in firstrib_rootfs/etc/pacman.conf comment out the line 39 #DownloadUser = alpm
database synchronization will be successful ,the assembly will continue

in archstrap_wiak_mod.sh dependencies added

archstrap_wiak_mod.sh

Code: Select all

#!/bin/bash
# Create archlinux rootfs without archlinux
set -e
# Parsing argumenst
rootfs=$(realpath "$1")
shift
while getopts -- ':r:g:i:p:' OPTION; do
    case $OPTION in
    r)
      mirror="${OPTARG[@]}"
      ;;
    g)
      format="${OPTARG[@]}"
      ;;
    i)
      ignore="${OPTARG[@]}"
      ;;
    p)
      pacman_conf="$(realpath "${OPTARG[@]}")"
      ;;
    *)
      echo "usage: archstarp (rootfs) [options]"
      echo "    -i : disable repo section"
      echo "    -r : used repo url"
      echo "    -g : repo index archive format"
      echo "    -p : use custom pacman.conf file"
      exit 1
    esac
done
[[ $UID -ne 0 ]] && echo "You must be root" && exit 31
# Define and check variables
[[ "$rootfs" == "" ]] && echo "Rootfs directory is invalid" && exit 1
[[ "$arch" == "" ]] && arch=$(uname -m)
[[ "$mirror" == "" ]] && mirror="https://mirrors.edge.kernel.org/archlinux/\$repo/os/\$arch"
[[ "$format" == "" ]] && format="tar.gz"
repo=$(echo "https://mirrors.edge.kernel.org/archlinux/\$repo/os/\$arch" | sed "s/\$repo/core/g;s/\$arch/$arch/g")
packages=(filesystem pacman base brotli coreutils bash ncurses libarchive openssl zstd curl gpgme
          expat xz lz4 bzip2 archlinux-keyring zlib libssh2 libassuan idn2 libpsl krb5
          libnghttp2 libnghttp3 e2fsprogs keyutils pacman-mirrorlist gnupg sqlite libgcrypt gawk
          file p11-kit findutils libp11-kit libtasn1 libffi mpfr gmp grep pcre libcap libxml2 icu audit libcap-ng attr pcre2 libxcrypt pam libseccomp
gcc-libs gettext glibc gzip iproute2 iputils licenses pciutils procps-ng psmisc shadow systemd systemd-sysvcompat tar util-linux) #wiak added
# Creating base directories
mkdir -p "$rootfs" && cd "$rootfs"
mkdir -p "$rootfs"/tmp/core-db
mkdir -p "$rootfs"/var/cache/pacman/pkg
mkdir -p "$rootfs"/etc/pacman.d/
# Define functions
fetch(){
    set +e
    url="$1"
    dest="$2"
    echo "D: $(basename $url)"
    if [[ ! -f "$dest" ]] ; then
        wget -c "$url" -O "$dest"
    fi
    set -e
}
deplist=()
getdep(){
    for pkg in $@ ; do
        name=$(echo $pkg | sed "s/>.*//g;s/<.*//g;s/>=*//g")
        desc=$(ls "$rootfs"/tmp/core-db/"$name"-[0-9]*/desc 2>/dev/null | tail -n 1)
        [[ ! -f "$desc" ]] && continue
        deps=()
        enable="false"
        echo $name
        cat $desc | while read line ; do
            line=$(echo $line | sed "s/>.*//g;s/<.*//g;s/>=*//g")
            if [[ "$line" == "%DEPENDS%" ]] ; then
                enable="true"
            elif [[ "$line" == "" ]] ; then
                enable="false"
            elif [[ "$enable" == "true" ]] ; then
                if ! echo ${deplist[@]} | grep $line &>/dev/null ; then
                    deplist+=($line)
                    getdep $line
                fi
            fi
        done
    done
}
get_url(){
    for pkg in $@ ; do
        name=$(echo $pkg | sed "s/>.*//g;s/<.*//g;s/>=*//g")
        desc=$(ls "$rootfs"/tmp/core-db/"$name"-[0-9]*/desc 2>/dev/null | tail -n 1)
        [[ ! -f "$desc" ]] && continue
        cat $desc | while read line ; do
            if [[ "$line" == "%FILENAME%" ]] ; then
                read filename
                echo $filename
            fi
        done
    done
}

# Download core database
cd "$rootfs"/tmp/core-db
fetch "$repo/core.db.$format" core.db.$format || true
tar -xf core.db.$format
cd "$rootfs"
# Install standalone pacman in rootfs
getdep ${packages[@]} | sort | uniq | while read pkg ; do
    echo $pkg
    name=$(get_url $pkg)
    fetch "$repo"/$name "$rootfs"/var/cache/pacman/pkg/$name
    tar -xf "$rootfs"/var/cache/pacman/pkg/$name || true
    rm -f "$rootfs"/.INSTALL "$rootfs"/.MTREE "$rootfs"/.BUILDINFO || true
done
# mirror list write
touch "$rootfs"/etc/pacman.d/mirrorlist
if ! grep "archstrap" "$rootfs"/etc/pacman.d/mirrorlist ; then
    echo "" >> "$rootfs"/etc/pacman.d/mirrorlist
    echo "# Added by archstrap" >> "$rootfs"/etc/pacman.d/mirrorlist
    echo "Server = $mirror" >> "$rootfs"/etc/pacman.d/mirrorlist
fi
# install custom pacman.conf if available
if [[ -f "${pacman_conf}" ]] ; then
    cat "${pacman_conf}" > "$rootfs"/etc/pacman.conf
fi
# pacman.conf configurations
sed -i 's/^CheckSpace/#CheckSpace/g' "$rootfs"/etc/pacman.conf
sed -i 's/^#ParallelDownloads/ParallelDownloads/g' "$rootfs"/etc/pacman.conf
sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf
sed -i "s/^[[:space:]]*\(CheckSpace\)/# \1/" "$rootfs"/etc/pacman.conf
sed -i "s/^[[:space:]]*SigLevel[[:space:]]*=.*$/SigLevel = Never/" "$rootfs"/etc/pacman.conf
for i in $ignore ; do
    sed -i -e '/^\['$i'\]/ { N; N; d; }' "$rootfs"/etc/pacman.conf
done
# Fix file conflict error
rm -rf "$rootfs/var/spool/mail"
# Dns configurations
cat /etc/resolv.conf > "$rootfs"/etc/resolv.conf
# Bind system
for i in dev sys proc run ; do
    mount --bind /$i "$rootfs"/$i
done
# Install base
busybox chroot "$rootfs" update-ca-trust
busybox chroot "$rootfs" pacman -Sy || true
mkdir -p "$rootfs"/newroot/var/lib/pacman/
mkdir -p "$rootfs"/newroot/var/cache/pacman/
cp -prf "$rootfs"/var/cache/pacman/* "$rootfs"/newroot/var/cache/pacman/
busybox chroot "$rootfs" pacman -Syy base bash pacman --noconfirm --overwrite \* --root=/newroot
# Umount all
for i in dev sys proc run ; do
    while umount -lf -R "$rootfs"/$i 2>/dev/null ; do true ; done
done
# replace rootfs with existing
mkdir -p "$rootfs/garbage"
mv "$rootfs"/* "$rootfs/garbage" 2>/dev/null || true
mv "$rootfs/garbage/newroot/"* "$rootfs/"
rm -rf "$rootfs/garbage"

# Dns configurations
cat /etc/resolv.conf > "$rootfs"/etc/resolv.conf
touch "$rootfs"/etc/pacman.d/mirrorlist

if ! grep "archstrap" "$rootfs"/etc/pacman.d/mirrorlist ; then
    echo "" >> "$rootfs"/etc/pacman.d/mirrorlist
    echo "# Added by archstrap" >> "$rootfs"/etc/pacman.d/mirrorlist
    echo "Server = $mirror" >> "$rootfs"/etc/pacman.d/mirrorlist
fi

busybox chroot "$rootfs" pacman-key --init
busybox chroot "$rootfs" pacman -Syyu --noconfirm
# pacman.conf configurations
sed -i 's/^CheckSpace/#CheckSpace/g' "$rootfs"/etc/pacman.conf
sed -i 's/^#ParallelDownloads/ParallelDownloads/g' "$rootfs"/etc/pacman.conf
sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf
sed -i "s/^[[:space:]]*\(CheckSpace\)/# \1/" "$rootfs"/etc/pacman.conf
sed -i "s/^[[:space:]]*SigLevel[[:space:]]*=.*$/SigLevel = Never/" "$rootfs"/etc/pacman.conf

[Sofiya]

Let's automate it - by adding code in archstrap_wiak_mod.sh

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf
And
sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

new " archstrap_wiak_mod.sh "

[Sofiya]

The assembly is going as quickly as expected.

Who's a good girl? I'm a good girl !!!

[Sofiya]

************************************ Topic Closed ***********************************************

[rockedge]

If you don't praise yourself, no one will.

Didn't I say something nice? I'll say again, Good work with the quick fix!

[Sofiya]

If you don't praise yourself, no one will.

Didn't I say something nice? I'll say again, Good work with the quick fix!

Don't pay attention to me, I'm just being capricious.

[Sofiya]

All updates have been made to GitLab.
Also, the "pamac manager" for Arch is unified to the latest version. Build from scripts works as before In normal mode

[wiak]

Let's automate it - by adding code in archstrap_wiak_mod.sh

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf
And
sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

new " archstrap_wiak_mod.sh "

Ah, good you found that. I've only just found time to login to the forum tonight or even view it. Been busy so found no time to investigate and happy I don't now need to... Okay I better remember where this post is so I can modify my own copy of archstrap_wiak_mod.sh.

[Sofiya]

Let's automate it - by adding code in archstrap_wiak_mod.sh

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf
And
sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

new " archstrap_wiak_mod.sh "

Ah, good you found that. I've only just found time to login to the forum tonight or even view it. Been busy so found no time to investigate and happy I don't now need to... Okay I better remember where this post is so I can modify my own copy of archstrap_wiak_mod.sh.

Here is your " archstrap_wiak_mod.sh "

https://gitlab.com/firstrib/firstrib/-/ ... iak_mod.sh

[wiak]

Let's automate it - by adding code in archstrap_wiak_mod.sh

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf
And
sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

new " archstrap_wiak_mod.sh "

Ah, good you found that. I've only just found time to login to the forum tonight or even view it. Been busy so found no time to investigate and happy I don't now need to... Okay I better remember where this post is so I can modify my own copy of archstrap_wiak_mod.sh.

Here is your " archstrap_wiak_mod.sh "

https://gitlab.com/firstrib/firstrib/-/ ... iak_mod.sh

So all of a sudden, we have two new pacman.conf options (well spotted Sofiya):

DownloadUser = username

Specifies the user to switch to for downloading files. If this config option is not set then the downloads are done as the user running pacman.
DisableSandbox

Disable the default sandbox applied to the process downloading files on Linux systems. Useful if experiencing landlock related failures while downloading files when running a Linux kernel that does not support this feature.

However, I just noted that I had some extra lines at the end of my original arch_wiak_mod.sh that crept in there by accident I think; do no harm but seem superfluous...

It may be a good idea to also use the new DisableSandbox option; I will think about this...

In above, Sofiya, I think the correct one is likely to be:

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf

rather than ever:

sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

but perhaps through experiment you have found differently?

Anyway - I'll fix my original archstrap_wiak_mod.sh and try it when I have time (but I have no time...).

But... that bash script archstrap.sh is long not maintained by its originator (2 years). I will likely move to using official arch small bootstrap release and build on top of that via built_firstrib_rootfs.sh f_plugin since official arch bootstrap release will always be well-maintained, which is the important thing - can also use that to make other small KLA/FR releases; but again might be a while till I move to using official arch small bootstrap release: https://geo.mirror.pkgbuild.com/iso/latest/ File: archlinux-bootstrap-x86_64.tar.zst

[Sofiya]

Ah, good you found that. I've only just found time to login to the forum tonight or even view it. Been busy so found no time to investigate and happy I don't now need to... Okay I better remember where this post is so I can modify my own copy of archstrap_wiak_mod.sh.

Here is your " archstrap_wiak_mod.sh "

https://gitlab.com/firstrib/firstrib/-/ ... iak_mod.sh

So all of a sudden, we have two new pacman.conf options (well spotted Sofiya):

DownloadUser = username

Specifies the user to switch to for downloading files. If this config option is not set then the downloads are done as the user running pacman.
DisableSandbox

Disable the default sandbox applied to the process downloading files on Linux systems. Useful if experiencing landlock related failures while downloading files when running a Linux kernel that does not support this feature.

However, I just noted that I had some extra lines at the end of my original arch_wiak_mod.sh that crept in there by accident I think; do no harm but seem superfluous...

It may be a good idea to also use the new DisableSandbox option; I will think about this...

In above, Sofiya, I think the correct one is likely to be:

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf

rather than ever:

sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf

but perhaps through experiment you have found differently?

Anyway - I'll fix my original archstrap_wiak_mod.sh and try it when I have time (but I have no time...).

But... that bash script archstrap.sh is long not maintained by its originator (2 years). I will likely move to using official arch small bootstrap release and build on top of that via built_firstrib_rootfs.sh f_plugin since official arch bootstrap release will always be well-maintained, which is the important thing - can also use that to make other small KLA/FR releases; but again might be a while till I move to using official arch small bootstrap release: https://geo.mirror.pkgbuild.com/iso/latest/ File: archlinux-bootstrap-x86_64.tar.zst

This disables during assembly so that synchronization can take place.

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf Line 123

and this one returns back to the default mode after synchronization has taken place

sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf Line 169

Original lines of " pacman.conf " from the package

Code: Select all

# Misc options
#UseSyslog
#Color
#NoProgressBar
CheckSpace
#VerbosePkgLists
#ParallelDownloads = 5
DownloadUser = alpm
#DisableSandbox

[wiak]

This disables during assembly so that synchronization can take place.

sed -i 's/^DownloadUser/#DownloadUser/g' "$rootfs"/etc/pacman.conf Line 123

and this one returns back to the default mode after synchronization has taken place

sed -i 's/^#DownloadUser/DownloadUser/g' "$rootfs"/etc/pacman.conf Line 169

Original lines of " pacman.conf " from the package

Code: Select all

# Misc options
#UseSyslog
#Color
#NoProgressBar
CheckSpace
#VerbosePkgLists
#ParallelDownloads = 5
DownloadUser = alpm
#DisableSandbox

Yes I realised that, and alpm is Arch Linux Package Manager (user) ... I think ..., which is for the case of this new Sandbox scenario - that's why I was suggesting maybe best instead to disable that sandbox (via DisableSandbox option) since as the man page says:

Disable the default sandbox applied to the process downloading files on Linux systems. Useful if experiencing landlock related failures while downloading files when running a Linux kernel that does not support this feature.

and we use great variety of kernels. However, may or may not be best to disable sandbox in practice.

[wiak]

But... that bash script archstrap.sh is long not maintained by its originator (2 years). I will likely move to using official arch small bootstrap release and build on top of that via built_firstrib_rootfs.sh f_plugin since official arch bootstrap release will always be well-maintained, which is the important thing - can also use that to make other small KLA/FR releases; but again might be a while till I move to using official arch small bootstrap release: https://geo.mirror.pkgbuild.com/iso/latest/ File: archlinux-bootstrap-x86_64.tar.zst

NO... I've definitely changed my mind... the rootfs created out of archlinux-bootstrap-x86_64.tar.zst is huge (1.1 GB) before adding anything (I just tried it)! We will stick with that archstrap for the moment.

[Sofiya]

But... that bash script archstrap.sh is long not maintained by its originator (2 years). I will likely move to using official arch small bootstrap release and build on top of that via built_firstrib_rootfs.sh f_plugin since official arch bootstrap release will always be well-maintained, which is the important thing - can also use that to make other small KLA/FR releases; but again might be a while till I move to using official arch small bootstrap release: https://geo.mirror.pkgbuild.com/iso/latest/ File: archlinux-bootstrap-x86_64.tar.zst

NO... I've definitely changed my mind... the rootfs created out of archlinux-bootstrap-x86_64.tar.zst is huge (1.1 GB) before adding anything (I just tried it)! We will stick with that archstrap for the moment.

take a look at " arch linux-bootstrap-x86_64.tar.zst "
Yes, I also wanted to write this, about the size