Problem using the sandbox

[je55eah]

Additional sfs files will be offered by the menu when starting a sandbox if those sfs files are loaded in the host system before starting a sandbox.

Code: Select all

load_sfs.sh --load sandboxsetup.sfs
rw-sandbox.sh

I have also created an autologin-fix.sfs to patch the lxc sandboxes which are started with only the first two layers until the next release.

Code: Select all

load_sfs.sh --load autologin-fix.sfs
load_sfs.sh --load sandboxsetup.sfs
rw-sandbox-lxc.sh

[je55eah]

If I create a sandbox and then immediately run sb2dir and dir2sfs the resulting sfs contains 11 files, some symlinks, some directories, and some tty things. Why isn't it empty and should these files be stripped out of every sfs because it seems like they will be included in each of them?

They are not in the autologin-fix.sfs I posted earlier because I created that sfs by simply creating a directory containing bin and autologin and runniing dir2sfs on it.

Here is the sfs containing unexpected files:
https://www.dropbox.com/s/w4iop56vgmac4ba/test.sfs?dl=0

[je55eah]

How does the xargs sed part transform these results?:

Code: Select all

# ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]*
/sys/fs/aufs/si_627cabd53966b79d/br0
/sys/fs/aufs/si_627cabd53966b79d/br1
/sys/fs/aufs/si_627cabd53966b79d/br2
/sys/fs/aufs/si_627cabd53966b79d/br3
/sys/fs/aufs/si_627cabd53966b79d/br4
/sys/fs/aufs/si_627cabd53966b79d/br5
/sys/fs/aufs/si_627cabd53966b79d/br6
/sys/fs/aufs/si_627cabd53966b79d/br7
# ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]* | xargs sed 's/=.*//';
/aufs/pup_rw
/aufs/pup_save
/aufs/pup_multi
/aufs/pup_ro11
/aufs/pup_ro10
/aufs/pup_ro
/aufs/kernel-modules
/aufs/pup_init
# 

[je55eah]

In the following code from rw-sandbox.sh:

Code: Select all

	} else if (mode == 2) {
		# get list of loop devices and files - index is $1 (loop devs)
		sub(/:/,"",$1)
		sub(/.*\//,"",$3); sub(/)/,"",$3)
		loopdev[$1]=$3

I believe it is stripping out the colon from $1 and the path from $3 as well as the final closing parenthesis from $3, but what is it doing with the entries which end with (deleted)) in $4?

Code: Select all

# losetup-FULL -a;
/dev/loop1: [0002]:1028 (/fd64.sfs (deleted))
/dev/loop11: [0021]:2665 (/aufs/pup_save/home/spot/Downloads/sandboxsetup.sfs)
/dev/loop0: [0002]:1035 (/kernel-modules.sfs (deleted))
/dev/loop10: [0021]:2620 (/aufs/pup_save/home/spot/Downloads/autologin-fix.sfs)

[jamesbond]

If I create a sandbox and then immediately run sb2dir and dir2sfs the resulting sfs contains 11 files, some symlinks, some directories, and some tty things. Why isn't it empty and should these files be stripped out of every sfs because it seems like they will be included in each of them?

Because you're using sandbox-lxc. While sb2dir works for sandbox and sandbox-lxc, it is more tuned for the standard sandbox. sandbox-lxc is used more to isolate applications, rather than for package building. You're welcome to use it for anything else, but as I said earlier, it's YMMV.

[jamesbond]

How does the xargs sed part transform these results?:

I will answer the question here for your sake, but going forward, please refrain from:
a) posting the same question in multiple threads,
b) posting out of topic questions (even in your own thread).

This is a generic programming question, and has nothing to do with "problems with sandbox".
It should have been posted in the "Programming" section.
There, it would have attracted more people to help you, and the answers would have also benefited more people instead of only yourself.

This code:

Code: Select all

ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]* | xargs sed 's/=.*//';

is functionally equivalent to:

Code: Select all

ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]* | while read -r filename; do
    sed 's/=.*//' $filename
done

But the top one is faster.

[je55eah]

@jamesbond

Thanks,

The xargs almost looks redundant to me, but that is probably because I don't fully understand it.

The real mystery is how erasing =.* transforms /sys/fs/aufs/si_627cabd53966b79d/br0 into /aufs/pup_rw

Hopefully I didn't post the same question in multiple threads. I see how this is a general programming question. It's also a question about the sandbox script. My mistake.

[JakeSFR]

The real mystery is how erasing =.* transforms /sys/fs/aufs/si_627cabd53966b79d/br0 into /aufs/pup_rw

In the above code sed works on files (their contents), not filenames.

Greetings!

[je55eah]

@JakeSFR
hmm, I think the contents of file are the output from ls presented as file contents through the pipe. Each line is grabbed by xargs and fed into sed which is printing the list of filenames into a temporary file by the next step.

[step]

@JakeSFR
hmm, I think the contents of file are the output from ls presented as file contents through the pipe. Each line is grabbed by xargs and fed into sed which is printing the list of filenames into a temporary file by the next step.

Look again at @jamesbond's careful use of $filename to explain xargs, then bring in @JakeSFR's distinction between file and filename. That command line is processing the contents of files named by the output of ls. Hint: find out what's in those files...

[jamesbond]

Ok, three things here.

1.

Hopefully I didn't post the same question in multiple threads.

I mean this and this.

2.
However due to your persistence, I have re-written sb2dir.sh to drop sandbox-created files from either sandbox or sandbox-lxc.

Download the attached file, gunzip it, and put it in /usr/bin replacing the existing sb2dir.sh (do this from OUTSIDE of the sandbox, not inside). Run sb2dir.sh --help and read the note carefully. "--clear-lxc" is the flag that you want.

However, please note that this __comes with a price__. When sb2dir drop, for example, /etc/hosts from sandbox-lxc, that means it will always drop that file __EVEN IF__ you manually edited that file and wants it to be saved. sb2dir is dumb that way, so that's just the price that you have to pay. If you have further special needs, please feel free to edit and customise sb2dir yourself.

3.
And finally, to follow up what @step and @JakeSFR already advised you, run the following and compare the results.

Code: Select all

ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]* | while read -r filename; do
    cat $filename
done

Compared with

Code: Select all

ls -v /sys/fs/aufs/si_627cabd53966b79d/br[0-9]* | while read -r filename; do
    cat $filename | sed 's/=.*//'
done

[je55eah]

@jamesbond @JakeSFR @step

Thanks fellas, I see it now. Xargs is required because it is triggering sed to change the contents of each file in the list processed by xargs. Each of those files contains the name that surprised me and sed strips away the ending beginning with the equals sign. and prints them out into a list for further processing.

James,

It didn't seem like the same thing to me. I appreciate what you have done, and I feel bad if you are telling me that I have burdened you with apparent special needs.

The contents of test.sfs are:

Code: Select all

/dev
/dev/console
/dev/fd
/dev/full
/dev/lxc
/dev/lxc/console
/dev/lxc/tty1
/dev/lxc/tty2
/dev/lxc/tty3
/dev/lxc/tty4
/dev/ptmx
/dev/random
/dev/stderr
/dev/stdin
/dev/stdout
/dev/tty
/dev/tty1
/dev/tty2
/dev/tty3
/dev/tty4
/dev/urandom
/dev/zero
/etc
/etc/netparams
/etc/rc.d
/etc/rc.d/rc.network.lxc
/etc/rc.d/rc.shutdown.lxc
/sbin
/sbin/poweroff
/sbin/reboot
/tmp
/tmp/utmp

The files I wrote about in the other thread are:

Code: Select all

/usr/bin/xwin
/usr/bin/wmexit
/mnt/sb
/mnt/sb/sandbox
/mnt/sb/sandbox/usr
/mnt/sb/sandbox/usr/bin
/mnt/sb/sandbox/usr/bin/xwin
/mnt/sb/sandbox/usr/bin/wmexit
/mnt/sb/sandbox/mnt
/mnt/sb/sandbox/mnt/sb
/mnt/sb/sandbox/mnt/sb/sandbox
/mnt/sb/sandbox/etc
/mnt/sb/sandbox/etc/profile
/mnt/sb/sandbox/etc/shinit
/mnt/sb/sandbox/etc/BOOTSTATE
/mnt/sb/sandbox/.wh..wh.orph
/mnt/sb/sandbox/.wh..wh.plnk
/mnt/sb/sandbox/.wh..wh.aufs
/etc/shinit

neither of those lists contain /etc/hosts

My aim here is to understand what is going on and I have been reporting my discoveries and asking for help with the things that confuse me. I now suspect that those files mentioned in the other thread were still not an accurate description of the first layer. Perhaps the base layer is in initrd and it is incorporated into pup_init at boot? I don't know.

Examining your new script, I see that there are some similar files and some differences:

Code: Select all

clear_aufs() {
	sed -ne '
# remove all the whiteout files
\_.wh._ d
p'
}

clear_std() {
	clear_aufs | sed -ne '
# sandbox created stuff and bash history
\_/etc/\.XLOADED$_ d
\_/etc/BOOTSTATE$_ d
\_/etc/profile$_ d
\_/etc/shinit$_ d
\_/usr/bin/xwin$_ d
\_/usr/bin/wmexit$_ d
\_/root/\.Xauthority$_ d
\_/root/\.history$_ d
\_/root/\.bash\_history$_ d
p'
}

clear_lxc() {
	clear_aufs | clear_std | sed -ne '
# and sandbox-lxc created stuff
\_/etc/rc\.d/rc\.sysinit\.lxc$_ d
\_/etc/rc\.d/rc\.shutdown\.lxc$_ d
\_/etc/rc\.d/rc\.network\.lxc$_ d
\_/etc/rc\.d/rc\.oneshot$_ d
\_/etc/X11/host-X11$_ d
\_/etc/hosts$_ d
\_/etc/hostname$_ d
\_/etc/inittab$_ d
\_/etc/netparams$_ d
\_/sbin/init$_ d
\_/sbin/poweroff$_ d
\_/sbin/reboot$_ d
\_/var/run/utmp$_ d
\_/var/run/wtmp$_ d
\_/dev/lxc/_ d; \_/dev/fd$_ d
\_/dev/console$_ d; \_/dev/stdin$_ d; \_/dev/stdout$_ d; \_/dev/stderr$_ d
\_/dev/full$_ d; \_/dev/null$_ d; \_/dev/zero$_ d
\_/dev/ptmx$_ d; \_/dev/random$_ d; \_/dev/urandom$_ d
\_/dev/tty$_ d; \_/dev/tty1$_ d; \_/dev/tty2$_ d; \_/dev/tty3$_ d; \_/dev/tty4$_ d;
p'
}

I'm impressed that you have a mental map of all the thousands of moving parts that make Linux work.

[jamesbond]

I appreciate what you have done, and I feel bad if you are telling me that I have burdened you with apparent special needs.

Don't worry about it. If we can help, we will. If we can't, well, that's just life.

My aim here is to understand what is going on and I have been reporting my discoveries and asking for help with the things that confuse me.

Then tell us what's your background. Are use an existing Linux user (from other distro)? Do you come from Windows? Etc. Then we don't have to guess what you already know and what you don't know yet, and we can go straight to the point.

I now suspect that those files mentioned in the other thread were still not an accurate description of the first layer.
Perhaps the base layer is in initrd and it is incorporated into pup_init at boot? I don't know.

Please read this document again: http://distro.ibiblio.org/fatdog/web/fa ... ystem.html in particular look at the picture. The "first layer" (at the very bottom) says "/aufs/pup_init" and the arrow (read "it comes from") points to "initrd".

I'm impressed that you have a mental map of all the thousands of moving parts that make Linux work.

LOL nowhere close. That's why we have a team - so we can help to cover each other's blindspots.