Discussion, talk and tips
https://forum.puppylinux.com/
https://bkhome.org/news/202205/a-rethin ... cture.html
https://bkhome.org/news/202205/fix-dete ... -room.html
I wonder if you should really be considering this as an elephant or a mouse, a nusance you's like to get rid off, but a uncompfortable necessity.
And I don't in any way like the idea of a shutdown without saving work and information gained while online or while working with an application.
The zram is great
But all saves should be under user control. Not pre-programmed.
(sorry - i think i have the opposite viewpoint to what you expressed...)
With this design, after bootup you can be doing the usual operations, such as surfing the web with the web browser, download a video, play a video, etc., and there will only be writes to the working-partition when you save a file in '/files'...........
Then at shutdown, everything in zram gets merged into the '.session' folder, and that folder has the entire free space of the working-partition to use. For, say, a 16GB flash-stick, the working-partition will be about 15GB..............
another interesting outcome with this design; at shutdown you could choose to either save the session or not save. After a session of web-browsing, shutting down without saving might be a desirable choice. However, remember that anything you save into '/files' is already saved.
Barrys seems to have both concerns covered?
In the back of my mind was the thought, take the elephant out of the room. In other words, remove the inbuilt support for containers.
Yes, I know that container support has been an integral feature of EasyOS right from the start in 2017. However, other security strategies have emerged, such as running each application as its own user (and in the case of Firefox and seaMonkey they already have their own sandboxes, which are light-weight containers).
I boot Fatdog with pfix=nox (to cli) and then
X:1 &
DISPLAY=:1 cwm &
X:2 &
DISPLAY=:2 cwm &
... two (or more) separate X servers. Typically I load a browser into ctrl-alt-F4 and office in ctrl-alt-F5.
For my laptop that wifi net connects things are actually quicker for browsing if I (tiger)vnc into another box that is hard wired and run chrome/whatever from there. 100Mbs+ rather than 35Mbs or so download speeds does make surfing much snappier. A form of physical containment (separation), but even if the browser is run on the same box but within a separate X that considerably raises the security bar. There's also the likes of qemu for separation, or DIY chroots etc. alternatives that each user might use however they desire.
EasyOS containers are perhaps an ejectable elephant.
williwaw wrote: ↑Thu May 26, 2022 2:43 amWith this design, after bootup you can be doing the usual operations, such as surfing the web with the web browser, download a video, play a video, etc., and there will only be writes to the working-partition when you save a file in '/files'...........
Then at shutdown, everything in zram gets merged into the '.session' folder, and that folder has the entire free space of the working-partition to use. For, say, a 16GB flash-stick, the working-partition will be about 15GB..............
another interesting outcome with this design; at shutdown you could choose to either save the session or not save. After a session of web-browsing, shutting down without saving might be a desirable choice. However, remember that anything you save into '/files' is already saved.
Barrys seems to have both concerns covered?
kvm/qemu has qcow2, somewhat similar to LVM - which are similar to layering but where instead of changes to individual files the 'changes' record changes in blocks (filesystem).
Something like the following works for me. Where a initial image is created and has something installed to it (NetBSD install image iso run/installed to the vHDD in this case), and where you can then just subsequently boot that image in which case all changes are recorded (similar to a conventional full install), or you can boot using a snapshot like approach, where the main image is ro and changes are recorded separately - similar to Puppy frugal style.
In that code I've used zram, 1GB fixed size allocation, into which the main image (ro) sfs is mounted and where the backing file (like a save file) is created to store/record changes.
Code: Select all
#!/bin/sh
# This script assumes we've already created/installed a kvm/qemu system
# such as using something like the following (where I installed NetBSD) ...
#modprobe kvm-amd
#qemu-img create -f qcow2 default.qcow2 40G
#QEMU_AUDIO_DRV=alsa qemu-system-x86_64 -m 2048 \
# -cpu host \
# -vga std \
# -cdrom NetBSD-9.2-amd64.iso \
# -drive if=virtio,file=default.qcow2,format=qcow2 \
# -enable-kvm \
# -netdev user,id=mynet0,hostfwd=tcp:127.0.0.1:2222-:22 \
# -device virtio-net,netdev=mynet0 \
# -soundhw ac97 \
# -smp 2
# Once that has been created/installed, we can subsequently boot that using
# a backing file, i.e. the main system is read only, and changes are stored
# in a savefile, simlar to Puppy, but where we're store changes to disk
# blocks rather than files
################################################################################
# We use zram to hold both the main ro image and store the changes (backingfile)
# qcow2 requires that the main image and backinfile are both on the same fs
echo 1 > /sys/block/zram0/max_comp_streams # zram for just one stream
modprobe zram num_devices=1
# cat /sys/block/zram0/comp_algorithm # what compression alg's are available
echo lz4 > /sys/block/zram0/comp_algorithm
echo 1G > /sys/block/zram0/disksize # set the size
mkfs.ext2 /dev/zram0 # Create a filesystem within zram
mkdir /zram
mount /dev/zram0 /zram
mkdir /zram/m
mount default.qcow2.sfs /zram/m # Mount the base ro system sfs image
cd /zram
qemu-img create -f qcow2 -b m/default.qcow2 backingfile # save/changes file
modprobe kvm-amd # Prepare to kvm/qemu boot
qemu-system-x86_64 -m 2048 \
-no-fd-bootchk -monitor stdio \
-vga std -k en-gb -usbdevice tablet \
-drive if=virtio,file=backingfile,format=qcow2 \
-device virtio-net,netdev=mynet0 \
-netdev user,id=mynet0,hostfwd=tcp:127.0.0.1:2222-:22 \
-cpu host -enable-kvm \
-smp 2
sleep 2 # Housekeeping
sync
sleep 1
umount /zram/m
umount /zram
rmmod zram
# All changes will be lost as the 'backingfile' is lost when we close the zram
# We could however add additional functionality to preserve that backing file
# and reuse that at the next boot instead of creating a new backingfile each
# time as above
################################################################################
kvm/qemu can run as quickly as bare metal installations, sometimes even quicker. Other times kvm can be slower than qemu, typically when there's lots of I/O involved. kvm is subject to hardware supporting that, otherwise you're limited to just qemu alone. A feature however is that qemu can be set to emulate hardware that you don't actually have, such as 4 cores on a 2 core system ...etc. Could be coded to use a common denominator type choice, so that each user was in effect booting using the same (emulated) hardware. That's an aside however, the above is predominately a indication of how qemu can layer a main sfs and save file into ram, where the save file is disposed of or written to usb only on demand/at-shutdown, otherwise contained/run in zram. But with similar restriction of available ram space limitations.
Thoughts are EasyOS boots a qemu instance, where the main image is mounted into zram (which uses relatively little space), backing (changes) file also created in zram for first boot, or copied from usb into zram for subsequent boots, which being a sparce file starts off small and grows dynamically, so if few changes are made during a session then the backing file is small. At the end of the session, when 'save' is clicked/invoked, the backing file is written back to usb.
... adding to my prior post, being a single file the backing (save) file would tend to grow with time, until it perhaps was the same size or larger than the main ro image (sfs). I note however that backing files can be compressed using lz4 or gzip/whatever to further reduce the filesize, but that would still entail long/slow writes back to usb when a 'save' was run.
It may however be possible to slice up that big file into smaller units and perhaps layer those units using more conventional file based overlays. A a simple example a single volume of 100 blocks separated into 100 single units/blocks (separate files), which was then overlaid and changes recorded such that only the changed units (blocks) had to be written back to usb. A combination of both filesystem blocks based overlays (qemu backing file) and file based overlays (conventional puppy style changed files overlays).
Never tried that myself, I'm only aware that large files can be split and then rejoined again, or there may be some alternatives to that such as LVM where a single visible (virtual) filesystem/volume is formed from multiple physical volumes.
Concept being to mount the usb base main sfs into zram, then read the entire backing/save also into zram, but via joining/layering using file type layering, so when a save is run only the changed blocks/files need to be written back to usb. The next boot then has the main set of blocks, overlaid with the changed blocks - forming the updated virtual volume (backing file).
But that could all get out of hand, too complex, maybe a simple recreation of the main image/sfs and rm the backing file would be better/simpler choice. lz4 with its standard compression setting can compess even large files relatively quickly, perhaps 30 seconds or so.
I know there has been a bit of discussion whether or not the protections offered by containers are significant. However, even with that aside I believe containers really enhance Dunfell's usability.
It is getting better now that Barry has been gradually expanding the repository for Dunfell... and I have been getting better at compiling things on my own that are not available.
But there are still times when I need a software that I am having difficulty compiling. In that case I just spin up a Buster or Puppy container and Presto - good to go.
I think this alone gives containers a reason for being - whether or not the security benefits are significant.
However, if the trend is towards the new Easy Bookworm then this might not be pertinent.
As part of my Fatdog container startup, originally formed out of Barry's work, I change the root password, delete /etc/wpa_supplicant that otherwise contains wifi passwords, delete ~/.ssh ssh keys and also android keys, add a iptables entry that blocks access to our router admin IP ...etc. Root has very limited capabilities, can't access disks etc. I also like that close/reopen a container is very quick/easy and that automatically clear out all of those cookies ...etc. that I accepted more often without having actually read what I was accepting.
Yesterday I received an email commenting on containers. The person reported that on his Chromebook, Crostini runs Debian Buster in a container, and it takes 30 seconds to startup. It displays a "Starting container..." message and you have to wait, but once running he says it is OK.
In EasyOS there is a slight delay at startup, but it is only a few seconds.
Yes, containers are useful, and do define what Easy is all about. There is no downside to having them there, even if someone doesn't use them. So, they are staying.
I don't necessarily need to release Easy Bookworm as a separate release, could just have it as an SFS, that Easy Dunfell downloads via the "sfs" (formerly "sfsget") icon on the desktop. As Airdale has posted, that is a good way to access the bigger repository, with the advantage that the apps will be running in a restricted secure container.
delete /etc/wpa_supplicant that otherwise contains wifi passwords
If you have wpa_passphrase you can encrypt the password, like this:
Code: Select all
# wpa_passphrase easyosuser barkbark
network={
ssid="easyosuser"
#psk="barkbark"
psk=2692c8c88df9fc93358c72066f96cfa654981573c4943fe6249034015c113f3e
}
#
You can delete the line with the unencryted pass key, of course.
EasyOS uses NetworkManager and there is no /etc/wpa_supplicant. Instead, there is /etc/NetworkManager/system-connections/ with the wifi connection file, in my case 'HUAWEIY9 Prime 2019.nmconnection'
The psk is in plain-text; however, the file has permissions 600, so cannot be read by apps running non-root. Firefox for example.
That file is not inside a container, so no security problem inside containers.
BarryK wrote: ↑Thu Jun 02, 2022 12:43 amI don't necessarily need to release Easy Bookworm as a separate release, could just have it as an SFS, that Easy Dunfell downloads via the "sfs" (formerly "sfsget") icon on the desktop. As Airdale has posted, that is a good way to access the bigger repository, with the advantage that the apps will be running in a restricted secure container.
trying to use sfsget but perhaps a different version of kernel_5.15....src.specs is missing from the repo?
thx
williwaw,
Uploading right now. Thanks for reporting.
sfsget now downloads the recently uploaded file, but sfsget still fails on account of other stuff missing from the repo.
Is there a way to update sfsget such that it is not looking for something that may not exist in the repo?
I actually have easy_0.1_amd64 on hand, so perhaps there is a way to install the sfs manually? I have copied easy_0.1_amd64 to wkg/sfs/easyos/debian/bookworm, and hope to run the sfsget package installer without running sfsget from the beginning. I'm not sure of the name or location of the installer script, though.
williwaw,
I have been doing some development work on the online sfs files, there are going to be issues for awhile.
Anyway, I will sync the pyro sfs files, so that error should go away.
There is also a horse in the room:
thanks Barry. Bookworm in a container seems to run fine. I really should be using the desktop container feature more, especially for testing. Glad that elephant wasn't as big as once thought.
.
Barry,
I have been trying to build a vanillaDpup into an sfs to run as a desktop in a container. Not successful yet, and It may be the subject of another thread once you complete your latest dev work with sfsget.
Edit from post above:
I am curious if creating a desktop container for a sfs created locally with dir2sfs, would be more straightforward if done in the easy container app, similar to the method used to create a container of an app?
sorry for the previous confusing suggestion., it was late and I was trying to use a hammer as a screwdriver