Corepup - a minimalist modular system

[wanderer]

.

this is the link to the corepup isos

https://www.smokey01.com/wanderer

hi rockedge
this thread can be moved to the dogs section
next to dcoredog etc

hi all
since we have a nice new forum
i will try to start my corepup thread again
i will put it in projects (because it is a project)
but i guess it can be moved or deleted if necessary

i will keep everything of importance in the first post
so people dont have to wade through endless nonsense
to figure out what is going on

remember the idea behind this project
is to try to combine the advantages of puppy and tinycore
such as using the symlink system instead of the layered file system
and adding puppy packages to a tinycore base
a big advantage is that as noted it is a very minimalist modular system
so it is easy to understand and modify

because of my very limited level of expertise
this is a very rudimentary effort
but hopefully it will be of interest or amusement to someone
it is at least to me

more to come as i figure things out

happy new forum

wanderer

[redquine]

Hi wanderer,

Great to see you here! What you've done so far with Corepup looks really promising; it ran beautifully on my old eeePC 701 with 512MB RAM.

Please let me know if there's anything I can do to help.

[wiak]

Yes, good to see you here wanderer. It is always fun reading up on your corepup experiments and developments. Certainly a good alternative non-layered-filesystem base for lego-like building a tiny system around puppy or puppy-like utilities and apps.

[wanderer]

thanks guys

ill keep plugging away at corepup
and post on this thread

i hope everyone will at least make a thread and post links to their stuff on the forum
this is a great central location for everything

wanderer

[ocpaul20]

This is obviously a stupid question and no doubt I have overlooked the obvious.

Is there a link to Corepup or did I just miss it?

[wanderer]

thanks ocpaul20 for your interest

the isos are now on the first post

wanderer

[wanderer]

hi all

i figured out how to put fredx portable firefox in corepup/tinycore
used the libraries from a newer tinycore

now i will see what i can do with vlc

wanderer

[greengeek]

Hi wanderer, i tried to load a .pet into corepup 6 by doubleclicking it, but corepup did not seem to recognise what to do with the .pet suffix.

So i changed the name of the .pet to .pet.tgz and was able to expand it by doubleclicking - but then i could not manually copy the files into the filesystem (/usr, /lib etc) because emelfm said "permission denied". (I could copy things like icons with no problems - but not libs or executable files)

Do you have any tips how i can try a .pet in Corepup please?

cheers!

[wanderer]

hi greengeek

corepup/tinycore does not default to root

if you want to move things switch to root
by sudo (command)
or by opening a filemanager or terminal as root

you may also have to sudo chmod 777 -R a lot of stuff so that it is found or usable

corepup uses tcz which are the same as sfs files
so everything that you want to use should be made into an sfs file and then named file.tcz
however a lot of things are already sfs files just named differently
you can put a file.tcz into the optional folder in the tce folder
and it can be loaded by the apps manager like any other tcz

wanderer

[pup]

please help me , man , i am newbie here
how to install palemoon 28 on this corepup?
is manual from tiny core web still apply to this distribution?
if not , is there any manual pdf on this distribution?

[rockedge]

I think I just downloaded the palemoon tarball then extracted it to /opt and clicked on the palemoon binary and made a menu entry.

It has been awhile since I've worked with Corepup.....I think the TinyCore Linux documentation will help also.

[fredx181]

I think I just downloaded the palemoon tarball then extracted it to /opt and clicked on the palemoon binary and made a menu entry.

It has been awhile since I've worked with Corepup.....I think the TinyCore Linux documentation will help also.

I think in TinyCore (but anyone correct me if I'm wrong, please) you cannot just drop files into the filesystem and expect it to be saved at reboot (as it does work in Puppy and most other "live" OS's).
Everything is very strictly organized by installing tcz files (in fact packages), quote from wanderer:

corepup uses tcz which are the same as sfs files

Fred

[rockedge]

yes there was some thing about the persistence. It has been a while since I've booted up and worked with Corepup.

[wanderer]

hi all

first i would use tinycore 11 in the corepup repository on the first post of this thread
since its libraries support new stuff like fredx portable firefox (if you load them from the tinycore repository)

then when you boot put in tce=sda1 and home=sda1 as bootcodes (or whatever drive you want your persistent folders to be on)

there may be a palemoon in the tinycore repository
which is accessed by using the app function
this would probably be the easiest since it would load the needed libraries automatically

if not you can make an sfs file from the palemoon folder and rename it palemoon.tcz
and put it in the tce/options folder on your hard drive and it can be loaded like any other tcz by the app function

the tinycore documents apply since corepup is just a slightly modified version of tinycore

tell me what you are doing
and i will try to help you

wanderer

[Luke]

Hi wanderer

The link to the corepup isos lead to:

TinyCore-11.0.iso 2020-08-29 14:42 19M
bionicpup32-8.0+22-uefi.iso 2020-10-06 12:20 273M
corepup-6.iso 2020-07-16 00:37 84M
corepup-7.iso 2020-07-16 00:38 15M
corepup-8.iso 2020-07-16 00:39 9.7M

Can you explain what is what ? what should i download ?

[wiak]

I think in TinyCore (but anyone correct me if I'm wrong, please) you cannot just drop files into the filesystem and expect it to be saved at reboot (as it does work in Puppy and most other "live" OS's).

Actually, it is pretty easy to arrange so that you can thereafter "just drop files into the filesystem and expect it to be saved". TinyCoreLinux provides the detailed documentation, but the general idea is:

http://tinycorelinux.net/concepts.html

The text file /opt/.filetool.lst lists files and directories to be backed up at power down and restored at reboot. The list may be changed manually (using vi, nano, etc) or via the scripts in the Tools menu

One of the great things about TinyCoreLinux is that its 'save file' (mydata.tcz) is usually pretty small. That's because apps/packages from tinycore repository that you add to the system don't need to be 'saved' in any such save file (or save folder or save partition). Rather they just get stored (as squashfs tcz files) in tce directory and auto-arranged to be pristine loaded into the running filesystem on next boot (configured in onboot.lst file I think). That's a very 'clean' and efficient way of organising persistence compared to live aufs/overlayfs approach where everything new added gets saved into persistence file or folder - which is a lot and basically all mixed together (whereas tinycorelinux keeps individual packages all in their own separate squashfs package). No aufs/overlayfs is in fact used - rather, as far as I remember it, the squashfs tcz files get loop mounted and then simply symlinked into the filesystem for use - it is an interesting, KISS neat approach.

Lots of info also in Tiny Core provided 'Book': http://tinycorelinux.net/corebook.pdf There are some additonal tricks to achieve persistence for some directories, such as /home (see pages 9 and 10 of that book for details)

[rockedge]

See section 7.2 in Corebook.pdf concerning file persistence in /opt. I stumbled on how to do it with palemoon at some point. I feel like I should boot my last experimental Corepup and check on it.

I started to install a LAMP and Zoneminder installation on Corepup to compare on how small I could go with the OS and still offer a good platform for a camera security system. I did not complete the quest because of compilation of dependency libs for ZM in Corepup became complicated and PERL was difficult to get going. Although that was sometime ago and might be worth another look.

[wanderer]

hi luke

download tinycore 11 first
and start with that

play with it a little
and go to the tinycore website and read the corebook

there are 2 basic variants
tinycore and dcore
both on the tinycore website
tinycore divides the dependencies up and loads only what is needed
it uses the tinycore repository
dcore loads the files from the debian stretch repository
and bundles them into one file both app and all its dependencies

there is an infinite number of variations possible
on how to build things
so you end up with only what you want
you can build it up and take it apart like leggos

you can also add puppy stuff
since sfs files from puppy are the same as tcz files in tinycore and dcore
you can add a pet by unsquashing it with uextract
resquashing it into an sfs file
and then renaming it a tcz
and loading it normally

this seems like a lot of stuff
but once you start to play with things
you will see how simple everything is

the point is that you are starting with a tiny core and building up
rather than trying to tear something apart and building down

post if you have any questions or comments
have fun

wanderer

[user1111]

No aufs/overlayfs is in fact used - rather, as far as I remember it, the squashfs tcz files get loop mounted and then simply symlinked into the filesystem for use - it is an interesting, KISS neat approach.

That's a very neat way of doing things. In my own Pugdog (cut down version of Bulldog (which is a inner layer of Fatdog)), I just use compressed tarball files (.tar.gz or .tgz) and extract them as desired/required. For instance the main boot is into just a busybox userland system (cli within a framebuffer) of around 5MB combined vmlinuz and initrd size (xz compressed); For wifi net connect and OpenSSH I extract a net.tgz into that - around 4MB filesize); Adding mc.tgz (as I like mc as a file manager and text editor) is also around a 4MB .tgz. With that set I can then do much of what I typically do, such as vnc into a full desktop/gui system for browsing/word processing ..etc. sshfs mount remote drives, and ssh into a ssh server for irc, mail ...etc. I also have a kvm/qemu tarball, that's around 15MB in size, which can be used to boot any iso/image files (or even other vmlinuz/initrd's), that again I use fbvnc to 'view' (access) - but at that level its somewhat pointless, as I might as well directly (natively) boot the iso, except if I want to have multiple OS's running concurrently (but again could just as well natively boot and then kvm/qemu boot other OS's within that).

So for me the building blocks pretty much stop after the 5MB vmlinuz/initrd + 4MB network/wifi/OpenSSH + 4MB mc. Thereafter I could have Windows, Linux, BSD ... whatever systems all concurrently available. 5.9 kernel has seen initial memory usage rise from 21MB to double that, around 42MB. Presently as I write this around 62MB is used.

A nice feature as I see it is that all of the usual background things that the likes of Google otherwise have running in the background on your system, aren't running locally. Have you ever noticed how even if you block port 80/443 http/https traffic in iptables, you'll still be able to load the google search screen and searching for something will still return results i.e. what google does have running on a system bypasses much of the system. I shift that into a server system that boots from a actual iso (DVD) and has the browser running in a kvm/qemu i.e. pretty much contained and that is rebooted (cleaned) relatively frequently (rebooted several times/day).

[wanderer]

with corepup/tinycore/dcore

the symlink system is one of the great strengths of this system

it is much more robust and flexible than the union file system
since it has no overhead

another great strength is having the ramdrive be the root filesystem
and having extensions symlinked into it
so at minimum only the ramdrive has to be loaded
to provide a working sytem
and then only the extensions one wants at that time need to be loaded

another great advantage is having the ramdrive
be very small and relatively unchanging
and contain all the base scripts and shell
so with the ramdrive
you have the complete base of the system in a small folder

another nice thing about this system is that it can use puppy sfs files
as compressed extensions
and symlink uncompressed application folders from the home folder
so it can use apps from puppy and other distros
in a straightforward way

the system is loaded in a frugal manner
and persistence is provided by the tce and home folders on the hardrive

etc

wanderer

[wiak]

with corepup/tinycore/dcore

the symlink system is one of the great strengths of this system

it is much more robust and flexible than the union file system
since it has no overhead

I've never experimented with using symlinks in that manner (other than in my also having many a actual tc installation) but I can't see any reason why it would not be possible to create an 'actual' Puppy using its core files (in addition to the corepup tc implementations), and various sfs drives (adrv and so on). Of course you'd have to make configuration types to emulate the various familiar Pupmodes and so on. Would be an interesting Pup to play with and compare.

Certainly, it is hard to imagine any distro more modular than tinycorelinux itself, but it does require a fair amount of work putting all its many parts together (since pretty much every single app/part of it is produced as a tiny sfs file) hence I'd like to see something similar (if possible) tried with larger Puppy sfs components.

[user1111]

the symlink system is one of the great strengths of this system

it is much more robust and flexible than the union file system
since it has no low overhead

Seems to me that it loop mounts each sfs under /tmp (folder name indicative of the sfs filename within /tmp/tcloop sub folder) and then does a sym link based cp (cp -ais) of the content into the main system.

Can use many loop devices (by default typically around 7 are created), so presumably creates additional loop devices as needed using something like mknod -m660 /dev/loop8 b 7 8

So no actual data copied, just sets up all of the 'pointers'

Certainly, it is hard to imagine any distro more modular than tinycorelinux itself, but it does require a fair amount of work putting all its many parts together (since pretty much every single app/part of it is produced as a tiny sfs file) hence I'd like to see something similar (if possible) tried with larger Puppy sfs components.

Approaching it from the other direction, adding a puppy.sfs as a tinycore sfs (another loop device and cp -ais the content) ... would add that on top of what is already there within tinycore. In Fatdog I use HDD based multi-session save, which creates a new/additional sfs each time you click 'save', so they also could be 'loaded'.

[wanderer]

here is an interesting thing you can do with symlinks

let us say you have a folder with an app in it named app1 (for example in home)
and you symlink it to root /
it is now mounted
if you change the name of the folder to another name say app1x
the symlinks cannot find it and it is now unmounted
if you change the name back again to app1
the symlinks can now find it again
and it is now mounted again
and on and on
by only changing the name of the folder

so you can do some interesting things
with ( low ) overhead

wanderer

[user1111]

In my Pugdog I have a minimal vmlinuz and initrd, basically just busybox and absolute minimum set of files (in /etc ....etc), enough to boot to cli/framebuffer and where busybox provides the main init (last line of my own init that is around 30 lines in total hands pid 0 over to busybox init, that invokes inittab ...etc.). Up to now I had been using tarball files to add to that, so a net.tar.gz for instance that I extract to / to add in /network/wifi/OpenSSH additional functionality; And again another for mc.tar.gz (for file manager and ncurses style text editor).

Recompiling the kernel to add in squashfs filesystem support and creating sfs's of the net and mc tarball files contents (net.sfs and mc.sfs), I can now boot and mount those
mkdir /mnt/net
mkdir /mnt/mc
mount net.sfs /mnt/net
mount mc.sfs /mnt/mc

... and then once booted I can 'load' those using symlink based copying ...

cd /
cp -ars /mnt/net/* .
cp -ars /mnt/mc/* .

... and it all works, I can wifi net connect, run mc ...etc.

I build my kernel directly from the source provided by kernel.org, so 'bulk standard' (no patching for aufs/whatever). What I've done is drop both net.sfs and mc.sfs into initrd, so they're also being 'copied to ram' by default. And if I built that initrd into the kernel, then that could all boot from the single vmlinuz file.

Thanks wanderer

[wanderer]

it is very easy to build and maintain multiple versions of the system
since 2 folders on the drive hold the extension files
the tce folder for the compressed extension files
and the home folder for the uncompressed extension files

all one has to do is download/gather the desired extension files
and place them in the appropriate folders

so for version 1 put what you want in tce1 and home1
for version 2 put what you want in tce2 and home2
for version 3 put what you want in tce3 and home3

the core will load the folders named tce and home
when it boots
so if you rename the folders without the numbers
you can load whatever version you want
and still maintain all the other versions
for use when you want them
in a very modular fashion

[wanderer]

hi rufwoof

i am trying to follow what you are doing
now digesting it
very interesting

what wiak is doing with weedog
is also very interesting

but a little complex for me

wanderer

[user1111]

hi rufwoof i am trying to follow what you are doing

Basically just compiling my own bespoke kernel with all modules/firmware built in. So very machine specific, but small.
Load the devx sfs for whatever system you use, download the kernel source tarball from kernel.org, extract that and then run make localyesconfig ... to create a .config file for your particular hardware. If prior to doing that you 'use' as much of your system as possible, camera, attach/mount usb's ...etc. then when make localyesconfig runs it should pick up on all of the modules that you used.

Once make localyesconfig finishes, run the actual build. Adding the number of cores you have as a -jN parameter helps speed up compile time. On my four core system for instance I run
make -j4

When that finishes you'll have a sub folder arch/x86/boot/bzImage ... that you can just rename to vmlinuz i.e. the kernel.

To further refine that, if you run make menuconfig in a terminal then you can wade through the settings to perhaps remove or add additional elements and then after saving the changes run make -j4 again to recompile. Such repeated recompiles usually run through pretty quickly, maybe a couple of minutes. After numerous repeats of that cycle I've got my vmlinuz down to around 5MB and where there's no need for additional modules/firmware (which otherwise can make up a considerable large part of a Puppy).

If/when you want to update the kernel to the next dot release, then download/extract that version, copy in your .config file and run make oldconfig ... to import that config into the new kernel source, and run make -j4 ... to produce a new bzImage (a.k.a. vmlinuz).

On top of that you of course need a initrd (and busybox within that). i.e. the 'userland'.

viewtopic.php?p=6922#p6922

viewtopic.php?f=60&t=964

I have a server i.e. a old desktop/tower that is connected to the TV, so I can boot the above, in which I also add framebuffer vnc, so I can boot and vnc into that desktop system and see the same screen (gui desktop) on both the TV and my laptop, and play videos/whatever as though the laptop was a advanced remote-control type 'terminal' device. If I open up the vnc port on my router, I can vnc into the desktop from anywhere. I find however that tunnelling vnc through a ssh tunnel where compression is set on works much better/quicker. So for instance if vnc is running on port 5900 and I open up port fowarding in the router for that, and my external IP address is 123.123.123.123, then on the client (laptop/remote system)

ssh -f -t -C -N -L 5900:localhost:5900 123.123.123.123

means I can just run vnc localhost on the laptop/client to have that route through the ssh tunnel to the desktop (and where -C = ssh compression is turned on).

When you can remote control your desktop/home system from anywhere, you can obviously start adding functionality to that, such as cameras attached to the home desktop/server ... or whatever. It also means that you can browse/whatever using your home system and internet connection remotely through a ssh tunnel which is more secure as pre-defined keys are involved (ssh), so the cafe/whatever wifi hot spot you may be using can't man-in-middle attack you. A bit like a VPN, but where you're the VPN service provider rather than outsourcing that to others (where trust/payments are involved).

I've been meaning to get into pi's for some time now i.e. replace the desktop server with a lower (electrical) power/smaller device, or potentially set up multiple pi servers. At least that is the intent.

A neat 'trick' is that the laptop isn't running X, is very lightweight (around 12MB with wifi/ssh/mc ..etc.) total size (vmlinuz and initrd), but has capacity to run full gui desktop (through vnc). Similarly the server doesn't have to be running X either, it can be headless (cli only). If you set up kvm/qemu then it can boot any iso (or image file, or vmlinuz/initrd) and be set to serve vnc (i.e. the gui desktop that your laptops sees) simply by adding -vnc :1 as a qemu parameter (that then is accessible by vnc'ing into port 5901). kvm/qemu runs at as good as native speeds, as though you had installed and booted it directly, but where qemu can boot any (or multiples of) iso files, so your laptop might vnc into one qemu running perhaps Windows, vnc into another running Puppy .. or whatever, and have access to/use those ... concurrently. And if those servers are left running, then you can disconnect the vnc session, and re-connect again later to drop straight into a gui desktop as it was last left ... near instantly (or where it had move onto, such as if rendering a large video, or doing a large compile/whatever). Yet another nice element is that you can set the vnc to accept multiple connections, so several can vnc into the exact same desktop. So if you're at home and your son/daughter vnc into the home server, you can see that on the TV (server) whilst typing text/whatever using your laptop whilst sitting on the couch. Or even have them play a sound through the speakers attached to the server - to attract your attention to turn the TV from normal broadcast channels to the VGA/whatever channel/input ('dad are you there?' ... type 'ring'). And they don't have to be running a Puppy to do that, most devices support vnc client (phone, Windows ..etc.).

Yes you can do similar in other ways, but a large part of Puppy/Dogs is the DIY fun element.

[user1111]

here is an interesting thing you can do with symlinks

let us say you have a folder with an app in it named app1 (for example in home)
and you symlink it to root /
it is now mounted
if you change the name of the folder to another name say app1x
the symlinks cannot find it and it is now unmounted
if you change the name back again to app1
the symlinks can now find it again
and it is now mounted again
and on and on
by only changing the name of the folder

so you can do some interesting things
with ( low ) overhead

wanderer

If you desire that 'load/unload' type functionality then the sym link approach is a good choice. Mount a sfs, cp as symlinks its content to / ... will however consume up to around 1.5 times the space/ram - assuming all running in ram and every file is at some point accessed (and assuming sfs compression results in 50% reduction). i.e. the sfs size and the extracted size (decompressed into cache as/when a file is read the first time). With tarball's you extract the entire content of the tarball to / ... but where that can't be undone (unloaded) during that session, at least not easily.

Seems to me that for space efficiency the tarball extraction is better IF most/all files within that will be accessed/used at some point during the session ... typically small units. If only a smaller subset of the files are being used then the sfs/sym-link approach is likely more space efficient ... typically large units, such as a full Puppy sfs. Overall with the added optionality to be able to 'unload' (such as via a simple folder rename) generally gives the sfs + sym links the edge IMO. Accordingly I've modified my Pugdog to use that approach, I store the net.sfs and mc.sfs in high xz compressed form within the initramfs (initrd) and just mount/cp -ars ... their content to / in order to 'load' them. Dropping that initrd into vmlinuz results in a 10.5MB total (xz compressed) vmlinuz size. Pretty good considering its a full busybox userland with all the modules/firmware required for my laptop built-in, supports wifi net connecting, has full OpenSSH (sshfs, sftp, scp, ssh), has mc for its file manager and text editor, and includes fbvnc ... that can be used to vnc into whatever (full gui desktops).

[wanderer]

hi rufwoof

corepup/tinycore/dcore
consists of 4 components

1 bootloader
2 kernel
3 initrd
4 compressed extensions

the extensions are loaded using symlinks
and can be many small individual pieces as in tinycore
or large bundles of dependencies supporting an app as in dcore

corepup also uses a simple system
that symlinks an uncompressed extension app folder
(usually located in home) to root

this keeps the initrd small and unchanging
the only changes are in which extensions are being loaded at that time
there is no layered filesystem ever used

[wanderer]

can you compare your system to this
it seems as i am reading your posts
that you variably compile/add many components to the kernel and initrd
rather than keeping them separate as extensions

also did you say your system was taken from pieces of fatdog

does it use puppy sfs files or pets

wanderer