using qemu to boot a firstribit or some other FR/KL creation

Converts most live isos into FR/KL distros


Moderator: Forum moderators

Post Reply
User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

PROJECT CLOSED...

EDIT EDIT EDIT: NOT ABANDONED AFTERALL. SEE: viewtopic.php?p=126276#p126276

Sometimes it is better to abandon an undertaking when results have proven to have major issues. That is the case with my tests of the method I introduced in this thread. I wrote the automating script and it worked for both FR/KL distros and Puppy Linux distro... BUT... In early tests I noticed a corrupted file... so I did an fsck of the underlying filesystem and hoped nothing to do with my experiment with qemu direct booting a frugal install distro... However, same issue occurred on testing boot of Puppy Linux frugal into qemu and I was forced to again fsck the underlying filesystem. Whilst the procedure definitely worked the host view of the filesystem clearly got messed up by qemu view of the same filesystem... Why that all resulted in corruption of the underlying system I have no idea, but, certainly for now, I will not be advising this method of booting into qemu and will not publish my script that makes it all very automatic and easy - last thing I want is some unsuspecting person to damage their system (despite already giving big DISCLAIMER in original post because I'd read of some possible issue without making much sense of the WHY)...

By all means experiment per the first post of this thread AT YOUR OWN RISK, but my experience suggests you will run into corrupt filesystem issues so BE WARNED. I don't myself intend taking this further for now, for these explained reasons. Pity really, but there it is... Sometime it is better to back down and abandon and so be it.
NOTE that I've left this post since others may wish to experiment and decide for themselves. But be warned, definitely caused corruption on my system (though fsck seems to have fixed everything back to okay for me).

ORIGINAL POST WAS:

I've tried to use qemu to boot a FR/KL distro directly from hard disk partition before but without success...

However, though I don't know the implications (danger of doing this), tonight I had success; at least with firstribit FR/KL_Fedora_LXQt I had created as a frugal install. I haven't tried, but expect same would work to directly qemu boot frugal install Puppy Linux using -append of Puppy boot arguments such as psubdir, and similarly with other frugal installed distros.

I did this in full install of Linux Mint, but imagine it would work with qemu in any distro (such as KLV-Airedale). I'm sure the following can be improved by qemu experts; I am really not proficient with it, but, as I say, I've tried options -kernel and -init and so on with qemu before, but got nowhere... till now. NOTE WELL that I ran the command from a terminal opened at the FR/KL_Fedora frugal install directory. The following worked as the attached image shows:

Code: Select all

qemu-system-x86_64 -enable-kvm -m 2G -vga cirrus -smp 2 -kernel vmlinuz -append "w_bootfrom=UUID=d355bb5d-de78-4189-9afd-3907fab409a3=/KL_Fedora w_changes=RAM0" -initrd initrd.gz -drive file=/dev/sda

Note that for the moment I'm using w_changes=RAM0 mode (i.e. no persistence); I have to experiment further to see if w_changes=RAM2 mode will work...

EDIT: well... wd_changes=RAM2 mode also 'seemed' to work (just used touch to create a few new files and on reboot they 'seemed' to be there, but some weird messages so for now I'd just recommend trying this with RAM0 mode, and even then at your own risk...:
EDIT2: Actually, it MAY be fine. I just realised I was checking the files from host system in below weirdness, and since under qemu control really the host probably hasn't a clue what's going on... (fine when checked on the qemu running FR/KL_Fedora system) I suspect RAM2 mode is also fine therefore, but don't blame me if not...
EDIT3: Per project abandoned note at top, I've decided this method is UNSAFE per the corrupt underlying filesystem I experienced during tests... tra la la - pity...

Code: Select all

# ls -al
ls: cannot access '.xsession-errors': Structure needs cleaning
ls: cannot access 'jnk11111111111': Bad message
ls: cannot access 'anyrubbish': Structure needs cleaning

Alternatively, instead of argument file=/dev/sda you can use: -hda /dev/sda

I've similarly quickly tried the same to boot normal KLV-Airedale frugal install using qemu, but for some reason I don't know yet, it didn't work.

I have also discovered that some firstribit creations won't boot like this in qemu because their final initrd.gz is currently too large... That could be fixed by trimming the initrd.gz using Tomas M code from Slax though; I will consider this matter further because I find this way of booting very useful compared to having to make frugal install onto a disk image or an iso.

Attachments
qemu_FR_Fedora_LXQt_from_harddisk.jpg
qemu_FR_Fedora_LXQt_from_harddisk.jpg (102.5 KiB) Viewed 1393 times

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

Clarity
Posts: 3647
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1530 times
Been thanked: 489 times

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by Clarity »

I offer this as a comment; nothing more.

A VM is an "independent" PC. The host is an independent PC. QEMU is merely a tool to create a PC so that an OS can run. The options you've provided it are merely a mere of providing INIT same as one would in the OS Menu's stanza. And the Host provides connectives services to facilitate what the Virtual Machine (aka virtual-PC) sees for LAN, video, sound, disc, disks, and other items as its real peripherals in its environment.

Glad to see this effort of describing a VM, you've presented, that works with a FirstRibIT outcome.

Thanks

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Yes, booting an OS directly via its kernel and initrd is briefly mentioned in the qemu documentation, which is why I tried it now and then, but previously without success. However I remained interested in the possibility since knew it would be more useful to me than qemu iso booting, but never successfully got it to work with a frugal install configuration and have never noticed anyone showing an example on Puppy Linux forum.

I am also now able to write a simple script (based on wd_grubconfig code) that can automatically boot up a qemu instance when run from its frugal install directory. Like wd_grubconfig itself, the new utility can be included by default with the distribution download.

In terms of safety, I do feel there is some risk letting qemu vm have full access to whole hard drive, which is something I have often seen and done before, but previously I didn't actually boot via qemu using vmlinux and initrd directly. My recommendation would be to use this technique on usb flash stick frugal installs if worried about safety, where and when user doesn't mind the risk of corruption to the overall drive partition. Such corruption is likely only by human action anyway, and not per se any fault by qemu... (for example, in the past..., I've daftly set up a bind mount and then deleted whole new directory forgetting it was bound to a critical system directory...)

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 6345
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2533 times
Been thanked: 2509 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by rockedge »

@wiak look into AQEMU GUI front end. There is a section to configure for direct booting of kernels and initrd.gz. Might make it easier to get going.

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

rockedge wrote: Tue Jul 16, 2024 2:36 am

@wiak look into AQEMU GUI front end. There is a section to configure for direct booting of kernels and initrd.gz. Might make it easier to get going.

I don't myself use that, but could be useful for others if easy to do from there. I'm having a look now; where about in there is the section you decribe? I haven't found it yet in my quick look.

EDIT: Okay, I see the tab section. I'll play with that to see if it is useful overall for frugal boot case.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 6345
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2533 times
Been thanked: 2509 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by rockedge »

I am still working on getting the desktop mainframe up and running with Hercules and MVS 3.8j but now that you mention it that will be the next attempted project.

Boot a QEMU VM with a Frugal installation! Cool.......

I am close to full success on booting MVS, I am through system generation steps :ugeek: :thumbup:

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Tried but got nowhere getting same to work via aqemu. Maybe I just don't understand what info it needs but even after giving paths to vmlinuz and so on it complained couldn't find the boot device.

Anyway, a little wd_grubconfig type script using what I know will work very conveniently in the end I'm sure - can always add further qemu options to it later (or even as a wee plugin).

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

One thing I find difficult to understand: when a build script creates a properly configured frugal installation out of the box, why would anyone want to pack all that up again into an inherently hard to read and impossible to write to single image? Yes, I know, if you go to the trouble of making a special Ventoy boot device, with some special care and attention by a distro developer, the iso or img can be booted, and if 'lucky' (meaning special help by extra dev work) even save persistence might work. But to ever say this arrangement is anything like as flexible or full-featured as extracted frugal install form is simply nonsense, sorry.

So could a firstrib or firstribit converted distro be turned back into an iso or img. Well yes it could, but a painful extra effort and involving also build compression time. But why bother when already have best use case in form of a frugal install, which even Ventoy disk can boot?

Firstribit in particular, required iso download by the user, and they must run script to convert into properly configured frugal install. The result is not intended for further redistribution; no iso or img form is thus required at all. If someone is unwilling to learn how to boot already made frugal then not my fault they are losing out. There are at least as many steps involved in making a somewhat inferior Ventoy boot mechanism ( though can be used to boot an extracted frugal install anyway). There are many steps involved in making an iso, or bootable image, and no, sorry, life is to short. Should use best boot method when provided by default and even properly configured - that is an extracted frugal install.

Of course if any person insists on making an iso or img, which is only for their own use anyway, then please go ahead. I dont even see the point. In fact, personally I hardly know anyone who hasnt taken the simple time required to master booting from extracted frugal installs. Ive seen plenty of issues, and had them to, with people trying yo get save persistence working after booting from an iso or other single file image format. Really an unnecessary, painful, dev time occupying tricky waste of time overall really. Most people don't use boot from iso except for quick tests of ready made live isos. Even Microsoft and Apple don't rely on booting from iso - far from it, and if it was better to do so I am quite certain they would have gone that road decades ago. Improving extracted frugal install booting is where that time we use in that booting area of dev work should primarily go, and not only in my opinion.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

What we do need is better mechanisms here to upgrade kernel and some other major components when required.

Yes, it is easy swapping over huge kernels, but that is a special construction that needs accordingly special compilation. The problem otherwise is that the initrd needs to contain modules that match kernel being used. On kernel upgrade, the initrd then needs rebuilt, but that can be done, just not inside an iso or other single file image without extraction and all the effort involved in putting it back together again thereafter! ;-) The Humpty Dumpty story... I doubt anyone wants to rebuild Humpty when it isn't necessary to do so anyway.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

So I've written the script or at least first attempt: firstrib2qemu.sh

which boots FR/KL distros from their normal frugal install directory no matter what disk or partition they are on.

Using it is a bit like using wd_grubconfig.sh: you need to simply copy the firstrib2qemu.sh script into the FR/KL frugal install directory, make it executable and run it. There is an optional firstrib2qemu.cfg configuration file that allows you to optionally modify some qemu parameters and also set the w_changes RAM mode required.

Well, far from every FR/KL distro is booting into qemu with it, but not to cry, some are... and may be a qemu parameter or two that simply need changed. Even with what is working it is very fun to use, and potentially very useful. For example the firstribit distros of Fedora LXQt and 4MLinux (nice one that is so similar to Puppy) and an old KLV that was built from a plugin are booting from their frugal installs straight into qemu on my Linux Mint XFCE full install machine. EDIT: KLV_Sway working too. Can't get FR_tinycorelinux past cmdline though - pity cos want networking via qemu.

Despite lots of other FR/KL distros not working, I will release these in case someone can find the likes of alternative qemu arguments that make others work. Too late at night to release at the moment, but I'll do a few further tests and release within next 24h so you can all have some fun with this (especially if you enjoy tinkering with qemu). Once I discover any other qemu arguments that improve 'anything' (sound for example) then I'll slot these extra bits into the script and also for use in its configuration file.

Note that all the UUID stuff is worked out automatically and exact format auto-provided to qemu, so in those cases where qemu successfully boots it, no user effort or input is needed... other than just run the script...

I could modify this to work with other frugal installed distros such as Puppy or DebianDog or whatever of course... If someone can be bothered writing a yad frontend they could of course; I prefer simple script... ;-)

Cheers for now...

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

wiak wrote: Tue Jul 16, 2024 5:36 pm

Despite lots of other FR/KL distros not working, I will release these in case someone can find the likes of alternative qemu arguments that make others work. Too late at night to release at the moment, but I'll do a few further tests and release within next 24h so you can all have some fun with this

Take two... there will be a delay. Script is working fine but to save me time longterm I'm revamping it slightly so will work with a couple of other forum distros too, via a plugin since some different code is needed to automate the boot arguments appropriately - hence will be another day or two prior to release depending how much time I can further give to this right now. I always find it is best to slow down prior to actual release of something new because it is often a nuisance when major changes are decided upon that effect the overall configuration and commandline arguments thereafter, and also it is better not to hurry a release, since why? I suspect it is not as if thousands, or even hundreds, are waiting... ;-) Maybe better to invest our time in making Video shorts for instagram that attract millions of views and bring in lots of dough for stupid content rather than providing Linux utilities, which may only be ever used by a handful of people? I suppose it is fine if we really want to use it ourself and then sharing makes sense since, why not...?

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

PROJECT CLOSED...

Sometimes it is better to abandon an undertaking when results have proven to have major issues. That is the case with my tests of the method I introduced in this thread. I wrote the automating script and it worked for both FR/KL distros and Puppy Linux distro... BUT... In early tests I noticed a corrupted file... so I did an fsck of the underlying filesystem and hoped nothing to do with my experiment with qemu direct booting a frugal install distro... However, same issue occurred on testing boot of Puppy Linux frugal into qemu and I was forced to again fsck the underlying filesystem. Whilst the procedure definitely worked the host view of the filesystem clearly got messed up by qemu view of the same filesystem... Why that all resulted in corruption of the underlying system I have no idea, but, certainly for now, I will not be advising this method of booting into qemu and will not publish my script that makes it all very automatic and easy - last thing I want is some unsuspecting person to damage their system (despite already giving big DISCLAIMER in original post because I'd read of some possible issue without making much sense of the WHY)...

By all means experiment per the first post of this thread AT YOUR OWN RISK, but my experience suggests you will run into corrupt filesystem issues so BE WARNED. I don't myself intend taking this further for now, for these explained reasons. Pity really, but there it is... Sometime it is better to back down and abandon and so be it.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Glutton for punishment; one last thought strikes my mind. Perhaps there is a possibility of no issue if in RAM0 mode. That would still be useful... I will assume not worth the risk, but will report back if RAM0 mode doesn't give me issues. I had been experimenting with RAM2 mode when I came across the filesystem corruption. Since a basically different machine accessing the same hard drive, but storing its memory view of that in its own totally different memory, it strikes me as not surprising that writes to disk from either 'computer' (the host or the guest) may turn out invisible to the other and on shutdown of the guest could well result in 'problems'... but can't see RAM0 mode (or not saving to flash in a Puppy using mode 13) having that issue, but I promise nothing... (Puppy has issue that it starts up in Pupmode 5 though unless it already has a save folder/file, so I'll try with firstrib RAM0 mode instead).

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

So, I haven't managed to exactly 'corrupt' my disk when using frugal2qemu with w_changes=RAM0 (no persistence) arrangement. That might sound logical. But it isn't quite as simple as that - if I write to a directory 'outside' of the save folder (such as frugal install actual partition) the file seems to save from the guest, but isn't first visible to the host... even when the qemu VM is shutdown: still what was written isn't seen... But it appears on reboot!!!, which is pretty weird to me. However, I tried fsck on the relevant partition after such writes and each time came up clean... so maybe it is safe, but then again, maybe it isn't. One thing I can say though is that using same with w_changes=RAM2 mode definitely didn't seem advisable and corruption was occurring when I tried that.

Which leaves me with a dilemma. I've closed the 'Project' as too dangerous, but maybe when used with FirstRib running under w_changes=RAM0 (no persistence mode) is actually safe and fine. And it certainly would be 'useful' to me despite no persistence. Actually I can't say 'no persistence allowed' since off the top of my head it should be possible to allow save on demand persistence to a different partition (not the one booted from). In fact, the same crops up with my very special KLfull2frugal arrangement, where an underlying fully installed distro is used as the base rootfilesystem in an overlyfs layered frugal install arrangement; that seems to work perfectly (and is most useful arrangement of all for my own needs), but the savefile or savefolder MUST be put in a different partition than the underlying (fully installed) main root filesystem. Alas I'm not quite so sure about frugal2qemu in terms of non-corruption if only running in RAM0 mode, but I 'suspect' it may be fine, and will be used by me, so should I really abandon the project from further publication or not??? or will DISCLAIMER - MAY VERY LIKELY BREAK YOUR SYSTEM be enough. i.e. passing on the risk to the user. Maybe a bit like giving someone the rm command with the caveat that if they use it at all they may delete their own system and especially if they use rm -rf on a critical part of the filesystem...???!

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

So the more I think about it, Ive almost decided to publish scripted method anyway but for FR RAM0 use only and probably use read only fs for qemu with temporary -snapshot files too... Nevertheless, if I do I want no blaiming me for any resulting catastrophe... you have been warned. All use at own risk........

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 6345
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2533 times
Been thanked: 2509 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by rockedge »

@wiak Now that I have the IBM mainframe virtual machine up and running and out of the way for now, I am totally interested in experimenting with the script!

I use QEMU to fire up frugally installed distro's but those are on a Virtual hard drive created for use with the QEMU virtual machine.

I will also experiment with an older KLV using the JWM-Rox desktop environment arrangement to test out booting a frugal install with a QEMU VM.

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

rockedge wrote: Wed Jul 17, 2024 7:14 pm

I use QEMU to fire up frugally installed distro's but those are on a Virtual hard drive created for use with the QEMU virtual machine.

Yes, that is fine of course since is effectively the VMs own drive. But what I'm using is exact same hard drive filesystem being used by two effectively separare computers at same time. Since they both keep own in memory tables and so on of the shared disk, changes by one computer are invisible to the other so corruption happens if writing involved. But, yes, for read only use or never actually writing back from temp files or RAM all is okay so I'll release scrpt soon with that strong disclaimer.

EDIT: I certainly find this script pretty useful even if just to look at new distros because I tend to always install as frugal and throw the iso away... Being able to boot from my many frugal installs (including of big firstribit distros), even as read only, is very useful to me, but read-only has to mean read-only... can't write anywhere on the underlying same disk as host without risking major corruption. A separate partition or media device not in use by the host at all would be fine though.
.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 6345
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2533 times
Been thanked: 2509 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by rockedge »

@wiak I tried several methods to boot a frugal HDD installation using QEMU and not having much success yet. I did manage to boot using AQEMU to set it up, an ISO using the kernels from other installs or any vmlinuz and initrd.gz. Tried out some pretty wild combinations.

Some strangeness definitely in operating these arrangements. Will keep on working on perhaps finding the right combination of QEMU arguments to boot a frugal HDD (or USB) frugal installation using a QEMU virtual machine.

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Yes, using -snapshot option to Qemu seems to prevent all writes back to underlying drive, which is less useful, but good enough to at least boot frugal install into qemu without risk of corrupting host using same disk... Seems safe.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

So the scripted project version of this has now effectively gone from abandoned to ongoing active again...

Not quite so ambitious in that no writes are allowed back to host filesystem at all (-snapshot qemu option seems to ensure that thus far...)

But, I am starting with w_changes=RAM2 mode, but not because I want save on demand (which isn't allowed here as I say) but because that mode includes any existing media upper_changes in second topmost overlay layer. That turns out to be very useful. Means I can add anything wanted in normal frugal install usage and later have exact same distro available as a quick qemu virtual machine inside whatever host distro I am working with...

In that respect it isn't a big deal not being able to write back to filesystem. However, via a 9p type shared disk (which I showed how to use on forum some long time back) it may be possible to arrange writing anyway... In fact might even prove possible to use 9p shared host disk for booting (but that's just an idea that may be a non-starter...). Plenty to experiment fruitfully with anyway.

By the way, I'm actually currently posting from FR/KL_Fedora firstribit frugal install running in qemu via the (now-renamed) frugal2qemu.sh utility I've written. Working nicely. Though primarily for my FirstRib-based distro needs, I've written it to currently (likely) work with Puppy Linux too via a simple plugin config file (frugal2qemu.cfg) with first line: distro=puppy
Can add other case statement stanzas to frugal2qemu.sh for other frugal installed distros; only real difference is the kernel arguments they need. Actually, an alternative is to use distro=puppy but include a q_append string containing appropriate kernel boot stanzas in the frugal2qemu.cfg config file... so very flexible and useful little script really. For example, the inbuilt puppy p_append string contains this:

Code: Select all

q_append="psubdir=/${subdir} pfix=nocopy pmedia=usbflash psave=${bootuuid}:/$subdir/ pdrv=${bootuuid}"

The q_ in the variable name is just a reminder to myself that this is for the qemu -append argument.
You will find/notice later that the values for subdir and bootuuid are automatically provided by a little w_grubconfig type function, which auto determines exactly the frugal install directory the script is run from.
Note that I am no expert on Puppy boot arguments, so may not be the best. You can always change these. I found them via @rockedge forum Boot parameters link: viewtopic.php?p=52875#p52875

NOTE ALSO that no grub setup was required whatsoever to use this since booting directly into qemu. The host system I was using was Linux Mint XFCE, but any suitable Linux including most any Puppy or KL distro that has qemu installed (with kvm active) would do

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

frugal2qemu.sh with frugal2qemu.cfg under final test.

Whilst I only really now know FirstRib boot arguments, so probably my boot results are lucky or could be improved/simplified, I have done my best to make the program work with frugal installs of all of:

FR/KL
Puppy
DebianDog (per screenshot attached)
FatDog (per screenshot attached)
EasyOS (per screenshot attached)

Key is that I don't allow writes so that is a limitation. If you decide to experiment and allow 'writing' back to save persistence be WARNED you would be on your own; my experiments confirm that is dangerous and tends to cause underlying host filesystem corruption. You don't want that ;-)

However, all seems fine for the script including -snapshot as means to prevent qemu writing to underlying host filesystem, and save folder should be loaded up okay (with whatever you put in that in actual frugal install boots).

NOTE AGAIN that no actual boot loader is needed with this script; i.e. no grub or similar required since I am booting the vmlinuz and initrd (whatever they are called by the distro implementation) directly via qemu. Only useful if you do have normal frugal installs of course...

I will publish this soon once I have completed final tests.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

Clarity
Posts: 3647
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1530 times
Been thanked: 489 times

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by Clarity »

wiak wrote: Thu Jul 18, 2024 4:21 am

... But what I'm using is exact same hard drive filesystem being used by two effectively separare computers at same time. ...

Sorry for the delay. Minor health issues.

In review, @wiak, as "I think" I know what you want. I think that there may be a minor misunderstanding from what the host has as physical and what QEMU (KVM) operates on. (YES, a host "can" corrupt a QEMU drive. Additionally, a VM "can" corrupt a host partition/drive that is defined by QEMU as 'raw' if written to.)

The current beauty of the QEMU implementations in the forum is that:

  1. We have a LAN connection to the host when using the features of the QEMU stanzas I demonstrate.

  2. you have SAMBA built-in the host, so create a host share

  3. Things you want both to see AT ANY TIME is to be read-written to the host's share(s) that you create...including printers, CDroms, USB sticks, and anything else you share from the host that the VM will see

  4. The tools to see the host's shares are built-in to forum distros

  5. Thus NO software in needed to be installed as all that's needed in built-in to each OS waiting for use by users.

In addition, if you want other PCs on the 'real' LAN to see what the VM offers:

  1. Setup a Bridge that the VM will use

  2. In the VM, its own SAMBA to create shares

  3. Things (aka 'shares') in the VM that you want ALL (including the host) to see will be seen by ALL LAN PCs as your VM(s) will appear to them as real PCs.

  4. Bridges are used to allow multiple VMs to get to the 'real' LAN over the host's LAN adapter (@fatdog/@jamesbond has posted somewhere how to set up a bridge for KVM use, IIRC

This is merely information that sometimes, even I, forget to mention in directions to users for safe operation of VM with their host.

Edit 1: 'raw' comment in paragraph 1

Last edited by Clarity on Tue Jul 23, 2024 8:19 am, edited 1 time in total.
User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Clarity wrote: Mon Jun 16, 1975 5:25 pm

In review, @wiak, as "I think" I know what you want. I think that there may be a minor misunderstanding from what the host has as physical and what QEMU (KVM) operates on. (YES, a host "can" corrupt a QEMU drive.)
...
[*]you have SAMBA built-in the host, so create a host share
[*]Things you want both to see AT ANY TIME is to be read-written to the host's share(s) that you create...including printers, CDroms, USB sticks, and anything else you share from the host that the VM will see
[*]The tools to see the host's shares are built-in to forum distros

What you are writing about is a totally different matter to the shared with host drive issues I refer to here. There remains no issue whatsoevef with file sharing between host and guess when using the likes of samba, ftp, or Plan 9 (9p protocol) file sharing. These have nothing to do with normal save on demand procedures. I will be investigating 9p as a possible workaround mechanism to allow save on demand to be safely undertaken - save on demand underneath operation is done by the initrd provision; nothing per se to do with the likes of user use of say samba SMB for that kind of basic user level file sharing. No misunderstanding on my part; seems like you do have a misunderstanding about why frugal2qemu needs to disable normal save on demand by a distro being booted in this manner from frugal install into qemu. It is simply not wiseto directly share the underlying disk with both host and guest at same time. Trust me, doing so usually results in physical host disk getting corrupted and requiring fsck to fix it (if lucky...). Repeat: this is nothing to do with samba one way or the other.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

Clarity
Posts: 3647
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1530 times
Been thanked: 489 times

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by Clarity »

"I think" I understand what you are saying as you are looking at the system during booting via the efforts surrounding the INIT system when booting and system's operation.

Correctly, you point out that my comments refer to system operations after desktop arrivals.

Clarity
Posts: 3647
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1530 times
Been thanked: 489 times

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by Clarity »

BTW: Your QEMU-FirstRibIT is very clever uses you provide.

The FirstRibIT2QEMU generates a running new distro within a KVM. Nice!

The hidden jewel is that a user could have KL running in a VM and use this script within that running system to create a VM. (I know this leads to a host with a VM which has a VM in the VM. But as @666philb and I showed years ago this runs well for various use/testing.)

There are 2 things I offer to consider:

  • If QEMU is not present when script is used, should an 'QEMU install' offer (to the user) emerge when the script is launched?

  • The script should either test for the KVM module loaded OR , by default, run @norgo's "qemu-ready" script to ensure the module is loaded; thus you wont have to generate test code on your own for use in your script.

User avatar
wiak
Posts: 4005
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 60 times
Been thanked: 1157 times
Contact:

Re: using qemu to boot a firstribit or some other FR/KL creation

Post by wiak »

Clarity wrote: Tue Jul 23, 2024 8:42 am

There are 2 things I offer to consider:

  • If QEMU is not present when script is used, should an 'QEMU install' offer (to the user) emerge when the script is launched?

  • The script should either test for the KVM module loaded OR , by default, run @norgo's "qemu-ready" script to ensure the module is loaded; thus you wont have to generate test code on your own for use in your script.

I take the view almost always that I cannot assume the scripts of other people being available on utility script target distros. In any case I most often have no control over the dependencies of the scripts /utilities of others. For example the norgo script you refer to was designed I imagine for Puppy Linux and even then is probably not included in all pups and may or may not work without additional work or dependencies in other distros.

Certainly I could later add a test for qemu and kvm components being installed and the necessary modules loaded, which is a pretty trivial check to make. Since I write for wide range of KL distro upstream types, I would not offer to install since so many package manager situations are involved including possible AppImages. I would instead simply print an appropriate install needed message. So that could be done easily enough.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

Post Reply

Return to “firstribit”