How to make a remastering tool use space on another partition?

[Luluc]

We may find ourselves in the situation where the space needed for remastering is not available on the same partition as the frugal installation, but it is available on another partition. Is there any clever trick we can pull to make the remastering tool use that space on the other partition?

[fredx181]

We may find ourselves in the situation where the space needed for remastering is not available on the same partition as the frugal installation, but it is available on another partition. Is there any clever trick we can pull to make the remastering tool use that space on the other partition?

Besides the "Quick-remaster" tool there should be another one too, in Menu named "Remaster-DebDog64" , or type in terminal remasterdog
(edit: if not available, install remaster-scripts)
It lets you choose a "working" directory on a partition of choice (must be Linux filesystem, e.g. ext4):
(but note that the outcome will be different than with quick-remaster, extra modules (e..g. kernel module) will be merged in the final .squashfs)

Screenshot.png (63.48 KiB) Viewed 733 times

EDIT: Where it says .xzm is wrong (outdated), must have extension .squashfs and recommended to name it 01-filesystem.squashfs, to replace the one in your frugal install.

[Luluc]

Thanks.

I just tried it, and it failed badly. Really bad.
I chose sdb1/newdog. It created a big "root" file structure there. But it was very flawed.
My original root structure has "sdc", a symlink to another partition. A shortcut, nothing more. For some reason, the remaster tool copied a lot (but not all) of that partition to the newdog file structure. The rest of the file structure was comprised almost entirely of empty directories.
I also have "sdd" which is - you guessed it - a symlink to yet another partition. But the remaster tool didn't copy anything from it. So the whole thing seems to be random. (Both partitions were mounted during the whole procedure.)
The remaster tool offered me the chance to clean that file structure before finalizing it so I removed everything it had copied from the sdc partition.
I finalized the remaster by choosing the compression method.
Finally, the result was a 64KB 01-filesystem.squashfs file. What?! My current 01-filesystem.squashfs has 1.6GB. Where did everything else go?
This is not what I wanted at all.

Besides the "Quick-remaster" tool

The Quick-remaster tool does exactly what I want except it doesn't let me select a working directory (with more space available).

[fredx181]

@Luluc
Strange... I just tried it before I posted about it.
Let's see if we did similar. Assuming:
- You chose a drive from the top dropdown box (is it symlink ? as I understand you mentioned)
- Filled in the entry for the working directory, just a name, not a path
- Filled in the last entry 01-filesystem.squashfs

I also have "sdd" which is - you guessed it - a symlink to yet another partition. But the remaster tool didn't copy anything from it.

Something went obviously wrong for you , probably has to do with the symlink , don't understand , the remaster tool should copy stuff from the actual booted system, not copy from a (symlinked) partition.
From which system did you run remasterdog ? Debiandog Bookworm ? or ...

edit: perhaps there's confusion with the subject of your other recent thread ?

[Luluc]

- You chose a drive from the top dropdown box (is it symlink ? as I understand you say)

- Yes, but that drive is not a symlink. I just have /sdb, /sdc and /sdd as symlinks in my / root directory.
- I filled in just a name, not a path. The tool created /newdog correctly.
- Filled in the last entry 01-filesystem.squashfs. Correct.

I ran remasterdog while running Debiandog Bookworm.

I suspect it copied so much stuff from sdc (it's a big partition, lots of data) that it ran out of space to add stuff from the booted system. But it should never have copied stuff from another partition, right?

[fredx181]

- You chose a drive from the top dropdown box (is it symlink ? as I understand you say)

- Yes, but that drive is not a symlink. I just have /sdb, /sdc and /sdd as symlinks in my / root directory.
- I filled in just a name, not a path. The tool created /newdog correctly.
- Filled in the last entry 01-filesystem.squashfs. Correct.

I ran remasterdog while running Debiandog Bookworm.

I suspect it copied so much stuff from sdc (it's a big partition, lots of data) that it ran out of space to add stuff from the booted system. But it should never have copied stuff from another partition, right?

Indeed, it should only copy from the booted system.
EDIT: But frankly I never tried it while having symlinks on / , could well be the problem, that the contents from those symlinks were copied
Running out of space could have caused the failure, if current 01-filesystem.squashfs has 1.6GB, the uncompressed root filesystem can be around 4-5 GB , I guess.
The remaster tool, btw, creates first empty directories .

[Luluc]

It failed again.

This time, I chose sda2 because it has a lot of free space.
But it's encrypted. It's /dev/mapper/csda2
Well, the remaster tool gives me these options:
mapper/csda2 mapper/csdc1 sda1 sda2 sdb1 sdc1
I chose mapper/csda2.

But remasterdog didn't write there, in encrypted (decrypted and mounted) sda2. Instead, it created /mnt/mapper/css/newdog which is inside sda1.
That doesn't work for me. The available space on sda1 is not much.

I shiver to think about what might happen if I chose raw sda2.
The script needs to be smarter.

[fredx181]

The script needs to be smarter.

Well, yes, not much tested for many cases.
Btw, I just tested with a symlink on / and went fine for me, it's just symlink and couldn't have caused the failure for you.

[Luluc]

Maybe it should just prompt us for a path instead of trying to guess everything. I would be a lot more comfortable with entering a full path.

[fredx181]

@Luluc
As you said that quick-remaster works for you, here's a modifiied "quick-remastergui" script that has extra checkbox for setting a custom directory:

quick-remastergui.gz

Remove fake .gz extension and make executable

(23.48 KiB) Downloaded 9 times

If selecting a directory , then of course you need to replace manually the 01-filesystem.squashfs in the frugal install dir with the new one before rebooting.
edit: and also just before the point of ISO creating, if selected.

Screenshot_2024-11-12_07-18-13.png (44.9 KiB) Viewed 577 times

(still curious though why remasterdog fails for you, i.e. not enough space in the partition or ....)

[Luluc]

Yes, that one works well. Many thanks!

[Luluc]

Something is wrong. Here is a very detailed description.

- I run synaptic. It only shows installed packages. Where are the other ones?
- I update packages. All the other packages become visible.
- I don't like how synaptic looks. Strange. I had made the changes to make it look the way I like it.
- I use Bookworm Dog a little.
- I remaster it.
- I reboot into another Linux. (...)

- I'm back to Bookworm Dog.
- I run synaptic. It only shows installed packages. Where are the other ones?
- I update packages. All the other packages become visible.
- I don't like how synaptic looks. Strange. I had made the changes to make it look the way I like it.
- I use Bookworm Dog a little.
- I remaster it.
- I reboot into another Linux. (...)

- I'm back to Bookworm Dog.
- I run synaptic. It only shows... Oh, come on. This is like Groundhog Day (the movie)!
- I create an empty file on my Desktop area (/root/Desktop).
- I remaster.
- I reboot.
- The empty file is there. It was saved.
- I open /root/.synaptic/. It's empty.
- I reconfigure synaptic.
- A synaptic.conf file is created inside /root/.synaptic/.
- It's also present in /mnt/live/mnt/sda1/bookworm/live/changes/upperdir/root/.synaptic/
- I remaster.
- I reboot.
- I open /root/.synaptic/. It's empty.
- I run synaptic, it's lost all the previous configurations.
- I make some changes to the GUI and close it.
- I open /root/.synaptic/. The synaptic.conf file is there.

- I remaster.
- I reboot.
- I open /root/.synaptic/. It's empty.
- I run synaptic, it's lost all the previous configurations.
- I make some changes to the GUI and close it.
- I DO NOT remaster. JUST reboot.
- Finally! Synaptic looks correct now.

- No changes. Just another remaster.
- Reboot.
- I open /root/.synaptic/. It's empty again.
- I run synaptic, it's lost all the previous configurations again. Amazing.

[Luluc]

FYI,
I just did some experiments with Fossa Dog.
Before calling it a day, I remastered and rebooted.
The exact same problem with synaptic happened again.

[fredx181]

@Luluc
It does some cleaning [(e.g. cache, logs etc..) by excluding files in the remaster, too much for your liking apparently. (edit: about root/.synaptic, I agree that it is better to keep)
You can edit the script , look at lines 238-268 and comment out what you want to keep e.g. /var/cache/apt, /var/lib/apt or/and root/.synaptic

I wiil probably make new version (package) soon, any suggestions besides what you mentioned already ?
(the idea behind the cleaning is that you can possibly share the remaster with others, without cache, logs etc.. and of course without personal stuff)

[Luluc]

I suggest maybe a check box to enable/disable this "cleaning." Or maybe a simple quick-remaster.conf configuration file in ~/ such as:
remove_man_docs = no
merge_modules = yes
clean_cache = no
create_iso = no
custom_directory = /some/path

I know I always want to keep man pages and docs and have to de-select that option every time, every time, every time. It would be nice to be able to change the defaults. Hey, maybe also a no_prompts config option so I could run the remaster non-interactively. But for that we need to be able to change/control the defaults.

Thank you.

[dcung]

I know I always want to keep man pages and docs and have to de-select that option every time, every time, every time. It would be nice to be able to change the defaults.

On the other hand, I always do not want to keep them. So, would be nice to keep it the way it is.

I agree, adding non-interactive feature is good.

[fredx181]

I'd call it ultimate laziness to complain about one or two options to select or deselect, sorry

[dcung]

I'd call it ultimate laziness to complain about one or two options to select or deselect, sorry

Not sure who you are referring to.
But, I really couldn't care less whether it is on or off.
Just pointing out that 'default' can not satisfy all in any case.

[fredx181]

I'd call it ultimate laziness to complain about one or two options to select or deselect, sorry

Not sure who you are referring to.
But, I really couldn't care less whether it is on or off.
Just pointing out that 'default' can not satisfy all in any case.

Well, just saying that what's default (or not default) shouldn't be very important (everyone has it's own choices) as it's easy enough to just click on what's preferred or not.

[dcung]

Well, just saying that what's default (or not default) shouldn't be very important (everyone has it's own choices) as it's easy enough to just click on what's preferred or not.

Indeed.

Anyway, I'm already regretting chiming in...
Cheers. Over and out.

[Luluc]

I'd call it ultimate laziness to complain about one or two options to select or deselect, sorry

Laziness has fostered a lot of innovation in the history of civilization. Machines are meant to serve man.

[fredx181]

I'd call it ultimate laziness to complain about one or two options to select or deselect, sorry

Laziness has fostered a lot of innovation in the history of civilization. Machines are meant to serve man.

Ok, maybe one day, you'll have a chip in your head that connects to your computer.
Then, the latest, greatest quick-remaster program scans exactly what you prefer (and you'll probably became one of the laziest human beings that ever existed ).

[dancytron]

IIRC, the version that you did for First Rib had a config file control what got trimmed.

I thought that was a good change.

Dan

[Luluc]

We shall never lose hope.

https://wiki.tcl-lang.org/page/RMMADWIM
.

[Luluc]

I may have run into another problem.

I have been doing experiments with FossaDog and used the new quick-remaster tool on it. It works except for creating an ISO image. That fails. There is some error message, but I don't know what it is. It closes very fast and I can't read it. I just catch a glimpse.

So I edited the quick-remaster tool that is included in FossaDog to remove the aggressive "cleaning" and used it. I couldn't choose the target partition of the new squash file, but ISO image creation worked.

[fredx181]

I may have run into another problem.

I have been doing experiments with FossaDog and used the new quick-remaster tool on it. It works except for creating an ISO image. That fails. There is some error message, but I don't know what it is. It closes very fast and I can't read it. I just catch a glimpse.

So I edited the quick-remaster tool that is included in FossaDog to remove the aggressive "cleaning" and used it. I couldn't choose the target partition of the new squash file, but ISO image creation worked.

I wouldn't recommend using a program on FossaDog that's made for Debiandog (as there are much differences between these)
Btw, I'd say better not use Fossadog anyway, because it hasn't been updated in years.

[fredx181]

*** Quick Remaster Updated (v1.6.1) ***

For Bookworm and Daedalus, to install: apt update; apt install quick-remaster

• added checkbox for to create remaster in custom selected directory
• and checkbox for more advanced excluding options
• some cosmetic changes
• added Info button (contains more info about excluding files/folders)
• the 'exclude list' is changed now to $HOME/excludelist and can be kept to be able to use again (sort of 'unattended' excluding)

Tried to find a balance between (over)complexity and a (hopefully enough) user-friendly way to configure more exact what to exclude.

EDIT: for a bit more control of what to exclude (for who wants that first time running this new version) :
Not possible to have a "full" /root/excludelist before running the remaster first time , so that you could edit it before, manually (unattended ).
Not very convenient, but the trick/workaround that I did was to first configure all to exclude from the GUI, click Ok, then:

• at the point that creating 01-filesystem.squashfs is running (with the progress bar), cancel it by closing that window
• start again by first editing /root/excludelist as preferred with text-editor and then run Quick-Remaster again (you'll get prompt Yes/No to use it or not, if exists)
  (can be useful to refine more, e.g. if you want only cache to exclude and keep the logs, just remove all var/log/... from the excludelist (just example)

2024-11-16_13-26-37.png (198.29 KiB) Viewed 226 times

[dancytron]

Remastered my main Bookworm install with manual exclude file (but I didn't change it).

All seemed to work correctly.

[Luluc]

I wouldn't recommend using a program on FossaDog that's made for Debiandog (as there are much differences between these)

OK, but what differences? I used it and it works fine except for ISO image.

Btw, I'd say better not use Fossadog anyway, because it hasn't been updated in years.

What needs to be updated? It installs, it runs, it uses official Ubuntu repositories. Focal Fossa is still supported by Ubuntu. What needs to be updated?

I like it because the repository has some packages that have been abandoned by Debian. Maintainers are very insensitive about that. Fossa also accepts some packages from Bionic. I suspect I may be happier using Ubuntu than Debian. I also use some old Windows applications thanks to Wine. Just because some softtware is "old" or "hasn't been updated" doesn't mean it's not good anymore.

[fredx181]

I wouldn't recommend using a program on FossaDog that's made for Debiandog (as there are much differences between these)

OK, but what differences? I used it and it works fine except for ISO image.

Btw, I'd say better not use Fossadog anyway, because it hasn't been updated in years.

What needs to be updated? It installs, it runs, it uses official Ubuntu repositories. Focal Fossa is still supported by Ubuntu. What needs to be updated?

I like it because the repository has some packages that have been abandoned by Debian. Maintainers are very insensitive about that. Fossa also accepts some packages from Bionic. I suspect I may be happier using Ubuntu than Debian. I also use some old Windows applications thanks to Wine. Just because some softtware is "old" or "hasn't been updated" doesn't mean it's not good anymore.

No, no, didn't mean that it (Ubuntu Focal Fossa) is old and therefore not good. just that I didn't put much effort in maintaining FossaDog (e.g. new ISO with updated kernel, updated browser, bug fixes, improvements etc..), for reasons (e.g. time, energy) now focused on just Debian/Devuan (with the build scripts and some more).

OK, but what differences? I used it and it works fine except for ISO image.

Ok, then you were probably lucky