Remastering Debiandog

[gabtech]

Which folders/files can be removed when remastering any Debiandog.

[dancytron]

You don't need to worry about the synaptic/apt related data, the script deletes that for you.

I always delete my Chrome caches and profiles, which if you use Fred's system end up in either /root/chrome or /home/puppy/chrome.

I always check /root and /home/puppy for any stray user settings from other browsers. I also delete /root/.cache, the contents of /var/logs, and /root/.xsession-errors.

You used to have to delete the docs and manuals manually, but there is now a checkbox to do that.

I use bleachbit too, which gets rid of a lot of the misc trash.

I'm sure there is more, but that's what I do.

[dcung]

I am not familiar with remastering.
I tried in StretchDog using "Quick Remaster Gui". Once with XZ, once with LZIP compression. Both time change dir didn't get cleaned up.
It created the new 01-filesystem.squashfs and backup the old one to 01-filesystem.squashfs.bak

Laptop has 1 SSD, with only one ext4 partition. I think, porteus boot, since I saw the utility was checking before proceeding.
My menu.lst file

Code: Select all

# menu.lst produced by grub4dosconfig-v1.9.2
color white/blue black/cyan white/black cyan/black
#splashimage=/splash.xpm
timeout 10
default 1

title FossaDog64
 uuid 0c865b1a-73b0-4c3a-8085-0f163669c6d6
 kernel /FossaDog64/casper/vmlinuz from=/FossaDog64/ noauto changes=/FossaDog64/casper/
 initrd /FossaDog64/casper/initrd1.xz

title StretchDog
 kernel /Stretch-1/live/vmlinuz1 from=/Stretch-1/ noauto changes=/Stretch-1/live/
 initrd /Stretch-1/live/initrd1.xz

Upon reboot, this is what on screen. Both times.

[fredx181]

Hi dcung, I will investigate later what can be wrong, when I have time, did you use the quick-remastergui included in the Stretch ISO or the updated to latest version ? (package "quick-remaster").
In the meantime: Can you run again quick-remastergui from terminal, check for possible errors and post here ?

Fred

[dcung]

Hi Fred,
I was using quick-remaster included in ISO.
Once I read your reply, I upgraded to latest version and tried again. Which I should have done. Usually, I upgrade all upgrade-able on other installation. And I thought this would make the new master big. I only want new master to have nvidia driver since I have 4 D630 laptops. So, I didn't upgrade anything.

The latest version works fine. The original version spat out error about making some link, and something about initrd1.xz, and manual clean of change dir. It went too fast to read, and unlike the later version, it didn't pause and proceeded to reboot.

Thank you Fred.

[dcung]

It looks like, I'm not meant to have a remaster....
The remaster have all my customization, but nvidia driver didn't work. nvidia-xconfig command not found too.
It reverted to VESA. Something that it needs, didn't get saved during remastering.
Anyway...easy enough...I just copy change dir from one laptop to another.

2021-06-08-113304_665x404_scrot.png (108.5 KiB) Viewed 2303 times

I noticed a minor cosmetic issue. Synaptic indicated that remaster version is 1.2.5.
The GUI showed version 1.2.4.

2021-06-08-113433_772x53_scrot.png (10.24 KiB) Viewed 2303 times

[fredx181]

@dcung
I think it's similar problem as reported here for Fossadog: viewtopic.php?p=7826#p7826 (and see my post after)
I've made the same fix now for the Stretch quick-remaster package: updated to v1.2.6

Code: Select all

apt update
apt install quick-remaster  # will install v1.2.6

So good chance it will be OK now when making new remaster, including the Nvidia driver.

EDIT:

Once I read your reply, I upgraded to latest version and tried again. Which I should have done. Usually, I upgrade all upgrade-able on other installation. And I thought this would make the new master big

I think it's best to upgrade (all packages or what you choose to upgrade) before remastering, btw it will take less space when upgrades are included in the remaster than when all upgraded packages are in your save storage, as in the remaster the files will be just replaced and take no (or probably not much) extra space.

Fred

[dcung]

Got these errors trying to get remaster v1.2.6.
Already retried in few hours, still errors. I will try again later.

Code: Select all

root@live:~# apt update
Hit:1 http://security.debian.org/debian-security stretch/updates InRelease
Ign:2 http://deb.debian.org/debian stretch InRelease                           
Err:1 http://security.debian.org/debian-security stretch/updates InRelease     
  Couldn't create temporary file /tmp/apt.conf.UoGMHD for passing config to apt-key
Hit:3 http://deb.debian.org/debian stretch Release                             
Err:4 http://deb.debian.org/debian stretch Release.gpg                         
  Couldn't create temporary file /tmp/apt.conf.ZalLPH for passing config to apt-key
Get:5 http://dl.google.com/linux/chrome/deb stable InRelease [1,811 B]         
Err:5 http://dl.google.com/linux/chrome/deb stable InRelease                   
  Couldn't create temporary file /tmp/apt.conf.QYMGZS for passing config to apt-key
Get:7 https://fredx181.github.io/StretchDog/amd64/Packages ./ InRelease [2,302 B]
Err:7 https://fredx181.github.io/StretchDog/amd64/Packages ./ InRelease
  Couldn't create temporary file /tmp/apt.conf.X55QeI for passing config to apt-key
Hit:6 https://ixpeering.dl.sourceforge.net/project/ubuntuzilla/mozilla/apt all InRelease
Err:6 https://ixpeering.dl.sourceforge.net/project/ubuntuzilla/mozilla/apt all InRelease
  Couldn't create temporary file /tmp/apt.conf.nIEHSR for passing config to apt-key
Ign:8 http://snapshot.debian.org/archive/debian/20200528 stretch InRelease
Hit:9 http://snapshot.debian.org/archive/debian/20200528 stretch Release
Err:10 http://snapshot.debian.org/archive/debian/20200528 stretch Release.gpg
  Couldn't create temporary file /tmp/apt.conf.QEHCHp for passing config to apt-key
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://security.debian.org/debian-security stretch/updates InRelease: Couldn't create temporary file /tmp/apt.conf.UoGMHD for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://deb.debian.org/debian stretch Release: Couldn't create temporary file /tmp/apt.conf.ZalLPH for passing config to apt-key
W: GPG error: http://dl.google.com/linux/chrome/deb stable InRelease: Couldn't create temporary file /tmp/apt.conf.QYMGZS for passing config to apt-key
E: The repository 'http://dl.google.com/linux/chrome/deb stable InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://fredx181.github.io/StretchDog/amd64/Packages ./ InRelease: Couldn't create temporary file /tmp/apt.conf.X55QeI for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://ixpeering.dl.sourceforge.net/project/ubuntuzilla/mozilla/apt all InRelease: Couldn't create temporary file /tmp/apt.conf.nIEHSR for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://snapshot.debian.org/archive/debian/20200528 stretch Release: Couldn't create temporary file /tmp/apt.conf.QEHCHp for passing config to apt-key
root@live:~# 

[dcung]

While waiting to get quick-remaster v1.2.6 for StretchDog, I tried both on FossaDog and BusterDog which are up-to-date.

FossaDog quick-remaster v1.2.6
BusterDog quick-remaster v1.2.7
Both remaster - nvidia driver did not work and reverted to VESA.

EDIT:
I 'followed' what Flying Cat did. viewtopic.php?p=7826#p7826
Overwriting from backup '/lib/modules/*kernel-number*-generic'
And nvidia driver works again on all remasters, StretchDog, FossaDog and BusterDog.
I don't know what difference the one remaster made and the backup?

[rcrsn51]

On the remastered machine, run: depmod -a

Both remaster did not keep nvidia driver.

Or do you mean that the file was actually missing from /lib/modules?

[fredx181]

@dcung Re: apt update failed, I think these errors, e.g.:

Code: Select all

Couldn't create temporary file /tmp/apt.conf.QEHCHp for passing config to apt-key

are important, maybe for some reason your /tmp has the wrong permissions, I did a Stretch remaster and from it apt update works fine, my /tmp has drwxrwxrwt :

Code: Select all

ls -l /
....
....
drwxrwxrwt   5 root root  200 Jun  9 11:44 tmp

If it's not as above, try;
chmod 1777 /tmp

[dcung]

Or do you mean that the file was actually missing from /lib/modules?

Note - I reworded above to make it clearer (I hope).
There were files. But I had to overwrite with ones from backup before nvidia driver works again.
Have you remastered your nvidia machine? If so, nvidia driver works?

On the remastered machine, run: depmod -a

Came back to prompt empty.

@fredx181
StretchDog -> I updated quick-remaster to v1.2.6.
Noticed that you've corrected/updated the GUI title accordingly (not showing v.1.2.4 anymore ). In FossaDog (1.2.6) & BusterDog (1.2.7) title still showed 1.2.4 if you feel like updating those. I know I can easily edit my local script file to correct it. FYI for those that download in the future from repo.

Then - I remaster working StretchDog with v1.2.6.
nvidia driver did not work in the resulting remaster.
Re-copy (overwrite) from backup, and it works again.
I'm ok with current workaround, so will stop investigating.
I will clean up now....got too many change directories....

I narrow down to these files that makes the difference. Can't be bother to narrow down further.
Interestingly the ones from remastered dated '2020', where did they come from?

Code: Select all

 FROM REMASTER - nvidia driver DOES NOT WORK
total 4108
   4 drwxr-xr-x  5 root root    4096 Jun 10 07:17 .
   4 drwxr-xr-x  5 root root    4096 Jun 10 07:17 ..
   0 lrwxrwxrwx  1 root root      37 Jun 10 06:37 build -> /usr/src/linux-headers-4.9.0-12-amd64
   0 drwxr-xr-x 22 root root     187 Jan 20  2020 kernel
1000 -rw-r--r--  1 root root 1023874 Feb 20  2020 modules.alias
 961 -rw-r--r--  1 root root  983474 Feb 20  2020 modules.alias.bin
   4 -rw-r--r--  1 root root    4018 Jan 20  2020 modules.builtin
   6 -rw-r--r--  1 root root    5327 Feb 20  2020 modules.builtin.bin
 390 -rw-r--r--  1 root root  398625 Feb 20  2020 modules.dep
 537 -rw-r--r--  1 root root  549411 Feb 20  2020 modules.dep.bin
   1 -rw-r--r--  1 root root     402 Feb 20  2020 modules.devname
 131 -rw-r--r--  1 root root  133407 Jan 20  2020 modules.order
   1 -rw-r--r--  1 root root     550 Feb 20  2020 modules.softdep
 480 -rw-r--r--  1 root root  490804 Feb 20  2020 modules.symbols
 592 -rw-r--r--  1 root root  605907 Feb 20  2020 modules.symbols.bin
   0 -rw-r--r--  1 root root       0 Jun 10 07:17 remaster.txt
   0 lrwxrwxrwx  1 root root      38 Jun 10 06:37 source -> /usr/src/linux-headers-4.9.0-12-common
   0 drwxr-xr-x  3 root root      27 Jun 10 06:37 updates

Code: Select all

 FROM BACKUP - nvidia driver WORKS
total 3979
   4 drwxr-xr-x  2 root root    4096 Jun 10 07:18 .
   4 drwxr-xr-x 20 root root    4096 Jun 10 07:18 ..
1001 -rw-r--r--  1 root root 1024205 Jun  8 10:14 modules.alias
 961 -rw-r--r--  1 root root  983915 Jun  8 10:14 modules.alias.bin
   6 -rw-r--r--  1 root root    5327 Jun  8 10:14 modules.builtin.bin
 390 -rw-r--r--  1 root root  398879 Jun  8 10:14 modules.dep
 537 -rw-r--r--  1 root root  549721 Jun  8 10:14 modules.dep.bin
   1 -rw-r--r--  1 root root     402 Jun  8 10:14 modules.devname
   1 -rw-r--r--  1 root root     550 Jun  8 10:14 modules.softdep
 482 -rw-r--r--  1 root root  492795 Jun  8 10:14 modules.symbols
 594 -rw-r--r--  1 root root  607931 Jun  8 10:14 modules.symbols.bin
   0 -rw-r--r--  1 root root       0 Jun 10 07:18 nvidia.txt

[rcrsn51]

I am still unclear as to whether the nvidia.ko driver got into the remaster.

Run: find /lib/modules -name nvidia.ko

Is there a blacklist file in /etc/modprobe.d for the nouveau driver?

If the driver is present, the depmod command would register it into the system. On the next reboot, it should take effect.

[fredx181]

Interestingly the ones from remastered dated '2020', where did they come from?

That's the whole point, these "Feb 20 2020" files are from the original k-4.9.0-12-amd64.squashfs, should be the updated files from Jun 8.
I cannot understand why these updated files are not in the remaster.
Just before you start remaster, are the files in /lib/modules/4.9.0-12-amd64/ showing the updated ones ? (e.g. "Jun 8 10:14 modules.alias")
If not, then perhaps you should do a "depmod -a" just before remastering (however, the Nvidia install should have done that, so again, I cannot understand what happened).
P.S. did you select "Merge currently loaded modules... .." at start of the remaster ?
(just like to know, so I could possibly try to reproduce exactly what you did)

EDIT: Please ignore above, found out what's the problem, see next post.

Fred

[fredx181]

Oooops.... I understand now why the old files from /lib/modules/4.9.0-12-amd64/ are in the remaster.
My bad, sorry, the module k-4.9.0-12-amd64.squashfs is later in alphanumerical order, so will override the (newer) files in remastered 01-filesystem.squashfs.
Need to think about solution for quick-remaster, perhaps make the merging of extra modules the default.
Renaming k-4.9.0-12-amd64.squashfs to e.g. 00-k-4.9.0-12-amd64.squashfs should fix it too (so that it's earlier in alphanumerical order).

Fred

[dancytron]

Oooops.... I understand now why the old files from /lib/modules/4.9.0-12-amd64/ are in the remaster.
My bad, sorry, the module k-4.9.0-12-amd64.squashfs is later in alphanumerical order, so will override the (newer) files in remastered 01-filesystem.squashfs.
Need to think about solution for quick-remaster, perhaps make the merging of extra modules the default.
Renaming k-4.9.0-12-amd64.squashfs to e.g. 00-k-4.9.0-12-amd64.squashfs should fix it too (so that it's earlier in alphanumerical order).

Fred

Because it's later in alphanumeric order than "01-filesystem.squashfs"?

[fredx181]

Because it's later in alphanumeric order than "01-filesystem.squashfs"?

Yes, the files in k-4.9.0-12-amd64.squashfs will have priority over the files that are inside "01-filesystem.squashfs" (which is lower in alphanumeric order), even if the files with the same path are newer in 01-filesystem.squashfs.

EDIT: Anyone have opinion how to fix this for quick-remaster ? I think probably the best/safest is to make it standard to merge extra modules in "live" in the remaster (and disable them by renaming to .bak, also the kernel .squashfs), so no choice not to merge then.

[dancytron]

Because it's later in alphanumeric order than "01-filesystem.squashfs"?

Yes, the files in k-4.9.0-12-amd64.squashfs will have priority over the files that are inside "01-filesystem.squashfs" (which is lower in alphanumeric order), even if the files with the same path are newer in 01-filesystem.squashfs.

That makes sense.

I may have seen that in Buster on my laptop installing TV card kernel drivers and then remastering, but I just pasted them in manually.

edit: as far as the solution, I'd make them sort in the correct order rather than merge them into one file again. Either by just changing the name or maybe by changing the script so it knows which one the kernel one is no matter what it is named.

[fredx181]

Because it's later in alphanumeric order than "01-filesystem.squashfs"?

Yes, the files in k-4.9.0-12-amd64.squashfs will have priority over the files that are inside "01-filesystem.squashfs" (which is lower in alphanumeric order), even if the files with the same path are newer in 01-filesystem.squashfs.

That makes sense.

I may have seen that in Buster on my laptop installing TV card kernel drivers and then remastering, but I just pasted them in manually.

edit: as far as the solution, I'd make them sort in the correct order rather than merge them into one file again. Either by just changing the name or maybe by changing the script so it knows which one the kernel one is no matter what it is named.

Thanks Dan. Well.. not sure yet, still thinking about it, in fact every extra module should be renamed then to 00-....squashfs (not only the kernel) as there's always a chance that the user has modified files that are also existing in one of the extra modules.

[rcrsn51]

This discussion just confirms my lack of interest in remastering.

It's simpler to run SaveFolderBackup, make an image of the changes folder and transfer it to the new machines.

[dancytron]

Yes, the files in k-4.9.0-12-amd64.squashfs will have priority over the files that are inside "01-filesystem.squashfs" (which is lower in alphanumeric order), even if the files with the same path are newer in 01-filesystem.squashfs.

That makes sense.

I may have seen that in Buster on my laptop installing TV card kernel drivers and then remastering, but I just pasted them in manually.

edit: as far as the solution, I'd make them sort in the correct order rather than merge them into one file again. Either by just changing the name or maybe by changing the script so it knows which one the kernel one is no matter what it is named.

Thanks Dan. Well.. not sure yet, still thinking about it, in fact every extra module should be renamed then to 00-....squashfs (not only the kernel) as there's always a chance that the user has modified files that are also existing in one of the extra modules.

I always unload the extra squashfs files before I remaster.

There is some Deja vu, we've had this conversation before...

[fredx181]

There is some Deja vu, we've had this conversation before...

I think I remember it was about something similar yes, it's a bit of a struggle to make quick-remaster to work with all possibilities.
For now I like to keep it simple as is (could make it more advanced but perhaps then it's not "quick' anymore)
I think for next version I add only an info message at some point (since the kernel is for newer DebianDogs standard a separate module):

2021-06-11-195645_652x169_scrot.png (25.7 KiB) Viewed 2312 times

EDIT: Added new quick-remaster with above change to all repos .

[dancytron]

Is there any reason not to change the k-4.9.0-12-amd64.squashfs to 00-k-4.9.0-12-amd64.squashfs all the time?

I'm trying to think of a case where you would want it on top of the 01-filesystem.squashfs and I can't.

FWIW, I've been using the quick remaster on my Stretch version and it's been bulletproof since the beginning.

[dcung]

Hi Fred,
I've used the updated quick-remaster v1.2.6 on StretchDog - one that displayed the note at the end of remastering process. Worked as expected.

However, the new one in FossaDog, displayed the note...that's it. v1.2.7 - Does not do anything. It just exited the script after displaying the note.

2021-06-14-163937_652x184_scrot.png (29.43 KiB) Viewed 2263 times

[fredx181]

However, the new one in FossaDog, displayed the note...that's it. v1.2.7 - Does not do anything. It just exited the script after displaying the note

Yes, thanks for reporting, I found that I terribly messed up the script (by wrong copy/paste) , fixed now, re-uploaded v1.2.7 for Fossa
(apt update and reinstall quick-remaster should do, I guess)

Is there any reason not to change the k-4.9.0-12-amd64.squashfs to 00-k-4.9.0-12-amd64.squashfs all the time?

I'm trying to think of a case where you would want it on top of the 01-filesystem.squashfs and I can't.

Yes, I think too, could do no harm to name the kernel squashfs always to 00-... in the future

FWIW, I've been using the quick remaster on my Stretch version and it's been bulletproof since the beginning.

I've used it a lot too and always worked fine, but that was without installing any e.g. extra drivers.

Fred

[dcung]

Yes, thanks for reporting, I found that I terribly messed up the script (by wrong copy/paste) , fixed now, re-uploaded v1.2.7 for Fossa
(apt update and reinstall quick-remaster should do, I guess)

All good now.

[dancytron]

Just used the new Quick Remaster on my Debian Dog Stretch, (systemd enabled, overlay squashloader installed, all squash files unloaded) and it seems to have worked fine in all respects.