SAVE Sessions - how does this work?

[Clarity]

This topic area is created to address questions or issues related to Session Processing of FATDOG during boot, AND during its ShutdownReboot.

[Clarity]

I have been revisiting Session processing to better understand it from a single feature-element vantage point: "Multi-Sessions"

FD has always had a feature where one of its developers, astutely, saw the advantages of how sessions were done for DVDs and adapted that technology, universally; meaning that it works the same no matter if saving to DVD OR to system storage.

On DVDs, sessions changes are timestamped for each time one is created as a unique file. And upon boot, the FD is smart enough to assemble all files appropriately. The advantage to this approach gives the user the opportunity to eliminate any particular timestamp file , if needed.

This ability appears to be replicated when storing on other system storage devices.

I have used this ability for several years without issue.

Today
I decided to try to do a pristine boot using multi-session for saving, but with its parm having a twist. TWIST: Boot pristine modifying the boot stanza, directing it to a subfolder within a pre-existing folder. In this case for this pristine boot, the subfolder does NOT exist.

The boot stanza change for the Multi-session Menu option

Code: Select all

savefile=ram:multi:label:Persistence:Sessions/v901new

where the v901new folder does NOT exist at boot time.

Problem
The 'new' folder was NOT created at shutdown and the session files were saved in the existing parent folder.

The naming of the session files "suggest"that a portion of Sessions processing subsystem 'knows' that the files belong to a session subfolder ... seen in its file naming multi-v901new-2023-12..., but the subfolder is never created to house the session files.

Additional findings
The subfolder is ignored by ALL current test. This fact exists no matter if the subfolder exist, or not, before the pristine attempt is accomplished. THUS, in all tests, the session files are ONLY landing in the Session directory's root and NEVER in the subfolder indicated in the savefile parm

Is this a bug? This account suggest this is a BUG in FATDOG savefile processing for this feature use.

P.S. I was sure, somewhere in my past, I remember that this was touted as a feature where if a non-existent folder was specified at boot via the parm, FD would create the specified folder for session saves. Are my memory banks failing me?

Lastly, although I have also proven that this point here in this post that I make this statement, "it does NOT MATTER that I am launching FD from its ISO file. Everyone, by now, knows that FD behaves exactly the same whether it is launched or is a traditional frugal: These are the same understanding and operation for FDs.

[Clarity]

• Is there something broken in Savefile processing?
  OR

• Is there something wrong in how the savefile is to be structured?

[fatdoguser]

Code: Select all

savefile=ram:multi:label:Persistence:Sessions/v901new

Shouldn't the :Sessions be prefixed with a forward slash

Code: Select all

savefile=ram:multi:label:Persistence:/Sessions/v901new

I use multisession saves, but to create .sfs files on HDD instead and they're working OK in Fatdog 901

[jamesbond]

@fatdoguser, the slash if implied if it is not given. But it's better to be explicit, I agree.

@Clarity, the last part of the name is not a folder, it is the session-filename pattern. So if you put /Sessions/v901new, you're telling Fatdog to keep the sessions in /Sessions and name the session files as multi-v901new-2023-12... - exactly what you experienced.

If you want to keep it in the folder /Sessions/v901new, you should use the parameter /Sessions/v901new/abc so that /Sessions/v901new/ is taken as the folder name, and your session files will be called as multi-abc-2023-12...

[Clarity]

Hello @jamesbond
OK, thanks for the explanation.

BTW: For those who have used it, the Control Panel's Savefile utility will generate the parm I've used.

Question
Does this mean the Utility is in need of some review in light of these findings?

Statement
My reason for doing this is my attempting to isolate work-test done within the distro from other/production work. This is one of ways I foresee to isolate FD work efforts.

If anyone knows of other methods to isolate work efforts via parameter 'namings' that FD would respect, please share those/any other approaches.

Thanks in advance

[jamesbond]

BTW: For those who have used it, the Control Panel's Savefile utility will generate the parm I've used.

By "Control Panel's Savefile utility" I assume you mean Control Panel -> Install -> Savefile Argument Builder.

Question
Does this mean the Utility is in need of some review in light of these findings?

No, the utility is working as intended.

The instruction on the utility says: "3. Drag your savefile, savefolder, or SFS into this box"

This means, the savefile/savefolder/multisession SFS must __already__ exist for the utility to work. If you have not created the folder (as you said in your first post), then the utility will not work properly.

[Clarity]

Thanks. I thought I also tested that condition. I will retest when home.

Edit:
As I travel home as a passenger, this occurs to me:

• I have a folder named Sessions where all of my sessions for all forum distros are maintained.

• In the case of FD, I use the multi-session methods to boot-shutdown.

• I'm attempting to keep one 'set' of FD versions in separate subfolders

• And even as it is clearly explained I'm missing some thought as follows:

  • Should the save file describe the subfolder OR is it to describe one of the 2 created multisession filenames in the savefile parm?

  • Is there a recommendation of how to have FD create the subfolder name at pristine boot via some manner of use of the savefile parm ... when the subfolder is NOT created? This would be to allow the session to be saved within a newly created folder at shutdown.

HOpe this makes sense as I continue to frame clear understanding of SAVEFILE parm; no matter which of the 3 types of sessions contents that FD addresses; namely a savefile, a savefolder, a multisession

[jamesbond]

Should the save file describe the subfolder OR is it to describe one of the 2 created multisession filenames in the savefile parm?

Hope this makes sense as I continue to frame clear understanding of SAVEFILE parm; no matter which of the 3 types of sessions contents that FD addresses; namely a savefile, a savefolder, a multisession

It is impossible to make the savefile/savefolder to behave similarly with multisession save.
Because savefile/savefolder only requires exactly one parameter: the name of the savefile (or the savefolder), while multisession save requires two parameters: the location of the folder, and the name of SFS files to be created.

Is there a recommendation of how to have FD create the subfolder name at pristine boot via some manner of use of the savefile parm ... when the subfolder is NOT created? This would be to allow the session to be saved within a newly created folder at shutdown.

Exactly like what I said here. If the folder(s) don't exist they will be created.

[Clarity]

OK, this report is an update.

Scenario

• Intel PCs (1 is 2010 vhile another is 2020)

• Each HDMI connected for both audio and video requirements

• Bare-metal booting of USB launchers; Ventoy as well as SG2D

• ISO file used for tests is FD v901

• Sessions are maintained on each's system's drive partitioned with its label named "Persistence"

• On that partition, it has a folder where all sessions are kept name "Sessions"
  Note: These are independent PCs each area for its sessions are local to each PC.

Test Plan
Note: All tests are 2-step: Pristine boot to desktop and reboot immediately after session are saved from the pristine boot.

• boot each PC via the aforementioned USB

• launch FD v901 ISO file to its GRUB2 menu

• Navigate the menu to edit the 'multi-session' stanza

• On the Linux line, change the savefile parm to reflect new, non-presently existing subfolder(s) within the Sessions folder

• Navigate the desktop and the control panel with appropriate changes

• Save the session

• After any additional desktop reviews, reboot the system exactly as before changing the savefile parm as reflected

• Observe when desktop emerges

Findings (in no particular order)

• Normal processing operations

  • If the savefile parm's subfolder is NOT present at pristine boot, the subfolder is created and session is saved at

• Desktop utility processing

  • At anytime during desktop work, the session can be saved by clicking the "Save session" icon on the right side of the desktop.

    • If no prior folder/subfolder is present, the utility will create the folder and deposit the session files within that folder.

    • If folder is already present and empty new session files will be added.

    • If folder is already present and has prior session save within, a new timestamped session file will be added to its present contents.

• Control Panel's Savefile creator utility

  • The recommended savefile parm must have an ending "/" after the subfolder's name for successful operations and subolder creation to occur on a pristine/follow-up boot.
    For example
    savefile=ram:multi:label:Persistence:Sessions/Fatdog/v901 will NOT direct/create the session within subfolder 'v901'
    BUT
    savefile=ram:multi:label:Persistence:Sessions/Fatdog/v901/ will direct/create the session within subfolder 'v901' as is desired. Note: "v901/" will work while "v901" will not.

  • The wording of the utility does not totally match findings. If I knew how to edit it to match these findings, I would and present it to development.
    Note: This is not critical as it does not break apps or system operations.

.
Conclusion

• The savefile parm MUST be set properly for folder/subfolder session finding during boot as well as saving upon shutdowns is directed properly for the session's files.

If there are operational questions, feel free to shoot them my way. I will do what I can to answer.

[fatdoguser]

Fatdog multisession saves are excellent IMO

I keep a extracted copy of fd64.sfs alongside the fd64.sfs

Code: Select all

unsquashfs fd64.sfs

that creates a squashfs-root of the contents).

Mostly I boot, use, shutdown my configured fd64.sfs, without saving any changes (all data is stored elsewhere). Periodically when I want to reconfigure things I either directly edit the contents of the squashfs-root folder tree, or run a 'save' that creates a new multi-session....sfs file, that I then extract into the squashfs-root folder

Code: Select all

unsquashfs -f -d squashfs-root multi....sfs

and then remove that multi-session file(s), and create a new fd64.sfs

Code: Select all

mv fd64.sfs del.me
mksquashfs squashfs-root fd64.sfs

I may leave the multisession sfs's as is for a while, across several reboots/days, just to make the changes are OK, so easily undone (by deleting the multiesession...sfs file(s) if the changes induce a more obscure issue/bug, but otherwise don't really have any multi-session files being stored/loaded, its all in the main fd64.sfs.

For sfs compression I prefer lz4 as its very quick to both create the new fd64.sfs and quick to decompress, but forfeits disk space (which nowadays is inexpensive). On a SDD with lz4 compressed sfs its near as quick as reading from ram, so no need to copy the fd64.sfs into ram at bootup.

But that's just one of the ways Fatdog can save changes, its very flexible, a plethora of possible choices. Part of that flexibility induces elements of complexity in the boot parameters, can be awkward to figure out the precise syntax required, such as whether a trailing slash is required or not. But that's a figure once, forget/use many process.

I mostly use Fatdog as my Linux based servers and as a build tool. On the server side its very quick/easy to keep Chrome updated to the latest version, just a matter of downloading the Debian RPM version of chrome, extracting that and copying the extracted chrome subfolder to replace the existing /opt/google/chrome/chrome folder. I tend to leave the servers kernel version as-is, but on clients keep them up to date with the latest kernel point release (I'm presently tracking 6.6.8 series for that https://kernel.org/ .. no aufs just the direct as-is kernel as I don't load/unload sfs's at the client side, only at the server side.

[Clarity]

This is a offer to @fatdog for an update to the SAVEFILE Argument Builder for novices to better reflect savefile options.
I think it might be a little clearer and helpful.

Fix Savefile Atgument Builder.jpg (67.63 KiB) Viewed 462 times

[Clarity]

Awhile ago, I approached this forum to understand if FD v901 could handle multiple sessions saved from past v901 multiple pristine times.

The responses to help indicated that it appears there is present code that during INIT of FD it has the intelligence present to pause and list them when multiple sessions are there in the save folder.

I am reporting my experience:

1. I move several past sessions into a common folder

2. I use FD's Savefile Arg builder to validate the savefile I will use to boot FD

3. I launch the FD v901

4. At FD's boot menu, I select the 'Multimedia' option (all of my sessions were multimedia sessions)

5. Editing that stanza's linux line, I add savefile=ram:multi:sda3:Sessions/v901

6. Hit the enter key, the boot is on-its-way

7. I was hoping to have the boot, at some point, pause with a list for me to select which past session to use in this boot

8. It does not pause going directly to desktop using 1 of the 3 sessions in that folder.

@fatdog rarely, if ever, has code issues, so I suspect that this is a user problem.

Ideas?

My FD Session folder

FATDOG Session folder.png (26.15 KiB) Viewed 453 times

[fatdoguser]

@Clarity The relevant code in your case is from line 1012 in init

Code: Select all

### cmdline processing - savefile - load session from multisession device
# $1-device $2-mountpoint, $3-savefile path, $4-don't load loading the last n session
load_session() {
	local spath sname depth sname max_session
	spath=${3%/*}; sname=${3##*/}; sname=${MULTI_PREFIX}${sname%.*}
	[ "$spath" = "$3" ] && spath=	# special case - no path specified

depth=1 && [ $SEARCH_DEPTH -ne 0 ] && depth=$SEARCH_DEPTH	# how deep to look for files
mkdir -p /mnt/tmp /mnt/dvdtmp
if mount /dev/"$1" /mnt/dvdtmp; then
	# count how many sessions there are, and reduce that by the number of sessions to skip
	max_session=$(find /mnt/dvdtmp/$spath -maxdepth $depth -name "$sname*" | wc -l)
	[ "$4" ] && max_session=$(( $max_session - $4 )) # reduce by number of sessions to skip
	
	while read -r p; do
		if [ "$p" ];then
			echo -n "Loading session ${p#/mnt/dvdtmp/} ... "
			if mount -o loop "$p" /mnt/tmp; then
				cp -af /mnt/tmp/* /mnt/tmp/.[^.]* /mnt/tmp/..?* $2
				umount -d /mnt/tmp
				
				# cleanup deleted files and whiteouts
				# do it here in the loop --- slower, but we also ensure we deleted big files
				# earlier so they don't take up valuable RAM space
				find $2 -name ".wh.*" | while read -r pp; do
					fname="${pp%.wh.*}${pp#*.wh.}"	# without the .wh. part
					if [ -e "$fname" ]; then		# if both file and whiteout exist, then ...
						if [ "$pp" -nt "$fname" ]; then # delete the older one
							rm -rf "$fname" > /dev/null	# delete the file if older
						else
							rm -rf "$pp" > /dev/null	# otherwise delete the whiteout
						fi
					fi
				done
				echo "done."
				
			else
				echo "failed."
			fi
			decz max_session && break
		fi
	done << EOF
	$(find /mnt/dvdtmp/$spath -maxdepth $depth -name "$sname*" | sort)
EOF
	fi
	umount /mnt/dvdtmp	
}

That 5th line from the end, the 'find' case, pipes into sort. Each multisession save file includes a timestamp format filename so basically it finds/loads each multisession save file in timestamp sorted order. There is a option that can be set to exclude the last n files being loaded but otherwise all multisession files will be loaded in the timestamp (filename) sorted order.

Which in your folders case, means multiple different sessions all getting loaded. The Fatdog code as-is does not include providing a choice option to load one particular multisession base + associated save sfs files out of a set of base + save sfs's files (different sessions).

For separate boots/sessions you need to keep the multisession base.sfs and its associated save.sfs's for each separate session ... separate. Different folders and specify the relevant folder when booting. Fatdog as-is wont look for and present choices for such different sets, just looks/uses the multisession files in the folder it is pointed at.

At least that is the way I see (read) it. May be wrong, but I don't think I am.

Unfortunately as-is that doesn't fit with your particular usage/requirement case, so instead using save file/folder would seem more fitting for you, where Fatdog will prompt for selection/choice when there are multiples, as per our postings a week or so ago viewtopic.php?p=112930#p112930

Mixed sessions base + save sfs's do not have a unique marker for each individual session, so clustered together into the same folder as you have there's no way to distinguish a clear individual session grouping.

I suspect if you've been running those sessions and adding saves that likely they're corrupted in the sense that they include a mixed bag of file changes/additions/deletions form across multiple different sessions. Best would be to boot and separately save outside of Fatdog what data you can, and then start afresh (or better still in your case, use a save file/folder style choice).

[fatdoguser]

I note that from fatdog help

• - “device” and “uuid” are subcommands (multi:dev:path:skip is equivalent to multi:device:dev:path:skip).
• - “dev” is device where to keep the multisession savefiles to. If not specified, this will be sr0, presumably the first CD/DVD drive on your computer. If this is not the case, you need to set it to the correct device.
• - “dev-uuid” is the device's uuid.
• - “path” is the path to the savefile including the filename. Path is optional, if you don't specify it, the default of /fd64save.ext4 will be used. Note that in multisession mode, only the basename (without extension) is used, and Fatdog64 will prepend “multi” prefix and append the current date and time of when the session is saved, for example for the default filename of /fd64save.ext4, the actual session savefile will be /multi-fd64save-2012-04-23T12-54+1000-save.sfs - with the date and timestamp will be changed with every save.

Not sure, but I think that still wouldn't be within your requirement as you'd have to specifically specify the basename as part of the kernel boot parameter.

[Clarity]

Tomorrow, I will repeat my efforts by creating save-folders instead of using multi (DVD) sessions as saves.

Thanks for the efforts to demonstrate proper use scenarios for having multiple sessions per activity needs and tests.

As my filelist shows, that folder not only contains multiple multi-session saves, but also 2 folders each having multi-session saves, each.

My Plan:

1. empty the folder and start fresh

2. within the v901 folder, I will save sessions starting each time from a pristine base. When booting each pristine, I will use the savefile parm to guide where sessions are to be maintained.

3. The parm savefile=ram:device:label:Persistence:Sessions/v901 will be used each time I exercise a pristine boot to create a particular session for a specific mission.

I will report findings here as others may have a need to know methods of maintaining specific and multiple sessions of FDs in a common location.

I seek to keep all sessions in a common folder. I will always direct FD menu at boot time with the savefile parm. And hope that when multiple FD saves are present, FD will pause at boot to allow selection of one of the specific session of choice for boot.

Thanks, again, @fatdog for this assistance.

[Clarity]

Before retiring, I decided to do an initial run with the fore-mentioned instruction:

1. launched the ISO file to FD's boot menu

2. chose the RAM stanzal to edit

3. added to the linux line savefile=ram:device:label:Persistence:Sessions/v901

4. FD proceeded to desktop

5. At desktop opend the Control Panel to for some initial tailoring (locale, keyboard, desktop, and finally hostname change)

6. After desktop restarted I checked a folder's contents and

7. reboot requested via Menu>Reboot

8. On the reboot screens, I HAD TO REENTER all the parms as was entered at boot time via the savefile parm...again

9. After navigating all of the screens of rebooting, I ended here

   

   FD session saving.png (47.28 KiB) Viewed 417 times

• Pushing the button to reboot, "NOTHING HAPPENED"
  repeating efforts and nothing happened.

• Exited desktop to console prompt and entering reboot command ... same thing "nothing happened"
  entered "poweroff now" ... nothing!

• tap the PC power button to see if system would detect and move to power off....nothing happened.

• Forced poweroff holding down power button.

This is the first time, ever for me, where FD locked a PC.

[Clarity]

Today, starting from a 'pristine' boot from v901 ISO file, I get the following for all selected packages when doing a system update via the FD Control Panel>Install>FD System Updater.

I am using the steps in the above post, completing all steps toward a 'clean' Shutdown without a system HANG as the above shows.

BUT, I stopped with this: Updater PM results

Code: Select all

+==============================================================================
| Upgrading openssl-3.1.1-x86_64-1 package using /var/slapt-get/./openssl-3.1.1-x86_64-1.txz
+==============================================================================

Pre-installing package openssl-3.1.1-x86_64-1...
sed: error while loading shared libraries: libacl.so.1: cannot open shared object file: No such file or directory
sed: error while loading shared libraries: libacl.so.1: cannot open shared object file: No such file or directory
cat: /var/log/setup/tmp/tmplist17987: No such file or directory
ERROR:  Package /var/slapt-get/./openssl-3.1.1-x86_64-1.txz did not install
correctly.  You may need to reinstall your old package
to avoid problems.  Make sure the new package is notFurther
corrupted.

my

???

I suspect that a forthcoming release may be the culprit. Should I wait for the update release? My plan is to use @Gobbi's method for adding the nVidia driver to the FD system for its only video adapter.

Further I would like to determine, once the driver is installed, if it shows up in FD Control Panel>Display>Choose Graphics Driver to see if the installed driver will be listed by the FD Control panel utility.

FD Control Panel.jpg (25.63 KiB) Viewed 322 times

HOpe this is useful of planned testing on the bare-metal PC whose ONLY video is HDMI to a 4K TV from the PC's only video adapter.