I know that Firefox has options on the lines of "set as default"
I assume you are talking about using the Default Apps Manager
If under selecting a default browser.
If the drop down selection list shows Firefox portable.
You should be able to select to use it.
To give more info need to know specifically which Firefox portable you use.
Exactly where you have it located in the operating file system.
Setting the default browser just tells anything that is coded to use the default browser will use this one.
Programs that need to use a browser have commands that say use default browser.
This keeps them from needing to specifically say use a specific named browser.
The program does not care what browser, just a default browser to use.
The things you do not tell us, are usually the clue to fixing the problem.
When I was a kid, I wanted to be older.
This is not what I expected 