You'll want to read this for Private Mode:

https://support.mozilla.org/en-US/kb/private-browsing-use-firefox-without-history

Use the regular Firefox icon to visit sites in windows that can store history, cookies, etc.

I have been using the regular Firefox icon to browse, but it still goes to private browsing.

Could you double-check that Firefox is not set to force private windows:

On the Settings page, Privacy & Security panel, History section, there's a general selector:

• Firefox will: Remember history => should be okay
• Firefox will: Never remember history => prevents restoring your session
• Firefox will: Use custom settings for history => need to check the details

With custom settings:

[_] Always use private browsing mode => must be un-checked [X] Remember browsing and download history => must be checked [?] Clear history when Firefox closes => use the Settings button to check what is being cleared and make sure that "Browsing & download history" and "Cookies" are NOT set to be cleared