How did you set Firefox never to remember history - was that in Options > Privacy? Did you clear history? If not, perhaps you are just seeing old history. If you think your settings are being ignored, see this page:

• http://kb.mozillazine.org/Preferences_not_saved

Please Update to the latest Firefox 19.0.x version.

• Help > About Firefox

• https://support.mozilla.org/kb/update-firefox-latest-version
• https://support.mozilla.com/kb/Updating+Firefox

The Firefox Firefox 14.0.1 version that you currently run is no longer supported with security updates

Do you mean that you see pages that you visit appearing in the History?

Does this also happen if you enter PB mode via the Firefox menu or Tool menu?

Start Firefox in Safe Mode to check if one of the extensions (Firefox/Tools > Add-ons > Extensions) or if hardware acceleration is causing the problem (switch to the DEFAULT theme: Firefox/Tools > Add-ons > Appearance).

• Do NOT click the Reset button on the Safe mode start window or otherwise make changes.

• https://support.mozilla.org/kb/Safe+Mode
• https://support.mozilla.org/kb/Troubleshooting+extensions+and+themes

This has been happening to me for years. I don't think Mozilla really has a solution. I've now decided to use Chrome. They've improved a lot.