Note that thumbnail capturing is done using a private container that has the same features as a private window. You can disable the thumbnailer by going to about:config and creating browser.pagethumbnails.capturing_disabled (Boolean) with value true.

You can delete the thumbnails folder in the Firefox profile folder to remove already stored web page thumbnails. The thumbnails folder is in the same secondary profile location as used for the disk cache (Local Directory on about:profiles).

After digging a bit I found out that top sites favicons are not a privacy issue. They are fetched only ones when we first visit that site and stored in favicons.sqlite.

And for recent site screenshot, chosen solution to this question works.

Just a heads up that you will not be able to set a custom shortcut icon with the thumbnailer disabled.