What is the difference between profile and permission set in salesforce?

The difference between profile and permission set in salesforce is - Anything you can do w permission sets you can do w profiles, but there are a few things that can as of now only be done w profiles. The key UI difference in my experience is that only profiles can be used to control page layout assignment. I took a look and at the top level, it looks like the following items are profiles only. There may be others, I have not been able to find a clean list

Marketing user checkbox
Page layout assignment
Desktop client access
Login Hours
Login IP Ranges
Session settings
Password policies
Delegated authentication
Two-factor authentication with Single Sign on

Organisation-Wide Email Addresses are assignable by Profile only (idea to fix this) Default record type per object

    Profile specific search layouts (winter 20)

Permission sets and profiles have come very close to parity. Now that record type assignment can be done in either, it's just a few key things that remain, and these likely would remain on profile, as a user can only be assigned one page layout at a time, so it wouldn't really make sense on a permission set. FWIW, besides for the page layouts, which I knew about from general experience, I came up with this list by looking at a perm set and a profile in the enhanced editor and noted which top-level sections were missing from the perm set.

Permission Set
Profile