What's the difference between accounts like this and normal accounts?
The primary difference is that an unregistered account doesn't have login credentials associated with it. This effectively means that we don't know who you are (well, outside of having your IP address and a name/email combo you provided) and your profile may be restricted from certain actions.
On Stack Overflow (and a few other sites in the network), you have to log in (i.e. have a "registered" profile) before you can ask questions. Answering doesn't have the same limitation.
If it has a persistent identity that other people can't just claim, in what sense isn't it registered?
See above: no login credentials. Usernames aren't unique on our sites, but nobody can have the same user profile id, so in essence, you claim that with your profile, registered or otherwise.
How does such an account get created?

How is it authenticated? If you log out, are you locked out forever?
It is cookie-based. When you log out, we remind you to register (and there are a couple other reminders), but if you go ahead and do it anyway... you can go through the account recovery process and potentially a profile merge (I'm currently investigating if we have a bug with unregistered profile recovery) in order to regain access to your unregistered profile. After account recovery, the profile will be registered to prevent another loss of access.
iamdranged