You didn't really "lose three fans" so much as three accounts were removed. Sock-puppet accounts. Unfortunately, these users hit a lot of folks - their vote percentages back to the puppet-master were only about 50-60% each, which means there were also quite a lot of votes to other users not related to that person.
In most cases, these users also voted for the same posts of those other users that did end up receiving their votes. Some users saw losses of much more reputation than the 45 you lost. It sucks, yes, but the votes were also fraudulent. You were receiving multiple votes on posts from one user who was accessing the site from multiple accounts, and the reputation changes are fair to the community in that they fix something that really shouldn't have happened.
The achievements menu still groups the net gain/loss from a user removal by user, which means each user will appear as a separate entry. The reputation history used to do the same thing, but was recently changed to just roll them all up into one "user was removed" event which no longer indicates how many users were actually involved, only how much reputation was lost due to that reason (although I'm not personally aware if that was an intentional change or not).