Even moderators cannot see which deleted user caused your loss/gain of reputation.
E.g., as a modmoderator viewing your reputation page, the "User was removed" is not a link, and "learn more" is exactly the same as what you can see - a link to https://stackoverflow.com/help/user-was-removed
If you really wanted to, the only way tois to become a Stack Overflow staff member with adminadministrator privileges.