If you type the hashtag on a single line in your About Me section it will render as a html header. Putting a space before the hash makes the symbol appear.
You don't want this
#This is what you want
#You don't want this
#This is what you want
And for why Stack Exchange chose to use hashtags in your About Me, I don't know. It's a social media thing.
#DealWithIt
On a more serious note:
Hashtags are just temporary social media "tags", unique identifier to help filter out and categorize content. SE can easily find twitter messages using the provided hashtag, so they need something similar for SO profiles.
The pattern of a twitter hashtag is unique enough to not have already been used - so profiles containing it are easily searchable for the devs.
I mean, they could have used something else for SO profiles, but you might as well keep it uniform. This is, after all, a social media campaign.
How about we add a predefined guid to our profile! That's a programmer thing to do.
10 million questions! {434a403b-073d-458c-854c-0e5f89e9a758}
#
then the#
shows correctly