I'm calling this the Kazoo problem, and it's a fundamental problem that leaving text boxes around the Internet creates.
To illustrate this, I'm calling the following thing a magic trumpet of knowledge. You should blow on it only when you have something amazing to bring into this realm, otherwise you waste the magic essence that follows and surrounds it in light:
Now the rest of the Internet comes along and says ...
Hey look! A kazoo full of glitter!!!! Row, row, row your boat, gently down the stream!!! Ooooh colors, look at all the colooors!!!!
Clearly, we've yet to find a way to differentiate what we call 'comments' from everything else on the Internet called 'comments'. At the same time, we've yet to wrap our head around a design for better moderation tools to manage them.
The fix for this is to fix what's really wrong, hiding them as you suggest is just hiding the larger problem, and doesn't fit at all into the very public & transparent nature of the site.