I would say the current flow of comment flagging ("needs moderator intervention" reason) is sufficient for dealing with users you think "followed a post not because they're interested in it, but because they are interested in destroying it". Can't agree more with Temani Afif's comment:
flag any innaproriate behavior, as simple as that. We should not implement a complex feature to fight something we are fighting since too long with other tools – Temani Afif
The diagram at the end of the post should show the decision-making process for comment flagging is clear and has tools to handle all cases (suggestions welcome in case I misunderstood or missed something). There is no need to introduce a system with the potential for being abused to replace what already exists.
Addressing your concern about the flagging system:
Flags won't necessarily work, because the user isn't technically breaking any rules. The objective isn't to have him banned, just to stop him from polluting the specific thread. – user000001
What you described is a good case where a moderator needs to step in, weigh the problem, have a talk with the offending user (or explain to you they don't think this was an abuse) to prevent them from doing this to others (no need for suspension the first time, but a warning might be warranted) and clear out the comments and/or lock the post should they deem this necessary.
By allowing users to "force unfollow" anyone who commented from their posts, we open a vast potential for abuse. Apart from concerns others brought up, editors (I speak from experience here) are often perceived as unwanted intruders by OPs, and the feature will only make their job harder.
or some (maybe 3) users should be allowed to force-unfollow the user
Granted, ensuring peer review reduces the possibility of abuse, but at the same time makes the feature duplicate the comment flagging even more since multiple comment flags from users greatly increase the chances of a moderator to step in.