Lately, I have made an habit out of reacting to clearly inappropriate close votes that I happen to run into around Stack Overflow with a comment to the question that clearly states my disagreement, in the spirit of Shog9's answer to Directly vote to not close a question. While that seems to be a reasonably effective strategy to prevent further bad close votes, it might be even more effective -- and perhaps even instructive -- if the close voters had the opportunity to read my objections and, should they agree, retract their votes. That being so, I propose there should be a special-purpose mechanism for notifying close voters that their votes are being questioned. In what follows, I will specify some constraints that would make such a feature work in a way that is reasonable for everyone involved.
In principle, casting a close vote is at least as engaged a form of participation in a question as leaving a comment, and so if the latter makes one a potential target of notifications it should be no different for the former. However, if anyone was free to send notifications to close voters under any circumstances, there would be legitimate concerns about the volume of notifications thus submitted, specially towards those doing shifts at the review queues (concerns of the sort raised against Ping close voters on closed question body edit). I believe such concerns would be adequately dealt with by adding four restrictions to my proposed notification pathway:
Notifying close voters should be necessarily accompanied by a comment (this should discourage drive-by notifications and mindless gut-feeling misuse of the feature).
Notifying close voters should only be an option for questions that are still open (after all, we already have the reopen queue for closed questions).
It should only be possible to notify close voters once during the lifetime of the question (this should largely prevent anyone's inbox from being flooded).
The author of the question should not be able to notify close voters (the system is not meant as a means to defend your question, but as another form of peer review of close votes -- in other words, as an out-of-band complement to the review queues.)
Lastly, I have no strong opinions about the UI of this feature. The obvious solutions appear to be either an extra option in the close vote dialog or some sort of variation on the "@" comment notification syntax, but if these are problematic any more sensible suggestions would be good enough.