In short, NO, not without a stronger consensus and much clearer guidance.
Question votes do not depend on the relevance or quality of the answers, nor should answer votes depend upon the relevance or quality of the question.
Once an answer is provided if it is good and downvoted the signals are confusing, to say the least.
However, “the stick” says stamp on questioners polluting the site with off-topic questions, “the carrot” says in the long term co-operation is better where a desire to help, and appreciation for help, comes across clearly.
Either may be appropriate but if downvoting an answer because the question is deemed off topic (bearing in mind there are grey areas where opinions may differ) there is all the more reason to explain such a downvote.
I have both answered a question and downvoted it in my time and admit a degree of inconsistency. A factor I take into account is the reputation of the OP. The more rep the more the chances that I will “bend the rules” to try and help (despite the theory that those Q posters should know better - they may be outside their area of expertise and/or be showing no sign of repeat offending).
Similarly for those posting a question for the first time, specially if I can somewhere slip in mention that, for SO, there is a better way (eg a different SE site).
For those who repeatedly ask off topic questions I think downvoting of answers should be considered, but do not do so myself since, as I understand it at present, this is not the approved approach, though as I understand your question, it asks what the approved approach is. We know active meta participants are a very small proportion of the total number of users with the right to downvote so without something much clearer than we have at present I think we must rely on “This answer is not useful” for guidance. In my opinion a good answer just does not fit within such guidance.
There is also that the system is geared up to incentivise “good housekeeping” and even if only temporary (off topic questions will presumably eventually be deleted and any -1’s then reimbursed) for a while downvoting answers for the sake of discouraging off topic questions is “back to front” (penalises those doing the housekeeping) – and also eats into the daily vote allowance.
From a personal perspective, I already have downvotes rather out of proportion to upvotes (2:1) and would prefer not to increase that ratio, even though most of the downvotes were for off-topic questions.
It seems many OP’s do not care about votes (they just want a solution to their problem) yet -1's to questions may be more effective in discouraging unwanted behaviour (eg question bans) than votes against their answers. As far as I can tell, others are generally reluctant to do so however (the average for the most recently closed 50 questions is between -3 and -2).
A compromise for the time being my be simply do not to upvote the answers in such circumstances.
Question can’t be answered while closed, so any answer must have been posted before “due process” is complete. Many times I have seen questions closed that clearly should not have been, at least for the reason stated. Answerers as well as questioners may be new to the system and not yet have read up on swathes of meta to fully appreciate the nuances of what is and what is not on topic. A downvote for their well-intentioned effort could be very discouraging.
By voting to close you're by definition trying to prevent others from answering
- yes exactly. Close it, prevent further answers.