I recently encountered an answer that I thought was off-topic. A small discussion with the author (poster?) showed me that it was, in fact, a good answer.
The problem arose when I tried to upvote the answer, which I found myself unable to do as my downvote had been locked. As a beginner, I frequently find myself in such situations where I am eventually proven otherwise, especially when it comes to conceptual questions.
My proposed solution is to allow the member to retract their vote by manually stating why they have done so. The same as manually explaining why a question should be closed, but applied to vote-retraction instead.
If this statement were viewable to everyone (like the edit history), I feel that it could help other members dispel their own doubts regarding an answer, and (optimistically) learn something new.