I have come across some questions recently where I saw that I up-voted either the question, an answer or both, like this one - Sort array by firstname (alphabetically) in Javascript, and out of curiosity I have wanted to know when I did so.
I think it would be nice if hovering over the up-vote or down-vote showed that information. It is possible to find out via the votes tab in one's profile but it's not that easy when there are many of them.