Vote on comments if they add something useful to the post.
If you asked someone to elaborate their answer, and their comment explains it... by all means upvote.
Depending on circumstance, you may also want to edit the content from the comment into the post. Consider that the OP apparently chose not to do this.
People also vote for comments that they find funny. Technically we shouldn't do that, but comment upvotes don't grant rep, and therefore don't give extra privileges. The only thing to be gotten from upvoted comments are a few badges. So upvoting a funny comment usually isn't harmful.
Keep in mind that when a post has a lot of comments, it will only show the ones with the most upvotes. In this case, you may want to be careful with voting for fun comments, and make sure they don't drown the actually useful comments.