Just to be clear, flagging these "code-only" answers is not appropriate.
You can find detailed guidance here on when "not an answer" flags are appropriate (required reading for all reviewers), but the summary is that they're only for posts that don't even attempt to provide an answer to the question. They are not for incorrect answers, or even low-quality or incomplete answers.
The "very low quality" flag seems very tempting, I know. I would have to agree that these types of answers are indeed "very low quality". But please don't flag them with that reason, because what that reason really means is "this answer is garbage and needs to be deleted". A more accurate name for the VLQ flag would be "needs to be deleted", or possibly "unsalvageable".
Okay, so we're agreed that these answers shouldn't be flagged. We're also agreed that they're not good answers. What to do?
One obvious option, and one that you mentioned in the question, is downvoting the answer. A downvote on an answer just indicates to future viewers that the answer is not helpful. There are many different ways that an answer can be not helpful, including technical incorrectness, being incomplete, being unclear, and/or containing merely a dump of code without adequate explanation/background. So, as others (including Servy) have said, downvoting these types of answers is definitely appropriate. Use your best judgment.
There is an even better option, if you are knowledgeable about the subject matter discussed in the answer, and that's editing the answer to improve it. I do this a lot. That this site is collaboratively edited is one of its best features. A lot of people forget (or don't know) how much the Stack Exchange model is inspired by Wikipedia. The golden rule for editing is just that you're supposed to respect the author's intent, but expanding an answer to be more descriptive and more complete is undoubtedly consistent with that. It is not reasonable to assume that an author intends an answer to be unhelpful. If you can make the answer better, then please consider doing so.
If you can't edit, and you don't feel qualified to judge the helpfulness of the answer, then just do nothing. Leave the answer for someone else to judge and/or improve. In a review situation, this would translate to "Skip". There's never any shame in skipping.
Another intermediate option would be commenting on the answer, providing advice/encouragement on how to improve the answer to meet our standards. Here's a sample of a "canned" comment that some of our more prolific reviewers use for "code-only" answers:
While this code snippet may solve the problem, including an explanation really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion. Please also try not to crowd your code with explanatory comments, as this reduces the readability of both the code and the explanations!
Markdown for copy/pasting:
While this code snippet may solve the problem,
**[including an explanation](//meta.stackexchange.com/q/114762)
really helps to improve the quality of your post**. Remember that you are
answering the question for readers in the future, and those people might
not know the reasons for your code suggestion. Please also try not to crowd
your code with explanatory comments, as this reduces the readability of both
the code and the explanations!
I guess down-vote might not be proper option (just because I think this is not useful to others)
No. A downvote is precisely there to indicate that you think that it's not useful to others. That's exactly what it's there for. If the OP personally thinks that the answer is useful, they can upvote it, as can anyone else who thinks that the answer is useful. Your vote is for you to reflect your opinion of the quality of the post. If you think it's not useful due to its lack of information (or any other reason), you should downvote it.