Skip to main content
3 of 3
deleted 18 characters in body
duplode
  • 34.3k
  • 9
  • 64
  • 108

I see this scenario as being similar to the one discussed in Are answers which merely summarize other answers acceptable? In fact, it can be argued that a benchmark compilation like the one given as an example here goes a little further than just providing a summary. A post of the form "given the evidence below, the solution from answer X should be preferred if you care about performance under conditions Y" does offer a solution to the reader, so the "not an answer" flag generally won't be applicable. Beyond that, it is a matter of whether the answer is substantial enough:

  • Is there enough content to justify a new answer? For instance, if the comparison is between just two answers to a recently posted question that have starkly different asymptotic behaviour, pointing out the difference in comments, which in turn might be acted upon by the answer authors, will likely work better than posting the comparison separately.

  • Does the answer cover why performance varies across the solutions, as opposed to just showing the benchmarks without comment? That makes it more likely for the answer to pay its weight, and also might help future-proofing it against language or environment changes.

  • Is the benchmark measuring something meaningful? This squares with VLAZ's concerns about benchmark quality, though I don't think that is a reason to rule out benchmark answers in general. Benchmarks can be done well or poorly, just like any other piece of code in an answer.

  • Is the benchmark relevant to the question? On the one hand, there are situations in which a performance comparison is expected to be of interest to most readers even if not explicitly asked for in the question (say, if the question is about data structure manipulation algorithms or numerical methods). On the other hand, there are also plenty of cases in which caring about benchmarks would be pointless micro-optimisation.

While it is plausible that a significant share of those benchmark answers would fail some of the criteria above (as VLAZ notes in the specific case of JavaScript Q&As), I don't see that as a reason for a blanket ban. The answers should be evaluated on their own merits, within the context of the Q&A they belong to, and, if they happen to be insubstantial, downvoted and deleted as appropriate.

duplode
  • 34.3k
  • 9
  • 64
  • 108