21

I've often seen users post answers that don't bring a new solution to the question asked, but simply calculate the time performance of other answers.

Answers like:

I ran multiple iterations of all other and answers and found that SomeUser's answer was the fastest

>>> %timeit def solutionUser1()
10000 loops, best of 3: 29.6 µs per loop

>>> %timeit def solutionUser2()
10000 loops, best of 3: 37.8 µs per loop

Here is an example: How can I check for NaN values?

I am of the opinion that they should be flagged as "Not an answer", especially when OP was not asking about the most optimal solution in terms of time performance.

15
  • 49
    Answers are not just for the OP. Comparing solutions is a useful bit of information for any problem, and almost one hundred people certainly found it useful to them. Commented Apr 1, 2023 at 13:58
  • 2
    @StoryTeller-UnslanderMonica or have been lied to by them. Answer arguing the case upcoming.
    – VLAZ
    Commented Apr 1, 2023 at 14:00
  • 1
    I urge anyboddy who has arguments for benchmark answers to write an answer laying out the reasons. Let's hope there is something more than "the answers clearly mislead many into thinking the results are reliable and useful, thus, because they were a convincing lie, we should leave them be"
    – VLAZ
    Commented Apr 1, 2023 at 14:50
  • 4
    A lot of those "benchmark answers" often contain a new and sometimes better answer (eg: provide new code/better solution/answer, either in term of speed, etc) rather than just comparing existing answers. As others better said, even if it was just comparing existing answers without providing a newer one, it still gives invaluable insight in the speed, performance, and inner working of those answers. Commented Apr 1, 2023 at 18:01
  • 3
    @NordineLotfi this scenario is addressed in the first line of the post. i'm specifically targeting answers that benchmark other answers, without adding their own Commented Apr 1, 2023 at 18:50
  • right, I failed to see that earlier, my bad :) @NicolasGervais I want to add that even when some do not add their own solution, they still bring valuable insight (for some, not all of them). Sometimes they also prove that it isn't always the fastest (eg: when using a large input, etc) Commented Apr 1, 2023 at 18:54
  • 4
    Answers composed only of benchmarking other answers seems to pretty clearly fit in the *This is commentary on another post, not an answer" bucket of reasons to delete. If someone considers detailed performance information on a particular approach to be useful, it could be edited into the answer that uses that approach. Posting a new answer which is essentially a collection of comments on other answers wouldn't be appropriate. Commented Apr 1, 2023 at 19:17
  • 3
    The linked answer doesn’t seem exemplary to what you are talking about. It has a thorough, reproducible benchmark that is broadly applicable and can be rerun on any setup that is expected to perform differently; it also provides a custom solution. How the scenario instead is presented in the question feels somewhere between a caricature and strawman. Commented Apr 2, 2023 at 15:35
  • 1
    If I could offer an example of a bad benchmark answer, check out this one on "How can I check for NaN values?" (Python). To summarize my comment there: "At this scale [nanoseconds], name and attribute lookup time are significant. If you only use local names, the difference ... disappears"; and I put a better comparison.
    – wjandrea
    Commented Apr 2, 2023 at 16:26
  • 3
    Raw benchmark data is of dubious value. Such answers must include the actual benchmark code in a form that can be easily executed. Your linked example does have code, but it relies on the timing facilities built into an ipython environment, so it cannot be run by a Python user without ipython.
    – PM 2Ring
    Commented Apr 2, 2023 at 17:08
  • 2
    This is a good question, but you thoroughly undermined its premise by linking to a question that explicitly asks
    – Ian Kemp
    Commented Apr 3, 2023 at 7:38
  • I edited the question to include an example that more closely exemplifies the type of answer I'm referring to. That would be a banchmarking answer in a question that did not ask for benchmarking in any way. Commented Apr 3, 2023 at 13:32
  • 1
    @NicolasGervais Why is that benchmarking answer worth explicit mention? It looks as if there there are a lot of questionable answers on that Q&A. There is a compilation answer that doesn't even add a benchmark. There is a demonstration answer for just a single other answer. There is a "what about that other thing" answer. There are several "throw arbitrary input at the solutions" answers. There are... questionable "solutions". Benchmarking seems like a lesser issue there. Commented Apr 3, 2023 at 14:35
  • @NicolasGervais Significantly altering your question after people here invested effort in responding to your original concern is not a good thing to do. Almost all of the comments and answers below were written before you moved the goalposts. At a minimum update your question again, to also reference the original SO question that bothered you. Whether the new question you link to is more suitable or not is beside the point, because it was not what prompted almost all of the comments and answers here.
    – skomisa
    Commented Apr 4, 2023 at 4:31
  • @skomisa It's merely an example of what I've described in the post. If other users decided to focus on this specific example, then they're the ones going off-topic. No matter how often I change the link, if it's an example of an answer that compares time performance of existing answers, without it being asked the asker, and without adding a new answer, the examples will be relevant and interchangeable. Note that no specific example prompted me to ask this question — I often see this kind of answers and don't save them. Commented Apr 5, 2023 at 13:04

5 Answers 5

42

I see this scenario as being similar to the one discussed in Are answers which merely summarize other answers acceptable? In fact, it can be argued that a benchmark compilation like the one given as an example here goes a little further than just providing a summary. A post of the form "given the evidence below, the solution from answer X should be preferred if you care about performance under conditions Y" does offer a solution to the reader, so the "not an answer" flag generally won't be applicable. Beyond that, it is a matter of whether the answer is substantial enough:

  • Is there enough content to justify a new answer? For instance, if the comparison is between just two answers to a recently posted question that have starkly different asymptotic behaviour, pointing out the difference in comments, which in turn might be acted upon by the answer authors, will likely work better than posting the comparison separately.

  • Does the answer cover why performance varies across the solutions, as opposed to just showing the benchmarks without comment? That makes it more likely for the answer to pay its weight, and also might help future-proofing it against language or environment changes.

  • Is the benchmark measuring something meaningful? This squares with VLAZ's concerns about benchmark quality, though I don't think that is a reason to rule out benchmark answers in general. Benchmarks can be done well or poorly, just like any other piece of code in an answer.

  • Is the benchmark relevant to the question? On the one hand, there are situations in which a performance comparison is expected to be of interest to most readers even if not explicitly asked for in the question (say, if the question is about data structure manipulation algorithms or numerical methods). On the other hand, there are also plenty of cases in which caring about benchmarks would be pointless micro-optimisation.

While it is plausible that a significant share of those benchmark answers would fail some of the criteria above (as VLAZ notes in the specific case of JavaScript Q&As), I don't see that as a reason for a blanket ban. The answers should be evaluated on their own merits, within the context of the Q&A they belong to, and, if they happen to be insubstantial, downvoted and deleted as appropriate.

13

I don't have a strong opinion on the question, but I do have several thoughts about the underlying topic.

"Time complexity" vs "performance"

The term "time complexity" is often misused - I edited it out of the title here because it fell into that misuse pattern. Not to get too prescriptivist, but it doesn't mean anything to do with the amount of time it takes for the program to run; it's specifically referring to how that time scales with the problem size (also sometimes referred to as big-O complexity, although there's a bit more pedantry there about best/average/worst cases).

On the other hand, while "performance" typically refers to runtime, it can be broader. In particular, it might refer to memory usage. However, optimizing a given implementation for memory usage will generally be either straightforward (choose smaller data types where they'll do the job) or infeasible (the language doesn't offer access to such types), and it's just generally not seen to be as interesting of a task.

Information should be as centralized as possible, but no more so

On one hand, we have way too many questions on Stack Overflow overall, which makes it harder to find canonicals; and one cause of this is the desire of many to recognize very fine distinctions and avoid closing questions as duplicates. I would argue that, for sufficiently simple and general values of X, "what's the fastest way to do X [in FooLang]?" should ordinarily not be considered a separate question from "how can I do X [in FooLang]?". This has the implication that, as long as

  • there are multiple sane, meaningfully distinct approaches to the problem;

  • they have meaningfully different performance characteristics (complexity and/or constant-factor differences, in speed and/or memory usage)

then a consideration of those performance characteristics is fair game.

Note here that when I say "approaches" I'm talking about either different algorithms, or fundamentally different ways to express an algorithm in the language. I'm expressly not talking about micro-optimizations like inlining function calls to avoid overhead, optimizing memory access patterns, etc. I am including things like "here is a good spot to apply the standard library's built-in memoization strategy". (Of course, there is some subjectivity in judging this.)

As long as we have answers for a given question that care about performance, then, it would be more useful to have one answer, perhaps CW, that does a proper apples-to-apples comparison.

I count 24 undeleted answers on How do I clone a list so that it doesn't change unexpectedly after assignment? (admittedly, I recently contributed to the total myself; that's because I wanted to point out a common XY problem and couldn't find a better way to give the necessary information - it takes too much explanation to just stuff a related question link into the comments or a question "see also" edit). Clearly, the problem here is not answers giving benchmarks; it's redundancy - re-explaining the same simple concepts multiple times, refactored across different answers in multiple ways, and showing the same pieces of code multiple times. If anything, a benchmark answer is especially valuable for a question like this because it can obviate many other answers - by just showing the one-liner code for each of multiple ways to do the task (along with labeling that way with its test result).

However, there are multiple answers that include benchmarks, and that is less than optimal. If I had my way, there would be a single answer on that question that discusses performance; and it would

  • Use a consistent timing methodology
  • Use a good timing methodology and be transparent about it (thankfully, in Python the standard library provides support for doing timing properly; so for a task this simple, we pretty much just need to use that)
  • Summarize relative timing results in a thoughtful manner (i.e., grouping approaches that are approximately equal in performance, and showing an approximate factor of speed-up/slow-down between groups, making some allowance for differences in hardware setups)
  • Repeat the test (on the same hardware) for each Python version where the groupings changed due to differences in implementation
  • Possibly consider alternate Python implementations (like PyPy)
  • Have a reasonable summary up top
  • Somehow, do all of that without being absurdly long.

But perhaps I want too many ponies. This already seems hard to do when the underlying programming task is extremely simple.

Micro-optimization should be all or nothing

If the problem involves actually implementing some algorithm and not just some simple expression like x.copy() or [*x] or x[:], there might be N sensible ways that could be identified, reasonably held to be distinct from one another, to micro-optimize the code.

If you feel that this justifies the existence of 2^N separate answers in the answer section, you are hereby officially uninvited from my board games night.

In my mind, at most two answers are justified per fundamentally different algorithm: the one that shows maximally readable, "elegant", FooLang-community-standard, "quality" code; and the one that does its utmost to shave CPU cycles or whatever (and explains and describes every individual thing it's doing in order to attain that goal). Unless there's a clear speed/memory tradeoff somewhere within that algorithm, in which case it might be necessary to wing it.

For copying a list, it makes sense to show every reasonable syntactic construct that could be used to copy a list. For an overall process that involves copying a list, however, we show a version of that process that copies the list in the intuitive x.copy() way (along with doing everything else similarly), and a version that copies the list in the fastest possible way. (This is eliding over the fact that x.copy() is supposed to be using the fastest known way, and if you find something noticeably faster then it's worth reporting on the CPython issue tracker. On the other hand, actually using that way directly - currently, that should be [*x] IIRC - should still help slightly because it avoids having to do the actual method lookup. But I digress.)

1
  • 5
    "Possibly consider alternate Python implementations (like PyPy)" I recommend against actually doing this. Microbenchmarks work well for CPython because it has a straightforward execution model. Involving any of the JIT-ed or compiled alternate implementations would make most benchmarks less meaningful. Commented Apr 2, 2023 at 15:40
-6

I am very much against these answers in the majority of cases.

My most frequent tags revolve around JavaScript and I can confidently say that I have not seen a single answer that collects measurements that is worth the bits it takes up in a database. Moreover, I expect almost all* such answers to be completely useless.

*because I do not want to simply say "all" and have somebody bring up a single example to prove the opposite. Perhaps the only example on the site.

Where the code runs makes a difference

This may sound harsh but for JavaScript things change all the time. What ran in 99ms today may run in 15ms tomorrow. Or 150ms the day after. All due to different updates to the engine prioritising or de-prioritising different operations. So, any measurement taken over a year ago is most likely not relevant at all today.

However, there is even more to it - I mentioned updates to an engine. But there are multiple JavaScript engines out there. What takes 99ms in Firefox, might take 15ms in Chrome or vice versa. In the same point in time. And an update to either or both can change these measurements drastically.

Benchmarks can be done wrong

Moving on, there are even more reasons why the measurements might be off. One really big one is that people who take them do not know what they are doing. I have seen many benchmarks that do not even measure the same thing in two solutions. One of the measurements might take into account irrelevant operations like traversing arrays instead of doing operations. Or one might take into account putting together a dataset. Or when comparing two solutions that mutate data, then the second one tested might be working with the already mutated dataset.

Benchmarks may look correct but are not measuring real performance

Next problem. Here is some required reading: microbenchmarks fairy tale and Which is faster?. The moral of reading these is that performance is hard to measure. A couple of lines of code and some sample dataset may have vastly different performance characteristics when ran individually versus when ran as a part of a real-life application against real-life data. The numbers you get measuring the solutions in the Q&A might be completely off.

One very common problem around this area in recent years are JIT compiler. Measuring the performance of two solutions without proper warm-up of the engine can give misleading results for the first solution tested. This is because JIT optimisations will not start running immediately but after some usage of the code.

Again, JIT optimisations can very well differ in a real-life application using real-life data, so even if the benchmark included a warm-up, the results might still be misleading.

Benchmarks might omit relevant differences between solutions, focusing on just speed of performance

This is very important to keep in mind. Different solutions often have more to them than just different numbers. For example in JavaScript to check if someObject has a key-value for foo one can test

  • someObject.foo !== undefined which returns false if the property exists but the value of it is undefined or if the property does not exist at all
  • someObject.hasOwnProperty("foo") which checks if the object has non-inherited a property called foo. In particular, it would return false if the property exists (even if it has a non-undefined value) but is inherited.
  • "foo" in someObject which checks if foo exists at all in the object, even checking the inheritance.

There is an overlap between these three when checking an object without any inherited properties has or does not have a property. And we either do not care if the value is undefined or it is not expect it to be that. Comparing the speed of the three throws all this aside to produce a set of numbers.

Similarly, often solutions do have relevant differences and it is most important to pick the one that fits your requirements. Rather than the fastest one.

The difference often does not matter

To come to this point we have to go through a lot of "ifs" - if the benchmarks are current, representative, accurate, if they even compare the correct things. Even if all of these pass, the numbers can still end up useless. Let's take three options for checking object properties above. Somebody did test them and the results were

  • === undefined : 2 787 485 operations per second.
  • hasOwnProperty: 2 673 798 operations per second.
  • in: 2 344 174 operations per second.

Which one is the winner in this case? Clearly...none of them. Yes, clearly one has the highest number yet the worst performance is still 426.589494 nanoseconds or 0.000426589494 milliseconds per operation. IF you used the worst one and decided to check a million records, the code would still run in well under half a second. That is still a huge "if" - hence the bold and the capitals. If you are not performing a million operations, then the difference would not matter.

Very often I see performance measurements for operations where the number should mean that the whole benchmark was done in vein. Yet it was posted and upvoted very likely misleading others that it provides useful information.

Why are benchmarks a bad fit for Stack Overflow

So far, what I said was broadly applicable to benchmarks. Somewhat geared toward JavaScript as that is where most my experience is, however, the points above are equally valid on-site and off-site. This point is specific to Stack Overflow.

Let's assume the benchmark do pass all the ifs above - they are representative, accurate, useful, etc. The problem is that they have to be maintained. And they very often are not. An answer from 2015 with an update from 2016 might still be missing the comparison with answers that were posted since. Or any updates made to previous answers.

It might not even be possible for somebody else to update the answer without entirely rewriting it, as any editor likely cannot run the benchmarks with the same parameters as the original author of the answer. They would be running the code on different machines, after all. And some answers even omit relevant setup, so any editor would have to either guess or entirely re-create the benchmark suite.

Any and all benchmark answers become unreliable the day they are posted. What if the author never comes back? What if the author is diligent in updating it but how long would they be doing that?

Simply put, all benchmarking answers have to have ongoing maintenance to stay relevant. Which I would argue does not fit in the Q&A model. Yes, other answers can also become out of date. Yet, any benchmark answer is by its very nature reliant on all other answers and has to be constantly updated to match. Add to that any of the possible changes in environments where the code would run or differences in how the code might behave under different circumstances and I would argue these answers are ill equipped to provide on-going useful knowledge.


What can we do about these answers?

Absolutely nothing right now. Even if the answer is grossly misleading and completely wrong, if it has high score count, we cannot do anything about it. Nor are users apparently interested in the comparison being correct. The votes are coming in from the comparison being there, despite its correctness.

What should happen instead is benchmarking answers should be removed. And if the speed of a solution is a factor, then edit all other existing answers adding a benchmark for the solution they provide.

This also relies on establishing very clear and objective guidelines on how things can and should be benchmarked. Perhaps that can be set up in the question, thus any answerer can do it themselves, or anybody can edit the results in.

I can already foresee one of the very first objections would be missing out on reputation. To which, I could point out the goal of Stack Overflow is to be a repository of knowledge. Not a reputation mining farm.

Yes, the current system is not well equipped to handle answers with contributions. That is a much deeper flaw that runs throughout the Q&A format set up for the site. "Meta answers" that comment on other answers are a symptom of it. Yet, I believe this is the right thing to do. As opposed to produce answers that are either irrelevant at the time of post, or are left to lose relevancy as time progresses and especially as new answers are posted.

15
  • 14
    Ok, but not everything is Javascript and I have seen very useful benchmarking answers for other languages. For example, although perhaps an esoteric one, python code to generate a list of the first N primes is useful in number theoretic analysis and experiments. The benchmarking "meta"-answers that are given I and many others have found very useful and the answerers typically put in a lot of work. Like any other answer good benchmarking answers get upvoted and bad ones get downvoted. And they're still relevant a decade later. Commented Apr 1, 2023 at 15:17
  • 2
    Last update to that answer is in 2017. By the Community bot. The last answer to that question is from 2022. There are 13 answers posted after the last update to the answer benchmarking all solutions. Note, I didn't check when the last actual update to the benchmarks is because I don't care. I can tell you that right now this answer is at least six years out of date and not showing a complete picture. How useful is it to have outdated and unrepresentative information? If it served such a good job, why didn't anybody update it for six years? I do believe you failed to show its importance.
    – VLAZ
    Commented Apr 1, 2023 at 16:07
  • 2
    I do also want to point out that your example isn't representative of the whole of SO. Thus "not everything is Javascript" goes both ways. Let's assume that the benchmark you showed me is correct. I haven't checked, I can't be bothered. But even if it is, the counter to your statement is simple: not every benchmark is that benchmark. Anecdotal examples aplenty. You've not actually shown they count for much. Or what should we do in general across SO. I've put forward arguments that are wide reaching, you've put forward a single example that I've even argued against.
    – VLAZ
    Commented Apr 1, 2023 at 16:10
  • 6
    Right, therefore a categorical administrative determination on how to deal with them would be wrong. Let the upvote/downvote mechanism do its job. Commented Apr 1, 2023 at 16:13
  • 6
    "Let the upvote/downvote mechanism do its job." it doesn't
    – VLAZ
    Commented Apr 1, 2023 at 16:13
  • 2
    In my experience it does. Commented Apr 1, 2023 at 16:14
  • 2
    "No u" now, is it? If you have something to argue, I'd advise you to write an answer. If you do believe in votes, then it feels a lot like you're shying away from the voting system while trying to make your point about the voting system being preferable.
    – VLAZ
    Commented Apr 1, 2023 at 16:16
  • So... how should we handle answers that don't answer the question, but evaluate the time complexity of other answers? This meta-answer seems to say these answers are bad, but how should we actually handle them? Commented Apr 1, 2023 at 17:00
  • 2
    @MisterMiyagi how do we handle answers that are ill equipped to provide useful knowledge? I guess we just leave them be and accrue upvotes. Let the visitors sort it out. And maybe remove them when our failure to maintain a useful repository of knowledge has real life consequences. We can't do anything about them. I mean, we could delete them but...anything with score over 3 is basically undeletable, so no. Judging by the votes, I guess my arguments for the uselessness failed to land. We usually remove useless content. Alas.
    – VLAZ
    Commented Apr 1, 2023 at 17:31
  • 3
    Sure, answers composed of benchmarking may well be wrong or useless, and that doesn't qualify them for deletion. But answers composed of benchmarking already existing answers when the question wasn't asking for benchmarking sounds like NAA to me - it's (bulk) commentary on other answers, rather than an answer in itself. Commented Apr 1, 2023 at 19:20
  • @CertainPerformance it's a failure of the Q&A model, as I said. And in the past, I've also remarked that such an answer "is ultimately leveraging the Q&A mechanism (the A part of it) to artificially supply a feature the Q&A mechanism needs.". Because, let's really think about Q&A for a moment. You have a question and any of the answers should be sufficient to answer it. Some might be better or worse but it should be possible to pick one and use it. If you have a "meta-ansswer" you cannot use it. That's not a proper Q&A format.
    – VLAZ
    Commented Apr 1, 2023 at 19:25
  • I'm puzzled that your arguments are getting a cool reception, and doubly so since the benchmark answer is ignoring the question that was asked ("How would one interweave them efficiently...?"). A code solution is obviously required, rather than a benchmark. From the SO documentation for "How do I write a good answer?", with my emphasis added: "Read the question carefully What is the question asking for? Make sure your answer provides that – or at least a viable alternative". The benchmark answer should deleted as it's clearly NAA.
    – skomisa
    Commented Apr 2, 2023 at 0:35
  • 7
    "If you have a "meta-ansswer" you cannot use it" While I'm not a fan of duplicate answers generally, I feel the need to push back on this. My experience has been that benchmarking answers consistently show the code that they're benchmarking (copied from the other answers and credited appropriately). That said, I think this answer makes many other good points. Commented Apr 2, 2023 at 2:05
  • 4
    Yes, bad benchmarking code is bad, and it can be misleading if you aren't actually timing the parts of the algorithm that you think you're timing. OTOH, a good benchmarking answer shows how to do it properly, explains what is actually being timed, the reasons for the performance of the different algos, with various data sets. Etc. And it provides benchmarking code that can be easily extended if new solutions arise. Disclosure: I've written quite a few benchmarking answers, but I almost always add a new solution (or several), and I encourage readers to run the code on their own machine.
    – PM 2Ring
    Commented Apr 2, 2023 at 17:20
  • However, I certainly agree that benchmarking JavaScript is a fairly pointless exercise because of the extreme variability of the environments where the code runs.
    – PM 2Ring
    Commented Apr 2, 2023 at 17:22
-8

First, that answer is not an answer; it is a commentary on another post. And, if it was an answer, then it's essentially just a thank-you. If you don't think alone is enough to warrant its deletion, then just look at the canned comments in the Low Quality Answers queue:

Enter image description here

As I said, its flaws are twofold:

  • The answer does not answer the question itself. It just comments on the other answers that are already there and gives a link to another question. It's not plagiarism because everything's cited, but the author doesn't add anything new. In my opinion, the best way to tell everyone that you tested every solution and determined the best one is to leave a comment with your reasoning and upvote the answer. This is the point of comments and feedback: to speak your mind on the answer's advantages and flaws. It might even have been useful to leave a comment on the other answers to point out that they weren't as fast as the best one.
  • Also, the gist of the answer can be summarized as this: "Paul's answer is the best." The whole post is fundamentally just a drawn-out and detailed thank you. As the canned comment from the LQA queue says above, the author should've just upvoted the answer. The whole point of the voting system is to push the best answers to the top (see the tour). And guess what, the answer that the author determined was the best did end up being the most upvoted and accepted, and it rose to the top.

At best, this answer is an index to find other answers. On its own, it doesn't remotely attempt to solve the problem put forth in the question. Don't get me wrong: I definitely agree that it is useful, but not as an answer (after all, it is the second most upvoted response). This is a pretty rare case, and the whole thing is already a post rather than a comment, so I think it'd be both best and reasonable to make it a Community Wiki to help it serve its non-answer role.

2
  • 7
    The answer did provide a custom numba solution, and the benchmark demonstrates that it is the competively efficient as requested by the question. I would really like to know why people insist this isn’t answering the question instead of just asserting this would be the case. Commented Apr 3, 2023 at 5:39
  • 5
    I feel this answer illustrates a downside of having canned close reasons. It allows people to forget that the goal is to have a searchable archive of top-quality questions and answers, and that the canned close reasons are the result of a trial-and-error process that has identified common reasons why some answers do not promote the overarching goal. The canned close reasons are a convenience to avoid having to type a custom close reason, not an end in themselves. It's entirely possible that an answer which seems to fit one of the close reasons is still a good answer. Commented Apr 3, 2023 at 15:57
-10

How should we handle answers that don't answer the question, but evaluate the performance of other answers?

You see the bit of your question I struck out? It's struck out because it's irrelevant.

If an "answer" doesn't attempt to answer the question as asked, it is objectively not an answer and should be actioned as such. That is the rule, that is the end of the decision tree.

No number of ifs, ands, buts, apparent usefulness, nonsensical Shog9 mental gymnastics, or any other qualification can or should be allowed to obscure the true nature of these not-answers, nor prevent them from being removed for being off topic.

7
  • 8
    "If an "answer" doesn't attempt to answer the question as asked" OP said it doesn't, but I'm not sure if I agree. Commented Apr 2, 2023 at 12:43
  • 2
    @HolyBlackCat From the SO documentation for How do I write a good answer?: "Read the question carefully What is the question asking for? Make sure your answer provides that – or at least a viable alternative.". How can that possibly be interpreted that to mean that running benchmarks on other answers constitutes a valid answer?
    – skomisa
    Commented Apr 2, 2023 at 18:18
  • 1
    Not directly relevant to this question, but relevant to your answer: XY problems should not be answered directly. Commented Apr 2, 2023 at 19:24
  • 1
    @skomisa Sorry, I don’t see how it categorically could not. For example, the answer linked includes the other solutions so it definitely provides a solution as well; the question asking "How would one interweave them efficiently" even explicitly asks for a solution that performs well - providing a benchmark for comparison seems to clearly answer how to efficiently do something. Commented Apr 3, 2023 at 5:33
  • 2
    @MisterMiyagi By that argument, an answer that plagiarises another is in and of itself a valid answer. Just because a benchmark "answer" appears to be more useful than pure plagiarism, does not make it one iota more of an actual answer.
    – Ian Kemp
    Commented Apr 3, 2023 at 7:34
  • 2
    @IanKemp Not even the caricature presented in the question plagiarises. Please tone down this excessive misrepresentation. Commented Apr 3, 2023 at 7:44
  • 1
    @IanKemp Exactly. While some may find such off topic benchmark answers interesting or helpful, the notion that this somehow makes them valid answers is absurd. The phrase "meta answer" used in the answer from VLAZ captures the situation precisely: a benchmark answer is an answer about other answers rather than the original question.
    – skomisa
    Commented Apr 4, 2023 at 4:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .