15

More than once I've seen bounties stating "answers must be tested", with the bounty donor then commenting on answers "read the above which says test your code".

While the language is never aggressive or attacking (definitely not deserving of flagging), I do find it an incredibly rude response to someone making a genuine attempt to help.

At the same time, I know the bounty section for any given tag tends to attract low quality answers hoping for fast reputation, so maybe it's not so unreasonable.

So is it OK to expect that anyone who answers or makes suggestions should go out of their way to test the solution for you?

I'd like a definitive answer I could link to, as I was surprised to find no "Stack Overflow is not a code writing service" page exists (I had always thought there was such a page for some reason).

10
  • i mean.. you don't have to answer it or comply with demands. Your answer is there for both the op AND future visitors. The bounty doesn't change that.
    – Kevin B
    Mar 15, 2018 at 15:29
  • Sure. Does not necessarily have to concern own answers though. I've seen reasonable answers deleted because of such comments. - but if I respond I'd rather know I was not in the wrong for doing so. Mar 15, 2018 at 15:33
  • 2
    deleted by... who? I'd assume that's just the owner of the answer buckling under peer pressure.
    – Kevin B
    Mar 15, 2018 at 15:33
  • 1
    Yes, exactly what I meant (and then of course, it's not available for future visitors) Mar 15, 2018 at 15:34
  • 3
    this seems quite fishy... and why i hate bounties in general. The person issuing the bounty is acting like they have no interest in the question or a solution. Looks like they're wasting people's time by offering bounties on old questions.
    – Kevin B
    Mar 15, 2018 at 15:56
  • 3
    Some users have impractically high expectations of their "investment". It is an easy trap, they'll expect their effort in scraping together enough rep for a bounty to be returned in kind. Quite a slog for some. No concept of how many years you invested in knowing what you know, they haven't done that yet. Just ignore it and post the answer you prefer. Mar 15, 2018 at 16:00
  • 6
    I don't smell anything particularly fishy about requiring that answers be verifiable. I've seen enough untested, low-effort and most importantly incorrect answers to bounty questions that were easily disproved (and easily earned themselves a downvote) by simply testing their code, so these bounty messages were likely borne out of frustration with such answers (as you've postulated).
    – BoltClock
    Mar 15, 2018 at 17:20
  • 3
    It only gets fishy when your answer is verifiable and you have tested and verified it yet the bounty setter simply chooses not to believe you. Then you have a right to feel shortchanged. But I've seen more bounties used for rep laundering between sock puppets than bounties withheld from answers that otherwise met their requirements.
    – BoltClock
    Mar 15, 2018 at 17:21
  • 3
    'OK, fine, please post your complete test specifications and how you wish to pay for this service' Mar 15, 2018 at 18:14
  • 3
    Only the OP can test to ensure sufficient fitness for their own purpose. It's unrealistic to attempt to outsource testing for free when it's overwhelmingly likely that the OP will not wish to put any effort into writing and posting test specs. harnesses, test cases, drive files, data tables etc. That would mean, you know, actual work. Mar 15, 2018 at 18:21

3 Answers 3

15

It's unconventional to require that code in a bounty be tested...but there's nothing really to stop them from saying that. At a minimum, it's a smell that there's something kinda dodgy going on. Usually I see that as a sign that someone wants to outsource their job on to us.

It's not the most polite thing someone could say and it's best to ignore it. If they don't like your answer but the rest of the community does and they don't pick one for themselves, it doesn't matter what requirements they put in place since the bounty will be self-awarded at the end.

2
  • 13
    To be clear, it's best to ignore it in the sense of not taking it to heart (feeling slighted, patronized, etc). Of course, there's never a reason not to test your code whether or not it's requested when it's not intended to be pseudocode.
    – BoltClock
    Mar 15, 2018 at 17:26
  • 1
    "someone wants to outsource their job on to us" Yes, and he don't even want to test it himself. Mar 15, 2018 at 20:33
6

'Stack Overflow is not a code writing service'...

It's not a test house either.

It's not unreasonable to expect that an answering user provide code that compiles and links, maybe with minimal changes of arg names or whatever. It IS however, unrealistic to ask them to provide any kind of testing.

Testing requires the right compiler, linker, environment, input data, output analysis. OP's will not provide specs and data for testing, (they just won't - accept it), and even if they did, the resulting task would be far too broad, extensive and onerous for SO volunteers.

What is needed for effective testing is the same compiler, linker, environment etc. etc. as the OP has.

Only one user has that, and that user is the only one who can effectively test.

'answers must be tested' - fine, you do it!

3
  • 10
    If you write an answer that is clearly and obviously not tested against the OP's criterion, it's fair to downvote. It's a reasonable expectation that one's answers are at least somewhat accurate.
    – Makoto
    Mar 15, 2018 at 18:45
  • Fair to downvote a broken answer; fair to post an answer without rigorously testing it; fair to request that "answers must be tested"; fair to ignore that request. All these things are well within bounds. Basically, caveat lector when you're using SO answers.
    – jscs
    Mar 16, 2018 at 13:00
  • I mean, frankly, if you expect that some stranger on the internet is going to reliably certify a bunch of code as functional and suitable for your project that they can't even see... well, I have a wonderful investment opportunity for you in the beautiful southern United States.
    – jscs
    Mar 16, 2018 at 13:04
4

(EDIT I've left the post as-is but see some of the conceptual refinements offered by commenters below the post.)

Have I completely misunderstood how Stack Overflow works? (Or did I just misunderstand the question...)

I always interpreted the purpose behind

  • providing an [mcve],
  • disallowing primarily opinion-based questions,
  • snippets

was to facilitate the development of high-quality answers elaborating a demonstrably workable solution.

I don't see how a bounty is any different. To me, a bounty is simply a polite, incentivized version of an all-caps title (à la "I REALLY NEED HELP!!1!").

I have been guilty of posting a very simple code fragment without testing it, but I do so aware that I risk a flood of down-votes because I forgot a comma and it throws an exception.

Overwhelmingly, I take time, usually learning something new in the process, write some code in an IDE, compile/lint it, and post that as an answer. No, Stack Overflow is not a code writing service, but isn't that what MCVEs are for?

I thought the idea of "answers must be tested" was implicit in every question.

That said, everything about suitability for a particular purpose still applies, just because some code compiles doesn't mean it's the best (or even a good) answer.


Now, playing devil's advocate to myself...

I've been awarded one bounty so far for an answer on updating the Dartium VM in a custom compile of Chromium. I did a bunch of reading and answered basically "no" with references, but didn't test it.

I was interested in the problem, but just compiling Chromium without the proper environment setup can take 8+ hours. Going more in-depth was incredibly prohibitive.

I was eventually awarded (I believe, automatically) the bounty because I offered the only answer with a sufficiently positive score.

12
  • 2
    lots of bounty question posters allow themselves to post bad questions, just because they know people will do an extra effort because of the extra rep. Good questions with bounties are rare. Personally I'm staying away from them, "answers must be tested" or not. Mar 15, 2018 at 20:32
  • 1
    note that nice users detecting a typo in a good answer (which hasn't been tested) won't downvote. Or will downvote with comment, wait until you fix it, then retract+upvote. If we tested all our answers (specially the ones where we know the language and the issue very well), that'd mean extra work. Sometimes it's worth taking the risk of not testing when you're sure of yourself, after all it's not your problem. Mar 15, 2018 at 20:34
  • @Jean-FrançoisFabre Yes I've seen that too, or nice users will just edit and fix it. I was being slightly facetious with the "flood of downvote" (though that did happen once).
    – msanford
    Mar 15, 2018 at 20:45
  • it happens when the question is bad, so the answer needs to be spot on (and tested :)) or you get rightfully slapped Mar 15, 2018 at 20:47
  • 2
    I think that your sentiment on what role an MCVE plays here is a bit...off. You're not far from the mark but this doesn't quite satisfy. The MCVE is more of a mental checklist for the OP to be sure that they have all the pieces ready to go for us when we wish to assist them with their code. The side effect is that the answers are high quality and that they demonstrate a workable solution. But, there are great questions without an MCVE that have great answers.
    – Makoto
    Mar 15, 2018 at 22:21
  • 2
    A bounty also represents "This is a more difficult question which takes time to answer, here is an incentive to invest the time and effort to do so". It is also a way to signal "This question has answers, but they are not good enough. Please, don't ignore this question." And apparently, some people also treat them as an open contract for hire. Well... you're certainly free to try, and fail.
    – Gimby
    Mar 16, 2018 at 8:49
  • @Makoto Upon reflection, you're absolutely right on [mcve], it is a positive side-effect rather than a primary goal.
    – msanford
    Mar 16, 2018 at 12:53
  • Good point @Gimby, it is richer and more subtle than just a "polite all-caps" as I suggested.
    – msanford
    Mar 16, 2018 at 12:54
  • Well also an ease of mind then: I'd find it hard to believe that you would get a rain of downvotes because you made a typo somewhere, you will get at least a comment and maybe someone will even just fix it. Now if you post an answer that is fundamentally wrong... then you'd better bring an umbrella.
    – Gimby
    Mar 16, 2018 at 13:01
  • To expand on Makoto's comment, MCVE doesn't apply to answers.
    – jscs
    Mar 16, 2018 at 13:06
  • @JoshCaswell Really? If the question contained one I find communicating the answer in an adapted form of the code is easier than in prose, even if prose will do.
    – msanford
    Mar 16, 2018 at 13:13
  • 2
    Sure, if you think that's the best way to communicate your answer, then do that by all means. All I meant was that there is not a requirement that answers provide an MCVE. If a prose explanation, or prose plus a line or two of code, are better, then that's fine too.
    – jscs
    Mar 16, 2018 at 13:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .