4

I just written some simple scripts, named it as a bot and executed it through browser's console when the browser has loaded with the close vote review page (Probably, the bot can be implemented all review pages with some little modifications in the script). I thought that probably SO would block executing the repetitive functionality from the script through asking any captcha. But it did not block the execution and it just allowed to cast all the close votes(40) in ~1.5 minutes. Is this good for us? or should we need any bot detector in this area?

Note: Actually the system has conducted some tests while the bot is reviewing the posts, but obviously it just expected a positive response, meaning closing the post. So that is not at all a problem for the bot, we can handle it manually or we can modify the script accordingly to bye pass that.

I did not disclose the code for that bot, since I don't know the sensitivity of this matter.

11
  • 3
    The audits! They do nothing! Apr 28, 2014 at 13:18
  • @FrédéricHamidi what do you mean by audits.? Apr 28, 2014 at 13:19
  • 2
    See What are review tests (audits) and how do they work? on the über-meta. Apr 28, 2014 at 13:21
  • If you manage to write a bot that does a good job (highly unlikely); excellent. If not it should fail the audits relatively quickly and get a big-ol review ban Apr 28, 2014 at 13:23
  • 5
    @Richard, actually the system has conducted some tests while the bot is reviewing the posts, but obviously it just expected a positive response, meaning closing the post. Either the bot was lucky, or we need more "Leave Open" audits :) Apr 28, 2014 at 13:24
  • 4
    @FrédéricHamidi I'm sure I've seen leave open audits out in the wild, perhaps 40 is too small a sample size to be sure of hitting one; 40 questions is probably about 3 audits, not impossible (assuming 50 50 chance) for all 3 to be leave close audits Apr 28, 2014 at 13:25
  • @Richard, yup, I've seen them too. I was even under the impression that the type of audit was linked to your recent activity (many close votes => leave open audit, many leave open votes => close audit). Now I'm not so sure... Apr 28, 2014 at 13:27
  • @FrédéricHamidi but in real time casting a close vote on a perfectly valid post doesn't matter right..? Does SO count it as a failure..? i am not sure about this. Apr 28, 2014 at 13:29
  • @Rajaprabhu, in an audit it is counted as a failure. Outside of audits, well, it is counted as a valid vote to close. Apr 28, 2014 at 13:30
  • 3
    @RichardTingle: It's incredibly trivial for a bot to detect review audits with 100% reliability, anyway. I won't spoil the method here, but it's been mentioned on meta before, and in any case, anyone with the appropriate skills to write a review bot should have no trouble working it out. Apr 28, 2014 at 17:05
  • There are multiple methods, not least the method @IlmariKaronen is probably referring to. You can always do what a human would do and check (a) the post does not exist, (b) the post has a significant discrepancy in votes, and (c) status of the question is different to what is shown. Apr 29, 2014 at 3:15

1 Answer 1

5

The close votes review queue is a bit of an odd case, in that you can get away with this for a little while and still not look out of the ordinary. It was the last queue to get audits, and even then I don't think they're as frequent as they should be.

Right now, for example, there are 37 users over the last 7 days with at least 10 reviews who have voted to close 100% of the items they were presented with in review there. There's a lot of terrible stuff in that queue that does deserve to be closed, but that seems a little high. Perhaps a few more "Leave Open" audit test cases could be sprinkled in there to keep people honest.

I can tell you that if you tried this in any other review queue, either the system or one of us would see it and end your reviewing pretty quickly. It's extremely difficult to maintain a streak of only one kind of vote in the other queues, and even if you find a way to game the audits (as some people have), you stick out like a sore thumb in our tools. We ban you for much longer if you do that than had you hit audits.

So, yes, the close votes queue could use a little tweaking in the way that audits are presented to users, but I don't believe that other safeguards are necessary to prevent review abuse from true bots. Humans who click "approve" on everything are the more common danger, and captchas, etc. aren't going to stop them.

1
  • @Stilly.stack: it is tracked internally, 10k users can see the questions they have skipped. Apr 29, 2014 at 3:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .