Timeline for Can we please tweak the algorithm that selects review audit questions
Current License: CC BY-SA 3.0
11 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Nov 29, 2017 at 21:08 | comment | added | Shawn | (cont.) and whats worse is flags weren't being reviewed for a couple weeks. So I could get re-banned on audits from weeks ago and I could not go pull those flags off once I knew their meanings had changed. I got tired of it and basically stopped reviewing. I just won't deal with the frustration of being punished for something with an always changing subjective meaning and with different meanings on different queues. | |
Nov 29, 2017 at 21:06 | comment | added | Shawn | Up until about a year ago I actually enjoyed reviewing. However my reviewing has dropped off to near zero. The reason for this was that I was actually trying hard and still somehow failing edge audits for bad audit examples that had a couple up votes from uninformed users. At the same time there was heated debate about the meanings of the different audit flags. For months I was fine and then all of a sudden my flags started getting rejected. I was then in "audit hell" as there is a post on this; basically my flags weren't aging off fast enough and one of the audits put me back in fail. | |
Nov 28, 2017 at 19:00 | comment | added | Servy | So people abusing review was indeed a problem before the incentives were in place (the review count, the rankings, the badges, etc.), but those incentives absolutely made things worse. Lots of people (myself included) said many times that these incentives should be removed, as they only ever encouraged people to do reviews who weren't interested in reviewing well. You can see for yourself how convinced SE was of that argument. | |
Nov 28, 2017 at 18:58 | comment | added | theMayer | Seems like it would be easier to get rid of the review count | |
Nov 28, 2017 at 18:56 | comment | added | Servy | Yes, it adds value, as can be seen by how unusable the system was without audits, which was the case for quite some time. The problems that they cause (which is honestly mostly just some frustration in reviewers ever once in awhile when they hit a bad audit, and need to learn to ignore it) absolutely pale in comparison to the amount of damage they've prevented (keeping out thousands and thousands of users just clicking buttons randomly to increase their review count, causing enormous amounts of damage). | |
Nov 28, 2017 at 18:54 | comment | added | Servy | It's there to ensure that people are "trying". It's not there to ensure that they're actually doing a good job. It's not attempting to do that, and it doesn't often accomplish that. (It occasionally helps people improve, incidentally, but it's not a primary purpose, nor something it consistently does.) | |
Nov 28, 2017 at 18:54 | comment | added | theMayer | Does the audit system actually add value (more than its cost, anyway)? Has this been asked or studied? | |
Nov 28, 2017 at 18:52 | comment | added | theMayer | Regardless of what the intent was, the reality is that the system is utilized to enforce standards. If not, it would not contain the words "you should click 'leave open'..." when you fail. | |
Nov 28, 2017 at 18:52 | comment | added | Servy |
And if we can't do that, let's up the privilege level for reviewing, disable the audit system, and be done with it. This has been shown to not be a solution. Reviewer quality is simply uncorrelated with reputation once you get to about the level where you're using most of the queues. When you look at the stats of people failing audits enough to get banned regularly, it's not just the low rep users.
|
|
Nov 28, 2017 at 18:51 | comment | added | Servy |
the purpose of auditing is not simply to catch robo-reviewers, but to reinforce site standards That's not what SE said when they created it and implemented it. It has been brought up as a possible side benefit, but it's not the goal of the system.
|
|
Nov 28, 2017 at 18:45 | history | answered | theMayer | CC BY-SA 3.0 |