Timeline for Let us bring an end to the "robo-reviewer" war: Phase 1 - 2
Current License: CC BY-SA 3.0
52 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Oct 23, 2014 at 18:08 | comment | added | Joe W | It seems like you would be better off suggesting the removal of the ability to suggest edits altogether with suggestions such as making reject votes worth more then accept votes. | |
Oct 23, 2014 at 8:58 | answer | added | Scimonster | timeline score: 4 | |
Jun 13, 2014 at 12:11 | comment | added | thegrinner |
they have made >X suggested edits (or have an edit badge) - all I see this doing is forcing people to make exactly the kinds of edits we don't want as too minor: they'll need to edit faster than anyone else to be counted, which means less time to make substantial edits.
|
|
Jun 12, 2014 at 14:08 | answer | added | zwol | timeline score: 10 | |
Jun 12, 2014 at 13:22 | history | edited | Sam | CC BY-SA 3.0 |
edited title
|
Jun 12, 2014 at 7:38 | answer | added | user1725145 | timeline score: 3 | |
Jun 12, 2014 at 4:42 | answer | added | Shog9 | timeline score: 23 | |
Jun 11, 2014 at 21:12 | comment | added | PlasmaHH | Maybe add something like votes being able to cancel out, and even if a certain threshold is reached, let it sit there for a while until no one cancles out more votes. | |
Jun 11, 2014 at 20:55 | answer | added | QueueHammer | timeline score: -1 | |
Jun 11, 2014 at 20:54 | answer | added | bd33 | timeline score: 2 | |
Jun 11, 2014 at 20:52 | answer | added | Bernhard Barker | timeline score: 8 | |
Jun 11, 2014 at 16:35 | comment | added | Sam | @Servy Agreed, I could be here all day discussing this (IMO, an interesting subject) with you, but I'll leave that for another post. (Already at 33 comments). | |
Jun 11, 2014 at 16:22 | comment | added | Servy | @Sam It's been proposed before (I'll see if I can find a dupe), but the main issue is that creating such a test that is both an effective tool at measuring the ability of the user to review, difficult to cheat on, and also not being prohibitively difficult to develop (both in the technical sense and in the sense of creating content for it) would be very difficult. There are all sorts of problems you need to deal with, depending on how you intend to go about doing it. I could discuss some of them, but it would likely be best for that to be in its own question, as it gets involved. | |
Jun 11, 2014 at 16:20 | comment | added | Sam | @mehow I guess we'll just have to wait for it to get on the community bulletin... | |
Jun 11, 2014 at 16:19 | comment | added | Sam |
@Servy I'm going to go "out on a limb" here, but perhaps we could revoke everyone's access to the SE queue, and then once a user meets with the above criteria he/she could be given X audits to do (without them knowing), and if they got >X of them right, they could be given access to the queue. Kind of like a "reviewers test" to be more certain that the user will vote appropriately.
|
|
Jun 11, 2014 at 15:59 | history | edited | Sam | CC BY-SA 3.0 |
added 235 characters in body
|
Jun 11, 2014 at 15:56 | comment | added | Sam | @Servy I see, well I suppose it would be more beneficial to sacrifice a few "ok" edits to stop a bunch of bad ones, than letting them all get through. I'll add it to the list of suggests anyway. | |
Jun 11, 2014 at 15:51 | comment | added | Servy | @Sam It's currently only really worth considering because bad reviewers have an inherent preference to approving posts they should reject, rather than rejecting posts that they should approved. Of course, when you end up, for whatever reason, in the position where reviewers end up rejecting items that they shouldn't, that system suddenly starts working against you. I'd say its an option worth considering, but not an option without its pitfalls. | |
Jun 11, 2014 at 15:51 | comment | added | user2140173 | ahh I wish I could set a bounty on this post | |
Jun 11, 2014 at 15:48 | comment | added | Sam | @Servy That sounds like a good idea (adding more weight on reject votes), and what if this were to be implemented as well then? | |
Jun 11, 2014 at 15:46 | comment | added | Servy | @Sam Adding more reviewers only helps if there is significantly more good reviewers than bad reviewers. While things certainly aren't as bad as they were a year ago, there are enough bad reviewers that you'll still see quite a lot of bad edits approved no matter how many votes you require (so long as it's a first to X; if you require more approve votes than reject votes, that's a different story). | |
Jun 11, 2014 at 15:44 | comment | added | Servy | @deviantfan Before audits were in place the suggested edit system was completely in shambles. Approximately zero suggested edits would ever be rejected, this included obvious vandalism, spam, literally anything. If it was suggested, it got approved, usually without ever being read by any of the 2-3 people that approved it. After the audit system was put in place the system went from "making the site worse through its assistance" to "causing a lot of problems, but still doing enough good to at least be worth existing". There were a lot of changes, but audits were the most important. | |
Jun 11, 2014 at 15:39 | vote | accept | Sam | ||
Sep 8, 2014 at 17:52 | |||||
Jun 11, 2014 at 15:39 | comment | added | Sam | @Servy I understand and agree with what you're saying but for an intervention regarding this issue for a community of this size would be enormous (and probably unlikely). Which is why I also suggested increasing the number of reviewers needed to approve an edit in the hope that the balance of good reviewer vs bad reviewers would be more even, if not, more in favour of the good reviewers. Maybe reviewers could be notified if the edit was finally approved or not, if it was approved and it was a bad edit, the good reviewers could flag the situation for a mod to ban/warn the bad reviewers? | |
Jun 11, 2014 at 15:32 | comment | added | deviantfan | Didn´t read everything so far, but PLEASE not more audit. They are annoying enough. (This is not because of personal bad experience. I think I failed nothing in my >300 edit reviews) | |
Jun 11, 2014 at 15:27 | comment | added | Servy | That's the main issue with this proposal, and what I was getting at with my earlier comments. This proposal to fix the review system is dependent on having a mechanism to first fix the review system. It only works to improve reviewers if you already have good reviewers, and if you've done that, then you've already won, there's no need to do anything else. | |
Jun 11, 2014 at 15:25 | comment | added | Servy | @Sam The problem there is that, with the audits as they stand, they're really only capable of kicking out people who don't even read the posts. There are still lots of really bad reviewers who are paying just enough attention to get past audits, while still not really understanding the edit system enough to actually reject inappropriate edits that aren't blatant vandalism. Yes, you said you want audits to be harder, but doing that successfully is of course non-trivial. If it can be done successfully you likely don't need to do anything else. | |
Jun 11, 2014 at 15:24 | answer | added | user2140173 | timeline score: 28 | |
Jun 11, 2014 at 15:24 | comment | added | Sam | @Servy This is why I suggested that the audit rate and difficulty should be increased. Maybe reviewers that continually fail audits should be permanently banned? | |
Jun 11, 2014 at 15:21 | comment | added | Servy | Since these bad edits are currently being largely accepted, if there isn't a major intervention when this system is first put in place, these same reviewers will keep approving bad edits, letting bad editors continue to become bad reviewers. There needs to be some form of intervention before a change like this is made to ensure that, at least for some short probationary period, we really only approve good edits, so that the people that get into the system are the people who know what should be approved, and so that a positive feedback loop is allowed to be created. | |
Jun 11, 2014 at 15:19 | comment | added | Servy | The main issue here is that it creates a feedback loop. If it ends up being a positive feedback loop, then that's great, but if you're not very careful, it could just as easily turn into a negative feedback loop. Currently people that just go around spamming super minor edits (often with a rather large percent of invalid edits mixed in) and end up with a very large percentage of their edits approved. Over 80% in my experiences, often much more. If that is allowed to happen at all, then they become reviewers, and continue to accept others making similar bad edits. | |
Jun 11, 2014 at 15:12 | history | edited | Sam | CC BY-SA 3.0 |
added 7 characters in body
|
Jun 11, 2014 at 15:03 | comment | added | Sam | @Payeli Sounds like a good idea, but that may lead to more "irrelevant" posts on Meta though, as users try to "jump" through that "hoop". | |
Jun 11, 2014 at 15:01 | comment | added | Infinite Recursion | Maybe you can make meta participation as a mandatory criteria. Like have at least x meta posts with y upvotes. Once they come to meta and see the consequences others face, they will improve. Bans are good, and awareness of prospective bans and troubles might deter them. | |
Jun 11, 2014 at 14:58 | comment | added | Sam | @Payeli I would say definitely (open to suggestions of course). If a users decides to start making bad edits again (s)he would have to be denied access to the queue. | |
Jun 11, 2014 at 14:55 | comment | added | Infinite Recursion | Will the privilege be revoked when the ratio drops? Like it revokes at present when rep drops below 2k? | |
Jun 11, 2014 at 14:51 | comment | added | The Guy with The Hat | Why don't we just get some sort of "moderation reputation" based on moderationy actions? | |
Jun 11, 2014 at 14:46 | comment | added | Sam |
@Payeli Yes, but to get into the queue you would have to have a good accepted:rejected edit ratio, so robo-editors would also be stopped (or at least reduced).
|
|
Jun 11, 2014 at 14:44 | history | edited | Sam | CC BY-SA 3.0 |
added 13 characters in body
|
Jun 11, 2014 at 14:43 | comment | added | Infinite Recursion | Thanks @ChrisF. Understood now. But then robo-editors will increase, they will race to make more edits->access the queue->get badges. Currently there is only one incentive for robo-editors, that's rep. Then there will be two incentives, rep+queue access. 2k users will make unrestricted edits just to get access to queues. Pushing users to make edits for the sake of being eligible to review later, that's scary. | |
Jun 11, 2014 at 14:28 | history | edited | Sam | CC BY-SA 3.0 |
added 24 characters in body
|
Jun 11, 2014 at 14:28 | comment | added | ChrisF Mod | @Payeli - it's not about restricting people making suggested edits, but restricting people reviewing them. | |
Jun 11, 2014 at 14:27 | comment | added | ChrisF Mod | @Sam - sounds good. | |
Jun 11, 2014 at 14:24 | comment | added | Sam | @ChrisF Ok, so how about, "they have made >X suggested edits, or have and edit badge"? | |
Jun 11, 2014 at 14:23 | comment | added | ChrisF Mod | @codeMagic - don't get me wrong, I'm annoyed about bad reviews too, but a suggested change like this needs to be solid. | |
Jun 11, 2014 at 14:20 | comment | added | codeMagic | @ChrisF "sorry, you may no longer review suggested edits --Community". But I only have 37 so I may not make the cut either. Even though I probably have a higher reject:accept ration than many others. But, in all fairness, Sam did say he didn't expect all of these to be implemented and I like the effort because I'm tired of reject silly/bad edits just to see them approved. | |
Jun 11, 2014 at 14:17 | comment | added | Infinite Recursion | I used to contribute edits before registering to SO for a long time, as per your scheme, unregistered users won't be able to propose any edits. They don't do it for rep, their contributions are often very good and honest. | |
Jun 11, 2014 at 14:11 | comment | added | ChrisF Mod | I appreciate that - but as an "old hand" I haven't made any suggested edits so your test is insufficient. I do have the "Strunk & White" and "Copy Editor" badges though.... | |
Jun 11, 2014 at 14:07 | comment | added | Sam | @ChrisF People that want to review edits should have at least made an edit and know how to make edits. Otherwise how else can you better guarantee that the user isn't going to approve a bad edit? | |
Jun 11, 2014 at 14:05 | comment | added | ChrisF Mod | "they have made >X suggested edits," - I haven't made any suggested edits as I got to 2K rep before they came in. So under your scheme I wouldn't be able to access the suggested edits queue. | |
Jun 11, 2014 at 14:00 | history | edited | Sam | CC BY-SA 3.0 |
edited title
|
Jun 11, 2014 at 13:53 | history | asked | Sam | CC BY-SA 3.0 |