Timeline for 1, 2, 3...test. Let’s increase the number of reviews & close votes for science!
Current License: CC BY-SA 3.0
24 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Mar 17, 2016 at 20:48 | comment | added | Shog9 | Regarding the accuracy of the automated heuristics... Might wanna have a separate thread for that, since posting data in comments is not going to work terribly well. | |
Mar 17, 2016 at 20:17 | comment | added | Servy | @NathanTuggy That seems like way more work, for, at best, the same benefit. | |
Mar 17, 2016 at 20:15 | comment | added | Nathan Tuggy | @Servy: I have an idea I don't think I've posted yet for using sorting to clump heavily delvoted reviews toward 20kers and heavily recommended reviews toward others, such that as few review actions of opposing type as possible are spent on a task dominated by a given type. I think that would be a bit safer than letting 2k review actions count as full, non-expiring downvotes. Prorating delvotes would be fairly harmless, though, by comparison. | |
Mar 17, 2016 at 20:12 | comment | added | Servy | @NathanTuggy Sure, or just count a "suggest delete" vote as half a real delete vote, but yes, I was aware that 3 delete votes on an answer would deleted it, and I'm also aware that that virtually never happens. | |
Mar 17, 2016 at 20:10 | comment | added | Nathan Tuggy | @Servy: I assume you mean prorate the delvotes, so a single delvote counts for two and two delvotes for four? Because 3 delvotes will already finish the review (whether cast in or out of the queue), as I expect you know. | |
Mar 17, 2016 at 18:15 | comment | added | gnat | @Magisch 60% insta-close is just the magic of system prioritizing CV queue (explained here). If we had something like that in LQ, life would be so much easier for everyone (human flags and items with larger amount of already performed reviews get higher priority, it's not a rocket science) | |
Mar 17, 2016 at 18:10 | comment | added | Magisch | @gnat I review c and sql CVs daily. They tend to be ~60% "I instantly close this", ~10% merits closer examination and ~30% Skip or leave open. | |
Mar 17, 2016 at 17:48 | comment | added | gnat | @Magisch I Skip these but this works worse than usual. First, because I'm afraid to miss something really worthy of deletion, like plagiarism and second, (likely most important) because these are mixed so randomly with straightforward garbage. Yet again, CV queue works so much smoother, it starts by giving me very easy items and gradually gets harder later so I can predict the load and manage my effort | |
Mar 17, 2016 at 17:14 | comment | added | NoDataDumpNoContribution | So the heuristics is wrong and produces more work instead of less? A good example for the importance of regularly monitoring the false positive rates of heuristics. | |
Mar 17, 2016 at 15:44 | comment | added | Brad Larson Mod | For example, these are all posts that were inserted into review by the heuristic this morning: stackoverflow.com/review/low-quality-posts/11669804 stackoverflow.com/review/low-quality-posts/11668147 stackoverflow.com/review/low-quality-posts/11669673 stackoverflow.com/review/low-quality-posts/11669865 and they all follow the "try this" pattern of answer. None are worth the time to review for deletion. | |
Mar 17, 2016 at 15:37 | comment | added | Brad Larson Mod | @bluefeet - I have noticed that the heuristic does tend to throw a lot of "try this: (code)" answers into review, because it skews towards shorter answers. These were many of the ones that I saw when I swept through and started wondering about the hit rate of the system heuristics. That's not to say that there aren't some people out there who blindly flag any answer with "try this" in it, but the system does seem to contribute more than a few of these. | |
Mar 17, 2016 at 15:28 | comment | added | Magisch | @bluefeet I feel horrible having to OK most of these posts, but answers are answers and them beeing bad doesn't make them delete worthy. Shouldn't people who go on bad flagging sprees get a flag timeout after so many declined ones? | |
Mar 17, 2016 at 15:27 | comment | added | Taryn StaffMod | @Magisch Flagging spree - bad flagging spree. | |
Mar 17, 2016 at 15:26 | comment | added | Magisch | @bluefeet I can attest to this beeing the case heavily. About 50-60% of the posts I review in the LQP Queue are "Try this: $code". Those are answers. Either the heuristic is not very good or a ton of people are on a flagging spree. | |
Mar 17, 2016 at 12:52 | comment | added | NoDataDumpNoContribution | Today in the VLQ I came across a zero scored question with four zero scored answers from 2009. Now anyone tell me that in the past everything was better on SO. | |
Mar 17, 2016 at 12:30 | comment | added | gnat | @Trilarion in my experience it's a friggin' problem in LQ queue, it breaks the flow and breaks it hard. This is especially noticeable when compared to CV queue which feeds me easiest items first. I wouldn't mind wasting my time, 20 (30) reviews a day isn't a big deal but the real harm is, as a result moderators get loaded by hundreds flags that could be handled by reviewers in a matter of minutes. I've seen blatant, textbook NAAs hanging in queue for hours until they eventually reach moderators - this just doesn't feel right | |
Mar 17, 2016 at 10:18 | comment | added | NoDataDumpNoContribution | @Jeffrey Bosboom Before we prioritize we should check if it is a problem at all. Maybe the heuristic is so strict that it identifies only positives (but not all of them). In that case maybe these should be prioritized and not ditched at all because it takes away work to identify them manually. First step is gathering data. | |
Mar 17, 2016 at 0:03 | comment | added | Taryn StaffMod | @JeffreyBosboom As I mentioned, I'm looking at a few different things... patience you must have. | |
Mar 16, 2016 at 21:44 | comment | added | Jeffrey Bosboom | If we aren't willing to ditch the heuristics entirely, can we at least prioritize the flagged posts (which a human has already judged to be a problem they organically found when browsing the site) over the heuristic-identified posts? (In a separate experiment, of course.) | |
Mar 16, 2016 at 18:49 | comment | added | ryanyuyu | @Servy that make a lot of sense. Since 3 20k users can delete a bad answer outside the queue, and it takes 6 "Recommend deletion" votes within the queue, it seems pretty logical that delete votes within the queue count for 2 "Recommend deletion" votes (2/6 = 1/3). | |
Mar 16, 2016 at 18:43 | comment | added | Brad Larson Mod | @bluefeet - Oh yeah, best to change one variable at a time. A few moderators were talking about this stuff yesterday, so I thought I'd toss these out as possible future items to look at. | |
Mar 16, 2016 at 18:42 | comment | added | Servy | To the second point, having users with the 20k rep to cast delete votes count for more than the 1/5 of the users necessary to delete the question would help. If those delete votes, rather than recommend delete votes, counted for, say, double, that might help. | |
Mar 16, 2016 at 18:41 | comment | added | Taryn StaffMod | This is a first test to see if we can make an impact in the queue. I'm not sure if we will make other tweaks with this test. I will say that I'm looking at a few other things, but we want to test one thing at a time. However, I'll see about pulling some numbers of what's being fed into the low quality review queue. | |
Mar 16, 2016 at 18:36 | history | answered | Brad LarsonMod | CC BY-SA 3.0 |