Timeline for Staging Ground Beta 1 Recap, and Reviewers needed for Beta 2
Current License: CC BY-SA 4.0
8 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Mar 10, 2023 at 8:34 | history | edited | NoDataDumpNoContribution | CC BY-SA 4.0 |
added 572 characters in body
|
Mar 7, 2023 at 6:43 | comment | added | NoDataDumpNoContribution | @KevinB "spend more than 30 seconds" 30 second really might be too optimistic. After all even reading alone of a question takes time, not to mention thinking about it and commenting on it. A couple of minutes sounds actually realistic especially if you need to come back and do multiple rounds of review per question. Maybe 3-4 minutes instead of 5-6 would be possible, but only if you are super efficient and focused. | |
Mar 7, 2023 at 6:38 | comment | added | NoDataDumpNoContribution | @YaakovEllis Sure there are variances but didn't you report average durations? I simply divide an average by a percentage. On average it really should be 15-20 minutes on total effort to produce one graduated (approved) question. Just think of the whole system as a blackbox with an input (reviewer time) and an output (approved questions) and relate input to output on average. I don't see where I would go wrong. | |
Mar 6, 2023 at 22:20 | comment | added | Yaakov Ellis StaffMod | Also, I don't know if you can extrapolate even from these numbers that there are 15-20 minutes spent per approved question. Posts that auto-graduated got almost no comments and review time. And of the deleted posts, most of them got significantly less reviewer time. Also, posts that were ok to be approved with no major changes (which are part of Approved) take much less time. The posts that take 5-6 minutes are posts that start off with a blocking review, and end up either approved, or abandoned (after a few feedback rounds). | |
Mar 6, 2023 at 22:19 | comment | added | Yaakov Ellis StaffMod | 5-6 minutes is the sum of all interactions, over initial read, and 2-3 comments (on average). I would also like to caution taking these numbers as anything other than a sign of "what happened in Beta 1". We are definitely working on adding features (like decline re-eval, and comment templates) that can lower the amount of time that reviewers spend per interaction. And I am definitely expecting that Beta 1 reviewers were spending more time than folks would after a wider release, simply due to their enthusiasm for exploring the new features and spending more time figuring things out. | |
Mar 6, 2023 at 21:32 | comment | added | NoDataDumpNoContribution | @KevinB The "market" will decide what is an acceptable time investment per graduated (approved) question. If I understood it correctly roughly 1% of incoming questions were handled by 50 reviewers in beta-1, so roughly 5000 reviewers in total would be sufficient for all questions if they will be able to deliver the amount of time they invested in this test for a long duration. Otherwise most questions will simply be auto-graduating. | |
Mar 6, 2023 at 20:49 | comment | added | Kevin B | i'd find it a bit concerning that for a given question the reviewer would be expected to spend more than 30 seconds on a given question, much less than the 5-6 minutes that has been reported as the average. Hopefully this is just evidence that the process hasn't been ironed out yet, rather than an acceptable avg amount of time. At the current rate i'd expect most questions to "graduate" without review at all if this were opened up to all questions. | |
Mar 6, 2023 at 20:44 | history | answered | NoDataDumpNoContribution | CC BY-SA 4.0 |