21

Triage is intended to limit visibility of the questions ("take that, wall of crap").

Wonder how efficient is the current implementation in maintaining that - specifically, how many questions manage to get answers while under review?


My understanding is, this could be found by analyzing the question timeline and counting occurrences of answer posted events that happen between events indicating start and end of the triage review.

This probably needs developer access because it appears to involve too many deleted questions to get reliable estimation from data available to regular users.

7
  • 1
    this visibility limitation is apparently not intended to be waterproof: for example any triage reviewer can get to the question using direct link and answer it. I am primarily interested to figure how much it "leaks" and amount of answers posted looks like a reasonably reliable indication for that
    – gnat
    Commented Jul 24, 2018 at 7:41
  • @pnuts my understanding is, no personal data is needed really: triage start and end events with timestamps (note question can pass through triage more than once) and answer posted events with timestamps, including deleted answers. I haven't seen this data in SEDE, did I miss something?
    – gnat
    Commented Jul 24, 2018 at 10:20
  • 1
    This is mostly a question of how many people answering questions use the homepage versus search, the tag lists, or other means. Only people primarily using the homepage to find questions to answer are affected.
    – Servy
    Commented Jul 24, 2018 at 14:42
  • 1
    @Servy it is a bit fuzzier than that because although the post I refer mentions only homepage I think I saw somewhere that some other pages also ignore it. A while ago I even tried to figure more details about that but recently realised that instead of drilling into this cumbersome stuff it is easier to just get the "aggregate stats blackbox style" instead. FWIW I also considered asking for views instead of answers but dropped that idea because it's too dynamic (triaged question in theory can improve to great shape, then collect hundreds views because of that, this is all too complicated)
    – gnat
    Commented Jul 24, 2018 at 14:50
  • @gnat I'm not saying you shouldn't look at the stats, I'm just saying that that's what the stats are going to show you. The magnitude of the different is going to indicate what percent of answerers primarily use the homepage to look for questions to answer.
    – Servy
    Commented Jul 24, 2018 at 14:52
  • agree @Servy that's about what I expect (as I wrote I'd prefer views as these directly relate to my question about efficiency but views turned out too complicated for reliable estimate so I switched to metric that appears more likely to get right)
    – gnat
    Commented Jul 24, 2018 at 15:34
  • I went to triage, got this question stackoverflow.com/review/triage/20406517, saw the question with 9 views and in the new c++ list stackoverflow.com/questions/tagged/c%2b%2b
    – Braiam
    Commented Jul 25, 2018 at 19:25

1 Answer 1

3

With the help of Nick Larsen, from our Data Team we (by which I mean he) pulled some numbers.

All time:

IsDeleted   AnsweredDuringReview    Total
       no                 606975    1566312  (38.7%)
      yes                  92199    1079537  ( 8.5%)

And below are some monthly breakdown graphs:

Graph: Review Queue Effectiveness by Month

Graph: Review Queue Answer Percentage by Month

A few notes:

  • About 10% of questions that end up in that queue, end up in there more than once. And there are questions that get answered after coming out of the review queue as well, the "All time" number above refers only to questions that were answered while in the queue.
  • There are a lot of confounding factors that contribute to these results, most notably average duration in the queue and average time to answer on the site as a whole over time, and many others, none of which were controlled for.
1
  • 1
    Did the welcome wagon stop us from deleting posts from review?
    – Luuklag
    Commented Apr 3, 2019 at 11:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .