17

I don't do a whole lot of reviewing work on Stack Overflow (I mostly answer, edit and comment). I was just checking out the triage queue and noticed I keep getting questions about PHP, Python, and WordPress.

Now, I barely know any Python or WordPress and know zero PHP. More concretely - I haven't answered questions about any of them, even (IIRC).

So why does it make sense to serve me questions about them? (And it's not as though they're also about abstract subjects in which I do have answers.)

(And yes, I realize I can skip. But I skip, skip skip and get more and more of them.)

26
  • 9
    Triaging (unlike other types of reviewing) should not require subject-matter expertise, so, yes, this is normal and reasonable. See also: meta.stackoverflow.com/questions/297317/…, meta.stackoverflow.com/questions/393298/…, meta.stackoverflow.com/questions/288874/…, meta.stackoverflow.com/questions/364975/… Oct 30, 2020 at 23:47
  • 11
    If you are not sure what the correct option is just skip the review. Oct 31, 2020 at 0:06
  • 10
    @SecurityHound They're aware of that, they even say "(And yes, I realize I can Skip. But I skip, skip skip and get more and more of them.)" - The question is, why show questions on which they may have no knowledge at all? Oct 31, 2020 at 0:10
  • 3
    @Nick - How can an automatic system know what topics the author knows. I know about tons of topics, never answered a question about them, and I can certainly tell the difference between a good question and bad question on topics I know nothing about Oct 31, 2020 at 0:15
  • 4
    @SecurityHound Well it could look at tags you've answered/asked questions in for one example (questions you've proven yourself to have some knowledge in), but it doesn't. And that's the point, because you don't need knowledge, I was just explaining how your comment "just skip the review." is unhelpful, OP knows that. Oct 31, 2020 at 0:20
  • 6
    @SecurityHound In addition to the skip button, maybe we could have a "Skip all questions with [tag]" button.
    – user3657941
    Oct 31, 2020 at 0:22
  • 2
    I can't help but notice that CodyGray linked to some old posts about Triage. The problem here is that is seems users have been getting review suspensions for making mistakes in Triage as if they should have been making decisions as subject matter experts. So I'm not sure those are valid arguments.
    – Scratte
    Oct 31, 2020 at 8:29
  • 4
    @SecurityHound Compared to the other queues, the amount of posts that one has to Skip in the queue is ridiculously high due to not being able to filter posts. I assume you already well know that if you review even a small amount.
    – Scratte
    Oct 31, 2020 at 8:32
  • 3
    @Scratte - It doesn’t take a SME to determine if a question meets the minimum requirements for a question. Oct 31, 2020 at 10:35
  • 1
    @einpklum Well, the subject of closing questions is germaine to who is doing the closing. When a new "triage guy" (which, as I said, I was once... but never did it again) starts reviewing questions with languages he doesn't even know - it is only encouraging the telling of new Stack Overflow users that they aren't competent "question askers." Having a giant infrastructure that encourages people to tell new users their questions are bad, rather than a system that encourages providing answers to questions is why people say Stack Overflow isn't a very nice place.
    – Y2020-09
    Oct 31, 2020 at 12:59
  • 1
    TL;DR Why would you ever close a PHP Question if you don't even know PHP? Very often the problem is the little details you never thought about.
    – Y2020-09
    Oct 31, 2020 at 13:07
  • 7
    We're not toxic or callous @TheLeb, people just don't read instructions on how to write a good question. We're not here to help individuals, we're here to make a library of knowledge to help the greater population Oct 31, 2020 at 14:15
  • 4
    "Why would you ever close a PHP Question if you don't even know PHP?" <- many reasons. No code while asking about why the code doesn't work; the English part of the question is unintelligible; a "can you write my program for me" question; request for a specific piece of software. But I better understand what you mean now. Oct 31, 2020 at 14:17
  • 2
    @SecurityHound It absolutely does take an SME to see if a post has enough information be OK with an edit from someone else than author. How do you know if there's enough information or if the post is really a typo and should be closed if you're not familiar with the tag?
    – Scratte
    Oct 31, 2020 at 17:48
  • 1
    @SecurityHound Reviewing based on "meets the minimum requirements for a question" is likely to get one review suspended. One needs to be sure in that queue. I'm also not comfortable with the message that comment sends to new reviewers, as they are the most likely to fall into that trap of thinking they can tell if a Question is fine based on this metric. The best option is to have a filter, so one doesn't need to skip 10 to review one post, as I did.
    – Scratte
    Oct 31, 2020 at 18:02

2 Answers 2

5

The fact that the Triage queue questions appear unrelated to the domain of expertise of the user seems unreasonable to me. I suggested a possible fix elsewhere. I am reposting it below, adding a few substantial improvements and changes.

The Triage queue should show more questions related to the "watched tags" of the reviewer. Alternatively, the Triage queue should show more questions with tags that are related to the tags appearing in the threads of the reviewer (questions, answers and comments by the reviewer).

Related tags are determined by their frequency of co-occurrence within the same question across the entire Stack Overflow. Using this definition, and are more related than, for example, and .

I understand that we want to triage all questions regardless of whether they have tags that are watched by the users. But the queue should not be entirely random. Instead, the Triage queue should be enriched up to a certain percent (between 0 and 100%, the optimum can be determined experimentally) by the questions with the watched or related tags.

Spiking in non-random questions into the Triage queue is not a novel idea. You occasionally get "test" questions designed to check "if you are paying attention" anyway under the current system. I simply suggest another type of spike-in questions - the "more related" type of questions. This would result in more meaningful reviews and less "skip, skip, skip" type of behavior.

SEE ALSO:

Something similar has been suggested many times before, see below. But the previous OPs asked mostly about using plain filters based on tags (which would limit the Triage queue to 100% watched tags, or something similar). I suggest a level of spike-in less than 100%, so that rarely watched tags or untagged questions get enough reviews as well.

Filter the Triage review queue by subject/expertise
Why is there no filter in triage review?
I only want to see items related to my favorite tags in the review queues
Review triage questions accordingly to my favorite tags

6
  • 1
    You're not being asked to assess the question content but the structure, is it clearly defined?, does the title make sense?, if it's a debugging problem has at least some code been included? None of this requires domain knowledge of the subject matter, just a bit if common sense.
    – user692942
    Oct 31, 2020 at 19:54
  • 1
    @Lankymart I think we should try to optimize the rate of reviews and the accuracy of reviews in the Triage queue. Clearly the OP and I both reduce our rate and/or accuracy under the current system. I suggest one possible way to optimize the system. Oct 31, 2020 at 21:14
  • By allowing people to review their watched tags you end up watering down the review process. Resulting in tags with high traffic seeing more reviews completed while low traffic tags will be left to stagnate in the review queue mire.
    – user692942
    Oct 31, 2020 at 21:17
  • 1
    @Lankymart I think that we should optimize the rate and accuracy of the reviews of all tags. I simply suggested to spike in the user-related tags. The spike in percent can be pretty low, most likely a lot less than 100%. Also I suggested to use related tags, not necessarily the watched tags. That’s a much broader selection of questions. Oct 31, 2020 at 22:16
  • Favourite, related the result is the same, you water down the review pool.
    – user692942
    Oct 31, 2020 at 22:21
  • 2
    @Lankymart I think that to resolve the difference in our views, SO should do experiments to optimIze the metrics that they care the most about. I could be wrong, but it may be that the optimal % of user related questions spike in could be not the current 0% but somewhere around 5-50%. Let the data guide our decisions! Oct 31, 2020 at 22:32
4

While you don't have perhaps the domain knowledge to answer the question, you do have the community knowledge to tell if the question is properly written.

It should be easy to tell questions that have formatting issues, have no code, are spam or blatant off-topic.

The problem goes for those questions that are, theoretically, properly written, but may lack some details that you're not aware of due to your lack of knowledge in the framework. For example, let's assume you have no idea on Docker, a user posted a docker-compose.yml file but no Dockerfile. You can't tell that file is required for Docker, because you don't know that fact.

So yes, you should skip those questions if they apparently seem well-written, and let users more knowledgeable tell if the question needs more detail.

Is it bad that SO gives you questions like that in triage? not necessarily. Maybe you cannot complete triage on questions out of your knowledge, but you are still able to quickly turn out the spam and very low quality content. Giving you these questions reduce the time a low quality post needs to be removed (since it reaches a broader audience), perhaps at the expense that some better detailed post missing details take a bit longer to close. That's OK to me, we're tackling down the most problematic questions first.

If a post without key details but well-written survives 30 minutes, maybe some people will already tell the author the missing details and flag the post anyway without it being in the triage. But a spam or malware post surviving 30 minutes is way more critical for the community.

2
  • 7
    But wouldn't it be better to direct questions to more-domain-knowledgeable people rather than uniformly? ... unless there's queue overflow (pun not intended) on some of the tags, I mean. Fair point regarding spam and blatant off-topics - although I've never gotten one of those yet to triage. Oct 31, 2020 at 14:13
  • @einpoklum No, because simply put, there isn't enough reviewers and by filtering the reviewers by their favorite tags you just end up watering it down further.
    – user692942
    Oct 31, 2020 at 19:56

Not the answer you're looking for? Browse other questions tagged .