-23

Can we please not use questions for review audits, at least in the close and reopen vote queues?

Audits with the Haskell tag have caused a lot of disagreement over the years. Here are some meta questions, all highly upvoted showing the community's disagreement with the audit, that I found showing the issue:

It seems that the Haskell community on SO allows more broad questions than most questions using other tags, which means that people not familiar with the tag or related questions (the majority of reviewers) will likely fail audits related to Haskell, not because they deal with Haskell topics, but because the questions are more broad than questions using other tags on Stack Overflow. Although we don't know the formula behind how audit questions are picked, it likely includes the length of the question and, since the questions are more broad, Haskell tagged questions tend to be longer which makes them more commonly picked to be audits.

The trouble brought by Haskell audits have caused some users to skip reviewing questions in the queue altogether. If we prevent questions with the tag from being audits, some Haskell tagged questions may be closed when the Haskell community doesn't want them to be, but they can be easily reopened because few will be wrongly closed. Should having separate communities within SO that follow different rules even be allowed?

Can we please prevent questions with the Haskell tag from being audits?

9
  • 33
    It would be better if, you know, the Haskellers followed the same rules as everyone else. When I am trying to solve a Haskell problem, half the time the questions/answers I find aren't even useful.
    – cimmanon
    Commented Jan 14, 2015 at 12:46
  • 17
    if people don't understand Haskell maybe they shouldn't be reviewing those posts. there's a skip button for a reason. Commented Jan 14, 2015 at 14:19
  • 8
    @NathanHughes While it does vary based on the particular type of review action, most of the queues are specifically designed such that domain knowledge isn't required in order to review the particular actions properly. While there are occasional exceptions, and one needs to be able to recognize when they come across one and skip the review, skipping all posts in other tags in all queues is just not sensible.
    – Servy
    Commented Jan 14, 2015 at 15:22
  • 8
    @NathanHughes You don't have to understand Haskell to know that a question like this one should be closed. But before Zach ran into it in an audit, tried to close it and got a failed audit, it was not closed. Removing Haskell questions from audits may not be a solution but nor is skipping questions that should evidently be closed.
    – Louis
    Commented Jan 14, 2015 at 15:33
  • 6
    I don't think we should omit tags from audits; instead, we should address the underlying issue, which seems to be that the haskell community needs cleaning up.
    – TylerH
    Commented Jan 14, 2015 at 16:22
  • 1
    @ZachSaucier The problem with ultra broad questions isn't that nobody wants to see a great answer to them; it's that they don't actually result in great answers when asked on SO. People don't go through the effort of writing a book on a tpoic only to post it as an answer to an SO question, nor does an SO answer have the tools to display a book's worth of information. When you need a book's worth of information, you just need to go put it in a book.
    – Servy
    Commented Jan 14, 2015 at 16:38
  • 3
    "It seems that the Haskell community on SO allows more broad questions than most questions using other tags, which means that" the Haskell community needs a reminder or two. Commented Jan 15, 2015 at 10:06
  • 1
    To be fair, getting people to skip Haskell reviews unless they're intimately familiar with Haskell is a good thing. It makes it more likely that the tasks will be left for the people best qualified to assess them correctly, instead of by people with zero or little familiarity with the language. Of course, there will always be questions that you can accurately assess even without being familiar with the language (generally, either really good or really bad ones), but not every task will be as easy to examine. Commented Dec 5, 2016 at 21:08
  • related: I do not understand why I failed this audit
    – gnat
    Commented May 23, 2019 at 9:02

3 Answers 3

36

people not familiar with the tag or related questions (the majority of reviewers) will likely fail audits related to Haskell.

...

The trouble brought by Haskell audits have caused some users to skip reviewing haskell questions in the queue altogether.

Wait, lemme see if I have this right... Folks who don't understand Haskell questions are deciding to stop reviewing Haskell questions because their lack of knowledge results in them repeatedly failing audits?

good.

That's why there's a Skip button. So if you don't know what you're looking at, you can leave reviewing it to someone else. Doesn't mean you need to be an enthusiastic Haskell user to review these questions, but if you're not even willing to do a bit of research then... Well, there's plenty of other questions in need of your attention.

11
  • 16
    The post is asserting that the audits are not being failed due to the reviewer's lack of knowledge of Haskell, but rather than the Haskell tag is moderating its contents under different guidelines than the rest of SO. If a question is Too Broad (in a way that's clear even without specific knowledge of Haskell) given SO's generally accepted guidelines, but the Haskell tag upvotes those questions and doesn't close them, the user fails an audit that they didn't actually lack sufficient knowledge to appropriately review.
    – Servy
    Commented Jan 14, 2015 at 16:59
  • 1
    one thing left to really accomplish that friggin' mission is to find a less "destructive" way to teach reviewers that There is no shame in using “Skip”. I mean, oh great that audits eventually teach them, but how many non-audit posts will be reviewed wrong before this happens
    – gnat
    Commented Jan 14, 2015 at 16:59
  • 3
    All-haskell audits, all the time, for the first 10 reviews?
    – Shog9
    Commented Jan 14, 2015 at 17:04
  • 6
    I'm skeptical, @Servy. If you suspect a a question is Too Broad and you see that it has an answer (something Review tells you) then the fastest way to validate your evaluation is to read that answer - which I doubt the folks failing these are doing. Either way, the problem isn't really the audits then. I mean, if we weren't using these questions as audits they would still exist. If there's a problem with how Haskell questions are being handled, maybe bring this up as a separate discussion and invite some of the folks active in the Haskell tag to participate & work out a solution?
    – Shog9
    Commented Jan 14, 2015 at 17:20
  • 3
    As Servy mentioned, the problem with the questions is that they're overly broad because the Haskell community allows them to be. I made this more clear by editing my question Commented Jan 14, 2015 at 17:21
  • And as I told Servy, @Zach: that's a problem independent of /review.
    – Shog9
    Commented Jan 14, 2015 at 17:22
  • @Shog9 I agree with that in entirely; it's why I downvoted the question. If the Haskell Tag isn't moderating its content appropriately, then that root problem needs to be addressed, rather than trying to hide it by not using that content as audits. I just want to make it clear that my first comment is merely summarizing the question, and asserting that you're not answering it. I'm not actually sure if the problem it alleges is in fact real, or saying that I agree with all of its assertions.
    – Servy
    Commented Jan 14, 2015 at 17:22
  • I'm unsure as to how to appropriately phrase such a post. I'd love to see someone else create a post for it :) Commented Jan 14, 2015 at 17:29
  • You could start by asking for advice in chat, @Zach
    – Shog9
    Commented Jan 14, 2015 at 17:33
  • 2
    How long until Lance comes along and edits out the image completely?
    – BoltClock Mod
    Commented Jan 14, 2015 at 18:42
  • 2
    yeah. Bring them couple slippery audits for a start. Then, don't forget to suspend for failing. Would be a great way to teach wouldn't it. No need for carrot if you can use a bigger stick, right? If seriously, why don't you simply make 10-20 skips required for Reviewer badge (somewhat similar to additional requirement of 25% question votes for Electorate badge)
    – gnat
    Commented Jan 15, 2015 at 6:22
13

You know by now what my suggestion would be about how to deal with controversial audit cases. This is particularly applicable to close vote reviews, where there can be significant differences of opinion on what is and is not acceptable on the site.

That said, let me comment about the argument that the Haskell community isn't policing itself to the same standards as other tags. As someone who has been tinkering with Haskell lately, and thus reading a good number of Haskell questions, I don't see a quality or policing problem in that community at all.

Due to the nature of the language and the current community of practitioners, you will get questions that involve more theory than in others. Some people seem to be reading these questions as too broad or opinionated. However, the best [haskell] questions still tend to provide the code they've tried and a clear statement of what they want to do.

See the subject of this audit dispute, where I think people are sticking too close to the letter of the law rather than the spirit of it by reading "What's the "right" way to do this in the Haskell world?" as soliciting open-ended opinions. I don't think the Haskell community is failing to police themselves by leaving that open. On the contrary, I think the rest of the community may be getting too restrictive if a question like that is no longer allowed.

2
  • 7
    THERE IS NO SUCH THING AS TOO RESTRICTIVE twitch
    – BoltClock Mod
    Commented Jan 14, 2015 at 17:58
  • 1
    one option for haskell folks is to protect their questions from being presented as known good close audits by bringing their own “human factor” into review audit composition/selection. Per my recollection of audit selection algorithm, single close vote (retracted immediately after casting) would guarantee that these won't be presented as known good close audits
    – gnat
    Commented Jan 16, 2015 at 7:39
7

TL;DR, NO, don't remove Haskell questions from the queue. If you're not sure whether a Haskell question is a good one, skip it. Actually, if you're not sure if any question is a good one, skip it. If it's indeed a bad one, it will attract attention sooner or later again, but killing a potentially nice topic just because you thought it might be a bit opinionated is IMHO not acceptable.


My two cents as a quite recent contributor.

There's one thing I want to get out of the way before saying anything else, and that's that I believe Haskell tag on SO is indeed quite different from tags for other, mainstream programming languages. There's a lot of possible explanations, but in short, because it's still a niche language, the posts of "I'd like to solve an (easy) problem with my code" appear in a much lesser proportion to more open posts about language and code design. My opinion is that we, as a community, are mostly learning haskell, learning how to approach problems with it, and there's little "rule-of-thumb" solutions to known problems.

Heck, I've asked a potentially problematic question not later than yesterday! I think it showcases the fact there's a lot of openness in the community about reasons and discussing the topics that (bear with me) whilst not perfectly practical in a given minute, can bring huge benefit to the community in terms of better explanation of why, not how things happen.

This can certainly be confusing for a reviewer, appearing as an endless, open-ended debate. In practice though, I haven't yet seen posts that would drift into oblivion of opinionated guessworking. If we were to kill any post that goes even slightly beyond "Specific programming problems", that could effectively mean that a lot of theoretical Haskell questions would have to be asked on ComputerScience.SE, but I hardly see them as belonging there. Let's keep SO a place where a creative process is still allowed to happen.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .