-25

There's been much discussion on how to reduce low quality posts on the site. Both as questions and answers.

I was having a discussion with someone about how we can improve the instructions to new users of how to post a question or an answer and it dawned on me, that in many cases, such instructions will just fall on deaf ears. For whatever reasons, language barriers, poor programming skills, unrealistic expectations and of course spammers and trolls - let's call this the "unfixable posters".

Improving the Ask a Question page will help some groups with language barriers and to some degree new (and old) programmers with the ability to follow instructions:
Let's improve Stack Overflow's "Ask a Question" page! (<-- we need to do this btw).
However, the remainder of the unfixable posters will just bypass any instructions and post rubbish.

Using minimum rep to post is unlikely to assist:
Low-rep users should not be able to ask questions

Restricting registration is unlikely to assist:
Should Stack Overflow be more restrictive about new user registrations?

What can we do to filter out these types of users without excluding potential newcomers to our community?

I propose a questionnaire pop up, when asking or answering a question for the first time. If it's done carefully it may assist in slowing down spammers and trollers. Although it would not be difficult to Google the answers, at least that would require an effort.

Much like a captcha, but to determine a minimal understanding before participating on the site.

The questionnaire would have to be carefully formulated and I wouldn't attempt to do that alone, but to give an idea of what I have in mind it could test that the user has a minimum level of knowledge of programming and basic mathematical skills. It would also serve as a basic English skills test.

These questions are a basic example:

  1. Name three programming languages.

  2. Name a programming IDE.

  3. What is 67 + (6*3 + 9)/2 - 1000?

and so on.

The questions could be asked as a series, so the user cannot proceed until that question is successfully answered.

This system, may also be helpful in reducing spam, maybe not trolls so much, although it could become tedious for them to have to continually jump through questions.

Some examples of low quality questions:

https://stackoverflow.com/questions/45884241/android-java-whats-program

enter image description here

Shell programming - matrix multiplication

enter image description here

Why this question isn't a duplicate?

This question is similar, but a lot broader and discusses many other features of the site- also it was asked over 5 years ago and the site stats have changed.
Should SO have a prequalification process for membership to weed out the 'noise'?

This question is similar, but different - it asks to do this across the board for all low rep users.
Programming Quiz to be able to ask questions

23
  • 2
    Make them answer the latest question on project euler. I like the idea but why would the questionnaire be different for a captcha?
    – George
    Commented Aug 25, 2017 at 9:36
  • @George because it would require programming knowledge (and a lot of regex) to pass it.
    – user3956566
    Commented Aug 25, 2017 at 9:37
  • 1
    But you even said yourself it would not be difficult to Google the answers
    – George
    Commented Aug 25, 2017 at 9:41
  • 4
    @George at least they're using Google! That's something some people don't seem to know how to do.
    – user3956566
    Commented Aug 25, 2017 at 9:43
  • 1
    @Keiwan I've edited the question - if you still think it's a dupe, that's fine
    – user3956566
    Commented Aug 25, 2017 at 10:00
  • 8
    I like the spirit, but I have some serious reservations about this proposal. Too long for a comment, too lazy to expand into an answer. But I think it would be better if we give everyone the benefit of the doubt and only exclude future posts when a problem has actually been identified. In other words, exactly like our current automatic bans, but tighten up the metrics. That solves the problem without unfairly hobbling new users, with the only cost being a couple of possibly low-quality questions. I remain dubious that a little quiz is going to prevent those anyway.
    – Cody Gray Mod
    Commented Aug 25, 2017 at 10:48
  • 1
    as currently stated your suggestion looks a bit too indiscriminate given that system already has ability to detect that question is going to be troublesome (and uses it, although not as efficiently as it could). I think that popping such a questionnaire right before pushing question to triage would make better sense
    – gnat
    Commented Aug 25, 2017 at 10:50
  • 1
    @CodyGray see above ^^^ - giving benefit of doubt makes sense except for maybe cases when system already gave up that doubt and is going to push the question into triage (cases like this probably qualify as "when a problem has actually been identified")
    – gnat
    Commented Aug 25, 2017 at 11:02
  • 1
    there you go - wrote it as an answer
    – gnat
    Commented Aug 25, 2017 at 12:50
  • 2
    I've gotta admit...I'm genuinely surprised to see a suggestion like this come from you.
    – Makoto
    Commented Aug 25, 2017 at 15:05
  • 2
    Turning up the quality filter would easily fix the examples you cited. They should not have ever gotten submitted, as they do not meet a reasonable minimum length, especially when you strip out hyperlinks.
    – Cody Gray Mod
    Commented Aug 25, 2017 at 15:22
  • 1
    upon re-reading more accurate title would be like "What can we do to filter out problematic types of users before requiring them to pass a quiz at first question". Also I have a feeling that tag feature-request is somewhat superfluous here, possibly discussion would be a better fit
    – gnat
    Commented Aug 25, 2017 at 16:45
  • 1
    @gnat I've rolled it back so you can undelete your answer
    – user3956566
    Commented Aug 26, 2017 at 3:26
  • 3
    Another problem is that any skilled/experienced developers who turn up to maybe answer questions, (ie. potentially extremely valuable members), are faced with 'trivial, timewasting interview questions from a place that I now don't want to work for', and may just not bother and turn to doing something else, like paid work. Commented Aug 26, 2017 at 8:42
  • 1
    Rather than programming or mathematical questions, ask questions related to the website. Like a multiple choice question from "How to ask page" or "Is this a duplicate for this question" etc. In this case even if they google, they have to go through the rules. Commented Aug 28, 2017 at 17:40

5 Answers 5

13

This proposal, like many others before it, all suffer from the same fatal flaw:

determine a minimal understanding

Let me be as blunt as I can here.

"Minimal understanding" is subjective.


I'm a software engineer with five years of experience in the industry. I've participated on this site for a little over five years. When I first came on, the understanding I had was limited to what I knew from college course work and my own experimentation with Python, and I used that to great success.

When I asked my first question here, any number of arguments could be made that I didn't have a "minimal understanding" of what the problem domain was.

  • Was the issue IO bound or CPU bound?
  • Was I experienced enough to understand any potential answers coming my way? (Obviously not)
  • Would I get the most value from an answer here, or would I be better served on some godforsaken forum elsewhere?

Or what about one of my more recent questions? An argument could be made for me not having a "minimal understanding" of the framework I'm using.

  • I had failed to check the actual produced JAR for any duplicates - I had thought that simply wasn't a thing.
  • I had considered the Oracle forums to see if they knew anything or had any insights, but stopped short due to their relatively slow response time and lack of cohesion.
  • Six answerers simply could not figure out what it was I was trying to get at, which led me to think that I was the one with the communication issue. (Turns out, they were guessing wildly.)

Or what about my worst question? I had demonstrated some understanding of Rails, and none of the patterns that Rails teaches you even apply to something like this. So, I sheepishly asked the community if they knew this was even possible.

  • Oh, I got slaughtered on this question. Worse, there's nothing I can do to fix it; that was a job ago and that code base is long since gone anyway.

So I'll say this again.

"Minimal understanding" is subjective.

Of the three questions I've provided for you above, what test could possibly be administered to determine that a potential OP has some level of minimal understanding to even ask this?

Let me tell you what wouldn't be useful:

  1. Name three programming languages.
  2. Name a programming IDE.
  3. What is 67 + (6*3 + 9)/2 - 1000?

None of these questions tell you if the person is competent at programming, or capable of answering a test. And I truly doubt that any test that could do that would mean that we'd get better questions. Quite the opposite; now that a decent-sized test has come up on Stack Overflow, we're not going to bother with Stack Overflow because asking is genuinely too much of a hassle.

I don't deny that there are a lot of crap questions out there. But there's no way to pre-screen the kind of askers you get here that will truly evaluate their level of experience or intellect.

And we haven't even begun to talk about ESL...

19
  • You make a good point -food for thought - I honestly thought it was a great idea when it was in my head hm. Btw your most recent question link is 404ed.
    – user3956566
    Commented Aug 25, 2017 at 22:30
  • I've fixed the link now. Feel free to peruse.
    – Makoto
    Commented Aug 25, 2017 at 23:13
  • Related
    – Shog9
    Commented Aug 25, 2017 at 23:33
  • LOL I see all the answers. Now tell me - why were you surprised? I genuinely want to know :)
    – user3956566
    Commented Aug 26, 2017 at 3:25
  • @Shog9 can we please make it easier to clear the close queue? both to close and delete low Q posts? Pleeeeease for the love of God .. can we solve this solution already? We need to reduce the number of close votes to 4 until it's manageable
    – user3956566
    Commented Aug 26, 2017 at 3:28
  • I've accepted this answer for two reasons: Firstly the subjectivity of it and also a comment Jon Clements made: .../...my concern would be putting genuine people off before they even post. I'm not sure how I'd feel going to a site with a well defined and researched problem at hand and then being asked questions to prove I'm worthy of doing so - I may not bother. Also - note to @Shog9 , maybe we can fiddle the heuristics to bump more posts into the first post and triage queues
    – user3956566
    Commented Aug 26, 2017 at 3:31
  • 2
    @YvetteColomb: Initially, I was surprised that you would suggest something which has had one or two incarnations of some form on Meta. Restricting users from posting has always been a thing around here, either with good or bad intentions. But I think the larger surprise was that...it almost seemed like this played into the common stereotype of new users, which was one I would want you to rise above. I get that some questions are just awful, but what you were proposing only isolates the community without solving any actual problems. It caught me off guard that you'd want to go that route.
    – Makoto
    Commented Aug 26, 2017 at 3:34
  • @Makoto ah I see, I understand. I guess I'm just pushing a bit for us to come up with a working solution to address poor posts. It's time consuming working with new users, but we don't want to put them off, I agree. We cannot keep poor content on the site either. The good thing about this post is, the community wants to filter after people have posted- so how can we best do that? It would be good if we could remove stuff more quickly, it can always be undeleted and reopened, if well edited. What do you think?
    – user3956566
    Commented Aug 26, 2017 at 3:42
  • 3
    @YvetteColomb: I think the community needs to find a better use for the pitchforks. I get that questions are rough but it isn't like that's an unknown thing. Machine learning may come in handy with dealing with this...but until then, what we should be focusing on is what's in our control; helping the users that have a shot and dealing with the users that don't in the most efficient way we have now - downvoting.
    – Makoto
    Commented Aug 26, 2017 at 4:29
  • 2
    We do need better (or at least, more up-to-date) heuristics, @YvetteColomb. It's trivial to put more posts into Triage; it's NOT trivial to put more posts into Triage that are likely to actually be problematic. Big fat problem w/ the close queue is always that folks use close votes or flags as a "super downvote" for things they don't like; tons of questions in there whose biggest fault is just being boring, makes it hard to identify those that are actually causing problems.
    – Shog9
    Commented Aug 28, 2017 at 17:06
  • @Shog9 I totally agree about the close vote queue - it drives me crazy seeing some of the posts people want to close - because it's too new or doesn't fit their idea of what a good question is - even though it may be perfectly clear and answerable to people who develop in that tag. The burninations are also an open slather and used as an excuse to just delete stuff that's sitting on the site causing no harm and in fact may be helpful. The whole close voting thing is another thing I wanted to address - I have a to do list. When you say posts "likely to be problematic", what do you mean?
    – user3956566
    Commented Aug 28, 2017 at 21:47
  • 1
    The greatest sin you can commit on Stack Overflow is wasting someone else's time, @Yvette. Questions that are off-topic, misleading, tar-babies, or change while being answered are the biggest culprits here; unfortunately, identifying them ahead of time is brutally hard - usually the best we can do is look for coincidental indicators like spelling, grammar, and code formatting.
    – Shog9
    Commented Aug 28, 2017 at 21:58
  • @Shog9 I hate the change while being answered - you should see the only meta post on my profile that actually really gets my goat and I jump on them fairly quickly. Yep it's an abuse of the site to take the programmers for granted that they are there to help as a free tutoring, debugging service or worse still - to write the code for someone else's job. "We help those who help themselves" - should apply.
    – user3956566
    Commented Aug 28, 2017 at 22:12
  • 1
    We need better filtering for ALL the question lists, @YvetteColomb. We've known that for years and even put significant research into trying to find solutions but all of that was abandoned before fruition; need to get back to it. Everything else is just re-arranging deck chairs.
    – Shog9
    Commented Aug 28, 2017 at 22:19
  • 4
    Naw, never that @YvetteColomb - use your downvotes. They're the most powerful, most underrated moderation tool on the site. Save close votes for "this is apocalyptic" scenarios.
    – Shog9
    Commented Aug 29, 2017 at 3:19
-1

As you yourself mentioned, the users can always Google for the answers. Even if they don't have to, I think a user can answer the questionnaire without actually knowing how to ask a good question. (I personally don't see the co-relation there).

About the comparison with widely accepted captchas, I see a major drawback with a questionnaire. While a captcha only intends to weed out robots, a questionnaire draws line using the knowledge of the person. Some might think of this as insulting and I think it is a bit rude on our part.

What we eventually want is, people to ask good question. We can still have a questionnaire, but instead of asking Programming or Mathematical questions we can questions about the website itself like -

  1. A multiple choice/fill in the blank question from the "How to Ask" page, or any other page we frequently keep linking in comments on Bad questions.
  2. A simple "Is this a duplicate of this" question. Since dupes are a major problem apart from bad questions.
  3. Choose a better answer from two. Say one has a descriptive answer regarding the problem and the other just has a bunch of links with little or no explanations.

The list can be grown according to the type of "bad questions" we see.

In this case even if they look up the answers from Google, they still have to read the rules or get a sense of what is good/bad in the process.

At the same time it is not that insulting, since it is not testing your knowledge but your understanding of the decorum of SO.

This questionnaire can then be stopped once the user has asked 2-3 good questions (we already have a sense of good questions since we have a badge for it) or when they gain a certain amount of reputation.

Again about trolls or spammers, I don't think we can do anything about it. Like you said those are "unfixable posters".

0
-7

Your proposal looks good, I specifically like how it concerns about avoiding to bother new users too indiscriminately.

I think it could be improved by taking into account that system already has ability to filter users whose question is going to be troublesome (and uses it, although maybe not as efficiently as it could). Specifically, I am thinking about popping such a quiz right before pushing user's first question to .

Just think of it, if a question passed above quality criteria that would trigger pushing into triage, user is quite likely good enough to be automatically bothered by quiz (they may still be poor asker mind you but that's too hard to tell automatically - otherwise triage heuristics would catch them).

On the other hand, if system has already detected that question is potentially troublesome, automatically running a quiz like you proposed could be helpful. It would block some part of the "unfixable posters" as you call them, thus decreasing load on triage reviewers, and as for another part that would manage to break through the quiz they would at least be throttled a bit more which is also a good thing.

15
  • 2
    Another good way of thinking outside the box - that's what I wanted to see if we could come up with some solutions - we have such a huge site and some good brains on here - surely we can figure something out.
    – user3956566
    Commented Aug 25, 2017 at 12:52
  • 7
    Or... We work out why the triage queue isn't working as intended and why people still let crap through and try and fix that... Manually reviewing failed audits in that queue that haven't led to an autoban is sometimes saddening - people will still say looks OK or requires editing to clear spam (and by clear I mean very very clear spam - fridge freezers, best buys in... Contact us on this number or email us at this address and view our latest offer on our website. Not those trying to wrap a genuine question around links or trying to hide links in punctuation etc...)
    – Jon Clements Mod
    Commented Aug 25, 2017 at 13:44
  • Well this has encouraged me to review the triage queue - at least if good reviews are made there - that's some garbage we can clear up quickly. The issue with getting rid of garbage is it's such a lengthy process to close vote and then delete vote. I really wish the site would just do something to make it easier for us to clean it up
    – user3956566
    Commented Aug 25, 2017 at 14:28
  • @JonClements if I read correctly your "or" is exclusive, why? I can't see could go wrong if you do that work out based on questions that got to triage after askers broke through quiz. Quiz will likely just make number of these questions smaller which is a good thing, no?
    – gnat
    Commented Aug 25, 2017 at 14:54
  • gnat - my concern would be putting genuine people off before they even post. I'm not sure how I'd feel going to a site with a well defined and researched problem at hand and then being asked questions to prove I'm worthy of doing so - I may not bother. However, if the questions not worthy of being on the site got properly triaged we're not asking the proof to be on the poster (heck they could still pass the "quiz" and post about how we should buy product xyz anyway) but rather it gets triaged into unsalvagable and is deleted there.
    – Jon Clements Mod
    Commented Aug 25, 2017 at 15:05
  • @JonClements that last comment would be good to put into an answer
    – user3956566
    Commented Aug 25, 2017 at 15:06
  • 1
    Sure it'd be an extra layer of filtering ingress but it'd still be triaged and there'd be a trade off between those that are determined to pass the test to post rubbish vs those put off passing the test to post something genuine and useful
    – Jon Clements Mod
    Commented Aug 25, 2017 at 15:07
  • 1
    @Yvette I may do so once it's a bit firmer in my mind. Those are just off the top of my head thoughts and observations - nothing concrete as such.
    – Jon Clements Mod
    Commented Aug 25, 2017 at 15:09
  • @JonClements basically we need to bring this issue to a head one way or the other - the community needs the tools to be able to keep the site clean - and while it can be argued that we have the tools - they're not effective enough - if the close queue is anything to go by.
    – user3956566
    Commented Aug 25, 2017 at 15:12
  • @JonClements your concern makes no sense to me sorry. We're talking about people who are already put off by the system (into triage): their questions are not going to be visible unless reviewers approve. Whether you show them that quiz or not is really minor compared to what they already have to pass through
    – gnat
    Commented Aug 25, 2017 at 15:13
  • @gnat haven't you just countered your own point? Assuming posts got triaged correctly - then it doesn't matter if you quiz someone apriori does it - good stuff gets posted and the bad stuff doesn't.
    – Jon Clements Mod
    Commented Aug 25, 2017 at 15:16
  • @JonClements sure, I did just that - I considered only questions that are going to pass triage successfully. We already force askers of these (potentially good) questions wait while triage reviewers check them and approve and compared to that passing the quiz is really minor, they at least don't have to wait while some random guy takes a look at what they wrote
    – gnat
    Commented Aug 25, 2017 at 15:25
  • (I know that my approach is invincible no matter how you try to find holes in it - because I invented it as a small, minimally intrusive extension over the feature that already works and is proven useful)
    – gnat
    Commented Aug 25, 2017 at 15:28
  • 1
    I know it's frustrating. We just need to avoid the "something needs to be done. Here's something - so let's do something" approach. I'd rather spend a bit of time brushing away the sand than risking putting people off and the opportunity of genuine pearls.
    – Jon Clements Mod
    Commented Aug 25, 2017 at 15:37
  • what is frustrating, that askers wait until triage reviewers check their posts? you can't do anything about this I'm afraid
    – gnat
    Commented Aug 25, 2017 at 15:38
-9

I have a counter-proposal. We give the new user a little quiz... after they posted their question.

And it is voluntary.

In exchange for correctly responding to a few simple questions, we give their question some extra exposure, like a bump to the front page. Or a few points. Nothing big, something like the +2 that you could also get for an edit.

The advantage of giving them a few points is that they now have something to lose. They have just become a little invested in Stack Overflow and their new account here!
We've discussed before that new users don't care about downvotes, because they believe they can just make a new account. Now that the new user has done a little work for their first points, a downvote will mean something.

The question could be about the How To Ask, or the latest Project Euler (as suggested by @George in their comment). Or anything like that.

Obviously it works only the first time. And it shouldn't give a user enough points to vote, lest it becomes a tool for sock puppeteers.

A voluntary quiz after posting might even be helpful for the asker.
Before they've posted their question, all they want is to put their question on the site. After they've posted it, it may be useful for them to take their mind off the question for a moment, and let the incubation effect do its work.

6
  • 3
    This is thinking outside the box - I like it
    – user3956566
    Commented Aug 25, 2017 at 12:50
  • 5
    Why not just answer someone else's question and get reputation?
    – Cody Gray Mod
    Commented Aug 25, 2017 at 12:52
  • 2
    @CodyGray well I guess that's the whole cyclic issue - is some people are not capable of providing good answers - so a questionnaire would sort that out. But I guess if it's after the fact it's just as a bonus and there could be a badge
    – user3956566
    Commented Aug 25, 2017 at 13:01
  • @CodyGray That's assuming there is a question that they can answer - and that is good enough that we actually want it on the site. Commented Aug 25, 2017 at 13:04
  • 7
    Sure, but I don't see the point in giving token amounts of reputation for completing tasks that are of absolutely no lasting value to the site. The whole design of the site is that you're supposed to contribute in both ways, and that earns you privileges and causes you to be invested. It serves all the purposes that this would, and more. The only merit in the original proposal is that it would hopefully block bad questions. We don't need more ways to earn reputation or benefits. Tried that with Documentation.
    – Cody Gray Mod
    Commented Aug 25, 2017 at 13:06
  • @CodyGray I see your point, if you consider rep something that represents lasting contributions to the site. (Even Docs rep was supposed to be for lasting contributions). If you look at rep as an indicator of how much trust the community has in a user, then this token amount of rep would mean something - a first little amount of trust gained. But more importantly, the new user will now lose something if they get a downvote. That, BTW, is why I suggested a small amount of points, and not a badge. Commented Aug 25, 2017 at 13:13
-14

Putting conditions or a "quiz" on participation would be counterproductive. While I agree that there are a lot of people who either don't have a good understanding of basic programming principles or are in a hurry to get an answer, I don't agree that we as a community should discourage people from participating here.

(As an aside....Personally, I haven't seen many trolls, spammers, or other undesirables on here but I work and don't live on here, either.)

Yeah, there's stupid questions and I've posted my fair share of them. It exposes my ignorance, but asking any question exposes a measure of ignorance. Slapping people down, whether it be by snarky comments or a pre-usage quiz, who are trying to learn hurts the intent of this site. This place exists to spread knowledge and the ones that would benefit the most from that intent are the ones that would be targeted by the pre-conditions suggested.

I do support the creation of a "newby coder overflow" or somesuch, a place where beginners can ask basic questions that seem to be beneath some users to answer like "How do I start making a program that does [blah]?", "where can I find a library that does [xyz]", and other questions that are deemed "off-topic" and closed. It would remove the "off-topic" questions from the community that the experienced users frequent, and give beginner programmers a chance to get their questions answered.

"But," you say, "Who would be there to answer the questions if we're all over here on stackoverflow?" We would. I have a bit of programming knowledge that I'd be happy to share, and I get the impression from some of the activity here that a lot of the users are teachers at heart. I think that there'd be just as much level of support on a newby community as there would be here.

My opinion, for what it's worth.

1
  • 1
    The blind leading the blind is always a good idea.
    – user4639281
    Commented Aug 25, 2017 at 23:06

You must log in to answer this question.