261

I’m Kristina, the first (and so far, only) User Experience Researcher here at Stack Overflow. As you probably know, on SO we have a bit of a problem with our new user experience. People new to SO (or maybe even new to programming!) come to us to find an answer to a programming issue, and that means that sometimes they ask a question themselves. When they ask a question, we show them a long page of rules for how to ask a question, and then throw them into it. (In case you forget what that looks like, here it is below.)

screenshot of ask a question rules

From here, one of several things happens:

  1. They are too intimidated by the wall of text (or get distracted or any number of things) and they just don’t ask it
  2. They ask a question, but it’s badly formed and it gets downvoted or deleted or snarked at
  3. They ask a question, but it’s a duplicate and gets closed right away (which is good for SO overall, but probably doesn’t feel great in-the-moment)
  4. They ask an appropriately-formed question and get a useful response

What we’re working on is reducing the instances of scenarios #1-3. We’re working with Denae Ford, a CS PhD student at NCSU (you might recognize her from her wonderful research work on gender-based barriers to Stack Overflow usage). Our first attempt to resolve this problem and make Stack Overflow friendlier to new folks (and hopefully also write a paper about it!), is to test a lightweight mentorship program.

Here’s how we envision this test working (about a month long, starting hopefully within the next month):

  1. When we see a first-time asker in the Ask a Question page, we show them a link to a chatroom in the sidebar
  2. We have a specially-created, moderated chatroom for people from the Ask a Question page and for people who volunteer to mentor (there are no rep minimums for this room)
  3. The mentors help with question formation, strategies for searching for existing questions, or with tagging. Mentors don’t actually answer programming questions in chat.

We hope that this or something like it will give new users a better first-time experience, thus increasing their likelihood of participating in the future, and generally make them better-equipped members of the Stack Overflow community. We also hope that the ratio of useful questions on the site will increase.

So, here’s what I’m asking from y’all:

  • What do you think about this idea? We’re just starting to put this test together and would love your feedback and thoughts - we want to give it as much likelihood to succeed as we can.
  • What concerns do you have with how we’re deciding to implement this initially? We’re trying to go as light on the technical requirements as possible, but we want to make sure we haven’t missed anything glaring.
  • And most importantly: are you interested in volunteering to be a mentor? A project like this would never work without knowledgeable and experienced Stack Overflow users to actually do some mentoring. As this is a test, we’d probably also want to talk to you about your experience afterwards.

If you want to be a mentor, sign up here.

69
  • 85
    When we see a first-time asker in the Ask a Question page, we show them a link to a chatroom in the sidebar I think this link would have to stand out more, it should be placed above the title of the question, that way it's less likely someone will miss it. The problem is mainly people haven't even gone to the How To Ask page before asking a question. Great idea to fix this issue btw, I hope this works!
    – George
    Commented Jul 26, 2017 at 13:26
  • 10
    I think this is a good idea generally. I would also love to volunteer (although I have very limited free time). Though I have doubts regarding the amount of new users that will actually notice this link and actually click on it. On the other hand, if there will be too many users at the same time in the chat, that could become very chaotic as you won't be able to have a proper conversation with a certain new user because of tens of messages posted from other users in between. Commented Jul 26, 2017 at 13:39
  • 31
    Good idea generally. Worth trying out and seeing whether it's viable.
    – Pekka
    Commented Jul 26, 2017 at 14:00
  • 21
    It's funny - only two days ago I thought about writing up a Meta suggestion on creating a new class of moderators, with the role of social workers where our current moderators have the role of cops - putting out fires but not having the time to help individual users much. A bit like the old SWAT team of nice suggestion. I wasn't sure whether it would be a good idea, but in this context and equipped with special chat rooms, it might make sense. It also might be a complete disaster - but it's certainly worth trying out!
    – Pekka
    Commented Jul 26, 2017 at 14:06
  • 56
    'Mentors don’t actually answer programming questions in chat' - how long will it take before someone attempts to get round that by expicitly asking a programming question, or by attempting to 'turn' an existing thread, (answers to the nearest 10ms, please). Commented Jul 26, 2017 at 14:14
  • 8
    Still, an attempt/trial to reduce the number of bad questions sounds good:) Commented Jul 26, 2017 at 14:15
  • 17
    Is there going to be any tag filtering? I could help people ask a C++ question or try and find a dupe for the, but I would be pretty limited in the help I could offer someone with PHP. Commented Jul 26, 2017 at 14:33
  • 26
    Off topic, I still think showing annotated examples of good and bad questions is the way to go teaching new users what works and what doesn't on Stack Overflow (To the extent that there is any rhyme and reason to what works around here and what doesn't.) Simple, understandable to ESL folks, relatively cheap to build.
    – Pekka
    Commented Jul 26, 2017 at 14:50
  • 7
    It should be trialed. If it turns into a massive disaster, it will still have value as a shield when disgruntled users fire off the all-too-common 'SO does not help new users' posts. We can link it and say 'we tried harder, and it just didn't work'. Commented Jul 26, 2017 at 15:00
  • 11
    How will you determine success/failure? Are you prepared to let it die if it does fail? Commented Jul 26, 2017 at 15:24
  • 15
    @PaulCrovella Oh, yes. We're killing it after about a month in any case - it's meant to be a short-term experiment where we gather data about the kinds of things people need help with, the way that people interact with each other in this setting, and how much getting personal help with asking a question actually helps people feel more welcome. So there really is no success/failure criteria here, because knowledge gained is a success - even if we learn that this is a horrible idea. :)
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 15:42
  • 12
    When you say that "there are no rep minimums for this room", that just applies to the new members asking for help to get their question in shape, right? Surely the mentors need site experience, and probably also domain-specific experience too. I suggest that mentors should have at least a bronze badge in some language tag.
    – PM 2Ring
    Commented Jul 26, 2017 at 17:39
  • 7
    @KristinaLustig Quick reality check. How many new SO users per day? How many questions from beginning users per day?
    – empty
    Commented Jul 27, 2017 at 19:51
  • 7
    @SteelToe 'new users mean know harm, its just they need a answer to a question fast.' That's exactly the approach that gets them down and close-voted:( Commented Jul 27, 2017 at 23:39
  • 11
    @SteelToe 'I know so many programmers that were turned off from the trolling and bias against first timers' do you have any evidence of trolling and bias against first timers, as distinct from bias against bad questions? No, you don't. How do I know? I have asked for evidence in many meta posts that claim thsi happens, and not one link has ever been posted. Now who is trolling, exactly? Commented Jul 27, 2017 at 23:42

35 Answers 35

256

I don't believe this is a good idea.

The main reason to me is that a new user will see a chat room and ask his question there. Then people in there will have to explain to him/her that they are not here to answer his/her question, but to help him/her ask the first question properly...

If he/she has the time it takes to ask a proper question, fine. Maybe he'll accept to be mentored that way. But I honestly do not think it will be the case for most users.

How do modern video games get new users into action as quickly as possible? With an undercovered tutorial. The first missions are the tutorials, but the story has started already, and information comes byte by byte, not all at once.

So why not do so here? Cody Gray started a post and I liked this answer.

The main idea is to guide the users step by step. A chat room will not solve the basic problem that a user has an immediate need of help and will find the fastest possible way to get help, even if it doesn't cope with the standards in place. Because he/she (wrongly) assumes that reading the whole tutorial to ask just one question is a loss of time.

So ask him, question by question, what is his problem. Then you can direct him easily towards a new question that can be asked, a question that already exists or a chat room that would help him solve his problem.

From some comments I receive for this answer, a new idea just came out. That if you want to aim at some undercover tutorial, make this chat-room, then make it monitored by learning bots. At the beginning they will need help from real mentors, but I'm pretty sure after a while they will be able to help 90% of the new users...

37
  • 23
    Yeah, this is a really interesting idea. We're starting this project on the assumption that throwing users a huge wall of text and expecting them to read that, process it, and use it, is inherently a flawed way of introducing them to a new platform. A step-by-step wizard-like solution - what you're describing - is another potential way of doing it. We decided to first try a kind of "get only the support that you need from other real humans" approach, but it definitely might be the wrong one. In that case, we'll go back to the drawing board, and the option you've presented is a solid alternate.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 14:23
  • 41
    One reason I really like this is that it's an experiment. Even if it doesn't work how it's expected to, we'll gain a better understanding of the types of questions and thoughts that new users struggle with. If we were to build an undercover tutorial in the future, this would be a low-cost way to research what should go into it, and what are all the human nuances of helping people get acclimated to this community.
    – Kurtis Beavers StaffMod
    Commented Jul 26, 2017 at 14:26
  • 4
    @KurtisBeavers You can get that by looking at the times people post on meta asking for help crafting a question. It's similar enough to this feature, except most people aren't told they can do it, and like this proposed feature, won't possibly scale up if it is used by a sizable portion of those that would benefit from using it.
    – Servy
    Commented Jul 26, 2017 at 14:31
  • 12
    I guess what I'm saying is the way this is pitched is that it isn't a proposed feature, but an experiment. It's OK if experiments fail or succeed, b/c they give valuable qualitative data either way. I also like the idea of the tutorial. It's totally possible the best solution is both or even something that hasn't been fully considered yet.
    – Kurtis Beavers StaffMod
    Commented Jul 26, 2017 at 14:42
  • 40
    @KristinaLustig 'throwing users a huge wall of text and expecting them to read that, process it, and use it, is inherently a flawed way of introducing them to a new platform' in general, I tend to agree. In the case of software development, however, is should be noted that reading, processing and using whole user manuals of text is just an project norm, and therefore it's not unreasonable to expect professional and/or enthusiast programmers to be able to handle one page of advice and follow a few links. Commented Jul 26, 2017 at 14:47
  • 5
    @MartinJames unfortunately I must disagree. People tend to read less and go straight to (what they believe is) the point. This is sad but true in every domain. With the ease now to build a small application from a short video tutorial (thinking about IoT), why would some people bother reading ? How many people have read MSDN docs for building a winforms ? Just add a button, double click on it and you can make some stuff... Commented Jul 26, 2017 at 14:55
  • 6
    @MartinJames "reading, processing and using whole user manuals of text is just an project norm" This is also a learned skill which most new users have not learned yet. (Otherwise they would not be "new".) Commented Jul 26, 2017 at 19:14
  • 11
    @Code-Apprentice well, new users of SO should be professional/enthusiastic programmers. They will have seen some documentation before. Commented Jul 26, 2017 at 19:17
  • 3
    @MartinJames I do not think that is a safe assumption. Even as a highly motivated beginner programmer, I had seen no technical documentation before my first day in a college level class. I gradually learned how to read docs over my entire college experience as a CS major. There are also many students in CS classes who are taking it as a "general elective" or as an alternative to a math requirement. These types of students are neither professional nor enthusaists. They just want to get a decent grade. These are the kinds of "new SO users" who I see this experiment targeting. Commented Jul 26, 2017 at 19:20
  • 16
    @Code-Apprentice "There are also many students in CS classes who are taking it as a 'general elective' or as an alternative to a math requirement." Then, as you say yourself in your next sentence, they "are neither professional nor enthusaists", and they have no business posting here. This site is explicitly for people who are serious about programming.
    – jscs
    Commented Jul 27, 2017 at 0:27
  • 7
    @Shawn: "The very first sentence on that page is 'Stack Overflow is a question and answer site for professional and enthusiast...'" Uh huh, and that's what we're talking about.
    – jscs
    Commented Jul 27, 2017 at 17:39
  • 4
    If they're actually students of the art, then I'm happy to consider them under the "enthusiast" rubric. It's the "They just want to get a decent grade." that I object to. (I wasn't a professional when I started using the site.)
    – jscs
    Commented Jul 27, 2017 at 17:57
  • 7
    @Shawn Ok, we disagree, but that's fine:) I'm surprised by your view, however, and have only admiration for anyone who would take that burden on. How may we direct those questions that are outside the class of 'professional and enthusiast' to yourself so that you can provide the help? Do you want links emailed to you directly, or are you going to set up your own site? Commented Jul 27, 2017 at 19:30
  • 6
    The site is about smart people asking questions and, hopefully, getting good answers. It depends on your definition of 'smart'. I like to think of the 'smart' askers as those who provide as much info in their questions as they reasonably can:inputs, outputs, error-messages, how testing was done, what was found during debugging. I think that 'smart' answerers are street-smart as well as skilled and experienced in software development, and recognise when they are being abused by those who will not help the helpers by making some effort first. Commented Jul 27, 2017 at 20:29
  • 12
    "The primary focus should not be high quality answers" That absolutely has been, and continues to be, the primary focus. Optimizing for Pearls, Not Sand. The vast majority of this site's traffic, and usefulness (to everyone, no matter their skill), comes from Googling, and the reason for that is that the answers are high quality. We already have forums for pages of back-and-forth working out what the problem is before getting to a solution. When you come to SO, you should get a solution, the end.
    – jscs
    Commented Jul 27, 2017 at 20:41
142

What concerns do you have with how we’re deciding to implement this initially?

Even though you explicitly mention that the mentors won't be answering programming questions, the biggest concern I see is that new users coming for help will see these volunteer users as people to directly answer their questions, not provide guidance on how to ask. This may be compounded when you have multiple new users in a single chatroom with multiple volunteers, since you'll have the potential for a lot of noise and crosstalk in the room itself.

Other than a written description of the role of these volunteers, do you have any other plans on how to enforce this idea? Are there any ideas on preemptively protecting the chat room from turning into a "can you answer my question?" chatroom, and protecting the volunteers from getting harassed into answering the question either directly in chat or under the final question?

Or are you going to wait and see if this scenario becomes a real problem?

8
  • 2
    Hey - you make a good point. Right now our plan is to make it very clear in the chat and with the mentors that this isn't for answering programming questions, it's for help with creating them (for a lot of reasons, including the greater good!). If we see this becoming a problem right away, there are a few avenues we have: continue as-is (with reinforcement of the rules), more active moderation by people from SO (like me!), or, if it's very bad, we might have to pause the program to think of a different implementation. I hope it doesn't come to that, but the only way to find out is to try.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 14:02
  • 119
    Evidence supporting the concern - my blog post on asking good questions starts (after a tinyurl) with "Please note: this blog post is not a suitable place to post a development question." I very regularly get comments asking questions. Some people just won't read what's in front of them...
    – Jon Skeet
    Commented Jul 26, 2017 at 14:43
  • @JonSkeet at first I read "I very rarely get comments [...]" And was very confused. How difficult would it be to check, automatically, if a comment is a question? SE has some things that do it (the word "problem" can't be in the title), but they are not generic/dynamic. They're hard-coded. It'd be interesting to explore automated answer demands from new users.
    – ItamarG3
    Commented Jul 26, 2017 at 19:59
  • 4
    @ItamarG3: Given that this is mentoring about a question, it would be reasonable to be quoting bits from the question within the chat, I'd say.
    – Jon Skeet
    Commented Jul 26, 2017 at 20:00
  • 3
    @JonSkeet true. But it might be worth investing time in a system that can detect repetitions of "so what's the answer to my question" and variations. It might also be good to see if a user is cooperative. If they're not, them that's the end of it: they shouldn't be mentored (not that they can't, but they shouldn't be. The latter often implies the former). (I'm slightly freaking out in the good way because Jon Skeet is answering my comment #.#)
    – ItamarG3
    Commented Jul 26, 2017 at 20:04
  • 9
    @ItamarG3: I suspect it's the kind of thing that's most easily written after we've seen what actually happens, but it would probably be worth the SO team thinking about possibilities beforehand, yes. And yes, uncooperative users are very annoying. (The number of people who've flatly told me that a code sample is complete, despite only consisting of a method on its own...)
    – Jon Skeet
    Commented Jul 26, 2017 at 20:05
  • 19
    @KristinaLustig Waiting for it to become a problem is simply going to de-incentivize people willing to help. We know it's going to be a problem, based on the rest of the site. This will only succeed if mentors are not driven away. There's already a huge amount of rubbish to wade through on the main site, let's not create another avenue of it. Why not liberally provide mentors with the ability to delete messages and/or timeout users - optionally with a popup explaining why the action was performed?
    – Rob Mod
    Commented Jul 27, 2017 at 0:20
  • Totally agree with this, it was my first though as I have seen it happen in plenty of channels across the web for a variety of reasons - poor understanding of the language used to describe the channels purpose being a large one, often (but not always) due to being non-English speaking. Liberal controls for mentors, with automatic notification of reason for deletion (rather than an implied "you're not good enough") would be a baseline requirement IMO.
    – Toby
    Commented Jul 27, 2017 at 11:54
94

So if someone is using this feature to get help improving their draft of a question, how are they intended to share that draft with the mentors? It's important that they do share the draft, so the mentors can see what they have so far and what they're missing or need to fix, but I can't imagine that putting the entire draft question in chat is possible/feasible. They're going to need some place to put the question and link to it in chat.

If this is rolled out just as described I suspect this is going to result in lots of people asking a question and then linking their existing questions in the chatroom, but at that point it's no longer serving its purpose, as the question will be getting downvotes/close votes/critique as appropriate, which is the thing this is trying to avoid. If people start hosting their drafts off site then mentors end up needing to click tons of possibly shady links to do their job, and moderators would need to let people constantly be posting links all over the place, providing tons of camouflage for spam or trolling (particularly given that there isn't going to be any rep requirement here).

If rolled out just as described it's also going to result in lots of people not providing enough information about the question they're considering asking to even get help crafting their question. If people start having problems just figuring out how to get help figuring out how to ask their actual question, well, if you think the current new user experience can leave a bad taste in people's mouth...

So for this to work you'd more or less need some way of letting people post their questions, but in a state where it doesn't show up in the question list/search/review queues/user profile, cannot be answered/voted on/closed (probably still want it to be editable and have comments enabled though), and could then be linked to in chat, and with the author having the ability to finalize it as a real question when they felt ready. Is this a feature you're planning to roll out with this?

35
  • 8
    Maybe some kind of "draft" status for questions? This would allow sharing of the question, but not full "question" status. Might not be a bad feature/solution overall, if users, say with >100 or >250 rep could 'approve' new questions, but they could all be seen, while allowing browsers/searchers to excl\ude 'drafts' from their browsing? Commented Jul 26, 2017 at 15:07
  • 2
    Maybe mentor could edit in a notice in question if posted on SO, that then is removed, to avoid some downvotes, close vote and comments while still in mentoring phase, they would just need a nice template. This in the future could be developed to an automatic question lock with notice, than then is removed. Commented Jul 26, 2017 at 15:31
  • 1
    @JonEricson But the intended workflow as laid out in the question is to link people to the chat from the ask question page, so they're going there before they've asked their question, and then they won't have a link to the question after they've posted their question. If the goal is to give people help improving their questions after they're asked (and the question doesn't lead me to believe that's actually the case) the link would need to be provided after they ask their question.
    – Servy
    Commented Jul 26, 2017 at 15:31
  • 6
    @KristinaLustig So you're just going to throw everyone in a chatroom without anything in place? That's simply going to end badly. I mentioned several major problems that you're going to have in my answer, and those types of problems aren't just going to be possible, they're going to be more or less mandatory. There won't be any "good" way for people to give the mentors enough information for them to be helpful in a productive way.
    – Servy
    Commented Jul 26, 2017 at 15:33
  • 3
    @PetterFriberg If people are asking bad questions on the site we don't want to discourage downvotes/close votes/comments on those questions. Those forms of feedback are all vital for the site to function. It's a bad thing if they aren't provided on the questions that need them. That would result in this feature doing more harm than good.
    – Servy
    Commented Jul 26, 2017 at 15:34
  • 1
    Providing the link to chat after the question is asked might be something we can try in the future. We don't want the chatroom to replace comments however. The ideal situation is for mentors to get involved as early in the process as possible. I suspect most people will have the chat in one tab as they work on their question in another. Sharable drafts would make this work out better, but I don't think they are strictly necessary. (And certainly not necessary for us to learn a thing or two from trying.)
    – Jon Ericson Staff
    Commented Jul 26, 2017 at 15:37
  • 6
    @JonEricson As for drafts being necessary, it may not be necessary for SO to have a heavily integrated and supported sharable draft feature (at first), but users need some way of sharing drafts with mentors, or mentors simply can't do their job, full stop. It might be enough for this experiment to just have people host their question on some random third party hosting site (perhaps a specific one that SO suggests they use) but if the mentors can't see what the draft question is, they can't critique it, or give advice as to what to do next.
    – Servy
    Commented Jul 26, 2017 at 15:41
  • 9
    The mentors definitely need to be able to see the draft of the question, and it has to be in the same form as it will eventually appear when the question goes live on the main site. Using 3rd party sites simply won't work, because Stack Exchange markup doesn't behave quite like anywhere else, and poor formatting is a significant issue for new askers (and as we all know, formatting in the chat rooms sucks). It's not just a matter of making the question look neat & pretty, in Python, incorrect indentation of the code alters control flow or is a syntax error.
    – PM 2Ring
    Commented Jul 26, 2017 at 16:11
  • 13
    @Servy: You aren't wrong. Perhaps this research will be just the thing to propel us to implement shareable drafts.
    – Jon Ericson Staff
    Commented Jul 26, 2017 at 16:18
  • 6
    @JonEricson Well I mean, we don't really need them now. If SO created sharable drafts they wouldn't have much use because there's no real place to use them. When this rolls out that would create the need for such a draft, and this feature will almost certainly be net harmful if there isn't some means of sharing a draft (whether integrated into the site or not). That doesn't mean that SO needs drafts, it means that a mentoring service needs drafts. If you need to run an experiment to figure out that a mentoring service requires some means of sharing the draft though...
    – Servy
    Commented Jul 26, 2017 at 17:18
  • 2
    @JonEricson So you think we should run an experimental mentoring service to see if having an integrated draft feature would be useful to SO even if there isn't a mentoring service? That makes no sense. If you want to find out if drafts would be useful for SO even if there isn't a mentoring service, then how does this experiment help you figure that out? And what's the point of running a mentoring service without any way of sharing drafts when we know, for sure that the mentors will need to see drafts of posts? That experiment doesn't tell you what you want to know.
    – Servy
    Commented Jul 26, 2017 at 17:49
  • 4
    @JonEricson I understand that. I'm saying that if you are trying to do research into whether or not a mentoring service might be useful, and you're running a small scale test to determine if it is, then you're going to need to some way of sharing a draft in that test. You don't need to run an experiment to figure out that in order for someone to critique a possible question and help someone improve it they need to see what they've done so far. There's no possible way for them to provide meaningful help otherwise. Knowing that doesn't take an experiment.
    – Servy
    Commented Jul 26, 2017 at 20:06
  • 6
    @JonEricson If you're going to honestly tell me that you need to run a month long experiment involving hundreds of users in order to figure out that in order for users to critique a potential question from a new user that the mentors need to see the prospective question, then that's...pretty absurd. Now if you said you weren't sure about the specifics of how to implement a native integration (I had a number of open ended questions on the subject in my answer) then sure, but the idea that there needs to be some way for the mentors to see a draft of the question doesn't take an experiment.
    – Servy
    Commented Jul 26, 2017 at 20:08
  • 2
    @JonEricson Do those rooms encourage users to paste large swaths of text/code in chat itself or to use an outside service like gist? Every room with a guideline on it that I'm aware of picks outside services, and it's almost always new users who need it beaten into them to use one. Commented Jul 27, 2017 at 1:01
  • 2
    @JonEricson While I understand what you're saying, testing out how well received a feature will be - without properly implementing said feature to make it useful - will yield questionable results at best.
    – Rob Mod
    Commented Jul 27, 2017 at 1:18
82

I signed up.

However, I think this project is doomed to fail.

The problem is that most of the people asking questions which fit your criteria simply lack the minimal understanding required to even formulate a good SO question with help. I spend a fair bit of time trying to teach programmers to fish in chat on this site, to so speak, and it's hard.

Really hard.

I expect that you will get a few types of people who actually enter these chat rooms:

  • 60% help vampires (SO doesn't like this term. You must embrace it - if you want this mentoring project to succeed you MUST understand that a large percentage of chat askers 100% fit this definition)
  • 30% people who have no idea what types of questions to ask, barely understand their problem, but can have the information drawn out of them with lengthy back/forth and uncovering of the massive XY problem(s) they are facing
  • 5% will solve their issue if you get them to write an MVCE
  • 5% have a good question which with minimal guidance will post it to SO and it will received well

My percentages are optimistic.

Of these categories, the first category is a writeoff. If you try to help that category without a ruthlessly rigorous system to filter your mentors will straight up quit.

The second need teachers/tutorials/books, not mentors.

The third and fourth are where this effort must focus its effort. However I am pretty confident that SO will focus on the first and second and doom the project.

Also, I don't know why this A/B study to making questions actually not suck by requesting information from the asker or something similar is not done instead of this. While a mentor might affect a few people, a better question asking experience would likely affect many people.


Just for kicks, I looked at the most recent question on Stack Overflow and found these:

4/5 of these are terribad questions from the formatting perspective. I don't know enough about PHP to know if 5 is too, it looks like a really basic syntax error to me, but hey code at least!

In all these cases a "mentor" would be far less effective than say... a preformed box of "what to put in a good question."

Wow, in the few minutes I took to write this up, two of the five questions were deleted (!).

21
  • 4
    Even though I think this is worth trying, a lot in here echoes my own worries - it could be that the couple of obviously capable and friendly newbies who frequently take to Twitter & co. after being treated badly on SO (the cases where most reasonable people are bound to think "SO should have treated this person better, no matter what the rules!") are the exception and there isn't an army of similar people behind them that just don't speak up. But then, maybe it's enough when we reach those couple of people? Even mentors won't be able to help everyone who comes to SO - but that's okay.
    – Pekka
    Commented Jul 26, 2017 at 15:03
  • 6
    @Pekka웃 I think that anyone who has ever been in any of the main site chat rooms for a while will feel similar to me. I hope this is optimizing for pearls, not trying to make sand into pearls...
    – enderland
    Commented Jul 26, 2017 at 15:04
  • that's a fair point. I'm not very familiar with that experience.
    – Pekka
    Commented Jul 26, 2017 at 15:04
  • 2
    @enderland Changes to the /ask page are on the table along with what Kristina is proposing here. Now that we have a PM for the DAG team (Q&A focused stuff), the /ask page is going to finally get some love and attention.
    – Taryn StaffMod
    Commented Jul 26, 2017 at 15:24
  • 4
    @bluefeet it would be even more awesome if users filled out that template and then were directed to mentors. Since I'm going to ask those exact same questions to 99% of "help me formulate a good SO question" people anyways. You could have them fill out a form not on SO directly with that information if you wanted, too, it wouldn't even have to be on SO directly.
    – enderland
    Commented Jul 26, 2017 at 15:26
  • 4
    The only way an experiment truly fails is when we fail to learn something from it.
    – Jon Ericson Staff
    Commented Jul 26, 2017 at 15:39
  • 3
    @JonEricson there are some experiments that are better than others to run and that can teach more even if they do fail, however...
    – enderland
    Commented Jul 26, 2017 at 15:43
  • 21
    @JonEricson I think you underestimate how much we already know about the things people struggle with when asking a question, and overestimate how likely this experiment is going to result in us coming up with new (practically implementable) scalable solutions on how to address them. It's a hard problem, one that all of us (SE itself and the community) have been striving to work on for years. It's simply a hard problem, due to the nature of how hard asking a good question is and how hard it is to get people to do the work to ask a good question when they don't want to.
    – Servy
    Commented Jul 26, 2017 at 15:48
  • 24
    Also the entire point of this post is feedback. How many people involved in this project have spent time mentoring people on SE chat who fit their target audience? I am often in the Python chat room and see the types of people you are looking to help here. I would strongly encourage Kristina (and others involved here) to actually go through the process of helping or reading the back/forths which happen with these sorts of users before just throwing blind ideas around. Your comment reads like "eh who cares, we'll try it and see what happens."
    – enderland
    Commented Jul 26, 2017 at 15:50
  • Help vampires shouldn't be much a problem in the mentoring room(s), since we won't actually be answering their technical question. OTOH, any new user entering the room needs to fully understand that the room is for fixing questions, not answering them, so if they persist in trying to get their technical question answered in the room they should be linked to the standard references re help vampires and kicked from the room.
    – PM 2Ring
    Commented Jul 26, 2017 at 15:51
  • 2
    @PM2Ring sure, but... oftentimes separating a help vampire from the second group is not clear until a long effort.
    – enderland
    Commented Jul 26, 2017 at 15:51
  • 7
    @KristinaLustig I would encourage you to stop by many of the SO chat rooms and ask people there, who have dealt with mentoring/helping people on a regular basis for years and ask their input before going too far down this proposal. A day or so of interactions will give you a ton of insight into designing a much better experiment.
    – enderland
    Commented Jul 26, 2017 at 15:58
  • 5
    In the Python chat room (where I'm one of several room owners) we have a policy that people can't ask for answers on their fresh SO questions (they need to wait a couple of days first), but they are permitted to ask for this kind of meta-help to fix up their fresh question. But we also get people who are asking questions that are too trivial or a bit too broad for the main site, and it's with those questions that we're prone to HV attacks. And these are all people who have enough rep to talk in chat, not raw 1-reppers, so they should have some knowledge of how to act properly on SO...
    – PM 2Ring
    Commented Jul 26, 2017 at 15:58
  • 3
    @PM2Ring yeah, that's the scary part. Chat is already flooded with... uh, sand, and that requires you to at least have some positive SO content.
    – enderland
    Commented Jul 26, 2017 at 15:59
  • 2
    @Kik I don't think that 'removing the ability to ask a question at all' is realistic on a Q&A site. The banner, however, seems more promising, though it lacks detail as described and would be misinterpreted. I suggest 'We have little interest in helping you except as a side-effect. Good questions and good answers might be useful to you, but SO is looking to help Fannie and Freddie: those users who look up SO Q&A next week, next year, or next decade '. Commented Aug 1, 2017 at 8:57
49

The warmth and inherent civility of communicating one-on-one with a live human being is the great feature of this project, but it could also be its downfall.

There's going to be a LOT of unreasonable people asking for a quick answer to their programming question and little interest in anything else.

These are the situations that suck the lifeblood out of the most idealistic mentor.

Avoiding these situations is much more easy on Stack Overflow as it currently works. You only contribute slices of time, every burden is shared, every interaction is somewhat asynchronous and there is no expectation of instant responses, there is strong cultural encouragement not to waste your time on too much one-on-one mentoring, et cetera.

How will Stack Overflow be protecting its mentors from the demands of unreasonable askers, given that it is much more difficult to bow out from a one-on-one conversation than from a Stack Overflow question?

Will there be clear-cut rules on what the purpose of mentoring is, and what it isn't, so mentors have something to rely on when a conversation becomes unproductive? "No programming questions" is a good start, but it might need more.

I'm even thinking this might need mods/employees watching over the conversations that go on and stepping in where they get out of control, but not sure whether that is feasible.

8
  • This isn't proposed as a one on one interaction. It's proposed as a chatroom filled with lots of mentors and lots of people looking for help.
    – Servy
    Commented Jul 26, 2017 at 15:36
  • 8
    In a shared chat full of people who need mentoring it'll be a lot of them answering those unreasonable people. Without very strict moderation that can quickly spiral into incentivising all the wrong behavior. Commented Jul 26, 2017 at 15:37
  • @Servy This isn't proposed as a one on one interaction right, yeah, I see. Hmmm
    – Pekka
    Commented Jul 26, 2017 at 15:53
  • 20
    Mentors that persist in actually answering the questions rather than helping fix them will need to get kicked, at least temporarily. Otherwise, chaos will ensue.
    – PM 2Ring
    Commented Jul 26, 2017 at 16:20
  • @PM2Ring Not just mentors, anyone in the chat. Commented Jul 29, 2017 at 12:06
  • @PaulCrovella True, but there won't be anyone in that chat room apart from the selected mentors & the raw newbies, and I'm assuming that the newbies mostly won't have the knowledge to be able to supply answers. OTOH, looking at a lot of the recent answers on SO, lack of knowledge doesn't seem to be much of an impediment for a lot of the people answering. ;)
    – PM 2Ring
    Commented Jul 29, 2017 at 12:16
  • @PM2Ring There's a lot of blind leading the blind out there, but the problem isn't whether they know the answer or not. The newbies would be in chat because they don't know what's expected of them on the site - prime candidates for screwing up in the chat as well. Commented Jul 29, 2017 at 12:20
  • @PaulCrovella I get the impression that Kristina plans on ensuring that everyone entering that chat room is aware of the purpose of the room and its ground rules. I don't disagree with what you're saying, but I still think that there's a greater risk of the mentors answering than of the newbies answering. But I guess only time will tell...
    – PM 2Ring
    Commented Jul 29, 2017 at 12:51
33

In light of this test plan moving forward, I feel I first have to ask about the assumption I see here:

What weight balance does Stack Overflow (the company) give to new user experience?

Do you have the directive to give a great new user experience at the cost of everything else? Or are you looking to provide an equal balance between new user experience and maintaining the high quality of Stack Overflow? There are many users who might suggest that a less-than-optimal or obstacle-laden new user experience is ideal because it weeds out people who just want to come here and get a code dump or have their work done for them, etc.

I think we can all guess which rough direction the company is pushing toward, but a response on this would help guide us as users in our recommendations and understanding of your ultimate goal. The balance you want to strike could make some suggestions completely out of the question, or make a hesitant reader think "Oh, then I have the perfect suggestion".


With that aside, my question for the Mentorship Project is:

How will you handle quality control of the mentors themselves?

Obviously a unified front is ideal, with equal measures of quality from each user, but as has been hinted at in other answers, different users have different ideas of 'how to ask a good question'. There are many users on Stack Overflow with 10,000 or 100,000 reputation who regularly answer duplicates, even when others leave comments asking them not to. Likewise, there are some users who think asking simple questions is the worst possible use of Stack Overflow, despite the Community Team saying "no, simple questions are fine, as long as they're useful, well-crafted, and not duplicates".

Will there be quality standards that mentors must meet, or else they risk being removed as a mentor? Can mentors flag or vote for other mentors to be removed or have a Community Team employee check out their mentoring quality? Or will you just try to discern the good from the bad during the invitation process and then turn a blind eye or have embarrassing break-up moments later on down the road when it turns out a mentor isn't that great after all?

9
  • 1
    This is a big issue, and I will not be attempting to mentor in the initial stages of this experiment; I will just lurk. Raising 'unhelpful' or 'abusive' flags on someone who does not immediately provide an answer to bad homework question dup #535 is not unknown on SO, and I want no part of mentoring until I understand the written, and unwritten, rules of the game. Commented Jul 26, 2017 at 20:00
  • 10
    I really think your "aside" in the first part is a really key question. It seems like SO (the company) cares more about new questions/askers than existing answerers. Commented Jul 26, 2017 at 21:07
  • 2
    @MikeMcCaughan Indeed, and if so, then that's fine, but I think we all would like it to be clarified in some concrete terms, because I see it being very easy for a new user UX project to reduce the quality of questions from new users vis-a-vis making it easier for new users to ask questions. By the very nature of them being new to the site, they'll be less likely to know what the site considers a 'good' question.
    – TylerH
    Commented Jul 26, 2017 at 22:19
  • 4
    I'm not sure it is a "seems" at this point. Recent Meta posts have made it abundantly clear.
    – jscs
    Commented Jul 26, 2017 at 22:29
  • 1
    @JoshCaswell it's clear from this question. It is assumed that all new users arrive to ask questions, and any skilled/experienced engineers who turn up to give free time to answer questions can be ignored. I expect that the chances of any kind of mentoring system to assist that class of new user to identify vamps, deadbeats, gimmeTehCodez, doAllMywork, doMyDebug, doMytesting etc. is approx. zero. I wonder how many answer a question or two, have their blood drained, leave and never come back? Commented Jul 27, 2017 at 11:30
  • 2
    @TylerH RE: your first point about new user experience. The DAG team takes question quality and new user experience seriously. I don't see them as mutually exclusive. We'd like to provide an experience that gives a new user a better chance of success. Mentoring may play a role in that. Or, this experiment may help us better understand how to programmatically help a new user to ask a better question. The ultimate goals are satisfied, engaged members AND quality content.
    – Joe Friend StaffMod
    Commented Jul 27, 2017 at 14:52
  • 1
    @JoeFriend Would the team be at all open to ideas for improving question quality at expense to the new asker experience? Just as an example: a time delay of 24 hours after registering before they're allow to ask a question on the site? Commented Jul 29, 2017 at 12:11
  • 2
    @PaulCrovella Short answer is yes. However, I think this could be implemented in a way that doesn't come off as "at the expense of the new asker experience." It would be interesting to run an experiment to determine how short of a delay is long enough to discourage the wrong behavior and encourage the right behavior. My hypothesis would be that even 30-60 minutes may be enough.
    – Joe Friend StaffMod
    Commented Jul 30, 2017 at 21:28
  • 1
    @JoeFriend I'd love to see the results of that experiment. I chose 24 hours based just on the number of "member since today" profiles I see asking low quality and off topic questions, so it's somewhat arbitrary. It'd be interesting to see the effect on the question signal to noise ratio for various delays. Commented Jul 30, 2017 at 22:38
30

My greatest worry over this is that the mentors will need a good grasp of SO policy and rules. To me, this is mods and/or experienced SO users. Since the free time of both those sets is a limited resource, the time spend on mentoring new users may be deducted from the time available to actually answer good questions and/or handle bad questions/users :(

5
  • 1
    True, but unfortunately most question are currently bad (duplicate or off-topic), I prefer spending my time helping out to avoid these, rather then to close vote review and flag offensive comments on them. Commented Jul 26, 2017 at 15:09
  • 4
    @PetterFriberg I can understand that, sure. I have always downvoted meta proposals that move the effort of fixing bad questions from the OP to the skilled and experienced engineers that answer good questions. My fear is that this proposal falls squarely in that bin, but I'm supporting it anyway as a trial/experiment, though I have that nagging feeling still that 'something must be done, this is something, so this must be done':) Commented Jul 26, 2017 at 15:18
  • 1
    I mean, it could even be counter-productive. In a similar fashion to the way parachutes, life-jackets and smoke-hoods would result in more air crash deaths, moderator time may be so diverted from 'normal' SO activities that the number of bad questions actually increases, (at a greater rate than now:). Commented Jul 27, 2017 at 19:25
  • For your first comment I don't really have an issue, who cares about where the effort is spent, important for me is that we end up with a useful Q/A for future users. For the experiment I guess it will be a duplicate finder experiment (at least that is my experience with users that ping me in chat), hence they have a question, I pass you a link to a duplicate end of story. However I have signed up, it would be interesting to see how they like to do this experiment and if we end up with a better ask question interface, I'm all in! Commented Jul 27, 2017 at 19:41
  • @PetterFriberg I don't have great hopes, but we'll see. Shoud be interesting, either way;) Commented Jul 27, 2017 at 20:13
29

When you want to ask a question, you need to follow these steps:

  1. Narrow down in one, or at most a couple of sentences, explaining at a high level what you're trying to do and why. Kind of a user story.
  2. Show what code you already have. Even if your question is a "How do I", you need to show a frame of reference for answerers to get them to know about the types and values of the input and output.
  3. Show what you have tried. "I have written this code (see #2) but it doesn't work" is not what you have tried, we want to see what you have tried to make that code work.

That's really all there is to it, but it requires a couple of paragraphs of text.

I strongly believe this format will not work in chat form, not in the first place because the "mentors" will get tired pretty quickly of explaining these three steps over and over again to impatient users who just want their problem solved.

Also, because mentors will be required to be somewhat of an expert (at least if we don't want a "blind leading the blind" situation), I'm afraid they won't have the persistence to keep hand-holding newbies into asking a good question and will quickly burn out, while tons of effort have already went in writing comprehensive pages that perfectly explain how to do so. There is absolutely no need to rehash those pages for everyone who wants to ask a question.

I also believe that the outcome of this initial experiment will be positive. The users who do visit and engage with a mentor in chat are already the ones interested in writing a good question, and the mentors present there will be happy if they helped their mentees - a win-win. That doesn't mean it'll scale up or won't require an extraordinary amount of effort compared to the results gained.

Instead the Ask a Question page should be redesigned.

5
  • 2
    You make a good point - there's definitely the danger that those who have signed on to mentor will get burned out - we'll watch out for that, and we'll also be interviewing mentors afterwards to understand their experiences. There's just so much benefit we can get out of running an experiment like this. It will help us to be able to say with certainty that X or Y thing does or does not work, and understand why that is. This is probably not the ideal solution, it's just the first one we're testing.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 14:55
  • 1
    @Kristina I'm all for experiments. I'm just not really convinced it'll work, and wanted to share that. Definitely don't let that hold you back.
    – CodeCaster
    Commented Jul 26, 2017 at 14:58
  • 7
    For what it's worth, I think the input from this project will help us do a better job of redesigning the ask page.
    – Jon Ericson Staff
    Commented Jul 26, 2017 at 15:31
  • Having the ask a question page require those steps (with separate inputs) would be really nice. The "show what code you have" section could be auto-formatted, which would resolve a lot of problems with new questions.
    – Andrew
    Commented Jul 27, 2017 at 17:09
  • I predict a mentor half-life of two days. Commented Jul 28, 2017 at 15:00
24

Your chat group would have to be populated by trained monkeys that keep spitting the same code phrases:

  • "Please include MCVS"
  • "Please specify the desired behavior"
  • "Please include the error message"
  • "A request for debugging help is not a question"
  • "A request for coding service is not a question"
  • "Please try to use a debugger"
  • "Please refer to the foundational text/reference/language definition"
  • "Please consult with your professor to improve your understanding"

If the goal is to reduce the time the community spends filtering out the bad questions, chat is exactly the wrong thing.

Most of the bad questions' OPs do not come here to learn how to ask. They come here to scratch an itch. They are not interesting in learning about questions, they need answers.

Your mentors will have to try to teach the people who are not interested in learning.

Instead, you need to assist people in asking questions. As was suggested on Meta before, we need a New Question Wizard.

The "Ask a Question" page for the new OP would have a bunch of links:

  • "My program crashes"
  • "My program does not compile"
  • "My program throws an exception"
  • "My program does not do what I tell it to do"
  • "How do I write code for my homework?"

The natural inclination of a less-experienced OP is to take the assistance. And then you have him/her! You can structure their questions, you can ask for every relevant element, you can even say "Please don't post your homework assignment here, this is not a homework site".

This will be much less labor-intensive than chat and likely more useful, too.

4
  • That is the essence. Askers should be obligatorily informed & assisted; there is already mentoring; it is ignored; questions should be close-voted, so askers will edit; not downvoted, it's not nice.
    – philipxy
    Commented Jul 29, 2017 at 1:09
  • @philipxy 'Askers should be obligatorily informed & assisted' well, that would have to be an automated system. Suggestions for such an automaton have been suggested before, and resisted as an unwanted barrier to new users:( Commented Jul 29, 2017 at 9:27
  • @MartinJames My comment doesn't commit. The obligation can be imposed as closure until improvement per comments. I agree auto is problematic. Certainly info & input could be better structured for those who want to ask good questions.
    – philipxy
    Commented Jul 29, 2017 at 9:49
  • 1
    @philipxy you may well be right. I'm gonna open a burner account, and go through the process of asking a question, right up to the point of actually submitting it, and then back out. I need to see for myself what new users get as advice:) Commented Jul 29, 2017 at 10:07
20

What do you think about this idea?

I like the idea of this idea. Improving the experience of first-time posters, and their posts themselves, is a worthwhile goal. I know quite a few people who think of Stack Overflow as too pedantic: "I never post on Stack Overflow because I just get downvoted and criticized for stuff that doesn't actually matter." Imho, improving this should be one of the core goals of the Stack Overflow team. So it's cool to see that people are actually thinking about the problem.

What concerns do you have with how we’re deciding to implement this initially?

In a nutshell: the people who don't read the wall-of-text aren't going to click a link to a chatroom, let alone spend the time to learn how to format their posts.

And most importantly: are you interested in volunteering to be a mentor?

Honestly? No. I spend a lot of time in some niche tags, and I consider myself a pretty kumbaya Stack-Overflow-ian. I err on the side of answering questions that are too broad, or not really fully formulated, because I know how overwhelming programming can be to a novice.

That being said, I've been a member (and moderator) of several forums at this point, and the main reason I prefer Stack Overflow is its zero tolerance policy on "code plz" questions. I think this idea of offering a helpful chat room vastly underestimates the number of posts that should simply be deleted immediately, because the OP didn't spend the bare minimum amount of time on it. And again, that type of user is not going to spend the time to click a link, talk about their problem, learn how to format their post, make revisions, get feedback, etc.

If other people are excited about being mentors, more power to them. But I'm afraid that, at best, mentors will spend 99% of their time on basically useless posts that simply should not be posted at all. And that really takes a toll, as evidenced by all of the forums I barely visit anymore.

It might be argued that this feature is not intended to help that type of user, and is instead geared towards users who want to spend the time to ask a well-formed question. But in that case, getting help in a chat room is not less work than reading a wall of text, so I'm not sure how much this will help those users or Stack Overflow's repuation. "Stack Overflow is so complicated that they have a chat room just for learning how to post!"

0
17
  • What do you think about this idea?

I think it's a well-intentioned, but misdirected idea. It seems to ignore what is, at least in my opinion, the biggest problem Stack Overflow has, which is a fundamental aspect of human nature. The people who are altruistically-minded, or even have the pairing of being reward-motivated and skilled enough to produce high-quality content for the site are enormously outnumbered by the people who just don't give a crap, and who rely on the gamification that motivates lower-quality handling of answering.

First and foremost on that question of lower-quality of answering, is people's willingness to write a new answer, generally of low-quality, for a question that is either unclear (so their answer is one of a half-dozen or more guesses) or which has already been asked and answered in a much clearer way (so the question should have been closed as a duplicate rather than being answered).

Put another way: this idea appears to focus on helping people that, frankly, are never going to contribute good, useful, high-quality content to the site, and who probably don't really want the help in the first place.

One problem Stack Overflow has always struggled with, as near as I can tell, is the dichotomy between the idea of helping individuals and building something that is useful for the community as a whole. IMHO, the latter is the real goal of Stack Overflow, and that goal is not served by spoon-feeding people who aren't willing or able to sign on to it.

But this mentoring idea does exactly that. It will greatly add to the work-load of dealing with low-quality contributors to the site, without appreciably improving the overall quality of the site. The people who find themselves in need of mentoring are still going to ask duplicate questions, will still show up having done little or no research of their own, and will still be asking questions that they should and could have answered for themselves by just reading the documentation.

Opinions vary, and I understand some people, even those answering questions instead of asking them, view Stack Overflow as just a large-scale helpdesk for programmers. But for me, the primary value here is in fact the building of an all-encompassing repository for good programming questions and answers, and this goal is at odds with a mentality that every person asking a question deserves an effort at answering their question.

  • What concerns do you have with how we’re deciding to implement this initially?

My biggest concern is the lack of collaboration tools that would help it succeed, first and foremost being a mechanism to pair up mentor and mentee, and to provide a mechanism for the mentor to have immediate access to the mentee's work as they receive help.

My other concern is, of course, as stated above: this idea seems likely to wind up dedicating the most resources to those least deserving of them. That is, the people who view Stack Overflow as just something to serve them, rather than something they can participate in and help improve.

  • And most importantly: are you interested in volunteering to be a mentor?

No way, no how. I've already seen how a person who is poor at asking questions can become a huge time suck if you let them. I've fallen into that trap more than once, and I work hard these days to avoid it. I pity those who are willing to volunteer for mentor service in this program; to be sure, they will once in a while wind up helping a person who genuinely wants to put their best foot forward and is willing to take constructive criticism to do so. But I'm sure the vast majority of interactions will be with the exact opposite of that. Our mentors are going to need a lot of patience to get through that.

In a perfect world, there'd be special training to help people understand how to ask good questions. In terms of research, presentation, and construction. Such training would be provided separately, and would be a prerequisite for asking questions on a site like Stack Overflow. Most importantly, it would have an outcome-testing aspect, ensuring that those who are certified as qualified authors of questions have in fact met the goals of that certification.

In fact, it's why I liked this answer and have myself thought a similar design would be appropriate. The idea of mentoring isn't bad, it's just that it's too costly for the reward when it requires an actual person for each person in need.

I favor instead a question-authoring user interface forced on inexperienced users that would guide them through the composition of a good question. This is by no means a trivial task, but IMHO a much more worthy and potentially fruitful area of research. Templates and automation together, to constrain how a new user presents their question (say, a form with a questionnaire format for example), and to evaluate in an automated way using, for example, modern language-processing/neural net pattern matching technologies to judge whether a question is coherent, provides adequate details, and a suitable code example.

On that latter point, things like detecting that a user is asking about code that doesn't work or which throws an exception, but has not provided an actual description of the behavior of the code, or any specific details about the exception, such as the type, message, and stack trace. It may even be feasible, in conjunction with a template approach, to know when a question likely requires a full MCVE, and whether that question has in fact included one.

Templates can provide that framework for the user to work within, and may also help simplify the job of the automation, by providing additional context for the pattern-matching. In a manner similar to the game-tutorial concept, there could even be multiple levels of complexity, starting from the most restrictive, and loosening up as the user gains experience and demonstrates competence, until they have full access and can write a post without any constraints at all.

If there's going to be research, I think it best to focus on solutions that reduce rather than increase the demand on volunteer services. Stack Overflow is already straining under the demand that exists today, so I don't think a new program that significantly adds to that work-load is likely to work out.

2
  • 3
    I too have these concerns, especially the time/effort of skilled and experienced developers diverted from answering good questions, and mods diverted from handling bad questions and/or users. I will lurk on the room to see what is happening and, if it seems to be directly useful, may even volunteer to mentor, (but yes, I can't see that happening, but I'm prepared to be proved wrong). If it tanks, then I can reference the remains as a shield in future 'SO does not help new users' posts:( Commented Jul 27, 2017 at 8:22
  • 10
    My thoughts on reading the announcement were along these lines as well. It's not enough that experts provide solutions, now the expectation is to walk through the formulation of the problems, too? The only saving grace here is that the program is explicitly time-limited.
    – jscs
    Commented Jul 27, 2017 at 11:30
15

Is it a single chatroom for all new users and mentors, or are new chatrooms created for more one-on-one interactions?

I don't know if you'll get enough mentors for true one-on-one coverage, but chatrooms can get too busy and too hectic if there is more than one conversation going on. Perhaps there could be a single room that then breaks up into smaller rooms after mentors interact with new users?

9
  • 2
    Yeah, right now we're planning it as a single chatroom for exactly the reason you mentioned - we don't think we'll have enough for 1:1 coverage. That single chatroom model might not be sustainable or scalable, but it'll be the easiest to implement. If we have a need for more infrastructure and smaller chats, then we'll definitely want to explore that, too, once we have the results from this experiment.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 13:45
  • 8
    I think it'll need to be a hybrid: start off in a joint chatroom (or possibly have a few, based on tags?) but then "pair up" users and mentors quickly.
    – Jon Skeet
    Commented Jul 26, 2017 at 14:44
  • 1
    You could probably do something similar to how loads for sites can be split between servers, maybe have a pair of mentors to a room, and split the new users between these rooms? Commented Jul 26, 2017 at 15:05
  • @JonSkeet Yup - if this works well and we want to implement this more formally/permanently, we'll need to invest more resources in figuring out how to scale it properly.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 15:16
  • 1
    I agree that it will probably get too hectic and confusing if it's all confined to a single room. IME it can get a bit bewildering if there are more than 3 or 4 conversations going on in the chat rooms I frequent. OTOH, these mentoring conversations should be fairly short, unlike actual problem-solving conversations that can go on and on. And I guess there is a benefit to having the core experience focused in a single room.
    – PM 2Ring
    Commented Jul 26, 2017 at 15:41
  • 1
    @PM2Ring I'm not so sure they'll always be short. You can look to plenty of meta examples on this one; people sometimes link to a question and ask how it can be improved, which is more or less what's going on in this proposal (but in chat). These posts end up in lengthy discussions often enough.
    – Servy
    Commented Jul 26, 2017 at 15:45
  • 2
    @KristinaLustig maybe spin up a new room for each case (like how each post can have its own associated chat room), and have a way to alert the pool of active mentors? For example, when a new room is created, can it be automatically linked into the single "mentors" room so whoever's there can click through? (There's probably a better way to do it; this is off the top of my head.) Commented Jul 26, 2017 at 16:03
  • Right @MonicaCellio. I had a similar idea, but didn't know how well it would play with the notification system. And that would need better coverage else a new user could just be sitting in a room by themself for a while.
    – ryanyuyu
    Commented Jul 26, 2017 at 16:05
  • 1
    @ryanyuyu transient notification is hard. You don't want to fill up the inboxes of mentors who aren't online at the moment (like plain old chat pings would do), but you do want a way for them to passively find out someone's waiting for help. We don't have something like "@here" to ping just the mentors currently in chat, so a feed into a dedicated room is the next-best thing I could come up with. Commented Jul 26, 2017 at 16:09
15

Here's my summary of my experiences dealing with those asking "bad" questions:

  • 95%+ of them clearly didn't even bother typing their question into Google (often I can just type the title of their own question into Google to get the answer).
  • 95%+ will ignore (or disregard) every single comment that attempts to get them to improve their question, no matter how polite those comments are.

Based on this, I assume the vast majority of users will do one of three things:

  • Try to ask their question there, expecting an answer, since it's less effort, or because why not.

    If they don't get an answer there, they'll just ask their question as usual.

  • Actively ignore it.

  • Be entirely oblivious to their own wrong-doing, or think this will just consist of the "mean" users from Stack Overflow, and just ask their question as usual.

The rest of the users (probably a fraction of a percent of the total) will:

  • Want to try it, but they'd be intimidated by having to figure out what exactly they're supposed to ask that's not their question and revert back to asking the question as usual.

  • Ask their question there (expecting meta feedback), be hit with a long list of requirements or consider the constructive feedback to be "mean" (consider that we generally expect users to do extensive research and basically rewrite their code before posting it here), give up and just ask their question as usual.

    Also, if they just ask their question there... shouldn't we just spend some effort "fixing" review and send it there instead? Isn't the First Posts queue essentially made for this very purpose?

    Aren't comments also made for this purpose? Why does this need to happen in chat? Why is this happening in chat better than it happening in comments? Because comments will be ignored and answers will come either way?

    If you want to prevent answers while resolving this, can't we just show the question in review, but hide it from public view (upon it being posted), preventing answers (and votes?) and let comments do what they were made for? Wouldn't that basically be an improved version of what's suggested here - less effort required by the asker, the asker can't really just ignore the feedback and ask anyway, it can (selectively) happen automatically instead of (or in addition to) being opt-in and it's using the existing review system. Comments probably also need some improvements, but I digress.

Also, not answering questions where we're not supposed to answer questions is hard, like really hard. Have you looked at the comments on just about every question? I can't imagine many people would manage to completely refrain from just answering the question sometimes in chat, which would entirely defeat the purpose.

In summary:

I don't know which users you're targeting here, but it sounds like we have very different views of what the average user on Stack Overflow looks like, and I'm not sure why you want to build an entire new system instead of just spending some time experimenting with what you already have.

3
  • @davidism What is your point vis a vis this answer? The answer says that being in chat is just another queue, waiting for people to address the very same points made in comments already, but don't.
    – philipxy
    Commented Aug 6, 2017 at 7:20
  • 1
    If the "under review" thing rolls out, I certainly want a way to opt out of seeing questions I can't vote on. Oh, and how is borderline spam going to be handled? Commented Aug 11, 2017 at 11:46
  • @JohnDvorak I don't know if preventing votes is a good idea (it's also an optional part of it), but my suggestion (if implemented well) seems better in every way to the whole mentorship project thing. The question should only show in review, which would make it trivially easy to avoid. What's borderline spam? Sounds like something that should be flagged instead of just downvoted. Borderline spam would probably also be easier to handle with my suggestion as opposed to the original suggestion (because there's an entire mature and extensive flagging, moderation and review system already there). Commented Aug 11, 2017 at 12:42
13

I see two possible outcomes of this:

  1. Very, very few people actually use the feature, because they're just here to ask their question, not to chat with people about how to ask their question. If this happens, the feature hasn't actually accomplished much of anything, even if those few people actually get good feedback. The problems you laid out will still be happening about as frequently as they're happening now.

  2. Lots of people use it; a significant enough portion of the people asking questions actually use this to help them ask better question. The end result, the chat room is unusable because there are too many people talking over each other, and nobody can follow what's being said, and also there will be so many more people asking for help than there will be people willing/able to help them that only a very small portion of the people asking questions actually get help, and those that do get help will tend to get more superficial help, and not get significant amounts of feedback, because the people helping them will be in such a hurry to get to the other people looking to get help.

Now that doesn't mean that the feature will be net harmful as a result of this, so it doesn't mean that this alone is reason to not do it, but I'd say it has a very low ceiling as far as possible benefits.

Honestly this feature seems about as useful as people just asking on meta when they want help critiquing a question they have (either one already posted, or one not yet posted), which is to say that it works great in very small numbers, but just doesn't scale up.

This might be more likely to be a useful feature on smaller sites, rather than sites like SO, where the ratio of regular users to questions is much lower, and it's actually practical to critique the quesitons of everyone asking for it. If this does happen though you'd need to be sure to not show the link to new users if the chatroom has no mentors in it, because in a smaller site there might not be mentors there 24/7.

14
  • Kind of a loose-loose situation... Commented Jul 26, 2017 at 14:24
  • 4
    What about the third option, where we have juuuust the right number of mentors and new question askers and everything is perfect? :D (But really, your point is really valid and something we're worried about, but we think we need to learn a lot more about this approach via experimenting in order to find out.)
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 14:27
  • 5
    @KristinaLustig The third option results in you waking up from your dream and finding that you actually ended up with option 1 (or you ended up in option 2 for a few days and then people stopped using it because it didn't help, and the mentors got overworked and the novelty wore off, and you ended up with option 1).
    – Servy
    Commented Jul 26, 2017 at 14:29
  • 2
    All prospective mentors should stock up on beer, just in case @Servy is right. Commented Jul 26, 2017 at 14:51
  • 6
    @MartinJames alcohol intake by mentors is possibly a good success / failure metric.
    – Kurtis Beavers StaffMod
    Commented Jul 26, 2017 at 14:56
  • 3
    The sensible way to address #2 is to have the link shown to a percentage of askers, and adjust that % according to the number of mentors available (with some hard cap). Obviously if this works we'll need a more robust system than a chatroom, but no sense in building it without testing first.
    – Shog9
    Commented Jul 26, 2017 at 15:02
  • 2
    @Shog9 Sure, that's just forcing yourself into option 1. There are lots of ways to do that, you suggested one (you could also go by some metrics, rather than choosing randomly, similar to what the first post/late answer/VLQ queues do). My point here is simply to say that of all of the possible options, there's a ceiling on how much good this can possibly do, even if it works perfectly, and the ceiling is fairly low. Even if you create a test that is effective a small scale, you won't be able to scale it up later.
    – Servy
    Commented Jul 26, 2017 at 15:06
  • 1
    or make multiple chat rooms and have people round robin sent to one of them?
    – enderland
    Commented Jul 26, 2017 at 15:46
  • @enderland That wouldn't remove the problems I've addressed in the post or the comments here. It would remove some of the general chaos, but it won't solve the problem that there's way more people looking to get help than there are people to help them. Splitting them up into multiple rooms won't magically create more mentors.
    – Servy
    Commented Jul 26, 2017 at 15:58
  • You said #1 was folks not using it because they don't care, @Servy. That's on them; removing the option because there aren't folks available to help is on us, and seems a reasonable approach to scale - certainly there are many thousands of unanswered questions on Stack Overflow itself for the same reason; ignorance on the part of one person does not create a responsibility for anyone else.
    – Shog9
    Commented Jul 26, 2017 at 17:29
  • 4
    @Shog9 But that's just it. Either the people asking questions don't want to use the feature, so it doesn't help, and it's their fault, or those people do ask for help in large numbers, and then we can't handle it, and that's our fault. If we actually run the experiment we'll be able to figure out who's "fault" it is that the system can't work, but my point in this answer is that there is no situation in which this works. We might be able to figure out why it doesn't work, but we already know, before we start, that it won't work.
    – Servy
    Commented Jul 26, 2017 at 17:34
  • Servy argues that there aren't enough potential mentors out there to make a difference. I tend to agree. Even for smaller sites, there might be less askers but also less experts/mentors, so even there the ratio might be too low. Commented Jul 28, 2017 at 19:48
  • Some might say that volunteering as a mentor is like volunteering for point duty in a combat hot zone, and will surely get you used/abused/suspended/banned. I could not possibly comment myself... Commented Jul 29, 2017 at 10:12
  • @Trilarion Personally I think some of the smaller sites at least have a shot. I'm not super active on any, but my interactions with some, and second hand knowledge of others, would tell me that there's at least enough of a possibility to run an experiment for some of the smaller sites. I don't think SO has any chance at all, so such an experiment is just a waste of time and resources.
    – Servy
    Commented Jul 31, 2017 at 13:04
12

I think it's positive that you're thinking about how to address the problem of new members not knowing the ropes.

My concern is that it assumes a reasonable majority of new members are open to being coached in how to ask a question, and will respond positively when invited into a chatroom where others refuse to answer their particular question, and instead give pointers on how to ask a better question.

In my experience, there is a high prevalence among new members of one of more of the following

  • Not bothering to take time to learn the ropes;
  • Skipping without reading through pages that provide guidance on the rules or expectations of the site;
  • Skipping guidance on how to ask questions in a way more likely to elicit a useful response;
  • Signing up with the hope of finding a free homework service;
  • Diving straight in with a question, without doing any search or otherwise checking that a similar question has been asked before.
  • If advised that they need to ask questions better, simply deeming that advice a waste of time.

The list goes on. But, in most cases, the behaviours are consistent with wanting a quick answer to questions they have asked, rather than guidance on how to contribute more useful questions to the site.

Not only SO is affected by this - it is quite common among forum sites that are open to questions from beginners as well.

More conventionally, mentoring programs often work better if the mentee (if that is a word) voluntarily seeking out a mentor for advice, rather than being taken aside (into a chatroom in this case) for a round of mentoring.

11

I suspect you'll need a serious contingent of users with room owner or moderator powers to keep that room in line. Users will ask programming questions in there, probably a lot of them. If you can't keep the noise level down in the room, it'll be horribly confusing.

I don't think anybody knows what will happen if we give a lot of 1 rep users access to chat. There is some potential for abuse there, but maybe the spammers are too slow to adapt to this.

One big concern for me would be simply whether this can scale. With thousands of new questions per day, this will require a very large number of mentors if it were to run at full scale, and not a test for a small percentage of users. I don't see that many users signing up for a potentially very frustrating job.

7
  • 2
    Yeah - we're prepared to throttle the % of users that the chat link is displayed to depending on adoption. We're not anticipating that there'll be an extremely high rate of people clicking into a sidebar link, but you're right - we'll need to be prepared for that.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 15:15
  • 5
    I like the chat room as a very low-effort step toward a more robust mentoring system. (I proposed one idea a few years ago.) A single chat room won't scale, but it will give us more evidence for whether a mentoring system would even work and what we'd need to do to maximize success. I can see an end state with many chat room to scale this up. I can also see an end state that isn't chat centric at all. This project is very much a trial balloon.
    – Jon Ericson Staff
    Commented Jul 26, 2017 at 15:25
  • @davidism Oh, yup! The chat link will only show up the first time you ask a question.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 18:19
  • 3
    @KristinaLustig no, I mean the mentors will need a system to know if this is a return user. Nothing stops users from bookmarking and re-joining chat just to keep getting help with their low effort drafts. As an owner of the Python room, we have a whole backend system outside of SO that tracks what users we've seen and whether they're vampires, so that all ROs are on the same page. I know mods have annotations, but that's not available to ROs.
    – davidism
    Commented Jul 26, 2017 at 18:21
  • @davidism interesting. Our current approach is to only let certain users (the ones who are flagged as mentors or who arrive at the chat directly from the ask-a-question page) participate in the chat - others are observers. There may still be holes for abuse there - I appreciate the flag. We'll keep an eye out. We many need some kind of timing mechanism or gate.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 18:23
  • @davidism Maybe I'm confused - does what I explained not address the issue of re-joining mentees?
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 18:25
  • @KristinaLustig Of course that just incentivises those people to create new accounts so that they can get access to the room again.
    – Servy
    Commented Jul 26, 2017 at 19:03
10
  1. They are too intimidated by the wall of text (or get distracted or any number of things) and they just don’t ask it
  2. They ask a question, but it’s badly formed and it gets downvoted or deleted or snarked at
  3. They ask a question, but it’s a duplicate and gets closed right away (which is good for SO overall, but probably doesn’t feel great in-the-moment)
  4. They ask an appropriately-formed question and get a useful response

I'm a big proponent of the idea of giving people a nice wide open space to type in to start with, and not giving them a wall of text they'll just skip as if it were a software licensing agreement. (I'm still mad I clicked that thing about not using iTunes for nuclear weapons, I should have read more carefully.)

The better time to hit them is probably after they've finished writing the question (or during), but before it's posted, to give them a chance to go back. And that also means you've got some data on what they put into the box to drive a guidance engine. I am a broken record on this one, but could you at least try this idea?:

Pre-flight screening checklist for first/early posts--adaptively pick three items, tune with metrics

Figure out what works and doesn't, based on how people's questions who are shown the varied prompts wind up being received. Maybe notice things like some screening questions are most effective on short questions, or perhaps they should depend on the tag, or whatever. Short and punchy--no wall.

What do you think about this idea?

Throwing people into chat just means they'll ask their questions in chat. But as with what I said above, the time to get involved in mentoring is after the question is composed...if they look at the situation and go "help!" Maybe a button so that if they're getting downvoted it can go "looks like you're downvoted, want to put your question on hold for a bit and ask for some assistance?"

5
  • "Looks like you're downvoted, want to put your question on hold for a bit and ask for some assistance?" Copyright Ⓒ 1996-2003 Microsoft Word. Clippit is a registered trademark of Microsoft Office Word 2003. All rights reserved.
    – AMDG
    Commented Aug 5, 2017 at 17:07
  • 1
    @LinkTheProgrammer I worked at Microsoft when they were doing research on how much people hated the paperclip. At first it was interpreted as saying the office assistant was a failed idea. But they found that many people who hated the paperclip liked the dog. There's something particularly condescending about the paperclip, in particular. Anyway, I think other automated assistive stuff (your question may be a duplicate, these words in the title may get it closed) are good, as long as they're just advice...if I mechanically can't post something because of a word in the title, that's bad. Commented Aug 5, 2017 at 19:24
  • I understand, but I was just making a comedic reference. In all realness, that pattern of Clippy has now become somewhat cliché that is all too informal: "It looks like you're trying to [...]; would you like some help with that?"
    – AMDG
    Commented Aug 5, 2017 at 21:19
  • I'd have to imagine that, because of the way our brain associates objects, the paper clip is probably associated with negative emotions: the stressful workplace that no one wants to think of—even at work itself; school; mundane tasks; etc. The dog is usually associated with positive emotions: playfulness, affections, etc.
    – AMDG
    Commented Aug 5, 2017 at 21:26
  • The bots in chat.stackoverflow.com/rooms/111347/sobotics which flag low-quality posts would be wonderful to integrate. They already have multiple heuristics for poor quality and anti-social behavior.
    – tripleee
    Commented Aug 11, 2017 at 10:59
9

I like that Stack Overflow is attempting to increase its already overwhelmingly inclusive approach to solving problems.

The mentors help with question formation

This is the only issue in my opinion, and it is certainly non trivial.

How? In the current framework how can mentors help with question formation?

If you can make this open to everyone in the community through improving the overall workflow that would be the ideal solution. Perhaps this would be in the form of sharing drafts and a new review queue.

In general, users at Stack Overflow do help with the formation of questions, through edits and comments. The main problem with the workflow for questions of a questionable nature is that many question askers are not attentive. It is a fire-and-forget process that is being used.

Getting a question asker (especially new ones) to realize that they need to be immediately attentive to their question, and also remain so for at least the next hour is very important. If creating a chat room that facilitates that interaction is the solution, then let's do it. But why should that singular chat room be any different than any other?

The approach of introducing a middle ground between question formation and question posting represents a barrier to asking. We have had a discussion on this topic before. The overall response was that a minor barrier would be ideal, but that there is a balance to strike between too invasive and too simple. I think that having a mentorship definitely falls on the invasive end as it essentially would imply to new users that if they did not take the mentor path and instead asked a question straight away then that was why their question was poorly received. I know that is more than likely not the reason, low quality content is never well received here, however perception is very important to new users.

From experience in chat, this could also quickly become a place where users simply ask their questions and attempt to get answers without posting - or worse get users to create their solutions. In other words, a haven for question banned users attempting to bypass their question restriction. These users would circumvent the intent of the mentorship as they are generally less interested in the process so much as simply having someone else do their work.

Stack Drafts

Similar to the creation of StackSnippets, there could be a StackDrafts. This could facilitate the sharing of drafts, allow a place for one-boxing, and get past the notion of a private area in order to allow the community at large to work on this. As well as a captch, anything at StackDrafts older than 60 days could be roomba'd - this would help to prevent abuse. It should also not be exposed to web crawling.

What it could look like from the asker

At any point during the ask question process, the asker may select a button that enters into mentor mode. "Get Help From A Mentor". Clicking this button enters the question into a Mentor Review Queue (where the request is populated by a text input similar to the flag for moderator UI), enters the asker into a chat room for that question specifically, and also replaces the button area with some feedback about the current mentor status of the post.

Overall layout with button:

Get Help From A Mentor

Modal for entering into mentor mode:

enter image description here

Status feedback for mentor mode

mentor mode status feedback

What it could look like from the mentor

The mentor sees the request in the queue and then clicks to mentor the post. Clicking the request has two actions. 1) It enters the user into a post specific chat, 2) It navigates the user to a rendered view of the draft. The post looks similar to a deleted post, except with many features not present - essentially it is a raw view of the render of the saved draft. Editing is explicitly not here, as the user should be the one responsible for creating such content to avoid abuse and encourage learning.

mentor POV

Who can mentor?

Anyone. The mentor review queue will simply be a queue where a user can either "Mentor" which will then take the action noted above, or skip, in which case they may view another. This will also allow multiple users to mentor one post, which may be desirable depending on the technology involved. It may also make sense to instead go the route of requiring at least one tag be used, and then limit the mentors to users having at least a bronze badge in said tag.

This would be one way I could envision making this work without greatly influencing the existing tools already built.

6
  • 12
    "Get Help From A Mentor" = get someone to answer your question. This is how it will be read.
    – user6655984
    Commented Jul 26, 2017 at 21:59
  • Can you think of alternative wording @Alex? The intent was to get someone to help compose the question.
    – Travis J
    Commented Jul 26, 2017 at 22:02
  • 5
    Maybe "Get advice on improving this post".
    – user6655984
    Commented Jul 26, 2017 at 22:09
  • I like that. Although it doesn't really mention the word "mentor".
    – Travis J
    Commented Jul 26, 2017 at 22:10
  • 7
    Suppose you have a site moderation problem. You think to yourself, "I know! I'll use a review queue." Well, now you have two problems.
    – jscs
    Commented Jul 26, 2017 at 22:19
  • Can you explain what moderation problem you are referencing, and how using a review queue in this circumstance would not be beneficial @JoshCaswell?
    – Travis J
    Commented Jul 26, 2017 at 22:20
7

One of the challenges the community has (and one of the reasons why Meta exists) is that the community as a whole doesn't agree 100% on what constitutes a good question. In some cases the differences are minor, but there are some can be polar opposites (code vs no code, what is a resource request, what is considered opinion based, are code review questions acceptable, any many more). *

So I see 2 possible issues arising due to this problem

  • A mentor, believing one set of guidelines, helps a new user to ask a question in a certain way or give the impression the question is on-topic and acceptable. But the community downvotes the question and the new user is still discouraged and is now mad at the mentor.
  • 2 (or more) different mentors in the same room feel differently about a specific user's question and provide conflicting guidance. The new user is still discouraged and is also confused.

This may be another scenario that you just have to wait and see how often this happens (the first bullet is almost guaranteed to happen, and the 2nd may be a problem as well, just less of an issue). But I think it is something you need to be prepared to handle with guidance for the user on how to handle conflicting info or mentors who are just wrong about how to ask a good question.

3
  • Is "are code review questions acceptable" something that is actually debated? We have Code Review for that. Commented Jul 28, 2017 at 1:12
  • @Scott Thank you for proving my point.. As someone witha close vote powers, i hope you are aware that just because a question is on topic onthe another site doesn't make out off topic here? Commented Jul 28, 2017 at 1:21
  • 1
    Yes, I am aware of that, but my understanding was that code review questions were off-topic for SO. Do you have any links to discussion about this? Commented Jul 28, 2017 at 2:05
7

I think I'm mentoring already for a very long time.

Often I comment on questions being new, kind of not very good but showing some potential instead of downvoting or closevoting them right away.

My comments aim at improving the question and/or making it acceptable. Quite often (say in ~50% of the cases) I get a response which (again in ~50% of these cases) results in an improvement of the question kind of making it acceptable. In other cases I may downvote the question later on.

Now this is much less effort than talking with every one before a question is posed because:

  • I can decide which question has potential
  • The comment system does not require me to be there the whole time (unlike a chat)

I would prefer that this avenue is explored more (more kind of reviewing with feedback instead of chat support) instead of the proposed feature. Maybe newly posed questions of new users or users with a bad track record should get into a special area first (think of quarantine) where the questions are scrutinized for validity instead of letting them free on the world the second the user clicks on "post my question".

In general I foresee a lack of available chat mentors. A human powered (for free) helpdesk for posing questions.. might not be possible.

1
  • 1
    Also the proposal reminded me a bit of the area 51 idea of Stackoverflow Academy. Commented Jul 28, 2017 at 20:09
6

I like this idea in principle, but there's one thing that has me concerned about its long-term viability:

Moving targets.

Stack Overflow is complex. It's very much the case that moderation of question quality is not consistent or is very much in a state of flux across disciplines. Worse, it's applied unevenly; if you get a user asking for feedback on a question of type A without a "foo", you may get users who are accustomed to questions of type B which very much mandates a "foo".

I'd consider myself a high-rep user, and I'd consider myself capable of being a mentor to others. I'm already a Comp-Sci tutor, and I have already mentored others on revolutionary career changes into software engineering. However, I'm just not sure that I'd be able to mentor Stack Overflow beginners very well, because unlike tutoring or career mentoring, what's "best" for the protege or what the protege needs to do in order to be successful here often changes by the hour.

Either one of two things needs to happen before I'd say that this could work well:

  • We need to shore up our moderation of questions in general, and have an easier-to-digest flow of what kinds of questions we think are good and which kinds we think are bad; or
  • We need to have mentors who have explicitly demonstrated the capability of understanding the multifariousness of Stack Overflow's questions, and what barometer of quality is being expected of the protege at any given time.

One of these things needs to be stable for a new user to find lots of success on Stack Overflow, and I have my doubts for either. I will remain optimistic for the latter, though.

2
  • 3
    We definitely have no illusions that this is long-term viable... it's meant to be a small and brief test through which we will hopefully gather lots of useful data. I like both of your suggestions a lot - we should definitely have clear definitions of questions that are good and not good, and we should make sure that mentors have the know-how to guide others. To be honest, I think this test may inevitably feel a bit wild-west-y. But thanks for sharing ways that we can try to avoid that feel.
    – kristinalustig StaffMod
    Commented Jul 26, 2017 at 15:36
  • 1
    This is essentially what I was trying to convey in my 2nd answer, but this is expressed so much better than my attempt. Commented Jul 26, 2017 at 15:52
6

What do you think about this idea?

Anyone who can be bothered going to a mentoring chatroom rather than banging a question down is likely to be writing good questions even without it, with the possible exception of indirect duplicates.

Which makes it futile in my opinion

6

I agree with many of the concerns expressed, particularly those of a noisy chat room, new users not bothering to click a link, and untrained mentors.

Here are some I'm not seeing addressed:

  • This sounds like a "done deal". As in even if someone links to a much better idea - even acknowledged by the OP in the comments - it won't stop this potential train wreck from falling off the cliff.

You have a problem. We have a problem. We all have a problem. Why implement a solution in a tunnel? If you want our feedback, at least don't make it sound like it doesn't matter.

  • How will you define the success or failure of this? By participation in this single chat room? By the number of down votes on a new user's question?

Before implementation of a new feature, it's normal to define this.

  • This isn't a repeat of my first point, instead it's asking why you haven't decided to revamp this "wall of text" into something more usable.

If there's something we all agree on, it's that it is a wall of text. You yourself used this as your first argument in your question! Now there's something you can do in a tunnel.

  • My last concern is the lack of using rep in any way.

"When we see a first-time asker in the Ask a Question page...people who volunteer to mentor (there are no rep minimums for this room)."

This cuts both ways. I don't have much rep (compared to many here) but I have learned what makes a good question. While I can appreciated someone giving me constructive criticism when I eventually ask my first question, it's nice to know I'll be treated like some student who created their account today. (Not.) Even more, this student who may not know how to properly ask a question can work the chat room. (Ouch.) I did see something to suggest the question was edited on this - but I'm just quoting what I currently see.

My conclusion? You, we, all of us know there's a problem. I don't believe this will help much. But if it does or doesn't - can you tell me how you'll know? (Point #2.)

2
  • 3
    The concern about being "treated like some student who created their account today" seems unwarranted. If you don't want to ask for help formulating your question, then don't ask. Nobody is proposing to force first-time askers into that chatroom. And by the way, reputation is visible in chat.
    – user6655984
    Commented Jul 27, 2017 at 2:35
  • Good point, and I agree. Maybe I should edit my thoughts. Let me work through that concern and edit. My concerns are making a distinction on what a bot (or some other objective way) can suggest what I should do with my question.
    – user7014451
    Commented Jul 29, 2017 at 2:24
4

What do you think about this idea?

I think it is great that you're trying to improve user experience for new users. Unfortunately, I think this approach is not a very good one.

I would much rather see a revised "newbie" question form for new users (with an option to bypass it if you are familiar with SO), that holds their hands through the entire process, and makes it difficult to submit a poorly formulated question.

I'm not sure how this would be implemented, though. Possibly by providing additional mandatory fields that would be incorporated into the final question (e.g. a field for MCVE, individual fields for "what is the problem you're trying to solve?", "what have you tried?", etc.).

What concerns do you have with how we’re deciding to implement this initially?

  • I think that the idea of a chat room will automatically invalidate this option for a significant portion of new users.

  • As other users have pointed out, out of those users who are willing/able to visit a chat room, a significant portion will want to ask their question in chat, and have it answered in chat.

  • Participating in this chat as a mentor sounds like a thankless, frustrating job that will quickly burn out those willing to participate.

And most importantly: are you interested in volunteering to be a mentor?

Sorry, but no. I don't have the time or patience.

3

Please disable chat-wide notifications when messages are flagged - either for the duration of the experiment, forever (...worth a shot), or even just for flags originating from the mentor room(s). These notifications are already an annoyance for many, and a room full of new people who understand neither established etiquette or the system they find themselves in will make the problem worse.

People who didn't sign up for this experiment shouldn't be bothered with its side effects.

1

Your User Experience Research is for us to mentor noobs in chat?

Sorry, I've got REAL problems to solve, no time to tweak some 12 year old's question on how to code his/her hobby project.


Since you've asked the community to mentor, which is usually a paid gig, can we ask something better be built based on all the solutions here? You guys are awesome devs, do it with AI or something!!

We've had this conversation so many times. I feel it is a major concern right now.

You can see all the links here have great "User Story" value. Cody Gray started a post and I liked this proposal for a question "menu".

Here are your concerns:

They are too intimidated by the wall of text (or get distracted or any number of things) and they just don’t ask it

They ask a question, but it’s badly formed and it gets downvoted or deleted or snarked at

They ask a question, but it’s a duplicate and gets closed right away (which is good for SO overall, but probably doesn’t feel great in-the-moment)

This is one of the best ways forward, it's an answer with a score over 300: Why is Stack Overflow so negative of late?

The idea is making downvotes contain a reason why. I've used the reasons at @WillSullivan's http://idownvotedbecau.se/.

Getting Downvotes and not knowing why is frustrating. This would solve that and give people feedback straight away, which would stop noobies feeling frustrated without even knowing why they suck. Please fix this bit first.

- I would recommend downvote reasons be confidential - only the OP can see the reasons to avoid "Group-Think".

- The HTML div downvote popup is a Comment Bubble, also notice how the design is an architectural door open plan.. When the OP edits their question, then one would hope the downvote reasons start to be reversed/go away...

18
  • I suggest adding 'shown', 'displayed' or 'demonstrated' to some items, eg 'No research' and 'No debugging'. This would be a defense against the 'I searched all over the internet and could not find any help' and/or 'I tried to debug it but could not find the problem' questions. Commented Jul 27, 2017 at 9:17
  • I was thinking that Will's project is hosted at GitHub so let the community set the reasons @MartinJames Commented Jul 27, 2017 at 9:20
  • 3
    I drew the red circle myself, honest! Commented Jul 27, 2017 at 9:56
  • 2
    Sorry, no 'freehand circles' upvote for you! It's too well-drawn, and so some evil posters might think you used an ellipse tool:( Commented Jul 27, 2017 at 10:50
  • Hey! That's a cheat edit! Meh - have an upboat anyway.. Commented Jul 27, 2017 at 10:57
  • 2
    -1 for not being red enough, too orange-y. Also @MartinJames what do you mean by an "upboat"? We do have a hairboat though.
    – kayess
    Commented Jul 27, 2017 at 13:38
  • Up voted just for the first phrase. Really strange outcome for a "UX researcher".
    – idmean
    Commented Jul 27, 2017 at 15:48
  • 12
    As much as people complain about downvotes without comments, they complain way more emphatically when they actually get comments to go along with their downvotes, even when they're constructive. People just don't like being informed that they've done something wrong.
    – Servy
    Commented Jul 27, 2017 at 16:18
  • Other than the -2 rep (which doesn't affect new users anyways), how would this be different from close votes?
    – user812786
    Commented Jul 27, 2017 at 17:57
  • 3
    @Servy and, if all those commenting using the current system were to be honest, instead of continually have to l... be economical with the truth, most of the regular skilled and experienced developers that answer questions on SO would join me on the 'suspended for a year' hall of infamy. It seems that anything other than total subservience gets flagged/meta'd as 'rude and abusive'. I have a nasty feeling that canned downvote comments, and/or the mentoring room, would/will go the same way, no matter how it is handled:( Commented Jul 27, 2017 at 18:07
  • On principle I hate this idea; it takes away a long-held convention and it also means that if I really don't think your question/answer is just any good, you now get to start a pitched debate with me explaining why your question about why your code doesn't compile isn't that trivial. However, I don't mind the idea of piloting this during a run like this. If nothing else, we can pilot it, get the data, and put the whole notion to rest.
    – Makoto
    Commented Jul 27, 2017 at 22:56
  • @Makoto, unlike commenting why you downvoted, this would be anonymous, thank you for having the mindset to try something, I think a lot of these ideas are worthy of AB Testing :) Commented Jul 27, 2017 at 23:13
  • Interesting... meta.stackexchange.com/questions/135/… Commented Jul 28, 2017 at 1:37
  • +1 for recommending some way to show downvote reasons Commented Jul 28, 2017 at 12:08
  • 1
    @JeremyThompson New users both complain when they don't get a reason for downvotes that they ought to get a reason, and then when they actually do get a reason, they complain even more and get even more upset than those that didn't get a reason at all. I'm not saying that you're wrong that new users complain when they get downvotes for no reason, just that this won't fix the problem, it will just make it worse.
    – Servy
    Commented Jul 31, 2017 at 13:08
0

Even this is a good idea, I don't believe that programmers or who wants to be a programmer need a better guide. Yes it might be harsh but doing research, trying to understand things is the part of the title. By doing this there might be problems like, Mentors don’t actually answer programming questions in chat.

I think they will answer

and chat will be an easier way to find answers for newbies or people who doesn't like to research.

-2

I think it's good idea, but the better idea would be to improve/implement the following things.

  1. Make the Ask Question page more intuitive so that the user doesn't have to go through the getting started guide. For example, in the question page suggest and auto-correct titles using machine learning and AI. In the question body show the code window, text window, exception window, etc... (may be some intuitive drag-gable components) so that there is no need to format the code; it will automatically format it. Add spelling and grammar auto-correction.

  2. Add a new user tab in the questions themselves and list all the new user questions there if the user is asking a question for first time or second time so the mentors can help and guide them easily.

  3. Using AI or machine learning implement an engine which will find all the possible duplicate questions, and their related questions, and put it in a queue "Possible Duplicates". Now mentors or reviewers can guide the new user and close the question if it is a duplicate, this will also save the time of other users.

2
  • Why would those in particular be "the best"?
    – Kevin B
    Commented Jul 27, 2017 at 15:33
  • @KevinB, yeah it will not be a best Idea, but I think better than the posted in the question
    – smali
    Commented Jul 28, 2017 at 6:13
-3

What do you think about this idea? We’re just starting to put this test together and would love your feedback and thoughts - we want to give it as much likelihood to succeed as we can.

I think it's a great idea that stands a chance to help people get better technically while keeping in mind the goal of SO to exist as a collection of High Quality questions and answers, all this while also keeping tempers lower. WIN WIN! :)

What concerns do you have with how we’re deciding to implement this initially? We’re trying to go as light on the technical requirements as possible, but we want to make sure we haven’t missed anything glaring.

Selection of mentors is crucial. The most technically advanced may not make for the best teachers. Unfortunately, there cannot be a one-size-fits-all to this, and only time and experience will tell. How much risk is Stack Overflow willing to take, considering that it is an experimental project?

And most importantly: are you interested in volunteering to be a mentor? A project like this would never work without knowledgeable and experienced Stack users to actually do some mentoring. As this is a test, we’d probably also want to talk to you about your experience afterwards.

Heck, yes! I've taught small children professionally before, and I'm a big fan of the ELI5 subreddit.

-4

I understand this is probably going to be a poorly accepted proposal, but I'd just like to say this as someone who uses SO occasionally, and every so often tries to use it like I use other stack exchange sites (i.e., using it all the time) but fails.

Thinking about it, the size of SO, and the results coming from that, is what turns me off. Majorly. An analogy: I hang out on CS Educators. It's an incredibly welcoming community. First posts by new users will sometimes have three welcome comments. Downvoting happens, but users are encouraged to pop into a chatroom. Top users devote significant amounts of time to helping them and cleaning up the site. This is true of most beta sites I've been on. On Physics.SE, the site I'm most active on, things are a little less friendly, but it's still a really welcoming site.

Size is the difference. Beta sites are small, closely knit communities, but they are ready to help others into the fold. Sites that are of medium size like PSE are still friendly, but they've lost a little bit because they don't care as much about user count. SO is a downright jungle. It's massive. No one cares about user count at all, because there's so many already. People are frantically trying to keep quality at some kind of level, and turning off many people in the process.

People aren't to be blamed. Tons of junk shows up that users have to deal with. It gets annoying, aggravating, painful. Just look at SO's CV queue. I head over there just to smile whenever the PSE queue gets long-ish, because there's thousands of questions in there.

My solution is kind of crazy, probably never going to be accepted, but - split SO up. Have a Python site, a JavaScript site. Or, rather, maybe do this: SO for general programming, and then tags split off into sites of their own as they get big enough. Or perhaps, you could have all the web-development tags under one umbrella. Whatever.

The point is, it's already been done to some extent, and worked. Look at TeX & Friends, or Mathematica.SE. They're friendly, helpful communities. Then step back into SO and look at the LaTeX and Mathematica tags. Chaos. This in fact also fits into the work you cited on gender disparity on SO. The biggest thing cited? The large, overwhelming communities turned women off disproportionately in comparison to men.

I myself would be all for it. I'd love having a dedicated Python site. I already am active on the dedicated TeX (and Mathematica, though not as much) site. Please, look at what has already worked.

6
  • 2
    Something similar has been proposed before: meta.stackoverflow.com/questions/253824/… FWIW, I think it's great to share opinions even when you don't think they'll be well received. Commented Jul 26, 2017 at 21:08
  • You make some interesting points, heather, but they're only tangentially related to this mentoring experiment proposal, which may explain the downvote and current lack of upvotes on this answer.
    – PM 2Ring
    Commented Jul 26, 2017 at 21:17
  • 1
    I agree with the sentiment. I hang out in the processing tag, and it's overall a more welcoming and "fun" environment than the java tag. So what you're talking about already happens on a tag-by-tag basis. The problem is, the large tags (where most of the questions are anyway) are already "too large" by themselves, even if you isolate them. Commented Jul 26, 2017 at 22:17
  • @PM2Ring I think it's really relevant. I think that only changes like this one will really make it more welcoming for new users, and that's why I don't think this project is the right step.
    – auden
    Commented Jul 26, 2017 at 23:12
  • @KevinWorkman indeed, that's true, but I think that just separating things, to some extent, from the flurry of SO would begin to help. Also, I think that it would help if (I know this is a meta tag) things were tagged beginnner/intermediate/advanced questions in a queue, meaning that users less willing to deal with new users could just ignore the beginner tag. That helps subdivide large tags further, and clean up old user/new user interaction a bit.
    – auden
    Commented Jul 26, 2017 at 23:13
  • 1
    I'm sure that I would not learn so much from SO if the questions were split up by site instead of tag. Commented Jul 27, 2017 at 18:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .