60

In this blog post The Loop: Our Community Roadmap for Q3 2020, the issue(s) with downvoting has been mentioned. It reads:

Downvotes research (July)

Receiving downvotes on Stack Overflow can be a frustrating and confusing experience. We currently ask users to downvote posts that are not useful or are unclear, but this can be subjective and interpreted in different ways. We will run a short targeted onsite survey to better understand what motivates users to downvote a post and use this data to inform inclusion and engagement opportunities.

There was also a recent discussion here about encouraging more downvotes: What can we do to encourage downvoting?

Do we have any ideas about what we could do or change to encourage people to spend a little bit more of their votes on downvoting posts?


  • I agree that "receiving downvotes can be a frustrating and confusing experience". I've had this experience early on Stack Overflow when I was learning the rules. Even recently, I posted a somewhat low-quality post, on Mathematics, which got multiple downvotes. I was confused as I have tried to include necessary information and format my question in a way that is legible. I later learned my terminology was not accurate and the question can be perceived as "Unclear" and that was the reason for downvotes (so I edited my post, and it got couple upvotes). However, I was not frustrated since I knew how Stack Exchange communities work and I didn't take the downvotes personally. But that's me, a user who has been around long enough to know that.

  • That's only one side of the story though. While getting downvotes may not be a pleasant experience, as I said here – Answer to What can we do to encourage downvoting? – (and many other users phrased it even better), we should encourage downvoting since it is crucial for effectively moderating the site and maintaining the quality.

What I feel is that the company has been focusing on the first aspect and overlooking the other. Well, not talking about the value of downvotes for curation in the blog post can be considered a sign of that. That said, it's great that we are getting a survey to reflect our thoughts.


What to discuss:

What can be done to encourage downvotes while making them less "confusing and frustrating", especially for newcomers? *



* I understand that these two issues have been discussed separately, but a focused discussion can be beneficial. It may even affect the questions in the survey (if it's not too late to change them).

10
  • 45
    I don't know about you, but I'm happy to downvote a few hundred more posts in the name of science.
    – Cody Gray Mod
    Commented Jul 28, 2020 at 22:27
  • 39
    It's death by a thousand little cuts, just one more to get the site they want instead of the one we provide for them. All you can do is ignore them. Commented Jul 28, 2020 at 22:37
  • 9
    Thank you for posting this. It needed to be asked. You're my hero of the week!
    – Dan Bron
    Commented Jul 28, 2020 at 22:44
  • 21
    I deleted irrelevant comments that had veered off into a tangential discussion about whether downvoters should leave a comment. We've had that discussion hundreds of times; I'm tired of it, and it's distracting from the point of this question. Whenever the question is, "what happened to the comments?", the answer is probably going to be "a moderator deleted them because they were useless."
    – Cody Gray Mod
    Commented Jul 28, 2020 at 23:03
  • 20
    In my tags I’m recently struggling to understand the upvotes, so I’m hopping there will be also an upvotes research as well.
    – ead
    Commented Jul 29, 2020 at 11:40
  • 17
    When any company has trouble understanding their own product, the instinct is always to "conduct some research." I understand that impulse, but why not ask some folks who already know the answers to your questions? There are literally hundreds of such people, some of which used to work for you. Better yet, why not try using your own product? Commented Jul 29, 2020 at 17:25
  • 14
    @ead For real. You downvote a bad question (literally google the title of the question with site:stackoverflow.com to find the duplicate types), it immediately gets upvoted by the FGITWs who have answered it. It's made significantly worse since questions got +10 reps because OP doesn't have to delete a question because it takes 5 downvotes to match one upvote.
    – adiga
    Commented Jul 29, 2020 at 17:58
  • 5
    More research is always good if it is conducted with an open mind. All the good things that come from downvoting surely deserve more data to be backed up as well. The company does this research probably because they identified it as a point of friction between users and I guess they may also have a different vision of what SO is than the meta community. Difficult to say without a clear statement about the vision for SO from the company side. Commented Jul 30, 2020 at 7:51
  • 1
    You gotta have balls of steel to post a Q on Mathematics.
    – mxmissile
    Commented Sep 9, 2020 at 13:55
  • 1
    Related (forward in time): Downvotes Survey launching Oct 15, 2020 Commented Oct 27, 2020 at 20:06

4 Answers 4

34

My thought process

I downvote questions when I think they are not a worthy addition to the information we have already collected here. When I look through new PHP questions I try to find questions I can answer. I open a new question and I ask myself "Is this question deserving my time and answer? Will people benefit by finding this question in the future?" If the answer is no, I downvote. I will then try to analyse if the question is even on-topic and vote to close if necessary. Then I move on to the next question.

I also downvote a lot of answers. I downvote sloppy answers, with no explanation, with bugs/broken code, with poor formatting (if I can't fix it easily myself) or if the suggestion is not the best suggestion in my opinion. Sometimes I downvote answers that are just lazily copying the same thing for the hundredth time or are just pointing a simple typo; I don't find them useful.

I also try to downvote off-topic questions, and answers which are not really answers.

My research

The primary reason why people find Stack Overflow unwelcoming is because they do not understand its purpose. They think this is a place where they can ask a question about their problem and get personal help. People get offended that we do not offer this kind of personal help. See Why do some people hate Stack Overflow?

Hear me out. Plenty of people come here for the questions, not for the answers. They are interested in asking about their own problem, not searching through existing answers for the one that solves it. They want someone to solve the problem for them. It's frustrating for them when we tell them the question needs more details, focus, or debugging details. They take downvotes as personal offence. They want help with their problem and someone thinks their problem is not interesting/not a good fit for Stack Overflow. See What is bad about Stack Overflow?

People must understand that SO does not offer personal support. This comment from Reddit explains SO purpose well:

SO q&a's are ideal for asking questions about and giving answers on standard situations and problems, so that's why it has a higher bar to uphold and needs a tighter-knit (possibly sometimes outwardly hostile) community for posing these questions and answers.

What can be done about the perceived hostility?

According to Joel Spolsky all questions are welcome even if they are the millionth duplicate. See Strange and maddening rules

We decided that newbies had to be welcome. Nothing was too “beginner” to be a reasonable question on Stack Overflow… as long as you did some homework before asking the question.

One thing I’m very concerned about, as we try to educate the next generation of developers, and, importantly, get more diversity and inclusiveness in that new generation, is what obstacles we’re putting up for people as they try to learn programming. In many ways Stack Overflow’s specific rules for what is permitted and what is not are obstacles, but an even bigger problem is rudeness, snark, or condescension that newcomers often see.

We could restrict questions. We could filter out low-quality ones, but would it work? Is this what the company is willing to do? I doubt it. The system is not good at judging quality of all questions automatically. We need simple heuristics, but beyond that it is our job to rate the content being posted here... which is what we do using downvotes and upvotes.

Maybe the post score should be hidden from the question asker? Do they really need to see the score? If there is something that can be improved we will either close it, edit or post a helpful comment. Post score is not very helpful to the OP.

Joel claims that rude, snarky and condescending comments are a problem. In my opinion, they are a symptom of a problem. Question asker gets frustrated with the jumping score, because they do not understand what it means for them. They post a rude comment under the question demanding an explanation! Curators get annoyed by lack of research, details or overall low quality. This frustration also leads to impolite comments.

My suggestion

  • Hide the question score from OP. At least for the first two days. Everyone else can see the question score, and the asker can see the score of the answers. Post score on answers is never hidden.
  • Improve the onboarding process for new users. Don't make it easier for them to ask questions. If they want to ask a question they must put effort into it! We need the system to explain it to them, guide them through question asking tutorial, and make them confirm that they understood the quality guidelines.
  • Give curators more power and better tools. We need a delete review queue, we need more votes, we need to stop being penalized for downvoting answers, we need improved review queues, and we need incentives for duplicate closures. We want to sieve through the content, but we should not feel restricted by the system in doing so, otherwise, we feel like we are not welcome to do so.
  • We need better search tools. Google is not enough. Many people don't have access to Google and some don't use Google at all. Stack Overflow search is notoriously bad at bubbling up the useful relevant content in search results to the point that people ignore the duplicate search popup altogether and search in external search engines.
9
  • 4
    I feel like all of your suggestions have been pitched and are floating around somewhere between here and Meta Stack Exchange. Some research should be done to see where those ended up.
    – Makoto
    Commented Jul 29, 2020 at 22:09
  • 3
    "otherwise, we feel like we are not welcome to do so". Exactly, we should care about making the site welcoming to newcomers (while more importantly educating them on how to use it), but why should all the SMEs and volunteer mods be forgotten? We need to feel welcome too.
    – M--
    Commented Jul 29, 2020 at 22:18
  • 3
    "Hide the question score from OP" No. It will not hurt any less and it will prevent people that really want and can improve from doing so. Down votes are useful for teaching new users about rules. After two days it will be too late, especially since they might be asking additional questions and dig deeper hole for themselves.
    – Dalija Prasnikar Mod
    Commented Jul 30, 2020 at 21:14
  • 3
    @DalijaPrasnikar Can you elaborate what rules does it teach a new user to obey? I am fairly certain that voting was not meant to be used as an educational tool. We use it only to rate content. OP is not meant to take votes personally. I would encourage you to reread this answer once more meta.stackoverflow.com/a/357437/1839439
    – Dharman Mod
    Commented Jul 30, 2020 at 21:41
  • 4
    If someone rates your content poorly then if you care you will try to figure out why. You will try to improve your content. If you don't see that it is poorly rated then you will not bother to do anything about it. That is educational part I was talking about. Yes, it would be better if people could learn by other means...but that is different story. Hiding down votes from new user is the worst possible scenario. It will not improve anything, it can only have negative impact on those that had bad start, but want to be better.
    – Dalija Prasnikar Mod
    Commented Jul 31, 2020 at 7:14
  • 3
    I mostly agree with everything you said, except that point that down votes should be hidden. That is the only reason I haven't up voted this.
    – Dalija Prasnikar Mod
    Commented Jul 31, 2020 at 7:15
  • 3
    @DalijaPrasnikar Most new users don't understand the post score the way we do. It's only a distraction and point of confusion. You could still see the score in your profile or in the timeline. I am just suggesting to hide it when they ask a question so they don't go berserk like they usually do.
    – Dharman Mod
    Commented Jul 31, 2020 at 10:06
  • 4
    I agree that they don't understand point score, but the only way they can grow on this site is to start understanding how it functions. The sooner, the better. Hiding only postpones the trouble. It will accomplish nothing. As soon as people get banned for their poor posts, you will have flood of questions on Meta "Why am I banned, when my posts were so good?"
    – Dalija Prasnikar Mod
    Commented Jul 31, 2020 at 11:41
  • I started using Stack Overflow as a freshmen in college over a decade ago, newly exposed to coding. Now look at me! Those downvotes were a clear signal to me that I was doing something wrong. In the carrot/stick adage, downvotes were the stick. That helped me realize I needed to shape up quick if I wanted to get answers to my questions. I don't think new users are incapable of understanding the point score on a question. If they aren't capable - are we really going to be able to help them anyways?
    – mason
    Commented Oct 15, 2020 at 18:01
63

Downvotes are now—and probably always will be—an essential feature of a functional content-rating system. If we want fewer people to have this poor experience, we need less downvote-worthy content to get posted on the site.

I'd like to see Stack Exchange focus their energy on researching why users downvote posts and using it to find better ways to guide users toward not posting content that merits a downvote in the first place.

Much of this content is coming from users who seemingly haven't read—or at least haven't internalized—the guidance provided by existing Help Center pages, like How do I ask a good question? or What topics can I ask about here? Figuring out the most common problems and proactively getting information about avoiding them in front of a user who might be about to run afoul of these issues would go a long way.

Imagine if the system saw a user posting about a crash without including an error message/stack trace and prompted them to provide it. Or, if it saw them type "What is a good library..." and informed them that recommendation questions are off-topic. A one-sentence question might receive a warning that such questions tend to be badly received. Images of code could be more thoroughly blocked.

Many of these are fixable (and I don't know what percentage of the problematic questions that they make up), but the asker tends to go through a long cycle without a lot of feedback. If they're really lucky, a curator has a bunch of helpful canned comments that explain why the post is being downvoted and provides that information to the asker. In so doing, the curator also risks becoming the victim of the asker's rage at their question being poorly received. Instead of understanding that the system/rules forbid such questions, the asker thinks it is just some self-important curator.

Whatever the most common reasons are, Stack Exchange is in the best position to help users craft a better post before it shows up on the site. Sure, it's all in the Help Center, on Meta, and in the FAQ if you look for it, but clearly a lot of people aren't reading these things. More proactive changes are clearly needed, and knowing the most common problems would certainly help inform those changes. I hope Stack Exchange uses their research to that end.

5
  • 21
    No amount of clear advice can help with a user whose sole aim is to get an answer to hand in on Monday morning. Such users, (and there are a lot of them), just don't care about misusing the effort of others as long as they get their answer NOW, and will freely ignore and/or click through any help links and nag screens:( Commented Jul 29, 2020 at 3:49
  • 5
    @MartinJames But those aren't the only users posting questions that don't align with the guidelines. I don't think anyone thinks we're somehow going to eliminate the need for moderation entirely.
    – BSMP
    Commented Jul 29, 2020 at 7:36
  • 2
    A good start would be to surface the links for the most critical Help Center articles in the left hand menu strip of the page, instead of hiding them behind an obscure icon. Commented Jul 29, 2020 at 17:20
  • 1
    It would be better if they read Jon Skeet's guide. It is much more actionable and to the point (and it has lists). (Yes, it is linked, but how many go there?) Commented Jul 29, 2020 at 18:37
  • 4
    I like the idea of guiding users while they make mistakes, instead of before or after. Sure, it might not catch deliberate offenders, but then again that's what downvotes are there for. Commented Jul 30, 2020 at 14:34
37

The company has received far too much signal in the sense that there are a lot of people online who bemoan how strict Stack Exchange's voting model is, and view it as a form of hostility, toxicity or antagonism against people who "just want to learn to code".

Put another way, the company hasn't received as much signal about the benefits of downvoting, content moderation or anything of the sort, since those kinds of posts or discussions don't happen in memes or on other social media platforms.

It's reached the level that people are going to be spending development time to investigate this, so I'm going to say what I'm thinking out loud.

We're gonna lose this fight.

We downvote for reasons which aren't easily patterned, but are largely tied to the notion of the question being unclear. There are some cases in which people do abuse the downvote feature for any given reason, but those are in the minority. I truly believe that will be overlooked due to the overwhelming amount of signal in complaining that downvotes are somehow unfair, unjust, or unwarranted.

I'm extremely pessimistic on this front, since we've been bludgeoned by data before.1 My gut tells me that this research is going to be used to justify changes to the voting system that we won't agree with, but also won't have a way to fight off since, well, there really are a lot of people who dislike it.


All of that is to say... you should downvote the poor quality content while we still have a chance to do so.


1: Yes, I know that there was a partial apology for this particular fiasco, but that doesn't change the fact that decisions were made based on incomplete data.

12
  • 6
    First: I agree with you. I’ve actually been expecting them to nerf downvoted for years, starting with when the true motivations behind “project reduplication of reduplication” came to light, then even more after change to increase rep rewards for upvotes on questions, with its transparent social justice veneer. I downvote less for “unclear”, and more for “asking others to do your work so you don’t have to”. Also, at least on my phone, when I click your “bludgeoned by data” link, Sara’s chat message is cut off, and I don’t see anything about “data” in it. But I assume it was the 0.015% thing.
    – Dan Bron
    Commented Jul 29, 2020 at 15:57
  • 9
    "those kinds of posts or discussions don't happen in memes" - oh god this is so true, every single time I see a meme on SO it's about the site being compared to a dictatorship or something. It's so sad to see most of the outsiders have this perception of SO. They only care about that one time when they asked a homework question and it got downvoted/closed, forgetting about the countless times the website has helped them through existing content. Meh. Commented Jul 29, 2020 at 16:42
  • 2
    I would add that technically, there are ways to tame this issue without damaging content quality. Triage, annotations, improving tooling for 10K/20k delete voters come to mind - and I bet there could be more. However, past experience (such as abandoning triage development) makes me pessimistic that these ways will be explored, things more likely will end up the way how you describe
    – gnat
    Commented Jul 29, 2020 at 16:42
  • 3
    @Marco Bonelli: Re "It's so sad to see most of the outsiders have this perception of SO": I am not sure that is the case. All the searches that result in finding a useful answer on Stack Overflow we never hear about (though the signal-to-noise level is slowly, but steadily decreasing). And the negative voices are amplified much more by social media (as negativity produces more engagement ("That can't be. I must stop it!") than positivity ("Ah, that is just corporate speak. I can ignore it.")) Commented Jul 29, 2020 at 18:09
  • 2
    @PeterMortensen good point which makes me even more sad :\ Commented Jul 29, 2020 at 18:15
  • 7
    "those kinds of posts or discussions don't happen in memes" - How's this? i.imgflip.com/49t4o1.jpg Commented Jul 29, 2020 at 18:21
  • 2
    @AndrewMyers Goddamn it how do I retweet this???
    – Dan Bron
    Commented Jul 29, 2020 at 18:35
  • 2
    I'm not sure about the conjecture in this answer. Would it also count as losing if SO became one platform among many, curators and answerers would emigrate somewhere else and take a copy of the worthy content with them? The company needs the money and money is only generated by high quality content, or isn't it? If the company would act rationally it would probably keep the downvoting, I'd say. Commented Jul 30, 2020 at 9:13
  • 1
    @Trilarion: Yes. We'd have lost the only thing that would matter in the sea of platforms and content - clout. Smaller sites may promise one thing or another, but not even the best content out there is reachable unless someone knows about how to get it. That's taken Stack Overflow decades to reach, and anyone or anything else starting from scratch is underestimating the level of effort it'd take to even approach a tenth of that.
    – Makoto
    Commented Jul 30, 2020 at 14:28
  • 2
    But still I'd say that downvotes increase the quality of the content and therefore generate revenue. Complaints on Twitter may talk loud but maybe money talks louder in the end. SO may try to square the circle with the downvote research and in the end may just give up and leave it like it is. Commented Jul 30, 2020 at 15:44
  • 6
    @Trilarion: I suppose I'm really taking the absolute pessimistic view on this. If what you were saying were accurate, they would have led with a neutral statement about the fact that they want to simply learn more about downvotes. Instead, they decided to suggest that receiving downvotes can be "frustrating" and "confusing". It isn't confusing at all if people bothered to read the tooltip text over the down arrow.
    – Makoto
    Commented Jul 30, 2020 at 15:52
  • 2
    I fully understand where you are coming from and I share all your concerns. I also agree that any mass emigration would only be the second best option. I just hope that the company sees the light at the end, because if not there is not much we can do, they are in power of this platform ultimately. If they go down the wrong path, which isn't really sure yet, I will have to emigrate and start somewhere else new though. What else could I personally do? Commented Jul 30, 2020 at 16:08
30

What I feel is that the company is focusing on the first point and overlooking the other

Well, it's blatantly clear that this is the case IMHO. They're trying to "optimize" the system to have as many users and posts as possible instead of good users and posts. You can't really save somebody from getting downvoted if they don't even bother reading the How to Ask tutorial. Of course that will be a frustrating experience for some, but is that really solvable? A robber could very well get frustrated because the cashier has an empty register. Is that something we care about?

It also really feels disheartening to see every single post from the company repeat this "let's make sure everyone is welcome no matter what" logic without acknowledging any of the MSO feedback and counterarguments.

Most importantly though, why should a downvote be a frustrating experience only for the asker? Sometimes I feel like I am the only one who is getting frustrated for casting that downvote, when the asker clearly doesn't even care about it as long as they get an answer. I surely can reach my daily dose of frustration very easily if I just keep looking for a decent question to answer under my favourite tags. This other side of the medal is also very important to consider.


Downvotes research: why do we need that?

I think the only reason we need this is for Stack Exchange (the company) to open their eyes. Of course, this is only assuming that they are willing to do so in the first place.

I'm really looking forward to seeing the results of this, and I'm hoping this research makes the company realize that people here on SO mostly use downvoting for good and legitimate reasons. I also hope they realize that some users don't feel welcome simply because they can't possibly be welcomed if they are here just to exploit the platform and its good hearted users to get their homework done for the next day.

The core issue here is not making sure people think about it twice before downvoting a post, but rather making sure that new users are able to understand how to avoid such scenarios in the first place. I think we already do a pretty good job at this, with the countless popups, tutorials and help center links. So what more should we do? If anything...

5
  • 3
    We could add a non-public vote-free pre-post stage with the required number of iterations (could be exactly 1 iteration for some questions) until the question is in an acceptable state (sort of closed and deleted by default). They still wouldn't like it (as it is a barrier to posting (immediately)), but it wouldn't be public and they get downvoted much less (less public shaming). Commented Jul 29, 2020 at 18:26
  • 1
    @PeterMortensen not quite sure I get your suggestion: "non-public pre-post stage"? What would the criteria to pass to the "post" stage be then exactly? Seems counter-intuitive to say the least. Commented Jul 29, 2020 at 18:34
  • I remember that somewhere someone from the company acknowledged that curation is hard and thankless work, but they only used that to suggest that curators would be happy if they hadn't so many possibilities to curate, if I remember it correctly. I think it was about potential automatic reopening of closed questions after editing. Commented Jul 30, 2020 at 7:59
  • I assume Peter is talking about a "sandbox" environment where new askers can post their question for clean-up before it goes onto the main site. This was actually tried a couple of years ago, in the Mentoring experiment.
    – PM 2Ring
    Commented Aug 9, 2020 at 10:36
  • It can be depressing to see all the low quality questions from new members who ignore (or haven't read) the Help centre info. In fact, some people have userscripts that block questions from low rep members so that they can look for good questions without getting too upset. Obviously, that's not good for the long-term health of the site, but I guess it's better than high rep members leaving the site totally.
    – PM 2Ring
    Commented Aug 9, 2020 at 11:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .