11

I was wondering whether the number of unanswered questions on Stack Overflow has increased over the past years.

Is there a graph I can see showing the percentage of unanswered questions on the y-axis and time in the x-axis?

8
  • this is the key question, isn't it? early on, there were far more answers posted a week than questions, but over time, this gap closed, then by 2020 the number of questions asked per week became larger than the number of answers posted per week. Perhaps if there was an answer to why there's been this long-term trend of less and less answers being posted, we'd have something that could be addressable.
    – Kevin B
    Commented Aug 30, 2023 at 20:19
  • @KevinB I wonder if the questions being asked nowadays require a higher knowledge level than earlier questions. If that is the case, then less people would be qualified to answer, resulting in less answers being posted. Just a thought, no data to back this up. Commented Aug 30, 2023 at 20:29
  • 1
    My assumption is there's a general amount of time that a given generation of users stay active on the site. For the first year, new users are very active and ask/answer/vote a lot. then year by year less and less of those users from that year stay. The retention of users past that one year mark seems to have been in decline for a long while, and these groups of year 2-6 users aren't being replaced by incoming users.
    – Kevin B
    Commented Aug 30, 2023 at 20:35
  • 6
    @DanielBlack from anecdotal experience - people seem to be posting plenty of basic questions which have copious dupes. So...not really.
    – VLAZ
    Commented Aug 30, 2023 at 20:36
  • so each data point is what? of the questions asked up to that point in time and what percentage were unanswered at that point in time? or questions asked since the last point in time, and how many were answered... before the next point in time? at any point in time later?
    – starball
    Commented Aug 30, 2023 at 20:54
  • I mean, if you take a count of how many questions are asked per day and compare it to how many answers are posted per day, is that not a relatively clear picture of how many unanswered questions you can expect to exist? it's not a direct measurement, as that'd be quite complicated given questions exist for more than one unit of measure,
    – Kevin B
    Commented Aug 30, 2023 at 20:55
  • if anything, it's a decent baseline, but certainly not the whole picture. Many questions receive multiple answers, which means there's even more unanswered questions than you might expect by just comparing incoming questions and answers 1:1
    – Kevin B
    Commented Aug 30, 2023 at 20:58
  • I can only speak for Blazor Tag questions, but I would say that more questions asked now are very context specific. Such as 1, Questions about supplier's libraries [often paid for] where the OP can't get it to do what they want to. 2. Trying to use an exotic cocktail of technologies that no one has an experience with [and often don't play well together]. 3. Asking unfocused or Newbie questions where you know you're going to get into a discussion [so you leave well alone]. Commented Aug 31, 2023 at 8:05

1 Answer 1

22

The obvious attempts to research this question will show an almost equally obvious bias: questions asked longer ago have had more time to receive an answer.

Let me instead focus on a different question: of questions asked within a certain date range, how many receive their first answer within, say, a week?

I wrote a SEDE query - a quite slow one, but the results should be cached now - to count questions that are answered within a week, grouped by the year they were asked. I also wrote a simpler query to get the total number of questions asked in each year (one might try to determine this by repeatedly using the site search, specifying date ranges like so; but this will not capture deleted questions.)

From here we can simply divide through and multiply by 100%:

Year Percent Promptly Answered % decline vs. previous year
2008 94.9
2009 89.9 5.3
2010 83.2 7.5
2011 78.8 5.3
2012 73.8 6.3
2013 68.4 7.3
2014 62.8 8.2
2015 61.3 2.4
2016 59.0 3.8
2017 56.8 3.7
2018 54.6 3.9
2019 54.1 0.9
2020 50.7 6.3
2021 48.2 4.9
2022 46.0 4.6
2023 43.0 (partial data) 6.5

I'd be cautious of over-interpreting this data, but several things stand out to me as potentially noteworthy:

  1. Prompt answer rates have consistently declined year over year since the beginning. This is almost certainly to be expected; as the low-hanging fruit gets picked, there's less worth answering, and duplicates (ideally!) get closed before they can be answered. Of course, there are other reasons, related to actual question quality. (I could look in to average question scores for questions by year; but it surely would be too much work for the database to consider only votes promptly cast, and anyway people would argue about how accurately these votes actually reflect question quality.)

  2. The rate of decline suddenly slowed in 2015 and again in 2019. This could be related to increased Meta activity in 2014 surrounding changes to policy trying to be "more nice" to new users, and to 2018-2019 changes to the Code of Conduct (originally planned out in 2018, leading to the drama surrounding the firing of Monica Cellio as a SE moderator in 2019; it would be interesting to check whether similar patterns are observed on other SE sites.)

  3. The rate of decline suddenly picked up in 2020 - it makes sense that this would correlate with the COVID bump, as total question volume bucked the longer-term downward trend and likely reflected a lot of newcomers. (Again, it would probably be too much work for the database to aggregate the questions by the age of the account asking the question, at the time of asking.)

  4. The rate of decline has increased again this year. This could of course be related to AI, but it's hard to see exactly how. It's notable that there have been new changes to the Code of Conduct which, by precedent, would be predicted to decrease the rate of decline.

But again, all of that is rather speculative. The most important thing here is: the current rate of decline is not out of line with historical data. In fact, if the trend from the first five years were extrapolated to the next ten, we would only be seeing about 35.5% of questions getting prompt answers today.

Note that the "% decline" figures here are based on ratios, not differences of the percentage figures between years. It doesn't make sense to extrapolate numbers like this linearly, because by construction they cannot go below zero; I am extrapolating them by ratio (i.e., exponentially) instead.

As for reasons, aside from the vague speculation included above, I'll leave it to someone else.

3
  • 3
    I rewrote your query slightly and expanded it to include the percentage and difference calculations in place so it can update as SEDE does (link to query). I agree with you that it's likely that a significant portion of of these "unanswered" questions are actually closed as duplicates of existing questions and have effectively been answered despite having received "no answers". That's somewhat harder to query for in SEDE.
    – Henry Ecker Mod
    Commented Aug 31, 2023 at 0:23
  • So in approximately 2032 no questions will be answered anymore on SO. Commented Sep 1, 2023 at 6:01
  • @NoDataDumpNoContribution no; that sort of conclusion is exactly why I am extrapolating by ratios rather than linearly. Commented Sep 1, 2023 at 12:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .