70

I was analyzing the proportion of questions which received at least one answer within 48 hours, using Bigquery:

select da, avg(answered)
from
(SELECT a.id, date(a.creation_date) as da, max(case when TIMESTAMP_DIFF(b.creation_date, a.creation_date, hour) <= 48 then 1 else 0 end) as answered
FROM 
`bigquery-public-data.stackoverflow.posts_questions` a 
left join
`bigquery-public-data.stackoverflow.posts_answers` b
on a.id = b.parent_id
group by 1, 2)
group by 1

Here's what I get:

This figure shows the proportion of questions asked on a given date which received at least one answer within 48 hours

This figure shows the proportion of questions asked on a given date which received at least one answer within 48 hours, and while there is clearly a decreasing trend, something happened on 31st May 2019, because of which there is a discrete drop (from 68 per cent on 30 May to 56 per cent on 31 May). The values remain low after that point. What happened on that day?

12
  • 51
    On what day was this data pulled? usually jumps like that, a year in the past, are due to roomba related tasks, and the jump simply exists approximately a year in the past from when the snapshot was taken.
    – Kevin B
    Aug 18, 2020 at 19:53
  • 21
    The data was pulled today itself, but the last update on Bigquery was on 31st May 2020. Which seems to be in line with your explanation. Aug 18, 2020 at 19:55
  • 4
    @HansPassant: that shouldn't be a problem given that I am only considering answers within 48 hours of the question being asked right? I can understand your concern being valid if that condition wasn't there. Aug 18, 2020 at 23:53
  • 1
    I'm not familiar with bigquery so I have a question. Is it correct that the query includes questions that get marked as dupes and questions that get closed for, e.g. low quality reasons? I'm wondering what causes the "declining trend"... more dupes, more low quality reasons or ... Aug 19, 2020 at 6:25
  • 5
    Roomba deletes questions with score <= 0 and no answer after a year. That's why the apparent rate of questions with an answer seem to be higher in the past. But they weren't really. The data before May 2019 would need to be adjusted and the deleted questions be added back in (depending on what you want to investigate). Aug 19, 2020 at 8:24
  • 1
    That's definitely true Trilarion, but it shouldn't result in such a stark change in trend as is seen.
    – Travis J
    Aug 19, 2020 at 8:26
  • 1
    There are also Roomba deletion waves for 9 and 30 days that would influence the apparently accelerating downwards trend at the end. Aug 19, 2020 at 8:27
  • 1
    @TravisJ Without the jump the trend is relatively stable and seems to change slowly but steadily over the years. Further investigation is needed. Is it more questions or less answers, do people ask more questions or asking more people questions (same for answers)? Aug 19, 2020 at 8:30
  • 1
    Don't Roomba tasks for 365 and 30 days only affect questions without an answer? and wouldn't that be excluded from the query in the question since it takes only question with an answer? (I'm assuming joining on the answer table would exclude questions without an answer)
    – bracco23
    Aug 19, 2020 at 10:19
  • 1
    @bracco23 no, that's not how joins work. The query is looking at ratio of answered to unanswered questions.
    – OrangeDog
    Aug 19, 2020 at 12:59
  • 1
    @OrangeDog dang, I really didn't see the left join there
    – bracco23
    Aug 19, 2020 at 13:03
  • 1
    @animuson why are you deleting wrong answers?
    – OrangeDog
    Aug 20, 2020 at 21:50

2 Answers 2

89

Nothing happened.

There is a cleanup process, usually referred to as "Roomba", that deletes unvoted, unanswered questions after a year. Any graphs that involve question counts will see this sharp drop (or rise, depending on the metric), exactly one year before the snapshot date.

1
32

Nothing happened.

If we run a functionally identical query on SEDE, but include deleted posts, we get the following picture:

Graph

As for why deleted questions matter, see OrangeDog's answer, and add user/moderator deletions of unanswered/late answered questions over time to that.

This means we're dealing with a near-constant decline in the percentage of questions that got a quick answer that's been going on since the start of Stack Overflow, without any clear speedups or slowdowns. If anything, this decline slowed down over 2018-2019, but this might've been undone in 2020.

Interpreting the findings can go many ways, e.g. more simple questions at the start, a more rapid growth of people asking than people answering, overall decline of the site, I leave that as an exercise to the reader.

7
  • 7
    "near-constant decline" At some point it must level off. My extrapolation is that in 2037 questions won't get answers anymore. :) Aug 19, 2020 at 14:31
  • 4
    @Trilarion that's probably why John Titor had to come back to solve his problems. No more SO to ask. :)
    – bracco23
    Aug 19, 2020 at 15:05
  • 14
    It's hardly surprising, over time most of the the "easy questions" will have been asked and answered already, So what is left are duplicates and difficult/specialist questions.
    – plugwash
    Aug 19, 2020 at 19:14
  • 8
    Plus the early answers have a huge reward (e.g. how to enumerate and enum in c# over 3000 rep) so there is less incentive wise now v time/reward. Early adopter advantage.
    – Andrew
    Aug 19, 2020 at 20:37
  • 5
    @plugwash: I agree with the difficult/specialist questions. A few years ago, the Rust tag was all about questions about the language. Nowadays, most non-duplicate questions are about libraries -- most of which I've never touched in my life. Answering a question about the language is usually relatively easy (with online compilers to help), answering questions about an issue with a particular library is requires much more effort -- and I don't care about most of those libraries to start with so I'm not really inclined to make the effort. Aug 20, 2020 at 8:51
  • @plugwash yeah; I imagine SE's decline in new questions and new user registrations is inevitable (unless they plan on allowing duplicates, or purging giant amounts of content). Seems like they should be focusing on one of the following, 1) Using Google hits to make money, or 2) making a research assistant service to help people understand what's already here (i.e., maybe paid 1 on 1 help). Or maybe the owners know the site is declining in new registrations and they're hoping nobody notices until they can sell it to somebody else.
    – jrh
    Aug 20, 2020 at 15:56
  • The next hot tech will boost it a bit. All the easy questions for the old hot tech have been answered already.
    – empty
    Aug 20, 2020 at 18:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .