50

In the Tavern, the topic of post quality came up recently. One of the ideas thrown out was to delay the ability of newly registered users from posting immediately - perhaps a 30 minute delay to 'encourage' the new user to do just a bit more research before filling out the textboxes. Thus, my request:

Is it possible to get information on how often brand new user accounts get their first question closed within a short period of time? I'm not sure what to define "short" as. An hour? Two? A day?

"Brand new", in this case, would be where account creation and question being posted are seconds to minutes apart.

And, because I don't want to make Shog's life too easy, would it be possible to see how many of those eventually get reopened?

9
  • Depends on what you mean by "brand new" - an account that's existed for less than 24 hours until its first question/answer, or an account that has made fewer than X questions/answers over a period of time. One could query around a few loose requirements like that through SEDE.
    – Makoto
    Commented Dec 15, 2015 at 4:08
  • 3
    Brand new = Account created and question posted within minutes of one another.
    – Andy Mod
    Commented Dec 15, 2015 at 4:09
  • 2
    Maybe it is just me being a pessimist but wouldn't a delay from creating an account just cause people to wait until they can post after they create an account rather then do anymore research or reading of site policies?
    – Joe W
    Commented Dec 15, 2015 at 19:15
  • Entirely possible, @JoeW. I wanted to get a bit of data though to help with that discussion.
    – Andy Mod
    Commented Dec 15, 2015 at 19:39
  • I remember Shog saying that users that create an account before asking a question fair better than those that join-n-ask immediately, found it meta.stackoverflow.com/a/310850/792066
    – Braiam
    Commented Dec 15, 2015 at 23:34
  • I don't know which way this cuts, but most websites try to have as few barriers as possible to joining because that's when most users are turned away (right at the beginning).
    – Owen
    Commented Dec 15, 2015 at 23:35
  • 9
    I can see the appeal of the idea, with all the influx of crappy questions from careless newbies. However, I would find it offensively condescending to be told "Please take 30 minutes to do some more research on your question, since you're obviously an idiot." when trying to ask a first question. Exactly the valuable new contributors would suffer from this feature the most. (And I suspect that the most enthusiastic idiots would gladly wait 30 minutes to ask their crappy questions regardless.) Commented Dec 15, 2015 at 23:47
  • 4
    SO would be well ahead if it wasn't so easy to create a new account to bypass a question ban or a rate limit. Wouldn't it be nice if users where invested in their account, ensuring they'd make an effort to keep it in good standing. Right now there is no motivation whatsoever for users to do this. I've always assumed a one-day waiting period would be appropriate to encourage this. Now throttling question rate by rep starts to get practical as well, giving yet more reasons to get the user to invest. Getting SE to agree with this is in the insurmountable obstacle. Commented Dec 16, 2015 at 15:22
  • 1
    I think it is better to show them more information (and making sure they process it) before posting instead of just having a waiting time (especially since some people might actually have researched on SO quite a lot before but waited with the registration until they had something to contribute). What about a multiple choice test (focused on some examples) to complete successfully before posting a question? Commented Dec 17, 2015 at 13:37

2 Answers 2

51

Just out of curiosity, I wrote an SEDE query to plot the total number of first questions, and the fraction of them that are closed, as a function of the account age when asked.

Alas, SEDE doesn't provide enough information about deleted questions to let them be associated with their authors, leading to a considerable systematic error (since closing and deletion tend to be highly correlated). Fortunately, Shog9 was kind enough to re-run the query on the full SO database and post the resulting graph. Here's what it looks like:

Graph of closed first questions as a function of account age when asked, with a logarithmic x-axis

(If you're curious about how this compares with the original graph I made, without deleted questions, you can find it in the edit history of this answer.)

The x-axis is the base 10 logarithm of the account age in seconds. The major peak in the total first question counts (blue line) is centered around 103 seconds ≈ 15 minutes; apparently, that's how long it takes for most people to ask their first question after registering, with most of the variation falling within plus or minus a factor of 10 of that (i.e. 1.5 minutes to 2.5 hours).

The red line in the graph, which shows the fraction of first questions that are closed, does show a certain amount of time-dependence. Ignoring the crazy wiggles on the left below the one second mark (which are almost certainly just random noise, amplified by dividing one small random number with another), the closure rate of first questions does peak at about 102.2 seconds ≈ 2.5 minutes after account creation, with over a quarter of all questions asked that quickly getting closed eventually.

Conversely, on the other side of the big peak in the blue line, the red line drops to a minimum around 103.5 seconds ≈ 1 hour, with only about 11% of questions asked at that point getting closed. (After that, the rate starts climbing again a bit, peaking at about 18% for questions asked 105.1 seconds ≈ 1.5 days after account creation.)

That said, the difference between the highest and lowest first question closure rate is still only a factor of about 2.5. And the highest peak in the fraction of first questions closed corresponds to a rather low overall first question rate — sure, questions asked less than 5 minutes (≈ 102.5 seconds) after account creation do have a relatively high risk of getting closed, but very few first questions are asked that quickly to begin with, so any intervention that only targets that small group of questions is unlikely to have much effect on the overall first question closure rate on SO.

Also, it's important to remember that, as any statistician will tell you, correlation does not imply causation. While these results do seem to somewhat support the impression that, on average, bad first questions tend to be asked a bit sooner after registering than good ones, that doesn't necessarily mean that forcing the authors of bad questions to wait a few minutes would cause them to improve their question at all.

Or, to put it in other words, a perfectly consistent explanation of these results would simply be that almost all new users actually ask their first question immediately after registering — the better questions just take, on average, a bit longer to type in.

Ps. The funny-looking oscillations in the blue line near 105 seconds ≈ 1 day can be explained by the fact that people have circadian rhythms, and so the time between registration and first question tends to cluster around multiples of 24 hours. The upward slope on the right, peaking at 107.5 seconds (= almost exactly one year) is a kind of an artifact of the exponential binning method used to plot the data; the rate of first-question-asking isn't actually increasing, but the bins into which the questions are grouped by account age before they're counted get exponentially wider the further right you go on the graph. After one year, the frequency of first questions starts to fall off faster than exponentially, presumably due to the automatic deletion of old unused user accounts.

7
  • 27
    ignoring deleted questions doesn't feel okay. Checking my own CV review stats, it looks like almost 2/3 questions I reviewed are deleted (CV stats page says I've got almost 28K reviews, while profile page shows only 10K, including all other kinds of reviews). Difficult to make conclusions having only about 1/3 of data available
    – gnat
    Commented Dec 15, 2015 at 7:15
  • 2
    "which is almost exactly one year; beyond that point, it seems the frequency of first questions starts to fall off faster than exponentially." that's because those Q's are normally delete by roomba
    – Braiam
    Commented Dec 15, 2015 at 18:11
  • 1
    @Braiam: That's one year between account registration and asking the first question. I don't think the roomba takes that into account. Commented Dec 15, 2015 at 18:16
  • 2
    Well, not roomba properly, but the cousin that deletes users that makes your datapoints looks like that.
    – Braiam
    Commented Dec 15, 2015 at 18:20
  • 4
    @Braiam: Thanks for the link, I didn't know that even existed. That does raise a question, though: in the linked answer, Shog says that the account roomba kicks in after 6 months (≈ 10^7.2 seconds) of inactivity, so one would expect the turning point in the graph to be close to that. But it's not -- it's clearly at 10^7.5 seconds, i.e. a full year. Has the time limit been increased since that answer was written? Commented Dec 15, 2015 at 20:58
  • Account deletion is throttled (wasted 15 minutes trying to find the link before posting that one instead), so anything older than 7 months (30 days if questions didn't reach 100 views + 6 months + 6-8 months) may be skewed.
    – Braiam
    Commented Dec 15, 2015 at 21:06
  • 2
    Here you go: same graph, with deleted questions included.
    – Shog9
    Commented Mar 31, 2016 at 0:55
9

I admit, I got nerd sniped on this one.

Here's the query.

Here's the graph:

enter image description here

This gives me the impression that, while it's a fairly rare occurrence (< 200 at any given point in time), the likelihood that this has happened has greatly increased since 2014. Also, I'm tempted to blame the spike in the latter half of December on hats, but they weren't fully turned on then.

1
  • 21
    The spike in December likely has to do with the roomba & how recent closed questions haven't been deleted yet.
    – hichris123
    Commented Dec 15, 2015 at 11:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .