37

It seems to me that most users with low reputation will accept answers in the comments with "That's it!" or "It worked, thanks!" but will not mark the answer as accepted -maybe because they are unaware on how the site works - while more seasoned users will mark when the answer is appropriate.

So, is there any statistical evidence or actual case studies that correlate the reputation a user has and the willingness to mark an answer as accepted?

11
  • 1
    Heh. I just got one of these an hour ago. I imagine that if you yourself have ever answered a single question, the acceptance rate on questions you ask is much higher (since you presumably understand better how the website works).
    – roippi
    Commented May 16, 2014 at 4:26
  • 17
    I don't think you really need the stats since this is obviously true. Almost every new user has trouble with this. Using a comment I often explain them the importance of voting / marking as answer and ask them to review their previous questions too. This often helps. Commented May 16, 2014 at 14:35
  • 1
    More Reputation = More Time on the Site = Understanding more of how the site works = Accepting answers to your questions. I know there was a time when I bothered to go search for my unanswered questions and marked a bunch as accepted. The time wasn't related to my rep, it was related to my knowledge of the site.
    – Rachel
    Commented May 16, 2014 at 15:01
  • 2
    @PatrickHofman It is not obviously true without the stats to back it up. It is only anecdotal :-) Commented May 16, 2014 at 15:04
  • I have a relatively low rate of marking answers "accepted", primarily because the questions I ask are often too hard to draw a valid response.
    – Hot Licks
    Commented May 16, 2014 at 21:25
  • @PatrickHofman Thanks for your efforts but even with them my impression is that a considerable part of users just pose one question take away the answer (or you never know if it was a helpful answer) and never come back at all. Maybe we can draw them a diagram next to the question. Anything that helps increase the acceptance rate a bit. Commented May 19, 2014 at 20:30
  • @Trilarion: I am afraid that despite how many effort is made, this at most will help a bit. Did you really read the beautiful interactive Getting Started? I doubt 99% of the new users did (I am sure since there is a badge for it). Commented May 19, 2014 at 20:39
  • @PatrickHofman You're right. There might not be much hope. But still seeing that not many people visit the about page we could take it as a sign to more prominently refer to it (during registration, when asking your first question, ..) and remind people on unmarked questions, .. Commented May 19, 2014 at 20:48
  • @Trilarion: good point. If you want to, create a feature request for it. I will support it. Commented May 19, 2014 at 20:50
  • 1
    @PatrickHofman Okay I will but probably only in the next days. I want to do it right and make some mockups how it could lock. Commented May 19, 2014 at 20:52
  • @PatrickHofman Oh I will just go through the process of a newbie registering and creating his first question and I will ask myself how to best refer to the about page and in which form. Cannot say now where and how this will be. I will just propose a bunch of things I think are possible and then you guys pick what you like. Commented May 19, 2014 at 20:58

3 Answers 3

43

There is significant correlation.

Questions asked by users with at most 100 rep have accepted answer in 40% of cases.

In case of users who have at least 100 rep this rate raises to 70%:

Following plot shows in detail how accept rate raises for low rep users. They are grouped in buckets of 5 rep points to avoid periodic noise (if your rep is 1 + 5k it's likely that you never accepted an answer). Accept rate by rep graph

6
  • Something looks wrong with the linked queries; the URLs encode min/max values but only the max value is used. eg the second returns results starting at 0 just like the first one does. Commented May 16, 2014 at 17:45
  • @DanNeely, it seems I accidentally overrode the old query when adding graph. It should work now.
    – zch
    Commented May 16, 2014 at 18:17
  • 5
    I think this approach conflates the question we're trying to answer - "what percentage of correctly-answered questions are not marked as answered by new users because they don't understand how the website works" with the fact that new users more frequently post bad/really bad/unanswerable questions that get closed.
    – roippi
    Commented May 16, 2014 at 19:54
  • @roippi, adding condition AnswerCount > 0 doesn't change much (other than moving graph up a few percent points). I think the result is quite robust.
    – zch
    Commented May 16, 2014 at 21:27
  • Even when limiting to non-closed questions with upvoted answers low-rep users are still less likely to accept. data.stackexchange.com/stackoverflow/query/194725/…
    – zch
    Commented May 17, 2014 at 0:50
  • What about users who have only posted one question vs. users who have posted at least two? Commented May 17, 2014 at 5:46
6

Before we build a Data query, let's find out whether we're asking the right question:

  • Are this really low-reputation users who do not accept answers? What does low reputation mean? How low?
  • Might it depend less on the reputation but on the number of questions a user has already asked? Maybe it was a good question, got many upvotes, but it is still the only question of that user.
  • Is StackOverflow simply responding too fast so that a newbie gets an answer, tries to accept but the 10 minutes period is not over yet?
  • Does it also depend on the fact that a newbie has never given an answer and not experienced that not becoming the accepted answer makes it hard to get required privileges?
  • Does it also depend on whether or not they have read the tour and earned the Informed badge?

With all the questions open, this is a first graph which might help the discussion: Find unanswered questions by noob and reputation

Screenshot of the query result

We can see the spikes at 1, 6 and 11 which are due to the 5 point reputation change per upvote. And we can see the number decreasing, which was expected.

Since I am new to OData, I make this a community wiki answer, so anyone can contribute here.

4
  • 3
    “And we can see the number decreasing” Isn't that just because the number of users decreases with increasing reputation?
    – svick
    Commented May 16, 2014 at 18:31
  • @svick, or they're more engaged with the site, know how it works and accept an answer.
    – dilbert
    Commented May 17, 2014 at 1:24
  • 1
    We could also argue to remove questions younger than 2 days, because the OP is still waiting for other answers. Commented May 19, 2014 at 6:01
  • @svick: Agreed, but what is a good proposal to have a better graph? Use percentage instead? I don't think that will help much. I think using reputation as one axis is not very good. I'd like to see a graph with number of questions on x-axis. Commented May 19, 2014 at 6:05
6

Generally, I see the advantage of "forcing function that requires they accept an answer on their first question before being allowed to ask a second." --msw.

I feels odd making a suggestion on this topic, having never posted a question myself...

Another idea...

For users who have earned a reputation under a specific threshold (say, 15; referencing Thomas W.' s graph). Find a way that might compel them to return to their question, after a time, and take some sort of action (loosely defined below).

If the hypothetical action is taken, the user would be free to ask another question immediately.

Otherwise, if the user attempts to post a new question, the system might inform the user of all previous previous questions that have been posted by the user, where the hypothetical action has not been taken; and that until taking the hypothetical action, the user is dis-allowed from posting new questions.

The 'hypothetical action(s)' include:

  • Accepting a suitable answer (of course).
  • Pressing a "No suitable answer yet" [NSAY] button.

When the [NSAY] button is pushed; it would show a timer that counts down until the button needs to be pushed again. Perhaps the timer's expiration period would get progressively longer each time it is pressed. Again referencing the graphs posted by Thomas W. and zch, perhaps the timer would first expire after 5-10 minutes (while the user is hopefully still present active at SO). When pressed next, the timer might expire after 8 hours, then perhaps 24 hours.

After pressing the [NSAY] button, it would become non-operational (grayed-out) until it expires. (Of course, it would not affect the users ability to accept an answer any time).

This [NSAY] button would be non-operational (grayed-out) permanently when the user has accepted an answer.


What behavior is best for low-rep question users? If it is that they should return periodically and review the answers; then perhaps something like a [NSAY] button would train them to do that. As their reputation improves, this "conditioning" would be eliminated. (Although it does taste like The Button on the TV show 'Lost').

4
  • I don't think it makes sense to require the user to keep pushing the button repeatedly if there has been no new activity (new answers, comments, edits) on the post. Also, I wonder how many of the users who haven't accepted an answer yet have posted more than one question? Commented May 17, 2014 at 5:45
  • Both good points @HarryJohnston. Perhaps the user should perceptive no correlation between the button and answering questions (if the purpose is to elicite a specific behavior (of checking back on their questions to see if a a suitable answer exists. Perhaps the timer wouldn't really start (or expire) until answers have been posted, or edited. As for your last question, I have wondered the same thing. In reality, users could side step all this effort by creating a new account for each new question. Commented May 17, 2014 at 5:55
  • 10 minutes is an absurdly short time-out for low volume tags. The time-outs should start at 24hrs.
    – tacaswell
    Commented May 18, 2014 at 2:06
  • @tcaswell, sure; 24hrs works for me. However, considering a 'newbie' session; the user may ask the question, wait a few minutes for an answer, and then go off and explore SO a little more. At this point, would we want the user to go back to their question and view the comments... and perhaps examine answers? I'm just looking for a way to instill the desired behavior. (And I worry that after 24 hours, the user may have lost interest in their earlier question.) For me, it is important that the user return to their question prior to leaving the site. Commented May 18, 2014 at 2:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .