133

In the Discussion on Sara Chipps' Meta answer, it was said that Meta consists of only .015% of Stack Overflow's active users.

[...] the fact that there are Millions of users on Stack Overflow whose needs aren't being met because in the past we've spent so much time on Meta which has .015% of Stack Overflow's active users and is not representative of the community as a whole. [...]

How was this number calculated? How is a Meta user defined? Reading something on the site? Having an account? Voting? Writing comments/answers/questions? Visited once in their lifetime? Visited in the last 6 to 8 weeks?

Bonus question: What fraction of free labour on Stack Overflow (writing answers/reviewing/helping others with edits&comments etc.) is carried out by these .015%?

18
  • 35
    A guess would be it's simply the unique meta visitors per month divided by the unique SO visitors per month. Regardless of how it is calculated, the "millions of users whose needs are not being met" part is very clearly hyperbole, unless SE actually interviewed a few million people.
    – l4mpi
    Commented Jul 25, 2019 at 13:29
  • 38
    @l4mpi I would go even further and say that millions daily visits shown in site stats more probably indicate users whose needs are being met
    – gnat
    Commented Jul 25, 2019 at 13:43
  • 6
    Based on the "potluck" line in the linked comment, I assume the bar is pretty high (e.g. I really doubt users at my level -- reading every day but rarely having the blend of strong opinion and eloquence/confidence necessary to write a post -- are counted among those users).
    – manveti
    Commented Jul 25, 2019 at 16:51
  • 12
    @CraigMeier That's why it is so important to know the exact definition. How else would I know if I'm included in the invitation to the potluck or not? It would be embarrassing to show up on her doorsteps with some delicious food in hands to find out that active meta user needs at least 100 answers (or whatever) Commented Jul 25, 2019 at 17:05
  • 7
    Not "the answer", but from SEDE you can get that 0.17% of all users have undeleted posts on meta (~40% have undeleted posts on main). Commented Jul 25, 2019 at 18:51
  • 1
    @thesecretmaster Thanks for this information, interessting! Do you happen to have the SEDE query still lying around? If yes, could I get a link, please? Commented Jul 25, 2019 at 19:17
  • # of users: select count(*) from users, number of meta users w/ undeleted posts select count(distinct OwnerUserId) from posts. Commented Jul 25, 2019 at 20:00
  • 8
    @samcarter I think that the bonus question is the most relevant of all
    – Lamak
    Commented Jul 26, 2019 at 22:27
  • 8
    Hello. I am a lurker who is concerned about EL&U's future, in order to understand what is happening behind the scenes and why its platform theme was changed (aka ruined) I try to catch up on the latest developments on SE. Thank you for posting, I was going to ask this very question on Meta or here because I was so taken aback by the figure. It was so specific it had to be authentic until I realised that number probably (I'm now going to check if my assumption is correct) took into consideration every single account ever created and not deleted on SO.
    – Mari-Lou A
    Commented Jul 27, 2019 at 6:22
  • 7
    Looking at the chat script, one must define what is an "active user". Is it anyone whose account has not been deleted? Is it someone who has an account, rarely if ever posts but visits and votes on meta? Someone who has an account, has earned 100 rep and posts only comments? Only upvotes? Only edits?
    – Mari-Lou A
    Commented Jul 27, 2019 at 6:30
  • 34
    I kinda think that the number was made up... just like how 70% of the statistics are made up on the spot..
    – Bhargav Rao Mod
    Commented Jul 29, 2019 at 1:23
  • 6
    @BhargavRao I don't believe your 70%, this sounds made up :) Commented Jul 29, 2019 at 8:01
  • 5
    As the person to whom that comment was addressed, I interpreted it as a made-up number, made hyperbolic perhaps to underscore a feeling where no immediate data was available. It was my understanding that Sara's point was that there are, or at least is a perception that there are, a large majority of SO users who do not participate in Meta (by whatever metric) and thus aren't being well-represented in Meta. With that being said, I would interested in some hard numbers on the active Meta users and what percentage of active SO users they make up.
    – TylerH
    Commented Jul 29, 2019 at 19:04
  • 8
    @BhargavRao Not that it really matters, "Studies have shown that accurate numbers aren't any more useful than the ones we make up."
    – Davy M
    Commented Jul 29, 2019 at 19:09
  • 2
    related: What data about meta has eluded Stack Exchange until recently? at MSE
    – gnat
    Commented Feb 18, 2020 at 14:26

1 Answer 1

123

This made me curious so I started messing with SEDE. To keep things simple, I'm going to look at the last year. Of course, since I'm just using SEDE, this will not take into account the hordes of Google lurkers.

NOTE: I am not a data scientist. These statements have not been evaluated by FiveThirtyEight. These queries are not intended to treat, cure, or prevent any disease. Consult your data scientist before beginning any protests.


Number of users with activity on the main site in the last year: 4,221,312
Number of users who have posted non-deleted questions or answers on Meta in the last year: 3,256
Number of users who have posted non-deleted comments on Meta in the last year: 4,959

Percentage, based on Meta commenters, within last year: 0.117%

Percentage, based on Meta posters, within last year: 0.0771%

If you do this without limiting to activity in the last year, you will get 10,779,915 for the total number of non-deleted users (ever) and 19,001 for the number of non-deleted users with non-deleted Meta posts (ever), giving...

Percentage, based on Meta posters, ever: 0.176%

Those numbers are higher than 0.015%, and aren't even accounting for whether the users are "active". So unless the 0.015% figure is really supposed to include people who don't have accounts, it's probably wrong. (Remember I'm not a data scientist.)

These numbers are a lot closer than they were in the previous version of this post. But that only makes these other statistics more impressive...


You mentioned something interesting in the comments that is the real inspiration for this post.

Bonus question: What fraction of free labour on Stack Overflow (writing answers/reviewing/helping others with edits&comments etc.) is carried out by these .015%?

Again, I'm going to simplify things by looking at the last year. Here is a list of the users who have commented on Meta in the last year, henceforth to be referred to as "Meta Users". Keep in mind that these Meta Users are about 1% of the users who have any activity during this last year.

Some of these statistics (especially Post History) might be a little skewed because of the inclusion of moderators and moderators aren't real users skew the statistics a little with their superhuman work on the site. Of course, if there are any moderators who haven't commented on Meta (and I think there are some), their stats wouldn't be counted among the Meta Users. But I'm pretty sure the most active moderators are included in the Meta Users.

a) As ratio of votes within last year (4.86%)

Number of votes in the last year: 1,408,932 (ignoring votes by the community user)
Number of votes by Meta Users in the last year: 68,409 (ignores downvotes on now-deleted posts, I think)
Percentage: 4.86%

b) As ratio of suggested edit votes within last year (23.0%)

Number of suggested edit votes in the last year: 906,061
Number of suggested edit votes by Meta Users in the last year: 208,118
Percentage: 23.0%

c) As ratio of (non-deleted) answers within last year (12.5%)

Number of non-deleted answers posted in the last year: 2,440,700
Number of non-deleted answers posted by Meta Users in the last year: 305,767
Percentage: 12.5%

d) As ratio of post history changes within last year (10.8%)

Number of post history changes in the last year: 12,675,953
Number of post history changes in the last year by Meta Users: 1,369,123
Percentage: 10.8%

e) As ratio of (non-deleted) comments within last year (16.3%)

Number of non-deleted comments in the last year: 7,641,649
Number of non-deleted comments by Meta Users in the last year: 1,245,870
Percentage: 16.3%


f), g) If someone knows a way to get a percentage of reviews or flags, feel free to comment or edit that in.

26
  • 4
    For percentage of reviews or flags you would need a CM to query the internal SEDE. The reviews tables in the public instance don't have a userid field. Flags is 100% private info.
    – rene
    Commented Jul 27, 2019 at 6:39
  • 2
    @rene Yeah, that's what I was beginning to suspect. I was hoping that there was a way to get generic flag counts in SEDE, since the helpful flags count is public. Same with reviews. I was hoping that there was a way to relate reviews to users, but I was having trouble understanding the structure of the database. If they specifically hid the userid column from the reviews table, that puts a damper on things. Not sure why they would do that for reviews. Commented Jul 27, 2019 at 6:50
  • 6
    You could support this FR ....
    – rene
    Commented Jul 27, 2019 at 6:53
  • 4
    There’s a very long tail of visitors to the main site that don’t interact with the site; so you’d need site visitor data to make an accurate assessment of who visits which properties.
    – George Stocker Mod
    Commented Jul 27, 2019 at 16:12
  • 8
    "There’s a very long tail of visitors to the main site that don’t interact with the site" One could maybe use these queries here to estimate the long trail. To come from 1.5% to 0.15% one would need roughly 10 times read-only visitors. The roughly 4000 active meta users generate 10% of the answers that 4 million people read. Maybe, difficult to say really. Commented Jul 27, 2019 at 17:19
  • 26
    @GeorgeStocker Yeah, I mentioned not being able to take into account Google lurkers. On the other hand, Sara specifically said "Stack Overflow's active users" [emphasis mine], so if they don't interact with the site I'm not sure how those numbers would be relevant... Commented Jul 27, 2019 at 19:56
  • 2
    Maybe she was referring to people who visit the site regularly, whether or not they have an account? I guess you could call that "active" usage, just not the first or second thing I thought of when I heard that term. Commented Jul 27, 2019 at 20:07
  • 7
    I just ran the numbers again - you appear to have the wrong numbers. The first query returns 4221312 users (over 10 times as manu users). 3292 active meta users gives 0.00078, or 0.078%. If you factor in eligibility (rep >= 5), you get 0.0022 or 0.22% (based off askers and answerers). What none of us can do though is factor in votes - users can vote without leaving a single answer or comment. And how big the overlap is also affects the final result. I don't think we can use SEDE to get an accurate result Commented Jul 28, 2019 at 20:13
  • 8
    That being said, if you use all the users and go off 3292, you get ~0.03% (which, divided by two, is 0.015). If you cut the amount of meta users in half, you also get 0.015%. If you want my guess, I'd say the number was calculated with all the users, not just the active ones, with a slightly different metric for what a meta user is defined as. Commented Jul 28, 2019 at 20:35
  • 24
    @GeorgeStocker: but the term 'active users' strongly implies it doesn't mean 'inbound Google searches who aren't logged-in/registered and don't have any site any activity'. Say I'm not logged into SO much of the time, but if I come here say 10 times a month via 10 different Google search referrals, do I show up as one user or 11? Are they/can they be uniquified or not? at least by IP address. It would seem disingenuous to cite those as 'active users' and put them in the denominator
    – smci
    Commented Jul 28, 2019 at 23:19
  • 7
    @RobertHarvey Mmm I do like the numbers better when we exclude those with less than 11 reputation, but I'm not too convinced that we can exclude such users when attempting to answer "How was the number of .015% of Meta users calculated?". There's been such a keen focus on attracting new users, regardless of the quality of their questions (and thus the upvotes they receive), that I can't expect that the official 0.015% statistic is excluding them.
    – Davy M
    Commented Jul 30, 2019 at 18:43
  • 6
    Unless a user has participated even a little (even American voters have to at least register to vote and then show up at the voting booth on election day), I don't think we can count them as being representative of the overall Stack Overflow community. We require 50 rep from users before we even grant them the privilege of commenting anywhere.
    – Robert Harvey Mod
    Commented Jul 30, 2019 at 18:49
  • 6
    @Mari-LouA Well, Sara is Tim's boss, and she's doubled down on that statistic. I guess it is premature to say it's official, but if it's not, it was very very premature to use it.
    – Davy M
    Commented Jul 31, 2019 at 21:55
  • 3
    @smci I recall several posts by SE employees where they refer Google unregistered as either unregistered or anonymous users. Also, the term "active" comes from the "last seen" and badges descriptions. It's historical meaning usually is that it has account and has been "active"/done something on the site.
    – Braiam
    Commented Aug 1, 2019 at 17:49
  • 4
    How do these numbers change if you consider voting as activity on meta. Many users simply vote their point of view here, without feeling the need to participate directly.
    – Travis J
    Commented Jan 29, 2020 at 23:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .