-22

I am doing a statistical analysis and I need a lower bound. I need to exclude from this particular analysis a certain segment of the user population because the analysis is about contributing users and the criterion is reputation. There is a fine line that separates the contributing community from the rest. That line corresponds to a certain reputation. That's the number I am looking for.

I know about the exceptions. I know about the 30k rep users with one post that is years old. I know about extremely active users with low rep because they offer bounties. Such exceptions are considered negligible. Consider the following facts about Stack Overflow users:

If we're interested in a statistical analysis of the contributing community we need to find that separation line, especially when we're talking about a line that separates the minority from the majority.

I am looking for that magical number that represents "decent reputation" in order to use it as a cut-off point to separate the statistical noise from the relevant data.

To clarify what I mean by "lower bound" or "cut-off point", consider this extremely awesome answer. In that analysis, Kevin Montrose chose 1000 rep as the lower bound, saying:

I've chosen 1000 rep as a good "earned your stripes" cutoff, below this point you get really freaky outliers

Kevin Montrose had a very good reason for not choosing the average rep. as the lower bound. He arrived at the 1000 rep. point by trial and error.

I am interested in both subjective answers (e.g. "I think 1000 rep. is a decent reputation") and objective answers that try to find that separation line by other methods.

There must be a way to find the border between contributing and not so contributing users.

This "lower bound", "decent rep.", "lowest rep of active users" or whatever you like to call it, can be used by anyone in future statistics.

27
  • 18
    I try not to judge people by their magical internet points but rather by how they act on the site. Reputation is a terrible metric for just about anything, except maybe seeing how many votes someone has received.
    – ivarni
    Commented May 31, 2018 at 5:20
  • You assume too much. This is has absolutely nothing to do with judging people. This is about statistics. I would appreciate if you can help me reword the question in a way that avoids triggering unjustified guilt.
    – q-l-p
    Commented May 31, 2018 at 5:32
  • 1
    Any reason you are considering only reputation and not badges and/or number of flags or any other contributions for statistical data?
    – Suraj Rao
    Commented May 31, 2018 at 5:41
  • 4
    and what are polls except ways to elicit people's opinions? Commented May 31, 2018 at 6:38
  • 2
    This is meta, right? Opinions are OK here. What are you talking about?
    – q-l-p
    Commented May 31, 2018 at 6:39
  • 1
    FWIW I don't think it should be closed, but I also don't see any kind of useful metric coming out of this. We have the privileges ladder for reputation already. I also fail to see why reputation should have any impact on who to take into account for a "feature request" we don't even know what is. High-rep users aren't the only users on the site and not all high-rep users even visit the site regularly.
    – ivarni
    Commented May 31, 2018 at 6:51
  • 2
    There obviously isn't a reputation border that makes users earn their stripes. You can review this answer, and see that there are >30K users that only ever asked a single question (and a broad one imo). I wouldn't trust them for a single second with their privileges, but luckily, they aren't active. Positive contribution takes priority over rep.
    – Erik A
    Commented May 31, 2018 at 7:36
  • 2
    I can sort of follow why you ask this, but I think the premise is questionable. How much rep you have can also vary with which tags you're active in. People who work in low traffic tags have a harder time earning rep than those who move in the more "popular" circles. But it doesn't mean their opinion is less worth... (I have a reaction everytime someone says getting 10k or whatever is easy and quick!) Commented May 31, 2018 at 10:56
  • 4
    I feel like this question has the whole XY problem going on. You're trying to apply an arbitrary cutoff for some mystery stats, but you've provided no use case or purpose for this data. This dramatically limits whatever participation people can do, as they have no idea what's going on.
    – fbueckert
    Commented May 31, 2018 at 13:29
  • 3
    (1) I have attempted to rephrase the question. I'm not sure if "seasoned contributor" is actually a great way of expressing what you are looking for. Still, the hope is that more neutral-sounding wording can help stem the bizarrely negative reception of this question. (2) Though I don't think it invalidates the question, @fbueckert has a point in that the ideal cutoff is likely to depend on what you want to find. Cf. Kevin's "freaky outliers", which presumably were outliers in the context of that particular analysis.
    – duplode
    Commented May 31, 2018 at 13:50
  • 2
    After reading your original meta post, I can't help but think you're trying to make a case for bias against new users here, based on whatever reputation level you can make work. If that's not what you're going for, I highly recommend you clarify what, exactly, you're trying to do.
    – fbueckert
    Commented May 31, 2018 at 14:37
  • 1
    In other words, we don't want users at or above a certain reputation level to be affected too much if the feature is implemented. Like increasing moderation burden? I can't think of another way that higher rep users would be affected by a feature request. If you can tell us what you're trying to avoid that might help others form an opinion of where the line should be.
    – BSMP
    Commented May 31, 2018 at 14:51
  • 1
    Not only there are four questionable close votes already, but one of them is "Off-topic/Not about Stack Overflow". Is that a joke?
    – duplode
    Commented May 31, 2018 at 15:27
  • 1
    Okay, but I'm having trouble figuring out what, exactly, you are trying to accomplish here. We have a goal, but we have absolutely no context as to why you're attempting this. Give us something more to work with, instead of a strange poll.
    – fbueckert
    Commented May 31, 2018 at 21:05
  • 1
    We get that you're trying to define a lower bound. What everyone's confused about is why you're dong this. What statistical analysis are you trying to do that you need help with defining a lower bound? Give us some context to work with.
    – fbueckert
    Commented Jun 1, 2018 at 13:25

3 Answers 3

9

I could come up with my own number but that would be too subjective.

Polling the community is pretty subjective too. You're just trading your own subjective opinion for the collected opinion of a group. You probably will get somewhat better results, but it's still inherently subjective.

It would probably be better to run your analysis with a few different cutoffs and see where you get the fewest outliers while also getting meaningful results. (That's incredibly vague, I know, but it's hard to be more specific without knowing what kind of analysis you're doing.)

1
  • You are, of course, right about the subjective part. While the comments and votes started coming, I realized that Kevin Montrose did just what you propose. But then the community reopened the question and I tried to cheat. I said to myself, What if we... let the community define "earned your stripes reputation"? So I added 2 answers to the question, as poll options. Because this was intended as a poll from the beginning. I thought a poll was OK if it served "the greater good". Or maybe I'm wrong. I don't trust untested assumptions. I prefer to be proven wrong. I had to do the test.
    – q-l-p
    Commented May 31, 2018 at 16:56
0

Kevin Montrose alluded to avoiding "freaky outliers" in his analysis as a reason for the 1000 rep cutoff. I would suggest another motivation: in the absence of bounty giving, 1001 rep is the minimum that a bronze badge holder can have. Tag badges are a decent, if partial, indicator of sustained contributions.

Here is a SEDE query for the reputation of tag badge holders. For an arbitrary example, David Fletcher, the bronze badge holder with second lowest reputation currently has 1190 rep (the recordist, enough rep to comment, has spent most of their rep on bounties and is currently at 396 rep).

If you feel a stricter cutoff would be more appropriate for your purposes, you can instead use the analogous figures for silver badges (4000 rep in principle, with 5000 being a more typical value) or gold badges (10000 or 12000).

Yet another option would be eschewing the rep cutoff altogether, and simply used ownership of tag badges of some class as criterion, as long as that makes sense for the specific investigation you want to carry out. (For Kevin that wouldn't have made much sense, as he was primarily concerned with reputation scores in the first place.)

0
-9

In order to find the line that separates the contributing community from the rest we first need to define the term "active user". It is expected that the separation line will shift depending on this definition.

"Active users" could be defined as those users who've been inactive for less than a certain amount of time T. We set the desired percentage P of active users from the total number of users of a certain reputation. To be democratic, I would set this to 1/2.

We already know the separation line is somewhere between 125 and 1000 rep. We scan this area and calculate for each rep. the ratio R = active users(rep) / total users(rep). If R > P then enough users of that rep. are active and we have to go deeper to a lower rep. Rinse and repeat until we reach the reputation for which R > P is false. This is the line.

We could fiddle with T and P see how the separation line shifts. For T = 1 and P = 1 you get the strictest possible definition of "decent rep." It will give you the lowest rep of daily users. The more you decrease T or P the more lenient you are with your definition of "decent rep."

1
  • While looking at recent activity can avoid artifacts related to the evolution of the site or to passive rep accumulation if these are a problem for your analysis, you'd want to make sure you don't end up eliminating more users than you'd like. Seasoned contributors sometimes take long breaks, or stop posting altogether. For instance, the [c#] tag currently has 590 gold badge holders. 136 of them haven't posted in the last 365 days.
    – duplode
    Commented Jun 1, 2018 at 13:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .