-14

As you may know, Stack Overflow has a penalty in a formula that determines whether to allow a post to go into HNQ or not:

Limited (by hotness score)

These sites can still contribute the same number of questions to Hot Network Questions, but their questions need higher hotness score to be selected.

Heavily (80% penalty factor)

80% penalty factor means that arbitrary hotness points for any question on Stack Overflow is 5x less than that of a question with similar activity from a non-penalized site.

Is it worth it?

How about reducing the penalty? (I'm not saying to remove it altogether, I'm suggesting to reduce it)

Times has changed and activity on Stack Overflow is arguably not that impressive that is used to be: questions generally do not attract really many votes, which apparently may lead to a few questions from Stack Overflow to be present in the list. I mean, check this, among 100 listed questions there're no posts from Stack Overflow at this time at all.

If I did not convince you... consider an outcome of following Data Explorer query (this one includes data from Jan 2022 to today, that's around 13.5 months):

Note: the linked query may look terrible but it serves its function at least, I have no idea how to properly compose these frikin TSQL queries, vastly different from MySQL that I'm used to

Note 2: I have no idea why Numbers use spaces as thousands separators and found no setting to tweak it

Let's then check out the formula to understand what does 80% penalty do:

(MIN(AnswerCount, 10) * QScore) / 5 + AnswerScore
-------------------------------------------------
         (QAgeInHours + 1) ^ 1.4

If we make some simplifications, it would mean that Stack Overflow has a same chance to reach hot network questions with a post of 50+ score as any other site has the chance to reach the HNQ with a post of 10+ score (50 - 80% penalty = 10).

Meanwhile, how many questions with a score of 50+ has Stack Overflow got during 2022? The answer is 134 according to Data Explorer, which puts Stack Overflow on 27th place in the table I've shown above. No wonder we see the posts from Stack Overflow so rarely in HNQ!

Of course this assumption is rough, we can see that the formula does prioritize answers, the general point remains; one could build a query to compare answer scores, I still doubt 27th-place-figure changes significantly, my point was to lessen the penalty and not remove it altogether

You see, Stack Overflow generated 1800 questions with a score >= 10, which may sound like a solid first place, until you consider that:

  • it's outrageously small compared to the total number of questions posted
  • 73% of all questions on Stack Exchange network during 2022 were asked on Stack Overflow. However, out of all questions with a score >= 10 only 16% of questions were asked on Stack Overflow during the same time.
  • as I said hotness score for Stack Overflow is divided by 5, and according to formula that would put Stack Overflow onto a 27th place in a table shown above:

For 2022 I have also checked stats but for posts with score >= 20, the results are almost the same so not including them


We can also see how these numbers changed throughout the years, but I believe here an impact from search engines is starting to outweight scores accumulated due to inner traffic. Probably activity also dropped, but I doubt that by such a large margin.

Year Total questions Score >= 10 Stack Overflow share SO score >= 10 share
2022 2,242,510 13,287 1,641,895 (73%) 2,133 (16%)
2021 2,203,458 21,573 1,552,201 (70%) 6,667 (31%)
2020 2,664,455 33,992 1,869,505 (70%) 14,492 (42%)
2019 2,578,082 50,531 1,765,594 (68%) 24,271 (48%)
2018 2,739,531 70,383 1,887,749 (69%) 38,042 (54%)
2017 3,020,634 92,953 2,115,000 (70%) 51,399 (55%)
2016 3,093,226 111,236 2,199,648 (71%) 61,620 (55%)

Check out the query that generates this (year-by-year) (especially revisions)

Having said that I further emphasize on the fact that Stack Overflow may not generate such a lot of high-score content that it is expected or was expected when the penalty was introduced.


Why am I proposing the change?

I believe that such a big penalty as 80% defeats the whole purpose of Hot Network Questions. One may argue that HNQ is useless, however I think not: it is specifically designed to drive attention to worthy posts. Considering there’s a lot of questions on Stack Overflow, it’s somewhat easy for a good question to “drown” among mediocre ones.

Bottom line:

Can we probably revisit the penalty value applied to the hotness score in a way of reducing it? It looks like Stack Overflow does not generate so much high-quality content that it would justify 80% penalty today.

oh yeah not so bottom line, p.s. incoming in 3...2...1...

P.S.: this also may relate to the other couple of penalized sites, however I do not expect this post to generate any feedback elsewhere, so I'm posting it here.

P.P.S. interesting over-the-years data actually, something to think of...

P.P.P.S. while writing this post and collecting data I'm now even more interested how is that possible that score>10 stats are so different for the past years (yeah search engines and all of that but that different?)

13
  • 18
    "it's outrageously small compared to the total number of questions posted" - that's the point. There's an exponential growth the lower you go. The point is to make sure Stack Overflow doesn't completely dominate hot network posts, which would completely push out most network sites. Commented Feb 11, 2023 at 23:02
  • 1
    @Zoe I have never requested removing the penalty, I suggested reducing it, in a current state stack overflow does not dominate, it does not even reach the list. One shouldn't pick an extreme case of rather having an enormous penalty or not having it all, there's a possibility to have something in between
    – nicael
    Commented Feb 11, 2023 at 23:07
  • 5
    Can you please add the missing section about benefits of the proposal for SO/SE? Commented Feb 11, 2023 at 23:43
  • 1
    @Alexei that’s basically the same as asking why would one even need Hot network questions, let’s then probably remove HNQ altogether? The answer is to drive more attention to interesting posts
    – nicael
    Commented Feb 11, 2023 at 23:46
  • 7
    No ... the real point of HNQ is to drive the people's attention to other sites.
    – Stephen C
    Commented Feb 12, 2023 at 5:22
  • 3
    The numbers you've shown are misleading. While question score (which you exclusively focus on) is a factor, it's not the dominant factor under most conditions. The "hotness" score will tend to be dominated by sum of the scores of the answers, rather than the question score, most of the time.
    – Makyen Mod
    Commented Feb 12, 2023 at 7:08
  • 3
    If you want to see if the current calculation is "unfair" (i.e. resulting in SO not being as represented on the HNQ list as much as other sites), you need to look at the number of questions that hit the HNQ from each site over a significant period of time (e.g. a year). Once you have that information, you can compare the total number of Hot Network Questions from each site. Doing that could then be used to see if some adjustment should be made to the existing calculation. I'd note that you'd also need to know what proportional representation was intended.
    – Makyen Mod
    Commented Feb 12, 2023 at 7:12
  • 2
    "it is specifically designed to drive attention to worthy posts" no. It's driving attention to clickbait-y posts. Maybe slightly strong of a word but it drives towards "interesting looking" questions, not necessarily "good" ones. Note that "interesting looking" according to the algorithm is also the 100th dupe of "do this basic thing for me" where half a dozen users immediately post an answer within a minute or two because...well, it's the 100th dupe - the subject matter is very well known."interesting looking" might also involve bad questions.
    – VLAZ
    Commented Feb 12, 2023 at 9:15
  • From what I observed, most of the time the HNQ from SO was about a temporary bug/breaking change due to a 3rd-party updating something, receiving many answers and upvotes because of "me too". Whether they're the kind that we want to highlight or not...
    – Andrew T.
    Commented Feb 12, 2023 at 10:35
  • 1
    Also, might be helpful, though only the top 10 (out of 150+) sites were listed: Which sites appear most often in the Hot Network Questions list?
    – Andrew T.
    Commented Feb 12, 2023 at 10:39
  • Shouldn't this be on the SE meta instead? Commented Feb 12, 2023 at 11:58
  • this is probably related: Technical site integration observational experiment "at least six answers right here and multiple comments point that hot network questions are expected to support the kind of integration you're looking for..." (side note to @KarlKnechtel - no, not on SE meta, because this is only about our site and whatever decision we make on that feature request, it should be decided by us, not by some strangers from all over the network. This firmly belongs to meta SO)
    – gnat
    Commented Feb 13, 2023 at 5:45
  • @ZoestandswithUkraine your concern about domination is wrong because system has hard limit of 5 questions from any site in the list. No matter how large is Stack Overflow, there just can't be more than five SO questions in the list
    – gnat
    Commented Feb 14, 2023 at 14:17

2 Answers 2

16

On your list there's only one site that comes within an order of magnitude of the number of questions in total asked in that time frame, and there's still a 1.6 million question gap between the two sites.

By the numbers, Stack Overflow is practically 95% of the network. If the algorithm were tweaked to allow any more questions from Stack Overflow into HNQ, then it wouldn't be Hot Network Questions, it'd be "Hot Stack Overflow Questions" instead.

I don't really...see a whole lot of reason to tweak the algorithm in light of that. It would mean that a lot of smaller network sites would lose traffic and engagement, and we already know what happens when something like that happens.

8
  • A total number of posts does not even nearly indicate how many would reach HNQ, that’s why I have provided a lot of numbers in addition to the total number of posts
    – nicael
    Commented Feb 12, 2023 at 0:18
  • As far as I understand the HNQ list consists of around 100 questions. Taking 2-3 of them (instead of around 0 right now) for the Stack Overflow is not something that would severely impact engagement on the other sites
    – nicael
    Commented Feb 12, 2023 at 0:25
  • 2
    @nicael: It's really the volume of potential questions that can make its way onto the site. But maybe here's a counterpoint - what benefit would it do to highlight top questions from Stack Overflow? Everyone already knows about this site on the network and they know what they get when they come here.
    – Makoto
    Commented Feb 12, 2023 at 0:57
  • 7
    Also, an important aspect of HNQ is building awareness of and interest in the rest of the network for Stack Overflow users.
    – Cody Gray Mod
    Commented Feb 12, 2023 at 5:18
  • 1
    @nicael There are about 180 sites on the network. With 100 Hot Network Question slots available, that means at least 45% of the sites on the network won't have any HNQs at a particular time. The issue isn't "is it as easy for a particular SO question to be on the HNQ list as it is for a question from another site". The issue is "does the current formula give SO the proportional representation on the HNQ list that is intended". To know that, you need to first look at relative number of Hot Network Questions on all sites and know what the intended representation is for SO.
    – Makyen Mod
    Commented Feb 12, 2023 at 7:59
  • Once those are known, they can be compared to see if the current formulas are working as intended. If they're not, then a look at adjusting the formulas/weighting is justified. You could, also, argue that the intended level of representation of SO isn't what it should be, but that's a different discussion than "let's change the weighting/formula" and should happen before getting into the details of the formula, because the formula serves the intent, rather than defining it.
    – Makyen Mod
    Commented Feb 12, 2023 at 7:59
  • this answer is plain wrong because system has hard limit of 5 questions from any site in the list. No matter how algorithm were tweaked, there just can't be more than five SO questions in the list (we can request SE dev team to make it less though if we want to - 4, 3, 2, 1 - now that would be very easy to do)
    – gnat
    Commented Feb 14, 2023 at 14:20
  • 1
    (on a further thought, it would be hard to blame answerer and those who upvoted for this mistake. The only purpose of the parameter discussed here is to keep Stack Overflow away from hot questions and the fact that even experienced users are unaware about basics of this feature kind of proves that it serves this purpose really well. Maybe we should keep it that way)
    – gnat
    Commented Feb 14, 2023 at 15:24
4

I wasn't going to go out on a limb but since many comments including top voted one and prior answer suggest that this feature is there to prevent SO dominating hot questions, I would like to clearly debunk this reasoning because it is obsolete for about 4 years now.

System currently works by simply limiting amount of hot questions from same site to no more than 5. No matter what penalty is or isn't there, there will be no more than five hot questions from Stack Overflow or any other site at all.

If we want, we can even request that no more than 4, or 3, or 2, or 1 questions from our site were in the hot list and dev team will do - and this won't even cost them any effort because this is just a simple change of a simple configuration parameter and it doesn't even need testing because it was already tested and proven to work as advertised on at least one site (IPS.SE).

Suming up, any concerns of SO dominating hot question list are not relevant anymore - system just won't let it happen because it uses hard limit of site questions in the list (5 or less of total 100). We needn't worry about that because system takes care of that automatically.


Now let's have a closer look at that penalty parameter you're asking about. Useful thing to understand about it is that for about 9 last years Stack Overflow is the only site where it makes a meaningful impact.

You may have seen at MSE various lists of other sites also having non-default values of this parameter - just ignore these. Parameter settings at these other sites don't make any difference worth talking about. Readers interested to find boring details about that can get to a more thorough explanation here.

Technically, this was maybe not the most elegant solution but for first few years it at least served some role preventing SO dominating hot list. However after hard limit of 5 questions was established it lost relevance in this regard so that currently purpose of this parameter can be described as in referred post:

a special hack only for SO questions, intended to make them drop off from the hot list much faster than questions from other sites in order to minimize risk of complaints about this feature at SO meta and therefore save efforts of SE dev team because it is so much easier to ignore similar complaints raised at smaller site metas.

You won't find above stated officially but that doesn't make it any less real, and under-representation of SO questions in the hot list you are talking about is exactly how it works.


Given above, I think we needn't worry about other sites and the real question we can ask ourselves is, do we want to discard the parameter which only purpose is to push our site (and only our site) questions out of hot network list?

Granted, it would be easier for us to moderate hot questions compared to some smaller sites (especially given that hard limit of 5). Sheer amount of SO users having rep-based privileges to moderate content (and actively exercising these privileges) makes it practically impossible for trolling, off-topic and duplicate questions to stay widely exposed in hot list for too long.

Still, this would be some extra burden to moderate such questions compared to what we have now when system does all that work for us and simply drops off our questions from hot list. Although this "luxury" comes at the expense of pushing out worthy questions as well.

3
  • 2
    Well, one thing to consider is that while the original intention of the penalty was to reduce the number of SO questions on the list during a time when there was no explicit limit per site, one thing it does today is it increases the required activity for a question to be selected. In my view, while removing this penalty won't have any effect on SO's domination in the list, it will mean that the specific questions selected will be more mediocre. This site has a lot more traffic and voting population so using the same voting standards as smaller sites may not be appropriate.
    – gparyani
    Commented Feb 14, 2023 at 12:09
  • @gparyani per my observations of questions in the hot list, voting standards in SO seems to be somewhere "in the middle" so to speak. It looks significantly lower than dozen or so top actively voting sites (like Worldbuilding, Code Golf, Tex, Mathematica etc) and somewhat above most of the rest of the network. In hot list I'd rather expect SO to be just one-of-the-many in this regard, that is slightly above the average but really far from top actively upvoting sites. This is just a guess though because there are too few our questions in HNQ to say for sure
    – gnat
    Commented Feb 14, 2023 at 13:52
  • ...speaking of mediocre questions in HNQ (or at least egregiously mediocre ones), I'd in fact expect us handling these better than most if not all other sites in the network. Unique thing I observe here is a pattern (or culture if you wish) of consistently downvoting such questions when these are widely exposed. I'd expect something like meta effect on steroids on such questions in HNQ and I would be quite surprised (and disappointed) not seeing it
    – gnat
    Commented Feb 14, 2023 at 13:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .