35

I was looking at the result of 2020 developer survey. It says

For five years running, Rust has taken the top spot as the most loved programming language.

My understanding of this sentence was: Among all survey takers, Rust is the most loved programming language. This seemed impossible. So I dug further ...

In the chart we see Rust is at 86.1%. At the top of the chart it says

% of developers who are developing with the language or technology and have expressed interest in continuing to develop with it.

This is even more confusing, because when we look at the survey result on the most commonly used programming languages we see that Rust is at 5.1%.
Take another look at the above quoted sentence. "% of developers who are developing with" Rust is 5.1%. Then how is it possible that "% of developers who are developing with" Rust and "have expressed interest in continuing to develop with it" becomes 86.1%?!
The only explanation is that the percentage must be interpreted as: Among those who are developing with Rust (which is 5.1% of survey takers) 86.1% of them have expressed interest in continuing to develop with it.

First of all, I don't think this interpretation is the first thing that comes to mind, when someone sees the following sentence

Rust has been the "most loved programming language" in the Stack Overflow Developer Survey every year since 2016. Wikipedia

Second, this is a weird way of measuring "loveliness" of a language (i.e. not considering the user base). For example if an obsolete programming language has 10 users, and 9 of them "expressed interest in continuing to develop with it", then this programming language should be regarded as the "most loved programming language" (90% loved)? I don't think so.

5
  • 11
    I may be too biased to answer, but this has been brought up in the comment thread of the survey feedback question, to which a suggestion was made to rename the concept to "most loved programming language among its community". I also can't find it right now, but I recall an official source saying that very low traffic languages are filtered out from the graph.
    – E_net4
    Commented Jun 12, 2020 at 11:19
  • 3
    I so misread that. In my quick look of the results, I just assumed that Rust was the "new" thing, and everyone was going there now.
    – Scratte
    Commented Jun 12, 2020 at 11:24
  • @Scratte In a way, people are moving to Rust in some cases (you can find stories from several companies with a bit of searching). However, the survey question does not intend to reflect that, but the overall user-base wishes to continue using it.
    – E_net4
    Commented Jun 12, 2020 at 11:31
  • 1
    It could be that Rust is so special in the way that almost all people who dislike it, really don't use it and not in the way that people generally would like it (if they would know it). Might be or might not be the case.If Rust became more often used, satisfaction with it might drop (and that might not even be the fault of the language). Commented Jun 12, 2020 at 11:42
  • 3
    I agree, it needs to read "Rust is the programming language with the highest satisfaction rating among its users".
    – TylerH
    Commented May 14, 2021 at 19:23

3 Answers 3

13

I believe there are two ways to avoid confusion and misleading:

  • Either add "in their community" to the section title and explanations (as suggested in the comments).
  • Or change the calculation to include all developers' community, which results in the following ranking (last column):

enter image description here

3
  • 3
    "in their community" conveys the false idea that developers using a language are part of a community. They're just users. They don't have to share anything besides that. Maybe "among practicers" or "among users" ? Commented Jun 13, 2020 at 10:56
  • 3
    "Among Users" sounds good to me.
    – LoMaPh
    Commented Jun 13, 2020 at 19:50
  • Adding absolute usage numbers as a confounder to the (more fuzzy) "like"-metric seems like an inherently bad idea. Moreover, which question do we want to answer? I think, "which languages are probably fun checking out" or something of the kind, for which "loved in its own community" is exactly the correct metric.
    – Raphael
    Commented Mar 16, 2021 at 9:19
8

In order to not be misleading, this number must always come with a clear explanation of what it is.

As far as I remember, it has always been the case on Stack Overflow pages. There never was any doubt for me.

The chart is titled

% of developers who are developing with the language or technology and have expressed interest in continuing to develop with it

The Stack Overflow blog even insists on the low number of Rust programmers this year.

Personally, this is the number I'm interested into: based on informed opinions about a language or framework, not a survey of the opinions of people who don't know what they're talking about.

2
  • Related: I have more confidence into this number than into the raw language users numbers as only people who answer internet surveys are counted. I know some developers who never heard of Stack Overflow. They're also the ones who don't know there are languages more recent than C++. And they're not rare... Commented Jun 12, 2020 at 11:26
  • 4
    "% of developers who are developing with the language or technology and have expressed interest in continuing to develop with it" does not accurately describe the meaning of the number. It should be "% of developers who are developing with the language or technology that have expressed interest in continuing to develop with it". The word "and" does nothing to indicate that "who are developing with the language" and "have expressed interest" are qualifying different numbers.
    – khelwood
    Commented Aug 9, 2020 at 6:39
5

The description

% of developers who are developing with the language or technology and have expressed interest in continuing to develop with it.

reads as an expression of the form % of {population} who {predicate}, meaning "this many per cent of the given population satisfy the given predicate".

It cannot reasonably be interpreted to mean what it is apparently intended to mean.

The intended interpretation seems to be

% of {developers who are developing with the language or technology} who {have expressed interest in continuing to develop with it.}

But instead of the indicated word "who" it uses the word "and".

% of {developers who are developing with the language or technology} and {have expressed interest in continuing to develop with it.}

This no longer makes sense, and the only interpretation of that that does make sense is

% of {developers} who {are developing with the language or technology and have expressed interest in continuing to develop with it.}

So as a description for the number given, this is not only confusing; it is actually untrue.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .