36

(This post is about the ongoing experiment regarding the display of a post's total score, discussed in New popup message when voting on a question/answer?, and not a duplicate of that thread)

May I kindly ask if you considered the ethics of deliberately lying to (some of) our users, when deciding to deploy the subject experiment?

I am certainly not the only one who sees the experiment as a kind of lying; quoting from the most upvoted answer to the thread above:

Questions like this:

[Screenshot of a delete-worthy question]

Really should be deleted ASAP [...] and basically lying to your readers about the question's actual vote count isn't helping.

and from some also highly upvoted relevant comments:

It feels like SO is lying to me

and

I absolutely cannot see the point of lying to your users about their downvotes. Or to other users either.

(emphasis mine in all above quotes)

Moreover, as explained by a moderator in the thread above:

You will only see this if you are logged into the site.

So, it would seem that we don't lie to "strangers", only to our own, dedicated, committed, and hard-working contributors, who are here trying to help?

I am perfectly aware that, as explained in the same answer:

You can see the real up/down vote counts at any time by clicking the score while viewing a post - even if you haven't earned the Established User privilege.

but this, I am afraid, can also be seen as "let's try to make the life of our dedicated contributors a little harder than already is", raising in itself also ethical considerations.


So, to wrap up:

  • If you did indeed consider such ethical implications before deciding to deploy, please do share your conclusions with us here explicitly (arguably, the chat rooms is not the appropriate forum for such discussions)
  • If not, I kindly suggest you terminate the experiment here and now on ethical grounds

From my side, I am temporarily suspending all voting & moderating activities until the experiment ends - or, alternatively, until I get a convincing argument as to why the experiment is not unethical.

12
  • 20
    They're running the experiment to get feedback like this. If the overall feedback is negative and it doesn't make much of an impact, they're not going to apply it permanently. Commented Oct 12, 2019 at 9:43
  • 9
    @Zoethetransgirl this is clear, but irrelevant to the issue - lying (even temporarily, only to a part of your users) is still lying, and experiments do have ethical considerations (I don't doubt the good intentions, I do doubt the implementation)
    – desertnaut
    Commented Oct 12, 2019 at 9:46
  • 2
    @Zoethetransgirl and unfortunately, does this rationale imply that if they found it does have an impact, they will keep it?? :(
    – desertnaut
    Commented Oct 12, 2019 at 10:08
  • 1
    I should probably have specified "make much of a positive impact". If most of the results are negative, they're not gonna keep it. Commented Oct 12, 2019 at 10:12
  • 4
    @Zoethetransgirl already understood that, so I am asking: if most of the results are positive, will they really keep it? Keep "lying"?
    – desertnaut
    Commented Oct 12, 2019 at 10:14
  • 3
    @desertnaut yes, but feedback like this makes it more likely for it to not be kept. If they for some reason decide to keep it, let's rather cross that bridge when we get there Commented Oct 12, 2019 at 10:20
  • 1
    @Zoethetransgirl yeah, but then it may be too late, and no bridge actually left to cross...
    – desertnaut
    Commented Oct 12, 2019 at 10:22
  • 7
    Altering numbers shown to users is nothing new on stackoverflow. Just take the reduced number of posts that is shown to be in the review queues. Commented Oct 12, 2019 at 12:46
  • 4
    @samcarter: That's not the same, though. If a question is scoring 0, -1 or lower makes a huge difference.
    – Cerbrus
    Commented Oct 13, 2019 at 6:02
  • 1
    @Cerbrus I think it is the same. In both cases stackoverflow just changes what the users sees to hide the underlying problems instead of solving them. Commented Oct 13, 2019 at 10:34
  • 5
    This post is an accusation that leaves no room for discussion. Commented Oct 13, 2019 at 17:01
  • 4
    It's not lying if you don't tell the whole truth. Although technically the experiment does indeed lie, it should display "<=0" instead of "0". Don't know why it hasn't been implemented like this. Commented Oct 13, 2019 at 21:02

4 Answers 4

25

A few notes here:

  1. We originally tried to make this work for everyone, however excluding anonymous users allows us to tie groups to accounts and thus avoid the frustration of the behavior changing throughout the day. (lots of folks complained about this early on)

  2. Lying or intentionally misleading people isn't something we're comfortable doing. That's why we made sure to enable the ability to view vote totals for all participants who see potentially-misleading scores, along with pop-up messages informing them of the experiment when voting.

  3. There's a good argument to be made that even passive misinformation can be problematic here - that's one reason why we've rejected such experimentation in the past. To that end, we're being very conservative about what effects we simulate here. This isn't a broad behavioral study or an attempt to manipulate folks' interactions, we're trying to better understand the effects of a small, rare, range of scores and so only those scores are touched (or examined).

I wanna say, #3 is the one that gives me pause: we're being as careful as possible, but I can't deny there will be situations where someone is going to feel that they've been mislead (in fact, several have already been reported). Based on my estimates prior to the start of this test, these should be rare - but I'm not happy about it.

At the same time, I've been fielding complaints from folks regarding downvoted posts for many, many years now. Authors, readers, even voters have claimed that our scoring system is fundamentally broken because of its transparency - that by making the score immediately apparent, it invariably influences how people vote, how they edit, how or whether they answer... In ways that are susceptible to unconstrained feedback loops or even outright manipulation.

Our response to those complaints has generally been some variation on, "nuh-uh!" And I'm concerned this has started to wear a bit thin. It's time we listened and, with as little disruption as possible, tried to collect some actual data on all of this.

9
  • I may like or may dislike hiding information, but in either case I acknowledge your right to manage the site as you see fit. That though does not apply to misleading information even if it is so light (the real info is actually available) and with honest good intentions as in this case, -1 for that. Please rethink your experiment. Also, congratulations on gathering data before acting, +10 for that. The total vote is still only +1 though. Commented Oct 14, 2019 at 22:04
  • 14
    I am accepting (and upvoting) what seems to me an honest, sincere, and reasonable answer to my exact question ("did you consider...?"); nevertheless, I still feel you have seriously miscalculated, hence I will keep absenting from any voting activities until this experiment is over. Thanks for taking the time to answer, I just wish you had done so earlier...
    – desertnaut
    Commented Oct 14, 2019 at 22:14
  • 3
    Btw. did you consider inserting relational operators in front of the score in order to fix the lie and be on a more ethical side? This answer unfortunately doesn't really say anything about it. You are not comfortable with lying and there seems to be an easy way out. I hope the change can be considered. Commented Oct 15, 2019 at 11:14
  • FWIW, while I'm skeptical of the value of the changes being tested, I don't view it as lying as others have. Seems like those people are being overly sensitive (or maybe they are being hyperbolic in their disapproval). Commented Oct 15, 2019 at 19:17
  • @PeterDuniho you mean, something like the conversation and edit wars and a post being (previously) locked because someone chose to use some Nancy (first name only), which may or may not be even a real name or person, in a Meta post? Well, maybe - opinions are always respected...
    – desertnaut
    Commented Oct 15, 2019 at 22:27
  • 5
    "lying or intentionally misleading people isn't something we're comfortable doing" - but you are still doing that unless/until the affected users vote or look at the vote breakdown. You might not consider it to be misleading to show me an answer with a highly negative score as having a score of 0 (or -1), but I certainly do. At least it seems like posts with negative score are currently shown as -1 and not 0, but especially for niche questions there's a huge difference between an answer at -1 and -5. Please opt me out of this experiment if possible.
    – l4mpi
    Commented Oct 16, 2019 at 10:33
  • 11
    By the way, if this is intended to reduce pile-on downvotes, it might backfire. I just downvoted a question at "-1" which was actuall at -7 at that time. I'm way more inclined to downvote a bad post at -1 than I am to downvote a post with a heavily negative score; after all I only have a limited amount of votes per day.
    – l4mpi
    Commented Oct 16, 2019 at 12:43
  • 7
    The insufferable mollycoddling of people needs to stop. This is a site for adults. If the horror of staring at a negative score of unicorn points is enough to destabilize someone, they have bigger problems than needing their programming problem solved and we, as a community, are completely incapable of helping them with that problem. Destroying our community for the sake of the sensibilities of people who have emotional issues that require clinical treatment is not logical - doubly so when it is clear that these destructive changes can in no way actually help those people.
    – J...
    Commented Oct 16, 2019 at 14:57
  • 4
    I would've been fine with this experiment if the notification hadn't just appeared when I downvoted a post. It reminded me too much of the "new user" notification box that just appeared one day. We get told what's going on only when someone posts a bug report about it. To me, one of the most welcoming parts of this community was how y'all used to be quite transparent about upcoming changes. Documentation, for all its warts, was announced ahead of time. People volunteered to be involved in it. This experiment, I'm sure, is not at the same level, but still, some forewarning is always nice. Commented Oct 16, 2019 at 16:59
31

It was possible to both perform the experiment and to be ethical.

Score : ≤0

Instead of

Score : 0

The former only hides information. Which in this case should pose no problem ethically. Nobody will be infuriated by a -10 in either case. People will wonder why their question/answer is not liked and remains a 0 in both cases.

3
  • Indeed it was...
    – desertnaut
    Commented Oct 14, 2019 at 21:36
  • 1
    I think this answer plagiarizes my answer. At the very least I would like to be credited. :) Just joking. Good idea. Maybe SO could still implement it. Commented Oct 14, 2019 at 22:05
  • 3
    @vicky_molokh ≤0 hides information without providing false information. 0 hides information and provides false information. I've edited the answer to add "only" in order to clarify Commented Oct 16, 2019 at 9:31
5

I'm not seeing the lie.

The question the experiment seems to be asking is,

If your question or answer is poorly received, would seeing a non-zero score affect your future participation or perception of the site?

The main point is that the question or answer wasn't received well. The implementation detail is how poorly it was received, from the perspective of the experiment.

It reminds me of a suggestion I had about four years back to actually allow for questions which were downvoted into oblivion (for a given numerical value of "oblivion") to be given a second chance by hiding their score as one of their functions.

I can respect that everyone here still has their pitchforks in a constant state of "ready", especially after what's been going on these last few weeks, but let's not be so eager to rush into this one without really thinking through what's going on...

11
  • You seem to be addressing a different question, which I have not asked (about the experiment rationale). What I ask is - even for well-intentioned experimental reasons, some of us here have been deliberately shown information that is false and inaccurate, with no hint provided about it beforehand. Performing a well-meant experiment aiming at improving the user experience does not provide card blanche for misleading your users, and it should have been the subject of an ethical consideration before deployed; so, what I ask is - did it? And if yes, what were the conclusions?
    – desertnaut
    Commented Oct 12, 2019 at 18:27
  • 7
    Let's not conflate "false", "inaccurate" or "misleading". The inaccuracy in the experiment is that the vote count is not accurate for public users. There's no indication that anything here is either false or misleading; in any circumstance, the question is not "well received" - and the definition of "well received" has long extended to net-zero posts. There really aren't a lot of people who can see post counts on their questions, and the vast majority of users that this would likely be aimed at are those sort of fly-by-night people who come in to post a question or answer once or twice.
    – Makoto
    Commented Oct 12, 2019 at 18:35
  • 1
    If it were the case that Stack Overflow were somehow insinuating that the post was well received, then yes, that would definitely be a lie. However, what's being stated is that the post isn't being well received.
    – Makoto
    Commented Oct 12, 2019 at 18:35
  • 15
    In my book, showing a zero score for a post that actually may be at -1, -2, or -3 counts as false, inaccurate, and misleading (now I cannot even know if a score is 0 due to no engagements, or if someone decided not to show me the actual votes), and honestly I cannot see it any other way; if you don't think so, we can of course just agree to disagree.
    – desertnaut
    Commented Oct 12, 2019 at 18:48
  • 4
    I do see it a different way...it's one of those things that has gnawed at me for years in participating on the site so I can see the motivation behind the experiment, and it's also apparent in Sara Chipps' first blog post here. Yes, we need to change how we do Q&A and how we convey the right signal to the users. I respect that we all disagree on how to go about that, so I'm fine with agreeing to disagree.
    – Makoto
    Commented Oct 12, 2019 at 19:06
  • 2
    Cool. Last point: trust me, I am not carrying any pitchfork (ready or otherwise), I am largely oblivious at what has been going on lately (I don't frequent Metas, and I don't even know who Sara Chipps is), and I only wrote this after having thought about it for 2-3 days, and seen how it practically affects me (and read about others, too). Again, I don't disagree that there are tons of things that can and should be improved here, I just suggest caution and serious consideration before jumping to implementation of "brilliant" ideas...
    – desertnaut
    Commented Oct 12, 2019 at 19:16
  • 1
    BTW (just thinking it along), the rationale you describe would make perfect sense if the experiment were confined to new users only; as is, what's the point of not showing the real score to seasoned & well-established users, like you or me? To see if it will affect our future participation???
    – desertnaut
    Commented Oct 13, 2019 at 12:34
  • @desertnaut An additional point they may be trying to check is if being able to see if a post is very negatively received (-2 votes or more) if it makes any difference on other users downvoting the question more. The reason I say this is because I've seen people complain about how a post may get "downvoted to oblivion" sometimes just minutes after being posted; I'd be interested in seeing if questions get less downvotes overall because less people may be swayed to downvote with the masses. Commented Oct 14, 2019 at 20:24
  • @GrumpyCrouton all these are valid points for sure, but here I'm asking something much more fundamental, and in principle irrelevant with the specific objectives of the experiment, whatever these might be and about which everyone of us just guesses, as they have not been explicitly disclosed
    – desertnaut
    Commented Oct 14, 2019 at 20:54
  • @desertnaut Well usually disclosing the objectives of an experiment involving so many people could skew the results making them meaningless. I'm sure they will be disclosed once the experiment is over. Commented Oct 15, 2019 at 13:30
  • 1
    @GrumpyCrouton you do have a point; let's see what will be disclosed afterwards, although as per Shog9'a admittance, the feedback received here @ Meta will also be taken into consideration
    – desertnaut
    Commented Oct 15, 2019 at 14:02
-17

I think the intention was not to lie but just not telling the whole truth. However the experiment does indeed lie (the math just doesn't add up and the displayed number is not always the score).

It can easily be fixed with the inclusion of a single character. Just display "≤0" instead of "0". But they didn't do it.

P.S.: There has been an official answer from Shog in the meantime and I can update this answer. I assumed SO wanted to just hide the truth and forgot to insert "≤", but even after asking if this is the case, the official answer doesn't mention anything of it. It mentions though that they were uncomfortable. But still the intention seems indeed to have been to lie. In summary: They intended to lie (but felt uncomfortable about it), they did it and they did not consider just hiding the truth with the usage of "≤". How ethical this experiment is/was is probably left to judge by the reader.

4
  • 10
    If a corporation pulls this kind of trickery on their tax report, they do get into a whole lot of trouble, though. Showing different numbers isn't a "partial truth". It's a lie.
    – Cerbrus
    Commented Oct 14, 2019 at 13:21
  • 1
    @Cerbrus I think the idea behind the experiment was not telling the real score. They implemented it by displaying 0, which is a lie. They could have easily rectified the situation by displaying <=0 instead. They didn't. Now I repeated my whole answer and don't know why I did it actually. Sorry. Commented Oct 14, 2019 at 14:02
  • 1
    Thanks for chiming in; to be honest, my problem with your answer is that it's not an answer: apart from the implementation ideas which are irrelevant to my question, you have ended up claiming "it's not a lie, but technically it is", which, let's face it, does not sound any helpful...
    – desertnaut
    Commented Oct 14, 2019 at 14:09
  • @desertnaut Thanks for the comment. I reformulated the answer a bit stating that I think that their intention was not to lie, but that they lied in the course of action. Actually, I mostly wanted to dwell on how easy it would have been to make it a non-lying experiment, even if that doesn't really answer your question. Sorry for that. Commented Oct 14, 2019 at 14:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .