86

Context

On 2024-08-22, the company decided to fully remove tag wiki excerpts from the top-level UI by removing tag wiki excerpts from the tag pages themselves, which drastically undermines our ability to curate tags. The company has been told how important the excerpt is by several people (1 2 3 4 5) and explicitly acknowledging their importance, and opted to go ahead and remove the excerpt from the last top-level place it was shown anyway.

This decision is actively harmful for all forms of tag curation, as it currently hides the often critical excerpt information behind several clicks with no real-world benefits for any end-users - in any target group, curator or otherwise. Misrepresenting what tags are for by hiding usage information only results in increased rates of incorrect tagging and, for certain tags, increased rates of off-topic questions.

From experiments, a few tags seem to be exempt from this and still have a functional excerpt, but the existence of an excerpt whitelist is unconfirmed and purely observational. It's unclear why these tags haven't been affected, so they may still break.

For some more concrete examples of why this is bad, see this meta post. The change hides critical information the community has built up over the past 15 years or so, and in many case actively misrepresents what the tags are for. The excerpts were already hidden and little used, but this change further hides the excerpts.

We can't force SE to change the UI back to using the tag wiki excerpt - but due to how the new excerpt is picked, we have a way to work around this change. That is the purpose of this here proposal - offering a way to restore existing functionality, even if SE refuses to restore it.

Conditions for executing the mass-editing

The company has been given a deadline of 2024-08-30 at 12:00 UTC to respond to this post, and for said response to be the immediate restoration of the tag wiki excerpts in the top-level UI, or a promise that they'll be brought back in the extremely near future. The goal with the deadline is to force them to respond as soon as possible, as this change is doing ongoing damage to tag curation. Waiting 6-8 weeks for them to maybe perhaps respond is not a viable decision.

While this can go in either direction, the proposal outlined in this post takes effect if Stack Exchange, Inc. chooses not to restore excerpts in the top-level UI.

If the excerpt is returned, or a plan is made to return it in the very near future, the mass-edits are automatically off the table. Due to restrictions explained in the next section, this new form of excerpts is not optimal. Optimally, we'd get our normal excerpts returned. Mass-editing serves as plan B to forcibly restore the usefulness of the 46177 tag wiki excerpts - if this plan is executed, it's the last resort.

Editing strategy and scope

The new "excerpt" system uses the first line, excluding HTML comments and headers, up to EOL, a period, or 215 characters are used; whichever comes first. The proposed mass-edit will therefore copy the excerpt into a special comment-delimited block (for visibility) in the main wiki, with a few changes to punctuation (read: removing periods and replacing them with another punctuation character, or replacing them with a unicode equivalent that doesn't trigger end-of-line detection) to fit as much of it in as possible.

The edit will insert the following section at the top of every tag wiki:

<!-- Begin new-style excerpt. See https://meta.stackoverflow.com/q/431374 for more information about restrictions of this section. If you want to edit the excerpt, edit here too, not just in the excerpt box above! -->
[Text currently in the actual excerpt goes here, with some automatic punctuation changes due to the aforementioned restrictions]
<!-- end excerpt --> 

The comment delimiters serve two purposes:

  1. Making editors aware of the problem and restrictions of the new-style excerpts documented only in this meta post.
  2. Allowing for easier automated removal of the excerpt should the real excerpt be brought back at some point.

The mass-editing will be a one-time occurrence to restore tag wiki excerpts to at least being functional at a scale. Maintaining them afterwards, and tweaking them to work with this new anti-community system will have to be done manually afterwards.

New excerpt restrictions

Note: These restrictions are observed behaviour of how the excerpt selection process works, and not policy. The excerpts can obviously be longer or otherwise break the restrictions, but if they do, the excerpt will not be displayed in full. This is one of the many reasons why getting the excerpt back is the most optimal strategy here, but again, this is the contingency plan if excerpts aren't brought back.

The following new restrictions have been imposed on the new-style excerpts:

  • The excerpt is selected from the first line of text that isn't a header, and that isn't an HTML comment. No other lines are considered
  • The character limit has been drastically reduced from 5-600 characters to just 215.
  • The excerpt must be on a single line (i.e. not containing linebreaks, it can obviously wrap in the editor)
  • The excerpt shown also stops at the first period
    • ... but apparently only if said period has a space after it. For example "next.js" doesn't cut off the excerpt at just "next" (see for a live example)

Note that the mass-editing process will not address anything but newlines and punctuation. Editing the excerpts to better fit these new requirements is best left to people after the excerpts have been moved.

How and when the edits will be performed

There are 46177 tags (69.95% of all tags) affected by SE's short-sighted decision. For obvious reasons, editing this by hand is incredibly time-consuming. If the mass-editing is approved, I will be writing an automation tool to handle all the edits. No action is required from the community at large to execute this proposal; I'll take care of making the automation tool and actually performing the edits. This also means that if SE has a problem with us working around their anti-community change, I'll take the heat.

As the API can't be used here, the speed of the edits are drastically reduced. With a working estimate of 5 edits per minute (~7200 edits per day), combined with some restrictions when running automation under mod accounts, I estimate the edits will take 7-14 days to complete. This assumes a few best-case scenarios, and assumes CloudFlare doesn't get in the way. It might be faster, it might be slower, but I hope it'll be possible to complete in under 14 days.

The development of the automation tool will start when the deadline has passed, and the edits will start as soon as the tool is operational. I estimate 1-2 days after the deadline to get everything working, with a large part of that time being waiting for the API to return all tags on the site for progress management.

Risks

One of the obvious risks is the duplication of content, particularly if any editors handle individual tags. I'll be adding a few safeguards to avoid this, particularly detecting if the excerpt is present anywhere in the body, and I plan to output any skipped tags somewhere for fully manual review. I hope that somewhere becomes chat, but this hinges on things I don't currently know. If chat isn't an option, a logfile will be used instead.

I'll still take point on editing or reviewing these, but if chat is a viable option, anyone can help edit tags that couldn't be edited automatically.

Aside this, and the obvious risk that I piss off someone at the company, there aren't any major risks. We're talking about trivially-automatable edits working with standardised input fields, and no major context-dependent changes, so I don't expect anything to break. I'll obviously keep an eye out for breakage, though.

Purpose of this meta post

This meta post exists to gather feedback on this plan, as well as to determine if we want to go forward with this. Going via meta is also meant to reduce the chance SE steps in if the plan is implemented. If the community consensus is to not perform the edits, the edits will not happen.

Again, optimally, SE would restore the excerpts so we don't have to do 46000 edits to get back to what we used to have. But if this proves to not be an option, this proposal gives us a way to put up a fight for the excerpts we've spent years creating. This is also a rare case where the change introduced by SE can be counteracted by us - even though that isn't optimal, it's certainly better than losing the excerpts.

Feedback will be accepted up to the same deadline given to SE (2024-08-30 at 12:00 UTC). I acknowledge that this is a short deadline, but the excerpts are important enough to warrant it.

41
  • 5
    so... you have to edit tags, to see whether or the tags that were selected make sense...
    – Kevin B
    Commented Aug 24 at 5:11
  • 5
    @starball what if I want to see how a tag is used by looking at a list of questions? My two options are: 1. Click the tag -> click "Go to Wiki" only then see the information or 2. Open the question -> edit post/tag only then be able to see the information. Besides, I don't think I don't think SE should get away with making a zero step process into a multistep process because at this point it seems like like (applying Hanlon's razor) they are dangerously inept at developing their own product. Yes, this is just one more instance but it has to stop at some point.
    – VLAZ
    Commented Aug 24 at 5:54
  • 27
    I still hope that SE will revert their decision, if not then we basically have no choice other than doing what you are proposing.
    – Dalija Prasnikar Mod
    Commented Aug 24 at 8:58
  • 6
    @dan1st SpencerG, who posted at least the recent tag-related posts, has been directly pinged in the mod backroom, coincidentally following a discussion where they were involved and active. The chance it has or will be seen is practically a guarantee. My message has not been acknowledged, but reading and not responding is very on-point for the company at the moment. I'm waiting on multiple responses. I will ping again on Monday and Thursday with increased ping lists if I continue not to get any acknowledgement, and after the deadline if they don't respond in time. Commented Aug 24 at 10:22
  • 3
    There's still every chance they deny having heard anything about it in spite of multiple attempts to directly contact the people involved (and on Monday and Thursday, non-involved employees), but SE will know even if they pretend not to. Commented Aug 24 at 10:28
  • 7
    "There's still every chance they deny having heard anything" the issue with the excerpts was brought up well over a month ago when they announced the experiment. Once again when the tag hover experiment concluded. Then SE explicitly acknowledged they had no idea how users used excerpts and asked for feedback. They've had plenty of warning. They can claim they've not seen this one, but it's not an excuse.
    – VLAZ
    Commented Aug 24 at 11:17
  • 15
    If they're doing everything they can to make it burn, it's about time to let it burn. Throwing even more volunteer time at this is sending the completely opposite message from what they should get. Commented Aug 24 at 12:24
  • 4
    @Lino They'll be copied over anyway, and they'll have to be manually edited on a per-case basis afterwards. The truncation doesn't happen during the editing process, only in the resulting display, so a full excerpt can be copied over without that being a problem. It'll just be cut off on the tag page. There aren't any good ways to uniformly handle these cases, so human editing after the fact is the best option. Commented Aug 24 at 15:04
  • 2
    Unfortunately, I don't currently have a number on how many excerpts are > 215 characters, so I don't know how much manual intervention is required. A lot of relevant information is present in the first 215 characters for most tag wikis I've seen though, so we do lose some information still, but far less than just losing the excerpts outright. It should be fairly easy to figure out with SEDE, but SQL joins on the large (presumably) Posts table often times out when I try writing queries, so I haven't bothered checking. (If no one else can get it to cooperate, maybe the blurry flower can help?) Commented Aug 24 at 15:10
  • 4
    @dan1st: This just feels like another step in the company's war with the community. I upvoted Zoe's proposal for a few reasons but it does make me uncomfortable for other reasons. Ultimately I'm with Zoe, but it's just hard to believe it has to be this way. Commented Aug 24 at 17:22
  • 9
    @PresidentJamesK.Polk Strictly speaking, nothing. If they decide they want to revert them, they can and will. This is the same reason they removed the excerpts in spite of our loud objections; there was nothing preventing them there either. Practically speaking, going against a community consensus burns goodwill, especially if the reversals also come with, for example, a suspension. It will make people leave, and further worsen their situation. We're already down 4 moderators so far in 2024. Commented Aug 24 at 17:57
  • 17
    For what it's worth, I consider the risk of reversals (and especially anything more than just reversals due to the consequences of the Monica incident in 2019) to be unlikely. Not impossible, but unlikely. That's part of why I'm planning to do all the edits; I don't know how it affects the likelihood of reversals, but I have the resources and connections to take any form of fight if they start it. And yeah, it's hard to believe this is what the situation has come to, especially after now 5 years of apparently empty promises to prioritise the community. But it is what it is Commented Aug 24 at 18:00
  • 9
    "Practically speaking, going against a community consensus burns goodwill". Judging by the amount of goodwill they've already burnt, I'm beginning to think the remaining amount of goodwill here is infinite. Commented Aug 24 at 21:30
  • 8
    Fun fact: the changes that are motivating this have not been made to Teams. This is interesting, because they usually roll out what they think are the best new features to Teams first, before the public site. On Teams, you still have both "Watch tag" and "Ignore tag" buttons, and it still displays the tag usage guidance.
    – Cody Gray Mod
    Commented Aug 25 at 12:51
  • 2
    @Starship clearly, the company wants the community to make the edits. They haven't said not to.
    – VLAZ
    Commented Aug 25 at 17:00

2 Answers 2

-3

Utilizing this script for these edits won’t be necessary. When we discussed the tag hover changes earlier this month, we were clear that the difficulties this introduced to curation activities were not anticipated. Because of this, on August 16th, 2024, we requested more feedback about curation activities and their use of tag hover descriptions to curate so we could better understand how this impacted them.

After reading the feedback, on August 22nd we approached the Stack Overflow moderators in their private mod chat to understand if moving forward with variant B of the tag hovers but with the tag wiki excerpts restored would solve the problem, as we had planned to make that change this week. We heard from a few moderators, but Zoe, we can’t help but notice that you never responded to our inquiry. As you might imagine, we are a bit surprised by this post.

As mentioned here, we will be reverting the changes made to the tag hover later this week. Regarding the tagged questions page, this change was communicated in this post on MSE on June 12th, 2024, in the section titled ‘Experiment 3: Recent Tags entry point’. We never received any feedback on this post about the change to the tagged question header being simplified as being an issue, other than some critiques to the links being moved. Due to that, we moved forward with it as a change. Because of the feedback we received on these two posts here, and here last week, we were already discussing whether we needed to reconsider changing the tagged question pages. Given that there is quite a bit of support here for tag excerpts to be restored to their previous state, we will post a separate meta post to understand better how this fits into use of the tagged question page. You can expect that later this week or early next week.

3
  • 13
    "We never received any feedback on this post about the change to the tagged question header being simplified as being an issue" but at no point does the announcement even explain that tag description would change on the tagged questions page. Sure, maybe the users should have meticulously swept through the sample image you shared then had to cross-reference against the real page, verifying where the text came from. But I feel that's not what the users should be doing.
    – VLAZ
    Commented Aug 28 at 14:41
  • 12
    But let's be brutally honest here - I personally am very fast losing any hope for leaving feedback. You (SE) have been notified you've introduced a breaking change and just...done nothing. Why should I really be giving any feedback when it seems it's often ignored. You got feedback for the tag excerpt on hover. With a question for why it was even changed. There has been 1. no answer and 2. You found further feedback about the excerpt unexpected. Don't wonder why you get no feedback in the face of this.
    – VLAZ
    Commented Aug 28 at 14:50
  • 3
    More feedback about the excerpt change on hover: meta.stackexchange.com/a/401396 meta.stackexchange.com/a/401511 meta.stackexchange.com/a/401913 Granted, these were posted was after the deadline. But how exactly should have users communicated this information such as to not make the same feedback coming later "unexpected"?
    – VLAZ
    Commented Aug 28 at 14:57
6

This is madness! *

As impressive as the commitment to doing this is, it does not change that the new status quo is fundamentally broken. Worse, it will make the status quo actually work, hiding just how broken it is. Worseningly worse, it will show that people actually consider it worth the initial and ongoing effort to make it work!

This is not the message that should be sent given such a change. It should fail, and it should fail fast.


*Madness? This is Stack Overflow!**

**Reference just for literary effect. No literal interpretation intended.

4
  • 10
    While I agree that in general, 'stupid should hurt', the people that are hurt by the company's actions in this case are the (rapidly diminishing) number of people who curate stuff for free on this site ... not the company or even its employees. Leaving it broken isn't going to make the company react, based on previous stupidity; instead it'll just make people to continue to stop participating in curation activities. Commented Aug 26 at 19:16
  • 1
    Leave it broken! The CMs involved couldn't bother to so much as post and own up to the disruption caused.
    – bad_coder
    Commented Aug 26 at 19:25
  • 3
    @RoddyoftheFrozenPeas Yes. Yes it will indeed make some people stop curating. That by itself isn’t a bad thing. There’s definitely a point when people should stop throwing their time at this. Commented Aug 26 at 19:30
  • 4
    This won't make the status quo work. Having the company remove excerpts from the UI in some places, then mass editing them back in so it shows properly in some but twice in other places leaves the site in a poor and inconsistent state. Maintaining them would be awful, as editors would need to be aware of the peculiarities of the "excerpt picking algorithm". These edits would make a good point not to change things against the communities wishes, but are not a workable solution to the problem at all. That's why I'm all for it!
    – Erik A
    Commented Aug 26 at 20:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .