Context
On 2024-08-22, the company decided to fully remove tag wiki excerpts from the top-level UI by removing tag wiki excerpts from the tag pages themselves, which drastically undermines our ability to curate tags. The company has been told how important the excerpt is by several people (1 2 3 4 5) and explicitly acknowledging their importance, and opted to go ahead and remove the excerpt from the last top-level place it was shown anyway.
This decision is actively harmful for all forms of tag curation, as it currently hides the often critical excerpt information behind several clicks with no real-world benefits for any end-users - in any target group, curator or otherwise. Misrepresenting what tags are for by hiding usage information only results in increased rates of incorrect tagging and, for certain tags, increased rates of off-topic questions.
From experiments, a few tags seem to be exempt from this and still have a functional excerpt, but the existence of an excerpt whitelist is unconfirmed and purely observational. It's unclear why these tags haven't been affected, so they may still break.
For some more concrete examples of why this is bad, see this meta post. The change hides critical information the community has built up over the past 15 years or so, and in many case actively misrepresents what the tags are for. The excerpts were already hidden and little used, but this change further hides the excerpts.
We can't force SE to change the UI back to using the tag wiki excerpt - but due to how the new excerpt is picked, we have a way to work around this change. That is the purpose of this here proposal - offering a way to restore existing functionality, even if SE refuses to restore it.
Conditions for executing the mass-editing
The company has been given a deadline of 2024-08-30 at 12:00 UTC to respond to this post, and for said response to be the immediate restoration of the tag wiki excerpts in the top-level UI, or a promise that they'll be brought back in the extremely near future. The goal with the deadline is to force them to respond as soon as possible, as this change is doing ongoing damage to tag curation. Waiting 6-8 weeks for them to maybe perhaps respond is not a viable decision.
While this can go in either direction, the proposal outlined in this post takes effect if Stack Exchange, Inc. chooses not to restore excerpts in the top-level UI.
If the excerpt is returned, or a plan is made to return it in the very near future, the mass-edits are automatically off the table. Due to restrictions explained in the next section, this new form of excerpts is not optimal. Optimally, we'd get our normal excerpts returned. Mass-editing serves as plan B to forcibly restore the usefulness of the 46177 tag wiki excerpts - if this plan is executed, it's the last resort.
Editing strategy and scope
The new "excerpt" system uses the first line, excluding HTML comments and headers, up to EOL, a period, or 215 characters are used; whichever comes first. The proposed mass-edit will therefore copy the excerpt into a special comment-delimited block (for visibility) in the main wiki, with a few changes to punctuation (read: removing periods and replacing them with another punctuation character, or replacing them with a unicode equivalent that doesn't trigger end-of-line detection) to fit as much of it in as possible.
The edit will insert the following section at the top of every tag wiki:
<!-- Begin new-style excerpt. See https://meta.stackoverflow.com/q/431374 for more information about restrictions of this section. If you want to edit the excerpt, edit here too, not just in the excerpt box above! -->
[Text currently in the actual excerpt goes here, with some automatic punctuation changes due to the aforementioned restrictions]
<!-- end excerpt -->
The comment delimiters serve two purposes:
- Making editors aware of the problem and restrictions of the new-style excerpts documented only in this meta post.
- Allowing for easier automated removal of the excerpt should the real excerpt be brought back at some point.
The mass-editing will be a one-time occurrence to restore tag wiki excerpts to at least being functional at a scale. Maintaining them afterwards, and tweaking them to work with this new anti-community system will have to be done manually afterwards.
New excerpt restrictions
Note: These restrictions are observed behaviour of how the excerpt selection process works, and not policy. The excerpts can obviously be longer or otherwise break the restrictions, but if they do, the excerpt will not be displayed in full. This is one of the many reasons why getting the excerpt back is the most optimal strategy here, but again, this is the contingency plan if excerpts aren't brought back.
The following new restrictions have been imposed on the new-style excerpts:
- The excerpt is selected from the first line of text that isn't a header, and that isn't an HTML comment. No other lines are considered
- The character limit has been drastically reduced from 5-600 characters to just 215.
- The excerpt must be on a single line (i.e. not containing linebreaks, it can obviously wrap in the editor)
- The excerpt shown also stops at the first period
- ... but apparently only if said period has a space after it. For example "next.js" doesn't cut off the excerpt at just "next" (see next.js for a live example)
Note that the mass-editing process will not address anything but newlines and punctuation. Editing the excerpts to better fit these new requirements is best left to people after the excerpts have been moved.
How and when the edits will be performed
There are 46177 tags (69.95% of all tags) affected by SE's short-sighted decision. For obvious reasons, editing this by hand is incredibly time-consuming. If the mass-editing is approved, I will be writing an automation tool to handle all the edits. No action is required from the community at large to execute this proposal; I'll take care of making the automation tool and actually performing the edits. This also means that if SE has a problem with us working around their anti-community change, I'll take the heat.
As the API can't be used here, the speed of the edits are drastically reduced. With a working estimate of 5 edits per minute (~7200 edits per day), combined with some restrictions when running automation under mod accounts, I estimate the edits will take 7-14 days to complete. This assumes a few best-case scenarios, and assumes CloudFlare doesn't get in the way. It might be faster, it might be slower, but I hope it'll be possible to complete in under 14 days.
The development of the automation tool will start when the deadline has passed, and the edits will start as soon as the tool is operational. I estimate 1-2 days after the deadline to get everything working, with a large part of that time being waiting for the API to return all tags on the site for progress management.
Risks
One of the obvious risks is the duplication of content, particularly if any editors handle individual tags. I'll be adding a few safeguards to avoid this, particularly detecting if the excerpt is present anywhere in the body, and I plan to output any skipped tags somewhere for fully manual review. I hope that somewhere becomes chat, but this hinges on things I don't currently know. If chat isn't an option, a logfile will be used instead.
I'll still take point on editing or reviewing these, but if chat is a viable option, anyone can help edit tags that couldn't be edited automatically.
Aside this, and the obvious risk that I piss off someone at the company, there aren't any major risks. We're talking about trivially-automatable edits working with standardised input fields, and no major context-dependent changes, so I don't expect anything to break. I'll obviously keep an eye out for breakage, though.
Purpose of this meta post
This meta post exists to gather feedback on this plan, as well as to determine if we want to go forward with this. Going via meta is also meant to reduce the chance SE steps in if the plan is implemented. If the community consensus is to not perform the edits, the edits will not happen.
Again, optimally, SE would restore the excerpts so we don't have to do 46000 edits to get back to what we used to have. But if this proves to not be an option, this proposal gives us a way to put up a fight for the excerpts we've spent years creating. This is also a rare case where the change introduced by SE can be counteracted by us - even though that isn't optimal, it's certainly better than losing the excerpts.
Feedback will be accepted up to the same deadline given to SE (2024-08-30 at 12:00 UTC). I acknowledge that this is a short deadline, but the excerpts are important enough to warrant it.