32

<! dedicated to Brad Larson (https://meta.stackoverflow.com/a/332611/839601) -> As far as I can tell there are some difficulties in the handling of certain kind of link-only answers. I would like to understand how many such answers are there - old, accepted, link-only answers with solid positive score.

I hope that there aren't too many because in this case we could make a list and arrange some kind of one-time clean-up effort - edit worthy ones, delete unworthy and put this whole affair to rest.


Have to admit, I don't quite understand how to reliably find link-only answers. The only SEDE script I am aware of is this one. Few years ago this script was used to cleanup link-only answers at Programmers. Per my recollection, it raised many false alerts which is OK at small site where people can further "filter" results manually but hardly acceptable at the size of Stack Overflow.

From what I observe in LQ review queue at SO it looks like more reliable scripts exist and are even regularly used to push link only answers into this queue. Their results aren't 100% perfect either but seem to be good enough. I wonder if such script can be used to help get statistics on what I am looking for.


With regards to what score posts to consider for this statistics, per my observations at meta troublesome answers tend to have score 10 or higher. I would be also interested in stats for posts with score 5 and higher - just in case if there are not too many of these, cleaning them up could provide some headroom and tame the flagging issues even more.

As for what to consider old enough, since we're talking about link only answers most natural cut off date seems to be March 2014. Just in case if this criteria picks too many posts, second best cut off date would probably be November 2012.

4
  • to avoid misunderstanding, I realise that it's possible for late (very late) upvotes and accepts to push some old answers into this troublesome category anytime - today, tomorrow, years later. I just expect that this will happen rarely
    – gnat
    Commented Aug 20, 2016 at 21:36
  • 8
    I'll try to dig into this early next week.
    – Taryn StaffMod
    Commented Aug 20, 2016 at 21:58
  • @bluefeet, recent answer from Shog seems to cover things quite thoroughly, would you be willing to add something over it? I am trying to figure whether it makes sense to wait for your answer before pinging SOCVR folks for help in planning the cleanup
    – gnat
    Commented Aug 23, 2016 at 9:30
  • 1
    Go for it @gnat, no sense in me duplicating some of what Shog did.
    – Taryn StaffMod
    Commented Aug 23, 2016 at 11:34

2 Answers 2

10

Laurel's query is a good start. Simply factoring out the length of the URL isn't enough though, since auto-linked URLs will repeat the URL in the link text... So you'll want to strip the entire link in these cases, but leave the link text in other situations where it actually contributes to the content of the post.

Parsing HTML with SQL makes one yearn for regular expressions... But if we make some assumptions (no code allowed, total length less than n characters, no more than one link per answer) and operate on the original Markdown, we can at least find answers that are very short without links without too much trouble:

select top 1000 AnswerId [Post Link], Score, x.CreationDate, Url, Len(replace(h.Text, Url, '')) LengthWithoutUrl
from
(select 
  a.Id as AnswerId,
  a.CreationDate,
  a.Score,
  substring(a.Body, charindex('<a href="', a.Body)+9, (charindex('"', a.Body, charindex('<a href="', a.Body)+9)-charindex('<a href="',a.Body)-9)) Url,
  (select max(Id) from PostHistory where PostId=a.Id and PostHistoryTypeId in (2,5)) LatestPostHistoryId
from
  Posts a
where
  a.PostTypeId = 2
  and Len(a.Body) < ##MaxAnswerLength?512##
  and a.Body like '%<a href="%'
  and a.Body not like '%</code>%'
  
  ) x 
join PostHistory h on Id=LatestPostHistoryId
where Score >= ##MinScore?5##
and Len(replace(h.Text, Url, '')) <= ##MaxBodyLengthWithoutUrl?29##
order by x.CreationDate desc

You can find this live here: https://data.stackexchange.com/stackoverflow/query/527543/find-some-link-mostly-answers-that-are-positively-received?MaxAnswerLength=512&MinScore=5&MaxBodyLengthWithoutUrl=29 - with the default parameters, it finds 1866 answers.

Note that with the default value of 29, the first result is from 2014 (because the len() function doesn't count trailing spaces) and the rest are older (because a similar but more robust technique blocks posting answers shorter than this). So if you want new stuff, increase that value somewhat.

Even still, I recommend reviewing these with a somewhat skeptical eye. To survive this long, they would tend to have some redeeming value; in many cases, either the question asks for a link or is trivially answered in just a few words.

12
  • On a bright note... We're down to only 215 link-only answers on Stack Overflow...
    – Shog9 Mod
    Commented Aug 23, 2016 at 1:17
  • interesting, thank you! Setting minScore 10 shows 503 posts at length without URL under 29. That looks doable - I think at Programmers about 2K answers were cleaned up. Even min score 7 shows 995, still realistic. Your note about keeping skeptical eye makes sense: in my experience when reviewing such posts it was worth paying attention to things usually considered secondary. It was quite frequently that either link content turned out good and easy to digest (meaning easy fix with edit) or question itself was blatant request for off-site resource (meaning that fix was to delete question)
    – gnat
    Commented Aug 23, 2016 at 7:20
  • 1
    I wonder how many of those are answers to closed questions... I clicked one question that had at least 3 answers that fit all but the score criteria.
    – Braiam
    Commented Aug 23, 2016 at 16:53
  • If you're operating on the Markdown, why are you looking for <a href= rather than [link](URL) ?
    – Ben Voigt
    Commented Aug 23, 2016 at 18:55
  • It's much, much faster to scan the rendered HTML first and then find the corresponding Markdown for matching posts, @Ben. Also more effective, since the raw Markdown might have raw URLs that are auto-linked, might have inline links, might have footnote-style links. However, the rendered HTML also pads out these short answers considerably, making it difficult to gauge the effective length of the answer without the link... Thus, I pull in the last revision and strip the URL to perform the length check. The internal check actually uses an HTML parser, which would be much nicer here.
    – Shog9 Mod
    Commented Aug 23, 2016 at 20:39
  • I should note that a previous version of this operated entirely on HTML and was much faster (don't even bother trying to run this on all answers). However, accuracy suffered greatly. YMMV, but if the goal is to find old, overlooked answers in need of some love, churning through tens of thousands of false positives isn't going to help.
    – Shog9 Mod
    Commented Aug 23, 2016 at 20:44
  • @Shog Yes that's perfectly sensible, but then maybe your answer should say it uses both the HTML and the Markdown. Right now you have "if we ... operate on the original Markdown"
    – Ben Voigt
    Commented Aug 23, 2016 at 21:50
  • Yeah... @Ben, on the one hand I did kinda breeze through the details of that query; on the other, not entirely the point of this question. On the gripping hand, I'm not exactly a SQL wiz, so I'd defer to anyone willing to make this less of an abomination.
    – Shog9 Mod
    Commented Aug 23, 2016 at 22:58
  • Shog, I would appreciate if you create chat room for this answer. At first I thought to discuss cleanup based on your script at SOCVR but after making list of points I wanted to clarify I figured that it would be hard to cover these without a dedicated room
    – gnat
    Commented Aug 24, 2016 at 21:25
  • 1
    Sure, @gnat. We can continue this discussion in chat.
    – Shog9 Mod
    Commented Aug 24, 2016 at 21:31
  • Shog, why does this post discuss parsing HTML with regular expressions? I thought that bobince taught us that you can't.
    – Nissa
    Commented Feb 16, 2017 at 2:13
  • Part of giving up programming for a living means sometimes using the wrong tool for the job, @Stephen.
    – Shog9 Mod
    Commented Feb 16, 2017 at 3:54
14

I wrote some queries a little while back that return a great deal of link only answers. These aren't something I would let flag everything they find, but they could be a start.

The last time I took up this endeavor, I realized that far too many of the link-only answers were merely symptoms. It's a bit hard to justify deletion of these types of answers when they're posted to an open recommendation question.

Eventually, the recommendation question problem took center stage. It's another battle, but I have also made a good query for it. (Of course, there is still more work to do...)


The best indicators of link-only answers are length and the presence of <a ></a> tags. This is pretty obvious, and I include both these factors in all my link-only queries (unless I'm trying something experimental).

Keep in mind that < (and >), when present should always be part of the post's HTML; any other usage of <, including code snippets, will be escaped.

Beyond this point, it depends on what you consider a link-only answer to be, and if you also want to include answers to closed questions. Like this one.


There are exactly 222 posts that are returned with:

select top 1000 id as [Post Link], body, score
from posts as a
where body not like '%<code>%' and body like '%<a href%'
and len(body)<100
and parentid is not null
and exists (select * from posts as p where p.acceptedanswerid= a.id)
and score>9

This is likely a very, very conservative number, since 100 characters isn't a lot, and 10 is pretty high of a score, especially when it also needs to be the accepted answer. It's also excluding answers that use any amount of code, since I was looking to avoid false positives as much as possible here.

One of the biggest areas where this can be improved is with length allowance, since the URL length is counted against the 100 character total.


I'm still digging through my old SEDE queries, but I found one query that looks promising. Its original purpose was to find questions that linked to deleted questions. It's actually quite incredible:

declare @sitebaseurl nvarchar(40) = N'https://stackoverflow.com/questions/'
declare @pattern nvarchar(60) = N'%'+ @sitebaseurl + N'[0123456789]%'
declare @closingtoken nvarchar(20) = N'%[/"#? <)'+char(10)+N']%'
declare @sitelen int = len(@sitebaseurl)

select top 50 p.id as [Post Link], body

, substring( p.body 
, patindex(@pattern, p.body) + @sitelen -- spos
, patindex(@closingtoken, substring(p.body,patindex(@pattern, p.body) + @sitelen, 18)) -1 -- epos
)  from posts p 

left outer join posthistory ph 
  on ph.postid = cast(substring( p.body 
       , patindex(@pattern, p.body) + @sitelen -- spos
       , patindex(@closingtoken, substring(p.body,patindex(@pattern, p.body) + @sitelen, 18)) -1 -- epos
       ) as int)
where body like 
'%https://stackoverflow.com/questions/[0-9]%'
and body not like '%11863521.aspx%'
and ph.id is null

I think that I can use some of that query to improve my other query.

5
  • interesting observation wrt "symptomatic" answers that point to recommendation question. We bumped into the same thing when cleaning up link-only answers at Programmers over 3 years ago. Back then moderator advised that the right way in these cases is deleting the question: "flag the question and ask for it to be deleted. If 9/10 answers are cr@p, it makes more sense to just get rid of it than commenting on every cr@p answer..." This way we got rid of many link-only answers
    – gnat
    Commented Aug 21, 2016 at 0:15
  • 1
    @gnat If we can get all (or enough of) the mods on board, that sounds like a really good idea. That's the challenging part, unfortunately.
    – Laurel
    Commented Aug 21, 2016 at 0:46
  • 1
    On a side topic, a way to assist preventing link onlies if to not count the char in a link, and to not allow repeating char e.g. ................................................... There would still be people posting nonsense, but it would force the average user to write something semi sensible
    – user3956566
    Commented Aug 21, 2016 at 1:47
  • 5
    @Laurel difference in scale makes it hard to directly reuse Programmers approaches over here. Think of it, SO is 200x larger but has only 4x more moderators. Even if they all agreed to delete, this would mean every one of them would have to do 50x more work than their peer at Programmers, that would be a murder. SO scale makes many things so much more difficult...
    – gnat
    Commented Aug 21, 2016 at 8:54
  • 2
    ...for example, look at my own request here, how I try to limit scope of work: "old, accepted, high score". And even then, I ask for stats to check if we can bite that chunk or not. For comparison, at Programmers we didn't bother with limiting at all, we simply went through and cleaned up everything, and it was apparently doable. Not easy mind you but still. Yeah 200x size difference matters
    – gnat
    Commented Aug 21, 2016 at 8:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .