47

The Stack Overflow question What is the best way to parse html in C#? is locked for historical significance. The point of locking is to preserve meaningful content that doesn't fit in with what Stack Overflow is today. However, unlike most other locked questions, this question is in essence technical — with minor alterations it could fit SO excellently — it's the top hit in Google, Bing, Yahoo and DuckDuckGo for "how to parse html in C#" (without quotes; searched in private mode).

Furthermore, this question extremely out of date — it's from 2008; it's 2015 now. The content in the question, unsurprisingly, is somewhat misleading — the top voted answer links to HAP (HTML Agility Pack); in 2008 that might have been a good option, but probably isn't anymore (HAP is a hand-written parser that has issues with self-closing tags and other fairly common HTML practices).

In essence:

  • This question covers a topic that is in-scope for SO, but is locked likely because it is poorly worded (the word "best" springs to mind).
  • It's extremely out of date.
  • It's existence discourages new questions covering this topic because duplicate content is frowned upon.

As a result, the question conflicts with Stack Overflow's core purpose — rather than getting good answers to practical, detailed questions, it's obfuscating good answers by misleading readers into missing out of advances in the period 2008-2015.

Related: Reopen or delete closed (and locked) questions that are in the top results for web searches? and Is there a process for re-opening or deleting locked questions?. I hope that this question makes the case for deletion more clearly than those.

14
  • 5
    Well... on recent questions on this topic most people still recommend HTML Agility Pack. Maybe this tool has evolved too since 2008 and is now better (I don't know, I have no experience with it). Commented Jan 17, 2015 at 12:28
  • 2
    If there is a better parser than HAP, then IMHO the top voted answer should be edited accordingly, preferably extended while keeping the context (i.e. explaining why HAP is no longer recommended). At least in a post that has serious traction with search engines. That's what's the whole point of SO: That outdated information can be updated without damaging "meta"-structures like search engine ranking.
    – Tomalak
    Commented Jan 17, 2015 at 12:46
  • 8
    @Tomalak: the point is: you cannot edit answers on closed questions Commented Jan 17, 2015 at 12:57
  • @ThomasW. the lock type could be changed to wiki lock in principle that allows editing. Commented Jan 17, 2015 at 13:19
  • @Thomas I know. That's why the question should be whether to re-open it solely on the basis that it is popular with search engines. I acknowledge that this is a whole new bag of hurt, though. :/
    – Tomalak
    Commented Jan 17, 2015 at 13:20
  • 2
    @eamon do you have a specific better answer in mind or do you just think there probably is one? Commented Jan 17, 2015 at 13:20
  • 7
    Ì don't think this question can be salvaged. It would still be a question asking for a library. Just delete it. Commented Jan 17, 2015 at 13:37
  • 7
    The top Google hit is not that SO question, it is a Codeproject.com project. Posted in 2014, it proposes HAP. So, problem solved. That such an awful site gets better ranking than SO is the much bigger issue. Commented Jan 17, 2015 at 17:43
  • 7
    @Hans: The top Google hit depends on your language and location. For me, SO is the top hit. And CodeProject is not that awful. IMHO it was one of the best sites before SO came up. Commented Jan 17, 2015 at 20:13
  • Old and unchanged content is going to be a problem. I see that more and more developers using the Google feature to show only search results e.g. of the last year in order to avoid getting outdated answers. Doing that, CodeProject (for me) is the top hit and even softwarerecs.stackexchange.com/questions/10773/… bubbles up. Commented Jan 17, 2015 at 20:16
  • 3
    @MartinSmith: A bit of both. I've used HAP before, but I regularly need to parse html, and HAP mis-parses quite a lot of html; even valid markup. So I occasionally look around for alternatives, I recently stumbled across CsQuery, which claims to follow the spec to the letter (using a parser related to the firefox html parser). It's been around for more than 2 years, I'm a little peeved in that I suspect I'd have found it earlier had this SO question not been locked. It wouldn't surprise me if there were other alternatives too; I really don't know; the SO thread is just so old. Commented Jan 18, 2015 at 10:40
  • 1
    @Tomalak: I don't think it's good practice to edit other's answers to radically change their content (and doing so requires considerable rep). Revisiting old threads is in any case not something stackoverflow encourages; new answers will with near certainty languish unseen at the bottom of the list. I think the right approach might be simply to revisit topics that are prone to change once in a while (say no more than once a year?) Honestly though, I'd be happy if at least this question got deleted, since it's content is particularly misleading. Commented Jan 18, 2015 at 10:45
  • 1
    Old and outdated is not something unique to this question, if you are going to delete that question then it begs the question are you going to do the same for every other outdated question on the site? Need to remember that just because an answer is outdated it does not mean that people are still not using the outdated versions and the question can still be useful to them. I think it is up to the person looking at the question to determine if an older post is still relevant to them.
    – Joe W
    Commented Jan 19, 2015 at 16:41
  • 1
    @EamonNerbonne You say you want to delete this question to make room for "fresh blood", so to speak, but any question of this nature would be squarely off topic nowadays (asks for a tool, library, etc.). What kind of question/answer are you envisioning that could replace this one and survive?
    – JLRishe
    Commented Jan 19, 2015 at 21:21

4 Answers 4

22

Usually I balk at this sort of thing. One person's "incorrect" or "out of date" answer is another person's "correct" and "in date" answer; we're supposed to let the community decide the best content through voting, not allow one individual to come onto meta and dictate what is and isn't true.

However, your argument is well-reasoned and well-presented, and I consider this case to be an exception to my general rule. That is, I support your proposal. In fact, I see the continued existence of questions like this (and the ensuing conundrum of what to do with them) as a semimajor flaw in the SE system.

If nothing else, a question like that now belongs on the Software Recommendations beta; if people really like this content then it can be posted there. Again I would agree with your sentiment, by suggesting that it may be best just to start from scratch were this particular question to find its way onto SR.

11
  • 8
    questions like this (200K+ views and likely hundreds incoming links that will rot after deletion) would better go (redirect) to museum.stackexchange.com, a separate site with a design clearly indicating that it doesn't belong to SO. Not that I find the content worth keeping (in this case, I feel rather opposite), just don't want SO to add too much into link rot over Internet
    – gnat
    Commented Jan 18, 2015 at 13:04
  • 6
    @gnat: Link rot is preferable to no link rot if the resource pointed to by the link is itself rotten. :) Commented Jan 18, 2015 at 14:08
  • 3
    those getting 404 not found won't see that content sucks. They will likely simply blame Stack Overflow for "making Internet worse"
    – gnat
    Commented Jan 18, 2015 at 14:43
  • 6
    @gnat: That's still preferable to them seeing crappy, outdated and/or incorrect content and thinking that Stack Overflow has put it there for them to learn. Commented Jan 18, 2015 at 14:45
  • 3
    well since this garbage is presented as belonging to Stack Overflow, you're maybe right
    – gnat
    Commented Jan 18, 2015 at 14:47
  • 1
    Maybe such questions should not be indexable/searchable? (but exact links should stay online, with a bigger warning at the top although). Maybe the actual, "vicious", internet-breaking deletion should be for illegal/spam content only?..
    – Vi.
    Commented Jan 19, 2015 at 22:07
  • 1
    That particular question can't belong to Software Recommendations as it is shopping list question... There is no SE site to my understanding where "recommend the best" type of question would be acceptable. Commented Jan 19, 2015 at 22:27
  • 2
    @AlexeiLevenkov: sort of on a tangent here, but in a sense, virtually every stackoverflow answer is a "recommend the best" type of question. That's kind of implicit in SO's setup - after all, you ask how to do X, and then pick the best answer for how to do so. But yeah, it's not good form - it's fortunately an easy flaw to fix, however. Commented Jan 19, 2015 at 22:37
  • 1
    @EamonNerbonne good question on SO is "recommend solution for my concrete problem/restrictions" - better solutions can only be upvoted when there is clear problem and/or clear list of requirements. The linked question just asks for "the best" without any specifics - so can't be considered good on SO/SR. Commented Jan 19, 2015 at 22:56
  • 1
    @AlexeiLevenkov: I definitely agree - I just think the distinction is pretty subtle. But yeah, using the word "best" tends to imply lack of specificity, as it does here too. Commented Jan 19, 2015 at 23:01
  • Do you happen to have any idea how to get this discussion to have a practical effect? As in, who needs to be prodded how to delete this question? Commented Jan 29, 2015 at 10:18
10

Let's pretend that this question were better written. Something like

I'm working on a project that needs to screen-scrape a web site. I tried using XDocument to parse the HTML that I get back from HttpClient, but it throws exceptions saying that the format of the XML is invalid. I thought of using regular expressions, but my team lead threw a fit when I mentioned them.

How should I parse HTML in .NET?

It seems to me that this is not a request for a tool recommendation. If .NET had a System.Net.Html namespace with an HtmlParser class, then that would be the answer, and nobody would consider it a request for a third-party tool.

But .NET does not have a good way to parse HTML, so we're left with no good way to answer people who have a legitimate requirement to parse HTML, unless they're able to use the WebBrowser control.

I don't personally care whether this particular question gets deleted or not, but I do care that we have an answer for those users who need to parse HTML.

So, what how do we achieve that?


P.S. This seems very familiar. I'm not sure whether I'm feeling deja vu, or whether I've answered a question like this before and just can't recall.

5
  • 2
    The question remains too broad. There's no end to the libraries or tooling (forking an external process that will produce an intermediary representation is a valid solution in some circumstances) that people can propose to solve the problem, and the question does not provide enough details to really rank the answers as to their adequacy towards a specific purpose. (I've used enough parsers to know that some are best in different situations.) Writing an implementation of an HTML parser in an answer would make the answer too long.
    – Louis
    Commented Jan 19, 2015 at 21:53
  • 2
    @Louis: that's fine. A real question where those issues matter should result in the people trying to answer it requestion clarification on those points via comments - the typical Q&A interaction on stackoverflow. Indeed that might mean there are eventually multiple different Q&A's on this, and that too is fine - as long as they all cover a different use case. Some of those questions might be popular; others might be niche tricks. All of that strikes me as being a lot healthier than a locked question that overshadows more relevant threads. Commented Jan 19, 2015 at 22:30
  • 2
    @EamonNerbonne A question that is too broad should be close-voted immediately. Closure pushes the question closer to deletion. I was under the impression that this is what John wanted to avoid.
    – Louis
    Commented Jan 19, 2015 at 22:34
  • @Louis: Close-voting immediately is another way of continuing the discussion - it's not always obvious what's too broad (or at least I think it's reasonable to expect different people to consider different things when making that decision). Regardless, after the comments/edits improving the original question, or after the original question was closed and a rewritten one posted, you would end up in a situation where there's a perfectly fine question about parsing HTML in some specific scenario. So I don't consider this a reason that HTML parsing in general should be taboo on SO. Commented Jan 19, 2015 at 22:44
  • 2
    @Louis: There are always exception to the rules. In this case, we are encountering a common problem, which don't have a canonical answer and most questions receive bad solutions to the problem. Therefore, it is totally acceptable for there to be canonical Q&A dedicated to show casing the usage of some of the better tools out there, which can be used for closing all similar questions as duplicate. Of course, such question should be seeded with some initial good content first to avoid horde of low quality contribution, and it should be regularly maintained by the community.
    – nhahtdh
    Commented Jan 20, 2015 at 7:19
6

"How do I parse HTML?" has a single correct answer: "Use the HTML5 parser algorithm."

"How do I parse HTML in C#" therefore also has a correct answer, namely whatever C# library is considered to be the best implementation of the HTML5 parser algorithm in that language. I would consider that an on-topic question for SO.

The HTML Agility Pack, as best I can tell, is not an implementation of the HTML5 parser algorithm and therefore it should not be the chosen answer to that question.

(I am, in general, dubious of the policy against library recommendations, because -- as this example demonstrates -- there may be solid, nonobvious, objective reasons to prefer one library over another.)

2
  • Those reasons to prefer one library over another should be the subject of specific questions about specific problems with "the other" library. Commented Jan 20, 2015 at 15:51
  • @JohnSaunders Someone searching for keywords like "parse html c#" is not likely to find such a question, which would probably appear to be about some specific corner case that HAP or whatever didn't handle properly.
    – zwol
    Commented Jan 21, 2015 at 1:11
3

We should change that particular question to "Canonical" and change lock to the other one that has single Wiki entry.

Why: We already have good duplicate for "I want to parse HTML in C# with regular expressions", but there are many similar questions that try to use Xml classes to parse HTML or string manipulation. Indeed How to use HTML Agility pack is acceptable alternative, but as many people pointed out there are better parsers now.

Also I believe top answer (HtmlAgilityPack) still way better that most people come up by themselves.

2
  • 3
    Maybe those other parsers are better. Maybe HAP still has advantages (e.g. a simpler API) - the point is that that discussion can't be had on a locked question. Commented Jan 19, 2015 at 22:33
  • The Zalgo question you cited is not a good dup for questions asking how to parse HTML with regex: meta.stackoverflow.com/questions/261561/…. It doesn't provide non-regex solution to the problem. All library answers are removed, and we are left with a bunch of joke answers and regex answers.
    – nhahtdh
    Commented Jan 20, 2015 at 7:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .