Regarding the close votes, the question has been reopened so I can no longer see the dupe-target it was closed as, but:
The question is about C# yet entirely fails to provide any information about what version of the .NET runtime is being used. That's an important detail; when it comes to debugging complex issues like this, it's a critical detail. Without the runtime version, I don't know which source code to look at to potentially help you (yes, the .NET source code is mostly available; your claim that "Due to the nature of not being able to see why the Ping is hanging" is completely false).
That, for me, is an automatic "needs details or clarity" close vote. Sorry, but regardless of how well-written your question is, if you fail to provide the most basic information, I have better things to do with my time than type comments in an effort to try to squeeze blood (information) out of a stone (you). The onus is on you as the asker to make your question answerable, not anyone else.
Regarding the downvotes:
While the question it may be well-described, at the end of the day it is describing an issue that has significant complexity, and thus requires significant investment from a potential answerer. Especially when you state "after hours (or days) of running" and "Most of the time all the tasks finish and are disposed; it is just randomly that they don't finish", it starts to sound like this is an environment- or configuration-specific issue that quite possibly can't be replicated by anyone else.
Effectively, you're asking for debugging by proxy, and that ain't how Stack Overflow works; this type of question would likely be much better suited to a forum post or GitHub discussion or reddit thread.