It's not just the pageload that takes a hit. It's the preview that takes a hit, too. I've noticed that typing answers on Physics and Math is slower than on SO and MSO, probably because the preview reruns the mathjax renderer every keystroke.
Yes, this could be improved to behave like prettify where prettyprint is only run once every X seconds. But you usually want an immediate preview for MathJax because TeX is hard.
Also, matlab on Stack Overflow is tiny compared to php and jquery and bash and asp.net and ruby and perl and .... a lot more. ALl of these languages love the dollar symbol, and have it to denote variables or something similarly common. Of course, code formatting protects these from MathJax, but not everyone uses code formatting.
In contrast, for matlab, you probably don't need TeX in every post. I agree that I'm no part of the tag community so I may be horribly wrong here, but most of your questions don't seem to be the type that require math.
I guess it could be loaded on a per-tag basis as well. Or use something like the fiddle listed here that let one decide to load MathJax dynamically.