Is this an issue that we just don't want to give rep to someone who improves the quality of the site, but only by a little?
Not really. Maybe just a touch, but it's pretty far down on the list of problems with too minor edits. Here are some much more important issues:
It consumes a lot of reviewer time. Time that could be better spent doing other things, like fixing the spelling grammar of other posts. (Imagine how many of these minor edits could be done in the time the 3-5 reviewers spend reviewing that edit.)
It locks the post from editing until the post is reviewed, inhibiting the ability of other users to make more substantial edits.
It draws reviewer time away from other suggested edits, causing them to lock up the posts for longer, and prevent other more substantial edits from being applied sooner.
It bumps the post on the front page, drawing attention to it and consuming the time of readers without much benefit.
The biggest problem with users getting rep with very minor edits is that it encourages them to make more of them, thus causing the above problems to happen over and over. When the users don't get their edits approved, they either learn not to make these edits, thus not causing those problems, or they get edit banned, thus being unable to cause those problems.
If the above is somewhat close to the reason, then is it OK if I correct grammar, spelling, formatting etc? Given that I no longer get rep for improving posts - does that make it OK to try and add value in that way?
It reduces the harm. 1, 2, and 3 don't apply. #4 does, but that's still pretty significantly reduced the net harm of the minor edits. So if you want to fix a few small things when you come across them naturally, that's generally fine, but when you go around actively searching for, say, a single common typo to fix, then you flood the homepage, and that gets disruptive. In short, fix it when you see it, don't go looking for it.
seems to be that "we" prefer incorrect grammar and wrong spelling, to correct grammar, and good spelling. But that doesn't make sense, it sounds a bit cynical, so I must be missing an important point.
You're right, it doesn't make sense, and that's not what's going on here. We don't like incorrect grammar and wrong spelling, but the problems causedfixed by those minor edits aren't as substantial as the problems caused by the person fixing it. When the medicine hurts more than the disease, you don't take it. That doesn't mean that it's good to be sick.