Should I have posted my improvement as a separate answer instead?
Yes. If you have come up with a different solution to the problem than another answerer, that you think is better, you should be posting it as your own answer, not editing someone else's answer to use your solution.
I cannot accept the answerer's answer as-is (because it does not fully answer my question) and I cannot accept my own answer, because this won't give the answerer due credit
That's your choice to make. If you want to accept an incomplete answer because you were able to use the information to come up with what you think is a better solution, you can. If you want to accept what you think is the best solution (in this case, the answer you came up with), you can do that to. If you can't decide and don't want to accept any answer, that's also fine.
If you think that an answer was useful, but don't feel that it was the best answer to the question, you can upvote it instead of accepting it.