Playing Devil's Advocate here, this question has a kind of...texture to it. It's not a question that immediately strikes me as a programming* question, but more of one of application design and engineering.
The main things that jump out at me are this:
But the code duplication and the entire set-up feels just bad. Same goes for using some kind of flag to tell parseValue() to throw/not-throw exceptions.
I'd like to avoid the code duplication, but the 'flag' option looks equally unappealing. Does anyone know of a better way to do this?
These are engineering opinions as opposed to concrete problems with code. There may be a problem with the engineering, but there may not be a problem with the code as written.
This is where the gray area of Stack Overflow really shows up; it feels like it could be on topic, but the reality is that we're scoped to handle practical programming* problems and not really ones where the developer is metaphorically at the whiteboard. There's more subjectivity in how this should be engineered to avoid the problems the OP has described from an engineering perspective, and we're not programmed equipped to deal with five or six conflicting opinions on the matter.
In that rationale, I could see why one would consider it off-topic.
*: I disambiguate between programmers and engineers in that one only writes code and the other engineers solutions.