"Go learn!" or especially "Go back to university, learn about algorithms and data structures before you try to tackle a problem like this" is not a helpful response (the latter sounds very elitist, as not everyone has the opportunity to attend university...).
If you're going to the effort of replying anyway, you could simply have said something like:
It's "big O" notation - used to describe the limiting behaviour of a
function.
And maybe include a link to the topic in case the want to read / learn more:
http://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common_functions
or
SO: What does O(log n) mean exactly?
I think the original SO OP comment is fair, you've included a notation they weren't familiar with (and hadn't used in the asking of their question):
What is O(log n)?
Sure, they could Google it, but there's no guarantee they see a good, short (single-line) summary of what something is. Don't write them a complete tutorial, but pointing someone in the right direction will help the OP and others.
Not everyone knows Big O (or regex or set notation etc. etc.) enough to know which search engine results are relevant. Remember, people come to programming from a variety of routes, not just an academic (mathematical or computer science) background, so not everyone will have the same underpinning knowledge.
If this is the only time the OP has encountered or needed to understand a specific notation, maybe 3 years at university isn't a good use of their time... If they continue in this area, they may well work out that more education may be the way to go - but that's their decision.
I'd treat these (side) questions the same for any topic. Say someone that didn't know Regular Expressions, for example - but an answer contained one. It's easy to assume everyone knows what they are, but not everyone does. Let them know what you've used, then point them in a constructive direction to start learning about it. - don't insult their intelligence or education level. As @Reto Koradi touches upon, more people than just the OP will view the question and there will be a number of them that are also unfamiliar with Big O.