The "modified [x] ago" indicator that appears next to a question when you look at unanswered questions, or active questions, indicates any modification that happened to the question itself or to the answers associated with the question. When someone deletes their answer, they become the user who last modified the question. And they remain the user who last modified the question until a new modification happens. This is similar to how someone who posts an answer becomes the user who last modified the question. As you can infer from what I'm saying, "modifying a question" here is more than just editing the text of the question: adding and removing answers, and editing them all count as "modifying the question".
At your rep level you won't see their deleted answer. At >=10K of reputation, you'll be able to see it.