The problem:
Say you have a chat about a problem but you have to go offline for the night or something and you want to continue later. I find it really hard to find the chat again and I probably will open a new chat with the same user, even though that is very difficult (I usually go back to the question from which I started the chat).
Why a better chat system:
I think the chat system has potential and I think it would be used more by users if there was an easy way to access it. I think people feel more comfortable when they can just explain their question to one or more people that are online instead of having to wait until somebody answers the question on StackOverflow and having to fear for reputation loss (I guess that more experienced users of SO will still ask questions the 'normal' way).
Some features that may be nifty:
A button next to the notifications and reputation change buttons we have now; this button can open a list of users that you have recently chatted with and give you access to the chat so you can continue chatting.
An invite button to invite users to a chat/chatroom, or to start a chat with this user. When you invite someone to a chat between two users, the chat can turn into a chatroom or a private group that you can invite people to (like the Facebook chat).
A few badges related to chatting:
first chat
orfirst chatroom
, which obviously correspond to creating a chat with another user and creating a chatroom for people to chat in.
Downsides:
Now we need to ask ourselves: "would improving the chat system also improve the whole site, or would it discourage asking a question on StackOverflow and encourage using chat instead?"
I don't think that there will be fewer questions, because people stick to a certain pattern. When you are used to doing something one way, it will be very hard for you to change to another way. That means that if you are used to asking questions you will probably not switch to a chat.