On EverythingTwo based sites, the ChatterBox is a side panel that shows up on every page. It houses quick conversations between users currently online. Postings don't stay more than a few minutes, are not archived, and have very limited length. It is modeled after an InternetRelayChat channel, and non-browser based clients exist to avoid having to hit refresh to get updates.
The JabberProject has often mused doing this through a Java plugin Jabber client that talked to a Jabber server. Jabber.com had a contract to do this, in fact.
BlogChat also provides something like this, only in real-time in the browser via DHTML and JavaScript tricks.
See also WhoIsOnline, BlogChat.