Why is the Real-Time Web Community Shooting Itself in the Foot?

2008 was supposed to be the year we began to see real-time web take shape. And while Twitter and FriendFeed have begun to show us some bits of what a real-time web might look like mostly it has been a year of discontent.

While I hate year end/beginning “predictions” (what am I Nostradamus?) I’m predicting 2009 won’t be much better. Why? Well that is the interesting part.

200901081030.jpg

I’ll let you in on a secret (shhh, this is just between you and me). Real-time services on the “web” are nothing new. We have a pretty good idea how they work (and don’t work). We know what the challenges are – and to a large degree how to architect/engineer the solutions. The problem is we aren’t leveraging the work that has already been done.

More after the jump…

For years HTTP, SMTP and POP were the web. Those three combined with the underlying transport layer (TCP/IP) to form the “killer application” web. Web 1.0 and Web 2.0 both were built on this framework and exposed both their incredible power and their weaknesses. We’ve developed a multitude of higher order solutions which leverage the power and compensate for the weaknesses – RSS and AJAX respectively provide an example of each.

What does this have to do with real-time? The foundation upon which the web was built isn’t entirely appropriate for synchronous real-time communication. Which is not to say we can not overcome those limitations by re-visiting or inventing parts of that foundation. We know this because we’ve solved one of the most intensive real-time synchronous problems – two way voice communication over internet protocol (VoIP).

200901081031.jpg

VoIP works because it threw out the foundation from IP up. Those of us involved/following it’s development from the early 1990’s on quickly realized that we couldn’t rely on TCP or any of the available application protocols (such as HTTP) because – while they incorporated important solutions for the transmissions of bulk data in response to specific requests – they were not appropriate for a synchronous exchange of data based on a standing agreement to exchange data (essentially this is what a VoIP phone call is). Other issues included efficient distribution of messages to many subscribers (multicasting), the delivery of many small chunks of data in a time sensitive manner, and the implicit understanding that every bit of data wasn’t critical to overall service quality (statelessness).

Because the door was thrown wide open to re-think the foundation of the web we saw proliferation of methods. Each company, group, and standards body developed and promoted a standard. Each screamed loudly that their standard was THE standard – and nothing happened. This was a 15 year period I refer to as the Protocol Wars. Thankfully, the protocol wars have ended, SIP won – and we have an intelligible way to execute on VoIP.

I believe that the problems of the real-time web are the problems that were faced and solved by VoIP. What I don’t understand is why the real-time web community isn’t working with the VoIP community.

In order to implement the real-time web (as opposed to talking about the real-time web) we will need to address multicasting, time-sensitivity, and communication negotiation. All problems already addressed by VoIP.

In order to implement the real-time web we will need to ensure every service/provider/network speaks the same underlying language (protocol) and implements that language in the same way (standard).

So, given that I believe much of the solution for the real-time web exists (which is different from saying it is complete) why do I believe 2009 won’t be much better? For two reasons:

  1. Simply because it appears to me that the real-time community is already beginning to engage in a protocol war.
  2. There is little or no interest on either side (VoIP/Real-Time web) in engaging to leverage the work already done.

So, here is my challenge to my friends in both the VoIP and Real-Time Web communities. Get in a room and figure it out. Extend SIP, fix XMPP, or just come up with something new – but don’t ignore one another. Please don’t accept another protocol war for the real-time web – because by the time you finally stop talking about the technology you may find that people have completely lost interest in creating solutions that leverage it.

But most of all, don’t make the mistakes we made with VoIP by focusing on the commodity (who will decide what the servers run) instead of getting on with creating the services you users really want.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s