People are obsessed with making their email “broadcast”. Push email is a big deal… but sadly it means you have to lock yourself into a single email service from a single provider. You want near real-time push email… Blackberry or Exchange… or you can pretty much forget about it.

So, why can’t we have an open alternative?

There are a multitude of ways to get real time notifications of new emails, and there is no reason for that mechanism to be an integral part of the email solution itself. Given that I set out to make a push email solution for my Google Apps accounts that would effectively notify on my iPhone or any other device.

As it turns out the solution is actually very simple. I created a simple server process that checks my inbox every few seconds and sends me an Instant Message (IM) when I have a new email. I also incorporated sender and subject filters – so you can get notified only for the emails you want.

I could have made it send me a Twitter DM, or an SMS, or plugged it in to PubSubHubBub, or RSS Cloud…

The point is, the actual mechanism for the notification isn’t really all that important…

More after the jump…

How does it work?

EmailToIM is a PHP daemon that will run on any system that has PHP installed. It periodically checks for new messages on your email server (IMAP only) and sends you an IM if it finds one.

For notifications on my iPhone I use BeeJive IM with Apple’s push notifications turned on.

Where can I get it?

I’ve set up a Google Code project for it here.

It is open source (GPL v3).

If you have any questions let me know.

2008 was supposed to be the year we began to see real-time web take shape. And while Twitter and FriendFeed have begun to show us some bits of what a real-time web might look like mostly it has been a year of discontent.

While I hate year end/beginning “predictions” (what am I Nostradamus?) I’m predicting 2009 won’t be much better. Why? Well that is the interesting part.

I’ll let you in on a secret (shhh, this is just between you and me). Real-time services on the “web” are nothing new. We have a pretty good idea how they work (and don’t work). We know what the challenges are – and to a large degree how to architect/engineer the solutions. The problem is we aren’t leveraging the work that has already been done.

More after the jump…

For years HTTP, SMTP and POP were the web. Those three combined with the underlying transport layer (TCP/IP) to form the “killer application” web. Web 1.0 and Web 2.0 both were built on this framework and exposed both their incredible power and their weaknesses. We’ve developed a multitude of higher order solutions which leverage the power and compensate for the weaknesses – RSS and AJAX respectively provide an example of each.

What does this have to do with real-time? The foundation upon which the web was built isn’t entirely appropriate for synchronous real-time communication. Which is not to say we can not overcome those limitations by re-visiting or inventing parts of that foundation. We know this because we’ve solved one of the most intensive real-time synchronous problems – two way voice communication over internet protocol (VoIP).

VoIP works because it threw out the foundation from IP up. Those of us involved/following it’s development from the early 1990′s on quickly realized that we couldn’t rely on TCP or any of the available application protocols (such as HTTP) because – while they incorporated important solutions for the transmissions of bulk data in response to specific requests – they were not appropriate for a synchronous exchange of data based on a standing agreement to exchange data (essentially this is what a VoIP phone call is). Other issues included efficient distribution of messages to many subscribers (multicasting), the delivery of many small chunks of data in a time sensitive manner, and the implicit understanding that every bit of data wasn’t critical to overall service quality (statelessness).

Because the door was thrown wide open to re-think the foundation of the web we saw proliferation of methods. Each company, group, and standards body developed and promoted a standard. Each screamed loudly that their standard was THE standard – and nothing happened. This was a 15 year period I refer to as the Protocol Wars. Thankfully, the protocol wars have ended, SIP won – and we have an intelligible way to execute on VoIP.

I believe that the problems of the real-time web are the problems that were faced and solved by VoIP. What I don’t understand is why the real-time web community isn’t working with the VoIP community.

In order to implement the real-time web (as opposed to talking about the real-time web) we will need to address multicasting, time-sensitivity, and communication negotiation. All problems already addressed by VoIP.

In order to implement the real-time web we will need to ensure every service/provider/network speaks the same underlying language (protocol) and implements that language in the same way (standard).

So, given that I believe much of the solution for the real-time web exists (which is different from saying it is complete) why do I believe 2009 won’t be much better? For two reasons:

1. Simply because it appears to me that the real-time community is already beginning to engage in a protocol war.
2. There is little or no interest on either side (VoIP/Real-Time web) in engaging to leverage the work already done.

So, here is my challenge to my friends in both the VoIP and Real-Time Web communities. Get in a room and figure it out. Extend SIP, fix XMPP, or just come up with something new – but don’t ignore one another. Please don’t accept another protocol war for the real-time web – because by the time you finally stop talking about the technology you may find that people have completely lost interest in creating solutions that leverage it.

But most of all, don’t make the mistakes we made with VoIP by focusing on the commodity (who will decide what the servers run) instead of getting on with creating the services you users really want.