Yesterday I took about and hour and created a FriendFeed to XMPP bridge. The advantage I had was that I was already running a XMPP server and had a working XMPP bot – these existed for my own personal use as well as for cosinity.
Why?
Primarily this is about information discovery. The ability to participate is challenging (see below) as FriendFeed is an aggregator.
As I said in an earlier post – I believe FriendFeed’s Real-Time functionality is missing two important capabilities.
- The ability to filter the feed by keyword(s).
- The ability to get real-time for the public timeline.
While there is nothing I can do about access to the public timeline – I can, via the API, create the ability to filter a feed by keyword(s).
Additionally, XMPP is a clean way to distribute the information – so why not have it published via XMPP?
How?
Since FriendFeed had not released any sample code or a Python/PHP library for real-time yet I had to modify the existing libraries to support real time. This turned out to be trivial – I simply made a new {fetch_RT} function which looks much like the existing {fetch} function. With that complete I could simply use this modified PHP library to grab a user’s real-time stream.
As I mentioned earlier I already had an XMPP server running and had created a bot for other reasons. I simply re-used that bot and the over the network XML API defined by the bot to send the FriendFeed stream out as XMPP messages.
There are several important things to note here about the architecture:
- De-Coupling the FriendFeed real-time processes from the XMPP bot processes is a very efficient model.
- Keeping the XMPP stream bot a “dumb pipe” (i.e., outbound message streaming only) is also very efficient.
- Clicking on the link at the front of the XMPP message opens the FriendFeed “conversation” in a browser.
- Since FriendFeed is an aggregator, participation via XMPP will be very problematic.
- For Example: If the message is a tweet do you want to reply with a tweet – or comment on the tweet in FriendFeed’s conversation?
- The bot would have to track a large number of variables for each message in order to properly distribute your comment/message/etc.
- This overhead would directly affect the ability of the service to scale.
- The scope of the feed (e.g., the apple room or My Home feed or my “the cool people list”) and the keywords for which you’d like an XMPP notification would be configured via a web interface.
- This – again – goes directly to keeping the service as scalable as possible.
- The down side is you can’t change scope or keywords via XMPP messages.
- There is the possibility of creating a separate “ffstreamcontrol” bot which would take these commands. It remains to seen if this is really required or a nice to have.
- Since the stream bot acts as a dumb pipe – it can rapidly scale by running multiple instances with the same UID and a different resource string. You won’t know (or care) which bot you are connected to – they all look the same. If one dies – the FriendFeed real-time processes will be programed to try the next one… and the next one… etc.
- Again – this is a very efficient model.
- Will provide high levels of redundancy
- The bots will all connect to a dedicated XMPP server – the XMPP messages will then be distributed via XMPP federation.
- XMPP federation protocols are very efficient.
The Plan:
My current plan is to build something similar to what you see below. The user will create XMPP streams based on scope (i.e., FriendFeed scope – For Example: the Debates 2008 room) and keyword(s). The user will be allowed multiple XMPP streams. All XMPP stream creation, deletion and modification will be done via a web interface.
The XMPP bot(s) will serve as a dumb pipe for content delivery.
Participation is problematic – and as such will be set aside for now. User’s can still participate by clicking on the link in the XMPP message – this link will open the FriendFeed conversation (here is one for example). This will allow the user to participate, subscribe to the user’s involved in the conversation, etc.
Please feel free to comment – let me know if you think this is headed in the right direction.
NOTE: This was written very quickly based on a prototype made yesterday. Please excuse any less than completely thought through concepts… it is a work in progress.
Brian Roy 
Brian, I suppose that these companies have their hands full trying to funnel data as fast as they can. Those of us who have used the Jabber/gTalk interface for Twitter and identi.ca not only for “realtime” updates but for track really don't want it any other way. Friendfeed to me was a good idea but I struggled with the interface. Their new beta of real time totally clicks with me. I can get it now except for the fact that in my line of work I am dependent on mobile access through my Blackberry. The gTalk interface not only of Twitter in the old days but Dustin's spies has been the way I can interact with my circle of friends and allow stream of keywords into my pocket during events. I am not a member of the iPhone elite, basically because of my like/dependence on IM. As usual, the manufacturers of these devices go short on memory and don't anticipate the savage use of bandwidth but hopefully they will realize they need to double up on the next rev so they can just get by when it comes out.I look forward to seeing how we can shape the feed from friendfeed into both a desktop and mobile experience. I suspect I will create a specific friends list to control bandwidth for my Blackberry, but we're used to stripping down and getting just the things we want and need (and that might not lock up our devices every 30 seconds). I am glad that knowledgeable developers and social media fans are creating these initial workarounds because it's not only a way to get what we want from the experience of these services but to help them see where their services need to go if they care about what their users experience. Twitter has failed miserably with how it treats their users and, I guess they don't care about the small group that saw what the future is going to be, rather be a mall chat room with no attendants and nothing we want to buy.
LikeLike
Aron – Thanks. Great points. Mobility is another key issue. XMPP provides a good solution for mobility – but the standard XMPP client isn't smart enough (IHMO) to deal with a complex set of states and entity relationships – as contained in the FF stream. I do however agree that solving for Social Media Mobility is an important issue.
LikeLike
Brian,This works great! Following aronski's suggestion below, I would suggest that there be a way to limit our xmpp feed to a specific list rather than getting the entire home feed. If someone follows 2 or 3 on FF, it's no big deal. But for me, I need a much smaller subset of the whole feed in real time, particularly when mobile (and I use a blackberry too).Thanks so much for doing this!
LikeLike
I sense that someone might have a chance to create a mobile client for this use…I'm sure it will be a chore, but perhaps it will open a door for a new kind of communication. I think that especially for mobile the idea of a smaller group (controllable?) will be important. Use Friendfeeds feature to filter before it goes to XMPP.
LikeLike
That will be included – you'll be able to specify a Room, A “list”, or your home feed. You can then optionally specify keywords.Is that what you are looking for?
LikeLike
FFStream will implement both scope – that is FF room, list, etc – and keyword filter(s).That way if you only want limited updates for the things/subjects/ people you really think are interesting that is what you'll get.
LikeLike
Yes, that's exactly what I need.
LikeLike
Brian, I suppose that these companies have their hands full trying to funnel data as fast as they can. Those of us who have used the Jabber/gTalk interface for Twitter and identi.ca not only for “realtime” updates but for track really don't want it any other way. Friendfeed to me was a good idea but I struggled with the interface. Their new beta of real time totally clicks with me. I can get it now except for the fact that in my line of work I am dependent on mobile access through my Blackberry. The gTalk interface not only of Twitter in the old days but Dustin's spies has been the way I can interact with my circle of friends and allow stream of keywords into my pocket during events. I am not a member of the iPhone elite, basically because of my like/dependence on IM. As usual, the manufacturers of these devices go short on memory and don't anticipate the savage use of bandwidth but hopefully they will realize they need to double up on the next rev so they can just get by when it comes out.I look forward to seeing how we can shape the feed from friendfeed into both a desktop and mobile experience. I suspect I will create a specific friends list to control bandwidth for my Blackberry, but we're used to stripping down and getting just the things we want and need (and that might not lock up our devices every 30 seconds). I am glad that knowledgeable developers and social media fans are creating these initial workarounds because it's not only a way to get what we want from the experience of these services but to help them see where their services need to go if they care about what their users experience. Twitter has failed miserably with how it treats their users and, I guess they don't care about the small group that saw what the future is going to be, rather be a mall chat room with no attendants and nothing we want to buy.
LikeLike
Aron – Thanks. Great points. Mobility is another key issue. XMPP provides a good solution for mobility – but the standard XMPP client isn't smart enough (IHMO) to deal with a complex set of states and entity relationships – as contained in the FF stream. I do however agree that solving for Social Media Mobility is an important issue.
LikeLike
Brian,This works great! Following aronski's suggestion above, I would suggest that there be a way to limit our xmpp feed to a specific list rather than getting the entire home feed. If someone follows 2 or 3 on FF, it's no big deal. But for me, I need a much smaller subset of the whole feed in real time, particularly when mobile (and I use a blackberry too).Thanks so much for doing this!
LikeLike
That will be included – you'll be able to specify a Room, A “list”, or your home feed. You can then optionally specify keywords.Is that what you are looking for?
LikeLike
Yes, that's exactly what I need.
LikeLike
I sense that someone might have a chance to create a mobile client for this use…I'm sure it will be a chore, but perhaps it will open a door for a new kind of communication. I think that especially for mobile the idea of a smaller group (controllable?) will be important. Use Friendfeeds feature to filter before it goes to XMPP.
LikeLike
FFStream will implement both scope – that is FF room, list, etc – and keyword filter(s).That way if you only want limited updates for the things/subjects/ people you really think are interesting that is what you'll get.
LikeLike
Brian, I suppose that these companies have their hands full trying to funnel data as fast as they can. Those of us who have used the Jabber/gTalk interface for Twitter and identi.ca not only for “realtime” updates but for track really don't want it any other way. Friendfeed to me was a good idea but I struggled with the interface. Their new beta of real time totally clicks with me. I can get it now except for the fact that in my line of work I am dependent on mobile access through my Blackberry. The gTalk interface not only of Twitter in the old days but Dustin's spies has been the way I can interact with my circle of friends and allow stream of keywords into my pocket during events. I am not a member of the iPhone elite, basically because of my like/dependence on IM. As usual, the manufacturers of these devices go short on memory and don't anticipate the savage use of bandwidth but hopefully they will realize they need to double up on the next rev so they can just get by when it comes out.I look forward to seeing how we can shape the feed from friendfeed into both a desktop and mobile experience. I suspect I will create a specific friends list to control bandwidth for my Blackberry, but we're used to stripping down and getting just the things we want and need (and that might not lock up our devices every 30 seconds). I am glad that knowledgeable developers and social media fans are creating these initial workarounds because it's not only a way to get what we want from the experience of these services but to help them see where their services need to go if they care about what their users experience. Twitter has failed miserably with how it treats their users and, I guess they don't care about the small group that saw what the future is going to be, rather be a mall chat room with no attendants and nothing we want to buy.
LikeLike
Aron – Thanks. Great points. Mobility is another key issue. XMPP provides a good solution for mobility – but the standard XMPP client isn't smart enough (IHMO) to deal with a complex set of states and entity relationships – as contained in the FF stream. I do however agree that solving for Social Media Mobility is an important issue.
LikeLike
Brian,This works great! Following aronski's suggestion above, I would suggest that there be a way to limit our xmpp feed to a specific list rather than getting the entire home feed. If someone follows 2 or 3 on FF, it's no big deal. But for me, I need a much smaller subset of the whole feed in real time, particularly when mobile (and I use a blackberry too).Thanks so much for doing this!
LikeLike
I sense that someone might have a chance to create a mobile client for this use…I'm sure it will be a chore, but perhaps it will open a door for a new kind of communication. I think that especially for mobile the idea of a smaller group (controllable?) will be important. Use Friendfeeds feature to filter before it goes to XMPP.
LikeLike
That will be included – you'll be able to specify a Room, A “list”, or your home feed. You can then optionally specify keywords.Is that what you are looking for?
LikeLike
FFStream will implement both scope – that is FF room, list, etc – and keyword filter(s).That way if you only want limited updates for the things/subjects/ people you really think are interesting that is what you'll get.
LikeLike
Yes, that's exactly what I need.
LikeLike