Comments on: Twitter, @replies and Multicasting

By: Twitted by 2ndtymekaty

Twitted by 2ndtymekaty — Thu, 14 May 2009 13:36:08 +0000

[...] This post was Twitted by 2ndtymekaty – Real-url.org [...]

By: briantroy

briantroy — Thu, 14 May 2009 04:43:36 +0000

I understand. But if Twitter is going to be a real-time service it will have to do EXACTLY what you need to do for video (or what I've done for VoIP conferenece calls).
That is one of my fundamental annoyances with Twitter and the "Real-Time" web community - they want real-time but only if it works within the constraints of the publish/subscribe architectures they already use. We need to re-think that.
XMPP is already building in Multicast (http://xmpp.org/extensions/xep-0033.html) SIP/SIMPLE has had it for years.
I'm not suggesting that part of Twitter won't continue to be publish/subscribe. I'm suggesting that if they really want to be real-time publish/subscribe won't get that done.

By: Brian

Brian — Thu, 14 May 2009 04:24:04 +0000

I think the part I'm stuck on is that yes, Twitter is a near-real-time system, but with the exception of the SMS gateway, Twitter is not a push mechanism.

My background with multicast is from a live video perspective. The theory is great: I have one server and thousands of clients waiting to be pushed the same data at the same instant.

With Twitter, my Twitter update doesn't get pushed to X people simultaneously. It gets saved into a database and waits until clients come by at their convenience and request/pull the data. Luckily, Twitter updates are immutable and can be indexed, cached, and optimized six ways from Sunday.

By: briantroy

briantroy — Thu, 14 May 2009 04:08:35 +0000

No, I never had it on. And I didn't notice a difference, I was commenting on:
1) The idea that the change would kill Follow Fridays
2) The fact that (which isn't changed by my misunderstanding the issue) multicast is vital to any large scale real-time system.

The same situation (regarding multicast) existed with VoIP. Multicast isn't about replicating the object in the DB - as a matter of fact the "object" should only exist once in the DB. It is about propagating the "object" to 1 or many endpoints/clients efficiently.

Think of it this way, the tweet only exists once (like a packet of real-time video) - but it has to get to a multitude of "viewers". We know the efficient way to do that is multicast. Iterating through each "viewer" and sending the same packet over and over again just won't work at scale.
The content of the "packet" isn't relevant - could be status, tweet, video, audio, etc. What is relevant is that you have to send it to many endpoints in the most efficient way possible.

I'll update the post to indicate I misunderstood the @ reply feature... My Bad.

By: briantroy

briantroy — Thu, 14 May 2009 04:01:18 +0000

Entirely possible. It does not seem CLEAR in any of the posts I've ready. The previous see all @ replies setting is both documented poorly and explained multiple ways by everyone. From the post Luigi cites:
What this means is that whenever you open a tweet with @, Twitter reads the submission as a reply. This:

@Sheamus is a great guy!

Is a very different message to this:

I tell you who's a great guy - @Sheamus!

Twitter absorbs these messages in different ways. The first, the reply, will be seen by the following people:

1. @Sheamus
2. Anybody who is following @Sheamus and the sender of the message
3. Anybody who, previously, had Twitter set to see all replies

That actually supports my supposition.

But hey.. If I'm wrong I'm wrong... thanks for pointing it out.

By: Brian

Brian — Thu, 14 May 2009 03:58:57 +0000

You never had “all @ replies” on? If you noticed a difference with Twitter's settings update, then you must have. If you had “@ replies to the people I'm following” selected, your stream should be the same now as it was before.

I have never seen nor heard about people seeing @ replies in their streams FROM people they don't follow. Your home screen (& statuses/friends_timeline API calls) should only show tweets from people you follow. Depending on the setting you had selected, you would either see 100% of tweets from those you follow (“all @ replies”), most tweets excluding those from your friends TO people outside of your network (“@ replies to the people I'm following”), or only non-conversational tweets (“no @ replies”).

Also, it would seem strange for Twitter to implement the multicast network protocol to replicate objects in a relational database model… Are you implying each user should have an independent db or table for the tweets of those they follow?

By: dalas

dalas — Thu, 14 May 2009 03:51:48 +0000

Luigi is right. You misunderstood the issue.

By: briantroy

briantroy — Thu, 14 May 2009 02:48:40 +0000

Not entirely sure that is the case (per the post you cited and others):
'Williams explained how this was now an important part of the network and went on to mention that 98 per cent of Twitter users left their replies on the ‘default’ setting – that is, they only saw @ replies to the people they are following.”

That being said… I may have this wrong in terms of who sees what based on an @ reply… but the issue of multicast in highly scalable real-time systems remains the same.

If I do have this wrong I'll update the post (once I can be sure). Part of my confusion is I never had All @ Replies turned on.

By: luigimontanez

luigimontanez — Thu, 14 May 2009 02:29:12 +0000

This isn't what the problem was at all. If some person I don't follow tweets about someone I follow, I'm not going to see that. If I follow @scobleizer and you (who I don't follow) tweet about @scobleizer, I would never see your tweet, no matter what my setting was. Twitter never did that, and probably never will.

The "show me all replies" checkbox was for viewing @replies from people I do follow directed at people I don't follow. So it's trimming @replies from people you already follow.

http://twittercism.com/all-replies/

By: briantroy

briantroy — Thu, 14 May 2009 00:43:36 +0000