The myth of “everything” – Responses to my view of Track

My post yesterday – specifically regarding the Great Track Debate – received several responses on Twitter, identi.ca and friendfeed. Many of these – really all of them were positive – but the debate is far from settled.

I learned something very important from one discussion in particular – that there is a myth that permeates the conversation. The myth of “everything”.

This myth is the adherence to two ideas:

  1. That somehow the “fire-hose” (as discussed here) represents the complete information about any given topic – that somehow the “fire-hose” is everything.
    1. At the root of it, this is the idea that everything is attainable.
  2. That – in order to monetize track – having everything is essential. For example, if you are trying to manage your brand you need every reference to it in real-time.
    1. At the root of it, this is the idea that everything is required and valuable.

The reason I refer to these two ideas collectively as the “myth of everything” is because when they are clearly stated and examined they are immediately recognizable as inconsistent with reality.

So let’s take the two ideas one at a time and examine how tenuous their attachment to reality is.

First, that the “fire-hose” represents everything about any given topic. The fire-hose is the sum total of what is said on a given service. In order for that to be everything that service would need to be participated in by everyone. That is a hard enough hurdle to overcome, but there is more – not only would it need to be everyone, it would need to be the only method by which they communicate their thoughts, ideas and feelings.

Even if you were able to combine the fire-hose of every service available – every social networking site, every blog, every micro-blog, every IM service, every news site – you would still be far short of everything.

So the idea that track only becomes valuable when it can capture “everything” is a myth. Everything is unattainable.

Second, that having everything is essential. Let’s suppose we were to allow that somehow everything was achievable. Even if it were would it be required? Would it provide value commensurate with the effort required to collect it?

Let’s evaluate this in terms of brand management. The assumption which underlies this is that it is required to respond in real time to every post which is misleading, false, or damaging. This assumption is flawed – the reality is that it is required to respond to a statistically relevant sample of those posts. You aren’t trying to refute every post – you are trying to move (or keep from moving) the average (or perhaps mean) opinion.

If any company were forced to staff enough positions to actively monitor and respond to every post made about them – they would immediately cease to be profitable. It isn’t scalable, and more importantly it isn’t required.

This holds true equally for politics.

So the idea that track delivers value because having access to everything is required and the primary driver of value is a myth. Everything is neither required nor valuable in real terms.

What the track community should be focused on – again IMHO – is not the fire-hose and the attendant myth of everything, but creating systems which can attain enough trackable scope to provide a statistically relevant sample of the posts in the social media universe.

I understand that, emotionally, it feels good to tap into some perceived “everything” and refute any and all posts that you think are misleading, false, biased, or offensive. But this isn’t about what feels good – at the end of the day it will be about what is effective. And to be effective everything is neither required nor valuable.

These two conclusions – that everything is unattainable and that – even if attained – is neither required nor valuable should allow us to dispense with the “myth of everything” and return to the point of track:

  1. Real-Time Information Discovery
  2. Real-Time Participation

The evolution of FFStream and the Great Track Debate

ff-filtered-twitter.pngAs those of you who follow me on twitter or friendfeed know FFStream – which I began discussing in this post – has evolved into FF-Filtered. The changes are not dramatic, but they are significant.

FF-Filtered is now focused on providing “your friendfeed – filtered” – and as that implies what it does is filter your friendfeed (home feed to be specific) by a list of keywords. If these keywords match the post title, comment, or user you receive the update in real time – in the browser or via IM, including GTalk, Jabber, AIM and Yahoo.

It isn’t track – as I’ve been repeatedly and vehemently told by the “community” over the last 5 days – more on that later.

Additionally – for the mobility set – we’ve added like, comment, post and filter updates via a mobile web page. If you click on the link in the IM from a mobile platform you get the following mobile web page:

IMG_0001.PNG

I’ve got a few more tricks up my sleeve – so look for more changes this week including a name change.

Now for the second half of the post… I’m going to talk about Track again… so sharpen your knives (or tongues) and get ready to revel in your abject disdain for my refusal to “go along” or shut up.

Let me say one thing first – if you want to attack my positions and opinions go for it. If you are here to attack my motives or me personally – GO AWAY NOW.

The last 5 days has been very illuminating for me – both in terms of the fervor of the “track community” and in terms of their point of view – which at times verges on dogma.

Let me attempt to “play back” what I’ve heard and then explain – as clearly as possible given my limited skills – my point of view.

Track – by the definition of the “track community” as led today by Steve Gillmor of Gillmor Gang/News Gang Live, is defined by the “fire-hose”. This fire-hose is the complete unabridged stream of posts occurring on any social media site. In the case of Twitter, this is the entire public timeline published in real time.

This is an important distinction – because it states that “track” can not be achieved without the “fire-hose”. More on this latter.

The second component of “track” is the ability to keyword filter the real time stream and deliver the filtered content in real (or near real) time.

The third component of “track” is the ability to insert posts into the public timeline from the same user interface you are viewing the stream in.

Those four things collectively comprise the “holy grail” of track.

I’m sure you will all let me know exactly how wrong I am… but I think those four capture the broad brushstrokes. Ok?

Before I attempt to explain my point of view – let me clarify one point. Regardless of my agreement or disagreement with the track community on any given point, there is one thing we vehemently agree on:

There is massive value in the ability to discover and participate in the social media stream in real (or near) real time. Our objective where that is concerned is the same.

When I consider track – I consider it in terms of the problems it attempts to solve. To me, track is an attempt to solve 2 very important problems:

  1. Real-Time Information Discovery
  2. Real-Time Participation

Any solution which solves those two problems would – by my definition – fall within the scope of a “track” service.

Now let me explain why (take a deep breath… you can throw something at me later). Where I differ with the “track community” on this issue is on the scope of the track-able data not what happens after the “track service” receives it. As importantly I fundamentally agree that the wider the scope of the data being tracked the more effective the track solution will be.

But, consider this – not every user wants to track the entire social media universe. To the contrary – most IMHO simply want to track their friends, family, co-workers, brand, market makers, influencers, power users, etc.

For those users a limited scope is a good thing. Beyond consideration of the scope of the data being tracked this service solves the exact same problems.

  1. Real-Time Information Discovery
  2. Real-Time Participation

So apply the duck test. It walks like a duck… it quacks like a duck… why isn’t it a duck?

It is my opinion – and you can feel free to take issue with it – that the track communities’ obsession with the “fire-hose” has actually retarded the growth of alternative track services. The obsession with scope has prevented the creation of useful (if limited by their limited trackable data) solutions under the banner of track – and that is a shame.

Every developer that seeks to solve the two problems should be embraced, encouraged and supported.

The real battle here is one of leverage. And the way to get the social media services to both open up their data and participate in the creation of a standard for doing so is to create a win-win. I believe track services that are useful and solve real problems (e.g., real-time brand monitoring) can and will provide the leverage that causes the change the community has been seeking.

If Twitter wants to pretend they ARE the social media universe – let them. It is abundantly clear from the success of friendfeed that no single service is or will be the social media universe – any service that ignores this will fail.

When compelling and broadly adopted services exist, which demand real-time un-scoped access to multiple underlying services, the individual services will have no choice but to “open their kimono” or face massive user defection.

So stop complaining about the lack of a “fire-hose” and figure out what those services are, who needs them the most, and how to drive that value to as many users as fast as possible. If you do that – you’ll get what you want… not today, not even tomorrow… but relatively soon.

I had intended to discuss standards, what I believe the high level components of an open track environment might look like, and why friendfeed is in the best position to lead standard development… but this has already gotten too long. I’ll set those subjects aside for another day.

If you’ve disagreed with everything else I’ve said – please remember – I share your goal. I’m not saying the outcome you seek isn’t valuable – I am just proposing a different course of action. I hope I’ve done so respectfully and without denigrating anyone or their point of view.

XMPP, Track and the Social Media Command Line

Last Friday I created a FriendFeed Real-Time bridge to XMPP. Over the weekend I’ve had several people receiving their home feed in real time over XMPP (GTalk to be specific). I also wrote a post about what I had done and what I intend to do with this going forward.

One thing I heard from the initial testers was the desire to “make it work like Track”. Consequently – I’ve spent quite a few mental cycles this weekend thinking about Track (as deployed by Twitter) and how that might work for FriendFeed Streams.

There are three basic components of Track

  1. The ability to track keywords (topics) and receive any tweets containing those keywords.
  2. The ability to post a new tweet.
  3. All interaction happens via IM (XMPP to be specific).

As I thought about these three basic components – and how they might be implemented for FriendFeed Streams I quickly realized that Twitter and FriendFeed are very different – and those differences have a huge impact on “Track” for FriendFeed Streams.

Let’s first consider the nature of Twitter.

Twitter is a stream of stateless tweets. Each tweet is an independent entity – bearing no relation to any other tweet in the timeline. Tweets only have relationships to the user that sent the tweet and, optionally, the user the tweet was directed at – via a @ reply. However, even those @ replies are simply stateless tweets – existing as an independent entity in the timeline.

It is that fundamental nature of Twitter that makes Track interesting – and to a certain extent possible. Since all tweets exist as stateless peer entities in a universal timeline – Twitter essentially creates a stateless stream of tweets. Track works by scanning and filtering this stream and injecting tweets into this stream.

Now let’s consider the nature of FriendFeed.

FriendFeed is a social media aggregator. It contains posts created organically (via the create post button in the FriendFeed interface) or via any number of aggregation channels (blogs, RSS Feeds, Twitter, Disqus, Flickr, etc). Each of these posts exist as state-full peer entities on the timeline.

FriendFeed posts are state-full because they exist as containers. They contain a separate groups of entities – Likes and Comments. Likes and Comments change change the state of the Post – and by doing so change the posts place in the timeline. For Example – If I comment on a 4 day old post it will move to the top of the timeline.

The Post as a container creates a conversation. This concept – the conversation – is totally foreign to Twitter.

FriendFeed, then, creates a state-full stream of conversations. Conversations appear in the stream one or more times – each time they are created or added to via a Like or Comment.

Now that we understand how Twitter and FriendFeed are similar and differ we can re-visit Track.

  1. The ability to track keywords (topics) and receive any tweets containing those keywords.
  2. The ability to post a new tweet.
  3. All interaction happens via IM (XMPP to be specific).

What is clear is that component #1 above is still viable and makes sense. It was the fundamental reason I chose to build FFStream. It still solves the problem of information discovery.

What poses a significant challenge is #2. I refer to this as the problem of participation. This is due to the state-fullness of the FriendFeed Post (or conversation). Where FriendFeed is concerned participation is a relative exercise – you can post a new conversation OR you can participate in an existing conversation. It is the relative nature of participation (relative to an existing conversation) that makes participation via IM so problematic.

This schism is completely analogous to the schism between the command line and the graphical user interface. As a matter of fact, an IM/XMPP interface for Social Media streams is simply a Social Media Command Line. You enter commands – often obscure, opaque and archaic commands – and thereby control the behavior of the application. And this model works… until the interaction model becomes so complex that a command line interface becomes impractical.

When the goal is to filter a stateless stream of data (ala grep) and insert data into that stateless stream – a command line works. However, when the stream becomes state-full – and contains entity relationships (i.e., conversations) filtering the stream is sensible – but the complexity of the required command line interface to inject data into the right entity in the stream makes the proposition a fools errand.

In short – FFStream is an ideal information discovery mechanism – far more powerful than Track because the stream is an aggregation of content and because the stream is a stream of conversations. It is not, however, a usable interface (i.e., command line) for participation in Social Media.

So the goal of FFStream will be to enable information discovery and encourage participation by providing a reference to the conversation. That reference will open a browser window with an appropriate interface (i.e., graphical user interface) for the participation in the conversation.

I know some will find this (to put it mildly) lacking. I know some will declare that I don’t get it. But there is a reason people vastly prefer the ease of use provided by a graphical user interface over the command line. If you are one of those who believe I’ve missed the point – please consider the command line vs. graphical user interface schism and how it relates to this conversation before declaring my ignorance.

54 Hours To Build A Company: A Look At Startup Weekend Phoenix

TechCrunch coverage of Phoenix Startup Weekend… Yeah – I was there working on Reserve Chute!!!

Twitrratr gets a nice bump… nice job guys.

Go Phoenix!

It’s been some time since we last covered Startup Weekend, a series of events that bring a roomful of developers and entrepreneurs together to develop new startups in only 54 hours. When the program originally launched last year, each weekend was geared towards building a single application, of which every participating member was a cofounder. Since then the format has changed – multiple companies are created at each event, and they don’t have to incorporate at the end of the weekend. Here’s a handful of the companies founded at last weekend’s event, which was held in Phoenix.

ReserveChute

With so much of our essential data making its way to the cloud, it’s becoming increasingly difficult to quickly make a local backup. Reserve Chute, an open source web app that will be available for use in the next few weeks, will automatically make backups of popular cloud based services including Gmail, Twitter, and Basecamp.

It’s unlikely that any of these services will be going out of business in the foreseeable future, and they almost certainly have redundant backup systems in place. But the prospect of having my entire photo collection or Email history wiped out is unnerving – having an easy way to back up these services would definitely give users some peace of mind, even if they never had to use it.

My Shelter Helper

My Shelter Helper is web page builder aimed to help animal shelters establish a presence on the web. Shelters can logon and after entering some basic information like their address and telephone number will be presented with a functional and good looking website. It’s a great idea, and I love the tagline: “Helping save animal’s lives.”

For now the service is generating some pretty barebones sites, but will introduce support for donations so that shelters can easily collect from benevolent animal lovers worldwide. I hope the team keeps working on this – anything that helps animals is a good thing, and plenty of people (like my mother, for example) would love to donate to their local animal shelter along with the national organizations.

Twittrratr

Awful domain name aside, the Twittrratr team has actually built a pretty cool Twitter site. After entering any keyword, Twittrratr will find related tweets and attempt to figure out if the subject is being spoke about in a positive or negative light. It’s a good idea but unfortunately it doesn’t work very well – oftentimes words that Twittrratr associates with a negative tweet aren’t being used to describe the keyword that was searched for. The team acknowledges that the system isn’t perfect and is open to suggestions (it’s still pretty impressive for 54 hours from conception to launch).

[From 54 Hours To Build A Company: A Look At Startup Weekend Phoenix]

What did you do last weekend?

I hope you had fun. I was at Phoenix Startup Weekend working with a team of smart, energized people to create a new company in a single weekend.

The result – Reserve Chute – is sheer genius.

This one time, at Phoenix Startup Weekend (http://phoenix.startupweekend.com) actually held in Chandler, AZ, a band of brothers, and one sister, came together to create a new company called Reserve Chute. With it’s main offering a personalized data backup service that permits users to grab their data from online sources like GMail and others and save them to a storage device of their choice. This is one cool and innovative service that should be on your “Must Have” list. Your data on your terms for when Web 2.0 becomes Web 2 point Oh no! What more could you ask for. For more information, visit http://www.reservechute.com and sign-up for this unique and powerful service offering.

If you use online applications to run your business you need to check Reserve Chute out. It might be the best thing you ever do for your business.

The following photos are courtesy of Adam Nollmeyer – Thanks Adam. You can find Adam’s work here.

UPDATE – The photos below are actually courtesy of ccl1111 – you can find his photos here. Adam’s photos are still great… be sure to check them out as well.

200810201001.jpg 200810201003.jpg

200810201003.jpg

OS X-installing EFi-X now shipping for $155

Fascinating… but not sure I’d be an early adopter on this.

What is really interesting is how many companies are taking a run a Apple’s refusal to let OSX load on non-apple hardware. It appears there are several lawyers that believe there is a valid defense available. We will just have to wait and see.

Filed under: ,

After a few false starts, the OS X-installing EFi-X dongle is finally shipping to consumers. Currently, two versions are up for grabs: the USB V1 for the average joe / jane and the USB V2 Developers Unit for, well, developers. In short, plugging this gem into your PC will enable select systems to install OS X, but we’d take a hard look at the fine print (and certified systems) before blindly plunking down $155 and hoping for the best.

[Via MacRumors, thanks Joseph]

[From OS X-installing EFi-X now shipping for $155]