Big Data – Spike Volume

One aspect of Big Data is the arrival pattern of the data being stored/analyzed.

Yesterdays Rangers win in the ALCS is a excellent example of this. At justSignal we monitor all of the MLB teams and when the Rangers won we experienced a phenomenal spike in data arrival. Right now – which is fairly representative of normal volumes – we are receiving 7.87 Tweets per Second from Twitter. Last night we peaked out at 43.5 Tweets per Second.

Here is what that looks like graphically:

Yankees Loss ALCS Tweets

Texas Rangers ALCS Win Tweets

Managing that kind of data arrival while continuing to generate meta data dimensions (transformations) and deliver data to clients in real time is no small task.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s