In February 1948, Communist leader Klement Gottwald stepped out on the balcony of a Baroque palace in Prague to address hundreds of thousands of his fellow citizens packed into Old Town Square. It was a crucial moment in Czech history – a fateful moment of the kind that occurs once or twice in a millennium.

Gottwald was flanked by his comrades, with Clementis standing next to him. There were snow flurries, it was cold, and Gottwald was bareheaded. The solicitous Clementis took off his own fur cap and set it on Gottwald’s head.

The Party propaganda section put out hundreds of thousands of copies of a photograph of that balcony with Gottwald, a fur cap on his head and comrades at his side, speaking to the nation. On that balcony the history of Communist Czechoslovakia was born. Every child knew the photograph from posters, schoolbooks, and museums.

Four years later Clementis was charged with treason and hanged. The propaganda section immediately airbrushed him out of history, and obviously, out of all the photographs as well. Ever since, Gottwald has stood on that balcony alone. Where Clementis once stood, there is only bare palace wall. All that remains of Clementis is the cap on Gottwald’s head.

Book of Laughter and Forgetting, Milan Kundera

The rise of social distribution networks

Over the past year there has been a rapid shift in social distribution online.    I believe this evolution represents an important change in how people find and use things online. At betaworks I am seeing some of our companies get 15-20% of daily traffic via social distribution — and the percentage is growing.    This post outlines some of the aspects of this shift that I think are most interesting.   The post itself is somewhat of a collage of media and thinking.

Distribution is one of the oldest parts of the media business.    Content is assumed to be king so long as you control the distribution flow to that content. From newspapers to NewsCorp companies have understand this model well.   Yet this model has never suited the Internet very well.     From the closed network ISP’s to Netcenter.   Pathfinder to Active desktop, Excite Lycos, Pointcast to the Network computer.   From attempts to differentially price bits to preset bookmarks on your browser — these are all attempts at gate keeping attention and navigation online.    Yet the relative flatness of the internet and its hyperlinked structure has offered people the ability to route around these toll gates.   Rather than client software or access the nexus of distribution became search.    Today there seems to be a new distribution model that is emerging.   One that is based on people’s ability to publically syndicate and distribute messages — aka content — in an open manner.    This has been a part of the internet since day one — yet now its emerging in a different form — its not pages, its streams, its social and so its syndication.    The tools serve to produce, consume, amplify and filter the stream.     In the spirit of this new wave of Now Media here is a collage of data about this shift.

Dimensions of the now web and how is it different?

Start with this constant, real time, flowing stream of data getting published, republished, annotated and co-opt’d across a myriad of sites and tools.    The social component is complex — consider where its happening.    The facile view is to say its Twitter, Facebook, Tumblr or FriendFeed — pick your favorite service.    But its much more than that because all these sites are, to varying degrees, becoming open and distributed. Its blogs, media storage sites (ie: twitpic) comment boards or moderation tools (ie: disqus) — a whole site can emerge around an issue — become relevant for week and then resubmerge into the morass of the data stream, even publishers are jumping in, only this week the Times pushed out the Times Wire.    The now web — or real time web — is still very much under construction but we are back in the dark room trying to understand the dimensions and contours of something new, or even to how to map and outline its borders. Its exciting stuff.

Think streams …

First and foremost what emerges out of this is a new metaphor — think streams vs. pages.     This seems like an abstract difference but I think its very important.    Metaphors help us shape and structure our perspective, they serve as a foundation for how we map and what patterns we observe in the world.     In the initial design of the web reading and writing (editing) were given equal consideration – yet for fifteen years the primary metaphor of the web has been pages and reading.     The metaphors we used to circumscribe this possibility set were mostly drawn from books and architecture (pages, browser, sites etc.).    Most of these metaphors were static and one way.     The steam metaphor is fundamentally different.  Its dynamic, it doesnt live very well within a page and still very much evolving.    Figuring out where the stream metaphor came from is hard — my sense is that it emerged out of RSS.    RSS introduced us to the concept of the web data as a stream — RSS itself became part of the delivery infrastructure but the metaphor it introduced us to is becoming an important part of our eveyday day lives.

A stream.   A real time, flowing, dynamic stream of  information — that we as users and participants can dip in and out of and whether we participate in them or simply observe we are are a part of this flow.     Stowe Boyd talks about this as the web as flow: “the first glimmers of a web that isnt about pages and browsers” (see this video interview,  view section 6 –> 7.50 mins in).       This world of flow, of streams, contains a very different possibility set to the world of pages.   Among other things it changes how we perceive needs.  Overload isnt a problem anymore since we have no choice but to acknowledge that we cant wade through all this information.   This isnt an inbox we have to empty,  or a page we have to get to the bottom of — its a flow of data that we can dip into at will but we cant attempt to gain an all encompassing view of it.     Dave Winer put it this way in a conversation over lunch about a year ago.    He said “think about Twitter as a rope of information — at the outset you assume you can hold on to the rope.  That you can read all the posts, handle all the replies and use Twitter as a communications tool, similar to IM — then at some point, as the number of people you follow and follow you rises — your hands begin to burn. You realize you cant hold the rope you need to just let go and observe the rope”.      Over at Facebook Zuckerberg started by framing the flow of user data as a news feed — a direct reference to RSS — but more recently he shifted to talk about it as a stream: “… a continuous stream of information that delivers a deeper understanding for everyone participating in it. As this happens, people will no longer come to Facebook to consume a particular piece or type of content, but to consume and participate in the stream itself.”    I have to finish up this section on the stream metaphor with a quote from Steve Gillmor.    He is talking about a new version of Friendfeed, but more generally he is talking about real time streams.     The content and the language — this stuff is stirring souls.

We’re seeing a new Beatles emerging in this new morning of creativity, a series of devices and software constructs that empower us with both the personal meaning of our lives and the intuitive combinations of serendipity and found material and the sturdiness that only rigorous practice brings. The ideas and sculpture, the rendering of this supple brine, we’ll stand in awe of it as it is polished to a sparkling sheen. (full article here)

Now, Now, Now

The real time aspect of these streams is essential.  At betaworks we are big believers in real time as a disruptive force — it’s an important aspect of many of our companies — it’s why we invested a lot of money into making bit.ly real time.  I remember when Jack Dorsey first saw bit.ly’s  plus or info page (the page you get to by putting a “+” at the end of any bit.ly URL) –  he said this is “great but it updates on 30 min cycles, you need to make it real time”.   This was August of ’08 — I registered the thought, but also thought he was nuts.    Here we sit in the spring of ’09 and we invested months in making bit.ly real time –  it works, and it matters.   Jack was right — its what people want to see the effects on how a meme is are spreading — real time.   It makes sense — watching a 30 min delay on a stream — is somewhere between weird and useless.   You can see an example of the real time bit.ly traffic flow to an URL  here. Another betaworks company, Someecards, is getting 20% of daily traffic from Twitter.   One of the founders Brook Lundy said the following “real time is now vital to what do.    Take the swine flu — within minutes of the news that a pandemic level 5 had been declared — we had an ecard out on Twitter”.    Sardonic, ironic, edgy ecards — who would have thought they would go real time.    Instead of me waxing on about real time let me pass the baton over to Om — he summarizes the shift as well as one could:

1. “The web is transitioning from mere interactivity to a more dynamic, real-time web where read-write functions are heading towards balanced synchronicity. The real-time web, as I have argued in the past, is the next logical step in the Internet’s evolution. (read)
2. The complete disaggregation of the web in parallel with the slow decline of the destination web. (read)
3. More and more people are publishing more and more “social objects” and sharing them online. That data deluge is creating a new kind of search opportunity. (read)”

Only connect …

The social aspects of this real time stream are clearly a core and emerging property.   Real time gives this ambient stream a degree of connectedness that other online media types haven’t.  Presence, chat, IRC and instant messaging all gave us glimmers of what was to come but the “one to one” nature of IM meant that we could never truly experience its social value.    It was thrilling to know someone else was on the network at the same time as you — and very useful to be able to message them but it was one to one.    Similarly IRC and chats rooms were open to one to many and many to many communications but they usually weren’t public.   And in instances that they were public the tools to moderate and manage the network of interactions were missing or crude.   In contrast the connectedness or density of real time social interactions emerging today is astounding — as the examples in the collage above illustrate.    Yet its early days.    There are a host of interesting questions on the social front.    One of the most interesting is, I think, how willthe different activity streams intersect and combine / recombine or will they simple compete with one another?      The two dominant, semi-public, activity streams today are Facebook and Twitter.    It is easy to think about them as similar and bound for head on competition — yet the structure of these two networks is fairly different.    Whether its possible or desirable to combine these streams is an emerging question — I suspect the answer is that over time they will merge but its worth thinking about the differences when thinking about ways to bring them together.      The key difference I observe between them are:

#1. Friending on Facebook is symmetrical — on Twitter it’s asymmetrical.    On Facebook if I follow you, you need to follow me, not so on Twitter, on Twitter I can follow you and you can never notice or care.   Similarly, I can unfollow you and again you may never notice or care.   This is an important difference.   When I ran Fotolog I observed the dynamics associated with an asymmetrical friend network — it is, I think, a closer approximation of the way human beings manage social relationships.    And I wonder the extent to which the Facebook sysmetrical friend network was / is product of the audience for which Facebook was intially created (students).   When I was a student I was happy to have a symmetrical social network, today not so much.

#2. The data on Facebook is assumed to be mostly private, or shared within private groups, Facebook itself has been mostly closed to the open web — and Facebook asserts a level of ownership over the data that passes through its network.   In contrast the data on Twitter is assumed to be public and Twitter asserts very few rights over the underlying data.    These are broad statements — worth unpacking a bit.    Facebook has been called a walled garden — there are real advantages to a walled garden — AOL certainly benefited from been closed to the web for a long long time.   Yet the by product of a closed system is that (a) data is not accessible or searchable by the web in general –ie: you need to be inside the garden to navigate it  (b) it assumes that the pace innovation inside the garden will match or exceed the rate of innovation outside of the garden and (c) the assertion of rights over the content within the garden means you have to mediate access and rights if and when those assets flow out of the garden.   Twitter takes a different approach.     The core of Twitter is a simple transport for the flow of data — the media associated with the post is not placed inline — so Twitter doesnt need to assert rights over it.    Example — if I post a picture within Facebook, Facebook asserts ownership rights over that picture, they can reuse that picture as they see fit.    If i leave Facebook they still have rights to use the image I posted.    In contrast if I post a picture within Twitter the picture is hosted on which ever service I decided to use.   What appears in Twitter is a simple link to that image.   I as the creator of that image can decide whether I want those rights to be broad or narrow.

#3. Defined use case vs. open use case.    Facebook is a fantastically well designed set of work-flows or use cases.   I arrive on the site and it present me with a myriad of possible paths I can follow to find people, share and post items and receive /measure associated feedback. Yet the paths are defined for the users.   If Facebook  is the well organized, pre planned town Twitter is more like new urban-ism — its organic and the paths are formed by the users.    Twitter is dead simple and the associated work-flows aren’t defined, I can devise them for myself (@replies, RT, hashtags all arose out of user behavior rather than a predefined UI.   At Fotolog we had a similar set of emergent, user driven features.  ie:  groups formed organically and then over time the company integrated the now defined work-flow into the system).    There are people who will swear Twitter is a communications platform, like email or IM — other say its micro-blogging — others say its broadcast — and the answer is that its all of the above and more.   Its work flows are open available to be defined by users and developers alike.   Form and content are separated in way that makes work-flows, or use cases open to interpretation and needs.

As I write this post Facebook is rapidly re-inventing itself on all three of the dimensions above.    It is changing at a pace that is remarkable for a company with its size membership.     I think its changing because Facebook have understood that they cant attempt to control the stream — they need to turn themselves inside out and become part of the web stream.   The next couple of years are going to be pretty interesting.       Maybe E.M. Forrester had it nailed in Howard’s End:  “Only connect! That was the whole of her sermon  … Live in fragments no longer.”

The streams are open and distributed and context is vital

The streams of data that constitute this now web are open, distributed, often appropriated, sometimes filtered, sometimes curated but often raw.     The streams make up a composite view of communications and media — one that is almost collage like (see composite media and wholes vs. centers).     To varying degrees the streams are open to search / navigation tools and its very often long, long tail stuff.  Let me run out some data as an example.     I pulled a day of bit.ly data — all the bit.ly links that were clicked on May 6th.      The 50 most popular links  generated only 4.4% (647,538) of the total number of clicks.    The top 10 URL’s were responsible for half (2%) of those 647,538 clicks.  50% of the total clicks (14m) went to links that received  48 clicks or less.   A full 37% of the links that day received only 1 click.   This is a very very long and flat tail — its more like a pancake.   I see this as a very healthy data set that is emerging.

Weeding out context out of this stream of data is vital.     Today context is provided mostly via social interactions and gestures.    People send out a message — with some context in the message itself and then the network picks up from there.   The message is often re-tweeted, favorite’d,  liked or re-blogged, its appropriated usually with attribution to creator or the source message — sometimes its categorized with a tag of some form and then curation occurs around that tag — and all this time, around it spins picking up velocity and more context as it swirls.    Over time  tools will emerge to provide real context to these pile up’s.   Semantic extraction services like Calais, Freebase, Zemanta, Glue, kynetx and Twine will offer a windows of context into the stream — as will better trending and search tools.      I believe search gets redefined in this world, as it collides with navigation– I blogged at length on the subject last winter.   And filtering  becomes a critical part of this puzzle.   Friendfeed is doing fascinating things with filters — allowing you to navigate and search in ways that a year ago could never have been imagined.

Think chunk
Traffic isnt distributed evenly in this new world.      All of a sudden crowds can show up on your site.     This breaks with the stream metaphor a little — its easy to think of flows in the stream as steady — but you have to think in bursts — this is where words like swarms become appropriate.    Some data to illustrate this shift.   The charts below are tracking the number of users simultaneously on a site.    The site is a political blog.    You can see on the left that the daily traffic flows are fairly predictable — peaking around 40-60 users on the site on an average day, peaks are around mid day.    Weekends are slow  — the chart is tracking Monday to Monday, from them wednesday seems to be the strongest day of the week — at least it was last week.   But then take a look at the chart on the right — tracking the same data for the last 30 days.   You can see that on four occasions over the last 30 days all of a sudden the traffic was more than 10x the norm.   Digging into these spikes — they were either driven by a pile up on Twitter, Facebook, Digg or a feature on one of the blog aggregation sites.    What do you do when out of no where 1000 people show up on your site?

The other week I was sitting in NY on 14th street and 9th Avenue with a colleague talking about this stuff.   We were accross the street from the Apple store and it struck me that there was a perfect example of a service that was setup to respond to chunky traffic.     If 5,000 people show up at an Apple store in the next 10 minutes — they know what to do.   It may not be perfect but they manage the flow of people in and out of the store, start a line outside, bring people standing outside water as they wait. maybe take names so people can leave and come back.   I’ve experienced all of the above while waiting in line at that store.   Apple has figured out how to manage swarms like a museum or public event would.    Most businesses and web sites have no idea how to do this.    Traffic in the other iterations of the web was more or less smooth but the future isnt smooth — its chunky.    So what to do when a burst takes place?   I have no real idea whats going to emerge here but cursory thoughts include making sure the author is present to manage comments etc., build in a dynamic mechanism to alert the crowd to other related items?    Beyond that its not clear to me but I think its a question that will be answered — since users are asking it.    Where we are starting at betaworks is making sure the tools are in place to at least find out if a swarm has shown up on your site.    The example above was tracked using Chartbeat — a service we developed.    We dont know what to do yet — but we do know that the first step is making sure you actually know that the tree fell — real time.

Where is Clementis’s hat? Where is the history?

I love that quote from Kundera.    The activity streams that are emerging online are all these shards — these ambient shards of people’s lives.    How do we map these shards to form and retain a sense of history?     Like the hat objects exist and ebb and flow with or without context.    The burden to construct and make sense of all of this information flow is placed, today, mostly on people.    In contrast to an authoritarian state eliminating history — today history is disappearing given a deluge of flow, a lack of tools to navigate and provide context about the past.    The cacophony of the crowd erases the past and affirms the present.   It started with search and now its accelerated with the now web.    I dont know where it leads but I almost want a remember button — like the like or favorite.   Something that registers  something as a memory — as an salient fact that I for one can draw out of the stream at a later time.   Its strangely compforting to know everything is out there but with little sense of priority of ability to find it it becomes like a mythical library — its there but we cant access it.

Unfinished

This media is unfinished, it evolves, it doesnt get finished or completed.    Take the two quotes below — both from Brian Eno, but fifteen years apart — they outline some of the boundaries of this aspect of the stream.

In a blinding flash of inspiration, the other day I realized that “interactive” anything is the wrong word. Interactive makes you imagine people sitting with their hands on controls, some kind of gamelike thing. The right word is “unfinished.” Think of cultural products, or art works, or the people who use them even, as being unfinished. Permanently unfinished. We come from a cultural heritage that says things have a “nature,” and that this nature is fixed and describable. We find more and more that this idea is insupportable – the “nature” of something is not by any means singular, and depends on where and when you find it, and what you want it for. The functional identity of things is a product of our interaction with them. And our own identities are products of our interaction with everything else. Now a lot of cultures far more “primitive” than ours take this entirely for granted – surely it is the whole basis of animism that the universe is a living, changing, changeable place. Does this make clearer why I welcome that African thing? It’s not nostalgia or admiration of the exotic – it’s saying, Here is a bundle of ideas that we would do well to learn from.  (Eno, Wired interview, 1995)

In an age of digital perfectability, it takes quite a lot of courage to say, “Leave it alone” and, if you do decide to make changes, [it takes] quite a lot of judgment to know at which point you stop. A lot of technology offers you the chance to make everything completely, wonderfully perfect, and thus to take out whatever residue of human life there was in the work to start with. It would be as though someone approached Cezanne and said, “You know, if you used Photoshop you could get rid of all those annoying brush marks and just have really nice, flat color surfaces.” It’s a misunderstanding to think that the traces of human activity — brushstrokes, tuning drift, arrhythmia — are not part of the work. They are the fundamental texture of the work, the fine grain of it. (Eno, Wired interview, 2008)

The media, these messages, stream — is clearly unfinished and constantly evolving as this post will likely also evolve as we learn more about the now web and the emerging social distribution networks.

Addendum, some new links

First — thank you to Alley Insider for re-posting the essay, and to TechCrunch and GigaOm for extending the discussion.    This piece at its heart is all about re-syndication and appropriation – as Om said “its all very meta to see this happen to the essay itself”.     There is also an article that I read after posting from Nova Spivack that I should have read in advance — he digs deep into the metaphor of the web as a stream.    And Fred Wilson and I did a session at the social media bootcamp last week where he talked about shifts in distribution dynamics — he outlines his thoughts about the emerging social stack here.   I do wish there was an easy way to thread all the comments from these different sites into the discussion here — the fragmentation is frustrating, the tools need to get smarter and make it easier to collate comments.