Architecture for civic participation

Last week’s brainstorming session on the use of social media for voter education got me thinking about the architecture that is needed for civic participation. The underlying concept is that the government provides basic infrastructure services and data. Citizens can participate in oversight and decision-making, and build tools for additional engagement, through access to services and data.

To facilitate participation, openness is needed in several layers.

  • open code and open data. These are two related families of practices that engage the community in the development and review of technology; and that make public information available to the public. Open data includes basic availability, as well as support for standards and licences that enable re-use and participation.
  • open APIs. Application programming interfaces enable developers to build on basic government infrastructure services, creating a broader ecosystem of applications that deliver value to the public without additional government funding, and that provide services that the government can’t.
  • Effective practices for social participation. Several attendees noted the problems with simple comment systems that devolve into anti-social anarchy, driving away constructive citizen participation. There are many techniques, tools, and social practices to overcome these problems. Solutions are context-dependent – there is no one-size-fits all solution.

It is exciting to participate in discussions such as the Social Media for Voter Education, the Hacking Open Government session at OSCON, and Transparency Camp West, coming up this weekend in Mountain View, that are helping to spread these ideas and encourage their implementation.

Of course Twitter is conversation

A couple of weeks ago, Mark Drapeau wrote a post that alleged that Twitter was not a tool for conversation, but for broadcast. It’s a provocative point, and is clearly false. Twitter isn’t a very good medium for extended conversation – but it’s obviously used for both conversation and broadcast.

The article uses statistics about the number of posts per Twitter account to infer that most Twitter activity is publishing. This isn’t a good interpretation of the facts for a couple of reasons. The low number of posts per account is almost surely evidence of a high rate of “dabbler” use. People sign up for Twitter, look around, and go away. The data about number of posts per account doesn’t say anything about people who are active on Twitter but use it primarily to consume content produced by others. There isn’t any evidence about the relative ratio of reading vs writing.

The second misreading relies on the Pareto principle – the highest volume of posts comes from a few people. This is true but irrelevant. Let’s say CNN has a service that publishes 100 updates per day on news stories. And two people have a conversation consisting of 5 posts each. These are two different, valid use cases. The existence of high-volume broadcast messages doesn’t somehow negate the fact that some people are talking to each other.

Direct evidence that that Twitter is conversation can be seen in Tweet Tweet Retweet a research paper by danah boyd and fellow researchers studying the use of Twitter. According to the paper, “36% of tweets mention a user in the form ‘@user’; 86% of tweets with @user begin with @user and are presumably a directed @reply.” The data uses on “a random sample of 720,000 tweets captured at 5-minute intervals from the public timeline over the period 1/26/09-6/13/09 using the Twitter API. This sample includes tweets from 437,708 unique users.” Another study with over 1 million tweets shows the same pattern – 39% tweets have an @user mention and 19% contain questions. (Thanks, Juan Carlos Muriente, founder of )

That looks like conclusive proof of the conversational use of Twitter. This surely dovetails with my own experience. In the last week, I’ve had conversations on distributed social networks, music, and Bay Area public transit. In these conversations I learned new information, met new people, shared ideas, and set the stage for follow-on activity. Twitter works for conversation, and the open nature of twitter sparks conversations that might not occur otherwise. It is true that Twitter is a not a good medium for in-depth, extended conversations. Messages are restricted to 140 characters. There isn’t visible threading (although thread info is kept in the data, allowing for threaded views such as Tweetboard.) The richest conversations sparked by Twitter often take place on Friendfeed, where replies are threaded in FriendFeed.

Twitter is good for short, fun and/or productive conversations that bring in often-unexpected relevant people through the social network. Deeper conversation and deeper collaboration need to segue into other modes. The next frontier for development, being pushed in different ways Google Wave,Citability, and other tools and concepts, will be means to connect shorter, real-time conversations with more in depth conversation and collaboration.

Facebook community fail

The devil’s promise with Facebook Connect was websites and communities wouldn’t need to worry their pretty little heads about user management and communication infrastructure. There was one true social network; and it lived in Facebook. All the site needed to do was cede their member login and identity to Facebook. In exchange, Facebook would bring to the site the real social network – all of your users, and all of their friends who use Facebook to share your good word. But it doesn’t work that way. I’d written about this in principle, but got bit by it in practice about a month ago.

I’m co-organizing an event on Social Media for Voter Education with California Secretary of State Debra Bowen. The event was originally scheduled for May 27, but the Secretary came down with strep and cancelled on the afternoon of the 27th. I used Facebook to manage the RSVPs. When I got the call from the Secretary’s office, I tried to use Facebook to notify the eighty-ish people who had signed up and said they would or might to come to the event that night. Unfortunately Facebook adds a delay if you want to send email to “many” people. That message didn’t get out until later that night. I used Twitter, an email to co-organizers, and old-fashioned social networking got the word out, but there were still some people who traveled to the event, only to find the “Postponed” sign on the door.

Facebook was the intermediary between our event and the participants, and when it came to crunch time, Facebook didn’t come through, and didn’t have a reliable way to reach people. Facebook has no obvious interest in making it effective for organizers to communicate effectively with the community. For an organization that needs reliable communication, outsourcing community management to Facebook isn’t a good deal. Groups are much better off with systems that let them manage and communicate with their own communities, using social network services as overlay but not as a core component.

If you are interested in the event itself, it has been rescheduled to July 29 at 7pm in San Francisco. I’m still using Facebook, because that’s the only way I can reach the people who signed up for the original event. And for the next event, I’ll want alternatives to Facebook with reliable communication.

iTunes, Last.fm and the politics of folksonomy

So, I’m listening to some perfectly nice folk/country/bluesish music by Eric Bibb in iTunes and I notice that the Genre column has the recording listed as Blues. I enjoy the multi-dimensional space of folk/country/blues/rock/etc, and like stuff at varying points in the coordinate matrix. This is the only album of Bibb’s I have, and I suspect it’s on the folkier side of his folk/blues mix. The reason this particular recording is categorized as “Blues” seems to be, er, at least as much visual as auditory.

For alphabetical reasons the song following Bibb is Eric Clapton’s cover of Going Down Slow, written bySt Lous Jimmy Oden and popularized by Howlin Wolf. Clapton’s cover is classified as Rock. The Clapton tune is much straighter blues than most of that Bibb recording. But Clapton is more vulnerable to sunburn.

Does anybody other than me find this aggravating? It seems *late* for this to be an issue. Obama likes star trek and jazz, he has the right to his choices as do the rest of us. Why do I need to look at this obsolete marketing category that classifies music ethnically not sonically.

Aha! The Last.fm tag cloud does a better job of things. The top tags for Bibb are blues, acoustic blues, folk and singer-songwriter. The top tags for Clapton are classic rock, blues, blues rock, guitar, singer-songwriter.

This issue is less vital than the discrimination that keeps gay families from legal protections of marriage, and other issues where real people get hurt by badly applied categories. It’s is more superficial than #amazonfail, the category mistake that pulled books on gay and lesbian themes out of Amazon search results and hence into more limited sales prospects.

The genre categories in iTunes are annoying throwback to the bad old days where music access was partitioned by segregated radio station. The result of the bottom up social network folksonomy is yet to fully express itself and yet to be measured. But I’d much rather look at the more nuanced, more accurate, less stereotyped tag cloud.

Facebook Open? not til they fix the privacy model

FaceBook has just taken two important steps away from being a walled garden by opening the API to stream data and by supporting OpenID. These things are very good. As someone who’s complained about the walled garden model, I think these are steps in the right direction. But these steps do not get FaceBook very far until and unless they fix the privacy model.

FaceBook is simultaneously too private and not private enough. This gets in the way of using it for information sharing AND for private information.

Facebook’s model for information about people is symmetric and mostly private. (Pages are a limited exception) I can only see information about you and from you if we mutually declare each other to be friends. This puts a break on the discovery of new people and new information. If you post an interesting link in Twitter, I can navigate to see your stream of tweets and choose to follow you. If you comment on my friend’s link in Facebook, I can’t see enough about you determine if I want to know more. Even if I could, “friending” is a different social gesture – I won’t friend you because I don’t know you. The mostly-private nature means that search is useless except to find people you already know.

ReadWriteWeb explains how this dramatically limits the utility of the newly open API:

Unfortunately, the data that developers are able to work with is severely limited. They will simply be able to make a call for a user to Facebook and get back the friends’ streams that this particular user has the permission to see. … Terms of Service will prohibit eyes outside of a user’s Facebook friends from seeing the massive amounts of friend-limited data. In other words, this is permission to build more interfaces for Facebook. That’s cool, but that’s not really what the world needs – more interfaces for giving Facebook love.

Meanwhile, Facebook’s model is not private enough. Facebook has been trying to be private but viral, and that makes it really hard to be private. Facebook actions leave trails all over the place. For example, if I comment on my friend’s link, others can see my comment. Facebook does have some granular controls over categories of friends and what is exposed to them. But these controls are not very easy to use. The API makes it possible to for developers to disclose information about your activities in unintended contexts, which may open new opportunities for privacy violation.

Until Facebook fixes its privacy model so that what’s open is open and what’s private is private, supporting open standards doesn’t make Facebook usefully open, and may make privacy issues worse.

Twitter, Facebook, and the unselfish API

Bernard Lunn’s ReadWriteWeb piece about the reverse network effect, writes that one of the ways that social networking services can wear out their welcome is by making their user base feel exploited. Intrusive ads, aggressive marketing, or onerous terms of service can create dissatisfaction and eventual exodus. The RWW article has the end user base of the service in mind, but I suspect the same dynamic pertains to the developer community. With that lens, it’s interesting to consider the very different ways that Twitter and Facebook handle APIs and integration.

Twitter’s API is unselfish. Using the straightforward REST API, developers can and do write clients, search tools, mapping tools, recommendation tools, analytics, personal organizing – a wide range of extensions. Twitter doesn’t do anything to constrain developers other than a rate limit. The lightest weight sort of integration is RSS, and Twitter generates RSS feeds for queries and streams, making it trivially easy to disseminate data. The availability of applications helps build the Twitter user base because they make Twitter more useful. Twitter’s business model is up in the air; but whether it moves toward paid accounts for power users, corporate users, advertising, there will continue to be plenty of room for complementary apps.

Facebook’s API is build to serve Facebook more than developers. The original API constrained developers to exposing a limited user interface within Facebook’s strict design. The functionality encouraged the creation of apps that expanded the Facebook user base because users were encouraged to spam their friends. Given the limits of Facebook, applications tended to be shallow. The most successful app developers needed to relentlessly focus on novelty because users would get bored with yesterday’s toy. Still, application developers put up with the limits because Facebook gave them access to oodles of users.

Then, with the move toward the Twitter-style user interface and the strategic shift toward Facebook Connect, Facebook hid and de-emphasized apps in the user interface. App providers, and users who were starting to like using Facebook for richer engagement were short on luck. Facebook Connect looks on the surface like it might provide developers with more breathing room. A developer can build a fully fledged application or community site, and take advantage of Facebook Connect, which lets users to bring their social network to the site.

But this is a deal with the devil. The problem is that when sites use Facebook Connect, they have minimal connection to their user base. An an application or community site wants to create the policies whereby the site communicates to the community, and the community talks to each other. With Facebook Connect, those rules belong to FaceBook. What’s worse, the member database is critical for a site to make money through ads, sales, donations, or services. With FB Connect, all your member database are belong to them. Another sign of Facebook’s weakness at supporting external sites can be seen in the lack of RSS feeds for public data like Pages. Facebook RSS is designed as a black hole. Content can be sucked into Facebook, and can’t get out. Facebook’s goal with APIs and integration is self-interested. They want to own the social graph, the user data, and the content; developers are sharecroppers on Facebook’s land.

I can see why a short-lived temporary site might want to use FB connect as a shortcut. For an established site, the viral aspect of Facebook may make Connect worth a try. But for a site that wants to build community and business value over the long haul, FB Connect is parasitic. Google’s Friend Connect has some less toxic properties – they are using standards for single signon, portable contacts, and portable lifestream data. The problem with Friend Connect is that doesn’t really have a social network. When there is a more open method with good social properties, applications and communities will go there.

Twitter’s unselfish API strategy will enable it to grow it’s community and provide win/win opportunities for developers. Facebook’s selfish strategy looks on the surface like it will help Facebook’s business success, but it risks running aground on RWW’s exploitatin principle – exploit your developers and they will leave when they get a chance.

How asymmetry scales

Bokardo predicts that Facebook will go asymmetric. He calls out two key reasons why: asymmetric networks are a a good fit for anyone with micro-fame, not just organizations, brands and bands. Asymmetric networks help people manage their attention – you don’t need to pay attention to every update from everyone following you.

There are a couple of other key reasons why asymmetric networks scale better. In Twitter there are a number of ways where asymmetry in a public network provides good returns to scale, as noted in yesterday’s post on premature predictions of peak Twitter
* Retweets get you information that was first posted by someone outside your network
* Searches let you find information outside your network
* Visible replies, like the lovely feature in TweetDeck that shows when someone mentions you even if you’re not following them, allow you to hail and engage people in conversation, and have others start conversations with you, even if you’re not following.

These features mean that the more people who join the network, the more interesting information will be amplified through it, and the more potentially interesting people you may discover. The level of context is fairly high – you can see what someone else has been Twittering, and see if they are interesting and relevant to you. And the level of obligation is low (you can follow someone without giving them the burden of accepting or rejecting you).

In Facebook, I can see when someone that I don’t know has commented on the update of someone I do know, but then I need to friend a stranger in order to learn more about them. Facebook’s mostly-symmetrical, mostly closed network makes it hard to learn new things and meet new people outside your existing network.

So, the reasons for asymmetry aren’t just about supporting fame, but enabling discovery with low social expense.

Peak Twitter?

There are several arguments going around predicting Peak Twitter. The discussion raises a number of interesting issues questions about social media and scale.

In Twitter is peaking, Steve Rubel describes the risks to Twitter as social trendiness and increasing messiness.

Too popular. Social networks seem to have a property in common with nightclubs, bars, and restaurants – they are popular for a while. Then the throng moves on. The digerati were on Orkut for a few minutes, before moving on to Facebook and Twitter. Popularity depends on community – Facebook and MySpace are bigger in the US, Bebo is big in Europe, Orkut is big in Latin America.

Rubel hypothesizes that the trend pattern is similar other pop culture trends, where hipsters create a trend, and then flee when the mainstream arrive. Rubel writes, “Just six months ago, the list of the top 100 users on Twitter read like a who’s who of geeks. That’s what made it a draw, for many, initially. Now, however, the list looks like People or US Magazine. Twitter is losing its geek creds as celebs flock to the service.” The difference is, a social network is a great many places, not one; the network is inhabited by millions of overlapping subcultures. Honestly, I haven’t heard of many of the pop culture celebrities who have recently joined Twitter, and the ones I’ve heard of, I don’t follow. I do follow some of my personal heroes, but they aren’t pop culture icons.

The argument that people magazine starlets and nba players will crowd out niche communities is the same mass media vision that there would be a handful of pop-culture centered websites that would crowd out the rest of the web. There are 270 million people on Facebook, which is a great many more than say, the 15 million people who visit Disney every year, and their subculture-centric Facebook experiences are different than the mass-produced Disney experiences.

Too big. The second argument is scale and disorganization. “Since replies are not threaded, celebs and corporations do not feel they have to respond to every Tweet.” This is a real challenge. Rubel rightly recognizes that tools are evolving to address the challenge. What’s missing is that personal needs are very different from organizational needs.

For personal use, the fact that Twitter is a flow is part of the charm. A twitter feed doesn’t carry the same perceived social obligation to keep up and respond as email or instant message. You can dip into the stream, step out, and come back later. For personal use, people need some better tools to manage their attention. Tweetdeck, which Rubel calls out as a good example, adds groups, search, and embryonic filtering into the basic experience.

The needs of non-celebrity individuals are different from the needs of corporations, politicians, and famous poeple. If your constituency has thousands to millions of people, you need very different tools to monitor the conversation than if you are following fifty or 100 people. If you’re an individual, and you miss an update from a friend or an interesting news link, no big deal. If you are striving to use Twitter for constituent listening and feedback, you want to notice complaints, suggestions, and kudos. You probably want to have multiple people listening to the account, listening for different products or topics, and working on responses.

Dunbar limit. In ReadWriteWeb, Bernard Lunn makes the opposite point, that size doesn’t matter. “In a social network, the value for existing users of a new user joining the network plateaus once users have most of their own contacts in that network.” For mostly closed, symmetrical networks such as Facebook and Linked In, this is true. For mostly open, asymmetrical networks such as Twitter, this is mostly false, which Lunn mentions briefly. I suspect that people will cap their participation at some augmented Dunbar limit of the number of people they can follow with social attention and time. But in Twitter, retweets, searches, and visible replies mean that the more people who join the network, the more interesting information will be amplified through it, and the more potentially interesting people you may discover. When you have your existing contacts on the networks, it is easy and to make new contacts if you wish. The level of context is fairly high – you can see what someone else has been Twittering, and see if they are interesting and relevant to you. And the level of obligation is low (you can follow someone without giving them the burden of accepting or rejecting you).

Exploitation. In the ReadWriteWeb post, Lunn makes the insightful point that social networks can fail when their hosts start to violate the implied social contract with their communities in the interest of making money from their investments. “If these businesses get too eager to monetize to justify those valuations, they may create the reverse network effect.” When they move to monetize, hosts may move toward intrusive advertising, marketing, privacy violations, or other steps that benefit the site’s commercial interest and go against the interests of the users. I see the potential risks even more broadly than Lunn does. Intellectual property terms of service, and increased control over content and customization can violate the perceived community social contract as much as intrusive ads and marketing can. There is some inertia to switching, but in the absence of monopoly, annoyed communities do pick up and go with some regularity.

Parasitism. In Mourning the loss of Twitter, Ross Mayfield predicts that Twitter will fall prey to the spam and other antisocial behavior that crippled Usenet and Email. Hopefully the Twitter ecosystem will evolve to meet the threats, and blacklist and social filtering tools will keep the parasites from killing the host.

Twitter is a fascinating experiment since the social scale dynamics of an asymmetrical, open network aren’t known. I suspect that the ecosystem will evolve social and topic filtering tools that will help it scale; time will tell. The platform strategy is helping already – third parties are building tools to search, manage, and respond to the twitter stream. And I hope that the Twitter management retains a good sense of environmental judgement and finds ways to make money that don’t feel exploitive to the community.

Facebook as a Twitter Wannabee

The new Facebook UI has become a stream of Twitter-like updates. The pattern builds on the addictive conversational nature of Twitter, but cripples some of the key ways that Facebook was different than Twitter. What made Facebook better than the earlier generations of YASNs is that it not only let you declare your friends but do things with your friends – share applications with them, share events, create groups, organize. The new Facebook hides the affordances for apps, events, groups.

By hiding the affordances for application functionality, are they making a really big bet on Facebook connect? Are they hoping that 3rd party services with independent web presense will integrate into the stream by delegating their member database to Facebook? This could be. The weakness of this strategy is that 3rd party services have no loyalty to Facebook and would just as well use some other technology. People just want to do things with their friends, with the least barrier to getting started.

Also, Facebook has FriendFeed-like discussion around assets, which is nice. The threaded comment UI is intuitive. It’s very helpful when you’re actually talking about an asset like a bookmark. But it lacks the transparency, discovery, and immediacy of Twitter conversations. With Twitter conversation, you can see someone replying to someone else, and find interesting new people. With Twitter conversation plus search, you can see someone asking a question and then follow the answers.

Also, Twitter conversation is present-focused in a good way. Facebook conversations are anchored to the original remark that happened to start the conversation. So if someone said something interesting 4 hours ago, you have to scroll back to find it. Which you probably won’t. With Twitter, if the conversation is ongoing, you’ll still hear it.

In summary: the Twitter mode for Facebook does give it some of the addictive quality of Twitter but in imitating Twitter, Facebook has sacrificed too much of what makes Facebook valuable. And in attempting to imitate Twitter, Facebook has missed some of the social dynamics that make Twitter good.

Aardvark vs. Twitter – the role of social in social search

ReadWriteWeb writes about Aardvark, a new IM-based Q&A service which raises the question about how valuable the social network is to search.

Frederic Lardinois at RWW says “Aardvark is a neat new service that lives in your IM client and which routes any question you might have to an Aardvark user who has the right expertise to answer your query. In return, Aardvark will also send you a few questions every day that fit your profile. You then decide to either answer the question or refer it to another friend. Of course, you can also always pass if you don’t know the answer.”

This is a very different sort of experience than Twitter, where you send out a question to people following you, and good Q&A may be forwarded through their networks.

Personally, I greatly prefer the Twitter model. IM is interruptive, Twitter is not. You can ignore the stream entirely, and pick up only the questions you want. With Twitter, the Q&A is interspersed with other sorts of information and conversation. A barrage of constant questions might feel more like an inquisitive pre-schooler.

With Aardvark, the questions come to you via IM, which is interruptive. I can’t imagine using that and having randome questions to answer in the middle of the day — maybe this would be fun for students and retirees. I frequently use IM and IRC, but maybe younger people who live with an open set of 8 IM chats woudn’t mind getting search questions by IM throughout the day, too.

Aardvark’s social feature feels anti-social to me. You can forward a question to a friend via IM. This is cool, since you may know exactly the right person who can answer the question. But it means that your friend also feels a social obligation to answer and feels social guilt for not answering. This is the reason I prefer Twitter questions to Linked In questions, multiplied by 1000. LinkedIn questions feel awkward because someone you know is asking you personally to respond. Twitter questions do not feel awkward because there is no obligation – if you answer you get good karma, and if not, you haven’t had to choose to ignore someone.

The Q&A opportunity in general is huge. People want questions answered and enjoy answering them. Yahoo Answers is huge. As of late 2008, Answers had nearly 150 million monthly visitors worldwide and 1.3 billion monthly page views. Yahoo Answers has a much more encyclopedia-like model, where you can search and browse for answers to questions. Aardvark is IM — does this mean that answers won’t be discoverable by others?

This real-life experiment — the Aardvark vs. Twitter models — will reveal something about the psychology of social search. Personally I’d greatly prefer Twitter, but perhaps Aardvark will find a demographic and psychographic that prefers its model.

Update: Rob Spiro of Aardvark says on Twitter that they are “definitely planning an aardvark-twitter integration, using Twitter as another communication channel.” “TwitVark” would be a great configuration, since it would combine the conversational atmosphere and optional social norm of Twitter with the social search filtering of Aardvark.