Paying for Caltrain

This week I went to a meeting of the Bay Rail Alliance, where the topic was paying for Caltrain. The agency is facing a grim deficit because it depends on earmarked state transit funds that are regularly raided for other uses.

To close an immediate budget gap, Caltrain is making changes including increases in parking fees and charges for employer-funded transit passes, and cutting back on mid-day service. Based on overwhelming community feedback, a worse proposal to eliminate weekend service was taken off the table.

Even with these changes, Caltrain’s revenue is unstable, unlike Bart, which gets some of its funding from local taxes. So the Bay Rail Alliance is interested in investigating potential sources of regional funding. If you’re interested, look for updates on the Bay Rail Alliance website.

While the operating budget is iffy, the capital situation looks promising. The Bay Area is a candidate to get stimulus funding targeted at high-speed rail. Since the stimulus funding needs to go to shovel-ready projects, what this means in practice is that stimulus funding would go to items including Caltrain electrification, and preparing the Transbay terminal to handle the long-awaited extension of Caltrain to the water’s edge. See Transbay Blog for good detail and ongoing coverage.

There are two underlying system problems that make these things a lot harder than they should be. The first is the underlying structural bankruptcy of the California budget process. The calls for reform seem to have quieted down a bit during the knock-down dragout budget battle in Sacramento but hopefully will pick up after the battle. (Comments on what’s going on would be welcome.) If reform goes anywhere, it will need a large constituency, and part of the alliance ought to be green; transportation is the single largest source of greenhouse gas emissions in California, and raiding transit budgets is not the way to get change.

The second system problem is the fragmented of the bay area transit system, were 26 seperate agencies serve a metro area of 7 million. Better regional governance would remove a lot of un-needed friction in creating a great system, but would take major reform.

iTunes, Last.fm and the politics of folksonomy

So, I’m listening to some perfectly nice folk/country/bluesish music by Eric Bibb in iTunes and I notice that the Genre column has the recording listed as Blues. I enjoy the multi-dimensional space of folk/country/blues/rock/etc, and like stuff at varying points in the coordinate matrix. This is the only album of Bibb’s I have, and I suspect it’s on the folkier side of his folk/blues mix. The reason this particular recording is categorized as “Blues” seems to be, er, at least as much visual as auditory.

For alphabetical reasons the song following Bibb is Eric Clapton’s cover of Going Down Slow, written bySt Lous Jimmy Oden and popularized by Howlin Wolf. Clapton’s cover is classified as Rock. The Clapton tune is much straighter blues than most of that Bibb recording. But Clapton is more vulnerable to sunburn.

Does anybody other than me find this aggravating? It seems *late* for this to be an issue. Obama likes star trek and jazz, he has the right to his choices as do the rest of us. Why do I need to look at this obsolete marketing category that classifies music ethnically not sonically.

Aha! The Last.fm tag cloud does a better job of things. The top tags for Bibb are blues, acoustic blues, folk and singer-songwriter. The top tags for Clapton are classic rock, blues, blues rock, guitar, singer-songwriter.

This issue is less vital than the discrimination that keeps gay families from legal protections of marriage, and other issues where real people get hurt by badly applied categories. It’s is more superficial than #amazonfail, the category mistake that pulled books on gay and lesbian themes out of Amazon search results and hence into more limited sales prospects.

The genre categories in iTunes are annoying throwback to the bad old days where music access was partitioned by segregated radio station. The result of the bottom up social network folksonomy is yet to fully express itself and yet to be measured. But I’d much rather look at the more nuanced, more accurate, less stereotyped tag cloud.

Google book settlement – what can we do?

The Google book settlement may be the most important undercovered tech policy / digital rights issue live today. And because it deals with the trailing edge of the long tail, it isn’t getting the attention it deserves.

What is this about? The Google books project was sued by the representatives of a small fraction of the millions of out of print books that it scanned. Instead of taking the case to trial, Google hammered out a settlement, which now needs to be approved by a judge.

The problem is that the settlement as it was written, has major anti-trust and anti-consumer implications. The settlement as written gives Google the exclusive right to scan out-of-print books and make them available. Anyone else – a competitor like Amazon, or an academic institution, or a person – would risk getting sued if they also wanted to digitize books. Legal scholar Pamela Samuelson called out the problems in The O’Reilly Radar Blog.

So is this a done deal? Not necessarily. The judge has the power to accept the agreement as written, require modifications, or reject the agreement. There are strong arguments to keep it but fix it.

Professor James Grimmelmann of New York Law School has written a clear paper explaining how to fix the settlement — remove the monopoly aspect by giving the same terms to future participants, and putting the public at the table by mandating library and reader representation on the registry board. A more detailed summary of Grimmelman’s recommendations can be found at the Law Librarian Blog. Months before the #amazonfail debacle, Grimmelmann recommended a provision that would prevent secret censorship by silently delisting books.

The US Justice Department is investigating anti-trust implications of the settlement, and the judge extended the deadline for parties to file briefs in the case by five months, from July to September.

So what can we citizens do? While individuals can’t lobby directly — this isn’t a legislative issue where you can contact your congress critter — parties with an interest in the case can express opinions to a judge.

The Electronic Frontier Foundation is recommending modifications including putting book scans in escrow. Public Knowledge also plans to file comment advocating keeping the settlement but modifying it to address problems with competition and access to orphan works. The American Library Association has weighed in. The Internet Archive and Consumers Watchdog have registered comments with the justice department, as reported by the New York Times.

But this issue isn’t getting as much attention as it deserves. It deals with the trailing edge of the long tail – works that are out of print, where the authors can’t be found. Issues that deal with new technology and consumer access – net neutrality and social media privacy seem more sexy. But over time, access to the scholarly knowledge and cultural resources locked in “out of print” works has cumulative value. Granting monopoly privileges is a slow drain on freedom, rights and knowledge.

Some of the larger organizations that often get engaged to protect consumer rights and digital freedoms — including Free Press, Consumers Union, and the Center for Democracy and Technology had not yet gotten involved when I last checked (before the original deadline). They may have gotten involved since — I’ll check, and would love to hear from folk in the know.

If you are a supporter of EFF, Public Knowledge, or the Internet Archive, thank them for their support on this issue. If you are a supporter of other groups that advocate digital rights, consumer protection, and academic freedom, please let them know that you care about this issue, and ask them to weigh in.

Wilco: I am trying to break your heart

“When you strip it down, it just sounds like a folk song.” That’s Jeff Tweedy of Wilco talking about their music early in the 2003 documentary about the making Yankee Hotel Foxtrot, which I watched this weekend after recently digging YHF out of the garage. Tweedy is right. Pull off the sonic layers and add half the words back to the fractured lyrics, and you have accessible, good folk and rock’n’roll. The live performances of Tweedy and the band make that clear. This music is not that hard.

But YHF was off-center enough that Reprise Records dumped the band when Tweedy wouldn’t take their advice to make the music more accessible. Wilco put the recording on the internet in the iterregnum before Nonesuch, another division of Time Warner, picked it up. Internet distribution only heightened interest in the recording and helped fans stay keep up with the band before the record came out.

The Wilco saga was a fairly early sign of the breakdown of the oligopoly. The tactics to try to preserve the economic scarcity of physical distribution in an age of digital download were unsustainable. The fact that YHF is a problem at all is a problem. Jim O’Rourke, who gets a speaking part of about 15 seconds, on the other hand, who was brought in to help production, is a ringer for music that resists easy. Nobody’s asking him about commercial music; that would probably keep the documentary from being produced.

Suroweicki argued in Slate that the conventional reading of artistic victory against commercial philistinism doesn’t hold because after all, it was another division of Time Warner that picked up the record; others have observed that Reprise didn’t have to have the grace to let the band buy their contract out. Still, Tweedy and manager didn’t have to have the balls and economic confidence to reject the advice to tone down the eccentricity and up the catchiness.

Interesting that it was Howie Klein, the music exec turned political blogger, whose ouster led to Reprise rejection of the record. Among other things, Klein has been one of the curators of the wonderful “Late Night Music Club”, a virtual fireside chat with youtube clips across wide range of excellent and interesting music irrespective of fashion and nominal genre. Communities like NLMC are taking the place of the radio playlist for music discovery, and that’s for the better.

In the Lefsetz Letter an entertainment industry lawyer makes the nostalgic argument in favor of the role of massmarket hits at creating common public consciousness. But the trade always was too high, in segregation, genre-focus, overplay, and the loss of cultural context in a narrative focused on hits. (Not to give Lefsetz a hard time; reading his blog, he is otherwise in favor of digital distribution and taking advantage of the long tail.)

Maybe we’ll eventually get a good “digg” for aggregating and voting up digital plays, which can play the role of a zeitgeist track. That wouldn’t be a bad thing, since it wouldn’t prevent people from discovering long-neglected performances on YouTube and discovering wonderful stuff through the playlists of friends and acquaintances. Network math works like that – there’s still a tall head in the age of the long tail – it’s just that you can get to the long tail now and you couldn’t before.

p.s. interesting that the Wikipedia definition of Playlist is now dominated by digital tools and the digital definition.

Green optimism

A few really good signs in recent months. In April the Honda Insight hybrid became the best selling car in Japan. Not the best selling hybrid, or best selling in some other niche. The best selling car.

Also, in 2008, “For the first time, investments in green energy overtook investments in fossil fuels for power generation ($140 billion vs. $110 billion)” according to a UN study. This is investments for power generation, not transportation fuel, but see above – electricity becomes automotive fuel.

The overall math on worldwide energy supply is scary but these are very good signs.

Backing up music with Windows and Mac

There are some things that Apple makes easy, and others that stay complicated. If you have more than one computer – especially if you have Mac and Windows – handling backup and synchronization is a pain.

I use a Windows machine mostly as a media player at home, and a MacBook for work and mobility. I had most of the iTunes library on the Windows machine, and a few things that I had bought on impulse while using the Mac. I wanted to use the Windows machine (an XP laptop with a busted battery and some Logitech speakers) as the main media player (because I own it), and the mac for sometime listening. I wanted it all backed up, ideally in a couple of places, and I had a 300G USB hard drive for backup.

Here’s what I needed to do:
* locate and import some missing backup files from a dead computer into the main collection (the key was to search for a distinctive song title)
* reformat the hard drive to FAT32 so that it could be read/written from Windows and Mac. It had been formatted so that Windows could read/write but Mac couldn’t write.
* use the external hard drive to move the Mac library onto the PC laptop
* carefully copy the Mac files into the PC folders (there were a few artists for whom I had different albums on each)
* turn on “Sharing” for the windows machine so I can listen at home

This multi-step process took a little bit of figuring out, with the help of Google and some nice folk at the Apple store. Once I figured out the steps, the implementation took a few hours but was pretty straightforward.

I signed up for an online backup service, but didn’t use it because it seems like it will take a few days to back up my collection and that’s not practical. To make a second backup that’s not in my house, the way to go seems like a USB keychain that lives in my bag or wallet. I’ll rip some CDs and see what size I need.

As the next step in the project, I’ll become the last person on the digital planet to RIP my CDs. Why am I only now getting around to ripping CDs and organizing a digital music collection? To make a long story short, I hadn’t taken care of the digital music collection because until Apple and the labels took off DRM I considered the digital stuff disposable, and bought as little DRM’d music as I could. I had spent the time in Austin mostly focusing on Austin music, mostly on CD. When I got to the Bay Area, I was heads down on work for a bit.

When I came up for air, I wanted to “true up” my music collection and taste; I didn’t want to just listen to the things I already liked and things that are nearly identical. So I’ve been doing a little exploring with the help of last.fm and youtube and wikipedia. That’s a longer story that may or may not make it to blog form.

Designing incentives for collaborative groups

The raw material for Designing Social Interfaces does a lovely job at outlining a range of patterns to incent participation for groups. The patterns reflects a powerful insight – communities vary in their level of competitiveness, and patterns should be used appropriately for the level of competitiveness in the community. “Haphazardly introducing competitive incentives into non-competitive contexts can create problems and may cause a schism within the community.”

Game communities have been leaders in using leaderboards ranking, named and numbered levels, and related techniques to incent participation by triggering the desire to beat the other guy. But for some people these techniques are disincentives: “In user-testing, we’ve seen some strong reactions to Numbered Levels from folks who make associations with ‘being graded’ or assessed. Others noted that numbers just “seem impersonal and kinda cold.”

However, techniques for more collaborative communities have been less well described. This post sketches a few I’ve observed. Reflecting on these patterns, they seem to fall into three groups:
* praise contribution
* make opportunities for participation visible
* make progress toward shared goals visible

Praise contribution

Badges
One pattern already described in the manuscript that can be used in a more cooperative setting is collectable acheivement. People can earn badges for completing tasks and post them to their page as marks of personal acheivement. There are also “gift” badges that can be given to others for “those who’ve distinguished themselves in some way: perhaps they’ve excelled at one particular skill that the community values; perhaps they are official representatives for the community or an affiliated organization; perhaps they have volunteered to be a helpful resource for others in the community. ”

Props
On Blip.fm, you can give someone else “props” for sharing a good song. Props give the receiver credit to use with others. This is a non-competitive form of Karma points – Karma points show appreciation and make it easy to compare people’s relative karma.

Improvisational sharing

Shared collections
A common collaborative pattern is the creation of a shared collection. Using a tags to collect material, a group to define contributors, or both, people define something to collect. On Flickr, for example, there are groups that collect pictures of Japanese bento boxes, makes models and serial numbers of airplanes, photos with certain color combinations. On Twitter in recent weeks, people have been sharing links to country/americana songs under the #twangthursday tag.

All that’s needed for this pattern to take hold is the ability to assemble content and people with a tag the group defines, to see the collection, and see new items.

Make opportunities for contribution visible

Gardening tools
In a large collaborative project such as a wiki, gardening tools reveal content that needs to be worked on; pages that are un-linked or un-tagged, content areas that need to be filled in and improved, new contributions that need review. These tools can be used with and without explicit rewards – one principle of “cognitive surplus” communities is that people will contribute and be creative if given a chance – simply making it easy for someone to see what needs to be done and do it is a motivator.

One example is the WikiHow editor’s toolbar which allows WikiHow editors to view edits that need review on new pages, featured pages, and more.

Visible progress toward shared goals

Fundraising widgets
To raise money for a cause, the person or group wanting to raise the money provides widgets that others can embed in their own sites. The widgets allow viewers to donate, and see progress toward the shared fundraising goal.

Information radiators
An Information radiator is a pattern described by agile software development proponent Alistair Cockburn. This is a shared bulletin board with information on the state of a shared project. For example, the “information radiator” for the Socialtext agile development process shows a burndown chart of completed stories for the iteration, whether there are failing automated tests, and the state of stories in progress. The metrics displayed in the radiator reflect items the team wants to track or improve; as the problems to solve change, the metrics may change.

There are probably many more patterns, and ways to organize them too. I’m going to put this to others to help with the brainstorming.

Credit to Peter Kaminski for ideas on Flickr photo sharing as a nomic game.

The death of the hit narrative

Bios and reviews of musicians often talk about careers in terms of hits; when a musician is good but not super-popular, the narrative is about whether this new record might be the one where the artist makes it big. But this story is starting to seem quaint.

Recorded music evolved in an economic context of mass marketing and mass distribution. To make money in this model, where there are high upfront costs and venture risk, you need to sell a ton of stuff. Mass media and mass distribution enable you to sell a lot of stuff. Thus, the evolution of music as a “hit-making” business. This story isn’t just inside baseball; some genius realized that the emergence of hits was a simple, dramatic story for the audience. “Greatest hits” shows told the story of which record was at number 12 on the charts and climbing.

The hit leaderboard takes attention off of the music itself and puts the focus on popularity. Music, within this narrative, lacks context, other than the genre in which it’s being ranked (r&b top 40, country top 40). Just as political coverage using the “horse race” narrative focuses on who’s up, who’s down, who’s stumbled, and ignores substantive coverage of the politicans’ records and campaign content, hit-centric coverage focuses on what’s climbing the charts at the expense of other aspects of the music.

A spot at the top of the charts has become a lot less relevant as tool for becoming aware of popular music, given the tools available for music discovery, surfing your friends’ collections on last.fm, trading earworms on blip, googling influences and band members. Out of factors that make music interesting and appealing, raw popularity seems like one, somewhat interesting factor, among others – what’s that sound? Who are they playing with? Where do they come from? What are their influences? What do they want to do next? Where are they playing live?

Of course, people in the know have always had access to their friends’ collections; the recommendations of the crotchety person at the record store; references in liner notes. And there were always subcultures that engaged with the music, not just the hit leaderboard. To take advantage of these things, you needed a strong social context and a fair amount of diligence. Many more people consumed what was provided on the radio and tv. The methods available to the cogniscenti have been democratized.

And this is changing the underlying story that is told about music. The dominant story was one of high peaks and low valleys. The change in music distribution seems to be enabling a tier of musicians who musicians who can make a living, more or less, playing live, selling indy, and exploring their music. Articles that ponder when an artist might hit the big time, and wondering why they haven’t seem irrelevant, as long as the artist is able to eat and stay out of the rain.

The new story, as it is emerging, is more picaresque – where did you come from, where are you now, where are you going?

Distributed social networking and distributed profiles

The nascent vision of distributed social networking imagines transparent ways of sharing profiles and friends. Is there anything in the OpenStack yet that allows profile fields to be distributed?

Here’s the model. If I’m involved in online communities relating to, say, environment, politics, music, and sports, then right now I have a different profile per site. So I want one profile. Right? Wrong. Each context finds different pieces of information relevant. And in each context, the information from other context might be more or less relevant. So what I actually want to do is to link each new site to my core identity, have a uri for each profile field or section, and be able to choose which ones get exposed in which other contexts.

This is one of the places where the corporate world may be a little ahead of the open web. When Socialtext started working on social networking, it was clear to us that we weren’t going to be the core source of basic data like a user’s name, address, and phone number – that information was managed by the corporate directory. Instead, what we would do would be to allow organizations to mirror data from their directory and other sources into Socialtext profiles, and then allow administrators to add new fields, too. The model is that the profile is a shell; you can pull things from elsewhere or create them here. And the profile you create here can be pulled elsewhere using a REST API, also.

In order to have social networking that reflects the irreducably distributed nature of our identities, the elements of the profile need to be pulled apart and reassembled, not just mirrored from place to place.

What Twitter won’t kill – RSS and cultural depth

At the Peninsula Dim Sum Tweetup, Chris Heuer expressed some concern that the shift toward realtime status and realtime search would usher in a dark new age of cultural amnesia in which people’s attention is riveted to the trivially new and we lose the ability to understand in context. This concern is a cultural expression of the technical prediction that in the age of Twitter, Google is dead, RSS is dead, non-realtime content is dead.

I’m less concerned. Sure, the shiny realtime is taking an increasing amount of attention share. And sure, the decentralized digg of realtime meme- and link-sharing is creating a powerful new toolset for discovery. But I strongly suspect that the internet will continue to foster interest in the deep as well as the shiny, and that they are connected. Also, the improved understanding of the value of streams will increase the use of of non-real time streams.

When twitter or some other realtime stream exposes a meme or a link, you can do a few things – pass it on, “like” it up, engage in current conversation, save it for later, and dive into the topic. Realtime helps you participate in the current conversation. Google search helps with the deeper dive.

There reason I’m not worried about amnesia is Clay Shirky’s math about the creative surplus. When hundreds of millions of people are networked, it only takes a very small number of more deeply engaged people to create Linux and Wikipedia. Sure, many more people may see and spread a meme than the number of people who dive deeply into it and organize around it. But math and accessibility favor the deep dive. Not many people will do it, but enough people will, and enough people will connect with each other, that we’ll get the depth and the action.

Let me give just one tiny example. Twitter, and specialized tools like Blip.fm and Last.fm make it easy to share immediate music updates. And then Google, Youtube, Myspace, Last.fm, Amazon and many more sites make it super-easy to find more about the music and get further into it. The stream is the tip of of the iceberg.

As for realtime displacing slower streams, the key is a better shared understanding of attention. Yes, realtime feeds replace RSS for immediate information, because it’s more immediate. But there are other, less-real-time uses for information feeds. As Lee Bryant and Shell Israel observe, RSS is a glue technology for exposing streams of information in relevant context, when there isn’t a need for immediate notice.

The first question is why is a stream needed at all. Is a source of information visible enough to the people closest to it, and are they the only people who want the information? No need for a stream. Do you want the information to be more visible, and visible to more people, get a stream. Do you want to share the information in a different context? Embed a feed. Does it not matter if the information is an hour old? Use an RSS feed. These stream management practices will become better understood, and the popularity of live streams will accelerate learning about how to use streams overall.