Clay Shirky

The Collapse of Complex Business Models

Clay Shirky - Thu, 04/01/2010 - 19:50

I gave a talk last year to a group of TV executives gathered for an annual conference. From the Q&A after, it was clear that for them, the question wasn’t whether the internet was going to alter their business, but about the mode and tempo of that alteration. Against that background, though, they were worried about a much more practical matter: When, they asked, would online video generate enough money to cover their current costs?

That kind of question comes up a lot. It’s a tough one to answer, not just because the answer is unlikely to make anybody happy, but because the premise is more important than the question itself.

There are two essential bits of background here. The first is that most TV is made by for-profit companies, and there are two ways to generate a profit: raise revenues above expenses, or cut expenses below revenues. The other is that, for many media business, that second option is unreachable.

Here’s why.

* * *

In 1988, Joseph Tainter wrote a chilling book called The Collapse of Complex Societies. Tainter looked at several societies that gradually arrived at a level of remarkable sophistication then suddenly collapsed: the Romans, the Lowlands Maya, the inhabitants of Chaco canyon. Every one of those groups had rich traditions, complex social structures, advanced technology, but despite their sophistication, they collapsed, impoverishing and scattering their citizens and leaving little but future archeological sites as evidence of previous greatness. Tainter asked himself whether there was some explanation common to these sudden dissolutions.

The answer he arrived at was that they hadn’t collapsed despite their cultural sophistication, they’d collapsed because of it. Subject to violent compression, Tainter’s story goes like this: a group of people, through a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex—agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive—each additional bit of complexity more than pays for itself in improved output—but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The ‘and them some’ is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn’t these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn’t because they don’t want to, it’s because they can’t.

In such systems, there is no way to make things a little bit simpler – the whole edifice becomes a huge, interlocking system not readily amenable to change. Tainter doesn’t regard the sudden decoherence of these societies as either a tragedy or a mistake—”[U]nder a situation of declining marginal returns collapse may be the most appropriate response”, to use his pitiless phrase. Furthermore, even when moderate adjustments could be made, they tend to be resisted, because any simplification discomfits elites.

When the value of complexity turns negative, a society plagued by an inability to react remains as complex as ever, right up to the moment where it becomes suddenly and dramatically simpler, which is to say right up to the moment of collapse. Collapse is simply the last remaining method of simplification.

* * *

In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT’s famous “five 9’s” reliability (services that work 99.999% of the time) would be valuable, but they couldn’t figure out how $20 a month, then the going rate, could cover the costs for good web hosting, much less leave a profit.

I started describing the web hosting I’d used, including the process of developing web sites locally, uploading them to the server, and then checking to see if anything had broken.

“But if you don’t have a staging server, you’d be changing things on the live site!” They explained this to me in the tone you’d use to explain to a small child why you don’t want to drink bleach. “Oh yeah, it was horrible”, I said. “Sometimes the servers would crash, and we’d just have to re-boot and start from scratch.” There was a long silence on the other end, the silence peculiar to conference calls when an entire group stops to think.

The ATT guys had correctly understood that the income from $20-a-month customers wouldn’t pay for good web hosting. What they hadn’t understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn’t very good.

This, for the ATT guys, wasn’t depressing so much as confusing. We finished up the call, and it was polite enough, but it was perfectly clear that there wasn’t going to be a consulting gig out of it, because it wasn’t a market they could get into, not because they didn’t want to, but because they couldn’t.

It would be easy to regard this as short-sighted on their part, but that ignores the realities of culture. For a century, ATT’s culture had prized—insisted on—quality of service; they ran their own power grid to keep the dial-tone humming during blackouts. ATT, like most organizations, could not be good at the thing it was good at and good at the opposite thing at the same time. The web hosting business, because it followed the “Simplicity first, quality later” model, didn’t just present a new market, it required new cultural imperatives.

* * *

Dr. Amy Smith is a professor in the Department of Mechanical Engineering at MIT, where she runs the Development Lab, or D-Lab, a lab organized around simple and cheap engineering solutions for the developing world.

Among the rules of thumb she offers for building in that environment is this: “If you want something to be 10 times cheaper, take out 90% of the materials.” Making media is like that now except, for “materials”, substitute “labor.”

* * *

About 15 years ago, the supply part of media’s supply-and-demand curve went parabolic, with a predictably inverse effect on price. Since then, a battalion of media elites have lined up to declare that exactly the opposite thing will start happening any day now.

To pick a couple of examples more or less at random, last year Barry Diller of IAC said, of content available on the web, “It is not free, and is not going to be,” Steve Brill of Journalism Online said that users “just need to get back into the habit of doing so [paying for content] online”, and Rupert Murdoch of News Corp said “Web users will have to pay for what they watch and use.”

Diller, Brill, and Murdoch seem be stating a simple fact—we will have to pay them—but this fact is not in fact a fact. Instead, it is a choice, one its proponents often decline to spell out in full, because, spelled out in full, it would read something like this:

“Web users will have to pay for what they watch and use, or else we will have to stop making content in the costly and complex way we have grown accustomed to making it. And we don’t know how to do that.”

* * *

One of the interesting questions about Tainter’s thesis is whether markets and democracy, the core mechanisms of the modern world, will let us avoid complexity-driven collapse, by keeping any one group of elites from seizing unbroken control. This is, as Tainter notes in his book, an open question. There is, however, one element of complex society into which neither markets nor democracy reach—bureaucracy.

Bureaucracies temporarily suspend the Second Law of Thermodynamics. In a bureaucracy, it’s easier to make a process more complex than to make it simpler, and easier to create a new burden than kill an old one.

In spring of 2007, the web video comedy In the Motherhood made the move to TV. In the Motherhood started online as a series of short videos, with viewers contributing funny stories from their own lives and voting on their favorites. This tactic generated good ideas at low cost as well as endearing the show to its viewers; the show’s tag line was “By Moms, For Moms, About Moms.”

The move to TV was an affirmation of this technique; when ABC launched the public forum for the new TV version, they told users their input “might just become inspiration for a story by the writers.”

Or it might not. Once the show moved to television, the Writers Guild of America got involved. They were OK with For and About Moms, but By Moms violated Guild rules. The producers tried to negotiate, to no avail, so the idea of audience engagement was canned (as was In the Motherhood itself some months later, after failing to engage viewers as the web version had).

The critical fact about this negotiation wasn’t about the mothers, or their stories, or how those stories might be used. The critical fact was that the negotiation took place in the grid of the television industry, between entities incorporated around a 20th century business logic, and entirely within invented constraints. At no point did the negotiation about audience involvement hinge on the question “Would this be an interesting thing to try?”

* * *

Here is the answer to that question from the TV executives.

In the future, at least some methods of producing video for the web will become as complex, with as many details to attend to, as television has today, and people will doubtless make pots of money on those forms of production. It’s tempting, at least for the people benefitting from the old complexity, to imagine that if things used to be complex, and they’re going to be complex, then everything can just stay complex in the meantime. That’s not how it works, however.

The most watched minute of video made in the last five years shows baby Charlie biting his brother’s finger. (Twice!) That minute has been watched by more people than the viewership of American Idol, Dancing With The Stars, and the Superbowl combined. (174 million views and counting.)

Some video still has to be complex to be valuable, but the logic of the old media ecoystem, where video had to be complex simply to be video, is broken. Expensive bits of video made in complex ways now compete with cheap bits made in simple ways. “Charlie Bit My Finger” was made by amateurs, in one take, with a lousy camera. No professionals were involved in selecting or editing or distributing it. Not one dime changed hands anywhere between creator, host, and viewers. A world where that is the kind of thing that just happens from time to time is a world where complexity is neither an absolute requirement nor an automatic advantage.

When ecosystems change and inflexible institutions collapse, their members disperse, abandoning old beliefs, trying new things, making their living in different ways than they used to. It’s easy to see the ways in which collapse to simplicity wrecks the glories of old. But there is one compensating advantage for the people who escape the old system: when the ecosystem stops rewarding complexity, it is the people who figure out how to work simply in the present, rather than the people who mastered the complexities of the past, who get to say what happens in the future.

Categories: Clay Shirky

Secretary Clinton’s Internet Freedom Speech, Abridged

Clay Shirky - Sat, 01/23/2010 - 22:26

[Ed. note: I attended Secretary Clinton's speech on internet freedom on Thursday the 21st, which I thought was a good combination of principle, policy, and illustrative stories. Talking to people afterwards, the commonest question was "What did the Secretary commit the State Department to?" The text below is my attempt to answer that question.

I don't have any inside information about the particulars of the State department's plans; the text below is simply an abridged version of the speech, from which I removed everything except statements you could judge future actions of the State Department on. Stripped of it's context (and with my apologies to the speech writers), my read of the speech is that the success or failure of our internet freedom policy will come down to our ability to live up to the principles outlined below. -clay]

[Topic Sentence]

We stand for a single internet where all of humanity has equal access to knowledge and ideas.

[Internet freedom as an aspect of human rights]

We need to synchronize our technological progress with our principles. In accepting the Nobel Prize, President Obama spoke about the need to build a world in which peace rests on the inherent rights and dignities of every individual. And in my speech on human rights at Georgetown a few days later, I talked about how we must find ways to make human rights a reality. Today, we find an urgent need to protect these freedoms on the digital frontiers of the 21st century.

[Freedom of expression]

But the internet is a network that magnifies the power and potential of all others. And that’s why we believe it’s critical that its users are assured certain basic freedoms. Freedom of expression is first among them. This freedom is no longer defined solely by whether citizens can go into the town square and criticize their government without fear of retribution. Blogs, emails, social networks, and text messages have opened up new forums for exchanging ideas, and created new targets for censorship.

[Exceptions for incitement to violence; disapproval of hate speech]

Now, all societies recognize that free expression has its limits. We do not tolerate those who incite others to violence, such as the agents of al-Qaida who are, at this moment, using the internet to promote the mass murder of innocent people across the world. And hate speech that targets individuals on the basis of their race, religion, ethnicity, gender, or sexual orientation is reprehensible.

[Acknowledgment of hard cases, which are nevertheless subordinate to peaceful political uses]

And we must also grapple with the issue of anonymous speech. Those who use the internet to recruit terrorists or distribute stolen intellectual property cannot divorce their online actions from their real world identities. But these challenges must not become an excuse for governments to systematically violate the rights and privacy of those who use the internet for peaceful political purposes.

[Freedom of worship]

But the freedom of worship also speaks to the universal right to come together with those who share your values and vision for humanity. In our history, those gatherings often took place in churches, synagogues, mosques and temples. Today, they may also take place on line.

Now, just as these technologies must not be used to punish peaceful political speech, they must also not be used to persecute or silence religious minorities.

We must work to advance the freedom of worship online just as we do in other areas of life.

[International coordination on cybersecurity]

We need more tools to help law enforcement agencies cooperate across jurisdictions when criminal hackers and organized crime syndicates attack networks for financial gain. The same is true when social ills such as child pornography and the exploitation of trafficked women and girls online is there for the world to see and for those who exploit these people to make a profit. We applaud efforts such as the Council on Europe’s Convention on Cybercrime that facilitate international cooperation in prosecuting such offenses. And we wish to redouble our efforts.

Countries or individuals that engage in cyber attacks should face consequences and international condemnation.

[Freedom to connect, likened to freedom of assembly]

The final freedom, one that was probably inherent in what both President and Mrs. Roosevelt thought about and wrote about all those years ago, is one that flows from the four I’ve already mentioned: the freedom to connect - the idea that governments should not prevent people from connecting to the internet, to websites, or to each other.

The freedom to connect is like the freedom of assembly, only in cyberspace.

[USG support for the broad goals]

The United States is committed to devoting the diplomatic, economic, and technological resources necessary to advance these freedoms.

[Support for dissidents; return to theme of human rights]

And I’m proud that the State Department is already working in more than 40 countries to help individuals silenced by oppressive governments. We are making this issue a priority at the United Nations as well, and we’re including internet freedom as a component in the first resolution we introduced after returning to the United Nations Human Rights Council.

[USG funding research into freedom-supporting tools]

We are also supporting the development of new tools that enable citizens to exercise their rights of free expression by circumventing politically motivated censorship. We are providing funds to groups around the world to make sure that those tools get to the people who need them in local languages, and with the training they need to access the internet safely.

[USG support for the broad goals II]

Both the American people and nations that censor the internet should understand that our government is committed to helping promote internet freedom.

[USG funding research into freedom-supporting tools II]

That’s why today I’m announcing that over the next year, we will work with partners in industry, academia, and nongovernmental organizations to establish a standing effort that will harness the power of connection technologies and apply them to our diplomatic goals. By relying on mobile phones, mapping applications, and other new tools, we can empower citizens and leverage our traditional diplomacy. We can address deficiencies in the current market for innovation.

And the State Department will be launching an innovation competition to give this work an immediate boost. We’ll be asking Americans to send us their best ideas for applications and technologies that help break down language barriers, overcome illiteracy, connect people to the services and information they need.

[Pro-freedom agenda in the market]

Increasingly, U.S. companies are making the issue of internet and information freedom a greater consideration in their business decisions. I hope that their competitors and foreign governments will pay close attention to this trend.

And censorship should not be in any way accepted by any company from anywhere. And in America, American companies need to make a principled stand. This needs to be part of our national brand.

[Support for Global Internet Freedom Task Force]

Now, we are reinvigorating the Global Internet Freedom Task Force as a forum for addressing threats to internet freedom around the world, and we are urging U.S. media companies to take a proactive role in challenging foreign governments’ demands for censorship and surveillance. The private sector has a shared responsibility to help safeguard free expression.

[February 2010 meeting with network services firms]

As part of our commitment to support responsible private sector engagement on information freedom, the State Department will be convening a high-level meeting next month co-chaired by Under Secretaries Robert Hormats and Maria Otero to bring together firms that provide network services for talks about internet freedom, because we want to have a partnership in addressing this 21st century challenge.

Categories: Clay Shirky

A Rant About Women

Clay Shirky - Sat, 01/16/2010 - 03:46

So I get email from a good former student, applying for a job and asking for a recommendation. “Sure”, I say, “Tell me what you think I should say.” I then get a draft letter back in which the student has described their work and fitness for the job in terms so superlative it would make an Assistant Brand Manager blush.

So I write my letter, looking over the student’s self-assessment and toning it down so that it sounds like it’s coming from a person and not a PR department, and send it off. And then, as I get over my annoyance, I realize that, by overstating their abilities, the student has probably gotten the best letter out of me they could have gotten.

Now, can you guess the gender of the student involved?

Of course you can. My home, the Interactive Telecommunications Program at NYU, is fairly gender-balanced, and I’ve taught about as many women as men over the last decade. In theory, the gender of my former student should be a coin-toss. In practice, I might as well have given him the pseudonym Moustache McMasculine for all the mystery there was. And I’ve grown increasingly worried that most of the women in the department, past or present, simply couldn’t write a letter like that.

This worry isn’t about psychology; I’m not concerned that women don’t engage in enough building of self-confidence or self-esteem. I’m worried about something much simpler: not enough women have what it takes to behave like arrogant self-aggrandizing jerks.

Remember David Hampton, the con artist immortalized in “Six Degrees of Separation”, who pretended he was Sydney Poitier’s son? He lied his way into restaurants and clubs, managed to borrow money, and crashed in celebrity guest rooms. He didn’t miss the fact that he was taking a risk, or that he might suffer. He just didn’t care.

It’s not that women will be better off being con artists; a lot of con artists aren’t better off being con artists either. It’s just that until women have role models who are willing to risk incarceration to get ahead, they’ll miss out on channelling smaller amounts of self-promoting con artistry to get what they want, and if they can’t do that, they’ll get less of what they want than they want.

There is no upper limit to the risks men are willing to take in order to succeed, and if there is an upper limit for women, they will succeed less. They will also end up in jail less, but I don’t think we get the rewards without the risks.

* * *

When I was 19 and three days into my freshman year, I went to see Bill Warfel, the head of grad theater design (my chosen profession, back in the day), to ask if I could enroll in a design course. He asked me two questions. The first was “How’s your drawing?” Not so good, I replied. (I could barely draw in those days.) “OK, how’s your drafting?” I realized this was it. I could either go for a set design or lighting design course, and since I couldn’t draw or draft well, I couldn’t take either.

“My drafting’s fine”, I said.

That’s the kind of behavior I mean. I sat in the office of someone I admired and feared, someone who was the gatekeeper for something I wanted, and I lied to his face. We talked some more and then he said “Ok, you can take my class.” And I ran to the local art supply place and bought a drafting board, since I had to start practicing.

That got me in the door. I learned to draft, Bill became my teacher and mentor, and four years later I moved to New York and started doing my own design work. I can’t say my ability to earn a living in that fickle profession was because of my behavior in Bill’s office, but I can say it was because I was willing to do that kind of thing. The difference between me and David Hampton isn’t that he’s a con artist and I’m not; the difference is that I only told lies I could live up to, and I knew when to stop. That’s not a different type of behavior, it’s just a different amount.

And it looks to me like women in general, and the women whose educations I am responsible for in particular, are often lousy at those kinds of behaviors, even when the situation calls for it. They aren’t just bad at behaving like arrogant self-aggrandizing jerks. They are bad at behaving like self-promoting narcissists, anti-social obsessives, or pompous blowhards, even a little bit, even temporarily, even when it would be in their best interests to do so. Whatever bad things you can say about those behaviors, you can’t say they are underrepresented among people who have changed the world.

Now this is asking women to behave more like men, but so what? We ask people to cross gender lines all the time. We’re in the middle of a generations-long project to encourage men to be better listeners and more sensitive partners, to take more account of others’ feelings and to let out our own feelings more. Similarly, I see colleges spending time and effort teaching women strategies for self-defense, including direct physical aggression. I sometimes wonder what would happen, though, if my college spent as much effort teaching women self-advancement as self-defense.

* * *

Some of the reason these strategies succeed is because we live in a world where women are discriminated against. However, even in an ideal future, self-promotion will be a skill that produces disproportionate rewards, and if skill at self-promotion remains disproportionately male, those rewards will as well. This isn’t because of oppression, it’s because of freedom.

Citizens of the developed world have an unprecedented amount of freedom to choose how we live, which means we experience life as a giant distributed discovery problem: What should I do? Where should I work? Who should I spend my time with? In most cases, there is no right answer, just tradeoffs. Many of these tradeoffs happen in the market; for everything from what you should eat to where you should live, there is a menu of options, and between your preferences and your budget, you’ll make a choice.

Some markets, though, are two-sided — while you are weighing your options, those options are also weighing you. People fortunate enough to have those options quickly discover that it’s not enough to decide you want to go to Swarthmore, or get money out of Kleiner Perkins. Those institutions must also decide if they will have you.

Some of the most important opportunities we have are in two-sided markets: education and employment, contracts and loans, grants and prizes. And the institutions that offer these opportunities operate in an environment where accurate information is hard to come by. One of their main sources of judgment is asking the candidate directly: Tell us why we should admit you. Tell us why we should hire you. Tell us why we should give you a grant. Tell us why we should promote you.

In these circumstances, people who don’t raise their hands don’t get called on, and people who raise their hands timidly get called on less. Some of this is because assertive people get noticed more easily, but some of it is because raising your hand is itself a high-cost signal that you are willing to risk public failure in order to try something.

That in turn correlates with many of the skills the candidate will need to actually do the work — to recruit colleagues and raise money, to motivate participants and convince skeptics, to persevere in the face of both obstacles and ridicule. Institutions assessing the fitness of candidates, in other words, often select self-promoters because self-promotion is tied to other characteristics needed for success.

It’s tempting to imagine that women could be forceful and self-confident without being arrogant or jerky, but that’s a false hope, because it’s other people who get to decide when they think you’re a jerk, and trying to stay under that threshold means giving those people veto power over your actions. To put yourself forward as someone good enough to do interesting things is, by definition, to expose yourself to all kinds of negative judgments, and as far as I can tell, the fact that other people get to decide what they think of your behavior leaves only two strategies for not suffering from those judgments: not doing anything, or not caring about the reaction.

* * *

Not caring works surprisingly well. Another of my great former students, now a peer and a friend, saw a request from a magazine reporter doing a tech story and looking for examples. My friend, who’d previously been too quiet about her work, decided to write the reporter and say “My work is awesome. You should write about it.”

The reporter looked at her work and wrote back saying “Your work is indeed awesome, and I will write about it. I also have to tell you you are the only woman who suggested her own work. Men do that all the time, but women wait for someone else to recommend them.” My friend stopped waiting, and now her work is getting the attention it deserves.

If you walked into my department at NYU, you wouldn’t say “Oh my, look how much more talented the men are than the women.” The level and variety of creative energy in the place is still breathtaking to me, and it’s not divided by gender. However, you would be justified in saying “I bet that the students who get famous five years from now will include more men than women”, because that’s what happens, year after year. My friend talking to the reporter remains the sad exception.

Part of this sorting out of careers is sexism, but part of it is that men are just better at being arrogant, and less concerned about people thinking we’re stupid (often correctly, it should be noted) for trying things we’re not qualified for.

Now I don’t know what to do about this problem. (The essence of a rant, in fact, is that the ranter has no idea how to fix the thing being ranted about.) What I do know is this: it would be good if more women see interesting opportunities that they might not be qualified for, opportunities which they might in fact fuck up if they try to take them on, and then try to take them on. It would be good if more women got in the habit of raising their hands and saying “I can do that. Sign me up. My work is awesome,” no matter how many people that behavior upsets.

Categories: Clay Shirky