Squeezed Books
Business Knowledge - Extracted, Compressed, Discussed
Summary: The Wisdom of Crowds
Differences between revision 4 and 5
Key Points
The thesis of the book, as the author states at the beginning:
under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them. Groups do not need to be dominated by exceptionally intelligent people in order to be smart.
Summary
Part I
Chapter 1
The book begins with a series of examples of “crowd wisdom”, ranging from the TV show “Who Wants to be a Millionaire?” and its “ask the audience”, to the stock market indicating the company most likely to be at fault hours after the Challenger disaster. These cases all demonstrate the four conditions that comprise wise crowds - independence, diversity of opinion, decentralization, and a way to aggregate the results. Similar results are to be found in sports betting, and in Google’s results, determined by examining the number of links pointing to any given page. One way to take advantage of this wisdom of crowds is through the use of ”prediction markets”, such as the Iowa Electronic Markets, where people buy and sell probabilities as if they were stocks. In the right circumstances, prediction markets are an excellent way of turning the knowledge of many people into reasonably accurate predictions.
Chapter 2
The importance of diversity is covered in the second chapter. A crowd can’t be wise if everyone always picks the same answer as everyone else. Examples include product markets, where there is usually an initial wide range of different attempts in a new market, which is quickly winnowed down to the successful designs; and honey bees, which send out scouts in all directions, but only return to those areas where flowers have been found. Diversity is important to “wise crowds”, because it expands the range of possible solutions proposed. In large groups, diversity comes naturally, but in smaller groups, it’s necessary to support and actively encourage it, to avoid the dangers of “groupthink”. When people give in to their conformist tendencies, and are afraid to stick their necks out, the quality of decisions suffers.
Chapter 3
Independence of action and thought is important for the wisdom of crowds. If everyone thinks alike, then they’re less likely to arrive at a good answer to a given problem, because they’re less likely to fall into “groupthink”. “The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes”.
American Football coaching is cited an example of the “herd mentality”, based on the work of David Romer examining the “best 4th down strategy” (pdf). It turns out that statistically, most teams would be better off trying to make the touchdown or 1st down, rather than going for the field goal, in many cases. However, since the accepted wisdom is to kick, going against the grain of the relatively small pool of decision makers (professional football coaches) would not be an easy choice to make consistently, especially for the risk averse.
Herding behavior often occurs because people seek safety in numbers, but it can lead to problematic results when independence is required. “Information cascades” are what occurs when an initial decision is made by a few people, and then more or less accepted uncritically by more and more people. This isn’t necessarily a recipe for disaster, as we can’t all evaluate everything in our lives, but must trust others to come to good conclusions. However, at times, it can be disastrous when the original information and decisions were wrong, but continue to be accepted by an ever-wider circle. Luckily, for most people, the more important a decision is, the more likely they are to examine the facts themselves, rather than simply fall in line. Information cascades actually work reasonably well much of the time, but the basic problem is that they are a sequential, rather than parallel process. If you’re trying to harness the wisdom of crowds, you must attempt to have all decisions made at the same time, rather than one at a time.
Chapter 4
This chapter covers decentralization - where it works, where it doesn’t and what can go wrong. Decentralized, aggregate behavior is a key aspect of things like free market economies, flocks of birds, and is something that has been touted as a virtuous way of running a company as of late, with small, self-organizing teams. Decentralization allows people, or more generally, components of a system, to act freely and independently of one another, and still interact to produce coordinated results.
Linux is cited as an example of a decentralized system with a central aggregator - Linus Torvalds. As most people know by this point, Linux is worked on collaboratively by many programmers throughout the world, but often, different people come up with competing solutions to the same problem. This is good at finding and testing diverse approaches to see, in practice rather than in theory, which one actually works the best. Ultimately, however, the ‘best’ solutions are not selected by popular vote, but by Linus, who is responsible for taking the results of the decentralized development process, and aggregating them into something useful by selecting the ‘best’ bits and pieces.
Also discussed is the decentralization of the intelligence community, and the negatives involved in the difficulty of sharing information, cited as one factor in the failure of the intelligence community to predict and prevent the 9/11 attacks. The problem, however, was not decentralization, but decentralization with no way to aggregate the results into something useful.
One such way of aggregating information was a proposed futures market based on potential events in the Middle East, and elsewhere, which was, however, not allowed to get off the ground due to squeamishness about the idea of buying and selling bets about, say, a leader’s chance of being assassinated in any particular year. This market could have been a useful tool, perhaps not in predicting precise events, but in collecting information about the general state of things in places where information is at times difficult to gather, and unfettered freedom of expression suppressed.
Links
The book’s web site: http://www.randomhouse.com/features/wisdomofcrowds/
Wikipedia page on the book: http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds