[Non-fiction/business] [******]
The Innovator's Dilemma is one of two books that I've read more than once in the last three years, and I expect to read it several more times in the course of my life. It covers several interrelated topics including disruption, value networks, shadow prices, resource dependence, and commoditization. It covers each of these topics well enough that it would be well worth reading to see what Christensen has to say about any one of them, the combination, and there common connection that constitutes the innovator's dilemma makes it indispensable reading for anyone remotely interested in business, economics, investing, evolution, or even artificial intelligence. Rereading this book a couple months ago, I realized just how profoundly it has affected the way I think. No other book that I've read in the past ten years has come close to affecting me as much as this one has.
The Innovator's Dilemma is well-written and well-organized, with a good index. It has excellent graphs and illustrations and compelling data. More importantly, the book's claims and analysis clearly fit the data. When there are lines in a graph, they are transparently meaningful. Frequently, in soft disciplines like social sciences, business, and economics, you see regression analyses of weak correlations that are just not very convincing. Most lines just don't fit most of data very well. This is never the case in The Innovator's Dilemma. When there's a line fit to the data in a graph, it's because the line fits the data tightly, not because there's a weak correlation that someone has happened to discover.
The Innovator's Dilemma is a book about why successful companies sometimes succumb to competition created by newcomers to their industry. The basic thesis is that they're chasing a local maximum, and the entrant competitors are chasing a different local maximum which happens to be higher and the existing companies can't beat the competition by continuing to improve at what they do best because they're chasing the wrong peak (under the circumstances).
I typically advocate caution against making arguments based that rely on non-cyclical, non-monotonic trends. People are good at spotting cycles, and they are good at noticing monotonic progressions, they are good at rationalizing away information that they disagrees with what they want to believe, and they have a tendency to overfit data and see patterns even in noise. The general arguments that people usually give when they are claiming that some phenomenon is neither cyclical nor monotonic (apart from noise) leaves too much room for motivated cognition and is too likely to involve overfitting the data. It's too easy to say "sometimes one must take one step backwards in order to take two steps forward" and dismiss evidence that contradicts your stance when you really ought to acknowledge that it contradicts your stance. Christensen does an excellent job of describing an actual non-monotonic trend correctly, and further of explaining why the existence of these sorts of trends causes so many problems for business.
Non-monotonic, non-cyclical trends do exist. Most local maxima are not the global maximum. A theory that deals with these type of phenomena needs to do so explicitly. Hill-climbing algorithms only find local maxima, and people who want to find a higher maximum need to find a different hill which involves going away from that maximum. A good discussion of non-monotonic trends ought typically be a discussion of hill-climbing algorithms. The Innovator's Dilemma is an excellent example of how to discuss this sort of subject well. (Even though it deals extensively with the manufacture of computer components, it is not a book about computer science so it doesn't really on this sort of language to make its points, but nonetheless it does make them.) Christensen describes value networks and resource dependence as two phenomena that essentially convert any well-run company in a competitive industry into a hill-climbing algorithm for optimizing what it produces within in its context. Companies that are operated by other principals fail if they are operating in a competitive market. However, because most local maxima are not global maxima, companies that are well-run get destroyed whenever another company that starts off climbing a different hill with a higher maximum competes with them. Christensen makes the argument much better than any brief summary can, but that's the essential theme of the book.
Christensen is writing about companies, but most of what he discusses is applicable to a wide range of other fields including biology and artificial intelligence. Have you ever wandered why mammals and birds seem to be able to live in a much wider range of habitats than fish, amphibians, or reptiles? This book gives me a better framework for answering that question than any book I've read that deals only with biology. You would think that since their is an aquatic niche for species as big as whales that fish would have found it before mammals did since they've had many more generations of optimizing for various aquatic roles. But they couldn't find the niches that mammals could adapt to on land because evolution is a hill-climbing algorithm and hill-climbing algorithms only find local maxima. I'm sure that many books that are actually about biology make this point in passing, but I've never read a book of biology that makes the point as forcefully or as well as The Innovator's Dilemma.
The other thing that the Innovator's Dilemma does exceptionally well is that it actually gives you a framework for thinking about how big the hills are and whether an attempt to reach a higher maximum is actually climbing the same hill or not. The general idea is that you have to be optimizing for criteria that is useless to the major extant market if you want to be disrupting the major players there. Hard drive manufacturers didn't care about making smaller hard drives because the manufactures of the server/desktop/notebook/whatever they supplied had other components built to use a specific-sized hard drive and didn't need a smaller one, were willing to pay a premium for more data stored on a bigger hard drive for a while, and weren't willing to a pay a premium to have a smaller hard drive. Eventually, however, because everything is improving, the smaller hard drive become good enough at everything that the larger hard drives do better, that people do want to make the switch. The discussion of what it means for a feature to become "good enough" in this context is like everything else in this book, very solid.
I did have one small nitpick that might annoy biologists. The Innovator's Dilemma opens with incorrect information about the lifespan of the fruitfly. Christensen quotes a colleague as saying that they "are conceived, born, mature, and die all within a single day." (Which is technically also ungrammatical.) This is simply not true. Fruitflies (Drosophila melanogaster, the species studied as a model organism) have a lifespan of about a month and spend a full four days as a larva and another four as a pupa, so it takes about ten days to produce a generation. As a cautionary tale, if you're going to include incorrect information about a field unrelated to the book you are writing, please don't do it as the very first thing you say. If that mistake had been anywhere else in the book, it wouldn't be worth mentioning, but seeing as it is in the opening line, it does call into question a little bit how well the fact-checking in the book was conducted. That said, most of the rest of the book is original research, and as previously stated, the evidence it presents for its findings is pretty compelling.
The only other possible complaint I might have is that there may be a little bit of overfitting in the graph of the bucket size of the largest available hydraulic excavator. The graph fits the data with two lines when it should probably just use one. There really is no evidence to suggest that fitting that data to two lines instead of one was justified. The point would be the same either way, and there would still be a good exponential fit with exactly one line. (It's a log scale plot so a line is an exponential fit.) It doesn't affect anything else in the book, but it does slightly contradict what I said about the solidness of the data analysis. But again, this is a tiny nitpick. I mention it only because it might annoy people who are interested in AI and sensitive to people overfitting data. Neither the mistake about the fruitfly nor this questionable fit have any impact on the subject matter. Both could be corrected and still support exactly the same message to approximately the same degree as the errant version.
All of this is to say that The Innovator's Dilemma is almost perfect. I wouldn't have commented on any minuscule details if I had weightier criticisms to levy. If you haven't read The Innovator's Dilemma you should read it; and if you have read it, I'd recommend that you read it again.
