The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos (Basic Books, 2015) 329 pages
When was the last time you saw a 12-word subtitle, I ask you? But after reading the book, I think a better subtitle would be “How Professor Domingos Took Bayes’ Theorem and Generalized It for Big Data.” And that’s still 12 words.
This is the “#1 Bestseller” in “Mathematical & Statistical Software” at Amazon. Which makes me sort of wonder how many categories they actually have and what the, say, #10 bestseller in that category might be.
Let me be clear: if you want to understand current research into artificial intelligence (AI), read this book. You might not understand everything, especially the discussion of Bayes theorem, but you’ll know enough to cut through the misconceptions.
What misconceptions? The ones about AI that I keep seeing online and hear when I talk to other people. Full disclosure: I have done PhD-level work in AI and work for a high-tech Silicon Valley company. I used to be a complete AI skeptic, but Domingos has convinced me not to be and that the whole field would be better off if it were called “machine learning” instead of AI, for reasons I will detail later.
What kind of AI misconceptions? Well, I saw one today from Tech Crunch (who should know better). In an article on “programming hate into AI” (HERE), the author says things like anything below the level of “coding in empathy” into an AI agent results in a “follower machine.”
The fallacy in this way of thinking is that you don’t really program anything “into” an AI agent. The whole point is to create a learning environment through rules or analogies or other methods and then “train” it to compile knowledge that can be used to answer questions about the world. For example, if we want an AI agent to be an expert in fruit, we can make rules like “if it’s round and red and…, then it’s an apple” or associations like “an apple has the characteristics round and red and…” or some other form of learning process. Then we can show the AI a picture and ask “Is this thing an apple?” and, if it’s done right, the response would be something like “it sure looks like an apple to me.” Intelligent robots and AI agents, as the book’s blurb mentions, “program themselves.”
And that’s the value of this book: it lets you cut through the nonsense written today about the future of AI and figure out what is really going on in the field today. Domingos is a professor and, like any good professor, he does a terrific job of introducing and explaining complex concepts to people who are not familiar with them.
The book surveys five different approaches to machine learning AI and attempts to combine them into a single “Master Algorithm” based on Domingos’s work with Bayes theorem and Markov logic networks as a unifier of all previous methods to achieve AI (page 246). Here are the five areas that Domingos details (they are nicely related on page 54, but the descriptions I took from page 291):
Symbolists who deal in “inverse deduction.” Starting on page 57, Domingos goes through the types of rules of induction that make up spam filters and the algorithms that banks use to detect fraud (“If your credit card was used to purchase one dollar of gas, it was stolen”, page 69). Is it 100% certain? No, and the problem over overfitting is a common one in AI (page 71). But the more apples our AI sees, the more it will be able to eliminate other round, red fruits like cherries or strawberries.
Connectionists who deal in “backpropagation.” Starting on page 93, Domingos goes over researchers who create AI algorithms based on “neural networks” and how the brain functions. One key concept that contributes to Domingos’s approach here is the “S-curve” (page 106). So our AI apple-detector will often err at first, then get a lot better, and then improve hardly at all.
Evolutionaries who deal in “genetic algorithms.” Starting on page 121, Domingos explores how learning systems can evolve themselves to become better. These are the people who create robot warriors that make versions of themselves that are smarter and smarter using 3D printers, a project that the military actually considered, according to Domingos. Domingos uses this as the starting point for his “Master Algorithm.”
Bayesians who deal in “probabilistic inference.” Starting on page 143, Domingos gets to the heart of his method, which is based on Markov logic networks. He derives MLNs from Bayes’s method of refining and weighing evidence (page 171). I have never seen a better explanation of what Bayes’ theorem actually does in action. Basically, the more real apples our apple AI sees, as opposed to plastic models or drawings, the better it should get (but it can never be 100% correct, because Bayes applies).
Analogizers who deal in “support vector machines.” Starting on page 177, Domingos points out that even a fake doctor, if given enough case histories, can match a set of symptoms with the most likely disease. So no matter how many green apples our apple AI sees, there will be enough other parameters to make a firm identification. This is where too many dimensions become an issue (page 188).
From page 203 to the end of the book, Domingoes shows how all five AI “tribes” can be combined to yield a more promising version of machine learning to form a foundation for the Master Algorithm. We might not have it yet, but we are getting closer (page 292).
To finish up this long essay, here’s why I think that most AI research should be done under the rubric of machine learning instead of AI. The problem with identifying all of AI with so-called “expert systems” is that AI has fragmented into many different fields, and these vary from those who can show you something working today to those who hope that someday, someway, their brand of AI might come to be. So here are my five “AI Tribes,” ranked from those I consider to be scientists working on firm ground to people who are fantasizing based on flimsy evidence and a lot of wishful thinking:
Machine Learning: What the Master Algorithm is all about. It’s not so much that a computer is AI smart, but that the whole network mining big data will know things about you that even you don’t know (like that you always pick up a case of beer when your spouse sends you for diapers). These are the most grounded and down-to earth of all AI researchers. Their smart robots are here already, out in the Internet, and you can download Weka open-source software or Domingos’s Alchemy machine learning AI environment (page 250)and use it today..
Robot Warriors: Science fiction writer Isaac Asimov’s “robot laws” make it impossible for smart robots to harm humans. Not only were these laws not foolproof (the whole point of his robot stories) but modern robots like military drones exist mainly to kill other humans. The big question today is not if robot soldiers will be able to kill humans – that’s the idea – but if they can do so without a human granting permission. It seems unlikely the delay would be feasible when patrolling mean streets after an invasion. Some fantasize that robot soldiers will only shoot and kill other robots, but it’s hard to see how this “rule” could ever be enforced in a world that still shrugs at wartime rape and torture.
Singularity Soldiers: These guys are in the Ray Kurzweil camp. They call the day when computers become smarter than people the “singularity” (Kurzweil did not invent the term, which computer scientist and science fiction writer Vernor Vinge used in a 1993 essay (page 286), but Kurzweil certainly popularized it). Then perhaps the machines will rise up and crush us, or decide to head for the stars, or keep us as pets. (The best exercise for AI algorithms might be to answer the question of what AI agents will do to humans.)
Consciousness Crusaders: I’m not even sure these guys are separate from the singularity brigade, or if they should be ranked before them. But I put them in their own category because their main goal is not to separate AIs from human society, but sort of to create a separate form of consciousness that we can use to better understand what it is to be human. Researchers here often posit some form of consciousness to other species like apes, dolphins, elephants, or even dogs. This may be so, but I’m not sure who might be interested in creating a robot version of the late, beloved Fido.
Upload Believers: These are the people who think that we can upload the “brains” of dying people into computers and then perhaps download them into other bodies (procured from…?). I remain skeptical that whatever it is that makes us human or conscious (or both) can be separated in any meaningful way from the sensory input system that is our body. It’s possible that you can in some fashion, but I doubt the new entity would be “you.” These guys deal more in religious fervor than scientific rigor, I think.