Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference

by Judea Pearl

Probabilistic Reasoning in Intelligent Systems is a complete and accessible account of the theoretical foundations and computational methods that underlie plausible reasoning under uncertainty. The author provides a coherent explication of probability as a language for reasoning with partial belief and offers a unifying perspective on other AI approaches to uncertainty, such as the Dempster-Shafer formalism, truth maintenance systems, and nonmonotonic logic. The author distinguishes syntactic and semantic approaches to uncertainty--and offers techniques, based on belief networks, that provide a mechanism for making semantics-based systems operational. Specifically, network-propagation techniques serve as a mechanism for combining the theoretical coherence of probability theory with modern demands of reasoning-systems modular declarative inputs, conceptually meaningful inferences, and parallel distributed computation. Application areas include diagnosis, forecasting, image interpretation, multi-sensor fusion, decision support systems, plan recognition, planning, speech recognition--in short, almost every task requiring that conclusions be drawn from uncertain clues and incomplete information. Probabilistic Reasoning in Intelligent Systems will be of special interest to scholars and researchers in AI, decision theory, statistics, logic, philosophy, cognitive psychology, and the management sciences. Professionals in the areas of knowledge-based systems, operations research, engineering, and statistics will find theoretical and computational tools of immediate practical use. The book can also be used as an excellent text for graduate-level courses in AI, operations research, or applied probability.

Genres: Science, Artificial Intelligence, Computer Science, Mathematics, Nonfiction, Textbooks, Philosophy

584 pages, Paperback

First published September 1, 1988

About the author

Judea Pearl (Hebrew: יהודה פרל) is an Israeli-American computer scientist and philosopher, best known for championing the probabilistic approach to artificial intelligence and the development of Bayesian networks.

Reviews

[Gavin · Rating 4 out of 5]

probability is not really about numbers, it is about the structure of reasoning

-Glen Shafer

By no means an introductory book; even chapter 1 will mean little to you if you haven't tried to model situations with both formal logic and probabilities before. (Some set theory wouldn't go amiss either.) Parts of it treat nearly-irrelevant dead controversies, just because he was still fighting off the McCarthy / production systems programme in the late Eighties. (For instance, I learned Dempster-Shafer theory in class, and it is sorta interesting and neatly evades Cox's theorem, but I still expect never to have to use it. It gets more than 50 pages here.) Bayesian networks, ingenious and progressive as they were, have peaked in use, though their children are still cutting edge and invaluable for human and nonhuman reasoning.

All that said: Pearl thinks very hard about ultimate matters. He didn't develop Bayesian networks (and causal models) as a hack, but instead as a consequence of showing probabilities to be better than the alternatives when tweaked for computation, subjective Bayesianism to be capable of handling causal inference, graphs as the natural data structure for both relevance and cause, and the causal/evidential decision theory distinction as primal.

On the surface, there is really no compelling reason that beliefs, being mental dispositions about unrepeatable and often unobservable events, should combine by the laws of proportions that govern repeatable trials such as the outcomes of gambling devices. The primary appeal of probability theory is its ability to express useful qualitative relationships among beliefs and to process these relationships in a way that yields intuitively plausible conclusions… What we wish to stress here is that the fortunate match between human intuition and the laws of proportions is not a coincidence. It came about because beliefs are formed not in a vacuum but rather as a distillation of sensory experiences...

We therefore take probability calculus as an initial model of human reasoning from which more refined models may originate, if needed. By exploring the limits of probability in machine implementations, we hope to identify conditions under which extensions, refinements and simplifications are warranted.

Building AI as feedback for formal epistemology! My favourite philosophers are technical like David Lewis; my favourite technical people are philosophical like Pearl.

He's also very good at taking us through a derivation and underlining the big implications (e.g. P(A) = \sum P(A|B_i) P(B_i) as a model for hypothetical reasoning: belief in event A is a weighted sum over belief in all the ways A can obtain). There's plenty of maths in here but I never struggled much, probably because of this qualitative care of his.

PRIS beats the arse off his own 2018 effort, perhaps because at this point he was still working incredibly hard to understand and synthesise competing approaches. Hard to rate. But if you want to seriously think about AI, you'll want to read it at some point.

----------------------------------

Misc notes

* McCarthy is to probabilities as Minsky is to neural nets. He sent us down a rabbit hole, chasing nonmonotonic logic solutions to a numerical problem. (See also Chomsky vs prob language models.)
* Others have used Pearl's vision to explain the ideal form of rationality, which humans depart from.
* His discussion of extensional (hacking out a generalised logic) vs intensional (possible-world counting) approaches to uncertainty seems fundamental to me, bedrock.
* The heart of the matter: Bayesnets are O(n) in variables, but actually with some complicated tweaks so are Dempster-Shafer galleries.
* Dempster-Shafer is an interesting example of the contingency of (the context of discovery of) mathematics. It didn't have to be developed (since probs are adequate for so much), and yet it was, and it evades the normal arguments against other uncertainty measures and is thus alive, if unpopular. (Compare noneuclidean geometries.) What other dominant calculi would get similarly competing theories, if we threw a few decades of brilliance at them?

[Todd Johnson · Rating 4 out of 5]

I'd give the first chapter or two of this book 5 stars. It's very well written, and the material is exceptionally well motivated. Pearl gives the "why" in a lot of places where others give only the "how" or the "what." That being said, the book shows its age in places. Apparently Daphne Koller and some others are writing a book which aims to replace this one, and which treats modern subjects such as iterated/generalized belief propagation and sampling more fully. Until then, this is a great place to begin studying graphical models, as long as it's supplemented with more recent papers.

[Moshe]

I have a lot to learn about probabilistic reasoning.

[Neil]

If you read Judea Pearl's "Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference" then you will see that the basic insight behind graphical models is indispensable to problems that require it. (It's not something that fits on a T-Shirt, I'm afraid, so you'll have to go and read the book yourself. I haven't seen any online popularizations of Bayesian networks that adequately convey the reasons behind the principles, or the importance of the math being exactly the way it is, but Pearl's book is wonderful.) - EY, https://www.lesswrong.com/s/3HyeNiEpv...

[Manoj Joshi · Rating 4 out of 5]

The approach is scientific and interesting especially concluding the book on discussions between a logicist and probabilistic! I learnt a lot especially the art of incising an event, the process and think it out in all means of probabilistic reasoning, applying in my field of VUCA research. Thx Judea

[Gary Lang · Rating 4 out of 5]

Great book on a topic that Dr. Pearl can rightly claim he pioneered the most useful approach for.

[Catwalker · Rating 5 out of 5]

I got this book as a reference on the use of probability and statistical distributions in modelling. I found it useful, but beyond that I was impressed by the clarity of the writing, the useful examples, and the discussions around the use of the various techniques.
The book is intended to describe the author's work on the applicability of probabilistic methods to AI that relies on automated reasoning under uncertainty. Many of the techniques are based on Bayesian inference. There is a chapter on the basics of Bayesian methods, and a comparison of Markov and Bayesian networks.
The book includes a number of less technical discussions aimed at giving the casual reader an overview of the issues in dealing with uncertainty in AI and decision networks. I found these sections the most interesting, as my interest was in understanding the advantages and limitations of models developed by others (rather than developing my own). These sections are indicated in the Table of Contents.

[Zach · Rating 4 out of 5]

Good thorough examples and clear writing put this book ahead of most textbooks' treatment of related subjects. Despite its early publication date, it is very forward-thinking, even in terms of computational paradigms, so that it seemed perhaps even more relevant today than at the time of writing.