Speaker: Rina Dechter
Title: Probabilistic Reasoning Meets Heuristic Search
Abstract:
Graphical models, including constraint networks, Bayesian networks, Markov random fields and inuence diagrams, have become a central paradigm for knowledge representation and reasoning in Artificial Intelligence, and provide powerful tools for solving problems in a variety of application domains, including coding and information theory, signal and image processing, data mining, learning, computational biology, and computer vision. Although past decades have seen considerable progress in algorithms in graphical models, many real-world problems are of such size and complexity that they remain out of reach. Advances in exact and approximate inference methods are thus crucial to address these important problems with potential impact across many computational disciplines. Exact inference is typically NP-hard, motivating the development of approximate and anytime techniques.
Existing algorithms typically take one of two approaches: Inference, expressed as message-passing schemes, or search and conditioning methods. In the past decade, my research group has developed several very successful algorithms for reasoning in graphical models based on combining heuristic search with message passing approximations. These algorithms are state-of-the-art. For example, in the past two approximate inference competitions (The Probabilistic Inference Challenge in UAI, 2012 and 2014) our algorithms were evaluated as leading at either first or second place. My plan is to describe the main principles underlying these developments.
In this talk I will describe the unifying framework of AND/OR search spaces for graphical models and show how they can facilitate problem decomposition and allow avoiding redundant specification (and computation) by exploiting the conditional independencies of the graphical model. The AND/OR search space size is bounded exponentially by the graphical models treewidth, making them comparable to inference algorithms such as variable-elimination or join/junctiontree decompositions. Yet, as search schemes they can facilitate many of the ideas developed within Heuristic Search and Operations Research communities, for all queries (e.g., satisfiability, optimization, weighted counting and their combinations) while enjoying the treewidth bound, automatically. Most significantly, they can allow trading off memory for time and time for accuracy, leading to anytime solvers.
I will subsequently show how approximate inference can be used for generating heuristic information for guiding the AND/OR search. This can be accomplished using the two ideas of mini-bucket partitioning schemes which relaxes the input problem by node duplication only, combined with linear programming relaxations ideas which optimize cost-shifting re-parameterization, to yield tight bounding heuristic information within systematic, anytime search.
AND/OR search guided by such inference-based heuristics has yielded some of the best state-of-the-art solvers for combinatorial optimization. Current research centers on extending these ideas beyond pure optimization, to weighted counting and to hybrid queries (i.e., max-product or min-sum queries such as marginal map and maximizing expected utilities in inuence diagrams), aiming for anytime behavior that generates not only an approximation, but also upper and lower bounds which become tighter with more time.