Browse course

By: Johannes Korbmacher

Boolean satisfiability


In the last chapter, you learned about Boolean algebra and its role as fundamental laws of reasoning and computation. In this chapter, we’ll further explore the applications of Boolean algebra in AI.

In particular, you’ll learn about how to efficiently test for valid inference in Boolean algebra. First, we’ll discuss the “naive” truth-table method, which suffers from the problem of combinatorial explosion. Then you’ll learn about a more efficient method for validity checking, which is known as the Davis–Putnam–Logemann–Loveland (DPLL) algorithm. This methods requires you to know about normal forms, so we’ll cover those, too.

The DPLL algorithm is the basis for many modern AI applications, especially in planning and diagnosis. You’ll explore some of these applications in the exercises.

Validity and Satisfiability


The problem that we’re tackling is how to computationally and efficiently test for valid inference. Why is this a problem?

Remember from Chapter 2. Valid inference that valid inference is truth-preservation from premises to conclusion:

$$P_1,P_2,\dots\vDash C\Leftrightarrow [P_1]\cap [P_2]\cap \dots\subseteq [C]$$

In Chapter 4.5 , we implemented this definition in the context of Boolean algebras. In this setting, the above condition is equivalent to saying that: $$P_1,P_2,\dots\vDash C\Leftrightarrow \text{ for all }\nu,\text{ if }\nu(P_1)=1, \nu(P_2)=1\dots\text{, then }\nu(C)=1.$$

In words, this means that the inference is valid iff for all Boolean valuations, where the premises are all true (have value 1), the conclusion is also true (has value 1). Using this definition, we could formally calculate the validity of an inference.

In Chapter 4.5 , we went through an example calculation to show that:

$$\mathsf{RAIN}\lor \mathsf{BIKE},\neg\mathsf{RAIN}\vDash\mathsf{BIKE}$$

The problem is that this was a “human-style” calculation. How can we teach a (non-intelligent) computer system the necessary skills to test for valid inference itself?

It turns out that for this purpose, it’s useful to reformulate the question in yet another, equivalent way using the concept of Boolean satisfiability. Consider a set of (propositional) formulas $\Gamma\subseteq \mathcal{L}$. We say that $\Gamma$ is satisfiable just in case there exists a Boolean valuation $\nu$ that makes all members of $\Gamma$ true, i.e. $\nu(A)=1$ for all $A\in \Gamma$.

Take the set $\Gamma=\Set{\mathsf{RAIN}\lor \mathsf{SUN},\neg \mathsf{BIKE}}$, for example. If we set $\nu$ such that:

We have:

This valuation shows that the set $\Set{\mathsf{RAIN}\lor \mathsf{SUN},\neg BIKE}$ is satisfiable.

A set that’s not satisfiable is called unsatisfiable. What does an unsatisfiable set look like? Consider $\Gamma=\Set{\mathsf{RAIN},\neg \mathsf{RAIN}, \mathsf{SUN}\land\neg \mathsf{BIKE}}$. Any valuation that makes all members of $\Gamma$ true will be such that:

This is because both $\mathsf{RAIN},\neg \mathsf{RAIN}\in \Gamma$.

But if $\nu(\neg \mathsf{RAIN})=-\nu(\mathsf{RAIN})=1$, this means that $\nu(\mathsf{RAIN})=0$. So our conditions give us that $\nu(\mathsf{RAIN})=1$ and at the same time $\nu(\mathsf{RAIN})=0$. But that’s impossible, meaning that the set is unsatisfiable.

The reason why we’re talking about satisfiability is that there’s a test for valid inference in terms of satisfiability, which looks as follows:

$$P_1,P_2,\dots\vDash C\Leftrightarrow \Set{P_1,P_2,\dots, \neg C}\text{ is \emph{un}satisfiable}$$

The idea is simply that $\Set{P_1,P_2,\dots, \neg C}$ is satisfiable just in case its possible to make all the premises $P_1,P_2,\dots$ true and at the same time the conclusion $C$ false (since $\nu(\neg C)=-\nu(C)=1$ just in case $\nu(C)=0$). To say that this is not possible is to say that we can’t make the premises true and the conclusion false, which is just another way of saying that whenever the premises are true, the conclusion is false.

Take our example from Chapter 4.5 again:

$$\mathsf{RAIN}\lor \mathsf{BIKE},\neg\mathsf{RAIN}\vDash\mathsf{BIKE}$$

We can show this using the previous test by checking whether the following set is satisfiable:

$$\Set{\mathsf{RAIN}\lor \mathsf{BIKE},\neg\mathsf{RAIN},\neg \mathsf{BIKE}}$$

We can quickly see that this set is unsatisfiable:

But that was again “human-style” thinking. How can we computationally implement this in a step-by-step procedure that a computer could follow? That’s what the rest of the chapter is about.

Truth-tables


The first method, we’ll discuss is the “naive” way of simply going through all relevant valuations by brute force.

Take our example from before:

$\Set{\mathsf{RAIN}\lor \mathsf{BIKE},\neg\mathsf{RAIN},\neg \mathsf{BIKE}}$

Since there are 2 propositional variables, there are $2^2=4$ relevant valuations.1 They are:

Now, using the recursion rules from Chapter 4.4 , we can simply calculate the truth-values of all formulas in our set. We get:

A simple inspection shows that in none of these cases, we have all formulas of our set true. Since we’ve inspected all valuations, we can conclude that the set is unsatisfiable.

A truth-table is a way to write down all the truth-values that a formula can take. Here’s how this works:

Here’s how this looks for the formula $(\mathsf{RAIN}\lor(\neg\mathsf{RAIN}\land \mathsf{BIKE}))$:

The truth-table method for validity checking is to use the truth-table for the formula: $$(((P_1\land P_2)\land \dots )\land \neg C)$$

For our example inference, we get:

This shows that the formula in question is unsatisfiable and thus the inference is valid.

The truth-table method is a working method for computationally determining the validity of inferences. So why doesn’t the chapter end here?

The problem is that truth-tables suffer from what’s known as combinatorial explosion. The problem is that the size of the necessary truth-table grows exponentially with the number of propositional variables in the formula, as in the following table:

Propositional variables Rows
1 2
2 4
3 8
4 16
5 32
6 64

Then, for each formula in the parsing tree, we need to calculate the truth-value in a given row. This quickly becomes painfully slow, even with modern hardware and for relatively simple, but real world applications.

We need a more efficient method.

Normal forms


The method we’ll be working with uses normal forms. Generally speaking, a normal form is a special way of writing formulas such that every formula can be re-written in this way. This is much better illustrated concretely, so let’s talk about Conjunctive Normal Forms (CNF), which are what we’re ultimately interested in.

A formula $A$ is in CNF just in case it is a conjunction ($\land$) of disjunctions ($\lor$) of propositional variables ($p,q,r,\dots$) or their negations ($\neg p,\neg q, \neg r,\dots$). This is still quite abstract, but we can work with that using some examples.

The following formulas are in CNF:

The following formulas are not in CNF:

So, some formulas are and some formulas aren’t in CNF. The crucial thing about CNFs, though, which is what make them a normal form, is that every formula can equivalently be re-written in CNF. What we mean by that is that for each formula $A$, there exists a formula $A_{CNF}$ such that for each valuation: $$\nu(A)=\nu(A_{CNF}).$$ A formula that’s equivalent in this way is equivalent for all (logical and AI) intents and purposes.

Here are the equivalent formulas for the non-CNF formulas:

These are easily verified using a truth-table for the two formulas and checking that they always have the same values.

Here’s one case:

As you can see in these two truth-tables: under each possible valuation, the two formulas have exactly the same truth-table.

The fact that each formula can be re-written in CNF is known as the CNF theorem. You’ll not learn the details of the proof, but you’ll learn how to transform formulas into CNFs.

Basically, all you need to do is to apply the following rules as many times as needed until your formula is in CNF:

The way to read these rules is that if you come across any of the left formulas within a non-CNF formula, replace the occurrence with the right formula.

Here’s how the procedure looks like in practice:

$$\mathsf{RAIN}\lor \neg(\mathsf{SUN}\lor \neg \mathsf{BIKE})$$ $$\leadsto\mathsf{RAIN}\lor (\neg\mathsf{SUN}\land \neg\neg \mathsf{BIKE})$$ $$\leadsto(\mathsf{RAIN}\lor \neg\mathsf{SUN})\land (\mathsf{RAIN}\lor \neg\neg \mathsf{BIKE})$$ $$\leadsto(\mathsf{RAIN}\lor \neg\mathsf{SUN})\land (\mathsf{RAIN}\lor \mathsf{BIKE})$$

Before we move to the DPLL algorithm, we need to talk about what CNF formulas look like in general. Above, we defined a CNF as a conjunction of disjunctions of propositional variables or their negations. First, some terminology:

$$\langle lit\rangle ::= \langle prop\rangle\mid \neg\langle prop\rangle$$

A convenient way of talking about CNFs is simply as sets of sets of literals. So we write: $$\Set{\Set{l_1^1,\dots,l_{n_1}^1},\dots \Set{l_1^k,\dots,l_{n_k}^k}}$$ instead of $$(l_1^1\lor\dots\lor l_{n_1}^1)\land\dots\land(l_1^k\lor\dots\lor l_{n_k}^k).$$

That is, there are $1,\dots,k$ clauses:

$$\Set{l_1^1,\dots,l_{n_1}^1}$$ $$\vdots$$ $$\Set{l_1^k,\dots,l_{n_k}^k}.$$

The first clause has length $n_1$, the second one length $n_2$…, and the last, the $k$-th one has length $n_k$.

This is obviously criminally abstract (though unfortunately necessary), but looking at a concrete example hopefully helps understand this quicker. The idea is simply that instead of instead of $$(\mathsf{RAIN}\lor \neg\mathsf{SUN})\land (\mathsf{RAIN}\lor \mathsf{BIKE}),$$ we just write: $$\Set{\Set{\mathsf{RAIN},\neg\mathsf{SUN}},\Set{\mathsf{RAIN},\mathsf{BIKE}}}$$

DPLL


The DPLL algorithm determines whether a given formula in CNF is satisfiable, i.e. whether there’s a Boolean valuation that makes the entire formula true.

Using the notation from before, we assume that our formula looks as follows: $$\Set{\Set{l_1^1,\dots,l_{n_1}^1},\dots \Set{l_1^k,\dots,l_{n_k}^k}}$$

The crucial thing to understand is that in order to make the corresponding formula true, it’s sufficient to make one literal from each clause true.

Let’s start with the “criminally abstract” way of saying what that means: we simply need to find a sequence: $$l_{i_1}^1\in \Set{l_1^1m\dots, l_{n_1}^1}$$ $$\vdots$$ $$l_{i_k}^1\in \Set{l_k^1m\dots, l_{n_k}^k}$$ of literals with a valuation $\nu$ such that $$\nu(l_{i_1})=1$$ $$\vdots$$ $$\nu(l_{i_k})=1$$

The DPLL algorithm is, essentially, a smart way of making this search easier than just going “brute force” through all the possibilities. In some cases, we still need to search through some possibilities, but by thinking about the problem, we can often cut down the search space significantly.

We do so, with the following rules, which we’ll introduce using less abstract examples:

Now we have all the ingredients to describe the DPLL algorithm in general:

It’s a mathematical fact that this algorithm always terminates and gives the correct answer: a valuation that satisfies the formula (we find an empty CNF along the way) or the answer that its unsatisfiable (all CNFs along the way end up with an empty clause).

Applications


The Boolean satisfiability problem (SAT) is one of the most important technical concepts in AI. It’s lies at the core of automated reasoning techniques (“theorem proving”), as well as things like automated planning and scheduling.

You’ve effectively already seen how automated reasoning can work:

This is, at the very core, how many automated reasoning programs, like Prover9, Z3, and Vampire, work.

In the exercises for this chapter, you’ll learn a bit more about the “planning” applications.

Further readings


To be added

Notes:


  1. In general, if there are $n$ propositional variables, there are $2^n$ possible Boolean valuations. ↩︎

  2. Note that $\Set{\Set{\neg \mathsf{SUN}},\Set{\neg\mathsf{SUN}}}=\Set{\Set{\neg \mathsf{SUN}}}$. ↩︎

Last edited: 24/09/2024