Lecture 12

Logic and neural networks

Johannes Korbmacher Lecture slides

Logical methods for AI

Logic and neural networks

Lecture 12

What you’ll be able to do

Neural networks

la_ai_ann
  • Networks of simple units compute truth-functions.
  • From formulas to networks: build them.
  • From networks to formulas: query them.

Perceptrons

01 zstep(z)

Calculating a perceptron

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Boolean gates

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Linear separability

0001 011 x₁x₂

Exclusive disjunction

w₁ + b > 0, w₂ + b > 0 and b ≤ 0 give w₁ + w₂ + b > 0.

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Hidden layers

x₁ XOR x₂ = (x₁ OR x₂) AND NOT(x₁ AND x₂)

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Universal representation

  • One hidden unit for each row with output 1
  • Row 1, 0, 1: h = step(x₁ − x₂ + x₃ − 1.5)
  • Output unit: y = step(h₁ + … + hₖ − 0.5)

Networks from truth-tables

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Knowledge bases

KB = {R → W, S → W},   (R → W) ∧ (S → W) ≡ (R ∨ S) → W

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.

Supervised learning

ai_learning_language01001111010101011010010101111110101010101101101010110111100000100000000001111111111111000000000
  • Output too low: raise the active weights and the bias
  • Output too high: lower them
  • Linearly separable examples only: never XOR

The bitter lesson

ai_tools
  • Sutton: search and learning scale with computation.
  • They overtake the understanding we build in.
  • So what has a network learned?

Universal approximation

  • sigmoid(z) = 11 + e⁻ᶻ
  • Outputs as probability estimates
  • Finding the parameters is learning.
10.5 −606 zsigmoid(z)

Rule extraction

Enable JavaScript to change the inputs and inspect the network. The formulas and worked examples in the chapter describe its calculation.