CS 381K: Artificial Intelligence: Lecture Notes


Copyright © 2006 by Gordon S. Novak Jr.

Permission is granted for individuals to make copies of these notes for personal use, or for instructors to make copies for classroom use.

Note: Many of these pages use math symbols such as: &forall &exist . Microsoft Internet Explorer will not display the math symbols, but Netscape or Firefox will.

Index

1. CS 381K \ \ Artificial Intelligence

2. Artificial Intelligence as Science

3. Scientific Goals of AI

4. A.I. as Engineering

5. Areas of Artificial Intelligence

6. Perception

7. Robotics

8. Natural Language Understanding:

9. Planning

10. Expert Systems

11. Theorem Proving

12. Symbolic Mathematics

13. Game Playing

14. Characteristics of A.I. Programs

15. Symbolic Processing

16. Knowledge Representation

17. Semantic Networks / Frames

18. Logic

19. Logic ...

20. A.I. is the Future of Computing!

21. Search

22. Why Search is Necessary

23. How Search Fits Into A.I.

24. Outline of Search Topics

25. State Space Search

26. Tic-Tac-Toe as a State Space

27. Search Tree

28. Solving Problems Using Search

29. Basic Recursive Algorithm

30. Basic Depth-first Search Algorithm

31. Comments on Search Algorithm

32. Recursive Depth-First Search

33. State Space Search Program

34. Notes on State Space Search Program

35. Missionaries and Cannibals

36. Missionaries/Cannibals Search Graph

37. Missionaries and Cannibals Representation

38. Functions for Missionaries and Cannibals

39. Testing Missionaries and Cannibals

40. Combinatoric Explosion

41. Do Faster Computers Help?

42. Irrelevant Operators

43. Search Order

44. Search Order

45. Depth-First Search

46. Bounded Depth-First Search

47. Iterative Deepening

48. Cost of Iterative Deepening

49. Using Heuristics to Guide Search

50. Hill Climbing

51. Simulated Annealing

52. Use of Simulated Annealing

53. Example of Simulated Annealing

54. Behavior of Simulated Annealing

55. Breadth-First Search

56. Breadth-First Search Algorithm

57. Uniform-Cost Search

58. Ordered Search

59. The Ordered Search Algorithm

60. Evaluation Functions

61. Heuristic Search: A*

62. Heuristic Search for Route Finding

63. Ordered Search for Route Finding

64. Effect of Heuristic Function

65. Admissibility of Heuristic Function

66. Informed Heuristic Functions

67. Features of Heuristic Functions

68. Heuristic Search Handles Local Maxima

69. Iterative Deepening A* (IDA*)

70. Beam Search: $MA^{*}$ and $SMA^{*}$

71. Forward vs. Backward Search

72. Search Tree vs. Search Graph

73. Symbolic Differentiation

74. Simple Symbolic Differentiation

75. Simple Symbolic Differentiation ...

76. Testing and Incremental Development

77. Symbolic Simplification: Rewrite Rules

78. Symbolic Simplification Programs

79. Incremental Testing

80. Beware of Copying Code!

81. Program vs. Pattern Matching

82. Pattern Matching Simplification

83. Pattern Matching Process

84. Pattern Matching

85. Rules for Pattern Matching

86. Equality vs. Matching

87. Pattern Matching

88. Copying and Substitution Functions

89. Matching and Substitution

90. Auxiliary Functions

91. Transformation Examples

92. Remaining Issues

93. Problem Reduction Search

94. Problem Reduction Search: Flowchart

95. Problem Reduction Representations

96. AND/OR Graph Example

97. Solution to an AND/OR Graph

98. Search of an AND/OR Graph

99. Counterexamples

100. Searching Game Trees

101. Game Trees

102. Static Evaluation Functions

103. Minimax Evaluation

104. Alpha-Beta Search

105. Implementing Alpha-Beta Search

106. Alpha-Beta Search Example

107. Game Tree Search

108. Samuel's Checkers Program

109. General Problem Solver (GPS)

110. Problems with GPS

111. Searching in Abstraction Spaces

112. Chronological Backtracking

113. Dependency Directed Backtracking

114. Genetic Algorithms

115. Outline of Genetic Algorithm

116. Waltz Filtering

117. Waltz Filtering for Vision

118. Waltz Filtering Process

119. Waltz Filtering: Example

120. Waltz Example

124. Constraint Satisfaction Problems

125. CSP as Backtracking Search

126. Constraint Network

127. Optimizing Search

128. Constraint Propagation

129. Other Techniques and Kinds of CSP

130. Difficulty of CSP

131. Eight Puzzle

132. Objectives of Eight-Puzzle Exercise

133. DENDRAL (Buchanan and Feigenbaum)

134. Underlying State Space:

135. Ways to Reduce Search Space: Heuristics

136. Goal Testing:

137. Search as a Basic Technique

138. Where Search Should Fit in an AI System

139. Knowledge Representation and Reasoning

140. Representation Hypothesis

141. Computation as Simulation

142. Alternatives to the Representation Hypothesis

143. Kinds of Knowledge

144. Knowledge Representation System

145. Knowledge Representation

146. Symbolic Representation

147. Retrieval: Matching Problem

148. Knowledge Representation Methods

149. Knowledge Representation: Hard Problems

150. Logic for Artificial Intelligence

151. Logical Representation

152. Propositional Logic

153. Interpretation in Propositional Logic

154. Equivalent Formula Laws

155. Ways to Prove Theorems

156. Backward Chaining

157. Backchaining Example

158. Forward Chaining

159. Forward Chaining Example

160. Resolution

161. Resolution for Propositional Calculus

162. Normal Forms

163. Logical Consequence

164. Resolution for Propositional Calculus

165. Resolution Step for Propositional Calculus

166. Resolution Step

167. Examples of Resolution Step

168. Example: Propositional Calculus Resolution

169. Resolution with Venn Diagrams

170. Example of Resolution with Venn Diagrams

171. Satisfiability Checking

172. Predicate Calculus (First-order Logic)

173. Interpretations in Predicate Calculus

174. Overview of Predicate Calculus Resolution

175. Prenex Normal Form

176. Order of Quantifiers

177. Skolemization

178. Standard Form

179. Proof by Contradiction

180. Herbrand's Theorem

181. Resolution for Predicate Calculus

182. Examples of Unification

183. Substitutions

184. Unification Algorithm

185. Unification Implementation

186. Simple Unification Algorithm

188. Soundness and Completeness

189. Resolution Strategies

190. Resolution Strategies ...

191. Resolution Example

192. Resolution Example

193. Resolution as Syntax

194. Natural Deduction

195. Linear Resolution

196. Backchaining Theorem Prover

198. Question Answering

199. Answer Extraction

200. Planning

201. The Frame Problem

202. Planning: Situation Calculus

203. Operators in Situation Calculus

204. Frame Axioms

205. Planning: STRIPS

206. STRIPS Operators for Blocks World

207. STRIPS: Operator Application

208. Selection of STRIPS Operators

209. Kinds of Planning

210. Criticality Number

211. Plan Monitoring

212. Weaknesses of A.I. Planning

213. Knowledge Rep. in Predicate Calculus

214. Rules

215. Forward Chaining

216. Backward Chaining

217. Importance of Backchaining

218. Reason Maintenance

219. Implementing Retraction

220. Truth Maintenance

221. ATMS

222. Closed World Assumption

223. PROLOG

224. Non-Monotonic Logic

225. Induction and Abduction

226. Predicate Calculus: Representation Language

228. Predicate Calculus as Programming Language

229. When to Use Logic

230. Semantic Networks and Frames

231. Property List Representation

232. Property Lists

233. Advantages of Property Lists

234. Frames

235. ``Frame'' Software Packages

236. Typical Features of Frames

237. Simple Frame Program

238. Example Frame Data

239. Shadowing

240. Multiple Inheritance

241. Example Frame

243. Advantages of Frames

244. Disadvantages of Frames

245. When to Use Frames

246. Object-Oriented Programming

247. Access to Objects

248. Internal Implementation is Hidden

249. Advantages of Objects

250. Object-Oriented Programming vs. Frames

251. Unique Features of Frames

252. Unique Features of Object-Oriented Systems

253. Example: Frame

254. Example: Object

255. Benefits of Frame Idea

256. Execution Time of Messages

257. Scripts

258. Some Languages / Inference Engines

259. Knowledge Bases

260. Overview of Knowledge Representation

261. Situated Action

262. Natural Language Processing (NLP)

263. Why Study Natural Language?

264. Model of Natural Language Communication

265. Minimality of Natural Language

266. Zipf's Law

267. Areas of Natural Language

268. Computer Language Understanding

269. Problems in Understanding Language ...

270. Outline of Natural Language Section

271. Morphology

272. Lexicon

273. Lexical Features

274. Size of Lexicon

275. Statistical Natural Language Processing

276. Part-of-Speech Tagging

277. AI View of Syntax

278. Grammar

279. Language Generation

280. Parsing

281. Ambiguity

282. Sources of Ambiguity

283. Foreign Languages

284. Formal Syntax

285. Notation

286. Phrase Structure Grammar

287. Recognizing Automaton

288. Chomsky Hierarchy of Languages

289. Regular Languages

290. Context Free Languages

291. Context Sensitive Languages

292. What Kind of Language is English?

293. Parsing

294. Top-down Parser

295. Bottom-up Parsing

296. Chart Parser

297. Augmented Transition Networks

298. Augmented Transition Networks

299. Separability of Components

300. Problems with Separability

301. Combining Syntax and Semantics

302. How to Combine Syntax \& Semantics

303. Natural Language as an AI Problem

304. Semantics

305. Semantics ...

306. Ambiguity

307. Reducing Ambiguity

308. Case Theory

309. Case Relations

310. Case Relation Example

311. Case Frames

312. Disambiguation Using Case Frames

313. Preference Semantics

314. Conceptual Dependency

315. Conceptual Dependency: Examples

316. Conceptual Dependency: Evaluation

317. Work by Schank and Students

318. Pronoun Reference

319. Deep Semantics Influences Parsing

320. Reference

321. Referent Identification

322. Scripts and Reference

323. Internal Representation

324. How Not to do Representation

325. Tokens

326. Time Sequence

327. Rhetorical Relations

328. Conclusions on Semantics

329. Simple Language Processing: ELIZA

330. Spectrum of Language Descriptions

331. Semantic Grammar

332. Semantic Grammar: Extended Pattern Matching

333. Example Semantics for a Semantic Grammar

334. Compositional Semantics

335. ATN Written in Lisp

336. Sentence Pointer Handling

337. Word Category Testing

338. Lexicon Example

339. Database Access

340. Building Database Access

341. Parser

342. Phrase Parsing

343. Grammar Compiler

344. An Example Query

345. Example Queries

346. Natural Language Interfaces

347. Problems with NL Interfaces

348. Menu-based Natural Language

349. Statistical NLP

350. Speech Understanding

351. Machine Translation Example

352. Sentence Understanding in ISAAC

359. Preposition Semantics

360. Expert Systems

361. Power-Based Strategy

362. Expert Reasoning

363. Expert Knowledge

364. Choosing a Domain

365. Rule-Based Systems

366. Advantages of Rules

367. EMYCIN

368. Rule-Based Expert Systems

369. Production Systems (OPS-5 family)

370. Recognize-Act Cycle

371. Production System (OPS-5)

372. OPS-5 Data

373. OPS-5 Rules

374. Conflict Resolution

375. Evaluation of OPS-5

376. Reasoning Under Uncertainty

377. Bayes' Theorem

378. Uses of Bayes' Theorem

379. Joint Probability Distribution

380. Chain Rule

381. Bayesian Networks

382. Computing with Bayesian Networks

383. A Heretical View

384. EMYCIN's Certainty Factors

385. Certainty Factor Meaning

386. EMYCIN Data CF's

387. EMYCIN Antecedent CF

388. Rule Certainty Factors

389. Certainty Factor Combination

390. Certainty Factor Combination

391. Summary of CF Computations

392. Contradictions:

393. Duplicate Rules

394. Rule Subsumption

395. Increasing Certainty

396. EMYCIN CF vs. Probability Theory

397. Sensitivity Analysis

398. Expert Systems vs. Decision Trees

399. Machine Learning

400. Rule Induction

401. Sample Decision Tree:

402. Example of Rule Induction

403. Final Decision Tree with Classifications

404. Algorithm for Rule Induction

405. Alternatives for {\tt select-feature}

406. Limitations of Rule Induction

407. Digital Low-Pass Filter

408. Getting Knowledge From Expert

409. Interaction with Expert

410. Conceptual Islands

411. Advantages of Conceptual Islands

412. Expansion with Conceptual Islands

413. Orthogonal Knowledge Sources

414. Neuron

415. Neuron Firing

416. Function of Brain Regions

417. Somatotopy: Sensory and Motor

418. Early Experimental Work

419. Function from Brain Injury

420. Positron Emission Tomography

421. Micro-electrodes

422. Hubel and Wiesel: Visual Cortex

423. Visual Processing Areas

424. Vision

425. Mental Rotation

426. Somatotopic Processing

427. Machine Vision

428. Digitization

429. Low-Pass Filtering

430. High-Pass Filtering

431. Convolution

432. Feature Detection

433. Hough Transform

434. Generalized Hough Transform

435. Segmentation

436. Shape from Shading

437. Lambert's Cosine Law

438. Object Identification

439. Motion

440. Other Sensory Modes

441. Adding Semantics to Vision

442. Practical Vision

443. Theory and Practice

444. Driving a Car

445. Driving: Architecture

446. Driving: Detailed Architecture

447. Control of Spacecraft

448. Remote Agent: Architecture

449. Engine / Valve Configurations

450. Understanding Machines from Movies

451. Understanding Machines: Example


CS 381K