Lecture Notes and Further Reading

- Some models of sequential computation
(handout
and
overhead presentation). Do not
linger too much on Turing machines; we will mostly use the random
access machine throughout the course. You may need to remember
these machines though for at least two reasons: (a) they
offer the simplest definition for nondeterminism, and (b)
computations on Turing machines are defined as graphs of
configurations, which has the merit of allowing the use of typical
graph algorithms (such as traversal) to analyze computations.
- NP-complete problems
(handout
and
overhead presentation). The main
resource for this section is Chapter 34 from Introduction to
Algorithms by Cormen et. al.
The path to proving that SAT is NP complete (through Bounded tiling) is taken from Elements of the Theory of Computation (Lewis and Papadimitriou, second edition, call number QA 267 .L49 1998). Note that Cormen et. al. use satisfiability of Boolean circuits (CIRCUIT-SAT) instead. The proof that Bounded tiling is NP-complete is simpler and more intuitive (hence my choice for this course). This being said, CIRCUIT-SAT is interesting in practice (as opposed to Bounded tiling) and the reduction presented in Cormen et. al. starts from the RAM rather than the Turing machine so it is all well worth reading. It is worth mentioning that Cook proved that SAT is NP-complete from scratch (without going through any intermediate problem).

The second part of the notes discusses a number of proofs of NP-completeness which are taken from Cormen et. al. The authoritative source of NP-complete problems is Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson (available in the library).

The original paper by S. A. Cook is available directly from its author here. Here is a version retyped using a modern typesetting system and so much more readable.

- Approximation algorithms
(handout
and
overhead presentation). Most
(but not all) of the material is discussed according to Chapter 35
from Introduction to Algorithms by Cormen et. al.
A much more detailed presentation of approximation schemes can be found in Approximation Schemes - A Tutorial by Schuurman and Woeginger. Many other papers on the matter exist (Google is your friend). It is worth noting that most presentations on the matter start from the Knapsack problem.

I do not have a definite reference for backtracking or branch and bound, but virtually all AI textbooks cover them. Here are some references: Chapter 2: Backtracking from Algorithms by J. Erickson for backtracking and Branch and Bound Algorithms - Principles and Examples by J. Clausen for branch and bound.

Applying backtracking on the kind of problems already discussed in class can also be seen in Algorithms and Complexity by H. S. Wilf. His Chapter 5 covers the whole matter discussed so far in the course so it may be interesting as an alternate point of view to the whole section on NP-completeness and approximation algorithms.

- Linear programming
(handout
and
overhead presentation). This
material is presented according to Cormen et. al., though in a
substantially abbreviated manner.
- Computational geometry
(handout
and
overhead presentation). The
basic processing algorithms (cross product, etc.) and also the
many-segment intersection algorthm are taken from Cormen et. al.,
which also covers the Graham scan and gift wrapping convex hull
algorithms.
An excellent overview of convex hull algorithms is Lecture 3: More Convex Hull Algorithms by David M. Mount, which also includes a detailed discussion on lower bounds (only available through the Way Back Machine these days; a somehow simplified version is shown in (the second part of) Lecture 3 and Lecture 4 from CMSC 754: Computational Geometry by the same David M. Mount). These documents present Quickhull directly with the Akl-Toussaint optimization. Quickhull is introduced in The quickhull algorithm for convex hulls by C. Barber Bradford, D. P. Dobkin, and H. Huhdanpaa (ACM Transactions on Mathematical Software, 22:4 (1996), 469-483). The Akl-Toussaint heuristic is introduced in A fast convex hull algorithm by (you guessed it) S. G. Akl and G. T. Toussaint (Information Processing Letters, 7 (1978), 219-222) which unfortunately does not appear to be available on line. The original paper actually presents a complete algorithm for finding the convex hull. The performance of the algorithm is further studied in A note on linear expected time algorithms for finding convex hulls by L. Devroye and G. T. Toussaint, (Computing, 26 (1981), 361-366).

The Wikipedia pages for gift wrapping, Graham scan, Quickhull, and Chan's algorithm are not bad at all and also provide cute animated demonstrations of the respective algorithms.

- Models of parallel computation
(handout
and
overhead presentation). These
lectures cover most of Chapters 1 and 2 from Parallel Computation:
Models and Methods by S. Akl Various painstaking details of the innards of the PRAM and
interconnection networks presented in Chapter 2 are intentionally
glossed over and will be discussed later on an as-needed basis.
As a motivating example of parallel approaches to algorithm design these lectures also cover Chapter 4 up to Section 4.8 (inclusive).

- Divide and conquer on the PRAM
(handout
and
overhead presentation). These
lectures cover most of Chapter 5 from Akl. Merging is presented according to The Design and Analysis of
Parallel Algorithms (by S. G. Akl, Prentice Hall, 1989), though
the resulting algorithm is similar (but not quite identical) to the
one shown in Chapter 5. Sorting algorithms are mostly taken from
The Design and Analysis of Parallel Algorithms. To see how
rank sorting can be done more efficiently check out
Fast Parallel Sorting
Algorithms
by D. S. Hirschberg (Communications of the ACM, 21 (1978), 657-661).
This paper also describes an efficient (and clever) parallel version
of bucket sort.
- Some algorithms on interconnection networks
(handout
and
overhead presentation). This
material is taken from the beginning of Akl's Chapter 7 and roughly
the first half of Chapter 9. Some more network topologies are presented according to Section 2.3.
- The parallel computation thesis
(handout
and
overhead presentation).
These lectures cover Chapter 5 from Parallel
Complexity Theory
by I. Parberry (Prentice Hall, 1987).
A substantially more in-depth discussion on this matter is Limits to Parallel Computation: P-Completeness Theory by R. Greenlaw, H. J. Hoover, and W. L. Ruzzo (Oxford University Press, 1995). This book presents the complete picture (though a bit of background from Parberry is still useful even if not strictly required) including a compendium of P-complete (and thus probably non-parallelizable) problems.

Summary on parallel computation: Here is a writeup on parallel computation that covers the whole second half of the term. While it does not contain much additional information compared to the lecture notes, this information is presented in a possibly more convenient format.

My lectures are recorded, as follows:

- The introduction (Lecture 1) was not recorded
- Models of computation: Lecture 1 (not recorded), Lecture 2
- NP-complete problems: Lecture 3, Lecture 4, Lecture 5, Lecture 6, first about 25 minutes of Lecture 7
- Approximation algorithms: rest of Lecture 7, Lecture 8
- Linear programming: Lecture 9, first about 40 minutes of Lecture 10
- Computational geometry: rest of Lecture 10, Lecture 11, Lecture 12
- Introduction to parallel models and algorithms: Lecture 13, Lecture 14, Lecture 16
- Divide and conquer on the PRAM: Lecture 17, Lecture 18
- Interconnection networks: Lecture 19, Lecture 20, first about 40 minutes of Lecture 21
- The parallel computation thesis: rest of Lecture 21, Lecture 22, Lecture 23
- Wrap up: Lecture 24 does not contain any new matter and so was not recorded.