Lecture Notes and Further Reading

  1. Some models of sequential computation (handout and overhead presentation). Do not linger too much on Turing machines; we will mostly use the random access machine throughout the course. You may need to remember these machines though for at least two reasons: (a) they offer the simplest definition for nondeterminism, and (b) computations on Turing machines are defined as graphs of configurations, which has the merit of allowing the use of typical graph algorithms (such as traversal) to analyze computations.

  2. NP-complete problems (handout and overhead presentation). The main resource for this section is Chapter 34 from Introduction to Algorithms by Cormen et. al.

    The path to proving that SAT is NP complete (through Bounded tiling) is taken from Elements of the Theory of Computation (Lewis and Papadimitriou, second edition, call number QA 267 .L49 1998). Note that Cormen et. al. use satisfiability of Boolean circuits (CIRCUIT-SAT) instead. The proof that Bounded tiling is NP-complete is simpler and more intuitive (hence my choice for this course). This being said, CIRCUIT-SAT is interesting in practice (as opposed to Bounded tiling) and the reduction presented in Cormen et. al. starts from the RAM rather than the Turing machine so it is all well worth reading. It is worth mentioning that Cook proved that SAT is NP-complete from scratch (without going through any intermediate problem).

    The second part of the notes discusses a number of proofs of NP-completeness which are taken from Cormen et. al. The authoritative source of NP-complete problems is Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson (available in the library).

    The original paper by S. A. Cook is available directly from its author here. Here is a version retyped using a modern typesetting system and so much more readable.

  3. Approximation algorithms (handout and overhead presentation). Most (but not all) of the material is discussed according to Chapter 35 from Introduction to Algorithms by Cormen et. al.

    A much more detailed presentation of approximation schemes can be found in Approximation Schemes - A Tutorial by Schuurman and Woeginger. Many other papers on the matter exist (Google is your friend). It is worth noting that most presentations on the matter start from the Knapsack problem.

    I do not have a definite reference for backtracking or branch and bound, but virtually all AI textbooks cover them. Here are some references: Chapter 2: Backtracking from Algorithms by J. Erickson for backtracking and Branch and Bound Algorithms - Principles and Examples by J. Clausen for branch and bound.

    Applying backtracking on the kind of problems already discussed in class can also be seen in Algorithms and Complexity by H. S. Wilf. His Chapter 5 covers the whole matter discussed so far in the course so it may be interesting as an alternate point of view to the whole section on NP-completeness and approximation algorithms.

  4. Linear programming (handout and overhead presentation). This material is presented according to Cormen et. al., though in a substantially abbreviated manner.

  5. Computational geometry (handout and overhead presentation). The basic processing algorithms (cross product, etc.) and also the many-segment intersection algorthm are taken from Cormen et. al., which also covers the Graham scan and gift wrapping convex hull algorithms.

    An excellent overview of convex hull algorithms is Lecture 3: More Convex Hull Algorithms by David M. Mount, which also includes a detailed discussion on lower bounds (only available through the Way Back Machine these days; a somehow simplified version is shown in (the second part of) Lecture 3 and Lecture 4 from CMSC 754: Computational Geometry by the same David M. Mount). These documents present Quickhull directly with the Akl-Toussaint optimization. Quickhull is introduced in The quickhull algorithm for convex hulls by C. Barber Bradford, D. P. Dobkin, and H. Huhdanpaa (ACM Transactions on Mathematical Software, 22:4 (1996), 469-483). The Akl-Toussaint heuristic is introduced in A fast convex hull algorithm by (you guessed it) S. G. Akl and G. T. Toussaint (Information Processing Letters, 7 (1978), 219-222) which unfortunately does not appear to be available on line. The original paper actually presents a complete algorithm for finding the convex hull. The performance of the algorithm is further studied in A note on linear expected time algorithms for finding convex hulls by L. Devroye and G. T. Toussaint, (Computing, 26 (1981), 361-366).

    The Wikipedia pages for gift wrapping, Graham scan, Quickhull, and Chan's algorithm are not bad at all and also provide cute animated demonstrations of the respective algorithms.

  6. Models of parallel computation (handout and overhead presentation). These lectures cover most of Chapters 1 and 2 from Parallel Computation: Models and Methods by S. Akl Various painstaking details of the innards of the PRAM and interconnection networks presented in Chapter 2 are intentionally glossed over and will be discussed later on an as-needed basis.

    As a motivating example of parallel approaches to algorithm design these lectures also cover Chapter 4 up to Section 4.8 (inclusive).

  7. Divide and conquer on the PRAM (handout and overhead presentation). These lectures cover most of Chapter 5 from Akl. Merging is presented according to The Design and Analysis of Parallel Algorithms (by S. G. Akl, Prentice Hall, 1989), though the resulting algorithm is similar (but not quite identical) to the one shown in Chapter 5. Sorting algorithms are mostly taken from The Design and Analysis of Parallel Algorithms. To see how rank sorting can be done more efficiently check out Fast Parallel Sorting Algorithms by D. S. Hirschberg (Communications of the ACM, 21 (1978), 657-661). This paper also describes an efficient (and clever) parallel version of bucket sort.

  8. Some algorithms on interconnection networks (handout and overhead presentation). This material is taken from the beginning of Akl's Chapter 7 and roughly the first half of Chapter 9. Some more network topologies are presented according to Section 2.3.

  9. The parallel computation thesis (handout and overhead presentation). These lectures cover Chapter 5 from Parallel Complexity Theory by I. Parberry (Prentice Hall, 1987).

    A substantially more in-depth discussion on this matter is Limits to Parallel Computation: P-Completeness Theory by R. Greenlaw, H. J. Hoover, and W. L. Ruzzo (Oxford University Press, 1995). This book presents the complete picture (though a bit of background from Parberry is still useful even if not strictly required) including a compendium of P-complete (and thus probably non-parallelizable) problems.

Summary on parallel computation: Here is a writeup on parallel computation that covers the whole second half of the term. While it does not contain much additional information compared to the lecture notes, this information is presented in a possibly more convenient format.

Lecture recording

My lectures are recorded, as follows: