Algorithms
The structures every program stands on, the algorithms that run on them, the proof that each is correct and its bound tight, and the six honest responses to a problem that has no fast algorithm. Every count on this path is produced by running the algorithm on the input in front of you. Nine courses and 112 lessons are available.
- Courses
- 9
- Lessons
- 112
- Level
- Advanced
- Assumes
- assumes Discrete Mathematics in full
Overview
Data Structures comes first because every later bound is quoted in terms of one. Prim and Dijkstra are a heap with a schedule, Kruskal is union–find with a sort, and the sweep line of the geometry course is a search tree that changes as the line moves. A reader who has not costed a heap operation by running it will read O(E log V) as a label rather than as a count of something.
Sorting and Selection comes second because it is where the analysis becomes probabilistic. Randomised quicksort’s exact expectation is the model for every argument in Randomised Algorithms, and the comparison lower bound Discrete Mathematics proved is used twice more here — by the adversary arguments and by the convex-hull lower bound — as a thing to reduce to.
Graph Algorithms precedes Greedy Algorithms and Matroids rather than following it, because the cut property and Kruskal are the instance the matroid theorem generalises, and a general theorem read before its instance is a definition rather than an insight. Dynamic Programming then follows Greedy for the same reason in the other direction: 0/1 knapsack is interesting because the density rule that solved the fractional problem has just failed on it.
Strings, Geometry and Randomised Algorithms are three applications of the first five courses and could be taken in any order; they sit here so that Intractability and Approximation, which needs all of them, comes last. It needs the probabilistic method to derandomise, subset-sum’s dynamic program to explain weak NP-hardness, König’s theorem to explain a tractable special case, and the two proof templates because an approximation ratio is a stays-ahead argument against a lower bound.
Courses
Each course assumes the ones before it and Discrete Mathematics in full, and nothing else. Data Structures pays for the operations Sorting and Graph Algorithms spend; Greedy and Dynamic Programming are two answers to the same question about optimal substructure, in that order because the boundary greed fails at is where the table begins; Strings, Geometry and Randomised Algorithms each apply the first five courses to one world; and Intractability consumes all eight.
Discrete Mathematics analysed algorithms and never built a structure to run one on. This course builds eight, and costs each by executing an operation sequence rather than by quoting a class — because the interesting facts here are all about the difference between a worst case, an amortised bound and an expectation, and those three are indistinguishable from a single fast run.
Discrete Mathematics proved no comparison sort beats n log n. That leaves open everything that matters: stability, whether the bound applies at all, what randomness buys, and how close to the bound a real sort gets. This is also the course where the analysis becomes probabilistic — randomised quicksort's exact expectation is the pattern every later expectation on this path follows.
Discrete Mathematics defined graphs and ran two algorithms on them. This course builds eleven, proves each correct, and counts what each one does on the graph you type. It is also where an algorithm first certifies its own answer: everywhere else on this path a lesson puts a measured count beside a proved bound, and here the maximum flow comes back with a cut of equal capacity attached.
This is the one course on the path where a lesson can do better than putting a measured count beside a proved bound. A greedy rule commits to a choice and never revisits it, so on a small instance the optimum is enumerable — and every verdict here is computed twice, once by running the rule and once by a search that has never heard of it. When the two disagree the lab names the subset that beats the rule, which is what a counterexample looks like when it is produced rather than quoted.
Greedy Algorithms and Matroids proved a rule correct wherever the structure allowed it. This course is what to do where it does not: the same optimal substructure, without the greedy choice property, evaluated by filling a table. Every page runs the table against at least one route that has no table in it — an exhaustive search, a memo-free recursion, or a re-costing of the object that came back — because a filled table is the easiest thing on this path to get confidently wrong.
This is the course where the asymptotically better algorithm loses, three times, on inputs one click apart. Naive matching beats Knuth–Morris–Pratt on prose with an absent pattern; Horspool beats both by a factor of three on English and loses to naive matching by a factor of five on a run of one letter; a rolling hash reports three positions that are not matches until the characters are compared. None of the bounds moves while any of that happens, and every page prints both columns.
Every other course on this path measures an algorithm. This one measures a PREDICATE: the same orientation test in exact integers and in double precision, on 29 magnitudes of the same input, with the wrong answers counted rather than warned about. The failure turns out to have a shape — the rounded test can lose a turn and provably cannot invent the opposite one — and the rest of the course is what that sign is used for, with degenerate input treated as the subject rather than excluded from it.
Everywhere else on this path a measured count sits beside a proved bound. Here the thing being bounded is itself random, and the easiest mistake in the subject is to run an algorithm a hundred times, take the average, and call it the expectation. So the labs enumerate: every tape of a shuffle, every execution of a quicksort, every contraction order, every base, every assignment. The exact fraction and the sample sit in the same row, and the gap between them is what the course is about.
Every course so far built algorithms that work. This one starts where none is known to, and it does two things with that. First it makes hardness constructive: a reduction is a construction, so each one here is built in front of you, both instances are solved exhaustively, and the map between their solution sets is checked for soundness, for injectivity and for surjectivity as three separate questions — because two of them have answers that change with the instance. Then it builds the six things that can honestly be done instead, and for each one it prints the algorithm's answer, the true optimum from an exhaustive search, the realised ratio and the promise, on one axis.
Recommended background
Discrete Mathematics in full, and its course on Algorithms and Complexity as a hard prerequisite. Specifically: big-O with witnesses, analysis of iterative algorithms, recursion trees and amortised analysis, the master theorem, the comparison lower bound, greedy and dynamic programming on one example each, complexity classes and reductions as definitions. Also, by name: strong and structural induction, recurrence relations, loop invariants; expected value, linearity of expectation, independence, variance, the geometric distribution; modular arithmetic and exponentiation, Fermat and Euler, the seeded generator; Graphs and Trees in full; satisfiability and normal forms. Each course names the slugs it uses.
From Algebra, only the-coordinate-plane and the 2×2 determinant, both for Geometric Algorithms, and the determinant is defined inline there so that a reader who has not taken Algebra’s Systems and Matrices loses nothing.
No programming. Every lesson reads pseudocode in Discrete Mathematics’ conventions — one-based indices, = for assignment — and none asks you to write or run code. The labs execute the algorithm so that you can watch it and count what it does.
Willingness to distrust a count you just watched. The lab measures cases; the lesson proves a claim about all of them. Every misconception on this path is a case mistaken for a class — a hash table that was fast on the keys you tried, a pivot rule that survived the inputs you chose, an approximation that matched the optimum on the instance in front of you.