# Learn Graph Theory > A free, interactive algorithm visualizer for learning graph theory and operations research. The platform offers 30+ step-by-step algorithm visualizations (BFS, DFS, Dijkstra, MST, TSP, network flow, and more), structured lessons, practice problems, and in-depth articles. Aimed at computer science students, coding-interview candidates, and educators. All content is free and available in English, French, Spanish, German, and Chinese. ## Algorithm Visualizations Interactive, step-by-step visualizations with real-time execution. Each page explains the algorithm, its complexity, and use cases. - [Breadth-First Search (BFS)](https://learngraphtheory.org/algorithm/bfs) - [Depth-First Search (DFS)](https://learngraphtheory.org/algorithm/dfs) - [Dijkstra's Shortest Path](https://learngraphtheory.org/algorithm/dijkstra) - [Bellman-Ford](https://learngraphtheory.org/algorithm/bellman-ford) - [Floyd-Warshall](https://learngraphtheory.org/algorithm/floyd-warshall) - [Prim's Minimum Spanning Tree](https://learngraphtheory.org/algorithm/prim) - [Kruskal's Minimum Spanning Tree](https://learngraphtheory.org/algorithm/kruskal) - [Boruvka's Algorithm](https://learngraphtheory.org/algorithm/boruvka) - [Tarjan's Strongly Connected Components](https://learngraphtheory.org/algorithm/tarjan-scc) - [Kosaraju's Strongly Connected Components](https://learngraphtheory.org/algorithm/kosaraju-scc) - [Topological Sort](https://learngraphtheory.org/algorithm/topological-sort) - [Articulation Points](https://learngraphtheory.org/algorithm/articulation-points) - [Bridges](https://learngraphtheory.org/algorithm/bridges) - [Traveling Salesperson Problem (TSP)](https://learngraphtheory.org/algorithm/tsp) - [Hamiltonian Path](https://learngraphtheory.org/algorithm/hamiltonian-path) - [Eulerian Path](https://learngraphtheory.org/algorithm/eulerian-path) - [Graph Coloring](https://learngraphtheory.org/algorithm/graph-coloring) - [Maximal Clique](https://learngraphtheory.org/algorithm/maximal-clique) - [Bipartite Check](https://learngraphtheory.org/algorithm/bipartite-check) - [Cycle Detection](https://learngraphtheory.org/algorithm/cycle-detection) - [Maximum Flow](https://learngraphtheory.org/algorithm/max-flow) - [Minimum Cut](https://learngraphtheory.org/algorithm/min-cut) - [Chordality](https://learngraphtheory.org/algorithm/chordality) - [Facility Location](https://learngraphtheory.org/algorithm/facility-location) - [K-Means Clustering](https://learngraphtheory.org/algorithm/k-means-clustering) - [Multi-Vehicle Routing](https://learngraphtheory.org/algorithm/multi-vehicle-routing) - [Capacitated Vehicle Routing](https://learngraphtheory.org/algorithm/capacitated-vehicle-routing) ## Lessons Guided lessons that build graph theory understanding from fundamentals to advanced topics. - [Introduction to Graphs](https://learngraphtheory.org/lesson/introduction-to-graphs) - [Fundamental Structures](https://learngraphtheory.org/lesson/fundamental-structures) - [Trees and Applications](https://learngraphtheory.org/lesson/trees-applications) - [Graph Traversal](https://learngraphtheory.org/lesson/graph-traversal) - [Shortest Path Algorithms](https://learngraphtheory.org/lesson/shortest-path-algorithms) - [Minimum Spanning Trees](https://learngraphtheory.org/lesson/minimum-spanning-trees) - [Graph Connectivity](https://learngraphtheory.org/lesson/graph-connectivity) - [Network Flow](https://learngraphtheory.org/lesson/network-flow) ## Articles In-depth explanatory articles covering algorithms, theory, and applications. - [Graph Theory in Biology](https://learngraphtheory.org/articles/graph-theory-biology.html) - [Introduction to Graph Theory](https://learngraphtheory.org/articles/introduction-to-graph-theory.html) - [Topological Sort Interview Questions](https://learngraphtheory.org/articles/topological-sort-interview-questions.html) - [Union-Find Interview Questions](https://learngraphtheory.org/articles/union-find-interview-questions.html) - [Graph Theory: Supply Chain Optimization](https://learngraphtheory.org/articles/graph-theory-supply-chain-optimization.html) - [Graph Theory in Cybersecurity](https://learngraphtheory.org/articles/graph-theory-cybersecurity.html) - [Vertices and Edges Explained](https://learngraphtheory.org/articles/vertices-and-edges.html) - [Directed vs Undirected Graphs Explained](https://learngraphtheory.org/articles/directed-vs-undirected-graphs.html) - [Weighted vs Unweighted Graphs Explained](https://learngraphtheory.org/articles/weighted-vs-unweighted-graphs.html) - [Simple Graphs vs Multigraphs Explained](https://learngraphtheory.org/articles/simple-graphs-vs-multigraphs.html) - [Trees in Graph Theory](https://learngraphtheory.org/articles/trees-in-graph-theory.html) - [Graph Representation Explained](https://learngraphtheory.org/articles/graph-representation.html) - [BFS Interview Questions](https://learngraphtheory.org/articles/bfs-interview-questions.html) - [DFS Interview Questions](https://learngraphtheory.org/articles/dfs-interview-questions.html) - [Dijkstra Interview Questions](https://learngraphtheory.org/articles/dijkstra-interview-questions.html) - [Finite and Infinite Graphs Explained](https://learngraphtheory.org/articles/finite-and-infinite-graphs.html) - [BFS vs DFS: Key Differences](https://learngraphtheory.org/articles/bfs-vs-dfs.html) - [Shortest Path Algorithms](https://learngraphtheory.org/articles/shortest-path-algorithms.html) - [Minimum Spanning Trees (MST)](https://learngraphtheory.org/articles/minimum-spanning-trees.html) - [Traveling Salesperson Problem (TSP)](https://learngraphtheory.org/articles/traveling-salesperson-problem.html) - [Vehicle Routing Problem (VRP)](https://learngraphtheory.org/articles/vehicle-routing-problem.html) - [Max-Flow Min-Cut Theorem & Network Flow](https://learngraphtheory.org/articles/network-flow-max-flow-min-cut.html) - [Graph Coloring Problem](https://learngraphtheory.org/articles/graph-coloring-problem.html) - [A* Search Algorithm: Step-by-Step Guide + Visualizer](https://learngraphtheory.org/articles/a-star-search-algorithm.html) - [A* Search Algorithm in AI: The Complete Guide](https://learngraphtheory.org/articles/a-star-algorithm-in-ai.html) - [Graph Algorithms Time Complexity](https://learngraphtheory.org/articles/graph-algorithms-complexity.html) - [History of Graph Theory](https://learngraphtheory.org/articles/history-of-graph-theory.html) - [Applications of Graph Theory](https://learngraphtheory.org/articles/applications-of-graph-theory.html) - [Graph Theory in Software Engineering](https://learngraphtheory.org/articles/software-engineering-concepts.html) - [Graph Algorithms for Coding Interviews](https://learngraphtheory.org/articles/coding-interviews.html) - [Graph Theory Interview Questions](https://learngraphtheory.org/articles/interview-questions.html) ## Key Pages - [Home](https://learngraphtheory.org/) - [All Algorithms](https://learngraphtheory.org/algorithms) - [All Lessons](https://learngraphtheory.org/lessons) - [Practice Problems](https://learngraphtheory.org/practice) - [Blog & Articles Index](https://learngraphtheory.org/articles/) - [FAQ](https://learngraphtheory.org/faq.html) ## Optional - [Developers](https://learngraphtheory.org/developers) - [Sitemap](https://learngraphtheory.org/sitemap.xml) --- # Full Article Text > The complete text of every article on learngraphtheory.org, inlined for AI ingestion. ## Graph Theory in Biology Source: https://learngraphtheory.org/articles/graph-theory-biology.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Why biology is full of graphs - 2. Four graphs, four different questions - 3. Protein networks: hubs, bridges and betweenness - 4. Robust to accidents, fragile to attack - 5. Modules, and why clustering means function - 6. Gene regulation: network motifs - 7. Genome assembly: walking every edge once - 8. Sequence alignment is a shortest path - 9. Phylogenetics: one tree out of astronomically many - 10. Food webs and extinction cascades - 11. Connectomes and epidemics - 12. What is easy, what is hard - 13. Modelling mistakes - 14. From model to practice - 15. Frequently asked questions - 16. References 1. Why biology is full of graphs Biology spent most of its history making lists. A list of genes, a list of enzymes, a list of the species in a lake. The lists got long and then they got complete, and completing them revealed something uncomfortable: knowing every part of a system tells you remarkably little about what the system does. The human genome was finished in 2003 and the number of protein-coding genes turned out to be around 20,000, not far off the count in a nematode worm with 302 neurons. The parts list was never going to be the explanation. What separates a human from a worm, and a healthy cell from a cancerous one, is not which components exist but which components interact with which. That sentence is a definition of a graph. A graph is a set of things together with a set of connections between them, and nothing else. The moment you write down which proteins bind to which, or which gene switches on which, or which species eats which, you have stopped making a list and started drawing a graph, whether or not you use the word. This is not a metaphor and it is not a presentation style. It matters because graphs arrive with two centuries of mathematics attached. Once a biological question is phrased as a graph question, a large body of theorems and algorithms becomes available at once, and problems that look like they need new biology turn out to need an existing algorithm instead. Genome assembly, the process that turns hundreds of millions of short DNA fragments into a chromosome, is a walk that uses every edge of a graph exactly once. Euler settled that problem in 1736 for the bridges of a Prussian city, and it is the same problem. The habit is older than most people assume. The word graph in this technical sense was coined by the mathematician James Joseph Sylvester in 1878, by analogy with chemical structure diagrams: molecules drawn as atoms joined by bonds. Chemistry gave graph theory part of its vocabulary, and a century later molecular biology gave it some of its largest data sets. This guide works through six biological systems: a protein interaction network, a gene regulatory network, a genome being assembled from reads, a pair of sequences being aligned, a set of species placed on an evolutionary tree, and a food web losing species one at a time. Each is kept small enough to check by hand and solved with a named algorithm. Every number here came out of code that was run, and each result was recomputed by a second, different method before it was written down. 2. Four graphs, four different questions Before building anything it is worth being precise about what kind of graph each biological system produces, because the kind decides which questions are even askable. Four distinctions do almost all of the work. Directed or undirected. If two proteins bind, the relationship is symmetric: "A binds B" and "B binds A" are the same fact, so protein interaction networks are undirected. If a transcription factor switches on a gene, the relationship runs one way, so gene regulatory networks are directed. This is not bookkeeping. Undirected graphs have connected components; directed graphs have reachability, cycles and feedback, and feedback is the substance of regulation. Asking whether a gene network contains a cycle is asking whether it contains a feedback loop, and the answer changes the biology. Weighted or unweighted. A food web edge can simply exist, or it can carry the biomass flowing along it. Sequence similarity graphs carry a score on every edge. Weights let you ask for the best route rather than merely a route, which is what turns alignment into a shortest path problem in section 8. Static or dynamic. Almost every network in this article is drawn as though it were fixed. Real cells are not. An interaction that exists in a liver cell may not exist in a neuron, and interactions appear and vanish across the cell cycle. Treating a time-averaged aggregate as though all of its edges were simultaneously present is the most common modelling error in this field, and section 13 comes back to it. Bipartite or not. Some biological data has two kinds of vertex with edges only between the kinds: drugs and their targets, hosts and their parasites, genes and the diseases they are associated with. Bipartite graphs bring their own algorithms, matching above all, which is how drug repurposing screens are often framed. Get these four right and the rest follows. Get them wrong and you will compute a number that is meaningless in a way no software will warn you about. 3. Protein networks: hubs, bridges and betweenness A protein interaction network, usually shortened to PPI network, has one vertex per protein and an undirected edge wherever two proteins physically bind. Large-scale versions are built by yeast two-hybrid screens or by affinity purification followed by mass spectrometry, and the published networks for yeast and human run to tens of thousands of edges. Rather than gesture at something enormous, we will use a small one: twelve proteins and eighteen interactions, small enough that every claim below can be checked by counting. Twelve proteins, eighteen interactions, three visible modules. Every number quoted in this section was computed on exactly this graph. The first thing to measure is degree, the number of partners a protein has. Here the degrees run A:5, then F and I with 4, then a group of five with 3, then a tail of four with 2, averaging 3.0. In real PPI networks this distribution is far more uneven: most proteins have a handful of partners and a small minority have hundreds. Networks with that shape are called scale-free, a term Barabasi and Oltvai popularised in their 2004 review, and the high-degree minority are the hubs. Hubs matter for a reason that was established experimentally rather than argued theoretically. In 2001 Jeong, Mason, Barabasi and Oltvai compared the yeast PPI network against the yeast deletion library, in which every gene has been knocked out in turn and the resulting cell scored viable or dead. Proteins with more interaction partners were substantially more likely to be essential. The paper is called Lethality and centrality in protein networks, and the correlation it reported is why degree became the first thing anyone computes on a biological network. Degree is not the only kind of importance, and this is where a small example earns its place. Consider protein E. It has three partners and on any list sorted by degree it is unremarkable. Now compute betweenness centrality, which counts, over every pair of proteins, what fraction of the shortest paths between them passes through a given vertex. Freeman introduced the measure in 1977 for social networks. On this graph the betweenness ranking is A at 23.0, I at 19.8, F at 12.2, and then E at 9.2, ahead of several proteins with more partners than it has. E scores highly because of where it sits, not how many neighbours it has. It is the entry point of the second module, so traffic between the first module and the second has to pass through it. In network terms E is a bridge rather than a hub, and the distinction has a biological reading: bridge proteins are candidates for cross-talk between pathways, and removing one does not delete a function so much as disconnect two functions from each other. A ranking by degree alone would never surface it. Two further numbers describe the whole graph rather than any single vertex. The average shortest path length is 2.197 and the diameter, the longest of all shortest paths, is 4: any protein reaches any other in at most four steps. Real PPI networks behave the same way at vastly larger scale, with thousands of proteins and a characteristic path length around 5. This is the small-world property that Watts and Strogatz formalised in 1998, and inside a cell it has a blunt consequence. A perturbation anywhere is a few steps from everywhere, which is a large part of why a drug aimed at one protein so reliably produces effects nobody designed. 4. Robust to accidents, fragile to attack The most cited result in network biology is not about any particular protein. It is about what happens when you start deleting them, and it was published by Albert, Jeong and Barabasi in Nature in 2000 under the title Error and attack tolerance of complex networks. The experiment is easy to state. Take a network, remove vertices, and after each removal measure the size of the largest surviving component. Do it twice: once removing vertices uniformly at random, which models accidents and mutation, and once removing them in descending order of degree, which models a deliberate attack. Then compare the curves. The same network under two removal strategies. Random damage barely registers; removing the same number of hubs takes the network apart. On a 300-vertex scale-free network grown by preferential attachment, removing 20% of the vertices at random leaves 78% of the network still connected in one piece. Removing the 20% with the highest degree leaves 9%. The network that shrugged off the first attack was destroyed by the second, and the only difference between them was which vertices got chosen. The twelve-protein network shows the same asymmetry at a scale you can verify by hand. Removing proteins at random, averaged over every possible choice, leaves largest components of 12, 11, 9.55, 7.96 and 6.46 as the number of removals goes from zero to four. Removing hubs in descending degree order, which here means A, then F, then I, then C, leaves 12, 11, 7, 6 and 4. Two well-chosen deletions cost more than four random ones. The explanation is the degree distribution. In a scale-free network the overwhelming majority of vertices have low degree, so a random deletion almost certainly hits a peripheral vertex whose loss disconnects nobody. The rare hubs hold everything together, and deleting one removes many edges at once. Robustness against random damage and fragility against targeted damage are not two properties in tension. They are one property viewed from two directions. The biological readings run both ways. On the fragile side, this is why hub proteins are enriched for essentiality, and why oncology has spent two decades trying to identify the hubs a tumour depends on. On the robust side, it is why organisms tolerate an enormous load of random mutation with no visible consequence, and why single-gene knockouts so often produce no phenotype at all. That last observation frustrated a generation of geneticists: most genes are not load-bearing, and the ones that are can be picked out by their position in the graph. 5. Modules, and why clustering means function Look again at the twelve-protein network and you can see three groups by eye. Proteins A to D are densely tied to each other, E to H form a second cluster, I to L a third, and only four edges cross between groups. That visual impression has a number behind it. The clustering coefficient of a vertex asks a specific question: of all the pairs of my neighbours, what fraction are themselves connected? If a protein has four partners there are six pairs among them, and the coefficient is the fraction of those six pairs that bind each other. Averaged over all twelve proteins this network scores 0.503, meaning roughly half of all the triangles that could close do close. A random graph with the same number of vertices and edges scores about 0.23. Real PPI networks are similarly clustered, and so are metabolic networks, neural networks and food webs. High clustering is what makes the word module meaningful. Proteins that all bind each other tend to be doing one job together: they form a complex, or sit in one pathway, or are recruited to the same place at the same time. This is the most useful inference in applied network biology, because it lets you annotate an unknown protein from its neighbours. If a protein of unknown function sits in a cluster whose other members all handle DNA repair, DNA repair is the hypothesis to test first. Whole pipelines rest on that idea, and they are community detection algorithms with biological names attached. One caution belongs here. A module found by an algorithm is a hypothesis, not a discovery. The algorithm partitions whatever you give it, and it will return modules from random data just as happily. 6. Gene regulation: network motifs Gene regulatory networks are directed. An arc from gene X to gene Y means the protein X produces binds the promoter of Y and changes how much of Y gets made. Because the arcs have direction, the interesting structures are patterns of flow rather than dense neighbourhoods, and in 2002 two papers from Uri Alon's group changed how people read them. The idea is this. Take a small subgraph, say three genes wired in a particular pattern, and count how often it occurs in the real network. That count alone means nothing, because some patterns are common purely as a consequence of how many arcs each gene has. So generate many randomised networks with exactly the same degrees, by repeatedly swapping the endpoints of pairs of arcs, and count the pattern in each. If the real count sits far out in the tail of that distribution, the pattern is a network motif: it occurs more often than the degrees alone can explain, which is evidence that selection put it there. A motif is not a pattern that occurs often. It is a pattern that occurs more often than a degree-matched random network can account for. The pattern in the figure is the feed-forward loop: gene X regulates Y, X also regulates Z directly, and Y regulates Z as well. In the eight-gene network here it occurs five times. Across 1,000 degree-preserving randomisations the average count was 1.80 with a standard deviation of 1.22, giving a z-score of 2.63, and only 17 of the 1,000 randomised networks contained five or more. On a network this small that is suggestive rather than conclusive; in the real E. coli transcription network Shen-Orr, Milo and Alon found the same pattern with z-scores in the tens, which is not a borderline result. What makes the feed-forward loop worth caring about is that its function can be derived rather than guessed. In the coherent version, where X activates both Y and Z and Y also activates Z, gene Z only switches on once both X and Y are present. Since Y takes time to accumulate after X appears, Z ignores brief pulses of X and responds only to sustained signals. The motif is a persistence detector, a noise filter built out of three genes. Change the signs and you get a pulse generator or an accelerated response instead. The wiring is the mechanism. Milo and colleagues found that different kinds of network are characterised by different motifs: transcription networks are rich in feed-forward loops, neural networks in a different set, food webs in another again. They argued that motifs are the elementary circuits from which the network is built, and the framing stuck, both because the statistics are checkable and because the circuits do something. One methodological point generalises well beyond biology. The randomisation must preserve degrees. Compare against an ordinary random graph and almost everything looks like a motif, since real networks have hubs and random ones do not, and the hubs alone generate a surplus of every three-node pattern. Getting the null model wrong is the standard way this analysis fails. 7. Genome assembly: walking every edge once Sequencing machines cannot read a chromosome. They read short fragments, from around 100 bases on a short-read instrument up to tens of thousands on a long-read one, sampled at random positions from many copies of the genome. A human genome arrives as hundreds of millions of these fragments with no record of where any of them came from. Assembly is the problem of putting them back together, and the modern solution is a graph. The construction is due to Pevzner, Tang and Waterman in 2001, and it is elegant enough to state in two sentences. Chop every read into overlapping substrings of length k, called k-mers. Then build a graph in which each k-mer is an edge, running from the vertex spelled by its first k-1 letters to the vertex spelled by its last k-1 letters. Reconstructing the sequence now means finding a walk that uses every edge exactly once, which is an Eulerian path. Assembly as an Eulerian path. When a repeat makes the walk ambiguous, the reads genuinely do not contain enough information to choose, and longer reads are the only fix. Take the sequence ATGGCGTGCA and read it as 4-mers. That gives seven k-mers, a graph with 8 vertices and 7 edges, and exactly one Eulerian path, which spells the original sequence back out. Assembly succeeded, and it succeeded because the graph had a unique answer. Now take AGGGTGGTTGGC, again as 4-mers. The graph has two Eulerian paths, spelling AGGGTGGTTGGC and AGGGTTGGTGGC. Both are consistent with every read that was observed. This is not a failure of the algorithm, and no better algorithm can fix it: the 3-mer TGG occurs twice, the walk arrives at that vertex more than once, and the reads carry no information about which way to leave it the first time. The ambiguity is in the data. What resolves it is longer reads. Reading the same sequence as 6-mers produces a graph with exactly one Eulerian path and a single reconstruction. This is the reason the sequencing industry has spent fifteen years chasing read length rather than read count, and the reason the human genome was only declared complete, gap to telomere, in 2022, more than twenty years after the first draft. The missing pieces were repeats, and repeats are precisely the structures that make an Eulerian walk ambiguous. There is a lovely piece of algorithmic history here. Finding an Eulerian path is easy: linear time, with an existence condition known since Euler. The natural-looking alternative, building a graph in which every read is a vertex and joining reads that overlap, requires a walk visiting every vertex once, which is a Hamiltonian path and NP-complete. Two formulations of one biological task, one tractable and one hopeless, separated only by the decision to make the reads edges instead of vertices. 8. Sequence alignment is a shortest path Comparing two sequences is the most-run computation in biology. Every BLAST query does it, every read mapper does it, and every claim that two genes are homologous rests on it. The standard algorithm is Needleman and Wunsch's, published in 1970 and taught everywhere as dynamic programming over a matrix. It is worth seeing that the matrix is a graph. Build a grid with one vertex for each pair of positions (i, j), meaning "the first i letters of sequence one have been aligned against the first j letters of sequence two". From each vertex draw three arcs: right, for a letter of sequence two against a gap; down, for a letter of sequence one against a gap; and diagonal, for aligning the two letters. Give each arc a cost, zero for a diagonal that matches and one otherwise. The best alignment is now the cheapest path from the top-left corner to the bottom-right, and any shortest path algorithm finds it. Aligning GATTACA against GCATGCU builds a grid graph with 64 vertices and 161 arcs. Needleman-Wunsch returns an edit distance of 4. A shortest path search over that graph, with no dynamic programming table anywhere, returns 4 as well. They agree because they are the same computation: the grid is acyclic, so filling the cells in order is exactly relaxing the arcs in topological order. Seeing it as a graph is not a party trick. It explains why Smith and Waterman's 1981 local alignment algorithm works: allowing a path to start fresh anywhere is adding a zero-cost arc from the source to every vertex. It explains affine gap penalties, which need three layers of grid rather than one because the state must remember whether a gap is already open. And it explains why alignment costs the product of the two sequence lengths, since that is the size of the graph, which is why fast aligners avoid building most of it. 9. Phylogenetics: one tree out of astronomically many A phylogenetic tree is a graph with no cycles: leaves are the species you observed, internal vertices are ancestors you did not, and edge lengths measure evolutionary divergence. Reconstructing one from present-day sequences is the central inference problem of evolutionary biology, and its difficulty is a counting problem before it is anything else. The search space is beyond astronomical, so the practical algorithms do not search it. Neighbour joining builds one tree greedily and gets the right answer when the distances behave. The number of distinct unrooted binary trees on n species, tabulated by Felsenstein in 1978, is the double factorial (2n-5)!!, and it detonates. Four species give 3 trees. Five give 15. Ten give 2,027,025. Twenty give about 2.2 x 1020. Fifty species give roughly 2.8 x 1074, which is more trees than there are atoms in the observable universe by a wide margin. Every one of those is a candidate answer, and phylogenetic studies routinely involve hundreds of taxa. Exhaustive search is therefore not merely slow, it is permanently impossible, and the situation is worse than that: finding the most parsimonious tree, the one requiring the fewest evolutionary changes, was later proved NP-hard, and maximum likelihood tree search is no better. Saitou and Nei's neighbour joining, published in 1987 and among the most cited papers in all of biology, sidesteps the search entirely. It takes a matrix of pairwise distances, repeatedly joins the pair of taxa that a specific criterion identifies as neighbours, and collapses them into one node, until a tree remains. It never enumerates alternatives, and it runs in cubic time. On the five-primate distance matrix in the figure it joins orangutan with gibbon first, then human with chimp, then gorilla with the orangutan and gibbon group, and it recovers every branch length exactly. That exactness is not luck. When the distances are additive, meaning they came from some tree in the first place, neighbour joining provably returns that tree. Real distances estimated from real sequences are only approximately additive, which is why real phylogenetics uses neighbour joining to produce a fast starting tree and then refines it under a likelihood model, and why the same data set can support different published trees. 10. Food webs and extinction cascades Move from inside the cell to a whole ecosystem and the mathematics does not change. A food web is a directed graph: one vertex per species, and an arc from prey to predator wherever the second eats the first. Species with no prey are basal, meaning plants, algae or detritus, and everything else ultimately depends on them. The model here has twelve species and seventeen feeding links, with algae and detritus at the base and an otter, a heron and a pike at the top. The first thing the graph gives you is trophic level, computed as one plus the average level of everything a species eats. Basal species sit at 1.00, herbivores at 2.00, and the top predators land on fractional values: the pike at 4.50, the otter at 4.75, the heron at 4.33. Fractional levels are not an artefact. They are the honest answer for an omnivore that eats at several levels at once. The best connected species is not the one whose loss does the most damage. Position in the graph decides that, and position is not degree. The question that matters for conservation is what happens after a species is lost. Delete a vertex, then delete any species left with nothing to eat, then repeat until the web settles. Those follow-on losses are secondary extinctions, and they are the reason ecosystems collapse faster than the direct pressure on them would suggest. The results on this web are counterintuitive in a specific and useful way. The minnow is the best connected species with five feeding links. Remove it and nothing else dies: everything that ate the minnow eats something else too. Remove the heron or the otter, both top predators, and again nothing follows. Now remove detritus, which has only two links and is not a species anybody campaigns to protect. Three species disappear in total: the detritus itself, then the insect that eats nothing else, then the frog that eats nothing but insects. A vertex with two edges caused more damage than a vertex with five. The pattern generalises. Basal species are load-bearing because everything above them depends on them, while a highly connected consumer sits in a part of the graph that offers substitutes. Removing both basal species, algae and detritus, costs all twelve species. Removing the three best connected, minnow, perch and snail, costs six. Twice the damage from the less impressive-looking intervention. Dunne, Williams and Martinez reported exactly this in 2002 across sixteen real food webs, and added a second finding worth carrying away: robustness increases with connectance, the number of links divided by the square of the number of species. Webs with more feeding links absorb more damage before they fragment, because more species have alternatives. The web modelled here has a connectance of 0.118, squarely in the range reported for real webs. The practical lesson is that conservation triage based on charisma, size or even link count measures the wrong quantity. The species whose loss propagates is found by simulating the removal on the graph, and the answer is regularly something small and unloved. 11. Connectomes and epidemics Two more areas deserve mention, because both reuse machinery already introduced. Connectomes. A nervous system is a directed, weighted graph of neurons joined by synapses. The first complete one was published by White, Southgate, Thomson and Brenner in 1986: the nematode C. elegans, 302 neurons and around 7,000 connections, reconstructed by hand from electron micrographs over more than a decade. Human work operates at coarser resolution, with vertices as brain regions and edges as fibre bundles or correlated activity, but the analysis is the one from section 3: degree, clustering, path length, modules, hubs. Bullmore and Sporns set out the programme in 2009, and the recurring finding is that brains are small-world and modular, with a densely interconnected core of high-degree regions, the "rich club", carrying a disproportionate share of long-distance traffic. Several psychiatric and neurological conditions show up as altered graph statistics. Those are correlations across groups, not diagnostics for individuals, and it is worth knowing that a functional connectome depends heavily on a correlation threshold chosen by the analyst, and that the statistics move when the threshold moves. Epidemics. Disease spreads across a contact graph, and its structure determines the outcome as much as the pathogen does. Pastor-Satorras and Vespignani proved a startling result in 2001: on a network with a scale-free degree distribution and unbounded variance, the classical epidemic threshold vanishes. In the well-mixed models taught in textbooks an infection with a low enough transmission rate dies out; on such a network it does not, because the hubs keep it alive. That reframed vaccination strategy, since immunising the high-degree individuals, or even acquaintances of randomly chosen individuals, who are high-degree more often than chance, beats immunising at random with the same number of doses. The same mathematics reappears in cell biology as signal propagation and in computer security as malware spread. The graph does not care what the vertices represent. 12. What is easy, what is hard Framing a biological question as a graph question does not make it solvable. It makes the difficulty visible, which is more useful, and the boundary falls in surprising places. Easy, meaning polynomial time and routine at scale. Degree, clustering coefficients and connected components are effectively free. Shortest paths, and therefore alignment, are cheap. Betweenness centrality on a sparse graph runs in time proportional to the product of the vertex and edge counts thanks to Brandes' algorithm. Eulerian paths are linear, spanning trees and flows are polynomial, and neighbour joining is cubic. Everything solved in this article is in this category, and all of it scales to graphs with millions of edges on a laptop. Hard, meaning NP-hard with no polynomial algorithm expected. Finding the most parsimonious phylogenetic tree. Finding the largest set of species that all interact, which is maximum clique. Deciding whether one network is a subgraph of another, which underlies motif search on larger patterns. Finding a Hamiltonian path, the reason the overlap formulation of assembly was abandoned. Optimal graph partitioning, in its exact form. Two observations make the boundary less discouraging than it sounds. First, hard problems in biology are usually attacked with heuristics that work well on the instances that actually arise: phylogenetics hill-climbs from a neighbour-joining start, and motif finders enumerate cleverly, which is fine for three- and four-vertex patterns. Second, the difference between the tractable and intractable formulations of one biological task is often just a modelling choice, as the k-mers-as-edges decision in assembly shows. Recognising which side of the line you are on before writing code is most of the benefit. 13. Modelling mistakes Five errors account for most of the wrong conclusions drawn from biological networks. None of them is exotic and all of them are still in print. Treating an aggregate as a snapshot. A published PPI network is the union of many experiments, in different cell types, under different conditions, over decades. Its edges never coexisted. Computing shortest paths across it assumes every interaction is simultaneously available, which is false. Where condition-specific data exists, filter to it; where it does not, treat path-based conclusions as hypotheses. Ignoring study bias. Well-studied proteins have more known interactions because more people looked, not necessarily because they have more real partners. Any analysis concluding that "the most connected proteins are the important ones" is partly rediscovering the field's publication history. The check is whether your result survives when the network is restricted to a single unbiased screen. Comparing against the wrong null model. This is the motif lesson from section 6 and it generalises everywhere. Real biological networks have hubs and heavy-tailed degree distributions. Compare any structural statistic against a uniform random graph and it will look extraordinary. The comparison has to preserve the features you are not testing, which usually means preserving the degree sequence. Reading correlation as an edge. Gene co-expression networks join genes whose expression levels correlate across samples. Correlation is not regulation, and the resulting graph is undirected while regulation is directed. These networks are useful for generating hypotheses and consistently misleading when read as mechanism. The same caution applies to functional connectomes built from correlated brain activity. Over-reading the scale-free claim. The observation that biological networks have heavy-tailed degree distributions is robust and important. The stronger claim, that they follow a clean power law, has been challenged repeatedly on statistical grounds, notably by Broido and Clauset in 2019, who found strict power laws to be rare across thousands of empirical networks. The useful conclusions in this article, hub essentiality and the error-versus-attack asymmetry, need only the heavy tail, not the exact functional form. Claim the tail, not the law. 14. From model to practice A short procedure for anyone about to build one of these graphs for real. Write down what a vertex is and what an edge means, in one sentence each, before touching data. Most confused analyses can be traced back to a graph in which the edges mean two different things, "binds" mixed with "is correlated with", or "eats" mixed with "competes with". If the sentence is hard to write, the graph is not ready. Decide directed or undirected, weighted or unweighted, on biological grounds. Not on the grounds of what the software defaults to. Every downstream metric inherits this choice, and a betweenness score computed on a graph that should have been directed is not an approximation, it is a different quantity. Compute the cheap descriptive statistics first. Vertex and edge counts, degree distribution, number of components, clustering coefficient, path length. These take seconds and catch data problems immediately: an unexpected second component usually means an identifier mismatch, and a suspiciously high average degree usually means duplicated edges. Choose the null model before computing the statistic you care about. Not after seeing the result. Perturb and re-run. Drop 10% of the edges at random and recompute your headline conclusion. Biological networks are incomplete and noisy, and a ranking that reshuffles when a tenth of the data is removed is a property of the sample rather than of the organism. If you want to build intuition for the algorithms behind all of this before applying them to biological data, the interactive visualizers on this site let you run breadth-first search, Dijkstra's algorithm and minimum spanning tree construction step by step on graphs you draw yourself, which is the fastest way to develop a feel for what these methods are actually doing. 15. Frequently asked questions How is graph theory used in biology? + Wherever biological objects interact. Proteins that bind form an undirected network, genes that regulate each other form a directed one, DNA reads form a de Bruijn graph whose Eulerian path is the assembled genome, species and their ancestors form an evolutionary tree, and species that eat each other form a food web. In each case the biology supplies the vertices and edges, and standard algorithms then answer the questions: which components are essential, which patterns are over-represented, which sequence explains the reads, which tree explains the distances, and which extinction triggers a cascade. What is a hub protein, and why do hubs matter? + A hub is a protein with far more interaction partners than average. They matter because of an experimental result: Jeong, Mason, Barabasi and Oltvai showed in 2001 that yeast proteins with more partners are substantially more likely to be essential, meaning the cell dies without them. Hubs also explain why networks behave so differently under random damage and targeted damage. On the scale-free network in this article, removing 20% of proteins at random leaves 78% of the network connected, while removing the 20% with the highest degree leaves 9%. What is a network motif? + A small subgraph that appears more often in a real network than in randomised networks with exactly the same degrees. The randomisation is the whole point: comparing against an ordinary random graph makes almost every pattern look significant, because real networks have hubs and random ones do not. In the eight-gene network here the feed-forward loop appears five times against a randomised mean of 1.80 and a standard deviation of 1.22, a z-score of 2.63, with only 17 of 1,000 randomised copies reaching five or more. Motifs matter because their function can be derived: the coherent feed-forward loop ignores brief pulses and responds only to sustained signals. Why is genome assembly an Eulerian path problem? + Because of how the graph is built. Chop every read into overlapping substrings of length k, then make each k-mer an edge running from the vertex spelled by its first k-1 letters to the vertex spelled by its last k-1 letters. Using every read exactly once now means using every edge exactly once, which is an Eulerian path, and that is solvable in linear time. The natural alternative, making each read a vertex and joining overlapping reads, requires visiting every vertex once, which is a Hamiltonian path and NP-complete. The same biological task is tractable or hopeless depending only on whether reads become edges or vertices. Why do repeats make assembly ambiguous? + Because a repeated sequence makes the walk arrive at the same vertex more than once, and the reads carry no information about which way to leave it first. In this article the sequence AGGGTGGTTGGC read as 4-mers produces a graph with two Eulerian paths, spelling AGGGTGGTTGGC and AGGGTTGGTGGC, both fully consistent with every read. No algorithm can choose between them, because the ambiguity is in the data rather than in the method. Reading the same sequence as 6-mers gives exactly one reconstruction, which is why read length matters more than read count and why long-read sequencing changed assembly. How many phylogenetic trees are there, and how do biologists find one? + The number of unrooted binary trees on n species is the double factorial (2n-5)!!, which grows past any possibility of search: 3 trees for four species, 15 for five, 2,027,025 for ten, about 2.2 x 10^20 for twenty and roughly 2.8 x 10^74 for fifty. Finding the most parsimonious tree is NP-hard. Neighbour joining, published by Saitou and Nei in 1987, avoids searching at all: it repeatedly joins the closest pair by a specific criterion and builds one tree in cubic time. When the distances are additive it provably recovers the true tree, which is exactly what happens on the five-primate matrix in this article, where every branch length is recovered exactly. Which species matters most in a food web? + Not the best connected one. On the twelve-species web in this article the minnow has the most feeding links, five, and removing it causes no secondary extinctions at all, because everything that ate it eats something else. Removing detritus, which has only two links, costs three species: the detritus, then the insect that eats nothing else, then the frog that eats nothing but insects. Removing both basal species loses all twelve, while removing the three best connected species loses six. Importance is a property of position in the graph, and it has to be found by simulating the removal rather than by counting links. Are biological networks really scale-free? + They have heavy-tailed degree distributions, which is well established. The stronger claim that they follow a clean power law has been challenged on statistical grounds, notably by Broido and Clauset in 2019, who found strict power laws to be rare across thousands of empirical networks. This matters less than it sounds, because the conclusions that get used need only the heavy tail: a distribution in which most vertices have few edges and a small minority have many is enough to produce hub essentiality and the asymmetry between random failure and targeted attack. The safe position is to claim the tail and not the law. 16. References The papers behind the results in this article, in chronological order. - Sylvester, J. J. (1878). “Chemistry and algebra.” Nature, 17, 284. - Needleman, S. B. and Wunsch, C. D. (1970). “A general method applicable to the search for similarities in the amino acid sequence of two proteins.” Journal of Molecular Biology, 48(3), 443–453. - Freeman, L. C. (1977). “A set of measures of centrality based upon betweenness.” Sociometry, 40(1), 35–41. - Felsenstein, J. (1978). “The number of evolutionary trees.” Systematic Zoology, 27(1), 27–33. - Smith, T. F. and Waterman, M. S. (1981). “Identification of common molecular subsequences.” Journal of Molecular Biology, 147(1), 195–197. - White, J. G., Southgate, E., Thomson, J. N. and Brenner, S. (1986). “The structure of the nervous system of the nematode Caenorhabditis elegans.” Philosophical Transactions of the Royal Society B, 314(1165), 1–340. - Saitou, N. and Nei, M. (1987). “The neighbor-joining method: a new method for reconstructing phylogenetic trees.” Molecular Biology and Evolution, 4(4), 406–425. - Watts, D. J. and Strogatz, S. H. (1998). “Collective dynamics of small-world networks.” Nature, 393, 440–442. - Albert, R., Jeong, H. and Barabási, A.-L. (2000). “Error and attack tolerance of complex networks.” Nature, 406, 378–382. - Jeong, H., Tombor, B., Albert, R., Oltvai, Z. N. and Barabási, A.-L. (2000). “The large-scale organization of metabolic networks.” Nature, 407, 651–654. - Jeong, H., Mason, S. P., Barabási, A.-L. and Oltvai, Z. N. (2001). “Lethality and centrality in protein networks.” Nature, 411, 41–42. - Brandes, U. (2001). “A faster algorithm for betweenness centrality.” Journal of Mathematical Sociology, 25(2), 163–177. - Pevzner, P. A., Tang, H. and Waterman, M. S. (2001). “An Eulerian path approach to DNA fragment assembly.” Proceedings of the National Academy of Sciences, 98(17), 9748–9753. - Pastor-Satorras, R. and Vespignani, A. (2001). “Epidemic spreading in scale-free networks.” Physical Review Letters, 86(14), 3200–3203. - Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N., Chklovskii, D. and Alon, U. (2002). “Network motifs: simple building blocks of complex networks.” Science, 298(5594), 824–827. - Shen-Orr, S. S., Milo, R., Mangan, S. and Alon, U. (2002). “Network motifs in the transcriptional regulation network of Escherichia coli.” Nature Genetics, 31(1), 64–68. - Dunne, J. A., Williams, R. J. and Martinez, N. D. (2002). “Network structure and biodiversity loss in food webs: robustness increases with connectance.” Ecology Letters, 5(4), 558–567. - Barabási, A.-L. and Oltvai, Z. N. (2004). “Network biology: understanding the cell's functional organization.” Nature Reviews Genetics, 5(2), 101–113. - Yildirim, M. A., Goh, K.-I., Cusick, M. E., Barabási, A.-L. and Vidal, M. (2007). “Drug-target network.” Nature Biotechnology, 25(10), 1119–1126. - Bullmore, E. and Sporns, O. (2009). “Complex brain networks: graph theoretical analysis of structural and functional systems.” Nature Reviews Neuroscience, 10(3), 186–198. - Compeau, P. E. C., Pevzner, P. A. and Tesler, G. (2011). “How to apply de Bruijn graphs to genome assembly.” Nature Biotechnology, 29(11), 987–991. - Broido, A. D. and Clauset, A. (2019). “Scale-free networks are rare.” Nature Communications, 10, 1017. --- ## Introduction to Graph Theory Source: https://learngraphtheory.org/articles/introduction-to-graph-theory.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What graph theory actually studies - 2. The definition, and what it leaves out on purpose - 3. Where it started: seven bridges and one impossible walk - 4. The vocabulary, and the first theorem - 5. The families of graphs - 6. Walks, trails, paths and cycles - 7. Connectivity, components and the edges you cannot lose - 8. Trees: the most useful special case - 9. How a graph is stored in a computer - 10. Traversal: breadth first and depth first - 11. The classic problems and their algorithms - 12. Five results worth knowing by name - 13. What is easy, what is hard, and why it matters - 14. Where graphs actually show up - 15. Mistakes beginners reliably make - 16. Where to go next - 17. Glossary - 18. Frequently asked questions - 19. References 1. What graph theory actually studies Graph theory is the study of one very small idea: a collection of objects, and a record of which pairs of them are connected. That is the whole subject. What makes it worth a century and a half of mathematics is that an enormous number of practical questions turn out to be questions about exactly that, and nothing else. Consider four problems that look unrelated. A delivery company wants the shortest route between two depots. A compiler needs the order in which to build a project's modules. A biologist wants to know which proteins interact, directly or through intermediaries. A network operator wants to know which cable, if cut, would isolate a region. These have nothing in common as stories. Structurally they are the same handful of problems asked about the same kind of object, and the algorithms that answer them are interchangeable. That transferability is why the subject is taught early and used everywhere: once a situation is written as a graph, a large catalogue of results becomes available at once, and none of them care what the vertices originally represented. Modelling is the act of throwing information away. Everything on the left that is not a connection is deliberately lost, and the object that remains is a graph. Notice what the picture on the right does not contain. The towns have moved, the roads are straight, and nothing records that one is twice as long as another. If those facts matter for your question you must add them back explicitly, as numbers on the edges. If they do not, throwing them away is exactly what makes the problem tractable. This article is a first course in one page: the definitions in order, the small results everything else rests on, how graphs are stored and searched in real code, the classic problems, and an honest map of which questions a computer can answer in seconds and which it cannot answer at all. Each section links to a deeper article if you want more on that one topic. 2. The definition, and what it leaves out on purpose Almost every popular account says a graph is "dots joined by lines". That picture is useful, and it is also why many people get stuck a few weeks later: the dots and the lines are a drawing of the object, not the object. The object is a pair of sets. Diestel's Graph Theory, the standard graduate reference, states it in its cleanest form: A graph is a pair G = (V, E) of sets such that E ⊆ [V]2, where [V]2 is the set of all 2-element subsets of V. Unpacked, that says four things: - V is a set of objects, called vertices (or nodes). Nothing at all is assumed about them. They may be cities, people, atoms, web pages or integers. The theory never looks inside a vertex; it only needs to be able to tell two of them apart. - E is a set of 2-element subsets of V. An edge is literally the set {u, v}. It is not an arrow, not a curve, and it carries no information beyond which pair it joins. - Because E is a set, an edge is either present or absent. It cannot appear twice. - Because each edge has two distinct members, no edge joins a vertex to itself. Those last two consequences are not extra rules somebody added; they fall straight out of the set theory, and a graph obeying them is called simple. Allowing repeated edges or self-loops means changing the definition itself, which is what multigraphs do in section 5. Two more pieces of notation appear everywhere. Write V(G) and E(G) when several graphs are in play. And the two size measures have names: the number of vertices is the order of the graph, the number of edges is its size, abbreviated in nearly all algorithm texts to n = |V| and m = |E|. What the definition leaves out is as informative as what it includes. There is no geometry, so two drawings of the same graph are the same graph even if one looks like a spiral and the other like a grid. There is no order on the vertices. There are no distances, capacities or costs; those come from an extra function, usually written w: E → ℝ, bolted on when a problem needs it. The bare object is deliberately impoverished, and that poverty is what makes theorems about it apply so widely. The companion guide on vertices and edges works the same definition through in more detail. 3. Where it started: seven bridges and one impossible walk The subject has a birthday. In 1736 Leonhard Euler, then at the St Petersburg Academy, sent in a paper titled Solutio problematis ad geometriam situs pertinentis, "the solution of a problem relating to the geometry of position". The problem came from the Prussian city of Königsberg, now Kaliningrad. The river Pregel divided the city into four land masses, joined by seven bridges, and the citizens amused themselves with a question: can you walk through the city crossing every bridge exactly once? Euler's first move is the move this whole article is about. The size and shape of the land masses are irrelevant, the length of the bridges is irrelevant, and the only thing that matters is which land mass connects to which, and how many times. Strip the rest and you have four objects and seven connections, which modern textbooks draw as a multigraph with four vertices and seven edges. His argument is short enough to give in full. Suppose the walk exists, and take any land mass that is not its start or end. Every time the walk arrives there it must also leave, so the bridges at that land mass are used in pairs and their number must be even. In Königsberg the four land masses had 5, 3, 3 and 3 bridges, all odd. A walk has only two ends, so at most two land masses may have an odd count. Four is too many, and no such walk exists. Why this argument matters more than the answer. Euler did not search for a route and fail. He proved that no route can exist, by counting a quantity that any successful route would have to respect. That style of reasoning, find an invariant, show the goal violates it, is what separates graph theory from puzzle solving, and it is why 1736 counts as the start of a field rather than the solution of a riddle. Euler stated the converse too, but did not prove it; that gap stayed open until Carl Hierholzer gave a constructive proof, published posthumously in 1873. The modern statement is clean: a connected graph has a closed walk using every edge exactly once, an Eulerian circuit, if and only if every vertex has even degree, and an open one, an Eulerian trail, if and only if exactly two vertices have odd degree. The full story is in the guide to the Eulerian path and circuit. The century that followed filled in the foundations, from Kirchhoff's spanning trees in 1847 to Sylvester borrowing the word "graph" from chemistry in 1878 and König's first textbook in 1936. That story is told in the history of graph theory. 4. The vocabulary, and the first theorem The rest of this article uses one running example, a graph with seven vertices and eight edges. It is small enough to check every claim by hand and large enough to be interesting. V = {A, B, C, D, E, F, G} n = 7 E = { {A,B}, {A,C}, {B,C}, {B,D}, {C,E}, {D,E}, {D,F}, {F,G} } m = 8 The running example for the whole article. Every term defined in this section can be read directly off this picture. Here are the terms, each one defined only from the two sets: - Adjacent. Two vertices joined by an edge. B and D are adjacent, A and D are not. - Incident. An edge is incident to each of its two endpoints. Adjacency relates a vertex to a vertex, incidence relates a vertex to an edge, and beginners routinely swap the two. - Neighbourhood. N(v), the set of vertices adjacent to v. Here N(D) = {B, E, F}. - Degree. deg(v), the number of edge ends meeting v, which in a simple graph is just |N(v)|. Here deg(B) = 3 and deg(G) = 1. - Leaf and isolated vertex. Degree 1, such as G, and degree 0. Isolated vertices are legal, and they are the most commonly lost piece of data when a graph is built from an edge list, because an edge list has no way to mention them. - Minimum and maximum degree. δ(G) and Δ(G), here 1 at G and 3 at B, C and D. With degree defined, the first theorem is one line away. The seven degrees sum to 2 + 3 + 3 + 3 + 2 + 2 + 1 = 16, exactly twice the eight edges, and that is not a coincidence about this graph. The handshaking lemma. In any graph, the sum of all vertex degrees equals twice the number of edges. Proof: count the pairs (v, e) where the vertex v is an endpoint of the edge e. Counting by vertices gives the sum of the degrees. Counting by edges gives 2m, because every edge has exactly two ends. Two counts of the same set must agree. That technique, counting one collection in two ways, is called double counting, and it is the workhorse of elementary combinatorics. The lemma has a corollary that surprises people the first time: the number of vertices of odd degree is always even. Here they are B, C, D and G, which is four. The reason is arithmetic: the total is even and the even-degree vertices contribute an even amount, so the odd-degree vertices must contribute an even amount between them, which takes an even number of them. In everyday terms, the number of people in a room who have shaken hands an odd number of times is even. Euler's Königsberg argument is this corollary applied to a walk. 5. The families of graphs The bare definition in section 2 is the most restrictive one. Every real modelling problem eventually needs a variation, and each variation is a specific, named change to what an edge is allowed to be. Knowing which family you are in decides which algorithms are even applicable, so this is not vocabulary for its own sake. Eight families, each defined by one change to what an edge may be. Most real models are a combination: a road network is a weighted directed graph, a dependency file is a DAG. Simple graphs are the default: no loops, no repeated edges, and every unqualified result in a textbook is about these. Multigraphs allow parallel edges and pseudographs allow self-loops too. Königsberg genuinely needs a multigraph, since two of its land masses were joined by two bridges, and a loop adds 2 to the degree of its vertex because both ends attach there. See simple graphs versus multigraphs. Directed graphs, or digraphs, replace the unordered pair {u, v} with the ordered pair (u, v), called an arc, so a digraph may contain either direction, both or neither, and degree splits into in-degree and out-degree. This is the right model whenever the relation is not symmetric: one way streets, "A follows B", "module A imports module B", "task A must finish before task B". See directed versus undirected graphs. Weighted graphs add a function w assigning a number to each edge: kilometres, minutes, price, capacity, similarity. Algorithms have strong opinions about those numbers. Dijkstra's algorithm requires them to be non-negative, Bellman-Ford tolerates negatives but not negative cycles, and breadth first search ignores them entirely, which is why running BFS on a weighted graph and calling the result a shortest path is one of the most common bugs in beginner code. See weighted versus unweighted graphs. Bipartite graphs split the vertex set into two parts with every edge running between them. Students and courses, applicants and jobs, buyers and products: any two sided matching situation is bipartite. A graph is bipartite exactly when it contains no odd cycle, and a single breadth first search that two-colours the vertices decides it in linear time. Complete graphs, written Kn, have every possible edge. Since an edge is a choice of 2 vertices from n, the count is n(n-1)/2, so K5 has 10 edges and K100 has 4,950. That is also the ceiling for any simple graph on n vertices, and it is what the density of a graph is measured against. Trees are connected graphs with no cycles, the subject of section 8. DAGs, directed acyclic graphs, are digraphs with no directed cycle, and they are the shape of every dependency and every schedule: spreadsheet formulas, build targets, Git commits and the operations of a neural network are all DAGs, and the algorithm that puts them in a valid order is topological sorting. One more is worth knowing by name: planar graphs can be drawn with no edges crossing, which matters for circuit layout and map colouring, and which section 12 returns to. 6. Walks, trails, paths and cycles Four words describe movement through a graph, they are used interchangeably in casual speech, and they mean four different things. Getting them straight prevents a surprising amount of confusion later, because theorems are stated with the precise word and the difference between them is often the entire content of the result. Each definition is the previous one with a repetition forbidden. The numbers show the order in which the route visits each vertex. - A walk is any sequence of vertices where consecutive ones are adjacent. Nothing is forbidden. A walk may cross the same edge and revisit the same vertex as often as it likes. - A trail is a walk with no repeated edge. Vertices may still repeat. Euler's bridge problem asks for a trail that uses every edge, which is why it is an Eulerian trail and not an Eulerian path. - A path is a walk with no repeated vertex, which automatically forbids repeated edges too. When somebody says "the route from A to G", this is what they mean. - A cycle is a closed path: it starts and ends at the same vertex and repeats nothing else. In the example, B, C, E, D, B is a cycle of length 4, and A, B, C, A is a triangle, a cycle of length 3. The length of any of these is its number of edges, not its number of vertices, which is an off-by-one waiting to happen. The distance d(u, v) is the length of a shortest path. Here d(A, G) = 4, along A, B, D, F, G; the route A, C, E, D, F, G also arrives but uses five edges, so it is a path and not a shortest one. The diameter is the largest distance between any pair of vertices, a compact way of saying how spread out a network is. One fact follows immediately and gets used constantly: if there is a walk from u to v, there is a path from u to v. Cut out the loop between any two visits to the same vertex and the result is a shorter walk, so the surgery ends at one with no repeated vertex. This is why reachability algorithms never consider walks at all. 7. Connectivity, components and the edges you cannot lose A graph is connected when every vertex can be reached from every other. When it is not, it falls apart into connected components, which are the maximal pieces that are internally connected. Connectivity is the first thing worth checking about any graph you did not construct yourself, because a surprising number of real datasets arrive in several pieces and most reported bugs of the form "the algorithm returned infinity" are that fact discovered the hard way. Within a connected graph, some parts of the structure are more critical than others. A bridge is an edge whose removal increases the number of components, and a cut vertex, or articulation point, is a vertex whose removal does the same. These are the single points of failure, and finding them is the standard first analysis of any network whose reliability matters. The running graph is connected, but only just. Two of its eight edges are bridges, and losing either one splits the network in half. In the running example the edges DF and FG are bridges, and D and F are cut vertices. Note what is not a bridge: none of the five edges that lie on a cycle, because a cycle always offers a detour. That is the general rule, and it is worth stating as a fact rather than an observation. An edge is a bridge exactly when it lies on no cycle. The same intuition explains why redundancy in real networks is measured in cycles: a second route is a cycle through the first. For directed graphs the notion splits in two: a digraph is weakly connected if ignoring arc directions leaves a connected graph, and strongly connected if every vertex reaches every other by following arcs the right way round. Tarjan's 1972 algorithm finds the strongly connected components in linear time, and in a dependency graph one with more than a single vertex is precisely a circular dependency. Computationally all of this is cheap. One breadth first or depth first sweep labels every component in O(n + m), and bridges and cut vertices come out of a single depth first search augmented with Tarjan's low-link values, also in O(n + m). There is rarely a reason not to check connectivity before doing anything else. 8. Trees: the most useful special case A tree is a connected graph with no cycles. It is the single most important special case in the subject, partly because trees show up everywhere in computing and partly because a great many hard problems become easy when the input happens to be one. What makes trees remarkable is how many different-sounding descriptions pick out the same objects. For a graph G on n vertices, all of the following are equivalent, and any one can serve as the definition: - G is connected and has no cycles. - G is connected and has exactly n - 1 edges. - G has no cycles and has exactly n - 1 edges. - Between every pair of vertices there is exactly one path. - G is connected, and removing any edge disconnects it, so every edge is a bridge. - G has no cycles, and adding any new edge creates exactly one cycle. The equivalence is proved as a cycle of implications, laid out in the guide to trees in graph theory. Two consequences are worth carrying around. The edge count is forced, so a "tree" with 100 vertices and 120 edges is not a tree and something upstream is wrong. And uniqueness of paths is why tree problems are easy: there is nothing to search for, because there is only ever one route. A forest is an acyclic graph that need not be connected, so it is a disjoint union of trees, and one with n vertices and c components has exactly n - c edges. A spanning tree of a connected graph is a tree subgraph containing every vertex, the cheapest skeleton that keeps the graph in one piece. Both traversals produce one for free as a side effect, and when edges carry weights, finding the lightest is the minimum spanning tree problem. Trees also come in a rooted flavour, where one vertex is singled out and the words parent, child, ancestor, subtree and depth become available, as in file systems, parse trees and heaps. Rooting is a choice laid on top of the graph rather than a property of it, which is the point of the article on rooted trees. 9. How a graph is stored in a computer Everything so far has been mathematics. The moment a machine has to answer a question about a graph, you must choose how it is laid out in memory, and that choice is not an implementation detail: it changes which operations are cheap by factors of thousands, and a badly matched representation is the most common reason a correct algorithm runs too slowly. There are three standard layouts, and they store exactly the same information. The same eight edges, three times. Which one you pick decides whether your algorithm reads one number or scans the whole structure. The edge list is the set E written out. It is compact, it is what a CSV file or an API hands you, and it is what Kruskal's algorithm wants, since that algorithm sorts edges by weight and never asks about a particular vertex. Its weakness is that "who are the neighbours of D" means scanning all m rows. The adjacency matrix is an n by n grid where cell (u, v) is 1 when the edge is present. Checking whether two given vertices are adjacent is a single lookup, and for undirected graphs the matrix is symmetric, so it stores every fact twice. The cost is space: n squared cells whether or not there are any edges. It is also the gateway to spectral methods, where the eigenvalues of the matrix, or of the closely related Laplacian, expose clustering and connectivity, the subject of spectral graph theory in machine learning. The adjacency list keeps, for each vertex, the list of its neighbours. Iterating over the neighbours of v costs O(deg v), which is optimal, and the total space is O(n + m). This is the default in practice, and the layout every traversal below assumes. OperationEdge listAdjacency matrixAdjacency list SpaceO(m)O(n2)O(n + m) Is u adjacent to v?O(m)O(1)O(deg u) Visit all neighbours of uO(m)O(n)O(deg u) Add an edgeO(1)O(1)O(1) Delete an edgeO(m)O(1)O(deg u) Iterate over all edgesO(m)O(n2)O(n + m) What decides the argument in practice is that real networks are sparse: the average number of neighbours stays in the tens no matter how large the network grows, since junctions have three or four roads and people have a bounded number of friends. For a graph with a million vertices and five million edges the adjacency list holds about ten million entries, while the matrix would need a trillion cells, several terabytes for a graph that otherwise fits comfortably in memory. Use the matrix when the graph is small, genuinely dense, or destined for linear algebra; use the adjacency list otherwise. The deeper treatment, including compressed sparse row layouts, is in graph representation. Building an adjacency list from an edge list takes four lines, and the comment in the middle is the part beginners get wrong: edges = [('A','B'), ('A','C'), ('B','C'), ('B','D'), ('C','E'), ('D','E'), ('D','F'), ('F','G')] graph = {v: [] for v in 'ABCDEFG'} # start from V, so isolated vertices survive for u, v in edges: graph[u].append(v) graph[v].append(u) # omit this line for a directed graph Starting from the vertex set rather than the edges is what keeps isolated vertices in the graph. Build the dictionary lazily from the edge list and any vertex with no edges silently disappears, changing n and breaking every calculation that divides by it. 10. Traversal: breadth first and depth first Almost every graph algorithm is a traversal with bookkeeping attached. There are two, they differ by one data structure, and understanding that difference is the highest-value hour a beginner can spend on the subject. Both start at a vertex, keep a collection of vertices discovered but not yet processed, and repeat: take one out, look at its neighbours, add the new ones. Breadth first search takes them out in the order they went in, using a queue. Depth first search takes out the most recently added, using a stack or the call stack of a recursive function. That single choice produces two completely different shapes of exploration. Same graph, same start vertex, same linear cost, two different orders and two different sets of tree edges. Here is breadth first search in full, returning the visit order, the distance from the start and the tree of parents that lets you reconstruct the actual routes: from collections import deque def bfs(graph, start): dist = {start: 0} parent = {start: None} queue = deque([start]) order = [] while queue: u = queue.popleft() # a queue: first in, first out order.append(u) for v in graph[u]: if v not in dist: # not yet discovered dist[v] = dist[u] + 1 parent[v] = u queue.append(v) return order, dist, parent order, dist, parent = bfs(graph, 'A') # order ['A', 'B', 'C', 'D', 'E', 'F', 'G'] # dist {'A': 0, 'B': 1, 'C': 1, 'D': 2, 'E': 2, 'F': 3, 'G': 4} The important property is in the dist dictionary. Because BFS finishes an entire layer before starting the next, the first time it reaches a vertex it has used the fewest possible edges, so BFS solves the shortest path problem on unweighted graphs in O(n + m). Reaching for Dijkstra when every edge costs the same is wasted work. Depth first search is the same skeleton with a stack instead of a queue: def dfs(graph, start): seen = set() order = [] stack = [start] while stack: u = stack.pop() # a stack: last in, first out if u in seen: continue seen.add(u) order.append(u) for v in reversed(graph[u]): # reversed, so the first neighbour is taken first if v not in seen: stack.append(v) return order dfs(graph, 'A') # ['A', 'B', 'C', 'E', 'D', 'F', 'G'] DFS does not give you distances, and the order in which it finishes vertices is the useful output rather than the order in which it starts them. That finishing order is what topological sorting, cycle detection, strongly connected components and bridge finding are all built from, following Tarjan's 1972 paper that turned depth first search from a technique into a toolkit. The one thing to remember. Use BFS when the question is about distance or the fewest steps, and DFS when the question is about structure: does a cycle exist, what depends on what, which pieces hang together. Both cost O(n + m) and both visit every vertex exactly once, so the choice is never about speed. A fuller comparison of the two, including the bugs each one invites, is in BFS versus DFS. 11. The classic problems and their algorithms With traversal in place, the standard catalogue is within reach. Each of these is a question people genuinely ask about real networks, and each has a named algorithm. Shortest paths. Unweighted, BFS answers it. With non-negative weights, Dijkstra's algorithm, published in a three page note in 1959, settles vertices in increasing order of distance and runs in O(m + n log n) with a good priority queue. With negative weights, Dijkstra's central assumption fails and you need Bellman-Ford, which relaxes every edge n-1 times in O(nm) and detects negative cycles as a bonus. For every pair at once, Floyd-Warshall does it in O(n3) with three nested loops, and when you have a goal and a sensible estimate of the distance remaining, A* search uses it to examine a fraction of the graph. The full decision tree is in shortest path algorithms. Minimum spanning trees. Find the cheapest set of edges that keeps a weighted graph connected. Kruskal's algorithm sorts the edges and adds any that does not close a cycle, using a union-find structure to test that in near constant time; Prim's grows one tree outward, always taking the cheapest edge leaving it. Both are greedy, both are provably optimal, and both run in O(m log n). This is the algorithm behind laying cable and fibre at minimum cost, and it appears inside clustering methods too. Ordering and flow. Given a DAG of dependencies, topological sorting produces an order in which every task follows what it depends on, in O(n + m), failing exactly when a cycle exists. Given pipes with capacities, maximum flow asks how much can move from a source to a sink; formalised by Ford and Fulkerson in 1956, it models traffic, bandwidth, supply chains and, through a standard reduction, bipartite matching. See network flow, max-flow and min-cut. Colouring. Label the vertices so that no two adjacent ones share a label, using as few labels as possible. That count is the chromatic number, and it models exam timetabling, frequency assignment and register allocation. Unlike everything above, this one is NP-hard and practice relies on heuristics. See the graph colouring problem. Tours. A Hamiltonian cycle visits every vertex exactly once, and the travelling salesperson problem asks for the cheapest one. It looks like a small variation on Euler's question from 1736, which is linear time, and it is among the hardest problems in the catalogue. Its practical cousin, routing a fleet from a depot under capacity limits, is the vehicle routing problem. ProblemAlgorithmComplexityRequires Reachability, componentsBFS or DFSO(n + m)Nothing Shortest path, unweightedBFSO(n + m)Nothing Shortest path, weightedDijkstraO(m + n log n)No negative weights Shortest path, negative weightsBellman-FordO(nm)No negative cycles All pairs shortest pathsFloyd-WarshallO(n3)No negative cycles Minimum spanning treeKruskal or PrimO(m log n)Undirected, connected Task orderingTopological sortO(n + m)Directed and acyclic Maximum flowDinic, OrlinO(nm) and betterCapacities Bipartite matchingHopcroft-KarpO(m√n)Bipartite Minimum colouringNone knownExponentialNP-hard Cheapest tour (TSP)Held-Karp, heuristicsO(n22n) exactNP-hard 12. Five results worth knowing by name A first course is partly a set of algorithms and partly a set of results that shape how you think about the objects. The handshaking lemma from section 4 is the first of them. These five come up just as constantly, in interviews, in papers and in conversation, and each can be stated in a sentence. Euler's criterion for traversing every edge (1736, completed by Hierholzer in 1873) A connected graph has a closed trail using every edge exactly once if and only if every vertex has even degree, and an open one if and only if exactly two vertices have odd degree. This is the valuable kind of theorem: it turns a search over an enormous space of routes into a check you can do by counting, in linear time. Euler's formula for planar graphs (1758) Draw a connected planar graph with no crossings and let f be the number of faces, counting the unbounded outer region. Then n - m + f = 2 The running example, drawn as in the figures, has n = 7, m = 8 and three faces: the triangle ABC, the quadrilateral BCED and the outer region, and indeed 7 - 8 + 3 = 2. The corollary has real teeth: any simple planar graph with at least three vertices satisfies m ≤ 3n - 6, so planar graphs are always sparse, and K5 with its 5 vertices and 10 edges cannot be planar since 3n - 6 is 9. Kuratowski's theorem of 1930 completes the picture: a graph is planar exactly when it contains no subdivision of K5 or K3,3. Hopcroft and Tarjan showed in 1974 that planarity can be tested in linear time. The four colour theorem (Appel and Haken, 1976) Every planar graph can be properly coloured with at most four colours, so no map needs more than four colours for countries sharing a border to differ. Francis Guthrie asked the question in 1852 and it resisted proof for 124 years. The eventual argument reduced the problem to a finite set of configurations and checked them by computer, which started a real philosophical argument about what a proof is; it was simplified in 1997 by Robertson, Sanders, Seymour and Thomas, and formally verified in Coq by Georges Gonthier in 2005. Note the asymmetry: four colours always suffice, but deciding whether three suffice is NP-complete. König's theorem (1931) In a bipartite graph, the size of a maximum matching equals the size of a minimum vertex cover. A matching is a set of edges with no shared endpoint, a way of pairing people to jobs; a vertex cover is a set of vertices touching every edge. Two apparently unrelated optimisation problems have the same answer, which is the first duality most students meet, and it is what makes maximum matching computable in polynomial time. In general graphs the equality fails and minimum vertex cover is NP-hard. The max-flow min-cut theorem (Ford and Fulkerson, 1956) In any flow network, the maximum flow from source to sink equals the total capacity of the smallest cut separating them: the most you can push through is exactly what the tightest bottleneck allows. This is duality again in its most quotable form, turning a maximisation over all flows into a minimisation over all cuts. It underlies image segmentation, project selection and reliability analysis, and König's theorem falls out of it as a special case. 13. What is easy, what is hard, and why it matters The most practically important thing a beginner can learn about graphs is not an algorithm. It is that two problems can be stated in almost the same words and sit on opposite sides of an enormous computational divide. Finding the shortest path between two vertices takes milliseconds on a graph with millions of vertices; finding the longest simple path between the same two is NP-hard and hopeless past a few dozen. Deciding whether a graph has a closed trail using every edge once is a degree check in linear time; deciding whether it has a cycle through every vertex once is NP-complete. Deciding whether two colours suffice is a single BFS; deciding whether three suffice is NP-complete. Where the standard problems sit. Recognising the column before writing code is worth more than knowing any individual algorithm in it. The formal statement is that a large family of graph problems is NP-complete, a notion introduced by Cook in 1971 and given its first substantial catalogue by Richard Karp in 1972, whose famous list of 21 problems is dominated by graph problems: clique, vertex cover, Hamiltonian circuit, chromatic number, feedback arc set and more. No polynomial time algorithm is known for any of them, and one for any one of them would give one for all of them. Nobody expects that to happen. The practical consequence is not despair, it is a change of question. When a problem lands in the right hand column you stop asking for the optimum and choose among four honest strategies: - Accept an approximation. For metric TSP, Christofides' 1976 algorithm guarantees a tour at most 1.5 times the optimum in polynomial time. - Use a heuristic and measure it. Local search such as 2-opt or simulated annealing regularly lands within a couple of per cent of optimal on thousands of cities, with no guarantee at all. - Exploit the structure of your instances. Hard in general does not mean hard for you: colouring is easy on chordal graphs, many problems are easy on trees and on graphs of small treewidth, and road networks have geometry that specialised routing algorithms exploit ruthlessly. - Solve smaller instances exactly. Integer programming solvers routinely prove optimality for TSP instances with thousands of cities. Exponential does not mean impossible, it means the ceiling arrives quickly. One caution: "NP-hard" describes worst cases as the input grows, not a verdict on your particular problem. A fuller treatment of the cost of each algorithm is in graph algorithms and complexity. 14. Where graphs actually show up The claim that graph theory is everywhere is easy to make and worth substantiating. Here is where the material in this article is doing work right now, on the device you are reading this on. Navigation. Every routing app models the road network as a weighted directed graph, junctions as vertices and road segments as arcs weighted by expected travel time. The query is a shortest path and the algorithm is an engineered descendant of Dijkstra and A*, using precomputed hierarchies so a continental route touches a few thousand vertices instead of tens of millions. One-way streets are why the graph must be directed; live traffic is why the weights change by the minute. Search and social platforms. The web is a directed graph of pages and links, and PageRank, described by Brin and Page in 1998, ranks a page by the probability that a random surfer following links ends up there, which is an eigenvector computation on the adjacency structure. On social platforms people are vertices and relationships are edges: Milgram's 1967 letter study produced the popular "six degrees", and a 2012 analysis of the whole Facebook graph put the average distance between two users at 4.74. Community detection, friend recommendation and influence estimation are all standard graph computations run at scale. Software engineering. Build systems, package managers and spreadsheet engines maintain a DAG and topologically sort it. Version control history is a DAG of commits, and a merge is a question about common ancestors. Compilers build control flow graphs for optimisation and interference graphs for register allocation, where assigning registers is literally graph colouring, and dead code elimination is a reachability query. See graph theory in software engineering. Logistics. Delivery routing is the vehicle routing problem, warehouse placement is facility location and supply chains are flow networks with capacities. Here the gap between a good and a poor algorithm is measured in fuel and payroll, and the field that studies it is operations research. Science and machine learning. A molecule is a graph of atoms and bonds, and searching a chemical database is subgraph isomorphism. Genome assembly reconstructs a sequence by finding an Eulerian path in a de Bruijn graph, which is Euler's 1736 criterion earning its keep 280 years later. Spectral clustering partitions data using the eigenvectors of a graph Laplacian, and graph neural networks generalise convolution to irregular structures by passing messages along edges. Power grids and telecoms are analysed for bridges and cut vertices because that is where cascading failures begin. A wider survey is in applications of graph theory. 15. Mistakes beginners reliably make These are the errors that show up over and over in student code, in interviews and in production bugs. Every one of them is cheap to avoid once you have seen it named. - Losing isolated vertices. Building the graph from an edge list means any vertex with no edges never appears. The order n silently changes, averages are wrong, and a component count is short. Build the vertex set first. - Marking visited too late in BFS. Mark a vertex when you push it onto the queue, not when you pop it. Marking on pop lets the same vertex enter the queue once per incident edge, which turns a linear algorithm into a memory problem. - Running BFS on a weighted graph. BFS minimises the number of edges, not their total weight. On a graph where a two-edge route costs 100 and a five-edge route costs 5, BFS confidently returns the expensive one. Weighted graphs need Dijkstra. - Running Dijkstra with negative weights. Dijkstra assumes that once a vertex is settled, no cheaper route to it can appear later. A negative edge breaks that assumption and the output is silently wrong rather than an error. Use Bellman-Ford. - Counting path length in vertices. Length is the number of edges, so a path through five vertices has length 4. This is the source of a large share of off-by-one errors in shortest path code. - Ignoring which family the graph belongs to. An algorithm that is correct on undirected graphs can be silently wrong on directed ones, and a result proved for simple graphs may fail on a multigraph. Check the family before reaching for the algorithm. - Assuming the graph is connected. Real data arrives in pieces. Check the component count before trusting any distance, diameter or average. - Recursing too deep in DFS. A recursive DFS on a path of a million vertices needs a million stack frames. Write the iterative version when the input can be large. - Adding only one direction in an undirected graph. An undirected edge must appear in both adjacency lists. Omitting the second append produces a graph that looks right in a drawing and behaves like a digraph in every traversal. - Believing the drawing. Two edges crossing in a picture means nothing. Only the sets matter, which is why "is this graph planar" is a real question and not something you settle by squinting at a diagram. - Assuming n - 1 edges means tree. It does only together with connectedness, or with acyclicity. A triangle plus an isolated vertex has 4 vertices and 3 edges and is not a tree. 16. Where to go next The useful next step is to build a graph and run something on it rather than read more definitions. Type the running example into the interactive visualizer, run breadth first search from A and watch the layers fill in, then run depth first search from the same vertex and compare the order. Ninety seconds of that does what no amount of prose does. After that, the natural sequence is the order of this article: vocabulary, traversal, weighted shortest paths, spanning trees, then the harder problems. The graph theory study roadmap lays out that path with a schedule and the structured lessons follow it interactively. For technical interviews, the graph theory for coding interviews guide covers the patterns that actually appear, alongside the algorithms cheat sheet. For textbooks: West's Introduction to Graph Theory is the standard undergraduate course in book form, Diestel's Graph Theory is the graduate reference and the source of the definition in section 2, and the graph chapters of Cormen, Leiserson, Rivest and Stein remain the clearest treatment of the implementations. A fuller comparison, including courses and video series, is in the best resources to learn graph theory. 17. Glossary Every term used above, in one place. TermMeaning Vertex (node)An element of V. The theory assumes nothing about what it is. EdgeA pair of vertices, {u, v} when undirected, (u, v) when directed. ArcA directed edge, with a tail and a head. Order, sizeThe number of vertices n, and the number of edges m. AdjacentTwo vertices joined by an edge. IncidentThe relation between an edge and one of its endpoints. Degreedeg(v), the number of edge ends at v. A loop counts twice. NeighbourhoodN(v), the set of vertices adjacent to v. Simple graphNo loops and no parallel edges. MultigraphParallel edges allowed; a pseudograph also allows loops. Walk, trail, pathAny route; a route with no repeated edge; a route with no repeated vertex. CycleA closed path of length at least 3 in a simple graph. Length, distanceEdges in a route; the length of a shortest path, written d(u, v). Connected, componentEvery vertex reachable from every other; a maximal such piece. Bridge, cut vertexAn edge, or a vertex, whose removal increases the component count. Tree, forestA connected acyclic graph; a disjoint union of trees. Spanning treeA tree subgraph containing every vertex of the graph. BipartiteVertices split in two, with every edge crossing between the parts. Complete graphKn, every pair joined, with n(n-1)/2 edges. DAGA directed graph with no directed cycle. PlanarDrawable in the plane with no edges crossing. IsomorphicIdentical up to renaming the vertices, so the same graph. Sparse, densem close to n, against m close to n2. 18. Frequently asked questions What is graph theory in simple terms? Graph theory is the study of connections. A graph is a set of objects, called vertices, together with a record of which pairs of them are joined, called edges. Nothing else is assumed, so the vertices can be cities, people, web pages or tasks. Because a great many practical questions depend only on which things are connected to which, one body of results and algorithms answers all of them at once. What maths do I need before learning graph theory? Much less than most people expect. Basic set notation, the idea of a function, and enough comfort with proofs to follow a counting argument are sufficient for a first course, and no calculus is needed anywhere. Linear algebra becomes useful if you go on to spectral methods, and probability if you go on to random graphs, but everything in this article needs arithmetic and careful reading only. What is the difference between a graph and a tree? A tree is a graph, specifically one that is connected and contains no cycles. Every tree is a graph, and most graphs are not trees. The useful properties follow from those two conditions: a tree on n vertices has exactly n-1 edges, there is exactly one path between any two vertices, and removing any edge disconnects it. Those constraints are why problems that are hard on general graphs are often easy on trees. What is the difference between BFS and DFS? Only the structure that holds the discovered vertices. Breadth first search uses a queue and explores layer by layer, so the first time it reaches a vertex it has used the fewest possible edges, which makes it the correct tool for shortest paths on unweighted graphs. Depth first search uses a stack, or recursion, and follows one branch as deep as it can before backtracking, which makes it the tool for structural questions such as cycle detection, topological ordering and finding bridges. Both visit every vertex once and both run in O(n + m) time. Where is graph theory used in real life? Route planning in navigation apps, PageRank in web search, friend and product recommendation on social platforms, dependency resolution in build systems and package managers, register allocation in compilers, genome assembly in bioinformatics, delivery routing in logistics, fraud detection in payment networks, and message passing in graph neural networks. Each one is a standard graph problem applied to a specific network. Is graph theory important for coding interviews? Yes. Graph questions are a reliable part of technical interviews at most large software companies, and the majority of them reduce to breadth first or depth first search with bookkeeping attached: grid traversal, counting islands, course scheduling by topological sort, cycle detection, and shortest paths on unweighted graphs. Fluency in the two traversals, plus the habit of building an adjacency list from whatever input format is given, covers most of what is actually asked. Why can computers not solve the travelling salesman problem? They can, for small instances, and they can get very close on large ones. What they cannot do is solve it exactly and quickly in every case, because the number of distinct tours through n cities is (n-1)!/2, which for just 20 cities is already more than 60 quadrillion. The problem is NP-hard, so no algorithm is known that escapes that growth in the worst case. In practice, exact solvers handle instances with thousands of cities, and heuristics such as 2-opt or simulated annealing land within a few per cent of optimal on much larger ones. How long does it take to learn graph theory? The foundations covered here, the definitions, both traversals and the standard problems, take most people two to four weeks of regular study. Being able to implement the classic algorithms from memory takes a couple of months of practice. The subject itself is open ended and still has active research, but the working knowledge that covers interviews and most engineering use is a small and finite body of material. 19. References The definitions, theorems, dates and complexity bounds above come from these sources, listed in chronological order. - Euler, L. (1736). "Solutio problematis ad geometriam situs pertinentis." Commentarii Academiae Scientiarum Petropolitanae 8 (published 1741), 128 to 140. The Königsberg bridges paper. - Euler, L. (1758). "Elementa doctrinae solidorum." Novi Commentarii Academiae Scientiarum Petropolitanae 4, 109 to 140. The polyhedron formula behind n - m + f = 2. - Kirchhoff, G. (1847). "Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanischer Ströme geführt wird." Annalen der Physik 148(12), 497 to 508. - Hierholzer, C. (1873). "Über die Möglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechung zu umfahren." Mathematische Annalen 6(1), 30 to 32. - Sylvester, J. J. (1878). "Chemistry and Algebra." Nature 17, 284. The first modern use of the word "graph". - Cayley, A. (1889). "A theorem on trees." Quarterly Journal of Pure and Applied Mathematics 23, 376 to 378. - Kuratowski, K. (1930). "Sur le problème des courbes gauches en topologie." Fundamenta Mathematicae 15, 271 to 283. - König, D. (1931). "Gráfok és mátrixok." Matematikai és Fizikai Lapok 38, 116 to 119. - König, D. (1936). Theorie der endlichen und unendlichen Graphen. Leipzig: Akademische Verlagsgesellschaft. - Ford, L. R. and Fulkerson, D. R. (1956). "Maximal flow through a network." Canadian Journal of Mathematics 8, 399 to 404. - Kruskal, J. B. (1956). "On the shortest spanning subtree of a graph and the traveling salesman problem." Proceedings of the American Mathematical Society 7(1), 48 to 50. - Prim, R. C. (1957). "Shortest connection networks and some generalizations." Bell System Technical Journal 36(6), 1389 to 1401. - Dijkstra, E. W. (1959). "A note on two problems in connexion with graphs." Numerische Mathematik 1, 269 to 271. - Floyd, R. W. (1962). "Algorithm 97: Shortest path." Communications of the ACM 5(6), 345. - Held, M. and Karp, R. M. (1962). "A dynamic programming approach to sequencing problems." Journal of the Society for Industrial and Applied Mathematics 10(1), 196 to 210. - Milgram, S. (1967). "The small world problem." Psychology Today 2(1), 60 to 67. - Cook, S. A. (1971). "The complexity of theorem-proving procedures." Proceedings of the Third Annual ACM Symposium on Theory of Computing, 151 to 158. - Karp, R. M. (1972). "Reducibility among combinatorial problems." In Complexity of Computer Computations, 85 to 103. New York: Plenum Press. - Tarjan, R. (1972). "Depth-first search and linear graph algorithms." SIAM Journal on Computing 1(2), 146 to 160. - Hopcroft, J. and Tarjan, R. (1974). "Efficient planarity testing." Journal of the ACM 21(4), 549 to 568. - Christofides, N. (1976). Worst-case analysis of a new heuristic for the travelling salesman problem. Report 388, Carnegie Mellon University. - Appel, K. and Haken, W. (1977). "Every planar map is four colorable." Illinois Journal of Mathematics 21(3). Part I, 429 to 490; Part II, with J. Koch, 491 to 567. - Fredman, M. L. and Tarjan, R. E. (1987). "Fibonacci heaps and their uses in improved network optimization algorithms." Journal of the ACM 34(3), 596 to 615. - Brin, S. and Page, L. (1998). "The anatomy of a large-scale hypertextual Web search engine." Computer Networks and ISDN Systems 30(1 to 7), 107 to 117. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. - Gonthier, G. (2008). "Formal proof: the four-color theorem." Notices of the American Mathematical Society 55(11), 1382 to 1393. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. - Backstrom, L., Boldi, P., Rosa, M., Ugander, J. and Vigna, S. (2012). "Four degrees of separation." Proceedings of the 4th Annual ACM Web Science Conference, 33 to 42. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Source of the definition quoted in section 2. Build the running example yourself Seven vertices, eight edges, and every definition on this page becomes something you can point at. Drop them into the visualizer, run BFS and DFS from A, and watch the two orders diverge. Open the visualizer Related Articles Vertices and Edges Explained Read more → BFS vs DFS Read more → Graph Theory Study Roadmap Read more → --- ## Topological Sort Interview Questions Source: https://learngraphtheory.org/articles/topological-sort-interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What a topological sort question is actually testing - 2. The two templates, and when each one wins - 3. Course Schedule: can every course be finished? - 4. Course Schedule II: return an order - 5. Alien Dictionary: recovering an alphabet - 6. Parallel courses: the minimum number of semesters - 7. Is the order unique? Sequence reconstruction - 8. The longest path, and the critical path - 9. Eventual safe states: sort the reverse graph - 10. Sort items by group: two levels at once - 11. The complexity answers - 12. Mistakes that fail the interview - 13. Frequently asked questions - 14. References 1. What a topological sort question is actually testing The words "topological sort" almost never appear in the question. You get courses with prerequisites, build targets, task lists, a recipe, or a dictionary in an alien alphabet, and the interview is watching for three things. Do you see the graph? Anything phrased as "X must come before Y" is a directed edge, and the answer is an ordering of the vertices. Do you get the arcs the right way round? This is the single most common failure, and it produces code that runs, returns an order, and is backwards. Do you know that the cycle check and the sort are the same computation? "Can this be scheduled" and "give me a schedule" are one algorithm with two different return statements. After that, the variations are all the same sweep carrying something extra: a level number, a duration, a count, a second graph. Once the template is automatic, the interesting part of every one of these problems is the modelling, not the code. The mechanics themselves are covered in the guide to topological sorting; this page is about the eight questions that actually get asked. Every worked example below was executed by script before it was written down. 2. The two templates, and when each one wins There are exactly two implementations worth knowing, and an interviewer will accept either. Write whichever one you can produce without hesitating, and be able to say why you might want the other. Kahn's algorithm, from his 1962 paper, is the iterative one. Count how many prerequisites each vertex still has, keep the ones at zero in a queue, and emit them. from collections import deque def kahn(n, edges): # edges hold (u, v) meaning u comes before v adj = [[] for _ in range(n)] indeg = [0] * n for u, v in edges: adj[u].append(v) indeg[v] += 1 # count arcs INTO v q = deque(v for v in range(n) if indeg[v] == 0) order = [] while q: u = q.popleft() order.append(u) for v in adj[u]: indeg[v] -= 1 # u is done, so v needs one fewer if indeg[v] == 0: q.append(v) return order if len(order) == n else [] # short output means a cycle The last line carries the whole cycle test. If some vertices never reach in-degree zero, they are waiting on each other, and len(order) < n is the proof. Notice there is no visited set anywhere: the in-degree counter already guarantees each vertex is emitted exactly once. The running example for this article. Watch vertex 3: it waits until both 0 and 1 have been emitted, which is exactly what "all prerequisites first" means. On this graph the queue starts as [1, 2], and the emitted order is 1, 2, 4, 0, 5, 3, 6, 7. All eight vertices come out, so there is no cycle. The DFS version is the other template. Run a depth first search and push each vertex onto a list when it finishes, then reverse. The subtlety that interviewers probe is the colouring. WHITE, GREY, BLACK = 0, 1, 2 # unseen, on the stack, finished def dfs_topo(n, adj): colour = [WHITE] * n out = [] def visit(u): colour[u] = GREY for v in adj[u]: if colour[v] == GREY: # back edge: we found a cycle return False if colour[v] == WHITE and not visit(v): return False colour[u] = BLACK out.append(u) # push on the way OUT, not the way in return True for v in range(n): if colour[v] == WHITE and not visit(v): return [] return out[::-1] # reverse post-order Three colours, not a visited set. A plain visited set cannot distinguish an arc back into the current recursion stack, which is a cycle, from an arc into a branch that was already finished, which is not. Say that sentence in the interview and the cycle-detection follow-up is already answered. Which to use? Kahn if the problem wants levels, counts, lexicographic order, or anything that benefits from processing sources in waves. DFS if you are already writing a depth first search for another reason, or if you want the reverse post-order for a strongly connected components pass. Both are O(V + E). The one practical difference: the recursive DFS needs stack depth proportional to the longest chain, which on an adversarial input of a hundred thousand chained tasks will hit Python's default recursion limit, and Kahn will not. 3. Course Schedule: can every course be finished? The question. There are n courses and a list of pairs [a, b] meaning "to take course a you must first take course b". Can you finish all of them? The modelling step is the entire question, and it is where most candidates lose it. The pair [a, b] says b before a, so the arc runs b → a, and it is indeg[a] that goes up. Getting this backwards still produces a valid topological sort of a different graph, so nothing crashes and the answer is silently wrong on any asymmetric test case. def can_finish(n, prerequisites): edges = [(b, a) for a, b in prerequisites] # b before a return len(kahn(n, edges)) == n That is it: run the sort, compare the count. Say out loud that a schedule exists exactly when the prerequisite graph is acyclic, because a cycle is a set of courses that each wait on another. One arc added, and Kahn emits 2 vertices instead of 8. The six that never move are exactly the cycle and everything downstream of it. Add the arc 7 → 2 to the running graph and the queue starts with only vertex 1, emits 1 and 4, and then empties. Six vertices are left stuck, and they are precisely the cycle 2 → 0 → 3 → 6 → 7 → 2 together with vertex 5, which sits downstream of it. The follow-up: which courses are the problem? The leftovers with non-zero in-degree are the vertices on or after a cycle, which is usually the answer they want. If they insist on the cycle itself rather than everything blocked by it, you need the DFS version: when you meet a grey vertex, the current recursion stack from that vertex onwards is the cycle. The trap. Reversing the arcs. Read the pair aloud as "a depends on b, so b comes first" before you type, and confirm the direction with the interviewer on a two-element example. 4. Course Schedule II: return an order The question. Same input, but return a valid order, or an empty list if none exists. This is kahn unchanged, which is why the two questions are usually asked back to back. The only new idea is one you must volunteer: the order is not unique, and the grader accepts any valid one. Two templates, two different answers, both correct. Counting them by brute force, this eight-vertex graph admits 49 valid orders. Kahn returns 1, 2, 4, 0, 5, 3, 6, 7 and the DFS returns 2, 5, 1, 4, 0, 3, 6, 7. Neither is more correct than the other, and an exhaustive count says this graph has 49 distinct valid orders. If your solution is being diffed against one expected answer, that is a broken test, not a broken solution. The follow-up: return the lexicographically smallest order. Replace the queue with a min-heap. At each step you pop the smallest available vertex rather than the earliest queued one, which greedily fixes the smallest possible value at every position. The cost goes from O(V + E) to O(V + E log V), and being able to state that trade-off is the point of the follow-up. On the running graph the smallest order is 1, 2, 0, 3, 4, 5, 6, 7. The trap. Returning order without the length check. On a cyclic input you hand back a partial schedule that looks entirely plausible, and every automated test with a cycle fails while your local run of the happy path passes. 5. Alien Dictionary: recovering an alphabet The question. You are given words sorted according to an unknown alphabet. Recover an order of the letters consistent with that sorting, or report that none exists. Nothing here looks like a graph until you notice what "sorted" tells you. Compare two adjacent words, find the first position where they differ, and you have learnt exactly one fact: that letter of the first word precedes that letter of the second. Everything after the first difference tells you nothing. Then topologically sort the letters. def alien_order(words): adj = {c: set() for w in words for c in w} indeg = {c: 0 for c in adj} for w1, w2 in zip(words, words[1:]): if len(w1) > len(w2) and w1.startswith(w2): return "" # "abc" before "ab" is impossible for a, b in zip(w1, w2): if a != b: if b not in adj[a]: # do not count a duplicate twice adj[a].add(b) indeg[b] += 1 break # only the FIRST difference counts ... # then Kahn over the letters Three details in six lines, and interviewers check all three. Only adjacent pairs. Comparing every pair of words adds edges the input does not justify. Only the first differing position, then break. The prefix rule: if a word is a strict prefix of the one before it, the input contradicts itself and the answer is the empty string, with no graph built at all. On the classic input ["wrt", "wrf", "er", "ett", "rftt"] the comparisons give t → f, w → e, r → t and e → r, and the sort returns "wertf". On ["abc", "ab"] the prefix rule fires and returns "". On ["z", "x", "z"] the edges z → x and x → z form a cycle, so the length check returns "" too. The follow-up: is the alphabet you returned the only one? That is the uniqueness question of section 7: the order is forced exactly when the queue holds a single letter at every step. Any letter that never appears in a comparison floats free, and its position is arbitrary. The trap. Seeding the graph from the letters that appear in comparisons rather than from every letter in every word. Letters that are never compared still have to appear in the output, and dropping them is the failure that gets caught by a hidden test rather than by your own. 6. Parallel courses: the minimum number of semesters The question. You may take any number of courses at once, as long as every prerequisite is already done. What is the fewest semesters needed? The answer is the number of levels in the DAG, and the level of a vertex is one more than the largest level among its predecessors. Carry that number through the same sweep. def min_semesters(n, edges): order = kahn(n, edges) if len(order) != n: return -1 # a cycle: never finishes level = [1] * n for u in order: # every predecessor of u is final for v in adj[u]: level[v] = max(level[v], level[u] + 1) return max(level) Because the loop runs in topological order, every predecessor of u has already contributed before u is read, which is the property that makes one pass enough. On the running graph the levels are {1, 2}, then {0, 4, 5}, then {3}, then {6}, then {7}, so the answer is 5 semesters. The same forward sweep answers both questions. Carry a counter and you get semesters; carry a duration and you get the deadline. The follow-up: what if you can take at most k courses per semester? The easy answer collapses. Unlimited parallelism is linear because greedily taking everything available is optimal; capping the width turns it into precedence-constrained scheduling on k machines, which is NP-hard in general. Two identical machines with unit tasks is the classic tractable case, solved by Coffman and Graham in 1972. Recognising that the follow-up changes complexity class, rather than trying to patch the loop, is what the interviewer is listening for. The trap. Assigning a level the first time a vertex is reached, as though this were an ordinary BFS from the sources. A vertex must wait for its slowest predecessor, so the level is a maximum, not a first arrival. The BFS-by-waves variant works only if you pop a whole layer at a time and never look at a vertex before its in-degree hits zero. Topological Sort Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 7. Is the order unique? Sequence reconstruction The question. Given a DAG, decide whether it has exactly one valid topological order. The usual dressing is sequence reconstruction: you are handed a sequence and a set of subsequences, and asked whether the sequence is the only one consistent with them. The test is one line inside Kahn's loop. while q: if len(q) > 1: return False # a choice existed, so the order is not forced u = q.popleft() ... If the queue ever holds two vertices, both are available and either may come next, so at least two valid orders exist. If it holds exactly one at every step, no choice was ever made and the order is forced. There is a second, equivalent way to say it that impresses: the order is unique exactly when consecutive vertices in it are joined by an arc, that is, when the topological order is a Hamiltonian path in the DAG. Both formulations are checkable in O(V + E), and quoting the Hamiltonian path version shows you understand why uniqueness is a structural property rather than an accident of the queue. On the running graph the queue sizes at the eight pops are 2, 2, 3, 2, 2, 1, 1, 1. The very first step already offers a choice between 1 and 2, so the order is not unique, which the 49 valid orders counted in section 4 confirm. On the chain 0 → 1 → 2 → 3 the sizes are 1, 1, 1, 1 and the order is forced. The follow-up: sequence reconstruction itself. Build the graph from consecutive pairs in each subsequence, then run the check above and additionally verify that the emitted order equals the given sequence. Both conditions are needed: a unique order that differs from the sequence you were handed is still a "no". The trap. Checking the queue size only once, at the start. A graph can begin with a single source and branch three steps later, so the comparison has to run on every iteration. Worth knowing: "every level holds exactly one vertex" is an equivalent test, because a forced order makes the levels a strict chain, so reasoning in levels is not wrong, it just costs you a second pass to compute them. 8. The longest path, and the critical path The question. Each task takes a known number of days and cannot start until its prerequisites are done. When does the project finish, and which tasks are the ones that decide it? This is the longest path problem, and on a general graph it is NP-hard. On a DAG it is linear, and the reason is the topological order: every predecessor of a vertex is final before that vertex is read, so a single forward pass settles it. def critical_path(n, edges, dur): order = kahn(n, edges) finish = list(dur) # earliest finish if nothing blocks it prev = [-1] * n for u in order: for v in adj[u]: if finish[u] + dur[v] > finish[v]: finish[v] = finish[u] + dur[v] prev[v] = u # remember who forced the delay end = max(range(n), key=lambda v: finish[v]) path = [] while end != -1: path.append(end); end = prev[end] return max(finish), path[::-1] Give the running graph the durations 3, 2, 4, 5, 1, 2, 6, 3 for vertices 0 to 7 and the earliest finish times come out as 7, 2, 4, 12, 3, 6, 18, 21. The project takes 21 days and the critical path is 2 → 0 → 3 → 6 → 7, whose durations add to exactly 21. That chain is what a manager means by "the critical path": slip any task on it by a day and the whole project slips by a day, while task 5 has slack and can drift for days before anyone notices. This is the Kelley and Walker method from 1959, and saying its name out loud costs nothing. The follow-up: shortest path instead. Change the comparison to < and you have single-source shortest paths on a DAG, in O(V + E), and it works with negative weights, which Dijkstra cannot. Any time an interviewer mentions negative edges on an acyclic graph, this is the answer, not Bellman-Ford. Compare with the general case in shortest path algorithms. The trap. Relaxing in the wrong order. Iterating over vertices 0 to n-1 instead of over the topological order gives a value that depends on the labelling: on this graph it silently reports 17 instead of 21, because vertex 0 is read before vertex 2 has contributed to it. The whole point of the topological order is that it makes one pass sufficient. 9. Eventual safe states: sort the reverse graph The question. A node is safe if every path leaving it reaches a terminal node, so you can never get stuck in a cycle. Return all safe nodes in ascending order. Phrased forwards this is awkward. Reverse every arc and it becomes a topological sort: peel off the nodes with out-degree zero, which are the terminals, and every time a node's remaining out-degree hits zero, all of its successors were safe, so it is safe too. def safe_nodes(graph): n = len(graph) rev = [[] for _ in range(n)] outdeg = [len(graph[u]) for u in range(n)] for u in range(n): for v in graph[u]: rev[v].append(u) q = deque(v for v in range(n) if outdeg[v] == 0) # terminals safe = [] while q: u = q.popleft() safe.append(u) for p in rev[u]: outdeg[p] -= 1 if outdeg[p] == 0: q.append(p) return sorted(safe) It is Kahn's algorithm with in-degree replaced by out-degree and the arcs reversed, which is worth saying explicitly because it demonstrates you recognise the template under a disguise. On the standard example [[1,2], [2,3], [5], [0,5], [5], [], []] the answer is [2, 4, 5, 6]: nodes 5 and 6 are terminal, 2 and 4 lead only into them, and 0, 1 and 3 sit on the cycle 0 → 1 → 3 → 0. The follow-up: do it with DFS instead. Three colours again. A node is safe if no arc from it reaches a grey vertex, and you can memoise the result per node so the whole thing stays linear. Interviewers often want to hear both, because the reverse-graph version is the one candidates rarely find on their own. The trap. Answering "the nodes not on a cycle". Node 3 lies on no cycle of its own by that reading, yet it has an arc into the cycle through 0, so it is unsafe. Safety is about every path from the node, not about the node itself. 10. Sort items by group: two levels at once The question. Items belong to groups, some items must precede others, and items in the same group must end up contiguous in the output. Return a valid order or an empty list. This is the hard variant, and the insight is small: run two topological sorts. One over the groups, using an arc between groups whenever an item in one must precede an item in another, and one over the items inside each group. Then concatenate the groups in group order, each filled with its own sorted items. The only fiddly part is the ungrouped items. An item with group -1 is constrained by nobody's grouping, so give each one a fresh private group of its own. Merging them all into a single group instead is the classic wrong answer: it forces unrelated items to be contiguous and can turn a solvable instance into an unsolvable one. Either sort failing means the whole instance fails, so the length check runs twice. The complexity stays O(V + E) across both passes, since every item and every dependency is touched a constant number of times. The follow-up: is one course a prerequisite of another? That is Course Schedule IV, and it wants reachability rather than an order. Process vertices in topological order and union each vertex's reachable set into its successors, using bitsets: O(V × E / 64) in practice, and the topological order is what guarantees a set is complete before it is copied forward. The trap. Sorting the groups but forgetting that a group can also be a cycle with itself through two items in different groups. Build the group graph from item dependencies only when the two groups differ, or you create self-loops that fail the sort for no reason. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together: the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 11. The complexity answers Have these ready, because they are asked verbatim and the answer is short. VariantTimeSpaceWhy Kahn, or DFSO(V + E)O(V + E)Every vertex is emitted once, every arc is relaxed once Lexicographically smallestO(V + E log V)O(V + E)The queue becomes a heap Levels, or longest pathO(V + E)O(V + E)One extra array carried through the same sweep Uniqueness checkO(V + E)O(V + E)One comparison per pop Reachability between all pairsO(V × E / 64)O(V2 / 64)Bitset union in topological order Two things to add unprompted. First, the graph is usually not given to you as an adjacency list: it arrives as a list of pairs, and building the list costs O(V + E) too, so quoting a bound that ignores construction is wrong. Second, a topological sort is not a comparison sort and is not bound by O(n log n): it is linear precisely because the input already supplies the ordering constraints rather than making you discover them. The recursive DFS version also uses O(V) stack depth in the worst case, which is a real limit rather than a theoretical one. A chain of 100,000 tasks exhausts Python's default recursion limit of 1,000 long before it exhausts memory. 12. Mistakes that fail the interview Ordered by how often they appear; the first two account for most rejected solutions. - Building the arcs backwards. The pair [a, b] in Course Schedule means b before a. Reversed, the code still returns an order, it is just an order for the mirrored problem. Read the pair aloud before typing. - Dropping the len(order) == V check. Without it a cyclic input yields a partial schedule that looks fine. The count is the cycle test, and the order is the by-product. - Using a visited set in the DFS version. Two states cannot distinguish a back edge from an arc into a finished branch, so you either miss cycles or invent them. Three colours, every time. - Losing vertices with no edges. Building the graph from the pair list alone silently drops every course with no prerequisites and no dependents. Start from the vertex count you were given. - Assigning levels on first arrival. A vertex waits for its slowest predecessor, so its level is a maximum over predecessors, not the first value that reaches it. - Relaxing outside the topological order. The longest-path and shortest-path sweeps are correct only because every predecessor is final when a vertex is read. Iterating over vertex indices instead quietly returns a smaller number. - Double counting duplicate dependencies. If the input can repeat a pair, either deduplicate before counting in-degrees or decrement once per stored arc. Counting a duplicate in one place and not the other leaves a vertex permanently stuck at in-degree one. - Claiming the order is unique. It rarely is, and asserting it invites the follow-up you have not prepared. Say "a valid order" and offer the uniqueness test if they want it. - Not asking about the input. Can dependencies repeat? Can a course depend on itself? Are vertex ids dense integers or arbitrary strings? Each answer changes the first ten lines you write. The habit that prevents most of these: before writing anything, say which direction the arcs run and what the answer is when the sort comes up short. If you cannot state both in one sentence, you are not ready to type yet. 13. Frequently asked questions What is a topological sort in simple terms? + It is an order of the vertices of a directed graph in which every arc points forwards, so nothing appears before something it depends on. Courses after their prerequisites, build targets after their inputs, tasks after the tasks that block them. It exists if and only if the graph has no directed cycle, and finding one takes O(V + E) time. Kahn or DFS: which should I write in an interview? + Whichever you can write without hesitating, since both are O(V + E) and both are accepted. Kahn is the better default: it is iterative so there is no recursion limit, its cycle test is a length comparison rather than a colour argument, and it extends naturally to levels, to lexicographic order with a heap, and to anything processed in waves. Reach for DFS when you already need a depth first search for another part of the problem, or when you want reverse post-order for a strongly connected components pass. How do I detect a cycle with a topological sort? + With Kahn, count what comes out: if fewer than V vertices are emitted, the ones left behind never reached in-degree zero and are exactly the vertices on or after a cycle. With DFS, colour the vertices white, grey and black, where grey means currently on the recursion stack; an arc into a grey vertex is a back edge, and a back edge is a cycle. A two-state visited set cannot make that distinction and will report cycles that do not exist. Is the topological order unique? + Almost never. The eight-vertex graph used throughout this article has 49 valid orders. The order is unique exactly when Kahn's queue holds a single vertex at every step, which is the same as saying that consecutive vertices in the order are joined by an arc, so the order is a Hamiltonian path in the DAG. If a problem statement expects one specific answer, it is usually asking for the lexicographically smallest one, which you get by replacing the queue with a min-heap. Can you topologically sort an undirected graph? + No, and the question is worth answering carefully because it is sometimes a test. An undirected edge imposes no order between its endpoints, so there is nothing to sort. If a problem hands you an undirected graph and asks for an ordering, either the direction is implied somewhere in the statement and you have to recover it, or the intended technique is something else, such as peeling leaves for minimum height trees. Why is the longest path easy on a DAG but hard in general? + Because a topological order lets you settle each vertex once. Every predecessor of a vertex has its final value before that vertex is read, so a single forward pass is enough and the cost is O(V + E). On a graph with cycles no such order exists, a path may not repeat vertices, and the longest simple path problem is NP-hard. This is why project scheduling, which is longest path with durations, is a linear-time computation in practice. Which interview problems are secretly topological sort? + Course Schedule I and II, Alien Dictionary, Parallel Courses, Sequence Reconstruction, Find Eventual Safe States, Sort Items by Group, Course Schedule IV, Minimum Time to Complete All Tasks, and any build-order, task-scheduling or dependency-resolution question. The tell is the phrase "must come before", or an input of pairs where the two elements are not symmetric. 14. References The papers that introduced these techniques and the texts that analyse them, in chronological order. - Kelley, J. E. and Walker, M. R. (1959). “Critical-path planning and scheduling.” Proceedings of the Eastern Joint Computer Conference, 160–173. - Kahn, A. B. (1962). “Topological sorting of large networks.” Communications of the ACM, 5(11), 558–562. - Knuth, D. E. (1968). The Art of Computer Programming, Volume 1: Fundamental Algorithms, Section 2.2.3. Addison-Wesley. - Coffman, E. G. and Graham, R. L. (1972). “Optimal scheduling for two-processor systems.” Acta Informatica, 1(3), 200–213. - Tarjan, R. E. (1972). “Depth-first search and linear graph algorithms.” SIAM Journal on Computing, 1(2), 146–160. - Tarjan, R. E. (1976). “Edge-disjoint spanning trees and depth-first search.” Acta Informatica, 6(2), 171–185. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Section 22.4. MIT Press. - Sedgewick, R. and Wayne, K. (2011). Algorithms, 4th edition, Section 4.2. Addison-Wesley. - McDowell, G. L. (2015). Cracking the Coding Interview, 6th edition. CareerCup. - Skiena, S. S. (2020). The Algorithm Design Manual, 3rd edition, Section 5.10. Springer. --- ## Union-Find Interview Questions Source: https://learngraphtheory.org/articles/union-find-interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What a union-find question is actually testing - 2. The template, and the two lines that matter - 3. Counting connected components - 4. Redundant Connection: the edge that closes a cycle - 5. Islands II: why BFS loses when the grid changes - 6. Accounts Merge: when the elements are not integers - 7. Most Stones Removed: choosing what to union - 8. Evaluate Division: weighted union-find - 9. Equality equations: the order you process them in - 10. Kruskal: union-find inside a spanning tree - 11. The complexity answers - 12. Mistakes that fail the interview - 13. Frequently asked questions - 14. References 1. What a union-find question is actually testing Union-find is the rare interview topic where the implementation is not the difficulty. Fifteen lines, two optimisations, no edge cases worth arguing about. Interviewers know that, which is why the questions are built somewhere else entirely. They test three things. Do you recognise a connectivity question? Anything phrased as "are these two in the same group", "how many groups are there", or "which change merges two groups" is union-find, even when the words are accounts, stones, equations or cables. Do you know when it beats a traversal? A single static graph can be swept by BFS or DFS just as fast; union-find wins when the edges arrive one at a time and the answer is needed after each one. Can you choose the elements? This is where the hard questions live, and where section 7 spends its time. The eight problems below are the ones that recur, each with the solution, the follow-up, and the error that loses the offer. Every worked example on this page was executed by script. If you want the data structure itself derived from scratch rather than recapped, that is the guide to union-find. 2. The template, and the two lines that matter Write this without thinking. Two optimisations, one line each, and interviewers ask about both by name. class DSU: def __init__(self, n): self.parent = list(range(n)) self.size = [1] * n self.count = n # number of components, free of charge def find(self, x): root = x while self.parent[root] != root: root = self.parent[root] while self.parent[x] != root: # PATH COMPRESSION: flatten the walk self.parent[x], x = root, self.parent[x] return root def union(self, a, b): ra, rb = self.find(a), self.find(b) if ra == rb: return False # already together: nothing to merge if self.size[ra] < self.size[rb]: # UNION BY SIZE: small tree under big ra, rb = rb, ra self.parent[rb] = ra self.size[ra] += self.size[rb] self.count -= 1 return True Three details to say out loud while you type them. union returns a boolean, and that return value answers half the questions on this page: False means the two were already connected, which means the edge you just tried closes a cycle. count is maintained by the merges, so component counting never needs a second pass. And the find above is iterative, which matters on a chain of a hundred thousand elements where the recursive version dies on the call stack. The same eight elements, the same seven unions, two data structures. The left one is what you get by writing parent[rb] = ra without checking the sizes. Why both optimisations? Union by size alone bounds the depth at O(log n), because a tree only gets taller when two trees of equal size merge. Path compression alone also gives O(log n) amortised. Together they give O(α(n)) amortised per operation, which is section 11. If you can only remember one, remember path compression: it is one line and it does most of the work in practice. One implementation note worth volunteering: union by rank and union by size are interchangeable for the bound. Rank stores an upper bound on the height, size stores the element count. Size is more useful in interviews because half the follow-ups ask for the size of the resulting component, and you already have it. 3. Counting connected components The question. Given n nodes and a list of undirected edges, how many connected components are there? The classic phrasing is LeetCode 323, and Number of Provinces is the same question with an adjacency matrix. With the template above there is no algorithm left to write. def count_components(n, edges): dsu = DSU(n) for a, b in edges: dsu.union(a, b) return dsu.count The running example for this article. Seven of the eight unions merge something; the highlighted one does not, and that single fact is the next four questions. On the running example, ten elements and the eight unions (0,1) (2,3) (1,2) (4,5) (6,7) (5,6) (0,3) (8,9) leave three components: {0,1,2,3}, {4,5,6,7} and {8,9}, of sizes 4, 4 and 2. Seven unions merged; union(0, 3) did not, because 0 and 3 were already in the same tree by then. Run one find on every element afterwards and the structure ends up as parent = 0 0 0 0 4 4 4 4 8 8: every element points straight at the root of its component, so every later query is a single hop. That flattening is path compression paying for itself. The follow-up: why not just run DFS? On a static graph, do. Both are linear and DFS needs no extra structure, so reaching for union-find on a fixed edge list is a small red flag rather than a plus. The honest answer is that union-find earns its place when the edges arrive over time, when you need the answer after each arrival, or when the graph is too large to hold as an adjacency list but the pairs stream past. Saying this unprompted separates you from candidates who pattern-match on the word "components". The trap. Returning len(set(find(v) for v in range(n))) without calling find, and instead counting distinct values of parent. Before compression the parent array holds intermediate nodes, not roots, so the count comes out too high. Maintain count in union and the question does not arise. 4. Redundant Connection: the edge that closes a cycle The question. A tree on n nodes has had one extra edge added. Find the edge that can be removed, and if several qualify, return the one that appears last in the input. This is the boolean returned by union, and nothing else. def find_redundant(edges): dsu = DSU(len(edges) + 1) for a, b in edges: if not dsu.union(a, b): # a and b were already connected return [a, b] # so this edge closes a cycle Process the edges in order and the first one whose union returns False is the answer. It is also automatically the last such edge in the input, because a tree plus one edge has exactly one cycle, so exactly one edge fails. On [[1,2],[2,3],[3,4],[1,4],[1,5]] the answer is [1,4], and on the triangle [[1,2],[1,3],[2,3]] it is [2,3]. The follow-up: what if the graph is directed? That is Redundant Connection II, and it is a genuinely harder problem rather than a variation. A directed version can fail in two ways: a node with two parents, or a cycle, and it can have both at once. The technique is to find the node with in-degree two, tentatively remove each of its two candidate edges, and test whether the rest forms a valid rooted tree with union-find. Knowing that the directed case splits into cases is enough; interviewers rarely make you write it. The trap. Using DSU(n) when the nodes are labelled 1 to n. Every union-find bug of this class is an off-by-one on the array size, and it surfaces as an index error on the very last node rather than as a wrong answer. Allocate n + 1 and ignore slot zero. Union-Find Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Islands II: why BFS loses when the grid changes The question. An empty m × n grid of water. Land is added one cell at a time. After each addition, report how many islands exist. This is the question that justifies the whole data structure, so treat it as the one to get right. Counting islands on a fixed grid is a flood fill, and it costs O(mn). Doing that after every one of k additions costs O(k × mn), which is quadratic and will time out. Union-find turns each addition into a constant amount of work, because adding land can only ever merge islands, never split them. def num_islands2(m, n, positions): dsu, seen, out, count = {}, set(), [], 0 for r, c in positions: if (r, c) in seen: # a repeated position adds nothing out.append(count) continue seen.add((r, c)) dsu[(r, c)] = (r, c) # a new island of one cell count += 1 for dr, dc in ((1,0), (-1,0), (0,1), (0,-1)): nb = (r + dr, c + dc) if nb in seen and union(dsu, (r, c), nb): count -= 1 # merged with a neighbour out.append(count) return out Each new cell starts as its own island, then merges with at most four neighbours, so each step costs O(α) and the whole run is O(k α(mn)). On a 3 by 3 grid with land added at (0,0), (0,1), (1,2), (2,1) the answers are 1, 1, 2, 3: the second cell joins the first, then the next two are isolated. Add (1,1) as a fifth move and it touches all three, so the sequence ends 1, 1, 2, 3, 1. The follow-up: what if land can also be removed? Say plainly that union-find does not support deletion, because there is no way to undo a merge once the paths are compressed. The real answers are to process the operations offline in reverse, turning deletions into additions, or to use a union-find with rollback, which keeps an undo stack and therefore gives up path compression in exchange for union by rank alone at O(log n). Naming "offline reversal" is usually enough. The trap. Forgetting that the same position can appear twice in the input. Adding land where land already exists must not increment the count, and the guard is one line. It is also the only hidden test case in this problem. 6. Accounts Merge: when the elements are not integers The question. Each account is a name followed by a list of emails. Two accounts belong to the same person when they share any email. Merge them and return the emails of each person, sorted. The structure is trivially union-find. What the question really tests is the plumbing: your elements are strings, and the array-based DSU needs integers. ids = {} for account in accounts: for mail in account[1:]: if mail not in ids: ids[mail] = len(ids) # assign each email a dense integer owner[mail] = account[0] dsu = DSU(len(ids)) for account in accounts: first = ids[account[1]] for mail in account[2:]: dsu.union(first, ids[mail]) # chain every mail to the first one Union every email in an account to the first email of that account, which is enough to make the whole account one component, then bucket the emails by root and sort each bucket. Merging John [a, b], John [c, b], Mary [m] and a second John [z] gives three people: John with a, b, c, Mary with m, and a different John with only z. That last group is the point of the question. The name is not the identity. Two accounts with the same name and no shared email are two different people, and a candidate who unions by name gets a plausible wrong answer that the sample input is deliberately built to catch. The follow-up: could you avoid the id mapping? Yes, by storing parent as a dictionary keyed by the string itself, which costs a hash per access instead of an array index. It is cleaner to write and slower to run, and saying which trade-off you are making is what is being marked. In a language without dictionaries in the hot path, or when the same structure is reused millions of times, the dense integer mapping wins. The trap. Building the name lookup from the root's email rather than keeping a map from email to name. After compression the root can be any email in the group, and if you recorded the name against a specific one you will attach the wrong name to a merged account. 7. Most Stones Removed: choosing what to union The question. Stones sit on a grid. You may remove a stone if it shares a row or a column with another remaining stone. What is the maximum number you can remove? Two insights, and the second is the one that makes it a good interview question. First, from any connected group of stones you can remove all but one. Remove them in the reverse of the order a spanning tree of the group is built, taking leaves first, and the last stone standing keeps the group legal at every step. So the answer is total stones - number of components, and the whole problem reduces to counting components. Second, and this is the part that is hard under pressure: do not union the stones. Union the rows and the columns. The elements are the rows and the columns, and each stone is one union between them. Six stones, six elements, one component, five removable. Each stone at (r, c) becomes a single union(row r, column c). Two stones end up connected exactly when they share a row or a column, or are linked by a chain of stones that do, which is the relation the problem describes. It also turns an O(k2) pairwise comparison into O(k α). On the six-stone example the whole board collapses to one component and the answer is 5. On [[0,0],[0,2],[1,1],[2,0],[2,2]] there are two components and the answer is 3. The follow-up: how do you keep rows and columns from colliding? They live in the same structure, so row 2 and column 2 must be different elements. Offset the columns by a constant larger than any row index, commonly c + 10001 for the stated limits, or use a dictionary keyed by ("r", r) and ("c", c). Mentioning the collision before the interviewer does is worth a lot on this problem. The trap. Counting components over all rows and columns that exist rather than only those that actually hold a stone. Empty rows are isolated elements and each one inflates the component count, so the answer comes out too small. Only create an element the first time a stone needs it. 8. Evaluate Division: weighted union-find The question. You are given equations such as a / b = 2.0 and b / c = 3.0 and asked to answer queries such as a / c, returning -1 when the answer cannot be determined. Most candidates build a graph and run a DFS multiplying edge weights along the path, which is a perfectly good answer. The stronger answer is weighted union-find: store, alongside each parent pointer, the ratio of the child's value to the parent's value. Then find returns both the root and the accumulated ratio to it, and any query is one division. The weight on a pointer is the value of the child divided by the value of its parent. Path compression has to rescale it, or the structure lies. def find(x): # returns (root, value of x / value of root) if parent[x] == x: return x, 1.0 root, wp = find(parent[x]) weight[x] *= wp # rescale while the path is flattened parent[x] = root return root, weight[x] With a / b = 2 and b / c = 3, the queries come out as a / c = 6, b / a = 0.5, c / a = 1/6, a / a = 1, and -1 for anything mentioning a symbol that never appeared, which is why x / x is -1 rather than 1 in the standard problem. That last case is a deliberate trick and it catches people who special-case equal arguments before checking that the symbol exists. The follow-up: how do you detect a contradiction? If union(a, b, v) finds that a and b already share a root, do not merge; compare the implied ratio against v instead. A mismatch beyond floating point tolerance means the input is inconsistent. The same structure with addition instead of multiplication answers "is this set of offset constraints satisfiable", which is how the technique appears in scheduling problems. The trap. Compressing the path without updating the weight, which is the mistake the figure warns about. The pointers stay correct, every subsequent query silently returns a wrong number, and the bug survives any test that only checks connectivity. 9. Equality equations: the order you process them in The question. Given equations such as "a==b" and "b!=c" over single lowercase letters, decide whether they can all be true at once. The solution is four lines and one idea: two passes, equalities first. dsu = DSU(26) for e in equations: if e[1] == '=': dsu.union(ord(e[0]) - 97, ord(e[3]) - 97) for e in equations: if e[1] == '!': if dsu.find(ord(e[0]) - 97) == dsu.find(ord(e[3]) - 97): return False return True Equality is an equivalence relation, so it partitions the letters into groups that must hold the same value. Inequality is not an equivalence relation and cannot be unioned at all; it can only be checked against the finished partition. Process them in one interleaved pass and the answer depends on the input order, which is the bug this question exists to catch: ["a!=b", "a==b"] would be accepted, because the inequality is tested before the union that contradicts it. Verified outputs: ["a==b","b!=a"] is False, ["a==b","b==c","a==c"] is True, ["a==b","b!=c","c==a"] is False, and the single equation ["a!=a"] is False because a letter is always equal to itself. The follow-up: what if the variables were not single letters? Exactly the id mapping from section 6: hash each name to a dense integer, or key the parent dictionary by the name. Nothing else changes, which is a good thing to point out because it shows you see the structure as separate from the encoding. The trap. Allocating the DSU over the letters that appear rather than over all 26. It works, and it costs you the two minutes you spend building the mapping for an alphabet that is already dense and tiny. Read the constraints before writing generic code. 10. Kruskal: union-find inside a spanning tree The question. Connect all the points at minimum total cost, where the cost between two points is their Manhattan distance. This is LeetCode 1584, and it is a minimum spanning tree wearing a hat. Union-find is not the answer here, it is the component that makes the answer work. Kruskal's algorithm sorts every candidate edge by weight and accepts an edge exactly when it joins two different components, which is the boolean returned by union. edges.sort() # by weight dsu, total, used = DSU(n), 0, 0 for w, a, b in edges: if dsu.union(a, b): # only if it connects two components total += w used += 1 if used == n - 1: # a spanning tree has n-1 edges break On the five points [[0,0],[2,2],[3,10],[5,2],[7,0]] there are 10 candidate edges, Kruskal keeps four of them with weights 3, 4, 4 and 9, and the total is 20. The early exit at n - 1 edges matters on dense inputs, where the candidate list is O(n2) and most of it is never needed. The follow-up: Prim or Kruskal here? For a complete graph on n points, Kruskal builds and sorts n(n-1)/2 edges, which is O(n2 log n), while Prim with an array scan runs in O(n2) and never materialises the edge list. On a dense instance Prim is the better answer, and knowing that Kruskal is the sparse-graph algorithm is the point of the question. The trade-off is worked through in minimum spanning trees and in Kruskal's algorithm. The trap. Adding the weight before testing the union, so rejected edges still contribute to the total. It produces a number that is close enough to look right on the sample and wrong on everything else. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together: the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 11. The complexity answers This is the one topic where the honest answer is slightly awkward, and interviewers ask precisely because of that. VersionAmortised per operationSource Neither optimisationO(n)The chain in the figure above Union by size or rank onlyO(log n)Depth doubles only on equal merges Path compression onlyO(log n)Tarjan and van Leeuwen, 1984 Both togetherO(α(n))Tarjan, 1975 Any pointer-based structureΩ(α(n))Fredman and Saks, 1989 α is the inverse Ackermann function, and it grows so slowly that α(n) ≤ 4 for every n that could be stored in any physical computer. So the practical answer is "effectively constant", and the correct answer is "O(α(n)) amortised, which is not the same as O(1)". The distinction is real: Fredman and Saks proved in 1989 that no structure of this kind can do better, so the α is not an artefact of the analysis. Two more numbers worth having. Space is O(n), two integer arrays. And the amortisation is per sequence, not per call: a single find can still walk a long path, it is the total over m operations that is bounded. Interviewers sometimes push on that, and "amortised, not worst case per operation" is the phrase they want. As a sanity check on how flat the trees really get: after 200,000 random unions over 100,000 elements and one find on every element, the deepest tree in the structure is one pointer. Every element points directly at its root. 12. Mistakes that fail the interview Ordered by how often they appear; the first three account for most rejected solutions. - Comparing elements instead of roots. if a == b where if find(a) == find(b) was meant. It compiles, it runs, and it answers a different question. - Skipping union by size. Path compression alone usually passes, so this survives testing and then degrades on adversarial input. Both optimisations are one line each; write both. - Recursive find on deep input. A hundred thousand chained unions is a hundred thousand stack frames. Write the iterative version, or say why the recursion depth is safe here. - Off-by-one on the array size. Nodes labelled 1 to n need DSU(n + 1). This is the single most common crash in these problems. - Unioning the wrong things. Stones instead of rows and columns, accounts instead of emails, names instead of identities. When the pairwise version looks quadratic, the elements are usually wrong. - Interleaving equalities and inequalities. Constraints that merge must all be applied before any constraint that only checks. Two passes, always. - Forgetting that union-find cannot delete. If the problem removes edges, say so immediately and offer offline reversal or a rollback structure. Trying to patch deletion into a compressed structure is a dead end. - Not maintaining the component count. Recomputing it with a loop over find after every operation turns a linear solution into a quadratic one, which is exactly the failure mode Islands II is built to expose. - Quoting O(1). It is O(α(n)) amortised. Say "effectively constant, formally inverse Ackermann" and the follow-up disappears. The habit that prevents most of these: before writing anything, say what one element represents and what it means for two of them to be in the same set. If you cannot finish both halves of that sentence, you have not modelled the problem yet, and the fifteen lines will not save you. 13. Frequently asked questions What is union-find in simple terms? + It is a structure that keeps track of which items belong to the same group, supporting two operations: find, which asks which group an item is in, and union, which merges two groups. Each group is stored as a tree of parent pointers and is identified by the root of that tree, so two items are in the same group exactly when they have the same root. It is also called a disjoint set union, or DSU. When should I use union-find instead of BFS or DFS? + Use a traversal when the graph is fixed and you sweep it once, since both approaches are linear and a traversal needs no extra structure. Use union-find when edges arrive over time and the answer is needed after each one, when the problem only ever merges groups and never splits them, or when you want the boolean "were these already connected" as part of another algorithm, which is what Kruskal's algorithm does. Union-find also does not need the adjacency list to exist at all, which matters when the pairs stream past rather than fitting in memory. Is union-find really O(1)? + No, and this is worth getting right. With union by size or rank plus path compression, m operations on n elements cost O(m times alpha of n) amortised, where alpha is the inverse Ackermann function. Alpha of n is at most 4 for any n that could physically be stored, so the practical behaviour is constant, but the bound is not O(1) and the difference is not a technicality: Fredman and Saks proved in 1989 that no structure of this kind can beat alpha. Say "effectively constant, formally inverse Ackermann, amortised rather than worst case per call". Union by rank or union by size? + Either, since both give the same asymptotic bound. Rank stores an upper bound on a tree's height and size stores how many elements it holds. Size is usually the better choice in an interview because a large share of follow-up questions ask for the size of the merged component, and with union by size you already have that number for free. Whichever you pick, attach the smaller tree under the larger one, never the reverse. Can union-find handle deletions? + Not directly. Once paths are compressed there is no record of how the trees were assembled, so a merge cannot be undone. Two standard answers exist. Process the operations offline in reverse, which turns every deletion into an addition and lets ordinary union-find run backwards. Or use a union-find with rollback, which keeps an undo stack of the changes made by each union and therefore has to give up path compression, leaving union by rank alone at O(log n) per operation. How do I use union-find when the elements are strings? + Two options. Assign each distinct string a dense integer the first time you see it and run the ordinary array-based structure, which is faster and is what you want when the structure is in a hot loop. Or store the parent map as a dictionary keyed by the string itself, which is shorter to write and costs a hash lookup per access. Both are correct; stating which trade-off you are making is what the interviewer is listening for. Which interview problems are union-find? + Number of Connected Components, Number of Provinces, Redundant Connection, Number of Islands II, Accounts Merge, Most Stones Removed, Evaluate Division, Satisfiability of Equality Equations, Min Cost to Connect All Points, Graph Valid Tree, Smallest String With Swaps and Regions Cut By Slashes. The tell is a question about whether two things belong to the same group, or a count of groups that has to survive a stream of merges. 14. References The papers that introduced these techniques and the texts that analyse them, in chronological order. - Kruskal, J. B. (1956). “On the shortest spanning subtree of a graph and the traveling salesman problem.” Proceedings of the American Mathematical Society, 7(1), 48–50. - Galler, B. A. and Fischer, M. J. (1964). “An improved equivalence algorithm.” Communications of the ACM, 7(5), 301–303. - Hopcroft, J. E. and Ullman, J. D. (1973). “Set merging algorithms.” SIAM Journal on Computing, 2(4), 294–303. - Tarjan, R. E. (1975). “Efficiency of a good but not linear set union algorithm.” Journal of the ACM, 22(2), 215–225. - Tarjan, R. E. and van Leeuwen, J. (1984). “Worst-case analysis of set union algorithms.” Journal of the ACM, 31(2), 245–281. - Fredman, M. and Saks, M. (1989). “The cell probe complexity of dynamic data structures.” Proceedings of the 21st Annual ACM Symposium on Theory of Computing, 345–354. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Chapter 21. MIT Press. - Sedgewick, R. and Wayne, K. (2011). Algorithms, 4th edition, Section 1.5. Addison-Wesley. - McDowell, G. L. (2015). Cracking the Coding Interview, 6th edition. CareerCup. - Skiena, S. S. (2020). The Algorithm Design Manual, 3rd edition, Chapter 8. Springer. --- ## Graph Theory: Supply Chain Optimization Source: https://learngraphtheory.org/articles/graph-theory-supply-chain-optimization.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Why a supply chain is a graph - 2. The anatomy: vertices, arcs and the numbers on them - 3. The network used throughout this article - 4. Lead time: shortest paths - 5. Capacity: maximum flow and the cut that limits you - 6. Cost: the transportation problem and minimum cost flow - 7. Which warehouses should exist at all? - 8. Designing the physical network: spanning trees - 9. The last mile: routing the vehicles - 10. Inside the factory: materials and schedules - 11. Resilience: what breaks, and how badly - 12. Two modelling tricks worth knowing - 13. What is easy, what is hard - 14. From model to practice - 15. Modelling mistakes that produce confident wrong answers - 16. Frequently asked questions - 17. References 1. Why a supply chain is a graph A supply chain is a set of places and a set of movements between them. Suppliers ship to plants, plants ship to warehouses, warehouses ship to stores, and something is true or false about every possible movement: it exists or it does not, it costs this much per unit, it takes this many days, it can carry at most this much per week. That description is already a graph. The places are vertices, the movements are directed arcs, and the commercial facts are numbers attached to the arcs. Nothing has been simplified away yet, and the moment the model exists, a century of algorithms becomes available: shortest path answers "how fast can this reach that", maximum flow answers "how much can we actually deliver", minimum cost flow answers "what is the cheapest plan", and connectivity answers "what happens when this burns down". This is not a metaphor invented for teaching. The mathematics of supply chains is network optimisation. Hitchcock posed the transportation problem in 1941 and Koopmans reached it independently in work presented in 1947 and published in 1949; Dantzig solved it with the simplex method in 1951; Ford and Fulkerson published the maximum flow algorithm in 1956 and the book Flows in Networks in 1962, and their examples were railway capacity and shipment planning rather than abstract graphs. The field that formalised all of it is operations research, and its central objects are graphs. What this article does is take one small network, four echelons wide, and answer every standard supply chain question on it. Every number quoted below was computed by solving the model, not estimated: the plans, the bottlenecks, the shortfalls after a failure, and the cost of each alternative. If you are new to the underlying vocabulary, the introduction to graph theory covers the definitions this article assumes. 2. The anatomy: vertices, arcs and the numbers on them Modelling is the act of deciding what to keep. Three decisions carry most of the weight. What is a vertex? Usually a physical location: a supplier site, a plant, a distribution centre, a customer region. Sometimes it is finer, such as a production line or a dock door, and sometimes coarser, such as a whole country in a strategic study. The rule is that a vertex is anything you might want to open, close, capacitate or lose, because those are the questions the model will be asked. What is an arc? A lane: an origin, a destination, and usually a mode. The same two facilities connected by road and by air are two arcs, not one, because they have different costs, times and capacities. Arcs are directed, since shipping east is not the same as shipping west; the distinction and its consequences are covered in directed versus undirected graphs. What goes on the arc? At least three numbers, and they answer different questions, so it matters which one you attach: - Capacity, in units per period. This limits what is possible and is the input to maximum flow. - Cost, per unit shipped. This decides what is cheapest and is the input to minimum cost flow. - Transit time, in days. This decides what is fast and is the input to shortest path. Beginners often collapse these into one "weight" and then wonder why the answer looks wrong. They are genuinely different objectives, and the cheapest route is frequently not the fastest one, as section 4 shows on this very network. The guide to weighted versus unweighted graphs makes the same point in the abstract. Two more attributes live on vertices rather than arcs: supply at the sources, demand at the sinks, and sometimes a fixed cost for a facility to exist at all, which is what turns a flow problem into a location problem in section 7. The whole model on one picture: 11 vertices, 14 arcs, and three numbers per arc. Everything else in this article is a question asked of this object. 3. The network used throughout this article The running example is deliberately small enough to check by hand and rich enough to break in interesting ways. Two suppliers feed two plants, the plants feed three distribution centres, and the centres serve four customer regions. EchelonNodesNumbers SuppliersS1, S290 units per week available at each PlantsP1, P2Convert supply into finished goods Distribution centresD1, D2, D3Capacities 55, 75 and 65 units CustomersC1, C2, C3, C4Demand 25, 35, 40 and 30, totalling 130 Each of the fourteen lanes carries a capacity, a cost per unit and a transit time, as drawn in the figure above. Total supply is 180 against demand of 130, so in the base case there is slack; section 5 removes it. One structural detail matters before any algorithm runs. The network is a directed acyclic graph: material only ever moves left to right, from supply towards demand. Real chains have returns, rework loops and transfers between warehouses, all of which create cycles, and the algorithms below still work. But the acyclic case is the one where the intuition is cleanest, and it is where most tactical planning models actually live. A second detail is that this is a single product, single period model. That assumption is doing a lot of work, and section 12 shows the standard graph construction that removes it. 4. Lead time: shortest paths The first question anyone asks of a network is how quickly it can respond. With transit times on the arcs, that is exactly the shortest path problem, and Dijkstra's algorithm answers it for every destination at once in O(m + n log n). Solving it from each supplier gives the service picture: FromC1C2C3C4 S16 daysS1-P1-D1-C17 daysS1-P1-D1-C29 daysS1-P2-D3-C38 daysS1-P2-D3-C4 S27 daysS2-P1-D1-C16 daysS2-P2-D2-C26 daysS2-P2-D3-C35 daysS2-P2-D3-C4 Three things fall out of this table that a spreadsheet would not have told you. The worst service in the network is 9 days, from S1 to C3, and it is the number a service level agreement has to be written against. The two suppliers are not interchangeable: S1 is faster to C1, S2 is faster to everything else, which is an argument for dual sourcing by region rather than by volume. And the fastest route to C3 from S1 goes through P2, not through the geographically obvious P1, because the P1 branch is slower at every step. Now compare with cost. The cheapest lane out of S1 is S1 → P1 at 2 per unit, and the fastest route to C3 avoids it entirely. Minimising days and minimising money are different optimisations over the same graph, and any planning tool that offers a single "best route" is quietly choosing one of them for you. The full decision tree of which algorithm applies to which variant is in shortest path algorithms. Two practical extensions are worth knowing. Adding a fixed handling time at each facility is done by putting the delay on the node, which section 12 turns into an arc. And when the question is "what is the fastest route that also costs less than X", you have a constrained shortest path problem, which is NP-hard in general and usually solved with Lagrangian relaxation or a labelling algorithm rather than plain Dijkstra. 5. Capacity: maximum flow and the cut that limits you The second question is how much the network can actually move. Add an artificial source feeding both suppliers with their available volume, and an artificial sink drawing each customer's demand, and the answer is a maximum flow computation. In the base case the answer is undramatic: all 130 units get through. What makes it interesting is where the binding constraint sits. Solving the flow problem also produces the minimum cut, and here the cut consists of the customer arcs themselves. In plain language: nothing inside the network is limiting anything, and the only reason more units do not flow is that nobody has ordered them. That is the healthy case, and it is worth confirming before anyone is asked to approve capital expenditure. Now raise every demand by 40%, a modest peak season. Demand becomes 182 units, and the network delivers 164. The max-flow min-cut theorem doing real work: the shortfall is not a vague "capacity problem", it is three named lanes and a number. The 18 missing units decompose precisely. Total supply is 180, so 2 units were never manufacturable regardless of the network. The remaining 16 are lost to structure, and the minimum cut names the structure exactly: P2 → D3 at capacity 50, D1 → C1 at 30, D2 → C3 at 35, plus the 49 units of C2 demand that sits on the source side. Those add to 164, which the max-flow min-cut theorem guarantees equals the maximum flow, and the arithmetic confirms it. This is the single most useful thing graph theory does for a supply chain, so it is worth stating plainly. The minimum cut is the investment list. Capacity added anywhere else changes nothing at all. Testing that claim on this network gives a result that no amount of intuition would produce: - 10 extra units on P2 → D3: throughput rises from 164 to 174. - 10 extra units on D2 → C3: also 174. - 10 extra units on D1 → C1: throughput rises to 165, and no further. One unit, not ten. - 10 extra units on any of the other eleven lanes: no change whatsoever. The D1 → C1 case is the instructive one. It is genuinely on the minimum cut, so the first unit of extra capacity does help, but after that a different constraint binds and the investment stops paying. A cut tells you where the wall is today; it does not promise the wall stays in the same place once you move it. In practice this is why capacity planning is done as a sequence of re-solves rather than a single ranking. 6. Cost: the transportation problem and minimum cost flow Feasibility is not a plan. The operational question is which of the many feasible plans is cheapest, and that is the minimum cost flow problem: satisfy every demand, respect every capacity, minimise the sum of flow times cost over all arcs. Its ancestor is the transportation problem, posed by Hitchcock in 1941 and independently by Koopmans, and solved efficiently by Dantzig in 1951 with a specialised simplex method. The modern general form is solved by the network simplex algorithm or by successive shortest paths, and Ahuja, Magnanti and Orlin's Network Flows remains the standard treatment. Solved on the running network, the cheapest way to deliver all 130 units costs 1,020, an average of 7.85 per unit: The optimal plan. Notice how little of it is obvious: C2 is served from two different centres, and C3 is split 35 and 5 across D2 and D3. Three features of the solution are worth reading carefully, because they are the features that surprise people. Two lanes carry nothing. S2 → P1 and P1 → D2 are perfectly usable and are never worth using at these prices. A network diagram cannot tell you this; only the optimisation can. It is also the answer to "why do we pay to maintain that lane", which is a question worth asking annually. Demand gets split. C2 is served 25 units from D1 and 10 from D2, and C3 is served 35 from D2 and 5 from D3. Single sourcing each customer from its nearest centre is a rule of thumb, not an optimum, and here it would cost more. Real models often add a constraint forbidding splits, and the price of that constraint should be measured rather than assumed. The answer came out in whole units. That is not luck. The constraint matrix of a network flow problem is totally unimodular, so when supplies and demands are integers, the linear program has an integer optimal solution automatically. This is why flow problems are solved as linear programs and still give shippable answers, and it is exactly what fails the moment you add a binary "open or closed" decision, which is the subject of the next section. 7. Which warehouses should exist at all? Everything so far took the network as given. The strategic question is which facilities should exist, and it changes the mathematics completely: opening a site costs a fixed amount whether it ships one unit or a thousand, and a fixed cost cannot be expressed as a cost per unit on an arc. Give the three distribution centres a weekly fixed cost of 250, 300 and 200, capacities of 55, 75 and 65 units, and a cost per unit for serving each customer region. Then the question is which subset to open, and for each candidate subset the serving cost is itself a transportation problem. With three sites there are seven subsets and we can simply solve all of them. Four of the seven options cannot even cover demand. Among the three that can, the one with the best transport cost is the worst overall. The winner is {D2, D3} at 855: 500 of fixed cost and 355 of transport. The result that matters pedagogically is the last row. Opening all three centres produces the lowest transport cost of any configuration, 305, because every customer can then be served from its cheapest source. It is still 200 worse in total, because the third site's fixed cost of 250 buys only 50 of transport savings. Optimising the flow inside a network you have already over-built is a good way to be efficiently wrong. This is the capacitated facility location problem, and unlike everything in sections 4 to 6 it is NP-hard. With three candidate sites, brute force over eight subsets is instant. With three hundred, it is not, and the field solves it with mixed integer programming: Balinski gave the standard formulation in 1965, Geoffrion and Graves solved a real multi-commodity distribution design with Benders decomposition in 1974, and modern solvers handle industrial instances routinely. The structure that makes it tractable in practice is exactly the one visible here: for any fixed set of open sites, the remaining problem is a network flow that solves in polynomial time. 8. Designing the physical network: spanning trees A different design question is not "where should facilities be" but "which connections should we build". Laying a private line, contracting a dedicated shuttle, or building a rail spur has a cost per link, and the requirement is that every facility can reach every other. That is the minimum spanning tree problem, solved by Kruskal's algorithm in O(m log n). On six facilities, two plants, three centres and a shared cross-dock hub, with eleven possible links priced between 3 and 10, the cheapest connected design costs 21 and uses five links: P2-H at 3, P1-D1 at 4, D2-H at 4, P2-D3 at 5 and D1-H at 5. Five links for six facilities is not a coincidence. A tree on n vertices always has exactly n - 1 edges, and that is the defining trade-off of the whole approach: a spanning tree is the cheapest way to connect everything, and it is also the most fragile way. Every one of those five links is a bridge, meaning its loss disconnects the network, and three of the six facilities are cut vertices. Section 11 puts numbers on what that costs. The practical lesson is that minimum spanning tree is the right algorithm for the wrong objective in most supply chain settings. What you usually want is the cheapest network that survives the loss of any single link, which is two-edge-connected network design, and that problem is NP-hard. The MST is still worth computing, because it is a lower bound: no connected design can cost less, so it tells you the price of the redundancy you are about to buy. 9. The last mile: routing the vehicles Everything above moves units between facilities. The final leg moves them to doors, and it is where a large share of distribution cost is incurred and where the mathematics gets hard. Give one vehicle a set of stops and ask for the shortest tour that visits each exactly once and returns to the depot, and you have the travelling salesperson problem. Give a fleet with capacities and ask which vehicle serves which stops, and you have the vehicle routing problem, introduced by Dantzig and Ramser in 1959 under the name "the truck dispatching problem" and generalised ever since into windows, mixed fleets, pickup and delivery, and driver hours. The difference from sections 4 to 6 is a difference in kind, not in degree. Shortest path, max flow and min cost flow are all polynomial: a modern solver handles a continental road network in under a second. TSP and VRP are NP-hard, and the number of possible tours through n stops is (n-1)!/2, which passes 60 quadrillion at just 20 stops. That is why practice runs on heuristics: the Clarke and Wright savings algorithm from 1964 is still a standard construction method, local search such as 2-opt and Or-opt improves the result, and metaheuristics such as large neighbourhood search drive commercial engines. Exact methods have improved enormously too, and instances with hundreds of customers are now solved to proven optimality, but the daily dispatch problem is solved by heuristics because it has to be answered in minutes. The modelling point worth carrying away: the routing layer sits on top of the flow layer. The flow model decides that D3 ships 30 units to region C4; the routing model decides the sequence of doors within C4 and which truck does it. Optimising them jointly is possible and is what integrated planning systems attempt, but the two-stage split is standard because each stage is hard for a different reason. 10. Inside the factory: materials and schedules Zoom into a single plant and the graphs do not stop. Two of them run the factory, and both are directed acyclic graphs answered by one pass in topological order. The first is the bill of materials. A product is made of components, each of which is made of components, and the arcs carry quantities. Exploding a customer order into raw material requirements means walking that graph from the top down, multiplying as you go. This is what material requirements planning does, formalised by Orlicky in 1975 and still the core loop of every ERP system. Requirement explosion and project scheduling are the same sweep over a DAG, carrying a quantity in one case and a duration in the other. For an order of 100 units of product A, the explosion gives 200 of B, 100 of C, 600 of D, 500 of E and 400 of F. Component E is the one worth pausing on: it appears under two different parents, so its requirement is 2 × 2 through B plus 1 × 1 through C, which is 5 per unit of A. Adding up the branches independently, which is what a naive spreadsheet does, double counts or undercounts precisely these shared components. Processing the items in topological order guarantees every parent is final before a child is read, which is why the sweep is correct in one pass. The second graph is the schedule. Tasks have durations and precedence constraints, and the project length is the longest path through the resulting DAG. On the seven-task plan in the figure the makespan is 25 days, along procure, fabricate, paint, assemble, test and pack. That chain is the critical path, from Kelley and Walker's 1959 method, and its practical meaning is sharp: any delay on it delays the order one for one, while subassembly carries 7 days of slack and could be delayed a full week without moving the delivery date by an hour. Note the asymmetry that makes this valuable. Longest path is NP-hard on a general graph and linear on a DAG, so scheduling is cheap precisely because precedence constraints cannot form a cycle. If they do, the plan is infeasible, and the same algorithm detects that too. 11. Resilience: what breaks, and how badly A cost model tells you what to do when everything works. A resilience model tells you what happens when it does not, and it is the same graph asked a different question: remove a vertex, re-solve the flow, and read the shortfall. Every bar is a re-solved optimisation, not an estimate. The worst single failure is the plant that the cheapest plan leans on hardest. Losing any single facility leaves the network able to deliver between 90 and 100 of the 130 units. The worst case is P2, at 90 units, a 31% shortfall, and that result is worth reading against section 6. The cheapest plan routes 70 units through S2 → P2 and 80 units through P2 in total, because P2 sits on the cheapest lanes. Cost optimisation concentrates flow, and concentrated flow is exactly what fragility looks like. The optimum and the risk are produced by the same property of the network. Individual lanes matter too, and unequally. The worst single lane is P2 → D3, whose loss costs 35 units; P1 → D1 and D3 → C4 cost 30 each; D2 → C3 costs 20; and S1 → P1, D1 → C2 or D3 → C3 cost only 5. Ranking mitigation spending by lane volume would get this ordering wrong, because volume is what the plan chose to send, not what the network would lose. The structural view from section 8 says the same thing in a different language. In a minimum spanning tree every link is a bridge and several facilities are cut vertices, so a cost-minimal physical network has, by construction, no redundancy at all. Redundancy is the cycles that the spanning tree removed. Buying resilience means deliberately buying edges that a cost model would reject. Two research threads are worth naming here. Sheffi's The Resilient Enterprise (2005) made the case that flexibility is a strategic asset rather than waste. And Simchi-Levi and colleagues, working with Ford, formalised the idea that risk should be measured by time to recover and the resulting profit impact rather than by the probability of a disruption, which is unknowable: their 2015 study found that the parts posing the greatest exposure were frequently low-value components from single-source suppliers that no spend-based analysis would ever flag. That is a graph question, and it is the one this section computes. 12. Two modelling tricks worth knowing Two constructions turn "the model cannot express that" into "the model expresses that fine", and between them they cover most of what beginners hit first. Node splitting, for capacity on a facility. Flow algorithms put capacity on arcs, but a warehouse has a throughput limit of its own. The fix is to replace the vertex with two: an "in" copy that receives every incoming arc, an "out" copy that sends every outgoing arc, and a single arc between them carrying the facility's capacity. before: --> [ D2 ] --> after: --> [D2_in] --(capacity 75, cost = handling fee)--> [D2_out] --> The same trick carries a handling cost or a fixed processing delay, which is how the lead times of section 4 absorb time spent inside a building rather than on a road. It doubles the vertex count and changes nothing else, and every flow algorithm in this article works unmodified afterwards. Time expansion, for inventory. A single-period model has no memory: whatever is produced must ship immediately. Real chains hold stock, and stock is movement through time rather than space. Build one copy of the network per period and add an arc from each facility in period t to the same facility in period t+1. Flow along that arc is inventory, its cost is the holding cost, and its capacity is the storage limit. The result is called a time-expanded network, and it is exactly why multi-period production planning is solvable at all: a problem that looks like it needs a new theory turns out to be an ordinary minimum cost flow on a graph T times larger. The same construction handles shelf life, by simply not building the arc that would carry stock past its expiry. Both tricks share a moral that is worth internalising. When a supply chain feature seems to need a new algorithm, it usually needs a new graph, and the algorithm you already have then applies unchanged. 13. What is easy, what is hard The most valuable thing a planner can know about their own model is which side of the tractability line it sits on, because that decides whether the answer is an optimum or a good guess. Supply chain questionGraph problemCost Fastest route, service commitmentsShortest pathO(m + n log n) Can we deliver it all? Where is the bottleneck?Maximum flow, minimum cutPolynomial Cheapest shipping planMinimum cost flowPolynomial Material requirementsTopological order on a DAGO(n + m) Project duration, critical pathLongest path on a DAGO(n + m) Cheapest set of connecting linksMinimum spanning treeO(m log n) Which facilities to openFacility locationNP-hard Delivery routes for a fleetVehicle routingNP-hard Cheapest network surviving any single failureTwo-edge-connected designNP-hard Production batching over timeLot sizing with setupsNP-hard in general The pattern is clean and worth stating: questions about flow are easy, questions about which discrete objects to build are hard. The moment a decision becomes yes or no rather than how much, total unimodularity is lost, the linear program stops handing back integer answers, and you are in mixed integer programming. Hard does not mean hopeless. Facility location instances with hundreds of candidate sites are solved to proven optimality every day, and vehicle routing heuristics land within a few per cent of the best known solutions on instances far beyond exact methods. What the line changes is the promise: on the top half of the table you can say "this is optimal", and on the bottom half the honest sentence is "this is the best we found, and here is the bound". A fuller treatment of the costs themselves is in graph algorithms and complexity. 14. From model to practice The gap between a correct model and a useful one is mostly not mathematics. Four things decide whether the work lands. The data is the project. Lane costs, capacities and transit times live in transport management systems, contracts and spreadsheets, and they disagree with each other. A model built on a lane cost table that is eighteen months old will produce a confident, precise, wrong answer, and the failure will be blamed on the optimisation. Budget most of the effort here. Choose granularity on purpose. A strategic network study can treat a whole region as one customer vertex; a weekly dispatch model cannot. Aggregating demand is legitimate and aggregating capacity usually is not, because averages hide exactly the peaks that create the bottleneck of section 5. Use a real solver. For flows, NetworkX and SciPy both ship minimum cost flow, and Google OR-Tools covers flows, routing and scheduling with an interface built for practitioners. For anything with binary decisions, a mixed integer programming solver such as Gurobi, CPLEX or the open-source HiGHS and CBC is the right tool. Writing your own network simplex is a good way to learn and a poor way to ship. Model the thing that actually varies. A deterministic model answers "what is best if next week is exactly this". Demand is not exactly anything, and the classic failure mode here is not an algorithmic one: Forrester described in 1958 how ordering policies amplify variability upstream, and Lee, Padmanabhan and Whang named it the bullwhip effect in 1997. No amount of optimising a single week's flow addresses it. The standard responses are scenario analysis, stochastic or robust optimisation, and a rolling horizon that re-solves as reality arrives. One last habit, which the numbers in this article are meant to demonstrate: re-solve rather than reason. The claim that a lane is critical, that a site earns its fixed cost, or that a capacity investment pays back is a claim the model can settle in milliseconds, and intuition about networks is unreliable in exactly the cases that matter. Section 5 found a lane where ten extra units of capacity buys one unit of throughput. Nobody guesses that. 15. Modelling mistakes that produce confident wrong answers A network model rarely fails loudly. It returns a plan, the plan looks reasonable, and the error is only visible to someone who knows where to look. These are the ones that recur. - Putting a facility's capacity on one of its arcs. A warehouse that can handle 75 units per week is not the same as a lane that can carry 75, and squeezing the limit onto whichever arc looks busiest quietly permits more or less throughput than reality allows. Split the node, as in section 12. - Using one number as the arc weight. Cost, time and capacity answer different questions, and a model that carries only one of them will confidently return the cheapest plan when it was asked for the fastest. On this network those two answers genuinely differ. - Answering a multi-period question with a single-period model. Without inventory arcs, everything produced must ship immediately, so the model will either declare a feasible plan infeasible or invent capacity it does not have. Time-expand the network instead. - Aggregating capacity along with demand. Averaging four weeks of demand into one is often defensible; averaging capacity is not, because the average hides exactly the peak that creates the bottleneck. The 40% peak in section 5 disappears entirely under monthly averaging. - Optimising cost with no service constraint. A pure cost objective will happily route everything along the slowest cheap lanes. Lead time has to enter as a constraint or a penalty, or the optimum will be one nobody can operate. - Ranking risk by volume. The busiest lane is the one the plan chose, which is not the same as the one whose loss hurts most. Re-solve without each candidate and rank by the shortfall, which is what section 11 does and what produces a different ordering. - Pricing a truckload as a cost per unit. Freight is frequently a step function: the second pallet on a half-empty truck is nearly free, the first pallet on a new truck is not. A linear cost per unit smooths that away and systematically under-values consolidation. Step costs need binary variables, which moves the model into mixed integer programming. - Forgetting that the arc list is a modelling choice. The optimiser can only choose lanes that exist in the data. A lane nobody entered is a lane that will never appear in the answer, and "the model says we should not use that route" is often just "nobody told the model the route exists". The common thread is that all eight produce plausible output. The defence is to test the model against a period you already lived through: if it cannot reproduce last quarter's actual flows to within a sensible tolerance, it is not ready to recommend next quarter's. 16. Frequently asked questions How is graph theory used in supply chain management? + Facilities become vertices and shipping lanes become directed arcs, and then the standard questions become standard algorithms: shortest path for lead times and service levels, maximum flow for throughput and bottlenecks, minimum cost flow for the cheapest shipping plan, minimum spanning tree for network design, topological order for bills of materials and production schedules, and facility location and vehicle routing for the strategic and last-mile decisions. Network optimisation is not an analogy for supply chain planning; it is the mathematics the field is built on. What is the difference between maximum flow and minimum cost flow? + Maximum flow asks how much can physically get through and ignores money entirely; it answers "can we serve peak demand, and if not, where is the wall". Minimum cost flow asks for the cheapest way to move a required quantity and ignores anything that is not priced; it answers "given that we can serve demand, what should we actually ship on each lane". In practice you run maximum flow first to check feasibility and find the bottleneck, then minimum cost flow to produce the plan. Why is the minimum cut so useful in practice? + Because it converts a vague statement into a list. The max-flow min-cut theorem says the maximum throughput equals the capacity of the smallest set of arcs whose removal separates supply from demand, so the cut is a precise answer to "which lanes are the constraint". Capacity added anywhere else buys nothing. On the network in this article, ten extra units on either of two named lanes buys ten units of throughput, on a third lane buys one, and on the remaining eleven lanes buys exactly zero. Is the cheapest network also the best network? + Almost never, and graph theory explains why crisply. The cheapest way to connect a set of facilities is a spanning tree, and a spanning tree has no cycles, which means no alternative routes: every link is a bridge whose loss disconnects the network. Redundancy is precisely the set of cycles that a cost-minimising design deletes. The same effect appears in the flow plan, where concentrating volume on the cheapest lanes is what makes a single failure expensive. Cost and resilience are competing objectives and should be priced against each other rather than assumed compatible. Which supply chain problems are NP-hard? + The ones that decide which discrete objects exist. Facility location, vehicle routing, lot sizing with setup costs, and designing a network that survives any single failure are all NP-hard. Everything about flow through a fixed network is polynomial: shortest path, maximum flow, minimum cost flow, spanning trees, topological ordering and critical paths. The dividing line is the moment a decision becomes yes or no rather than how much, because that is when the linear programming relaxation stops returning integer answers by itself. What software solves these models? + For pure network flows, NetworkX and SciPy both ship minimum cost flow solvers, and Google OR-Tools covers flows, routing and scheduling with a practitioner-oriented interface. For anything with binary decisions, such as opening facilities or assigning trucks, use a mixed integer programming solver: Gurobi and CPLEX commercially, HiGHS and CBC in open source, usually through a modelling layer such as Pyomo, PuLP or JuMP. Writing your own network simplex is an excellent way to understand the algorithm and a poor way to ship a planning system. How do I model inventory held between periods? + With a time-expanded network. Make one copy of the whole network for each period and add an arc from each facility in period t to the same facility in period t plus one. Flow along that arc is inventory carried forward, its cost is the holding cost and its capacity is the storage limit. The multi-period problem then becomes an ordinary minimum cost flow on a graph that is T times larger, solvable with exactly the same algorithm. The same construction models shelf life by simply not building the arc that would carry stock past its expiry date. 17. References The foundational papers and the standard texts, in chronological order. - Hitchcock, F. L. (1941). “The distribution of a product from several sources to numerous localities.” Journal of Mathematics and Physics, 20(1–4), 224–230. - Koopmans, T. C. (1949). “Optimum utilization of the transportation system.” Econometrica, 17 (Supplement), 136–146. - Dantzig, G. B. (1951). “Application of the simplex method to a transportation problem.” In T. C. Koopmans (ed.), Activity Analysis of Production and Allocation, 359–373. New York: Wiley. - Ford, L. R. and Fulkerson, D. R. (1956). “Maximal flow through a network.” Canadian Journal of Mathematics, 8, 399–404. - Forrester, J. W. (1958). “Industrial dynamics: a major breakthrough for decision makers.” Harvard Business Review, 36(4), 37–66. - Dantzig, G. B. and Ramser, J. H. (1959). “The truck dispatching problem.” Management Science, 6(1), 80–91. - Kelley, J. E. and Walker, M. R. (1959). “Critical-path planning and scheduling.” Proceedings of the Eastern Joint Computer Conference, 160–173. - Ford, L. R. and Fulkerson, D. R. (1962). Flows in Networks. Princeton: Princeton University Press. - Clarke, G. and Wright, J. W. (1964). “Scheduling of vehicles from a central depot to a number of delivery points.” Operations Research, 12(4), 568–581. - Balinski, M. L. (1965). “Integer programming: methods, uses, computation.” Management Science, 12(3), 253–313. - Geoffrion, A. M. and Graves, G. W. (1974). “Multicommodity distribution system design by Benders decomposition.” Management Science, 20(5), 822–844. - Orlicky, J. (1975). Material Requirements Planning. New York: McGraw-Hill. - Ahuja, R. K., Magnanti, T. L. and Orlin, J. B. (1993). Network Flows: Theory, Algorithms, and Applications. Englewood Cliffs: Prentice Hall. - Lee, H. L., Padmanabhan, V. and Whang, S. (1997). “Information distortion in a supply chain: the bullwhip effect.” Management Science, 43(4), 546–558. - Sheffi, Y. (2005). The Resilient Enterprise: Overcoming Vulnerability for Competitive Advantage. Cambridge, Massachusetts: MIT Press. - Toth, P. and Vigo, D. (eds.) (2014). Vehicle Routing: Problems, Methods, and Applications, 2nd edition. Philadelphia: SIAM. - Simchi-Levi, D., Schmidt, W., Wei, Y., Zhang, P. Y., Combs, K., Ge, Y., Gusikhin, O., Sanders, M. and Zhang, D. (2015). “Identifying risks and mitigating disruptions in the automotive supply chain.” Interfaces, 45(5), 375–390. - Chopra, S. and Meindl, P. (2015). Supply Chain Management: Strategy, Planning, and Operation, 6th edition. Boston: Pearson. --- ## Graph Theory in Cybersecurity Source: https://learngraphtheory.org/articles/graph-theory-cybersecurity.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Why security questions are graph questions - 2. The attack graph: nodes, arcs and weights - 3. The network used throughout this article - 4. The easiest way in: shortest attack paths - 5. Sixteen ways in, and which host carries them - 6. Breaking every route: the minimum cut - 7. What one control actually buys - 8. Blast radius: what one compromise reaches - 9. How fast it spreads: the epidemic threshold - 10. Active Directory: the graph attackers already use - 11. Detection: provenance graphs and guilt by association - 12. The software supply chain graph - 13. What is easy, what is hard - 14. Modelling mistakes - 15. From model to practice - 16. Where to go next - 17. Frequently asked questions - 18. References 1. Why security questions are graph questions A vulnerability scanner produces a list. It tells you that this host runs an outdated library, that one exposes an administrative interface, a third has a weak service account. Every item gets a severity, the list is sorted, and the top of it gets fixed. Attackers do not read that list the way defenders do. An intrusion is a sequence: a foothold somewhere unimportant, a credential harvested there, a service that trusts that credential, a share that trusts that service, and eventually something that matters. Each step may be individually unremarkable. The combination is the breach. That difference is exactly the difference between a set and a graph. A list of weaknesses has no structure; a set of weaknesses plus the transitions between them is a directed graph, and once it exists the questions defenders care about become standard algorithms. Which route is easiest? Shortest path. Which controls break every route? Minimum cut. What does one compromise reach? Reachability. When does malware stop dying out on its own? The largest eigenvalue of the adjacency matrix. This is not a new idea in the literature. Phillips and Swiler proposed graph-based vulnerability analysis in 1998, Sheyner and colleagues automated attack graph generation with model checking in 2002, and the approach has been standard in research ever since. What is new is that the tooling finally caught up: modern estates are large enough that nobody can hold the paths in their head, and small enough graphs can be solved in milliseconds. Every number in this article was computed by solving the model, not estimated. If the graph vocabulary is unfamiliar, the introduction to graph theory covers the definitions used here. 2. The attack graph: nodes, arcs and weights Three modelling decisions carry the weight, and each one has an honest trade-off. What is a node? The simplest useful choice is a host, which is what this article uses. Research models are often finer: a node is a state, a pair of a machine and a privilege level, so that "user on web01" and "root on web01" are different vertices. That is more faithful and much larger, since the state space multiplies. Coarser models exist too, where a node is a whole subnet. Pick the granularity at which your controls operate, because the model exists to compare controls. What is an arc? A transition the attacker can make: an exploitable service, a trust relationship, a reused credential, a share mount, a phishing target. Arcs are directed, because compromise flows one way. A workstation that mounts a file share gives you an arc into the share, not out of it, and getting that direction wrong inverts every result. What goes on the arc? At least one number, and the choice determines what "shortest" means: - Attacker effort, a difficulty score. Shortest path then means the easiest intrusion. This article uses effort. - Success probability. Multiply along the path instead of adding, or add the negative logarithms and use the same shortest path algorithm unchanged. - Control cost. The price of the mitigation that removes that arc. The minimum cut over these numbers is the cheapest way to break every route, which is section 6. Where do the numbers come from? Usually a scoring system such as CVSS exploitability, calibrated by someone who knows the estate. They are estimates, and the honest position is that the ranking is far more robust than the absolute values. If you cannot defend a score of 3 against a score of 4, you can still defend that a public web exploit is easier than stealing a domain admin credential, and that ordering is what drives the results below. Ten hosts, sixteen transitions. Every question in this article is a question about this object, and nothing else. 3. The network used throughout this article The running example is a small enterprise, deliberately ordinary. The internet reaches three exposed systems: a public web server, a mail gateway and a VPN concentrator. Behind them sit two workstations and an application server, then a file server and a database, and finally the domain controller, which is what the attacker wants. ZoneHostsWhy it is in the model Perimeterweb01, mail01, vpnThe three ways in from the internet Users and applicationws01, ws02, app01Where footholds land and credentials live Datafile01, db01The assets, and the trust they carry Identitydc01The objective: domain compromise Each of the sixteen arcs carries an effort score and a control cost. The effort scores say that exploiting the public web application costs 3, that an internal service call from the DMZ into the app tier costs 2, and that stealing cached domain admin credentials off a workstation costs 8, which is hard but not impossible. Those relative judgements are the model's only real input. One structural note before any algorithm runs: this graph has no cycles, because every arc moves the attacker inward. Real attack graphs do have cycles, since an attacker can pivot back and forth, and every algorithm used below handles them. The acyclic case simply makes the worked examples easier to check by hand. 4. The easiest way in: shortest attack paths The first question is the one a penetration tester answers by hand over a fortnight: what is the easiest route from the internet to the domain controller? With effort on the arcs it is a shortest path problem, and Dijkstra's algorithm answers it for every asset at once. The cheapest intrusion costs 12 and runs through the servers, not the people. The workstation routes, which get most of the attention, are more expensive. The answer is internet → web01 → app01 → db01 → dc01 at a total effort of 12. Read the steps: exploit the public web application (3), use the trusted internal service call into the app tier (2), reach the database the application is entitled to query (3), and abuse the database service account against the domain controller (4). Two things about that result matter more than the number. Not one of those four steps is individually alarming. A web application vulnerability rated 3 out of 10 does not top a risk register. Neither does a service call between two systems that are supposed to talk to each other. The route is dangerous as a composition, and no per-host severity score can express a composition. This is the fundamental argument for attack graphs, made by Phillips and Swiler in 1998 and repeated in every paper since. The easiest route avoids the humans. Phishing is the most discussed initial access vector, and here the phishing route to the domain controller costs 14, not 12. The model does not say phishing is unimportant; it says that on this estate, with these scores, the server path is cheaper. Solving for the attacker's cheapest option rather than the defender's most feared option is precisely what the algorithm is for. The same computation gives the effort to reach every other asset: web01 costs 3, app01 costs 5, the file server costs 8, the database costs 8. Those are the numbers to put in front of an audit committee that wants to know how far the perimeter really is from the crown jewels. 5. Sixteen ways in, and which host carries them The cheapest path is one answer. Blocking it is not a strategy, because the attacker simply takes the next one. The useful question is how many routes exist and which assets they run through. Enumerating every simple path from the internet to the domain controller on this graph gives 16 distinct routes, costing between 12 and 25 with a median of 19. Sixteen is a small number precisely because the example is small; a real estate with a few thousand hosts routinely has more attack paths than there are atoms worth counting, which is why enumeration is a teaching device and the metrics below are the production technique. Counting how many of those routes pass through each host produces a ranking, and the ranking is not the one a perimeter report would give you. Exposure and importance are different measurements. The internet-facing web server carries the fewest routes of any host in the estate. The file server sits on 12 of the 16 routes, three quarters of them. The public web server, the single most scrutinised machine in most organisations, sits on 3. Nothing about the file server would flag on an external scan: it is not exposed, it runs nothing exotic, and it exists to hold documents. It is critical because of where it sits in the graph, and only a graph can say so. Enumeration is also where this approach stops scaling, and it is worth seeing why. Sixteen routes come out of ten hosts and sixteen arcs. Add a second file server that both workstations can reach and the count roughly doubles; a real estate with a few thousand machines and a flat internal network has a path count with more digits than anyone will ever read. The metrics below all avoid enumerating, which is what makes them usable on a real network. This measure is a security-specific cousin of betweenness centrality, introduced by Freeman in 1977, which counts the fraction of shortest paths passing through a vertex. Betweenness over all pairs is the standard network-science measure and is computable in O(nm) by Brandes' algorithm. For defence, counting paths between the specific pair that matters, the attacker's entry and the asset you care about, is usually more actionable: it answers "if I harden one machine, how many routes does it disturb" rather than "how central is this in general". Noel and Jajodia made the same argument for sensor placement in 2008: put your detection where the attack paths concentrate, not where the assets are most valuable, because the concentration points are where you get the most coverage per sensor. 6. Breaking every route: the minimum cut Ranking hosts tells you where to look. The stronger question is what set of controls would break every route at once, and how little that could cost. Give each arc the cost of the control that removes it, then compute the minimum cut between the internet and the domain controller. The max-flow min-cut theorem guarantees that the cheapest such set is exactly the minimum cut, and the algorithm returns it in polynomial time. The same machinery is covered in network flow, max-flow and min-cut. Three controls at a total cost of 7, and every one of the sixteen routes is gone. Verified by re-enumerating the paths afterwards: zero remain. The answer is three controls totalling 7: stop the DMZ web server calling into the app tier (3), stop mail attachments executing on workstations (2), and land VPN users in a restricted segment rather than beside the workstations (2). Re-enumerating the paths after applying them returns zero. Notice where the cut falls. Every one of the three controls sits at the boundary between the perimeter and the interior, and none of them touches the domain controller, the database or the file server. The instinct to harden the crown jewels first is not what the mathematics recommends: the cheapest complete fix is at the narrowest point of the graph, and here that is the first hop inward. The same question about machines rather than links uses the node splitting trick. Replace each host with an "in" copy and an "out" copy joined by an arc of capacity 1, give the real arcs infinite capacity, and the minimum cut now counts hosts instead of links. The answer here is 3 hosts: web01, mail01 and vpn, which are exactly the three that face the internet. That is a reassuring sanity check on a small example and a genuinely useful computation on a large one, where the equivalent set is rarely obvious. A caution about what is and is not polynomial. Finding the cheapest set of arcs or hosts to cut is a minimum cut, and it is fast. Finding the cheapest set of security measures is not the same problem: one patch may remove several arcs at once and one arc may need several measures, which turns it into a hitting set problem. Jha, Sheyner and Wing proved in 2002 that finding a minimum critical set of measures in an attack graph is NP-hard. Model the controls carefully, and know which of the two problems you are solving. 7. What one control actually buys Budgets rarely fund three controls at once, so the practical question is which single one to buy first. Removing each arc in turn and re-solving gives an answer, and the answer is sobering. ControlCostAttacker effortRoutes remaining Nothing (baseline)01216 Block db01 → dc015147 Block web01 → app0131413 Block internet → web0141413 Block app01 → db0151413 Block mail01 → ws012128 Block ws01 → file0131213 The best single control raises the attacker's effort from 12 to 14. That is all. No individual measure on this network buys more than two points of difficulty, because the graph is richly connected and the attacker simply switches to the next-cheapest route. This is the quantitative version of a familiar security truth: defence in depth is not a slogan, it is a consequence of the fact that single cuts in a dense graph do very little. The table also shows that the metric you choose changes the ranking. Blocking the mail gateway to workstation transition halves the number of routes, from 16 to 8, and leaves the attacker's easiest path completely untouched at 12. If your board reports "attack paths eliminated" you would call that control a success; if it reports "attacker effort" you would call it useless. Both numbers are real, they measure different things, and quoting only one is how security programmes end up optimising the wrong quantity. The best value on this table is blocking the database's route to the domain controller: cost 5, effort up to 14, and routes down to 7. It is the only control that improves both metrics substantially, which no amount of intuition would have identified. 8. Blast radius: what one compromise reaches Attack paths ask how an intruder gets in. The complementary question is what happens once they are in somewhere, and it is a plain reachability computation: from a compromised host, which assets can eventually be reached? One traversal per host answers it in linear time. Nine of the ten hosts can eventually reach the domain controller. The mail gateway alone reaches six of the nine other machines. The ranking inverts the exposure ranking. The public web server, the most exposed machine in the estate, reaches 4 assets. The mail gateway reaches 6. A workstation reaches 5. Exposure measures who can reach you; blast radius measures who you can reach, and the two produce different priority lists from the same graph. The number that should stop a meeting is this one: nine of the ten hosts can eventually reach the domain controller. Only the domain controller itself cannot, because nothing sits beyond it. On a real estate that figure is the single most useful output of the whole exercise, because it converts "we have flat networking" from an opinion into a measurement. Blast radius is also what makes containment decisions tractable during an incident. When a host is confirmed compromised, the set of machines that need investigation is its forward reachable set, and the set that could have infected it is its backward reachable set, computed on the reversed graph. Both are one traversal, and both are far more precise than isolating a whole subnet on instinct. 9. How fast it spreads: the epidemic threshold Ransomware and worms do not follow a single path; they spread. Modelling that needs a different question: given a network and an infection that propagates between neighbours and gets cleaned up at some rate, does it die out or take the estate? The answer is one of the most useful results in network science, and it is exact. For a very broad class of spreading models, the tipping point depends on a single number: the largest eigenvalue of the adjacency matrix, written λ₁. An infection whose spread-to-cleanup ratio is below 1 / λ₁ dies out on its own; above it, it becomes endemic. Wang, Chakrabarti, Wang and Faloutsos proved this in 2003, and Chakrabarti and colleagues generalised it in 2008. The threshold is not a metaphor. Two simulations, one either side of it, over 600 runs each: one settles at 5.5 infected hosts, the other is extinct in 21 steps. On the lateral movement graph here, nine hosts and thirteen links, λ₁ is 3.573, so the threshold is 0.280. Simulating an infection at 40% of that ratio, averaged over 600 runs, it is extinct by step 21. At four times the ratio it stabilises at 5.5 of the 9 hosts and stays there indefinitely. The threshold predicted both outcomes before either simulation ran. What makes this operationally interesting is that λ₁ is something you can change. Isolating the file server from the workstations and the application tier removes three links and drops λ₁ from 3.573 to 2.570, raising the threshold from 0.280 to 0.389. That is a 39% larger margin: infections that would previously have taken hold now die out. Two facts make the eigenvalue easier to reason about than it first looks. It always lies between the average degree and the maximum degree of the graph, which here means between 2.889 and 5, and 3.573 duly sits between them. And it is dominated by the densest part of the network, so the fastest way to lower it is to reduce the connectivity of the most connected host. That is exactly what the segmentation above does: the file server has degree 5, the highest in the estate, and cutting three of its links takes it to 2 and drops the maximum degree of the whole graph from 5 to 3. The most connected machine is the one to isolate, and degree is a one-line calculation you can run before touching any eigenvalue code. This reframes segmentation. "Segment the network" is usually justified with a story; here it is an intervention on a computable quantity, with a before and an after. The lineage of the idea runs back to Kephart and White, who built directed-graph epidemiological models of computer viruses for IEEE Security and Privacy in 1991, and to Staniford, Paxson and Weaver, whose 2002 analysis of worm propagation showed how quickly the curve moves when the graph is dense. 10. Active Directory: the graph attackers already use Everything so far has been a model built by a defender. The most consequential graph in enterprise security is one that already exists, that nobody deliberately designed, and that attackers have been querying for years: Active Directory. An AD environment is a graph whether or not anyone draws it. Users are vertices, groups are vertices, computers are vertices, and the arcs are the relationships the directory already stores: member of, admin to, can reset password of, has a session on, owns, has generic write over. Every one of those relationships is a transition an attacker can use. In 2016 Robbins, Vazarkar and Schroeder released BloodHound and gave the talk that named the technique, "Six Degrees of Domain Admin". Its insight is exactly the insight of this article: the individually harmless facts compose. A help desk group that can reset passwords for a group that contains a user who happens to have an active session on a server where a domain admin logged in last Tuesday is a four-hop path to total compromise, and no individual link in it looks like a misconfiguration. What BloodHound does is collect those relationships and run a shortest path query against them. It is Dijkstra, on a graph nobody had thought to draw. The result changed defensive practice, because the paths it surfaced were both real and invisible to every other tool in use. Three lessons transfer to any environment: - The data already exists. Directory relationships, cloud IAM policies, Kubernetes role bindings and SaaS sharing permissions are all graphs sitting in a database, waiting to be queried as one. - Sessions are arcs too. A logged-in session creates a temporary edge from that machine to that identity's privileges, which is why "who is logged in where" is a security question and not an inventory question. - Cleanup is a graph edit. Removing one nested group membership can delete thousands of paths, and there is no way to see that without the graph. 11. Detection: provenance graphs and guilt by association Attack graphs are about prevention. Two more graph techniques run on the detection side, and they use different graphs entirely. Provenance graphs record what actually happened on a system: processes, files, sockets and the causal relations between them. A process reads a file, writes another, spawns a child, opens a connection. King and Chen introduced backtracking with their BackTracker system at SOSP in 2003: given a detection point, such as a suspicious file, walk backwards through the causal graph to find how it got there. Forward traversal from an entry point tells you the damage; backward traversal from a symptom tells you the root cause. Both are graph traversals over the same recorded structure. The modern version correlates those flows against known attacker behaviour. HOLMES, published at IEEE Security and Privacy in 2019, maps suspicious information flows in a provenance graph onto the tactics and techniques of an attack lifecycle and raises an alert when the pattern of flows resembles an intrusion rather than ordinary activity. The engineering difficulty is scale: provenance graphs grow by millions of edges an hour on a single busy host, which makes efficient reduction and querying the entire research problem. Guilt by association is the second technique, and it is graph inference rather than traversal. Build a bipartite graph of machines and files: a machine is connected to every file it has seen. Most files and machines are unlabelled, but a few are known good and a few are known bad. Belief propagation then spreads those labels along the edges, on the assumption that files appearing on many infected machines are suspicious and machines holding many bad files are compromised. This is what Chau, Nachenberg, Wilhelm, Wright and Faloutsos built as Polonium in 2011, running over a graph of roughly 60 billion machine-file edges from Symantec telemetry, and reporting around 85% true positive detection. The technique matters because it needs no signature and no sandbox: a file nobody has ever analysed can be judged by the company it keeps. The same shape of computation, a bipartite graph plus label propagation, drives fraud detection in payments and abuse detection on social platforms. 12. The software supply chain graph The last graph is the one your build system walks. A modern application declares a handful of direct dependencies, each of which declares its own, and the transitive closure routinely runs to hundreds or thousands of packages. That closure is a directed acyclic graph, and it is an attack surface. The security question is a reachability question. If a package deep in the graph is compromised, which of your applications execute its code? That is forward reachability from the compromised node in the reverse dependency graph, and it is the query every organisation scrambles to answer in the first hour of a supply chain incident. Teams that maintain a software bill of materials answer it in seconds; teams that do not spend days grepping. Two properties of the graph make this dangerous in ways a list would not reveal. Depth hides risk: a package you never chose, three levels below one you did, runs with the same privileges as your own code. Popularity concentrates it: the packages with the highest in-degree are the highest-value targets, because compromising one reaches thousands of downstream projects at once, which is precisely the pattern documented in the review of real open source supply chain attacks by Ohm, Plate, Sykosch and Meier in 2020. The useful defensive metrics are graph metrics. Count the transitive closure size, not the direct dependency count. Rank dependencies by how many of your applications reach them. Watch for packages with a single maintainer and a large in-degree, which is exactly the risk profile that has produced several of the best known incidents. The technique is identical to the blast radius computation in section 8, run over a different graph. 13. What is easy, what is hard Attack graph analysis is unusual among security techniques in having a clean complexity story, and knowing which side of the line a question falls on saves a great deal of wasted effort. Security questionGraph problemCost Easiest route to an assetShortest pathO(m + n log n) What can this compromise reach?ReachabilityO(n + m) Cheapest set of links to cutMinimum cutPolynomial Smallest set of hosts to isolateMinimum vertex cutPolynomial Where does spread tip over?Largest eigenvaluePolynomial Which hosts are choke points?Betweenness centralityO(nm) Enumerate every attack pathAll simple pathsExponential in the worst case Minimum set of security measuresHitting set on the attack graphNP-hard Cheapest hardening under a budgetNetwork interdictionNP-hard The pattern is the familiar one: questions about flow and connectivity are cheap, questions about which discrete things to change are expensive. Enumeration is the trap in the middle. It is intuitive, it is what every demonstration does, and the number of simple paths can grow exponentially with the size of the network, which is why serious tools compute metrics over the graph rather than listing its paths. Ammann, Wijesekera and Kaushik made exactly this argument in 2002 when they proposed a compact, monotonic representation that scales polynomially instead of enumerating. One metric worth knowing by name is k-zero day safety, proposed by Wang, Jajodia, Singhal, Cheng and Noel in 2014. It asks how many distinct unknown vulnerabilities an attacker would need to reach an asset, which sidesteps the unanswerable question of how likely each individual exploit is. It is a graph distance under a different weighting, and it is a good example of the field's better instinct: measure structure, not probability. 14. Modelling mistakes An attack graph that is wrong is worse than none, because it produces confident, specific, incorrect priorities. These are the failures that recur. - Getting the arc direction wrong. A workstation mounting a share creates an arc into the share. Reversing it makes the file server look safe and the workstation look critical, and nothing about the output looks broken. - Modelling hosts when the risk is identities. If the real transition is a credential that works in three places, an arc between two machines does not capture it. This is why the Active Directory graph in section 10 is a separate model and not a refinement of this one. - Treating scores as measurements. The effort numbers are judgements. Trust the ordering, distrust the third decimal place, and test whether the conclusion survives perturbing the scores. If the recommended control changes when a 3 becomes a 4, say so. - Forgetting that the graph is a snapshot. A laptop that joins the VPN adds arcs; a decommissioned server removes them; a temporary firewall exception during a migration can open a route that no diagram ever recorded. An attack graph is only as current as the inventory behind it. - Enumerating paths on a real estate. It works beautifully on ten hosts and never finishes on ten thousand. Compute cuts, centrality and reachability instead, all of which are polynomial. - Reporting a single metric. Section 7 showed a control that halves the number of attack paths and does not slow the attacker down at all. Publish attacker effort and route count together, or the programme will optimise whichever one is on the slide. - Ignoring the arcs you cannot remove. Some transitions are the business: the application must query the database. The model should mark those as fixed, so the optimiser stops proposing controls that will never be approved. 15. From model to practice Four things separate a diagram that impresses a meeting from a model that changes decisions. Build the graph from data you already have. Firewall rule sets, cloud security group definitions, vulnerability scan output, Active Directory relationships and EDR telemetry all describe edges. A model assembled by hand in a workshop is out of date the week after the workshop; a model generated from configuration is regenerated nightly. Start with reachability, not attack paths. The cheapest valuable result is the blast radius table from section 8, because it needs no exploit scoring at all, only connectivity. "Nine of our ten hosts can reach the domain controller" is a finding that lands, and you can produce it before anyone argues about CVSS. Use the tools that exist. MulVAL, the scalable attack graph generator published by Ou, Boyer and McQueen in 2006, is still the reference implementation in research. BloodHound covers the identity graph. NetworkX or a graph database handles the analysis once the edges exist. None of the algorithms in this article needs to be written from scratch, and the graph chapters of any algorithms text cover the ones that do. Re-solve rather than argue. Every claim in this article was a claim the model settled in milliseconds: that the easiest route avoids the workstations, that the file server carries four times the routes of the web server, that the best single control buys two points of effort, that segmentation raises the epidemic threshold by 39%. Intuition about networks is unreliable in exactly the cases that matter, and the whole value of building the graph is that you stop needing it. 16. Where to go next The fastest way to internalise this material is to build a graph rather than read about one, and the barrier is lower than it looks. Ten hosts and sixteen arcs, which is all this article used, fit in a text file, and every result above came out of a few dozen lines of ordinary code. A sensible order to learn the pieces: get comfortable with breadth first and depth first search, since reachability and blast radius are nothing more than a traversal with bookkeeping. Then shortest path algorithms, which give you the easiest-route analysis and, with negative logarithms on the arcs, the most-likely-route analysis too. Then max-flow and min-cut, which is the whole of sections 6 and 7 and the single most under-used result in defensive security. After that the useful direction is structural rather than algorithmic: directed versus undirected graphs settles a surprising number of modelling arguments, and graph representation decides whether your analysis runs in a second or an hour once the estate is large. The complexity boundaries in section 13 are laid out more generally in graph algorithms and complexity. If you would rather start from the security side, the shortest path to a real result is to export your Active Directory relationships and query them, because that graph already exists and nobody had to model it. The finding that follows is usually the same one this article ends with: the number of machines that can eventually reach the domain controller is far higher than anyone in the room expected. 17. Frequently asked questions What is an attack graph? + A directed graph whose vertices are the states an intruder can occupy, usually hosts or host-and-privilege pairs, and whose arcs are the transitions between them: an exploitable service, a trust relationship, a reused credential. Weights on the arcs record how much effort each step costs, how likely it is to succeed, or how much the control that removes it would cost. Once the graph exists, the questions defenders ask become standard algorithms: shortest path for the easiest intrusion, minimum cut for the cheapest complete fix, reachability for blast radius. Why is a graph better than a list of vulnerabilities? + Because breaches are compositions, and a list cannot express a composition. On the network in this article the easiest route to the domain controller is made of four individually unremarkable steps, none of which would reach the top of a severity-sorted list, and their combination is the cheapest intrusion available. A list also cannot tell you that the file server sits on three quarters of all routes while the internet-facing web server sits on under a fifth. Those are properties of the structure, not of any single host. How do I find the cheapest way to block every attack path? + Put the cost of each mitigating control on the corresponding arc and compute the minimum cut between the attacker's starting point and the asset. The max-flow min-cut theorem guarantees the cheapest set of arcs that separates them is exactly that cut, and it is computed in polynomial time. To count hosts rather than links, split every host into an in copy and an out copy joined by an arc of capacity one and give the real arcs infinite capacity; the same algorithm then returns the smallest set of machines to isolate. What is the epidemic threshold, and why does it matter for ransomware? + For a wide class of spreading models, an infection dies out on its own if its spread-to-cleanup ratio is below one divided by the largest eigenvalue of the network's adjacency matrix, and becomes endemic above it. That result is due to Wang, Chakrabarti, Wang and Faloutsos in 2003. It matters because the eigenvalue is something segmentation changes: on the network in this article, isolating the file server from the workstations and the app tier drops the eigenvalue from 3.573 to 2.570 and raises the threshold by 39%, turning outbreaks that would have taken hold into ones that fade. What is BloodHound doing, mathematically? + Running shortest path queries on a graph built from Active Directory relationships. Users, groups and computers are vertices; membership, administrative rights, password reset rights, ownership and active sessions are arcs. The tool collects those relationships and finds routes from a low-privilege account to Domain Admin. The technique is ordinary graph search; the contribution was recognising that the directory already contains the graph, and that chains of individually reasonable permissions compose into total compromise. Can attack graph analysis scale to a real network? + The analysis scales; naive enumeration does not. The number of simple attack paths can grow exponentially with the size of the network, so listing them is hopeless beyond toy examples. Everything else in this article is polynomial: shortest paths, reachability, minimum cuts, centrality and the eigenvalue are all comfortably computable on graphs with millions of edges. The standard research answer, from Ammann and colleagues in 2002 and the MulVAL generator in 2006, is to use a compact representation whose size grows polynomially and to compute metrics over it rather than enumerating paths. Where do the effort scores come from, and what if they are wrong? + Usually from a scoring system such as CVSS exploitability, adjusted by someone who knows the estate. They are judgements rather than measurements, and the honest position is that the ordering is much more reliable than the values: you may not defend 3 against 4, but you can defend that a public web exploit is easier than stealing a domain admin credential. Test the conclusion by perturbing the scores. If the recommended control changes when one score moves by a point, report that instead of pretending the model is precise. Metrics such as k-zero day safety exist precisely to sidestep the scoring problem by counting distinct unknown vulnerabilities instead. 18. References The papers that established these techniques, in chronological order. - Ford, L. R. and Fulkerson, D. R. (1956). “Maximal flow through a network.” Canadian Journal of Mathematics, 8, 399–404. - Freeman, L. C. (1977). “A set of measures of centrality based upon betweenness.” Sociometry, 40(1), 35–41. - Kephart, J. O. and White, S. R. (1991). “Directed-graph epidemiological models of computer viruses.” Proceedings of the IEEE Symposium on Security and Privacy, 343–359. - Phillips, C. and Swiler, L. P. (1998). “A graph-based system for network-vulnerability analysis.” Proceedings of the New Security Paradigms Workshop, 71–79. - Ammann, P., Wijesekera, D. and Kaushik, S. (2002). “Scalable, graph-based network vulnerability analysis.” Proceedings of the 9th ACM Conference on Computer and Communications Security, 217–224. - Sheyner, O., Haines, J., Jha, S., Lippmann, R. and Wing, J. M. (2002). “Automated generation and analysis of attack graphs.” Proceedings of the IEEE Symposium on Security and Privacy, 273–284. - Jha, S., Sheyner, O. and Wing, J. (2002). “Two formal analyses of attack graphs.” Proceedings of the 15th IEEE Computer Security Foundations Workshop, 49–63. - Staniford, S., Paxson, V. and Weaver, N. (2002). “How to own the Internet in your spare time.” Proceedings of the 11th USENIX Security Symposium, 149–167. - King, S. T. and Chen, P. M. (2003). “Backtracking intrusions.” Proceedings of the 19th ACM Symposium on Operating Systems Principles, 223–236. - Wang, Y., Chakrabarti, D., Wang, C. and Faloutsos, C. (2003). “Epidemic spreading in real networks: an eigenvalue viewpoint.” Proceedings of the 22nd International Symposium on Reliable Distributed Systems, 25–34. - Ou, X., Boyer, W. F. and McQueen, M. A. (2006). “A scalable approach to attack graph generation.” Proceedings of the 13th ACM Conference on Computer and Communications Security, 336–345. - Chakrabarti, D., Wang, Y., Wang, C., Leskovec, J. and Faloutsos, C. (2008). “Epidemic thresholds in real networks.” ACM Transactions on Information and System Security, 10(4), 1–26. - Noel, S. and Jajodia, S. (2008). “Optimal IDS sensor placement and alert prioritization using attack graphs.” Journal of Network and Systems Management, 16(3), 259–275. - Chau, D. H., Nachenberg, C., Wilhelm, J., Wright, A. and Faloutsos, C. (2011). “Polonium: tera-scale graph mining and inference for malware detection.” Proceedings of the SIAM International Conference on Data Mining, 131–142. - Wang, L., Jajodia, S., Singhal, A., Cheng, P. and Noel, S. (2014). “k-zero day safety: a network security metric for measuring the risk of unknown vulnerabilities.” IEEE Transactions on Dependable and Secure Computing, 11(1), 30–44. - Robbins, A., Vazarkar, R. and Schroeder, W. (2016). “Six degrees of Domain Admin.” DEF CON 24. - Milajerdi, S. M., Gjomemo, R., Eshete, B., Sekar, R. and Venkatakrishnan, V. N. (2019). “HOLMES: real-time APT detection through correlation of suspicious information flows.” Proceedings of the IEEE Symposium on Security and Privacy, 1137–1152. - Ohm, M., Plate, H., Sykosch, A. and Meier, M. (2020). “Backstabber's knife collection: a review of open source software supply chain attacks.” Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA), 23–43. --- ## Vertices and Edges Explained Source: https://learngraphtheory.org/articles/vertices-and-edges.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. The formal definition of a graph - 2. Vertices: what they are, and what they are not - 3. Edges, adjacency and incidence - 4. Loops and parallel edges: where the definition has to change - 5. Degree and the first theorem in graph theory - 6. Degree sequences: which lists of numbers are graphs - 7. Directed edges: arcs, in-degree and out-degree - 8. Weighted edges, and why weights sit outside the graph - 9. How many edges can a graph have? - 10. Storing vertices and edges in code - 11. Deciding what a vertex should be - 12. Terminology traps across textbooks - 13. Glossary of symbols - 14. Frequently asked questions - 15. References 1. The formal definition of a graph Almost every introduction says a graph is "dots joined by lines". That picture is useful and it is also the reason so many people get stuck later: the dots and the lines are not the mathematical object. The object is a pair of sets. Diestel's Graph Theory, the standard graduate reference, opens with the definition in its cleanest form: A graph is a pair G = (V, E) of sets such that E ⊆ [V]2, where [V]2 denotes the set of all 2-element subsets of V. Unpack that and everything else in this article follows: - V is a set of objects called vertices. Nothing is assumed about them. They can be cities, people, web pages, chemical atoms or integers. The theory never looks inside them; it only cares which ones are distinguishable. - E is a set of 2-element subsets of V. An edge is literally the set {u, v}, not an arrow or a curve. It records nothing except which pair it joins. - Because E is a set, an edge is either present or absent. It cannot appear twice. - Because each element of E has exactly two distinct members, an edge cannot join a vertex to itself. Those last two consequences are not restrictions someone added later. They fall straight out of the set theory, and a graph that obeys them is called simple. Allowing repeated edges or self-loops means changing the definition itself, which is section 4. Two more pieces of standard notation you will meet everywhere. When several graphs are in play, write V(G) and E(G) to say which graph you mean. And the two size measures have names: the number of vertices is the order of the graph, the number of edges is its size. Most algorithm texts abbreviate these to n = |V| and m = |E|, and that is the convention used throughout this site. The whole vocabulary on one picture. Vertices are elements of V, edges are 2-element subsets of V, and every other term in this article is defined from those two. This graph is the running example for the rest of the article. Its vertex set is V = {A, B, C, D, E, F}, so the order is n = 6, and its edge set is E = { {A,B}, {A,C}, {B,C}, {B,D}, {C,D}, {D,E}, {E,F} } so m = 7 2. Vertices: what they are, and what they are not A vertex carries no structure of its own. In the formal object it is an anonymous element of a set, and everything you can say about it comes from the edges that touch it. This is worth insisting on, because it is exactly what makes graph theory transferable: a theorem proved about vertices is a theorem about airports, transistors, proteins and Git commits at the same time. Three consequences that trip people up in practice: - Vertices are distinguishable but otherwise interchangeable. Two graphs that differ only in what the vertices are called are isomorphic, and graph theory treats them as the same graph. When you label vertices in code you are adding bookkeeping, not mathematics. - An isolated vertex is still a vertex. A vertex with no edges at all is perfectly legal and is called isolated. Beginners often build a graph from an edge list and silently lose every isolated vertex, which changes the order of the graph and quietly breaks anything that divides by n. - The empty graph exists, and so do infinite ones. Nothing in the definition forbids V = ∅, and nothing in it forbids V being infinite either, which is the subject of the guide on finite and infinite graphs. Most texts allow it and a few exclude it by fiat; the important thing is to know which convention your source uses before you trust an edge case. The historical note is worth a line, because the vocabulary is not stable across a century of literature. Harary's classic 1969 text calls them points and lines. Physicists and many applied papers say nodes and links. Chemists, following Sylvester's 1878 note in Nature that gave the subject the word "graph" in the first place, spoke of atoms and bonds. All four vocabularies describe the same pair of sets. 3. Edges, adjacency and incidence An edge joins exactly two vertices, and those two are its endpoints or ends. From that single relationship the two words that beginners most often swap get their precise meanings: - Adjacency is a relation between two vertices. Vertices u and v are adjacent, written u ~ v, when {u, v} ∈ E. Adjacent vertices are also called neighbours, and the set of all neighbours of v is its neighbourhood, written N(v). - Incidence is a relation between a vertex and an edge. The edge {u, v} is incident to u and incident to v, and to no other vertex. The distinction sounds pedantic until you meet a sentence like "two adjacent edges". It is legal, and it means two edges that share an endpoint. Vertices are adjacent through an edge; edges are adjacent through a vertex. Keeping the two relations apart is what lets you read a definition such as a proper colouring ("adjacent vertices get different colours") without ambiguity. In the running example, N(B) = {A, C, D}, so B has three neighbours. The edge {B, D} is incident to B and to D. The edges {B, D} and {C, D} are adjacent to each other, since both are incident to D. 4. Loops and parallel edges: where the definition has to change Real systems produce two things the set definition cannot express. A road that leaves a roundabout and returns to it is an edge from a vertex to itself. Two separate flights between the same pair of airports are two distinct edges joining the same pair. Under E ⊆ [V]2 neither exists: {A, A} is a one-element set, and a set cannot hold the same element twice. The fix is not a footnote, it is a different definition. Bondy and Murty's Graph Theory carries an explicit incidence function alongside the two sets: G = (V, E, ψ) where ψ: E → unordered pairs of (not necessarily distinct) vertices Now an edge is an object in its own right with an identity, and ψ reports which pair it joins. Two different edges may map to the same pair, which gives parallel edges, also called multi-edges. An edge may map to a pair whose two entries coincide, which gives a loop. West's Introduction to Graph Theory takes the same route, defining a graph as a vertex set, an edge set, and a relation associating each edge with its endpoints. The resulting vocabulary: TermLoops allowed?Parallel edges allowed?Definition needed Simple graphNoNoG = (V, E) with E ⊆ [V]2 MultigraphUsually noYesIncidence function ψ PseudographYesYesIncidence function ψ Two practical warnings. First, "multigraph" is not used consistently: some authors let it include loops, others do not, so check the source before quoting a theorem. Second, a loop contributes 2 to the degree of its vertex, not 1, because both of its ends attach there. That convention is not arbitrary, and the next section explains exactly why it has to be that way. Unless a text says otherwise, "graph" means "simple graph". Every result in the rest of this article assumes it, and the companion guide on simple graphs vs multigraphs works through exactly what the two allowances change and which standard bounds stop holding without them. Vertices and Edges Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Degree and the first theorem in graph theory The degree of a vertex v, written deg(v) or d(v), is the number of edges incident to it. Equivalently, and more usefully, it is the number of edge ends that meet at v. In a simple graph it equals the size of the neighbourhood, deg(v) = |N(v)|. Two related quantities appear constantly in bounds and proofs: the minimum degree δ(G) and the maximum degree Δ(G). A graph in which every vertex has the same degree k is k-regular. Now the oldest theorem in the subject, which comes from Euler's 1736 paper on the bridges of Königsberg, the paper that founded graph theory: Handshaking lemma. For every finite graph, the sum of all vertex degrees equals twice the number of edges: ∑v∈V deg(v) = 2m. The proof is one line of double counting. Count the pairs (vertex, edge end meeting it). Summing over vertices gives ∑ deg(v). Summing over edges gives 2m, since every edge has exactly two ends. Both count the same finite set, so they are equal. Notice that this argument is also what forces a loop to count twice: a loop still has two ends, and both land on the same vertex. Every edge contributes exactly 2 to the total degree, one at each end. Seven edges, total degree 14, and an even number of odd-degree vertices. Check it on the running example: the degrees are 2, 3, 3, 3, 2, 1, which sum to 14, and the graph has 7 edges. The lemma has an immediate and very useful corollary. Corollary. In any graph, the number of vertices of odd degree is even. Split the sum into odd-degree and even-degree vertices. The total is even and the even-degree part is even, so the odd-degree part must be even too, which forces the count of odd terms to be even. That is why a party can never contain exactly three people who have each shaken an odd number of hands, and it is the same parity argument that decides whether an Eulerian path exists. 6. Degree sequences: which lists of numbers are graphs Write the degrees in non-increasing order and you get the degree sequence. For the running example it is (3, 3, 3, 2, 2, 1). The natural question runs the other way: given a list of numbers, is there a graph with exactly those degrees? A list for which one exists is called graphic. The handshaking lemma already rules out half the candidates: any sequence with an odd sum is not graphic. But that test is far from sufficient. Consider (3, 3, 1, 1). The sum is 8, which is even, and no vertex asks for more than the 3 neighbours available. Yet no simple graph realises it: the two degree-3 vertices must each join all three other vertices, which forces both degree-1 vertices to have degree 2. Two classical results settle the question completely: - The Erdős–Gallai theorem (1960) gives an exact criterion: a non-increasing sequence with even sum is graphic if and only if, for every k, the sum of the first k terms is at most k(k-1) plus the sum over the remaining terms of min(di, k). It is a closed-form test, checkable in linear time after sorting. - The Havel–Hakimi algorithm, from Havel (1955) and independently Hakimi (1962), is the constructive version: remove the largest degree d, subtract 1 from the next d entries, re-sort, and repeat. The original sequence is graphic exactly when this reduction ends in all zeros, and the steps double as a recipe for building a graph that realises it. Run Havel–Hakimi on the failing example to see it work: (3, 3, 1, 1) remove the 3, subtract 1 from the next three entries (2, 0, 0) re-sorted remove the 2, subtract 1 from the next two entries (-1, -1) negative, so the sequence is NOT graphic And on the running example, where it succeeds: (3, 3, 3, 2, 2, 1) → (2, 2, 1, 2, 1) → sorted (2, 2, 2, 1, 1) (2, 2, 2, 1, 1) → (1, 1, 1, 1) → sorted (1, 1, 1, 1) (1, 1, 1, 1) → (0, 1, 1) → sorted (1, 1, 0) (1, 1, 0) → (0, 0) → all zeros, so the sequence IS graphic One warning that catches people out: a graphic sequence can be realised by several non-isomorphic graphs. Knowing every degree does not pin down the graph. 7. Directed edges: arcs, in-degree and out-degree Replace the unordered pair with an ordered pair and you get a directed graph, or digraph: D = (V, A) with A ⊆ V × V An element (u, v) of A is an arc or directed edge, running from its tail u to its head v. Because the pair is ordered, (u, v) and (v, u) are different arcs and both may be present. Bang-Jensen and Gutin's Digraphs is the standard reference for the terminology, and it keeps "arc" for the directed object precisely to avoid the ambiguity of calling both things edges. Degree splits in two: - Out-degree d+(v), the number of arcs with tail v. - In-degree d-(v), the number of arcs with head v. The handshaking lemma splits with it. Every arc has exactly one tail and exactly one head, so counting arc tails and arc heads separately gives ∑v∈V d+(v) = ∑v∈V d-(v) = |A| Note the missing factor of 2: in the undirected case each edge contributed two ends to one sum, here each arc contributes one end to each of two sums. The full comparison, including orientations, the three kinds of connectivity and which algorithms survive, is in the guide to directed vs undirected graphs. Vertices with in-degree 0 are sources and those with out-degree 0 are sinks, which is exactly the vocabulary topological sorting and network flow are built on. 8. Weighted edges, and why weights sit outside the graph Shortest path problems need distances, flow problems need capacities, and scheduling problems need durations. None of that is in G = (V, E), and it is not supposed to be. A weighted graph is a graph plus a function: w: E → ℝ assigning a real number to each edge Keeping the weights in a separate function rather than inside the edges is what lets one graph carry several cost models at once. The same road network is one graph with three weight functions: kilometres, minutes and fuel. The companion guide on weighted vs unweighted graphs follows that idea through algorithm choice, negative weights and vertex strength. Swapping the function changes every shortest path without touching a single vertex or edge. It also explains why algorithms come with conditions on w rather than on the graph. Dijkstra's algorithm requires w(e) ≥ 0 for every edge; Bellman-Ford tolerates negative weights but not negative cycles. Those are constraints on the function, and the underlying pair of sets is indifferent to them. 9. How many edges can a graph have? In a simple graph on n vertices, an edge is a choice of 2 distinct vertices from n, so the maximum is the binomial coefficient mmax = C(n, 2) = n(n - 1) / 2 The graph that achieves it, where every pair of vertices is adjacent, is the complete graph Kn. For the running example, n = 6 gives a ceiling of 15 edges, and the graph uses 7 of them. The ratio 2m / (n(n-1)) is the density, here 0.47. That single bound is the reason two phrases dominate the algorithms literature: - A dense graph has m close to its maximum, so m = Θ(n2). - A sparse graph has m far below it, typically m = O(n) or O(n log n). Almost every large real network is sparse: road maps, social graphs and web graphs all have average degree in the single or double digits regardless of how many vertices they hold. Sparsity is not a detail. It decides which data structure to use, which is the next section, and it is why a complexity of O(m log n) beats O(n2) on real inputs even though the two are identical in the worst case. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 10. Storing vertices and edges in code There are three standard ways to hold V and E in memory, and the trade-offs between them are set out in Cormen, Leiserson, Rivest and Stein's Introduction to Algorithms. The full comparison, including compressed sparse row and the density threshold at which a matrix becomes the smaller option, is in the guide to graph representation. One graph, three encodings. The adjacency matrix is symmetric because the graph is undirected, and every 1 above the diagonal is mirrored below it. RepresentationSpaceIs u adjacent to v?Visit all neighbours of vBest for Adjacency matrixΘ(n2)O(1)Θ(n)Dense graphs, constant-time edge tests Adjacency listΘ(n + m)O(deg(u))Θ(deg(v))Sparse graphs, traversal Edge listΘ(m)O(m)O(m)Algorithms that sort edges, such as Kruskal The practical rule follows from section 9. Traversals such as BFS and DFS spend their whole running time walking neighbour sets, so on a sparse graph the adjacency list gives O(n + m) where the matrix would force O(n2) just to scan rows of zeros. On a dense graph the matrix wins on both space and simplicity. An edge list looks primitive until you meet Kruskal's algorithm, which sorts all edges by weight and never needs neighbour lookups at all. 11. Deciding what a vertex should be The hardest part of applying graph theory is not any theorem. It is choosing what the vertices are, because that choice fixes everything downstream and is rarely unique. Take a road network. The obvious model makes intersections the vertices and road segments the edges, which is what a routing engine wants: a path in the graph is a route on the ground. But turn restrictions and turn costs live at intersections, not on roads, and this model has nowhere to put them. The standard fix inverts the choice: make each road segment a vertex, and join two of them when you may legally drive from one into the other. Now a turn is an edge and can carry a cost. That inversion is a formal construction, not a hack. The line graph L(G) has one vertex for every edge of G, with two of them adjacent when the corresponding edges of G share an endpoint. It goes back to Whitney's 1932 paper on congruent graphs, and it is the reason a problem about edges can often be handed to an algorithm that only knows about vertices. Matching problems, for instance, become independent-set problems on the line graph. A short checklist that catches most modelling mistakes: - Can two of your vertices be joined more than once? If yes you need a multigraph, or you need to aggregate the parallel edges into one weight. - Is the relation symmetric? "Is friends with" usually is, "follows" and "depends on" are not. Get this wrong and you will use an undirected algorithm on a directed problem. - Does the relation involve more than two things at once? An edge joins exactly two vertices. A committee of five people, or a chemical reaction with three reagents, is a hypergraph edge, and squashing it into ordinary edges loses information. - Does the structure change over time? A standard graph is a snapshot. Temporal or dynamic graphs are separate models with their own literature. 12. Terminology traps across textbooks Graph theory grew in several fields at once, so the same object has several names and a few names mean different things to different authors. These are the ones that actually cause errors: You may readIt usually meansWatch out for Node, point, siteVertexNo difference in meaning; "node" dominates in computing, "point" in Harary's older text Link, line, arc, bondEdge"Arc" specifically means a directed edge in most modern texts ValencyDegreeCommon in chemistry and in British texts GraphSimple graphA minority of applied papers let "graph" include loops and parallel edges MultigraphParallel edges allowedSome authors also allow loops here, others reserve that for "pseudograph" Order and size|V| and |E| respectivelyEasy to swap; "size" is the edge count, not the vertex count PathA walk with no repeated vertexSome texts allow repeats and say "simple path" for the restricted version CycleA closed pathIn directed settings a single arc pair u to v and v to u already forms one The safe habit is the one every paper follows: state your conventions once at the start, then hold to them. When you quote a theorem, quote the definition it was proved under. 13. Glossary of symbols The notation used across this site and in the references below. SymbolReads asMeaning G = (V, E)the graph GA vertex set together with an edge set V(G), E(G)vertex set, edge set of GUsed when more than one graph is in play n, |V|, |G|order of GThe number of vertices m, |E|size of GThe number of edges {u, v}the edge u vAn undirected edge, often abbreviated uv (u, v)the arc from u to vA directed edge, tail u and head v u ~ vu is adjacent to vThere is an edge joining them N(v)neighbourhood of vThe set of vertices adjacent to v deg(v), d(v)degree of vThe number of edge ends meeting v d+(v), d-(v)out-degree, in-degreeArc counts by tail and by head δ(G), Δ(G)delta, capital deltaMinimum and maximum degree in G Kncomplete graph on n verticesEvery pair of vertices adjacent, n(n-1)/2 edges L(G)line graph of GOne vertex per edge of G, adjacent when the edges share an end w: E → ℝweight functionAssigns a number to each edge 14. Frequently asked questions What is the difference between a vertex and a node? There is none. They are two names for the same thing, and which one you meet depends on the field. Mathematics texts say vertex, computer science and network science usually say node, Harary's classic 1969 book says point, and chemistry says atom. The plural of vertex is vertices. Pick one word and use it consistently within a document. What is the difference between an edge and an arc? In most modern texts an edge is undirected and is written as the unordered pair {u, v}, while an arc is directed and is written as the ordered pair (u, v) with a tail and a head. Because the pair is ordered, the arcs (u, v) and (v, u) are different objects and a digraph may contain both. Some authors say "directed edge" instead of arc, which means exactly the same thing. Can an edge connect a vertex to itself? Not in a simple graph. Under the standard definition an edge is a 2-element subset of the vertex set, and {v, v} has only one element, so it is not a valid edge. An edge joining a vertex to itself is called a loop, and to allow one you must move to a definition with an explicit incidence function, which is what multigraphs and pseudographs use. In such a graph a loop adds 2 to the degree of its vertex, because both of its ends attach there. What is the degree of a vertex, and what is the handshaking lemma? The degree of a vertex is the number of edge ends that meet it, written deg(v). The handshaking lemma, which goes back to Euler's 1736 paper on the bridges of Königsberg, says the degrees of all vertices sum to exactly twice the number of edges, because every edge contributes one end at each of its two endpoints. Its best known corollary is that the number of vertices of odd degree is always even. How many edges can a graph with n vertices have? A simple undirected graph on n vertices has at most n(n-1)/2 edges, since an edge is a choice of 2 distinct vertices out of n. The graph that reaches this maximum is the complete graph K_n. A simple digraph can have up to n(n-1) arcs, because each ordered pair counts separately. Multigraphs have no upper bound at all, since parallel edges may be repeated freely. Is every list of numbers a valid degree sequence? No. A list that is realisable by some simple graph is called graphic. The handshaking lemma gives a quick necessary test, since the sum must be even, but it is not sufficient: (3, 3, 1, 1) has an even sum yet no simple graph has those degrees. The Erdős–Gallai theorem of 1960 gives an exact criterion, and the Havel–Hakimi algorithm from Havel (1955) and Hakimi (1962) both decides the question and constructs a realisation when one exists. 15. References The definitions, theorems and attributions above come from these sources, listed in chronological order. - Euler, L. (1736). "Solutio problematis ad geometriam situs pertinentis." Commentarii Academiae Scientiarum Petropolitanae 8 (published 1741), 128 to 140. The Königsberg bridges paper, and the origin of the degree argument. - Sylvester, J. J. (1878). "Chemistry and Algebra." Nature 17, 284. The note that introduced the word "graph" in its modern sense. - Whitney, H. (1932). "Congruent Graphs and the Connectivity of Graphs." American Journal of Mathematics 54(1), 150 to 168. Source of the line graph construction. - König, D. (1936). Theorie der endlichen und unendlichen Graphen. Leipzig: Akademische Verlagsgesellschaft. The first book devoted entirely to graph theory. - Havel, V. (1955). "A remark on the existence of finite graphs" (in Czech). Časopis pro pěstování matematiky 80, 477 to 480. - Erdős, P. and Gallai, T. (1960). "Graphs with prescribed degrees of vertices" (in Hungarian). Matematikai Lapok 11, 264 to 274. The exact criterion for graphic sequences. - Hakimi, S. L. (1962). "On Realizability of a Set of Integers as Degrees of the Vertices of a Linear Graph. I." Journal of the Society for Industrial and Applied Mathematics 10(3), 496 to 506. - Harary, F. (1969). Graph Theory. Reading, Massachusetts: Addison-Wesley. The classic that calls vertices "points" and edges "lines". - Bollobás, B. (1998). Modern Graph Theory. Graduate Texts in Mathematics 184. New York: Springer. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. Defines a graph by a vertex set, an edge set and an endpoint relation. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. Source of the incidence function formulation used in section 4. - Bang-Jensen, J. and Gutin, G. (2009). Digraphs: Theory, Algorithms and Applications, 2nd edition. London: Springer. Standard reference for arcs and directed degrees. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. Source of the representation costs in section 10. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Source of the definition quoted in section 1. Build a graph and watch it work Drop in your own vertices and edges, then run a traversal over them step by step. Every term on this page becomes something you can point at on screen. Open the visualizer Related Articles History of Graph Theory Read more → BFS vs DFS Read more → Graph Theory Study Roadmap Read more → --- ## Directed vs Undirected Graphs Explained Source: https://learngraphtheory.org/articles/directed-vs-undirected-graphs.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. The two definitions, side by side - 2. Edges, arcs, tails and heads - 3. Degree splits in two - 4. What changes in the matrix and the list - 5. Orientations and the underlying graph - 6. Robbins' theorem: which streets can go one-way - 7. Connectivity becomes three different questions - 8. Cycles, DAGs and topological order - 9. Which algorithms transfer, and which break - 10. Choosing: is your relation symmetric? - 11. Converting between the two - 12. Common mistakes - 13. Glossary - 14. Frequently asked questions - 15. References 1. The two definitions, side by side The difference between a directed and an undirected graph is one word in the definition: whether the pair joining two vertices is ordered. Everything else in this article, including which algorithms still work, is a consequence of that word. An undirected graph is the standard object covered in the guide to vertices and edges. Following Diestel's Graph Theory: G = (V, E) with E ⊆ [V]² an edge is an unordered pair {u, v} A directed graph, or digraph, replaces the unordered pair with an ordered one: D = (V, A) with A ⊆ V × V an arc is an ordered pair (u, v) Because (u, v) and (v, u) are different ordered pairs, both can be present at once, and a digraph containing both is said to have a digon between u and v. In the undirected world there is nothing to distinguish: {u, v} and {v, u} are the same set, so the edge exists once or not at all. Diestel gives a more general formulation worth knowing, because it is the one that survives contact with real data. A directed graph is a pair (V, E) of disjoint sets together with two maps init: E → V assigning each edge its initial vertex ter: E → V assigning each edge its terminal vertex Here an arc is an object in its own right rather than a pair, so the definition admits parallel arcs and loops without any special pleading. It is the directed counterpart of the incidence-function definition that multigraphs need, and it is why a flight schedule with three separate daily flights from A to B is still a perfectly good digraph. The formal relationship to logic is exact and worth stating once: an undirected graph without loops is precisely an irreflexive symmetric relation on V, while a digraph is an arbitrary binary relation on V. Direction is what you get when you stop insisting the relation be symmetric. The running example. The digraph on the right has six arcs; the undirected graph on the left is its underlying graph, where the two arcs between D and E collapse into one edge. These two graphs are the running example for the whole article. The digraph is V = {A, B, C, D, E} A = { (A,B), (B,C), (C,A), (C,D), (D,E), (E,D) } 6 arcs and the undirected graph on the left is its underlying graph, with 5 edges, since the opposite arcs between D and E become the single edge {D, E}. 2. Edges, arcs, tails and heads The vocabulary changes along with the definition, and the changes are not decorative. Bang-Jensen and Gutin's Digraphs, the standard reference on the directed side, is careful to reserve separate words so that a statement can never be ambiguous about which object it means. UndirectedDirectedNotes Edge {u, v}Arc (u, v)Many authors say "directed edge" for arc; the meaning is identical EndpointsTail u and head vThe arrow points at the head u and v are adjacentv is an out-neighbour of uAnd u is an in-neighbour of v. The relation is no longer symmetric Degree deg(v)Out-degree d+(v), in-degree d-(v)Two numbers where there was one Walk, path, cycleDirected walk, path, cycleEvery step must follow an arc forwards ConnectedStrongly, unilaterally or weakly connectedOne notion splits into three, see section 7 Tree, forestArborescence, branchingA tree with every arc pointing away from a root Two terms deserve their own line because they are routinely mixed up. An oriented graph is a digraph with no digons: you took an undirected graph and chose one direction for each edge. Every oriented graph is a digraph, but a digraph containing both (u,v) and (v,u) is not an oriented graph. This distinction is the whole subject of section 5. 3. Degree splits in two In an undirected graph the degree of a vertex counts the edge ends meeting it, and the handshaking lemma says those counts sum to twice the number of edges. In a digraph each arc has one tail and one head rather than two symmetric ends, so the single count splits into two: - Out-degree d+(v): the number of arcs whose tail is v. - In-degree d-(v): the number of arcs whose head is v. and the one identity splits into two as well: undirected ∑v∈V deg(v) = 2m every edge has two ends directed ∑v∈V d+(v) = ∑v∈V d-(v) = |A| every arc has one tail and one head The missing factor of 2 catches people out. It is not a different theorem, it is the same double-counting argument applied to a set whose elements now contribute to two separate sums instead of twice to one. Check it on the running example. Out-degrees are A 1, B 1, C 2, D 1, E 1, summing to 6. In-degrees are A 1, B 1, C 1, D 2, E 1, also summing to 6, which is the number of arcs. On the underlying undirected graph the degrees are 2, 2, 3, 2, 1, summing to 10, which is twice its 5 edges. Two named vertex types come out of this immediately and have no undirected counterpart at all: - A source has in-degree 0: nothing points at it. - A sink has out-degree 0: it points at nothing. Sources and sinks are the entry and exit points of flow networks and the starting and finishing positions of a topological order. In an undirected graph the concepts are simply not expressible. 4. What changes in the matrix and the list Direction shows up in storage as clearly as it does in the definition, and the differences are the ones set out in Cormen, Leiserson, Rivest and Stein's Introduction to Algorithms. Symmetry is the visible signature of an undirected graph. On the right, C reaches D but D does not reach C, so the two mirrored entries disagree. The adjacency matrix. For an undirected graph the matrix is always symmetric, A = AT, because {u, v} and {v, u} are the same edge. For a digraph it generally is not, and that asymmetry carries real information: - Row sums are out-degrees, column sums are in-degrees. In the undirected case both give the degree, which is why the distinction never comes up. - The transpose AT is exactly the reverse digraph, with every arc flipped. Reversing an undirected graph does nothing, which is another way of saying that A = AT. - The entry (Ak)uv counts walks of length k from u to v in both worlds, but in the directed case those walks must respect the arrows, so the count is generally different from (Ak)vu. The adjacency list. An undirected graph stores every edge twice, once in each endpoint's list, so the lists hold 2m entries. A digraph stores each arc once, in the tail's list, giving m entries. This has a practical consequence that surprises people the first time: to walk a digraph backwards you need a second structure, the reverse adjacency list, because a vertex's list tells you where you can go, not where you came from. Kosaraju's algorithm for strongly connected components is built directly on that observation and traverses the reverse graph on its second pass. Two counting bounds follow. A simple undirected graph on n vertices has at most n(n-1)/2 edges. A digraph with no loops has at most n(n-1) arcs, exactly twice as many, because each ordered pair is now its own slot. 5. Orientations and the underlying graph The two worlds are connected by a pair of constructions that go in opposite directions, and naming them properly removes a lot of confusion. - Given a digraph D, its underlying graph is what you get by forgetting the arrows: replace each arc (u, v) by the edge {u, v} and discard duplicates. The running example's underlying graph has 5 edges, one fewer than the digraph's 6 arcs, because the digon between D and E collapses. - Given an undirected graph G, an orientation of it is a digraph obtained by choosing exactly one direction for every edge. The result is an oriented graph: no digons, since each edge yields one arc. These are not inverse operations. Taking the underlying graph loses information that no orientation can restore, and a graph with m edges has 2m distinct orientations, since each edge is an independent binary choice. The running example's underlying graph has 5 edges and therefore 32 orientations, of which the original digraph is not even one, because the original has a digon. That raises the question the next section answers. Of those 2m orientations, is any of them good, in the sense that you can still get everywhere? Directed vs Undirected Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 6. Robbins' theorem: which streets can go one-way In 1939 Herbert Robbins published a short paper in the American Mathematical Monthly with the memorable title "A theorem on graphs, with an application to a problem of traffic control". The problem is exactly the one a city planner faces: if every street becomes one-way, can drivers still reach every part of town? Robbins' theorem. A connected undirected graph has a strongly connected orientation if and only if it has no bridge. A bridge, also called a cut edge, is an edge whose removal disconnects the graph. A connected graph with no bridges is exactly a 2-edge-connected graph, one where every edge lies on a cycle. (Connectedness matters here: a disconnected graph can be bridgeless without being 2-edge-connected.) A bridge admits only two orientations and both of them strand one side. Everything else on a bridgeless graph can be oriented so that the whole graph stays mutually reachable. One direction of the proof is the easy one and is worth seeing, because it explains the whole result. Suppose e = {u, v} is a bridge, so removing it splits the graph into a component holding u and a component holding v. Any orientation must send e one way or the other. If it becomes (u, v) then nothing on v's side can ever return to u's side, because e was the only connection and it now points the wrong way. If it becomes (v, u) the same argument runs in reverse. Either way the orientation fails to be strongly connected. The converse, that every bridgeless connected graph does admit a strongly connected orientation, is the substantial half, and the standard proof runs a depth-first search and orients tree edges away from the root and back edges towards it. The running example makes the theorem concrete. Its underlying graph contains the triangle A, B, C, which is bridgeless, but the edges {C, D} and {D, E} are both bridges. So by Robbins' theorem no orientation of that graph is strongly connected, which is precisely why the digraph in the figure is not strongly connected no matter how you redraw the arrows. Nash-Williams generalised the result in 1960: every 2k-edge-connected undirected graph has a k-arc-connected orientation, of which Robbins' theorem is the case k = 1. The practical reading is unchanged. One-way systems are safe exactly where the road network has redundancy, and a single road connecting a suburb to the rest of town can never be made one-way without cutting it off. 7. Connectivity becomes three different questions In an undirected graph, connectivity is a single yes or no: is there a path between every pair of vertices? Direction breaks that into a hierarchy. The classification is due to Harary, Norman and Cartwright's Structural Models, and it is the piece of directed-graph theory most often skipped and most often needed. A digraph iswhen, for every pair u and vRunning example Strongly connectedu reaches v and v reaches uNo: D cannot reach A Unilaterally connectedu reaches v or v reaches uYes: A reaches D, which is enough for that pair Weakly connectedthe underlying undirected graph is connectedYes Disconnectednot even weakly connectedNo Each condition implies the one below it, so strong implies unilateral implies weak. The running example sits exactly in the middle of the hierarchy, which is the common case in practice: you can get from the A, B, C triangle out to D and E, but never back. The useful refinement is to stop asking about the whole digraph and ask about its parts. A strongly connected component, or SCC, is a maximal set of vertices in which every vertex reaches every other. Every digraph partitions uniquely into SCCs, and contracting each one to a single vertex produces the condensation, which is always acyclic. That last fact is not an accident: if the condensation had a cycle, every component on it would reach every other, so they would all have been one SCC to begin with. Two strongly connected components and the condensation they induce. Contracting each component always leaves a DAG, whatever the original digraph looked like. Finding the SCCs takes linear time. Tarjan's 1972 paper "Depth-first search and linear graph algorithms" does it in a single depth-first traversal using low-link numbers, and the Kosaraju-Sharir method does it with two passes, the second over the reverse digraph. Both run in O(n + m), and both have no undirected counterpart, because in an undirected graph the connected components fall out of any single traversal. 8. Cycles, DAGs and topological order The word "cycle" quietly means something stricter once arrows are involved, and the gap causes real bugs. In a simple undirected graph, a cycle is a closed walk with no repeated vertex, and it needs at least three vertices, since going along an edge and straight back is not considered a cycle. An undirected graph with no cycles is a forest, and a connected one is a tree. In a digraph, a directed cycle must follow the arrows the whole way round, and a digon counts: the two arcs (D, E) and (E, D) form a directed cycle of length 2. A digraph with no directed cycles is a DAG, a directed acyclic graph, and DAGs carry a property nothing in the undirected world has: A digraph has a topological order, a linear arrangement of its vertices in which every arc points forwards, if and only if it is acyclic. Kahn's 1962 paper in Communications of the ACM gave the standard algorithm: repeatedly take a vertex of in-degree 0, output it, and delete it along with its outgoing arcs. If the digraph empties, the output is a topological order; if it stalls with vertices remaining, every survivor lies on a cycle. The details are in the guide to topological sorting. Two traps follow from all this: - The underlying graph having a cycle says nothing. The running example's underlying graph contains the triangle A, B, C, and its digraph does contain a directed cycle there. But orient that same triangle as A → B, A → C, B → C and you have a DAG whose underlying graph still has a cycle. Undirected cycle detection cannot answer a directed question. - Cycle detection is a different algorithm. In an undirected graph a DFS finds a cycle when it meets any already-visited vertex that is not the parent it came from. In a digraph that test is wrong: you must find a back edge to a vertex still on the current recursion stack, since an arc into an already-finished vertex is harmless. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 9. Which algorithms transfer, and which break The practical question is which parts of an undirected toolkit survive the move. The pattern is clearer than it first appears: anything that only follows edges forwards transfers, and anything that relies on symmetry does not. ProblemUndirectedDirectedWhat changes BFS and DFSWorksWorksSame code, follow out-arcs only. Reachability is now one-way Shortest path, non-negative weightsDijkstraDijkstraNothing. Dijkstra never assumed symmetry Shortest path, negative weightsUnbounded, or NP-hardBellman-FordA single negative undirected edge can be walked back and forth, so it is already a negative cycle: shortest walks are unbounded, and restricting to simple paths makes the problem NP-hard Connected componentsOne traversalTarjan or Kosaraju-Sharir for SCCsThree notions of connectivity instead of one Cycle detectionAny non-parent visited neighbourBack edge to a vertex on the recursion stackThe undirected test gives false positives on a digraph Minimum spanning treeKruskal, PrimDoes not applyThe directed analogue is the minimum arborescence, solved by Chu-Liu/Edmonds, not by a greedy edge sort Eulerian circuitConnected and every degree evenConnected and d+(v) = d-(v) for every vThe parity condition becomes a balance condition Maximum flowModel as two opposite arcsNativeFlow is directed by definition; Ford and Fulkerson posed it on a digraph Topological sortMeaninglessKahn or DFSNeeds arrows to have anything to order The minimum spanning tree row is the one that catches experienced people. Kruskal's and Prim's algorithms are greedy on a symmetric cost structure, and neither survives orientation. The right directed question is the minimum spanning arborescence: choose a set of arcs of least total weight so that every vertex is reachable from a fixed root. Chu and Liu in 1965 and Edmonds in 1967 solved it independently, and the algorithm looks nothing like a sorted edge scan: it selects each vertex's cheapest incoming arc, then contracts any cycle that forms and repeats. 10. Choosing: is your relation symmetric? The modelling question has one form: if the relation holds from u to v, must it hold from v to u? If yes, use an undirected graph. If no, or if you are unsure, use a digraph, because a digraph can always express a symmetric relation but not the reverse. RelationSymmetric?Model "is friends with" on a social networkYes, by construction on most platformsUndirected "follows" on a social networkNoDirected "links to" between web pagesNoDirected. Brin and Page's PageRank is defined on this digraph "co-authored a paper with"YesUndirected "cites"No, and it is usually acyclic in timeDirected, very nearly a DAG "is connected by a two-way street to"YesUndirected, unless costs differ by direction "depends on" between build targetsNoDirected, and it must be a DAG or the build cannot run "can be reached in one flight from"Usually but not alwaysDirected, since one-way routes exist One case deserves special attention because it looks symmetric and is not. An undirected edge can carry only one weight. If the cost of going from u to v differs from the cost of coming back, the relation is mutual but the model must still be directed. Cycling uphill and downhill, uploading and downloading on an asymmetric link, and exchanging currency in one direction versus the other are all mutual connections with two different costs, and each of them forces a digraph with two arcs carrying different weights. 11. Converting between the two Three conversions come up constantly, and each loses or invents something you should be aware of. - Digraph to undirected (symmetrisation). Replace each arc by an edge. This invents reachability: after symmetrising the running example, D reaches A, which was false in the original. Use it only when you genuinely want to ask a question about the underlying structure, such as whether the digraph is weakly connected. A stricter variant keeps only the mutual pairs, giving the "reciprocal" graph, which invents nothing but discards a great deal. - Undirected to digraph (orientation). Choose one direction per edge. This discards mutual reachability unless the graph is bridgeless, which is exactly Robbins' theorem in section 6. - Undirected as a digraph (arc doubling). Replace each edge {u, v} by both arcs (u, v) and (v, u). This is what almost every graph library does internally, and it is faithful for traversal and shortest paths. But it makes every edge a directed cycle of length 2, so any DAG test now answers false, every connected component becomes one strongly connected component, and a cycle detector reports a cycle for every single edge. The representation is fine; running directed structural algorithms on it is not. 12. Common mistakes - Building the adjacency list one-way for an undirected graph. The single most common graph bug in existence. Reading an edge list and appending only adj[u].push(v) silently produces a digraph, and your BFS then finds no path along an edge it can see. - Building it both ways for a directed graph. The same mistake inverted, and harder to spot: the code runs, the paths look plausible, and every answer that depended on direction is wrong. - Using undirected cycle detection on a digraph. Reporting a cycle whenever DFS meets a visited non-parent vertex flags plenty of DAGs as cyclic. Digraphs need the back-edge-to-the-recursion-stack test. - Assuming reachability is symmetric. "Is there a path from u to v" and "is there a path from v to u" are two different queries in a digraph, and answering one does not answer the other. - Forgetting you need the reverse graph. "Which vertices can reach v" cannot be answered from the forward adjacency list in less than a full scan. Build the reverse list once if you ask that question often. - Reaching for an MST on a digraph. Kruskal and Prim have no directed version. What you want is a minimum arborescence. - Symmetrising to reuse a library. Convenient, and it changes the answer. Every conversion in section 11 has a cost, so make it deliberately. 13. Glossary TermMeaning Arc (u, v)A directed edge, from tail u to head v DigraphA directed graph, D = (V, A) with A ⊆ V × V DigonA pair of opposite arcs between the same two vertices Oriented graphA digraph with no digons: one direction chosen per edge Orientation of GThe oriented graph produced by directing every edge of G Underlying graphThe undirected graph obtained by forgetting all arrow directions Reverse digraphEvery arc flipped; its matrix is AT In-degree, out-degreed-(v) arcs arriving, d+(v) arcs leaving Source, sinkIn-degree 0, out-degree 0 respectively Strongly connectedEvery vertex reaches every other, following arrows SCCA maximal strongly connected set of vertices CondensationThe digraph of SCCs contracted to single vertices; always a DAG DAGA digraph with no directed cycle ArborescenceA directed tree with all arcs pointing away from one root BridgeAn edge whose removal disconnects an undirected graph 14. Frequently asked questions What is the difference between a directed and an undirected graph? An undirected graph joins vertices with unordered pairs {u, v}, so the connection works both ways and the relation is symmetric. A directed graph, or digraph, uses ordered pairs (u, v), so an arc runs from a tail to a head and the reverse arc is a separate object that may or may not exist. Everything else follows from that: degree splits into in-degree and out-degree, the adjacency matrix stops being symmetric, and connectivity splits into strong, unilateral and weak. Is an undirected graph just a digraph with arcs in both directions? For storage and for traversal, yes, and that is exactly how most libraries represent undirected graphs. For structural questions, no. Doubling every edge into two opposite arcs turns each edge into a directed cycle of length 2, so a DAG test always fails, every connected component becomes a single strongly connected component, and a cycle detector fires on every edge. The representation is faithful; running directed structural algorithms on it is not. Does Dijkstra's algorithm work on directed graphs? Yes, without any modification. Dijkstra's algorithm only ever relaxes edges leaving the vertex it has just settled, so it never relies on symmetry. Its real requirement is that weights are non-negative, which is a condition on the weight function rather than on direction. Note the reverse point too: shortest paths with negative weights are really a directed problem, because a single negative undirected edge can be traversed back and forth and is therefore already a negative cycle, which leaves shortest walks unbounded and shortest simple paths NP-hard. What is the difference between a digraph and an oriented graph? An oriented graph is a digraph with no digons, meaning it never contains both (u, v) and (v, u). Equivalently, it is what you get by taking an undirected graph and choosing exactly one direction for each edge. Every oriented graph is a digraph, but a digraph with a pair of opposite arcs is not an oriented graph. An undirected graph with m edges has 2 to the power m distinct orientations. When can every street in a city be made one-way? Exactly when the street network has no bridge, that is, no single road whose removal would split the town in two. This is Robbins' theorem of 1939: a connected undirected graph has a strongly connected orientation if and only if it is bridgeless. The reason a bridge fails is easy to see, since whichever of its two directions you choose, nothing on the far side can ever come back. Do minimum spanning tree algorithms work on directed graphs? No. Kruskal's and Prim's algorithms are greedy on a symmetric cost structure and have no directed version. The directed analogue of the problem is the minimum spanning arborescence: pick the cheapest set of arcs so that every vertex is reachable from a chosen root. Chu and Liu in 1965 and Edmonds in 1967 solved it independently, and the method is different in kind, selecting each vertex's cheapest incoming arc and then contracting any cycle that appears. 15. References The definitions, theorems and attributions above come from these sources, listed in chronological order. - Robbins, H. E. (1939). "A theorem on graphs, with an application to a problem of traffic control." American Mathematical Monthly 46(5), 281 to 283. The bridgeless orientation theorem in section 6. - Ford, L. R. and Fulkerson, D. R. (1956). "Maximal flow through a network." Canadian Journal of Mathematics 8, 399 to 404. Flow posed on a digraph from the beginning. - Nash-Williams, C. St. J. A. (1960). "On orientations, connectivity and odd-vertex-pairings in finite graphs." Canadian Journal of Mathematics 12, 555 to 567. The k-arc-connected generalisation of Robbins. - Kahn, A. B. (1962). "Topological sorting of large networks." Communications of the ACM 5(11), 558 to 562. - Chu, Y. J. and Liu, T. H. (1965). "On the shortest arborescence of a directed graph." Scientia Sinica 14, 1396 to 1400. - Harary, F., Norman, R. Z. and Cartwright, D. (1965). Structural Models: An Introduction to the Theory of Directed Graphs. New York: Wiley. Source of the strong, unilateral and weak classification. - Edmonds, J. (1967). "Optimum branchings." Journal of Research of the National Bureau of Standards 71B(4), 233 to 240. - Tarjan, R. E. (1972). "Depth-first search and linear graph algorithms." SIAM Journal on Computing 1(2), 146 to 160. Linear-time strongly connected components. - Sharir, M. (1981). "A strong-connectivity algorithm and its applications in data flow analysis." Computers & Mathematics with Applications 7(1), 67 to 72. The two-pass method usually paired with Kosaraju's name. - Brin, S. and Page, L. (1998). "The anatomy of a large-scale hypertextual Web search engine." Computer Networks and ISDN Systems 30(1 to 7), 107 to 117. PageRank on the web digraph. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. - Bang-Jensen, J. and Gutin, G. (2009). Digraphs: Theory, Algorithms and Applications, 2nd edition. London: Springer. The standard reference for directed graph terminology. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. Source of the representation costs in section 4. - Chartrand, G., Lesniak, L. and Zhang, P. (2015). Graphs & Digraphs, 6th edition. Boca Raton: CRC Press. A textbook that develops both objects side by side. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Source of both definitions quoted in section 1. Watch direction change the answer Build a graph, flip its edges to arcs, and run the same traversal twice. The set of reachable vertices changing in front of you is the fastest way to internalise everything on this page. Open the visualizer Related Articles Vertices and Edges Explained Read more → Topological Sort Explained Read more → BFS vs DFS Read more → --- ## Weighted vs Unweighted Graphs Explained Source: https://learngraphtheory.org/articles/weighted-vs-unweighted-graphs.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. A weight is a function, not part of the graph - 2. Unweighted means every weight is 1 - 3. The shortest path is not the shortest path - 4. What a weight means: three ways they combine - 5. Which algorithm the weights choose for you - 6. Negative weights, and why Dijkstra breaks - 7. Problems that only exist when weighted - 8. Storing weights, and the zero-versus-infinity trap - 9. Degree becomes strength - 10. When to add weights, and when not to - 11. Common mistakes - 12. Glossary - 13. Frequently asked questions - 14. References 1. A weight is a function, not part of the graph A graph is a pair of sets, G = (V, E), and nothing in that definition mentions numbers. Distances, costs, capacities and durations arrive from outside, as a separate function attached to the same edge set: G = (V, E) the graph: which pairs are joined w: E → ℝ the weight function: what each join costs The standard texts are deliberate about this separation. Bondy and Murty define a weighted graph as a graph together with an assignment of a real number to each edge, and then define the weight of a subgraph as the sum of its edge weights, which is exactly what a shortest path or a minimum spanning tree minimises. Diestel treats weights the same way, as extra data laid over an unchanged combinatorial object. Keeping w outside the graph is not pedantry, it buys three things: - One graph can carry several cost models. A road network is one G with three functions over it: kilometres, minutes, litres of fuel. Swapping the function changes every answer without touching a vertex or an edge. - Conditions attach to the function, not the structure. "Dijkstra needs non-negative weights" is a statement about w. The graph is indifferent. - Structural results survive. Connectivity, planarity, bipartiteness, degree sequences and the handshaking lemma are properties of (V, E) alone, so adding weights cannot change any of them. An unweighted graph is then simply a graph with no such function supplied, and the next section shows that this is the same as supplying the most boring function there is. 2. Unweighted means every weight is 1 The cleanest way to hold the two cases in one head is to stop treating "unweighted" as the absence of weights and start treating it as a particular choice of them: An unweighted graph is a weighted graph with w(e) = 1 for every edge. The weight of a path is then its number of edges, so "shortest path" means "fewest edges". Everything follows from that single substitution. Breadth first search, which finds the fewest-edge path, is exactly what Dijkstra's algorithm degenerates into when every weight equals 1: the priority queue never needs to reorder anything, because distances come off it in non-decreasing integer order anyway, and a plain FIFO queue does the same job in O(n + m). Edward Moore's 1959 paper "The shortest path through a maze" posed and solved precisely this unit-weight problem, and is the paper the algorithm is usually traced back to. The substitution runs the other way too, which is where the cost of weights shows up. Give the same graph arbitrary positive weights and the FIFO queue stops working, because a path with more edges can now be cheaper. You need a priority queue, and the running time goes from O(n + m) to O(m log n) with a binary heap, or O(m + n log n) with the Fibonacci heap of Fredman and Tarjan (1987). 3. The shortest path is not the shortest path Here is the whole distinction in one picture. The same five vertices, the same five edges, asked the same question, give two different answers depending on whether the numbers are there. Fewest edges and least weight are different objectives. The single edge from A to D is the shortest path in the unweighted graph and the worst one in the weighted graph. Written out, the graph is V = {A, B, C, D, E} E = { {A,B}, {B,C}, {C,D}, {A,D}, {D,E} } w = 1 1 1 7 2 and the two questions have these answers: QuestionAlgorithmPath foundEdgesTotal weight Fewest edges from A to DBFSA → D17 Least total weight from A to DDijkstraA → B → C → D33 Fewest edges from A to EBFSA → D → E29 Least total weight from A to EDijkstraA → B → C → D → E45 Notice that the weighted answer uses more edges in both rows. That is the normal case, not a contrived one: a motorway detour is longer in junctions and shorter in minutes. Running BFS on a weighted graph does not give you an approximate answer, it gives you the answer to a different question, and the gap between them is unbounded. Raise the weight on {A, D} to a million and BFS still returns it. 4. What a weight means: three ways they combine "Weighted graph" is a container, not a meaning. Before choosing an algorithm you have to answer a prior question: how do the weights along a path combine into the value you care about? There are three common answers and they lead to three different problems. The same three numbers on the same path give three different path values. Which one you want decides the algorithm before any code is written. Combination ruleWeight meansPath valueProblem and method AdditiveDistance, cost, time, hopsSum of the edgesShortest path: BFS, Dijkstra, Bellman-Ford BottleneckCapacity, bandwidth, reliability of the weakest linkMinimum edge on the pathWidest path, also called maximax or minimax; solved by a modified Dijkstra or from a maximum spanning tree MultiplicativeProbability a link works, transfer ratesProduct of the edgesMost probable path: substitute -log w and it becomes additive The multiplicative trick is worth spelling out because it recurs everywhere from routing to natural language decoding. Maximising a product of probabilities along a path is the same as minimising the sum of their negative logarithms, since -log is monotone decreasing and turns products into sums. Because every probability is at most 1, every -log w is non-negative, so Dijkstra applies directly and no special algorithm is needed. There is one more distinction that causes more modelling bugs than any of the above, and it is not about arithmetic at all: Does a bigger number mean closer or further? In a distance graph, a large weight is bad and you minimise. In a similarity graph, a large weight is good and you maximise. The two are opposites, and the file format does not tell you which one you have. Correlation networks, co-purchase graphs and embedding similarity graphs are all similarity-weighted, so feeding them to a shortest-path routine computes the path through the least similar links. If you need a distance from a similarity, convert deliberately: d = 1 - s for a similarity bounded in [0, 1], or d = 1/s, or d = -log s. Each choice changes the ranking of paths, so it is a modelling decision rather than a formality. 5. Which algorithm the weights choose for you Once the combination rule is additive, the shape of the weight function alone decides the algorithm. This is the practical core of the weighted-versus-unweighted distinction. WeightsUseTimeWhy All equal (unweighted)BFSO(n + m)A FIFO queue already produces non-decreasing distances Only 0 and 10-1 BFS with a dequeO(n + m)Push a 0-edge to the front, a 1-edge to the back, and the deque stays sorted Small integers, bounded by CDial's bucket queueO(m + nC)Buckets replace the heap when the range of distances is small Arbitrary non-negativeDijkstraO(m log n), or O(m + n log n) with a Fibonacci heapThe greedy settle step needs distances to be non-decreasing Any real, no negative cycleBellman-FordO(nm)Relaxing every edge n-1 times needs no ordering assumption Negative edges, all pairsJohnson's algorithmO(nm + n² log n)Reweight once with Bellman-Ford so every weight becomes non-negative, then run Dijkstra from each vertex Two entries in that table deserve a note. 0-1 BFS is the neat observation that if weights are only 0 or 1 you never need a heap at all: a double-ended queue keeps the frontier sorted for free, which recovers linear time. Johnson's algorithm, from his 1977 paper in the Journal of the ACM, is the standard way to keep Dijkstra's speed on graphs with negative edges: it adds a potential function that makes every reweighted edge non-negative while preserving which paths are shortest. There is also a striking result at the boundary of the unweighted case. Thorup showed in 1999 that single-source shortest paths on an undirected graph with positive integer weights can be solved in linear time, matching BFS, by exploiting the structure of integer weights rather than comparing distances. No comparable linear-time result is known for arbitrary real weights in the comparison-addition model, which is a reminder that "weighted" is not one problem but a family whose difficulty depends on what the weights look like. Weighted vs Unweighted Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 6. Negative weights, and why Dijkstra breaks Dijkstra's 1959 note assumed non-negative weights, and the assumption is load-bearing rather than decorative. The algorithm is greedy: once it removes a vertex from the queue it declares that vertex settled and never revisits it. That is sound only if no path discovered later can be cheaper, which is exactly what non-negativity guarantees, since extending a path can only add to its cost. Introduce one negative edge and the guarantee fails. Here is a counterexample small enough to trace by hand, and free of ties so the pop order is forced: Dijkstra settles B at 2 while C is still on the queue. The arc from C is worth -5, so the true distance is -2, but B has already been closed and the answer is never revisited. Trace it: the queue pops S at 0 and relaxes A to 1 and C to 3. It pops A at 1 and relaxes B to 2. It pops B at 2 and marks it settled. Only then does it pop C at 3 and find the arc C → B of weight -5, which would give B a distance of -2. Because B is already settled, the improvement is discarded and the algorithm reports 2 instead of -2. Two clarifications that matter more than the counterexample itself: - Negative weights are not the same as negative cycles. A graph can have negative edges and still have well-defined shortest paths, which is exactly the case Bellman-Ford handles in O(nm). What breaks the problem entirely is a cycle of negative total weight, because you can go round it repeatedly and drive the cost to minus infinity. Bellman-Ford detects that condition rather than silently returning nonsense. - On an undirected graph a single negative edge is already a negative cycle. Walk along it and back and you have paid 2w < 0. So negative weights are effectively a directed-graph topic; on undirected graphs the shortest-walk problem becomes unbounded and the shortest-simple-path problem becomes NP-hard. The companion guide on directed versus undirected graphs covers that boundary. Negative weights are not exotic. Arbitrage chains price currency conversions as products, which become sums of negative logarithms, and a profitable cycle appears in the model as a negative cycle. That is the standard textbook application of negative-weight detection, and it is why Bellman-Ford is worth its extra factor of n. 7. Problems that only exist when weighted Some questions are not harder without weights, they are empty. The clearest case is the minimum spanning tree. In an unweighted connected graph every spanning tree has exactly n - 1 edges, so every spanning tree is minimum and the problem is solved by any traversal: the BFS or DFS tree is already an answer. Add weights and the question becomes real, because spanning trees now have different total costs, and finding the cheapest is what Borůvka in 1926, Kruskal in 1956 and Prim in 1957 each solved. On the running example the minimum spanning tree takes {A,B}, {B,C}, {C,D} and {D,E} for a total of 5, and rejects the expensive {A,D} at 7. Unweighted, all four spanning trees of that graph would be equally good. ProblemUnweightedWeighted Shortest pathFewest edges, BFS in O(n + m)Least total weight, Dijkstra or Bellman-Ford Minimum spanning treeTrivial: every spanning tree tiesThe real problem: Kruskal, Prim, Borůvka Maximum flowUnit capacities, a special caseCapacities are the weights; the whole subject MatchingMaximum cardinality matchingMaximum weight matching, a different algorithm Widest pathMeaninglessBottleneck objective, section 4 Clustering and community detectionBased on edge presenceBased on edge strength, which changes the communities found CentralityCounts of paths and neighboursWeighted variants; degree becomes strength, section 9 Maximum flow is the mirror image of the spanning tree case. Capacities are the weight function, so an unweighted flow network means unit capacities, which is the special case where max flow reduces to counting edge-disjoint paths by Menger's theorem. Ahuja, Magnanti and Orlin's Network Flows is the standard reference for the general weighted treatment, where each arc typically carries both a capacity and a cost, two weight functions on one graph. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 8. Storing weights, and the zero-versus-infinity trap Both standard representations extend in the obvious way, and both have a failure mode that is worth naming. Adjacency matrix. Instead of 0 and 1, entry (u, v) holds the weight of that edge. The trap is immediate: what goes in the cells with no edge? Zero is the tempting default and it is wrong, because zero is a perfectly legal weight and the two cases become indistinguishable. Use ∞ for "no edge" in shortest-path contexts, since that is the identity for minimisation, and keep 0 on the diagonal. In an unweighted matrix the same cell means "no edge" with the value 0, which is precisely why code ported from unweighted to weighted breaks here. unweighted A[u][v] = 1 if joined, else 0 weighted A[u][v] = w(u,v) if joined, else ∞ (0 on the diagonal) sentinel bug A[u][v] = 0 for "no edge" makes a zero-weight edge invisible, and every distance collapse to 0 Adjacency list. Each entry becomes a pair rather than a bare vertex, so the list holds (neighbour, weight). Nothing else changes, and this is why the adjacency list is the default for weighted work: the memory overhead is one number per stored edge, and the traversal loop is identical. A third format matters specifically for weighted graphs. The edge list of triples (u, v, w) is the natural input to Kruskal's algorithm, which sorts the whole list by weight, and to Bellman-Ford, which relaxes every edge in turn. Neither needs neighbour lookups, so neither needs an adjacency structure at all. 9. Degree becomes strength Weights change descriptive statistics as well as algorithms. The weighted analogue of a vertex's degree is its strength, the sum of the weights of its incident edges: deg(v) = number of incident edges the unweighted count s(v) = ∑ w(e) over edges incident to v the weighted total Barrat, Barthélemy, Pastor-Satorras and Vespignani introduced the term in their 2004 PNAS paper on weighted networks, and the reason it matters is that the two quantities can rank vertices completely differently. An airport with many tiny regional routes has high degree and low strength; a hub with four enormous long-haul routes has low degree and high strength. Asking "which is the most important airport" gives a different answer depending on which you compute, and neither is wrong. The same split runs through the rest of network analysis. Newman's 2004 paper "Analysis of weighted networks" shows how clustering coefficients, modularity and centrality all acquire weighted versions, and that the weighted and unweighted versions of a measure frequently disagree on the same data. When you report a network statistic, saying whether it used the weights is not a footnote, it is part of the definition. 10. When to add weights, and when not to Weights are not free. They cost you linear-time algorithms, they add a modelling decision at every step, and they introduce scale sensitivity that an unweighted graph simply does not have. Reach for them when the answer genuinely depends on magnitude: - Add weights when the edges are measurably unequal in a way that changes the decision: road lengths, link capacities, transaction amounts, correlation strengths, similarity scores. - Stay unweighted when the relation is binary in nature (adjacency of countries, presence of a dependency), when the numbers you have are noisy proxies you would not defend, or when the question is purely structural, such as connectivity or bipartiteness. - Threshold instead when weights exist but are unreliable. Keeping edges above a cutoff and discarding the rest turns a noisy weighted graph into a defensible unweighted one. State the cutoff, because results usually depend on it. Two cautions specific to weighted data. First, scale matters: multiplying every weight by a positive constant leaves shortest paths and minimum spanning trees unchanged, since both minimise a sum, but it changes any statistic that compares weights against an absolute threshold, and a negative multiplier inverts the problem entirely. Second, units must agree before weights are added together. Mixing minutes with kilometres in one weight function produces numbers that no algorithm can interpret, and nothing in the code will complain. 11. Common mistakes - Running BFS on a weighted graph. The most common of all. It returns the fewest-edge path, which is a correct answer to a different question, and the error is unbounded, as section 3 shows. - Using 0 as the "no edge" sentinel. Fine until a genuine zero-weight edge exists, then silently wrong. Use infinity for minimisation problems. - Feeding a similarity graph to a shortest-path routine. It will faithfully find the route through the weakest links. Convert similarity to distance first, and say how. - Reaching for Dijkstra with negative weights. It does not merely lose optimality guarantees on some inputs, it returns concretely wrong numbers, as in section 6. Use Bellman-Ford, or Johnson for all pairs. - Assuming a negative edge means a broken problem. Only a negative cycle makes shortest paths undefined. Bellman-Ford handles the rest and reports the cycle if one exists. - Adding weights of different units. Minutes plus kilometres is meaningless, and no algorithm will tell you. - Reporting a weighted network statistic without saying so. Degree and strength, and their derived centralities, routinely rank the same vertices differently. - Forgetting that structure is unchanged. Connectivity, bipartiteness and degree sequences do not depend on w. If a weighted algorithm gives an answer that contradicts one of them, the bug is in the weighting, not the theory. 12. Glossary TermMeaning Weight function w: E → ℝAssigns a number to each edge; not part of G = (V, E) Unweighted graphEquivalently, a weighted graph with w(e) = 1 everywhere Weight of a pathThe sum of its edge weights, under the additive convention Distance d(u, v)The minimum weight over all paths from u to v Bottleneck valueThe minimum edge weight along a path; maximised by the widest path Negative cycleA cycle of negative total weight; makes shortest paths undefined Strength s(v)The sum of the weights of the edges at v, the weighted degree Minimum spanning treeA spanning tree of least total weight; trivial when unweighted ReweightingShifting weights by a potential so they become non-negative, as in Johnson's algorithm ThresholdingTurning a weighted graph unweighted by keeping only edges above a cutoff 13. Frequently asked questions What is the difference between a weighted and an unweighted graph? A weighted graph carries a function w assigning a number to each edge, on top of the graph G = (V, E) itself. An unweighted graph has no such function, which is the same as every edge having weight 1. The practical consequence is that "shortest path" means fewest edges in the unweighted case and least total weight in the weighted case, and those are frequently different paths. Can I use BFS on a weighted graph? You can run it, and it will answer a different question: it returns the path with the fewest edges, ignoring the weights entirely. That is not an approximation of the least-weight path and the gap between them has no bound. Two exceptions are genuine: if every weight is equal, BFS is correct and faster than Dijkstra; and if the weights are only 0 and 1, a deque-based 0-1 BFS gives the correct weighted answer in linear time. Why does Dijkstra's algorithm fail with negative weights? Because it is greedy: when it removes a vertex from the priority queue it declares that distance final and never revisits it. That is sound only when extending a path cannot reduce its cost, which is exactly what non-negative weights guarantee. With a negative edge, a cheaper route can appear after the vertex has been settled, and the improvement is discarded. Section 6 gives a four-vertex example where Dijkstra returns 2 and the true distance is -2. Use Bellman-Ford instead, or Johnson's algorithm for all pairs. Is a minimum spanning tree meaningful in an unweighted graph? Not really. Every spanning tree of a connected graph on n vertices has exactly n-1 edges, so with equal weights they all have the same total and every spanning tree is minimum. Any BFS or DFS traversal already produces one in linear time. The minimum spanning tree problem only becomes interesting when the edges have different costs, which is why Kruskal's and Prim's algorithms are weighted algorithms by nature. Do weights change whether a graph is connected? No. Connectivity, bipartiteness, planarity, degree sequences and cycle structure are all properties of the pair (V, E) alone, and the weight function sits outside it. Adding, removing or rescaling weights cannot change any of them. If a weighted computation seems to contradict a structural fact, the error is in the weighting or the code, not in the theory. How do I handle probabilities or similarities as weights? Convert them to an additive cost first. For probabilities, the value of a path is the product of its edges, and maximising a product is the same as minimising the sum of negative logarithms, so replace w by -log w and run Dijkstra: every probability is at most 1, so every -log w is non-negative. For similarities, decide explicitly on a distance, such as 1 - s, 1/s or -log s. Feeding raw similarities to a shortest-path routine finds the path through the least similar links, which is almost never what was wanted. 14. References The definitions, algorithms and attributions above come from these sources, listed in chronological order. - Borůvka, O. (1926). "O jistém problému minimálním" (About a certain minimal problem). Práce Moravské Přírodovědecké Společnosti 3, 37 to 58. The earliest minimum spanning tree algorithm. - Kruskal, J. B. (1956). "On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem." Proceedings of the American Mathematical Society 7(1), 48 to 50. - Prim, R. C. (1957). "Shortest Connection Networks and Some Generalizations." Bell System Technical Journal 36(6), 1389 to 1401. - Bellman, R. (1958). "On a Routing Problem." Quarterly of Applied Mathematics 16(1), 87 to 90. Shortest paths that tolerate negative weights. - Dijkstra, E. W. (1959). "A Note on Two Problems in Connexion with Graphs." Numerische Mathematik 1, 269 to 271. The non-negativity assumption is stated here. - Moore, E. F. (1959). "The Shortest Path Through a Maze." Proceedings of an International Symposium on the Theory of Switching, Part II, 285 to 292. Harvard University Press. The unit-weight case, now known as BFS. - Johnson, D. B. (1977). "Efficient Algorithms for Shortest Paths in Sparse Networks." Journal of the ACM 24(1), 1 to 13. Reweighting to remove negative edges. - Fredman, M. L. and Tarjan, R. E. (1987). "Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms." Journal of the ACM 34(3), 596 to 615. Dijkstra in O(m + n log n). - Ahuja, R. K., Magnanti, T. L. and Orlin, J. B. (1993). Network Flows: Theory, Algorithms, and Applications. Englewood Cliffs: Prentice Hall. The standard reference for capacities and costs as weights. - Thorup, M. (1999). "Undirected Single-Source Shortest Paths with Positive Integer Weights in Linear Time." Journal of the ACM 46(3), 362 to 394. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. - Barrat, A., Barthélemy, M., Pastor-Satorras, R. and Vespignani, A. (2004). "The Architecture of Complex Weighted Networks." Proceedings of the National Academy of Sciences 101(11), 3747 to 3752. Source of vertex strength. - Newman, M. E. J. (2004). "Analysis of Weighted Networks." Physical Review E 70, 056131. Weighted versions of the standard network measures. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. Source of the weighted-graph definition in section 1. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Change one weight and watch the path move Build the graph from section 3, run Dijkstra, then raise the weight on a single edge and run it again. Seeing the route jump is worth more than any amount of reading about it. Open the visualizer Related Articles Vertices and Edges Explained Read more → Directed vs Undirected Graphs Read more → Shortest Path Algorithms Read more → --- ## Simple Graphs vs Multigraphs Explained Source: https://learngraphtheory.org/articles/simple-graphs-vs-multigraphs.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Three definitions, two allowances - 2. Graph theory was founded on a multigraph - 3. What a loop does to degree - 4. What multiplicity changes, and what it cannot - 5. Every simple-graph bound stops holding - 6. Storage: where the adjacency matrix runs out - 7. Which algorithms care - 8. When a multigraph is not optional - 9. Simplifying, and what it costs - 10. Directed multigraphs - 11. Common mistakes - 12. Glossary - 13. Frequently asked questions - 14. References 1. Three definitions, two allowances Two things a set of unordered pairs cannot express: an edge that joins a vertex to itself, and two distinct edges joining the same pair. Whether you permit them gives three different objects, and the names are worth getting right because theorems are stated about specific ones. The simple graph is the default in most of the literature, and its definition is the one from the guide to vertices and edges: G = (V, E) with E ⊆ [V]² each edge is a 2-element subset of V Because E is a set of 2-element subsets, {v, v} is not admissible (it has one element) and the same pair cannot appear twice (a set holds each element once). Both restrictions are consequences of the notation rather than decisions someone made. To lift them you need a different formalism. Bondy and Murty's Graph Theory gives edges an identity of their own and adds a function saying which pair each one joins: G = (V, E, ψ) with ψ: E → unordered pairs of (not necessarily distinct) vertices Now e1 and e2 can be different elements of E with ψ(e1) = ψ(e2) = {u, v}, which is a pair of parallel edges, and ψ(e) = {v, v} is a loop. Diestel reaches the same place with two maps sending each edge to its ends, and West with a relation associating each edge with its endpoints. The formalism differs; the content does not. ObjectLoopsParallel edgesNeeds Simple graphNoNoE ⊆ [V]² MultigraphUsually noYesAn incidence function PseudographYesYesAn incidence function The same four vertices under the three definitions. Each step to the right buys an allowance and costs a formalism. Two warnings about the vocabulary itself, because they cause real confusion when reading papers: - "Multigraph" is not used consistently. Some authors allow loops in a multigraph, others reserve that for "pseudograph", and a few use "multigraph" for any graph at all. Check the source's own definition before quoting one of its theorems. - "Graph" usually means "simple graph". Most texts say so once in chapter one and then never repeat it. A result stated for "graphs" often has an unstated simplicity hypothesis, and section 5 shows how badly some of them fail without it. 2. Graph theory was founded on a multigraph This is not a marginal case bolted on afterwards. The problem that started the subject is a multigraph, and it stops being the same problem if you simplify it. Euler's 1736 paper on the bridges of Königsberg models four landmasses joined by seven bridges. Two bridges connect the north bank to the island, and two more connect the south bank to the island. Those are parallel edges, and no simple graph can hold them. Left, the real bridges: four odd-degree landmasses, so no walk crosses every bridge exactly once. Right, the same map with the parallel bridges merged: only two odd vertices, and the walk becomes possible. Euler's criterion is about degree parity, and the parallel bridges are what push the degrees to where they are: Königsberg multigraph 7 edges degrees 3, 5, 3, 3 four odd → no Eulerian trail Underlying simple graph 5 edges degrees 2, 3, 2, 3 two odd → a trail exists Delete the duplicates and the answer flips. The multiplicity is not decoration on the model, it is the model, and the historical detail in the history of graph theory is inseparable from the formalism. Anyone who loads the seven bridges into a data structure that silently deduplicates edges will conclude that Euler was wrong. 3. What a loop does to degree Degree counts edge ends meeting a vertex, not edges. A loop has two ends and both of them land on the same vertex, so: A loop at v contributes 2 to deg(v). Parallel edges each contribute 1, exactly as separate edges do. That convention is forced, not chosen. The handshaking lemma counts the pairs (vertex, edge end at it) two ways, and every edge including a loop has exactly two ends, so ∑ deg(v) = 2m holds for pseudographs unchanged. Give a loop degree 1 and the oldest theorem in the subject breaks immediately. Some consequences worth having in mind: - A vertex with a loop and no other edges has degree 2, not 0, and is not isolated. - The number of odd-degree vertices is still even, since the proof only uses ∑ deg(v) = 2m. - In a simple graph deg(v) = |N(v)|, the size of the neighbourhood. In a multigraph that identity fails: three parallel edges to one neighbour give degree 3 and one neighbour. Code that computes degree as the length of a deduplicated neighbour set is computing the wrong number. 4. What multiplicity changes, and what it cannot The useful question is not "is my graph simple" but "does the property I am computing depend on multiplicity". The answers split cleanly, and the split is not obvious. The smallest possible experiment. Doubling one edge leaves connectivity and colouring alone, and changes the min cut, the spanning tree count and the girth. PropertyParallel edgesLoopsWhy Connectivity, componentsNo effectNo effectReachability only needs one edge between a pair PlanarityNo effectNo effectA multigraph is planar exactly when its underlying simple graph is BipartitenessNo effectDestroys itA loop is an odd closed walk of length 1 Proper vertex colouringNo effectMakes it impossibleParallel edges impose the same constraint twice; a loop demands a vertex differ from itself Degree, handshakingEach counts 1Counts 2Edge ends, not edges Girth (shortest cycle)Drops to 2Drops to 1Two parallel edges form a cycle of length 2 Edge connectivity, min cutChangesNo effectEvery parallel copy must be cut too Number of spanning treesChangesNo effectEach parallel copy gives a distinct tree Maximum flowChangesNo effectParallel capacities add Eulerian trail or circuitChangesAdds 2 to a degreeParity of degrees is the whole criterion Three of those rows deserve their reasoning spelled out. Colouring ignores parallel edges but dies on loops. A proper colouring requires the ends of every edge to get different colours. A second copy of {u, v} repeats a constraint that is already there, so the set of proper colourings, and therefore the chromatic number and the chromatic polynomial, are exactly those of the underlying simple graph. A loop demands c(v) ≠ c(v), which nothing satisfies, so a pseudograph with a loop has no proper colouring at all and its chromatic polynomial is identically zero. This is why graph colouring is almost always stated for loopless graphs. Spanning tree counts do depend on multiplicity. Two vertices joined by a single edge have one spanning tree; joined by two parallel edges they have two, because choosing either edge gives a different tree. Kirchhoff's matrix-tree theorem of 1847 counts them from the Laplacian, and it is stated for multigraphs precisely because the multiplicities enter the matrix as off-diagonal counts. Electrical networks, which is where Kirchhoff met the problem, routinely have components in parallel. Cuts and flows depend on multiplicity. The minimum number of edges whose removal disconnects u from v is 1 when a single edge joins them and 2 when two do. Since max flow equals min cut, the same holds for flow: k parallel unit-capacity edges carry k units. This is exactly why a multigraph with unit capacities is the natural unweighted model of a flow network. Simple Graphs vs Multigraphs Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Every simple-graph bound stops holding A large fraction of the standard results carry an unstated simplicity assumption, and they do not degrade gracefully without it. They fail outright. Standard resultSimple graphMultigraph Maximum number of edgesm ≤ n(n-1)/2Unbounded: parallel copies may repeat freely Sparse or dense classificationm = O(n) vs Θ(n²)Meaningless without a multiplicity bound Adjacency matrix is 0/1YesNo: entries are counts Degree equals neighbourhood sizedeg(v) = |N(v)|Fails; degree can exceed the number of neighbours Euler's formula for planar graphsn - m + f = 2Still holds, since it counts faces not simplicity Planar edge boundm ≤ 3n - 6 for n ≥ 3Fails: parallel edges bound faces of length 2 Handshaking lemma∑ deg(v) = 2mStill holds, with loops counted twice The two rows that survive are worth noticing as much as the ones that fail. The handshaking lemma and Euler's polyhedral formula are both proved by counting incidences, and counting arguments do not care whether two edges happen to join the same pair. The bounds that fail are the ones proved by choosing distinct pairs of vertices, which is precisely the step a multigraph invalidates. The practical version of this section: when you look up a bound, check whether its proof counts incidences or counts pairs. The first kind travels to multigraphs, the second does not. 6. Storage: where the adjacency matrix runs out The three standard representations degrade very differently, and the differences decide which one you use. The adjacency matrix stops being a 0/1 matrix. The natural extension holds the number of edges joining each pair, so entry (u, v) becomes a count, and a loop conventionally puts 2 on the diagonal so that row sums still give degrees. It works, but it has a fatal limitation for real data: a count cannot carry per-edge information. If your three parallel flights each have a different price, a matrix of counts has nowhere to put them. simple A[u][v] ∈ {0, 1} multigraph A[u][v] = number of edges joining u and v pseudograph A[v][v] = 2 × (number of loops at v) so that row sums are degrees The adjacency list keeps duplicates. The list for u simply contains v as many times as there are edges joining them. Traversal code is unchanged, and BFS or DFS will consider the same neighbour repeatedly, which is harmless when the visited check is on vertices. The edge list becomes the natural format. This is the representation that actually fits the mathematics: each edge is a record with its own identity, so parallel edges are simply distinct records, and per-edge attributes have somewhere to live. That is the incidence-function definition of section 1 expressed as a data structure. One design point follows, and it is the single most useful thing to take from this section: In a multigraph, edges need identities. A pair of endpoints no longer identifies an edge, so anything that refers to an edge, whether a matching, a spanning tree, a flow or a deletion, must refer to an edge ID and not to (u, v). Almost every multigraph bug traces back to that one sentence. Storing a spanning tree as a set of vertex pairs, or a visited-edge set keyed on (u, v), quietly conflates parallel edges and produces answers that are wrong in ways no type checker will catch. 7. Which algorithms care Most traversal-shaped algorithms are indifferent to multiplicity, because they mark vertices. The ones that mark or select edges need attention. AlgorithmOn a multigraphWhat to watch BFS and DFSWork unchangedThey visit a duplicated neighbour twice and skip it; the visited set is on vertices DijkstraWorks unchangedRelaxation naturally keeps the cheapest of several parallel edges Kruskal, PrimWork unchangedThe cycle test rejects the redundant copies automatically Eulerian trail or circuitNeeds the multigraphEvery edge must be traversed once, so parallel edges are separate obligations; mark edge IDs, not pairs Chinese postmanNeeds the multigraphThe algorithm's whole method is to duplicate edges, creating parallel copies deliberately Max flowWorks, and multiplicity mattersParallel capacities add; keep them separate or sum them explicitly Karger's min cutProduces multigraphsContracting an edge merges vertices and creates parallel edges; deduplicating them destroys correctness MatchingCare neededParallel edges give alternative choices for the same pair; loops are never in a matching Vertex colouringIgnore parallel edgesSimplify first; a loop means no colouring exists The Karger row is the one that surprises people, and it is worth stating in full because it inverts the usual instinct. Karger's randomised minimum cut algorithm repeatedly contracts a uniformly random edge, merging its two endpoints into one vertex. Contraction turns two edges that pointed at the two merged vertices into two parallel edges pointing at the new one, and the algorithm's probability analysis depends on keeping every copy, because the chance of contracting an edge is proportional to how many copies there are. Simplify the intermediate graph and the algorithm stops being correct. The same is true of the contraction step in Borůvka's spanning tree algorithm. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 8. When a multigraph is not optional Multigraphs are not a curiosity to be normalised away. They are the honest model whenever two entities can be related more than once and the individual relations matter: - Transport networks. Two cities joined by three different flights, or a road and a rail link between the same pair. Each has its own duration, price and capacity. - Electrical circuits. Components in parallel between the same two nodes, which is exactly the setting where Kirchhoff developed the matrix-tree theorem in 1847. - Transaction and payment graphs. Two accounts may transact many times; collapsing that into one edge loses amounts, timestamps and the count itself, which is usually the signal being looked for. - Chemical reaction and molecular graphs. Double and triple bonds are parallel edges in the classical graph model of a molecule. - Knowledge graphs and RDF. Two entities related by several distinct predicates. This is why such data is usually stored as triples, which is an edge list with a label per edge. - Anything built by contraction. Karger's algorithm, Borůvka's algorithm, and the condensation steps inside many approximation algorithms all create parallel edges as they run, whatever the input was. - Eulerian route problems. Street sweeping and postal routes need every physical street traversed, and two streets between the same junctions are two obligations. 9. Simplifying, and what it costs Turning a multigraph into a simple graph is often the right move, and it is only safe if you know which of the properties in section 4 you are about to change. There are three standard ways, and they answer different questions: MethodKeepsLosesRight for Collapse parallel edges to one, drop loopsConnectivity, planarity, and the chromatic number, though note that dropping a loop turns an uncolourable graph into a colourable oneCuts, flows, spanning tree counts, Eulerian structureStructural questions Collapse and sum the weightsTotal capacity, so max flow and min cut surviveIndividual edge attributesFlow and cut problems Collapse and take the minimum weightShortest path distancesThe alternatives, so cuts and flows breakRouting Note that the second and third rules are incompatible: summing is right for capacities and wrong for distances, taking the minimum is right for distances and wrong for capacities. Which one applies depends on how weights combine along a path, which is the subject of the companion guide on weighted vs unweighted graphs. Choosing the wrong collapse silently answers a different question, and the graph will look perfectly reasonable afterwards. A fourth option is often better than any of them: keep the multigraph and let the algorithm handle it. BFS, DFS, Dijkstra, Kruskal and Prim all run correctly on multigraphs as they stand, so simplification frequently buys nothing and costs information. 10. Directed multigraphs Everything above transfers to directed graphs, with one extra distinction worth naming. In a digraph, the arcs (u, v) and (v, u) are already different objects, which is not multiplicity but direction: that pair is a digon, and an ordinary digraph is already allowed to contain one without any extension to its definition. Plenty of digraphs contain none at all: a DAG never does. Multiplicity in the directed setting means two or more arcs with the same tail and the same head, which again needs an incidence-style definition, as in Bang-Jensen and Gutin's Digraphs. The practical consequences carry over directly: in-degree and out-degree count arcs rather than distinct neighbours, a directed loop adds 1 to both, and Eulerian conditions on a directed multigraph still compare in-degree with out-degree at every vertex. 11. Common mistakes - Loading a multigraph into a structure that deduplicates. A Set of pairs, an adjacency matrix of booleans or a database unique constraint on (u, v) all silently discard parallel edges. The graph then looks fine and every count is wrong. - Identifying edges by their endpoints. In a multigraph (u, v) names a set of edges, not one. Matchings, spanning trees, flows and visited-edge sets must key on edge IDs. - Computing degree as the neighbour count. Correct in a simple graph, wrong the moment an edge is duplicated or a loop appears. - Giving a loop degree 1. It contributes 2, and the handshaking lemma depends on it. - Applying m ≤ n(n-1)/2. That bound, and everything derived from it including sparse-versus-dense reasoning, needs simplicity. - Simplifying before an Eulerian, cut or flow computation. All three depend on multiplicity, as Königsberg demonstrates on the founding example of the subject. - Deduplicating inside a contraction algorithm. Karger's and Borůvka's algorithms create parallel edges deliberately and need them kept. - Assuming a library does what you expect. Graph libraries differ on whether adding an existing edge creates a duplicate, is ignored, or raises. Check, once, in a test. 12. Glossary TermMeaning Simple graphNo loops, no parallel edges; E ⊆ [V]² MultigraphParallel edges allowed; loops allowed or not depending on the author PseudographBoth loops and parallel edges allowed Parallel edgesTwo or more distinct edges with the same pair of ends; also called multi-edges MultiplicityThe number of edges joining a given pair of vertices LoopAn edge whose two ends are the same vertex; contributes 2 to its degree Incidence function ψMaps each edge to the pair of vertices it joins, giving edges their own identity Underlying simple graphWhat remains after collapsing parallel edges and deleting loops Loopless graphParallel edges permitted, loops not GirthLength of the shortest cycle; 2 with parallel edges, 1 with a loop 13. Frequently asked questions What is the difference between a simple graph and a multigraph? A simple graph allows at most one edge between any pair of vertices and no edge from a vertex to itself, because its edge set is a set of 2-element subsets of the vertex set. A multigraph permits several distinct edges joining the same pair, which requires a different definition in which edges have their own identity and an incidence function says which pair each one joins. A pseudograph permits loops as well. Is the Königsberg bridge problem a multigraph? Yes, and necessarily so. Two bridges join the north bank to the island and two more join the south bank to the island, so the model has parallel edges and cannot be a simple graph. It matters: the seven-bridge multigraph has degrees 3, 5, 3 and 3, all four odd, so no Eulerian trail exists, which was Euler's answer in 1736. Collapse the parallel bridges and the degrees become 2, 3, 2 and 3, only two of which are odd, so a trail would exist. Simplifying changes the answer. Does a loop count once or twice in the degree? Twice. Degree counts the edge ends meeting a vertex, and a loop has two ends, both attached to the same vertex. The convention is forced rather than chosen: the handshaking lemma says the degrees sum to twice the number of edges, and its proof counts each edge's two ends, so giving a loop degree 1 would break it. A vertex carrying only a loop has degree 2 and is not isolated. Do parallel edges change the chromatic number? No. A proper colouring requires the two ends of each edge to differ, and a duplicate edge merely repeats a constraint that already exists, so the proper colourings of a multigraph are exactly those of its underlying simple graph, and the chromatic number and chromatic polynomial are unchanged. A loop is different: it would require a vertex to differ in colour from itself, so a graph with a loop has no proper colouring at all. Can I just simplify a multigraph before running an algorithm? Only for properties that do not depend on multiplicity. Connectivity, planarity and colouring survive simplification. Minimum cuts, maximum flows, spanning tree counts, girth and Eulerian trails do not. If you must collapse weighted parallel edges, sum the weights for capacities and take the minimum for distances, and note that those two rules are incompatible. Often the better answer is not to simplify at all, since BFS, DFS, Dijkstra, Kruskal and Prim all run correctly on multigraphs unchanged. How do I store a multigraph in code? Give every edge an identity. An edge list of records, each with its own ID, endpoints and attributes, is the direct expression of the incidence-function definition and the format that scales to per-edge data. An adjacency list works too, holding a neighbour once per parallel edge. An adjacency matrix can only hold counts, so it cannot carry per-edge attributes, and a matrix of booleans silently deletes multiplicity. Whatever you choose, never key a set of visited or selected edges on the endpoint pair. 14. References The definitions, theorems and attributions above come from these sources, listed in chronological order. - Euler, L. (1736). "Solutio problematis ad geometriam situs pertinentis." Commentarii Academiae Scientiarum Petropolitanae 8 (published 1741), 128 to 140. The Königsberg bridges, modelled as a multigraph. - Kirchhoff, G. (1847). "Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Vertheilung galvanischer Ströme geführt wird." Annalen der Physik 148(12), 497 to 508. The matrix-tree theorem, developed on electrical networks with parallel components. - Harary, F. (1969). Graph Theory. Reading, Massachusetts: Addison-Wesley. Distinguishes graphs, multigraphs and pseudographs. - Karger, D. R. (1993). "Global Min-cuts in RNC, and Other Ramifications of a Simple Min-cut Algorithm." Proceedings of the 4th Annual ACM-SIAM Symposium on Discrete Algorithms, 21 to 30. The contraction algorithm that creates parallel edges as it runs. - Karger, D. R. and Stein, C. (1996). "A New Approach to the Minimum Cut Problem." Journal of the ACM 43(4), 601 to 640. - Bollobás, B. (1998). Modern Graph Theory. Graduate Texts in Mathematics 184. New York: Springer. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. Defines a graph by a vertex set, an edge set and an endpoint relation, which admits loops and parallel edges. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. Source of the incidence-function definition in section 1. - Bang-Jensen, J. and Gutin, G. (2009). Digraphs: Theory, Algorithms and Applications, 2nd edition. London: Springer. Directed multigraphs. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. - Wilson, R. J. (2010). Introduction to Graph Theory, 5th edition. Harlow: Prentice Hall. Develops multigraphs alongside simple graphs from the first chapter. - Chartrand, G., Lesniak, L. and Zhang, P. (2015). Graphs & Digraphs, 6th edition. Boca Raton: CRC Press. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Source of the simple-graph definition and of the multigraph formulation with two end maps. Build the seven bridges yourself Lay out the four landmasses, add the two parallel bridges to the island, and check the degrees. Then delete one duplicate and watch the parity change. Open the visualizer Related Articles Vertices and Edges Explained Read more → Eulerian Path and Circuit Read more → History of Graph Theory Read more → --- ## Trees in Graph Theory Source: https://learngraphtheory.org/articles/trees-in-graph-theory.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. A tree has seven definitions, and they are all the same one - 2. The equivalence theorem, and how its proof works - 3. The leaf lemma - 4. Forests, and counting components for free - 5. Spanning trees - 6. Counting labelled trees: Cayley's formula - 7. The Prüfer bijection, worked end to end - 8. The matrix-tree theorem - 9. Unlabelled trees: a much harder question - 10. Centres, radius and diameter - 11. Distance in trees, and the double BFS trick - 12. Free trees, rooted trees, ordered trees - 13. Where trees show up in computing - 14. Common mistakes - 15. Glossary - 16. Frequently asked questions - 17. References 1. A tree has seven definitions, and they are all the same one Ask three people to define a tree and you will get three answers. One says it is a connected graph with no cycles. Another says it is a graph with exactly one path between any two vertices. A third says it is a connected graph with n - 1 edges. All three are right, and so are four more definitions besides, because these conditions are equivalent: any graph satisfying one satisfies all of them. That is unusual, and it is the reason trees are the most useful special case in the subject. A structure with seven equivalent characterisations gives you seven different ways to prove something about it, and in practice you pick whichever one makes the proof shortest. Start with the standard definition, the one in Diestel and in Bondy and Murty: A tree is a connected acyclic graph. A graph whose components are all trees is a forest. A vertex of degree 1 in a tree is a leaf. Everything below is about a finite simple, undirected graph, which is the standard setting. A tree cannot have a loop or a parallel edge anyway, since either one is a cycle. The running example for the whole article: eight vertices, seven edges, four leaves. Every claim below is checked against this tree. Written out, the running example is V = {1, 2, 3, 4, 5, 6, 7, 8} E = { {1,3}, {2,3}, {3,4}, {4,5}, {5,6}, {5,7}, {7,8} } degrees 1:1 2:1 3:3 4:2 5:3 6:1 7:2 8:1 n = 8 m = 7 = n - 1 degree sum = 14 = 2m leaves: 1, 2, 6, 8 2. The equivalence theorem, and how its proof works Here is the result in full. It appears in essentially this form in West, in Bondy and Murty, and in Diestel, and it is worth memorising because each line is a tool. Theorem. For a graph G on n vertices, the following are equivalent: (1) G is a tree, that is, connected and acyclic. (2) Any two vertices of G are joined by exactly one path. (3) G is connected and has n - 1 edges. (4) G is acyclic and has n - 1 edges. (5) G is connected, and removing any single edge disconnects it (minimally connected). (6) G is acyclic, and adding any single edge creates a cycle (maximally acyclic). (7) G is connected and every edge is a bridge. The proof is not one argument but a cycle of implications, each of them short. It is worth seeing the shape, because it explains why the conditions feel so different from one another and yet describe the same object. StepWhy it holds (1) → (2)Connectivity gives at least one path. If two distinct paths joined the same pair, their union would contain a cycle, contradicting acyclicity. (2) → (5)One path between every pair means connected. Deleting the edge {u, v} destroys the only path from u to v, so the graph falls apart. (5) → (1)If a cycle existed, any edge of it could be deleted while leaving the graph connected, since the rest of the cycle still joins its ends. So there is no cycle. (1) → (3)Induction on n. A tree has a leaf (section 3); delete it and you have a tree on n - 1 vertices, which by induction has n - 2 edges. Put the leaf back and you have n - 1. (3) → (4)Suppose G were connected with n - 1 edges and still had a cycle. Delete one edge of that cycle: the graph stays connected, but now has only n - 2 edges, and a connected graph on n vertices needs at least n - 1. The contradiction rules out the cycle. (4) → (1)An acyclic graph with k components and n vertices has exactly n - k edges (section 4). With n - 1 edges, k = 1, so it is connected. (1) ↔ (6)Adding {u, v} to a tree closes the unique existing u to v path into a cycle. Conversely, maximal acyclicity forces connectivity, since two vertices in different components could be joined without creating any cycle. Two of these deserve emphasis because they are the ones people actually use. "Connected and n - 1 edges" is the cheapest test. Counting edges is O(m) and a connectivity check is O(n + m), so you can decide treehood in linear time without ever looking for a cycle. Note that neither half suffices alone: a triangle plus an isolated vertex has 4 vertices and 3 edges but is not a tree, and neither is a 4-cycle, which is connected with 4 edges. "Minimally connected" and "maximally acyclic" are the same object seen from two directions. A tree sits exactly on the boundary: it has as few edges as connectivity allows and as many as acyclicity allows. That is why trees turn up whenever a problem asks for a cheapest connecting structure, which is precisely the minimum spanning tree problem. All seven conditions hold on the running example, and each was checked directly: it is connected, acyclic, has 7 edges on 8 vertices, has exactly one path between each of the 28 vertex pairs, every one of its 7 edges is a bridge, and every one of the 21 missing edges closes a cycle when added. 3. The leaf lemma One small result carries most of the inductive proofs in the subject. Leaf lemma. Every finite tree with at least two vertices has at least two leaves. The proof is a favourite because it uses nothing but the definition. Take a longest path P in the tree, say from u to v. Such a path exists because the tree is finite. Now consider u. If u had a neighbour w outside P, then P could be extended by that edge, contradicting maximality. If u had a second neighbour on P, that would close a cycle, contradicting acyclicity. So u has exactly one neighbour and is a leaf, and the same argument applies to v. Two consequences follow immediately and both are used constantly: - Induction on trees always has a base case to remove. Delete a leaf from a tree and what remains is a tree with one fewer vertex. That single move is the engine of the proof that a tree has n - 1 edges, of the Prüfer encoding in section 7, and of the centre-finding algorithm in section 10. - The bound is tight. A path has exactly two leaves, so "at least two" cannot be improved in general. At the other extreme, the star K1,n-1 has n - 1 leaves. The running example has four leaves, 1, 2, 6 and 8, comfortably more than the guaranteed two. A useful sanity check when debugging tree code: if your structure claims to be a tree and reports fewer than two leaves, something is wrong, and the usual culprit is an accidental cycle. The lemma is also exactly what fails on infinite graphs. The one-way infinite path has a single leaf and the two-way infinite path has none at all, which is one of the cleanest illustrations of what finiteness was buying, covered in the guide to finite and infinite graphs. 4. Forests, and counting components for free A forest is an acyclic graph, connected or not. Every component of a forest is a tree, and that gives a counting identity worth knowing by heart: A forest with n vertices and k components has exactly n - k edges. The proof is one line: each component is a tree, so a component with ni vertices contributes ni - 1 edges, and summing over the k components gives n - k. Setting k = 1 recovers the tree case. Read the identity backwards and it becomes a tool rather than a fact: k = n - m the number of components of a forest, computed from its size alone, with no traversal That is genuinely useful. If you know a graph is acyclic, counting its vertices and edges tells you how many pieces it is in without running a search. It is also the identity behind the standard union-find invariant: every successful union merges two components and adds one edge, so the running count n - (unions so far) is the number of components at any moment. One warning. The identity assumes acyclicity. For a general graph, m ≥ n - k always holds, with equality exactly when the graph is a forest, so a graph with more edges than n - k necessarily contains a cycle. That inequality is the fastest way to prove a graph has a cycle without finding one: if m ≥ n, there is a cycle somewhere. 5. Spanning trees A spanning tree of a connected graph G is a subgraph that is a tree and includes every vertex of G. It is the minimal skeleton that keeps the graph in one piece. Every connected finite graph has one, and the constructive proof is worth knowing because it is also an algorithm: while a cycle exists, delete any edge of it. Deleting a cycle edge cannot disconnect the graph, since the rest of the cycle still joins its endpoints, and the process terminates because each step removes an edge. What remains is connected and acyclic. Equivalently, and more practically, the tree of discovery edges produced by any BFS or DFS traversal is already a spanning tree, found in O(n + m). Three facts about spanning trees that come up repeatedly: - Every spanning tree has exactly n - 1 edges, whatever the graph looked like. So on an unweighted graph all spanning trees tie, and the minimum spanning tree problem is only interesting once edges carry weights. - The number of spanning trees can be enormous. The complete graph Kn has nn-2 of them, which is Cayley's formula again, viewed from the spanning-tree side. - A tree is its own unique spanning tree. Obvious once stated, and a useful degenerate case for testing code: any spanning-tree counter should return exactly 1 on a tree, which the matrix-tree computation in section 8 confirms for the running example. For infinite graphs the statement "every connected graph has a spanning tree" is still true but needs the axiom of choice, and is in fact equivalent to it. That boundary is discussed in the finite and infinite graphs guide. Trees in Graph Theory Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 6. Counting labelled trees: Cayley's formula How many different trees can be built on a fixed set of n labelled vertices? The answer is one of the most quotable results in combinatorics, published by Arthur Cayley in 1889. Cayley's formula. The number of labelled trees on n vertices is nn-2. The first few values grow fast, and they are worth seeing because the smallest ones can be checked by hand: nnn-2Check 21The single edge, and nothing else is possible 33A path on 3 vertices, once for each choice of middle vertex 416Verified by exhaustive enumeration of all edge subsets 5125Verified the same way 61296Already past hand-checking The counts for n = 4 and n = 5 above are not quoted from a book; they were produced by enumerating every subset of n - 1 edges from the C(n, 2) candidates and keeping the connected ones, giving 16 and 125 exactly. A word on what "labelled" means, because the distinction is the whole subject of section 9. Cayley counts trees whose vertices are distinguishable, so the path 1 - 2 - 3 and the path 2 - 1 - 3 are different trees even though they have the same shape. Strip the labels and there is only one shape of tree on three vertices. Several proofs of the formula exist, including a double-counting argument on rooted forests and a determinant argument via the matrix-tree theorem. The most illuminating is a bijection, and it is short enough to work through completely. 7. The Prüfer bijection, worked end to end Heinz Prüfer gave a proof of Cayley's formula in 1918 by constructing an explicit bijection between labelled trees on n vertices and sequences of length n - 2 drawn from {1, …, n}. Since there are exactly nn-2 such sequences, the formula follows at once. Encoding strips the smallest leaf repeatedly and records its neighbour. Six removals reduce eight vertices to two, which is why the sequence has length n minus 2. Encoding. While more than two vertices remain, find the leaf with the smallest label, write down the label of its unique neighbour, and delete the leaf. Stop when two vertices are left. On the running example this produces, step by step: remove leaf 1 → write 3 remaining: 2,3,4,5,6,7,8 remove leaf 2 → write 3 remaining: 3,4,5,6,7,8 remove leaf 3 → write 4 remaining: 4,5,6,7,8 remove leaf 6 → write 5 remaining: 4,5,7,8 remove leaf 4 → write 5 remaining: 5,7,8 remove leaf 5 → write 7 remaining: 7,8 Prüfer sequence: (3, 3, 4, 5, 5, 7) length 6 = n - 2 Decoding. The inverse runs the same idea backwards. Give every vertex a counter equal to one plus the number of times it appears in the sequence, which will be its degree. Then repeatedly take the smallest vertex whose counter is 1 and is not yet used, join it to the first remaining entry of the sequence, and decrement both counters. When the sequence is exhausted, join the two vertices still holding a counter of 1. Running this on (3, 3, 4, 5, 5, 7) returns exactly the original edge set, which is what makes the correspondence a bijection rather than merely a summary. The single most useful property of the encoding is this: Vertex v appears in the Prüfer sequence exactly deg(v) - 1 times. In particular, the leaves are precisely the labels that never appear. Check it against the running example. Vertex 3 has degree 3 and appears twice; vertex 5 has degree 3 and appears twice; vertices 4 and 7 have degree 2 and appear once each; and the leaves 1, 2, 6 and 8 appear not at all. That correspondence turns questions about degree sequences into questions about how often symbols occur in a string, which is why Prüfer sequences are the standard tool for counting trees with prescribed degrees and for sampling a labelled tree uniformly at random: generate a random sequence of length n - 2 and decode it. 8. The matrix-tree theorem Cayley's formula counts spanning trees of the complete graph. Kirchhoff's matrix-tree theorem, which predates it by four decades and came out of his work on electrical networks in 1847, counts the spanning trees of any graph. Build the Laplacian matrix L = D - A, where D is the diagonal matrix of degrees and A is the adjacency matrix. Then: Matrix-tree theorem. Delete any one row and the corresponding column from L. The determinant of the remaining (n-1) × (n-1) matrix is the number of spanning trees of the graph. The choice of which row and column to delete does not matter. Three computations make the theorem concrete, and all three were carried out rather than quoted: GraphSpanning treesCross-check K4, the complete graph on 4 vertices16Agrees with Cayley: 44-2 = 16 C4, the 4-cycle4Delete any one of the 4 edges and a spanning tree remains The running example tree1A tree is its own only spanning tree The cycle case is the one to hold on to as intuition: a cycle on k vertices has exactly k spanning trees, one for each edge you choose to drop. The theorem also explains a remark from the guide to multigraphs: parallel edges genuinely change the spanning tree count, because they enter the Laplacian as off-diagonal multiplicities, so two vertices joined by two parallel edges have two spanning trees rather than one. 9. Unlabelled trees: a much harder question Cayley's formula is clean because labels make trees easy to tell apart. Ask instead how many trees there are up to isomorphism, meaning how many distinct shapes exist, and the problem becomes genuinely hard. nLabelled trees (nn-2)Unlabelled trees 111 211 331 4162 51253 612966 71680711 The two columns tell completely different stories. The labelled count has a one-line closed form; the unlabelled count has none. There is no known formula for the number of trees on n vertices up to isomorphism, only a generating-function treatment and an asymptotic result due to Richard Otter in 1948, which shows the count grows like C · αn n-5/2 for constants worked out numerically. The reason for the gap is symmetry. Dividing the labelled count by n! would be right only if every tree had trivial automorphism group, and most do not: a path can be reflected, a star can have its leaves permuted arbitrarily, and each symmetry means several labellings collapse to the same shape. Counting orbits under the symmetric group is exactly the hard part, and it is the reason the problem needs Pólya's enumeration machinery rather than a formula. For a working programmer the practical form of this distinction is tree isomorphism testing: deciding whether two trees have the same shape. Unlike general graph isomorphism, this is solvable in linear time, by canonically hashing each subtree bottom up from the leaves and comparing the results at the centre. That the general problem is hard while the tree case is easy is one more instance of the pattern running through this whole article. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 10. Centres, radius and diameter The eccentricity of a vertex is its greatest distance to any other vertex. The radius is the smallest eccentricity in the graph, the diameter the largest, and the centre is the set of vertices attaining the radius. For trees these have unusually clean behaviour, first worked out by Camille Jordan in 1869. Jordan's theorem. The centre of a tree consists of either one vertex or two adjacent vertices. Never three, never two non-adjacent ones. Compare that with a cycle, where every vertex is in the centre, and the sharpness of the tree case is clear. Peel the leaves, then peel the new leaves, and keep going. Whatever survives is the centre, which for this tree is the adjacent pair 4 and 5. The proof doubles as the algorithm. Repeatedly delete all current leaves simultaneously. Every deletion reduces the eccentricity of every surviving vertex by exactly 1, so it preserves which vertices are minimal, and the process ends with one or two vertices left. On the running example: start 1 2 3 4 5 6 7 8 strip leaves 1, 2, 6, 8 → remaining 3 4 5 7 strip leaves 3, 7 → remaining 4 5 ← the centre eccentricities 1:5 2:5 3:4 4:3 5:3 6:4 7:4 8:5 radius 3 diameter 5 centre {4, 5}, adjacent as Jordan requires The leaf-stripping result was checked against a direct computation of all eight eccentricities, and the two agree exactly: the vertices of minimum eccentricity are precisely 4 and 5. The algorithm runs in O(n), which is why it is the standard way to root a tree "in the middle", for instance before canonical hashing in isomorphism testing. One more identity holds in trees and is worth remembering: radius = ⌈diameter / 2⌉ here: 3 = ⌈5 / 2⌉ It follows because a tree's diameter is realised by a unique path, and the centre sits at that path's midpoint. In a general graph only the weaker radius ≤ diameter ≤ 2 · radius holds. 11. Distance in trees, and the double BFS trick Because there is exactly one path between any two vertices, distance in a tree is simpler than in any other class of graph. There is nothing to optimise: the unique path is the shortest path, so no weights, no priority queue and no Dijkstra are needed to find it. That uniqueness gives a neat and widely used algorithm for the diameter: Double BFS. Run a breadth first search from any vertex and let a be a farthest vertex found. Run a second search from a and let b be a farthest vertex from it. Then the path from a to b is a diameter, and dist(a, b) is the diameter's length. Two linear passes, no weights, no cleverness. On the running example, starting at vertex 1 the search reaches vertex 8 as a farthest vertex, and a second search from 8 returns vertex 1 at distance 5, matching the true diameter computed by taking the maximum over all eccentricities. The reason it works is worth stating, because the trick fails on general graphs and people transplant it anyway. The key claim is that a farthest vertex from any starting point is always an endpoint of some diameter. In a tree this holds because the unique paths force the farthest vertex to lie at the end of the longest path; in a graph with cycles the claim is simply false, and the two-pass method can return an underestimate. On a general graph, computing the diameter needs all-pairs distances. Some other distance facts that hold in trees and nowhere else in general: - Removing any edge splits the tree into exactly two components, since every edge is a bridge. That is what makes divide-and-conquer on trees, such as centroid decomposition, work so cleanly. - The path between two vertices can be recovered from parent pointers alone once the tree is rooted, in time proportional to the path length, which underpins lowest-common-ancestor techniques. - Distances satisfy the four-point condition, a metric identity characterising exactly those distance matrices that come from trees, which is the basis of phylogenetic tree reconstruction from genetic distance data. 12. Free trees, rooted trees, ordered trees Everything so far has been about free trees: connected acyclic graphs with no distinguished vertex and no ordering among the neighbours of any vertex. Computer science almost always works with something more structured, and Knuth's The Art of Computer Programming is careful to separate the three levels, because the counts differ at every one. ObjectExtra structureExample count on 3 nodes Free treeNone. Just a connected acyclic graph1 shape Rooted treeOne vertex is designated the root, which orients every edge away from it2 shapes: a path rooted at an end, or at the middle Ordered treeThe children of each node have a left-to-right order2 shapes, and the distinction bites from 4 nodes upwards Rooting is not a change to the graph, it is a change to the question. The underlying edge set is identical; what a root adds is a direction, and with it the whole vocabulary of parent, child, ancestor, descendant, depth and height, which the companion guide on rooted trees covers in detail along with the standard traversals. The counting difference is the sharpest way to see that these really are different objects. Ordered binary trees on n nodes are counted by the Catalan numbers, giving 1, 1, 2, 5, 14, 42 for n = 0 to 5, whereas free trees on the same number of vertices are far fewer. Every extra piece of structure you insist on multiplies the number of distinct objects. A practical note that follows from section 10: when an algorithm needs a root and none is given, rooting at the centre is usually the right default. It minimises the height, which bounds the depth of any recursion you run over the tree. 13. Where trees show up in computing Trees are the most common structure in computing that is genuinely a graph, and it is worth separating the cases where the tree is the data from those where it is a certificate produced by an algorithm. Trees as data. The hierarchy is the point: - File systems. Directories and files form a rooted tree, at least until symbolic links and hard links are allowed, at which point it becomes a general graph and the guarantee of a unique path is lost. That is exactly why link loops break naive directory walkers. - Parse trees and abstract syntax trees. Every compiler front end produces one. Being a tree is what makes recursive evaluation well founded: a subexpression cannot contain itself. - The DOM. An HTML document is an ordered rooted tree, and CSS selectors are queries over ancestor and sibling relations in it. - Search trees, tries and heaps. Binary search trees, B-trees and tries are trees whose shape is constrained to bound the depth, which is exactly the height of the rooted tree. - Decision trees. Each internal node tests a feature and each leaf carries a prediction; the unique root-to-leaf path is the explanation of the model's output. Trees as certificates. Here the tree is the output of an algorithm and encodes a proof: - BFS and DFS trees. Every traversal of a connected graph produces a spanning tree of discovery edges. The BFS tree additionally certifies shortest distances in an unweighted graph, and the DFS tree's back edges are what let you detect cycles and find bridges. - Shortest path trees. Dijkstra's algorithm outputs one: a spanning tree in which the path from the source to any vertex is a shortest path. Note it is generally not a minimum spanning tree, and confusing the two is a classic error. - Minimum spanning trees. Kruskal, Prim and Borůvka each certify the cheapest connecting subgraph, discussed in the MST guide. - Union-find forests. The disjoint-set structure literally is a forest, and path compression is an operation that flattens its trees to keep the height near constant. - Merkle trees. Hash trees in version control and distributed systems use the unique-path property so that a single leaf change propagates along exactly one path to the root, making verification logarithmic. One clarification worth making, because the terminology misleads people: a Git commit history is not a tree. A merge commit has two parents, so the history is a directed acyclic graph. Git's "tree" objects are something else entirely, namely the directory snapshots. The distinction between a DAG and a tree is precisely that a tree has a unique path between any two nodes, and a merge destroys that. 14. Common mistakes - Checking only one half of the definition. "Connected" alone admits cycles; "n - 1 edges" alone admits a triangle plus an isolated vertex. You need a pair of conditions from the theorem in section 2, and connected plus n - 1 edges is the cheapest. - Assuming a shortest path tree is a minimum spanning tree. They optimise different things: one minimises each distance from the source, the other minimises total edge weight. They frequently differ. - Using the double BFS diameter trick on a graph with cycles. It is valid only on trees, where the unique-path property makes a farthest vertex a diameter endpoint. On general graphs it can silently underestimate. - Confusing labelled and unlabelled counts. There are 125 labelled trees on 5 vertices and only 3 shapes. Dividing by n! does not convert between them, because trees have symmetries. - Forgetting that rooting changes nothing structurally. A root adds a question, not an edge. The underlying free tree is unchanged, so any structural fact proved for free trees still applies. - Expecting the leaf lemma on infinite trees. The two-way infinite path is acyclic and connected with no leaf at all. - Treating a DAG as a tree. A DAG can have several paths between two nodes; a tree cannot. Any algorithm relying on path uniqueness, including naive memoisation keyed on a node, will misbehave. - Building a "tree" that has a cycle. The fastest runtime check is the edge count: if a supposed tree on n vertices does not have exactly n - 1 edges, stop and look for the bug. 15. Glossary TermMeaning TreeA connected acyclic graph; equivalently any of the seven conditions in section 2 ForestAn acyclic graph; every component is a tree. With n vertices and k components it has n - k edges LeafA vertex of degree 1. Every finite tree with at least 2 vertices has at least 2 Spanning treeA subgraph that is a tree and touches every vertex of the host graph BridgeAn edge whose removal disconnects the graph. In a tree, every edge is one EccentricityThe greatest distance from a vertex to any other Radius, diameterThe minimum and maximum eccentricity. In a tree, radius = ⌈diameter / 2⌉ CentreThe vertices of minimum eccentricity. In a tree this is one vertex or two adjacent ones Cayley's formulaThere are nn-2 labelled trees on n vertices Prüfer sequenceA length n - 2 encoding of a labelled tree; vertex v appears deg(v) - 1 times LaplacianL = D - A; any cofactor counts the graph's spanning trees Free vs rooted treeA free tree has no distinguished vertex; rooting adds a root and orients every edge away from it 16. Frequently asked questions What is a tree in graph theory? A connected graph with no cycles. Six other conditions describe exactly the same object: exactly one path between every pair of vertices; connected with n-1 edges; acyclic with n-1 edges; minimally connected, so removing any edge disconnects it; maximally acyclic, so adding any edge creates a cycle; and connected with every edge a bridge. Any one of them can be taken as the definition, which is why trees are so convenient to prove things about. Why does a tree have exactly n - 1 edges? By induction, using the fact that every finite tree with at least two vertices has a leaf. Delete a leaf and its single edge: what remains is still connected and still acyclic, so it is a tree on n-1 vertices, which by induction has n-2 edges. Adding the leaf back gives n-1. The same counting extends to forests: a forest with n vertices and k components has exactly n-k edges, so the component count can be read off as n minus the edge count. How many trees are there on n vertices? It depends whether the vertices are labelled. With labels, Cayley's formula of 1889 gives exactly n to the power n-2: that is 16 trees on 4 vertices and 125 on 5. Without labels, counting distinct shapes, there is no closed formula at all: the counts run 1, 1, 1, 2, 3, 6, 11 for n = 1 to 7, and only an asymptotic result due to Otter in 1948 is known. The gap exists because trees have symmetries, so many labellings collapse to the same shape. What is a Prüfer sequence used for? It is a bijection between labelled trees on n vertices and sequences of length n-2 over the labels, which proves Cayley's formula immediately since there are n to the power n-2 such sequences. It is also practical: because a vertex appears exactly deg(v)-1 times in the sequence, questions about degree sequences become questions about symbol frequencies, and you can sample a uniformly random labelled tree simply by generating a random sequence and decoding it. How do I find the centre or the diameter of a tree? For the centre, repeatedly delete all current leaves at once until one or two vertices remain; those are the centre, and Jordan proved in 1869 that a tree's centre is always one vertex or two adjacent ones. For the diameter, run a breadth first search from any vertex, take a farthest vertex found, and run a second search from there: the greatest distance in the second pass is the diameter. Both are linear time. The double search trick is valid only on trees, and can underestimate on a graph with cycles. What is the difference between a tree, a spanning tree and a DAG? A tree is an undirected connected acyclic graph. A spanning tree is a tree that sits inside a larger connected graph and reaches all of its vertices, so a graph has many spanning trees while a tree is its own only one. A DAG is directed and has no directed cycle, but it may well have several paths between two nodes, which no tree can. That last point is why a Git commit history, where a merge has two parents, is a DAG and not a tree. 17. References Sources for the definitions, theorems and attributions above, together with the standard texts in which this material is developed, listed in chronological order. - Kirchhoff, G. (1847). "Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Vertheilung galvanischer Ströme geführt wird." Annalen der Physik 148(12), 497 to 508. The matrix-tree theorem. - Jordan, C. (1869). "Sur les assemblages de lignes." Journal für die reine und angewandte Mathematik 70, 185 to 190. The centre of a tree is one vertex or two adjacent vertices. - Cayley, A. (1889). "A Theorem on Trees." Quarterly Journal of Pure and Applied Mathematics 23, 376 to 378. - Prüfer, H. (1918). "Neuer Beweis eines Satzes über Permutationen." Archiv der Mathematik und Physik 27, 142 to 144. The bijection of section 7. - Borůvka, O. (1926). "O jistém problému minimálním." Práce Moravské Přírodovědecké Společnosti 3, 37 to 58. - König, D. (1936). Theorie der endlichen und unendlichen Graphen. Leipzig: Akademische Verlagsgesellschaft. - Otter, R. (1948). "The Number of Trees." Annals of Mathematics 49(3), 583 to 599. Asymptotics for unlabelled trees. - Kruskal, J. B. (1956). "On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem." Proceedings of the American Mathematical Society 7(1), 48 to 50. - Prim, R. C. (1957). "Shortest Connection Networks and Some Generalizations." Bell System Technical Journal 36(6), 1389 to 1401. - Harary, F. (1969). Graph Theory. Reading, Massachusetts: Addison-Wesley. - Knuth, D. E. (1997). The Art of Computer Programming, Volume 1: Fundamental Algorithms, 3rd edition, section 2.3. Reading, Massachusetts: Addison-Wesley. The free, rooted and ordered distinction. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. Chapter 2 develops trees and distance. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition. Cambridge, Massachusetts: MIT Press. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Section 1.5 covers trees and forests. Build a tree and try to break it Lay out the eight-vertex example, count the edges, then add one more edge anywhere and watch a cycle appear. Delete an edge instead and watch it fall into exactly two pieces. Both are the equivalence theorem in action. Open the visualizer Related Articles Rooted Trees in Graph Theory Read more → Minimum Spanning Trees Read more → Vertices and Edges Explained Read more → --- ## Graph Representation Explained Source: https://learngraphtheory.org/articles/graph-representation.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Representation is not an implementation detail - 2. The running example - 3. The edge list - 4. The adjacency matrix - 5. The adjacency list - 6. What “sparse” actually means - 7. Compressed sparse row - 8. The incidence matrix - 9. Direction, weight, multiplicity and loops - 10. Algebraic representations - 11. When the matrix wins - 12. Graphs that change - 13. Implicit graphs: storing nothing at all - 14. Compressed and succinct representations - 15. A decision procedure - 16. Common mistakes - 17. Glossary - 18. Frequently asked questions - 19. References 1. Representation is not an implementation detail A graph is an abstract object: a set of vertices and edges, nothing more. A computer cannot store an abstract object. It stores bytes, and the choice of which bytes decides, before you write a single line of algorithm, what your program is capable of. That claim is easy to state and easy to underestimate, so here is the sharpest version of it. Breadth-first search runs in O(n + m) time on an adjacency list and O(n2) time on an adjacency matrix. Same algorithm, same graph, same output. Only the storage differs. On a graph with a million vertices and fifty million edges, that is the difference between about 51 million operations and about a trillion: a factor of roughly 19,600. No amount of constant-factor tuning recovers that. The representation was the algorithm's asymptotic complexity all along. The reason is simple once seen. Both versions of BFS do the same thing at each vertex: enumerate its neighbours. An adjacency list answers “who are v's neighbours?” in time proportional to how many there are. An adjacency matrix answers it by scanning a whole row of length n, most of which is zeros. Summed over all vertices, the list costs 2m and the matrix costs n2. This is exactly the observation that Hopcroft and Tarjan built their linear-time graph algorithms on in the early 1970s, and it is why the adjacency list became the default in every algorithms course since. But the adjacency list is not always the answer, and treating it as the automatic default is its own mistake. Ask whether two given vertices are adjacent and the matrix answers in one memory access while the list scans a neighbour sequence. Multiply the graph by itself and the matrix hands you walk counts for free. Store a genuinely dense graph and the matrix uses less memory, not more. Run on a GPU and neither structure is what you want. So the honest framing is not “which representation is best” but which question are you going to ask most often, and how big is the graph. This article works through the seven representations that matter in practice, derives the cost of each operation on each, and ends with a decision procedure. Every number about the example graph below was computed by script rather than asserted, and the arithmetic is reproduced so you can check it. 2. The running example One small weighted graph carries the whole article. It is deliberately tiny enough to write out completely in every representation, and deliberately irregular enough that the representations look genuinely different. V = {0, 1, 2, 3, 4, 5} E = { {0,1}:4 {0,2}:3 {1,2}:2 {1,3}:5 {2,4}:7 {3,4}:1 {3,5}:6 } n = 6 m = 7 degree sum = 14 = 2m degrees 0:2 1:3 2:3 3:3 4:2 5:1 The running example. Six vertices, seven weighted edges. Every table in this article encodes exactly this graph. Two facts about it will keep coming back. Its neighbour sets, written in sorted order, are 0 → 1, 2 1 → 0, 2, 3 2 → 0, 1, 4 3 → 1, 4, 5 4 → 2, 3 5 → 3 and its density is 7 / 15 = 46.7%, since a simple graph on 6 vertices admits at most C(6,2) = 15 edges. That is very dense by real-world standards, which is a useful corrective: toy graphs in textbooks are almost always dense, and the intuitions they build about representation are almost always wrong for production data. We will fix that in section 6. 3. The edge list The simplest representation is to write down the edges and stop. edges = [ (0,1,4), (0,2,3), (1,2,2), (1,3,5), (2,4,7), (3,4,1), (3,5,6) ] An array of m triples. Space is Θ(n + m) if you also keep a vertex count, and Θ(m) if the vertex set is implicit in the edges. Nothing is precomputed, nothing is indexed. The consequence is that almost every query is a full scan. “Are 1 and 4 adjacent?” requires walking all seven edges. “What are 3's neighbours?” requires walking all seven edges. Both are O(m), which is catastrophic if you do it inside a loop over vertices, because that turns into O(nm). And yet the edge list is not a naive choice, because three important situations want exactly this shape: - Algorithms that iterate over edges rather than vertices. Kruskal's algorithm sorts all edges by weight and considers them in order; it never asks for a neighbour list. Bellman-Ford relaxes every edge n - 1 times; again, pure edge iteration. For these, the edge list is not merely adequate, it is optimal, since any other representation would have to be flattened back into edge order. - Interchange and storage. Every graph file format on disk is an edge list, because it is the only representation that is order-independent, append-only and trivially parseable. When you download a dataset from SNAP or the DIMACS collections, you get an edge list, and your first step is to convert it. - Streaming. If the graph does not fit in memory at all, an edge list is what arrives, one edge at a time, and the semi-streaming model of computation is built on the assumption that this is all you get. The practical rule is that an edge list is a transport format and an iteration format, not a query format. Real systems read an edge list and immediately build something indexed. That conversion is the subject of section 7, and it is cheaper than people expect: a counting sort over vertex identifiers does it in O(n + m). 4. The adjacency matrix Number the vertices 0 to n - 1 and build the n × n matrix A where A[u][v] = 1 if {u, v} is an edge and 0 otherwise. For the running example: 0 1 2 3 4 5 row sum 0 [ 0 1 1 0 0 0 ] 2 1 [ 1 0 1 1 0 0 ] 3 2 [ 1 1 0 0 1 0 ] 3 3 [ 0 1 0 0 1 1 ] 3 4 [ 0 0 1 1 0 0 ] 2 5 [ 0 0 0 1 0 0 ] 1 36 cells, 14 of them nonzero Three structural properties fall straight out, and each one is a usable check when debugging. - The matrix of an undirected graph is symmetric. A[u][v] = A[v][u] always, so half the storage is redundant. Directed graphs give up this symmetry, which is precisely what makes direction visible in the algebra. - The row sums are the degrees: 2, 3, 3, 3, 2, 1, matching the list in section 2. The total number of ones is 2m = 14, since each edge contributes two cells. - The diagonal is zero for a simple graph, because a nonzero diagonal entry is a loop. What you buy is constant-time adjacency. Asking whether 1 and 4 are adjacent is one array read, A[1][4], independent of degree. No other representation in this article does that without hashing. What you pay is Θ(n2) space regardless of how many edges exist, and Θ(n) time to enumerate one vertex's neighbours regardless of how few it has. That last cost is the one that bites. Vertex 5 has a single neighbour, but reading row 5 to discover this touches six cells. Scale that to a million vertices and finding the neighbours of a degree-one vertex touches a million cells. The O(n2) in matrix BFS is entirely this effect, accumulated. The bit-packing refinement. If the graph is unweighted, each cell needs one bit, not one byte and certainly not one 32-bit integer. Packing rows into machine words cuts memory by a factor of 8 against a byte matrix and 32 against an int matrix, and it does something more interesting: it lets you operate on 64 neighbours per instruction. Intersecting two neighbourhoods, which is the inner loop of triangle counting and of many clique algorithms, becomes a word-wise AND over n/64 words rather than a loop over n entries. We return to this in section 11, because it is the main reason dense representations survive. 5. The adjacency list Store, for each vertex, a sequence of its neighbours. adj[0] = [ (1,4), (2,3) ] adj[1] = [ (0,4), (2,2), (3,5) ] adj[2] = [ (0,3), (1,2), (4,7) ] adj[3] = [ (1,5), (4,1), (5,6) ] adj[4] = [ (2,7), (3,1) ] adj[5] = [ (3,6) ] Space is Θ(n + m): one slot per vertex plus 2m = 14 neighbour entries for an undirected graph, or m for a directed one. Enumerating the neighbours of v costs Θ(deg(v)), which is optimal, since you cannot list k things in less than k time. This is the representation that makes linear-time graph algorithms possible, and its adoption has a precise history. Tarjan's 1972 depth-first search paper and the Hopcroft and Tarjan companion algorithms of 1973 are explicit that the O(n + m) bounds depend on adjacency-list storage; the same procedures on a matrix are O(n2). Aho, Hopcroft and Ullman's 1974 textbook then made the list the standard presentation, and it has been the default in traversal code ever since. The cost is that adjacency testing is no longer constant. To answer “are 1 and 4 adjacent?” you scan adj[1], three entries, and find nothing. In general that is O(deg(u)), or O(min(deg(u), deg(v))) if you are careful enough to scan the shorter list. On a graph with a few very high-degree vertices, which is what every social or web graph looks like, that minimum can still be millions. The neighbour ordering question. Nothing in the definition says the neighbour sequences must be sorted, and most code leaves them in insertion order. Sorting them costs O(m log m) once and buys two things: binary-searchable adjacency in O(log deg(u)), and linear-time neighbourhood intersection by merge, which is what fast triangle-counting implementations rely on. If you intersect neighbourhoods at all, sort. The implementation trap. The textbook picture of an adjacency list is an array of linked lists, and the textbook picture is bad advice on modern hardware. A linked list dereferences a pointer per neighbour, and each dereference is a potential cache miss costing on the order of a hundred cycles. A vector> is better, since each vertex's neighbours are contiguous, but it still scatters n separately allocated blocks across the heap and pays a per-vertex allocation header. The fix is section 7. Hash-based variants. Replacing each neighbour sequence with a hash set gives expected O(1) adjacency testing while keeping Θ(n + m) space, which looks like the best of both. In practice the constant is unkind: a hash set costs several times the memory of a packed array of integers, destroys iteration locality, and makes the neighbour scan, which is the operation you actually do most, meaningfully slower. Use it when adjacency queries genuinely dominate traversal, and measure rather than assume. 6. What “sparse” actually means Everything above turns on one word. A graph is sparse when m is close to n and dense when m is close to n2, and the practical dividing line is the density density = m / C(n,2) = 2m / (n(n-1)) which is the fraction of possible edges that exist. The running example sits at 7/15 = 46.7%, which is enormously dense. Real graphs are not like this. A social network with a million users and fifty million friendships, an average of 100 friends each, has density 1.0 × 10-4: one hundredth of one percent. Road networks are worse still, with average degree under 3 because intersections have a bounded number of roads. The web graph, citation graphs, protein interaction networks and dependency graphs are all in the same regime. Here is what that costs, computed for exactly that million-vertex graph: RepresentationFormulaBytes for n = 106, m = 5 × 107 Adjacency matrix, one byte per celln2931 GiB Adjacency matrix, one bit per celln2 / 8116 GiB vector> adjacency list≈ 40n + 8m420 MiB Compressed sparse row8(n+1) + 8m389 MiB The matrix is roughly 300 times larger than the sparse structures even when packed down to single bits, and it does not fit in the memory of any ordinary machine. This is not a marginal preference. It is the difference between a program that runs and one that cannot be started. The crossover is worth knowing precisely. A bit-packed matrix costs n2/8 bytes; a sparse structure storing a 4-byte target per directed arc costs about 8m bytes. The matrix wins when n2/8 < 8m, that is when m > n2/64, which corresponds to a density above roughly 3.1%. Above that, use a matrix; below it, do not. Almost every graph you will meet outside of small combinatorial search problems is below it by three orders of magnitude. One graph, three encodings. The matrix spends 36 cells to record 14 ones; the list spends 14 entries. At this density that hardly matters, and at realistic densities it decides everything. The full operation-by-operation comparison, with d written for the degree of the vertex involved: OperationEdge listAdjacency matrixAdjacency listCSR SpaceΘ(n + m)Θ(n2)Θ(n + m)Θ(n + m) Is u adjacent to v?O(m)O(1)O(d)O(d), or O(log d) sorted List the neighbours of uO(m)Θ(n)Θ(d)Θ(d), contiguous Degree of uO(m)Θ(n)O(1)O(1) Iterate over all edgesΘ(m)Θ(n2)Θ(n + m)Θ(n + m) Add an edgeO(1)O(1)O(1) amortisedΘ(n + m) rebuild Delete an edgeO(m)O(1)O(d)Θ(n + m) rebuild BFS or DFSO(nm)Θ(n2)Θ(n + m)Θ(n + m), faster constant The table in essentially this form is the standard presentation, going back to Aho, Hopcroft and Ullman and reproduced in the graph chapter of Cormen, Leiserson, Rivest and Stein. Read it as a statement about which question you ask, not about which row is best. The only cell where the matrix is uniquely strong is adjacency testing, and the only cells where the edge list is strong are whole-edge iteration and appending. Everything else belongs to the sparse indexed structures. Graph Representation Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 7. Compressed sparse row The representation that production graph code actually uses is not the array-of-vectors adjacency list. It is compressed sparse row, borrowed wholesale from sparse linear algebra, where it has been standard since Gustavson's work in the early 1970s and is documented in Duff, Erisman and Reid as the canonical sparse storage scheme. Graph people sometimes call it the forward star representation, or simply a flattened adjacency list. The idea is to concatenate all the neighbour sequences into one array and keep a second array recording where each vertex's slice begins. offsets = [ 0, 2, 5, 8, 11, 13, 14 ] length n + 1 = 7 targets = [ 1, 2, 0, 2, 3, 0, 1, 4, 1, 4, 5, 2, 3, 3 ] length 2m = 14 weights = [ 4, 3, 4, 2, 5, 3, 2, 7, 5, 1, 6, 7, 1, 6 ] length 2m = 14 The neighbours of vertex v are targets[offsets[v] .. offsets[v+1] - 1]. For vertex 2 that is positions 5 through 7, giving neighbours [0, 1, 4] with weights [3, 2, 7], which matches adj[2] exactly. Degree comes for free as offsets[v+1] - offsets[v], recovering 2, 3, 3, 3, 2, 1 without touching the target array at all. CSR in full. Two flat arrays replace n separately allocated neighbour lists, and a vertex's neighbours become a contiguous slice. Asymptotically this is identical to an adjacency list. In practice it is substantially faster, for four reasons that have nothing to do with big-O: - Locality. A vertex's neighbours occupy consecutive bytes, so scanning them streams through cache lines instead of chasing pointers. The hardware prefetcher can see the access pattern and stay ahead of it. - No per-vertex allocation. Two allocations replace n + 1 of them. On the million-vertex example that removes about 38 MiB of headers and, more importantly, removes a million opportunities for the neighbour blocks to be scattered. - Trivially serialisable. The whole structure is two integer arrays, so it can be memory-mapped from disk, sent over a network, or handed to a GPU with no pointer fixing. - Index compression. Since targets holds vertex identifiers, a graph with fewer than 232 vertices needs only 4 bytes per entry, and one with fewer than 216 needs 2. The offsets array does need 64-bit entries once 2m exceeds 231, which is a real and frequently made mistake at billion-edge scale. Building CSR from an edge list is O(n + m) and does not require sorting. Count the degree of every vertex in one pass, prefix-sum the counts into offsets, then make a second pass placing each edge into its slot using a moving cursor per vertex. This is a counting sort by source vertex, and it is the standard ingestion path in every serious graph library. The price is rigidity. Inserting one edge shifts every subsequent entry in targets, so the structure is effectively immutable: you rebuild it in Θ(n + m) rather than update it. That is a fine trade when the graph is loaded once and queried many times, which describes most analytics workloads, and a bad one when the graph changes constantly. Section 12 deals with the second case. 8. The incidence matrix The third classical matrix indexes vertices against edges rather than against vertices. Label the seven edges e1 through e7 in the order they were listed, and set B[v][e] = 1 when v is an endpoint of e: e1 e2 e3 e4 e5 e6 e7 0 [ 1 1 0 0 0 0 0 ] 1 [ 1 0 1 1 0 0 0 ] 2 [ 0 1 1 0 1 0 0 ] 3 [ 0 0 0 1 0 1 1 ] 4 [ 0 0 0 0 1 1 0 ] 5 [ 0 0 0 0 0 0 1 ] column sums all 2 row sums 2,3,3,3,2,1 = degrees The shape is n × m, so space is Θ(nm), which is worse than the adjacency matrix for any graph with more edges than vertices. Nobody stores a graph this way for computation. The incidence matrix earns its place for a different reason: it is the bridge between graph theory and linear algebra. Two identities make the point. For the unsigned matrix above, B BT = A + D, where D is the diagonal matrix of degrees. Substituting the running example's numbers confirms it exactly. If instead you orient each edge arbitrarily and write -1 at its tail and +1 at its head, the signed incidence matrix Bs satisfies B_s B_s^T = D - A = L, the Laplacian independently of which orientation you chose. That identity is the reason the Laplacian is positive semidefinite, and it is the entry point to spectral graph theory. Diestel develops this line further, using the incidence matrix to define the cycle space and the cut space of a graph, two vector spaces over the field of two elements whose dimensions are m - n + c and n - c for a graph with c components. The incidence matrix is also the natural setting for flow problems: the constraint matrix of a network flow linear program is the signed incidence matrix, and its total unimodularity is what guarantees that the linear program has integer optimal solutions. One further note. The incidence matrix handles multigraphs more gracefully than the adjacency matrix does, since parallel edges are simply distinct columns rather than a count crammed into one cell. Hypergraphs, where an edge may join more than two vertices, have no sensible adjacency matrix at all but a perfectly natural incidence matrix with column sums greater than two. If you ever need to generalise past ordinary graphs, this is the representation that generalises. 9. Direction, weight, multiplicity and loops Everything so far assumed a simple undirected graph. Four common departures change what each representation must do, and the differences are where implementation bugs cluster. Direction. In a directed graph the adjacency matrix stops being symmetric, and A[u][v] = 1 means an arc from u to v only. The adjacency list stores each arc once instead of twice, so the neighbour arrays hold m entries rather than 2m. That halving is the single most common source of off-by-a-factor-of-two memory estimates. The real complication is that a directed graph has two neighbourhoods. adj[v] gives successors; predecessors require either scanning the entire structure or storing a second copy with every arc reversed. Sparse linear algebra calls that second copy compressed sparse column, and any algorithm that walks backwards, including reverse reachability, Kosaraju's strongly connected components procedure and backward Dijkstra in bidirectional search, needs it. Budget for two structures, not one. Weight. Weights can live in the matrix cells directly, replacing the 1 with the weight. The subtlety is what a non-edge becomes: 0 is a legitimate weight, so a 0 cell is ambiguous. The convention is to store ∞ for absent edges in shortest-path settings, which is exactly what Floyd-Warshall assumes on entry, and 0 in flow settings where a zero-capacity arc and an absent arc really are the same thing. Pick one deliberately. In sparse structures the weight goes in a parallel array indexed identically to targets, as in the CSR listing above, which keeps the two in lockstep and preserves locality. Storing pairs interleaved instead is also fine and sometimes better; storing weights in a separate hash map keyed by edge is almost always worse. Multiplicity. Parallel edges break the adjacency matrix's basic premise, since a cell holds one value. The usual repair is to store the multiplicity as an integer count, which works for counting problems but discards per-edge data such as distinct weights or identifiers. Adjacency lists take multi-edges without complaint: the same neighbour simply appears more than once. If you need per-edge attributes on a multigraph, store edge identifiers in the neighbour arrays and keep the attributes in a separate edge table indexed by those identifiers, which is what most graph databases do. Loops. A loop at v puts a nonzero on the diagonal. The convention that catches people out is that in an undirected graph a loop contributes 2 to the degree of v, so the standard undirected adjacency matrix stores A[v][v] = 2 for a single loop in order to keep the row-sum-equals-degree identity true. Plenty of code stores 1 instead and then quietly reports wrong degrees. In an adjacency list the same question becomes whether v appears once or twice in its own neighbour sequence, and the honest answer is that you must decide and document it, because both conventions exist in the literature. 10. Algebraic representations Once a graph is a matrix, matrix operations mean something. This is not a curiosity; it is the basis of an entire style of graph computation. Powers of the adjacency matrix count walks. The entry Ak[u][v] is exactly the number of walks of length k from u to v, which follows by induction from the definition of matrix multiplication. On the running example: A^2 = [ 2 1 1 1 1 0 ] [ 1 3 1 0 2 1 ] [ 1 1 3 2 0 0 ] [ 1 0 2 3 0 0 ] [ 1 2 0 0 2 1 ] [ 0 1 0 0 1 1 ] Read off A2[1][4] = 2: there are two length-2 walks from 1 to 4, namely 1→2→4 and 1→3→4. Check it against the picture. The diagonal A2[v][v] is 2, 3, 3, 3, 2, 1, which is the degree sequence again, because a length-2 walk from v to itself is a step out to a neighbour and back. Going one power further, trace(A3) = 6, and dividing by 6 gives one triangle, which brute force confirms is {0, 1, 2}. The division by 6 counts the three starting points and two directions of each triangle. The Laplacian. Define L = D - A: L = [ 2 -1 -1 0 0 0 ] [ -1 3 -1 -1 0 0 ] [ -1 -1 3 0 -1 0 ] [ 0 -1 0 3 -1 -1 ] [ 0 0 -1 -1 2 0 ] [ 0 0 0 -1 0 1 ] Every row sums to zero, so the all-ones vector is in the kernel and L is singular. Kirchhoff's matrix-tree theorem says that deleting any one row and the matching column and taking the determinant counts the graph's spanning trees. All six cofactors of the matrix above equal 11, and enumerating all C(7,5) = 21 five-edge subsets and testing each for acyclicity finds exactly 11 spanning trees. The theorem is not an approximation; it is an identity, and it turns a counting problem that looks exponential into one determinant. The Laplacian's eigenvalues carry more. The multiplicity of the eigenvalue 0 is the number of connected components. The second smallest eigenvalue, Fiedler's algebraic connectivity, measures how hard the graph is to disconnect, and the sign pattern of its eigenvector gives a usable graph bisection. This is the machinery behind spectral clustering and behind a large part of Chung's spectral graph theory. Graphs as linear algebra over semirings. The deepest version of this idea is that many graph algorithms are matrix operations, once you change the arithmetic. Replace (+, ×) with (min, +) and matrix multiplication becomes shortest-path relaxation, so An-1 over the min-plus semiring is the all-pairs shortest-path matrix. Replace it with (OR, AND) and it becomes reachability. Breadth-first search from a source is repeated multiplication of a sparse frontier vector by the adjacency matrix over a boolean semiring. Kepner and Gilbert set this out systematically, and it is the specification the GraphBLAS standard implements. The payoff is practical: expressing an algorithm as sparse matrix-vector products lets it inherit decades of tuned parallel linear algebra, which is how many GPU graph frameworks are built. 11. When the matrix wins Given section 6, it would be easy to conclude that adjacency matrices are a teaching device. They are not, and it is worth being precise about the four situations where the matrix is the right answer. Small n. If n is a few hundred, n2 is a few tens of thousands of cells and the memory argument evaporates. Floyd-Warshall computes all-pairs shortest paths in Θ(n3) time and Θ(n2) space on a matrix, with a three-line inner loop and near-perfect cache behaviour; for n in the hundreds it routinely beats running Dijkstra n times on a sparse structure, despite the worse asymptotics. Competitive programming and operations research are full of this regime. Genuinely dense graphs. Above the roughly 3.1% density crossover computed earlier, the matrix is smaller as well as faster. Complement graphs, similarity graphs with a permissive threshold, and constraint graphs from scheduling problems land here regularly. Bitset parallelism. This is the strongest argument. Pack each matrix row into machine words and set operations on neighbourhoods become word-parallel. Transitive closure via the Four Russians technique, introduced by Arlazarov, Dinic, Kronrod and Faradzev in 1970, computes reachability in O(n3 / log n) by precomputing results for blocks of bits; the same trick with plain 64-bit words gives a very large constant-factor win with almost no code. Triangle counting, maximum clique via branch and bound, and Boolean matrix products all lean on this. A sparse structure simply cannot do 64 adjacency tests in one instruction. Access to fast matrix multiplication. Some graph problems reduce to matrix multiplication and inherit its exponent. Seidel's algorithm computes all-pairs shortest paths in an unweighted undirected graph in O(nω log n) time by repeated squaring of the adjacency matrix, where ω is the matrix multiplication exponent. Alman and Vassilevska Williams brought ω below 2.3729 in 2021, and subsequent refinements have pushed it slightly lower still. These bounds are largely theoretical, since the algorithms achieving them have constants that make them impractical, but the reduction is real and it exists only because the graph is a matrix. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 12. Graphs that change Every structure above was described as if the graph were fixed. Many are not, and update cost is a dimension the standard comparison table underplays. The clean cases are the extremes. An adjacency matrix supports both insertion and deletion in O(1), since both are a single cell write; its problem was never update speed. An edge list appends in O(1) but deletes in O(m), because it must find the edge first. CSR does neither: any structural change rebuilds the whole thing in Θ(n + m). Adjacency lists sit in between and reward a little care. Appending a neighbour to a dynamic array is O(1) amortised. Deleting is O(deg(u)) to locate the entry, but only O(1) to remove it once found, provided you swap the last element into the hole rather than shifting everything down. If you also need to delete the reverse copy in an undirected graph, store each entry's position in its twin so the second deletion is O(1) too, which is exactly what the classic array-based edge representation with paired indices does. Three patterns cover most real needs: - Batch and rebuild. Accumulate updates in a small side buffer, answer queries against CSR plus buffer, and rebuild CSR when the buffer grows past a threshold. A rebuild costs Θ(n + m), so spreading it over Θ(m) updates is O(1) per update whenever m is at least n, which covers essentially every real graph. This is what most analytics systems do, and it is usually enough. - Tombstoning. Mark deleted entries rather than removing them, and compact periodically. Removing the entry becomes O(1) once it has been found, and no other entry moves, so indices into the neighbour arrays stay valid. Locating the entry is still O(deg(u)) unless you already hold a handle to it, and every later scan has to skip dead entries. - Genuinely dynamic structures. When updates and connectivity queries interleave and both must be fast, the data structure literature has answers. Link-cut trees and Euler tour trees maintain a changing forest in logarithmic time per operation, and general dynamic connectivity on arbitrary graphs is built on top of them, at O(log2 n) amortised per update. These are considerably more intricate than anything above, and you should reach for them only when the batch-and-rebuild pattern has been measured and found wanting. One warning specific to hardware. A structure that is fast in the asymptotic table can be slow in practice because updates fragment it. An adjacency list that has grown by a million individual insertions has its neighbour blocks scattered across the heap in allocation order, and a subsequent traversal pays for that scattering on every vertex. Periodically rebuilding into CSR is often worth it purely to restore locality, even when no asymptotic bound changes. 13. Implicit graphs: storing nothing at all There is one more representation, and it is the one people forget exists: do not store the graph. An implicit or procedural graph is defined by a function. Instead of a data structure you supply a successor routine that, given a vertex, generates its neighbours on demand. Nothing is materialised until it is visited. This is not a fringe technique. It is how essentially all of state-space search works: - Puzzle and game state spaces. The graph of Rubik's cube configurations has about 4.3 × 1019 vertices. Storing it is not a question of engineering effort: one bit per state is already more than five exabytes, and the arcs, at eighteen per state, would run past six zettabytes. Its successor function, apply one of the eighteen face turns, is a dozen lines of code. Search algorithms work perfectly well on it. - Planning and model checking. The reachable state graph of a concurrent program is generated by executing transitions. Explicit-state model checkers store only the visited set, never the edges. - Geometric and grid graphs. A pathfinding grid has an obvious successor function, four or eight offsets with a bounds and obstacle check. An adjacency list for a 4096 by 4096 grid with eight-way movement holds 134 million entries, every one of which a two-line function reproduces exactly. This matters directly for A* on grids. The consequences are worth stating plainly. Space drops from Θ(n + m) for the graph to Θ(|visited|) for the search, which is what makes the technique viable at all. In exchange you lose everything that requires seeing the whole graph: you cannot count edges, compute a degree distribution, or run any algorithm that iterates over all vertices. You also cannot cheaply ask for predecessors unless you write a second function for them, and regenerating a neighbourhood costs CPU every time instead of a memory read, which can be the more expensive option for a heavily revisited region. Implicit representation is also what licenses the memory-bounded search family. Iterative deepening A* keeps only the current path rather than an open list, trading repeated regeneration for linear space, and it is only sensible because regeneration is possible at all. 14. Compressed and succinct representations At web scale, even CSR is too large, and two distinct research lines attack that. Exploiting structure. The WebGraph framework of Boldi and Vigna is the standard reference here. It observes that if you order web pages by URL, pages from the same site end up with nearly identical outlink sets, and their target lists are close together numerically. Encoding each list as a reference to a similar earlier list plus a small correction, then gap-encoding the remaining targets with a variable-length code, brings the web graph down to a few bits per link, an order of magnitude better than raw 32-bit identifiers. The technique is entirely dependent on a good vertex ordering, which is the general lesson: compression on graphs is mostly a relabelling problem. Blandford, Blelloch and Kash proved a complementary result for separable graphs, which includes planar graphs and most meshes, showing that a separator-based ordering gives O(n)-bit representations that still answer an adjacency query in constant time. Succinct data structures. A different tradition asks for representations whose size approaches the information-theoretic minimum while still answering queries without decompression. Jacobson's 1989 work introduced the rank and select primitives that make this possible, and Munro and Raman extended it to trees and other structures. A rooted tree on n nodes needs about 2n bits rather than the n pointers a naive encoding spends, and navigation still runs in constant time. For a graph the general problem is harder, but the framing is the right one: the number of distinct labelled graphs on n vertices with m edges gives a lower bound of roughly m log(n2/m) bits, and how close a representation gets to that is a meaningful way to judge it. Neither line is something to reach for by default. Both cost query time, both cost implementation complexity, and both are worth it only when the graph genuinely does not fit. The practical intermediate step, and the one most people should try first, is simply renumbering the vertices so that neighbours have nearby identifiers. That alone improves cache behaviour on ordinary CSR measurably, and it costs one breadth-first traversal. 15. A decision procedure Collapsing everything above into something usable at a keyboard: Four questions settle almost every case. Density and mutability do most of the work. - Can you generate neighbours from a rule? If the graph is a state space, a grid, or anything procedurally defined, use an implicit representation and store only what you visit. - Is the graph dense, or is n small? Above about 3% density, or below roughly a thousand vertices, use a bit-packed adjacency matrix. You get constant-time adjacency and word-parallel set operations, and above the density crossover, less memory as well. Below it, at small n, the matrix is the larger structure and it simply does not matter. - Does your algorithm only sweep over edges? Kruskal, Bellman-Ford and anything streaming want an edge list. Do not build an index you will never query. - Does the graph change after loading? If not, build CSR. If it changes rarely, build CSR with an update buffer and rebuild periodically. If it changes constantly, use adjacency lists of dynamic arrays with swap-and-pop deletion. Then apply two corrections. If the graph is directed and you need predecessors, build the reversed structure too and pay the second copy. If adjacency testing genuinely dominates your workload rather than neighbour iteration, sort the neighbour arrays for binary search before you reach for hash sets. 16. Common mistakes - Assuming the adjacency list is always right. It is the right default, not a universal answer. On a 500-vertex dense graph, an adjacency matrix is smaller, simpler and faster. - Quoting O(n + m) for an algorithm running on a matrix. Traversal on an adjacency matrix is Θ(n2). The bound belongs to the representation, not to the pseudocode. - Forgetting the factor of two. An undirected graph stores 2m entries in any adjacency structure, because every edge appears in both endpoints' lists. Sizing an array at m is a buffer overflow, not a performance issue. - Using a 32-bit offsets array. CSR offsets index into an array of length 2m. Past about two billion directed arcs this overflows silently and corrupts every neighbour lookup. Targets can stay 32-bit as long as n is under four billion; offsets cannot. - Storing zero for a missing weighted edge. Zero is a valid weight. Use a sentinel of ∞, or store presence separately, or use a sparse structure where absence is simply an absent entry. - Building CSR by sorting. The counting-sort construction is O(n + m). Sorting the edge list first is O(m log m) and unnecessary, though it is a reasonable shortcut if you wanted sorted neighbour arrays anyway. - Not storing the reverse graph. Predecessor queries on a directed CSR require a full scan of the targets array. If you do them at all, build the transposed structure once. - Choosing a representation from asymptotics alone. An adjacency list and CSR are both Θ(n + m), and CSR is routinely several times faster because of locality. Constant factors on memory-bound graph code are not a rounding error. - Mishandling loops in the degree count. A loop adds 2 to the degree in an undirected graph. Whatever you decide the adjacency structure should store, make the degree function agree with it. 17. Glossary TermMeaning Edge listAn unindexed array of m endpoint pairs. Optimal for edge iteration, O(m) for everything else Adjacency matrixAn n × n array of 0/1 cells. Θ(n2) space, O(1) adjacency test, Θ(n) neighbour scan Adjacency listPer-vertex neighbour sequences. Θ(n + m) space, Θ(deg) neighbour scan CSR / forward starA flattened adjacency list: an offsets array of length n + 1 and a targets array of length 2m CSCThe same structure built on the transposed graph, giving predecessors instead of successors Incidence matrixAn n × m vertex-by-edge array. Θ(nm) space; the algebraic bridge, not a storage choice Densitym / C(n,2), the fraction of possible edges present. The crossover for matrix storage is near 3% LaplacianL = D - A. Rows sum to zero; any cofactor counts spanning trees; eigenvalues describe connectivity Implicit graphA successor function in place of stored edges. Space becomes proportional to what is visited Semiring formulationGraph algorithms written as matrix products with the arithmetic replaced, for example (min, +) for shortest paths 18. Frequently asked questions Which graph representation should I use by default? + An adjacency list, or compressed sparse row if the graph does not change after loading. Real graphs are sparse, typically far below 1% density, and both structures use space proportional to n plus m rather than n squared. Switch to an adjacency matrix only when density exceeds roughly 3% or n is under about a thousand. Why is BFS slower on an adjacency matrix? + Because finding one vertex's neighbours means scanning a whole matrix row of length n, most of which is zeros. Over all n vertices that is n squared cell reads, whereas an adjacency list touches only the 2m real entries. For a graph with a million vertices and fifty million edges the ratio is about 19,600 to one. What is the difference between an adjacency list and compressed sparse row? + They store the same information with the same asymptotic costs. CSR concatenates every neighbour sequence into one flat array and keeps a second array of starting offsets, so it makes two allocations instead of n plus one, keeps each vertex's neighbours contiguous in memory, and can be memory-mapped or copied to a GPU directly. The tradeoff is that CSR cannot be updated in place; adding an edge means rebuilding it. When is an adjacency matrix actually the better choice? + Four cases. When n is small enough that n squared is trivial, which is the Floyd-Warshall regime. When the graph is dense enough that a bit-packed matrix is genuinely smaller, above about 3% density. When you need word-parallel set operations on neighbourhoods, as in triangle counting or clique search. And when you want to reduce a graph problem to fast matrix multiplication, as Seidel's all-pairs shortest-path algorithm does. How much memory does each representation really need? + For a graph with a million vertices and fifty million edges: a byte-per-cell adjacency matrix needs 931 GiB, a bit-packed one 116 GiB, a vector-of-vectors adjacency list about 420 MiB, and compressed sparse row about 389 MiB. The sparse structures are roughly 300 times smaller than even the bit-packed matrix, which is the difference between a program that runs and one that cannot start. Do I need to store the graph at all? + Not if the neighbours of a vertex can be computed from a rule. Grids, puzzle state spaces and the reachable states of a program are all defined by a successor function, and search algorithms only ever need the neighbours of the vertex they are currently at. Space then scales with what you visit rather than with the size of the graph, which is the only reason searching a space of 4.3 times 10 to the 19 Rubik's cube states is possible. How do I represent a directed graph's predecessors? + Build a second structure on the reversed graph, which sparse linear algebra calls compressed sparse column. There is no way to get predecessors cheaply from a successor-indexed structure other than a full scan. Any algorithm that walks backwards, including reverse reachability, Kosaraju's strongly connected components procedure and bidirectional search, needs that second copy, so budget for double the memory. 19. References Sources for the definitions, complexity bounds and techniques above, together with the standard texts in which this material is developed, listed in chronological order. - Arlazarov, V. L., Dinic, E. A., Kronrod, M. A. and Faradzev, I. A. (1970). “On economical construction of the transitive closure of a directed graph.” Soviet Mathematics Doklady, 11, 1209–1210. - Gustavson, F. G. (1972). “Some basic techniques for solving sparse systems of linear equations.” In Sparse Matrices and Their Applications, Plenum Press, 41–52. - Tarjan, R. E. (1972). “Depth-first search and linear graph algorithms.” SIAM Journal on Computing, 1(2), 146–160. - Hopcroft, J. and Tarjan, R. E. (1973). “Algorithm 447: efficient algorithms for graph manipulation.” Communications of the ACM, 16(6), 372–378. - Aho, A. V., Hopcroft, J. E. and Ullman, J. D. (1974). The Design and Analysis of Computer Algorithms. Addison-Wesley. - Duff, I. S., Erisman, A. M. and Reid, J. K. (1986). Direct Methods for Sparse Matrices. Oxford University Press. - Jacobson, G. (1989). “Space-efficient static trees and graphs.” Proceedings of the 30th Annual Symposium on Foundations of Computer Science (FOCS), 549–554. - Seidel, R. (1995). “On the all-pairs-shortest-path problem in unweighted undirected graphs.” Journal of Computer and System Sciences, 51(3), 400–403. - Chung, F. R. K. (1997). Spectral Graph Theory. CBMS Regional Conference Series in Mathematics 92, American Mathematical Society. - Munro, J. I. and Raman, V. (2001). “Succinct representation of balanced parentheses and static trees.” SIAM Journal on Computing, 31(3), 762–776. - Blandford, D. K., Blelloch, G. E. and Kash, I. A. (2003). “Compact representations of separable graphs.” Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 679–688. - Boldi, P. and Vigna, S. (2004). “The WebGraph framework I: compression techniques.” Proceedings of the 13th International World Wide Web Conference (WWW), 595–602. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Chapter 22. MIT Press. - Kepner, J. and Gilbert, J., editors (2011). Graph Algorithms in the Language of Linear Algebra. Society for Industrial and Applied Mathematics. - Diestel, R. (2017). Graph Theory, 5th edition. Springer, Graduate Texts in Mathematics 173. - Alman, J. and Vassilevska Williams, V. (2021). “A refined laser method and faster matrix multiplication.” Proceedings of the 32nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 522–539. --- ## BFS Interview Questions Source: https://learngraphtheory.org/articles/bfs-interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What a BFS question is actually testing - 2. The template to write from memory - 3. Shortest path in an unweighted graph - 4. Number of islands - 5. Rotting oranges: multi-source BFS - 6. Word ladder: implicit graphs and meeting in the middle - 7. Level order traversal of a binary tree - 8. 0-1 BFS: when BFS beats Dijkstra - 9. Is the graph bipartite? - 10. Course schedule: BFS topological sort - 11. The complexity answers interviewers expect - 12. Mistakes that fail the interview - 13. Frequently asked questions - 14. References 1. What a BFS question is actually testing Almost nobody is asked to "implement BFS". You get a problem that does not look like a graph, and the interview asks three things: can you see the graph, do you know BFS is the tool, and can you write it without a bug. The signal is the word fewest, or any synonym: minimum steps, shortest transformation, earliest minute, closest exit. BFS answers those, and only when every step costs the same. That clause is the whole game: when steps cost the same BFS gives the exact minimum in O(V + E); when they do not it is simply wrong, and reaching for it is the mistake the question was built around. The eight below are the ones that recur. Each is presented the way it goes: the problem, the solution, the follow-up the interviewer asks next, and the error that loses the offer. Every worked example was executed by script. 2. The template to write from memory One template covers every question here. You should be able to produce it in two minutes without thinking, because interview time belongs to the modelling, not the typing. from collections import deque def bfs(start, neighbours): dist = {start: 0} q = deque([start]) while q: u = q.popleft() for v in neighbours(u): if v not in dist: # mark ON ENQUEUE, never on dequeue dist[v] = dist[u] + 1 q.append(v) return dist Four details separate a clean pass from a shaky one. - Mark visited when you enqueue, not when you dequeue. Marking on dequeue lets a vertex be pushed once per incoming frontier edge, blowing the queue up to O(E). Distances stay correct, so it passes the tests and fails the review. - Use a real queue. deque.popleft() is O(1); list.pop(0) is O(n) and silently turns a linear algorithm quadratic. - The distance map doubles as the visited set. Two structures where one will do is two chances to forget an update. - neighbours is a function, not a data structure. That is what lets the same eight lines solve a grid, a word puzzle and a state space unmodified. Most interview graphs are never materialised at all, a point developed in graph representation. BFS visits in layers. The queue holds at most two consecutive layers at any moment, which is where its memory cost comes from. The property that makes this work is the layer invariant: every edge joins vertices in the same layer or in consecutive ones, never skipping. All eight edges above satisfy it, and it is why the first time BFS reaches a vertex is along a shortest path. Say it out loud: "BFS finds the shortest path" without a reason sounds memorised. 3. Shortest path in an unweighted graph The question. Given an unweighted graph and two vertices, return the shortest path length and the path itself. The base case. The only addition to the template is a parent pointer. def shortest_path(adj, src, dst): dist, parent = {src: 0}, {src: None} q = deque([src]) while q: u = q.popleft() if u == dst: # early exit: stop when POPPED break for v in adj[u]: if v not in dist: dist[v] = dist[u] + 1 parent[v] = u q.append(v) if dst not in dist: return None path, cur = [], dst while cur is not None: path.append(cur) cur = parent[cur] return dist[dst], path[::-1] On the graph in the figure this returns distance 4 and the path 0 → 1 → 3 → 5 → 6. Say unprompted that it is a shortest path, not the one: 0 → 2 → 3 → 5 → 6 is equally short, and which you get depends on adjacency order. The follow-up: can you exit early? Yes, and the subtlety is where. Testing for the target when you pop is always correct. Testing when you push also works for plain BFS and saves a layer, but stops being correct once weights appear, so the pop-time check is the habit worth having. The worst case is unchanged at O(V + E). The trap. When the interviewer says "now the edges have weights", do not patch BFS. Switch to Dijkstra's algorithm, or to the deque trick in section 8 if the weights are only 0 and 1. Candidates who make BFS re-visit vertices to cope with weights are writing a slow, buggy Bellman-Ford by accident. 4. Number of islands The question. Given a grid of '1' (land) and '0' (water), count the connected groups of land. Diagonals do not connect. There is no explicit graph, which is the point. Vertices are land cells, edges are shared sides, so a cell has at most four neighbours and you never build an adjacency structure. def num_islands(grid): if not grid: return 0 R, C = len(grid), len(grid[0]) seen, count = set(), 0 for i in range(R): for j in range(C): if grid[i][j] != '1' or (i, j) in seen: continue count += 1 seen.add((i, j)) q = deque([(i, j)]) while q: r, c = q.popleft() for dr, dc in ((1,0), (-1,0), (0,1), (0,-1)): a, b = r + dr, c + dc if 0 <= a < R and 0 <= b < C \ and grid[a][b] == '1' and (a, b) not in seen: seen.add((a, b)) q.append((a, b)) return count Every cell is enqueued at most once and each does constant work, so this is O(R × C) time. Space is the visited set plus the queue, also O(R × C) in the worst case, when the grid is entirely land. The follow-up: BFS or DFS? Either works, because you are labelling components rather than measuring distance. Prefer BFS for a practical reason: recursive DFS on a 106-cell grid of solid land recurses a million deep and overflows the stack. If you pick DFS, say you would write it iteratively; that sentence is often the whole point of the follow-up. The comparison is in BFS vs DFS. The trap. Mutating the input grid, writing '0' over land instead of keeping a visited set, is a legitimate optimisation, but say that you are doing it. Silently destroying the caller's data is a code-review failure, not a cleverness. BFS Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Rotting oranges: multi-source BFS The question. A grid holds empty cells (0), fresh oranges (1) and rotten ones (2). Each minute, every rotten orange rots the fresh oranges orthogonally adjacent to it. Return the number of minutes until none are fresh, or -1 if that never happens. This separates people who memorised BFS from people who understand it. The instinct is to run a BFS from each rotten orange and combine the results, which is complicated and slow. The answer is to put every rotten orange in the queue before the loop starts. BFS then expands one shared wavefront, and each cell is reached first by whichever source is nearest. def oranges_rotting(grid): R, C = len(grid), len(grid[0]) q, fresh = deque(), 0 for i in range(R): for j in range(C): if grid[i][j] == 2: q.append((i, j, 0)) elif grid[i][j] == 1: fresh += 1 minutes = 0 while q: r, c, t = q.popleft() minutes = max(minutes, t) for dr, dc in ((1,0), (-1,0), (0,1), (0,-1)): a, b = r + dr, c + dc if 0 <= a < R and 0 <= b < C and grid[a][b] == 1: grid[a][b] = 2 # mark on enqueue fresh -= 1 q.append((a, b, t + 1)) return -1 if fresh else minutes Worked on this grid: 2 1 1 0 minute each cell rots: 0 1 2 . 1 1 0 2 1 2 . 0 0 1 1 1 . 3 2 1 two sources, 7 fresh oranges, 0 remaining, answer = 3 Two sources, one wavefront. Each cell is claimed by whichever rotten orange reaches it first, and the two fronts meet in the bottom row at minute 3. Complexity is O(R × C), unchanged by the number of sources. That multi-source BFS costs the same as single-source BFS is the insight being tested. The follow-up: what if an orange can never rot? That is the -1 case, and why the fresh counter exists. Do not detect it by comparing visited cells against the grid size: empty cells are not oranges and the arithmetic goes wrong. Count fresh up front, decrement on each rot, check what remains. Empty the two cells beside the bottom-left orange and it is sealed off, so one stays fresh and the answer is -1. The trap. The empty grid. Zero fresh and zero rotten should return 0, and an off-by-one returning 1 is the most common wrong submission. 6. Word ladder: implicit graphs and meeting in the middle The question. Given a start word, an end word and a dictionary, find the length of the shortest chain where each step changes exactly one letter and every intermediate word is in the dictionary. The graph has one vertex per dictionary word and an edge between words differing in one position. Building it explicitly costs O(N2 L) time, which is the slow solution most candidates write first. The fast one never builds it: generate neighbours on demand by trying all 26 letters at each of the L positions and testing against a hash set, exactly as the code below does. One substitution per position regenerates the word itself and the visited check discards it. For ten-letter words that is 260 lookups per vertex, independent of dictionary size. def ladder_length(begin, end, word_list): words = set(word_list) if end not in words: return 0 q, dist = deque([begin]), {begin: 1} while q: w = q.popleft() if w == end: return dist[w] for i in range(len(w)): for ch in "abcdefghijklmnopqrstuvwxyz": nxt = w[:i] + ch + w[i+1:] if nxt in words and nxt not in dist: dist[nxt] = dist[w] + 1 q.append(nxt) return 0 The follow-up: make it faster. The expected answer is bidirectional BFS, introduced by Pohl in 1971: search forward from the start and backward from the end at once, always expanding the smaller frontier, and stop when they meet. A one-directional search to depth d with branching factor b touches about bd vertices; two searches of depth d/2 touch 2bd/2. That is a halving of the exponent, not a constant factor. With branching factor 10 and depth 6, meeting in the middle turns 1.1 million visited vertices into about 2,200. The deeper the answer lies, the more this buys. The trap. Bidirectional BFS needs predecessors as cheaply as successors: free here, since the relation is symmetric, but a reversed copy on a directed graph. The meeting point also needs care. The answer is the sum of the two depths, and stopping the instant a vertex appears in both visited sets is only valid when you expand a whole layer at a time. 7. Level order traversal of a binary tree The question. Return the values of a binary tree grouped by depth, one list per level. The only new idea is processing a whole layer at a time, and the trick is to record the queue's length before the inner loop. def level_order(root): if not root: return [] out, q = [], deque([root]) while q: level = [] for _ in range(len(q)): # snapshot the layer size FIRST node = q.popleft() level.append(node.val) if node.left: q.append(node.left) if node.right: q.append(node.right) out.append(level) return out Capturing len(q) in the range call is what makes this work: the loop runs exactly as many times as there were nodes on the level, even though the queue grows during it. Reading the length inside the loop silently merges levels, and is the classic bug here. A tree needs no visited set: no cycles, one parent per node. Say that you are dropping it because the input is a tree, since dropping it silently on a graph is a hang. The follow-ups. Zigzag order reverses level on odd depths, rather than queueing backwards. Right side view is the last element of each level. Minimum depth is the depth of the first leaf popped, and here BFS genuinely beats DFS, which must explore the whole tree. 8. 0-1 BFS: when BFS beats Dijkstra The question. Every edge has weight 0 or 1; find the shortest distance from a source. Variants appear as a grid where some moves are free, or as "minimum walls to break". Dijkstra solves this in O(E log V) and is accepted. The answer being fished for runs in O(V + E): use a double-ended queue, pushing a relaxed vertex to the front across a 0-weight edge and the back across a 1-weight edge. The deque then holds at most two distinct distance values at once, which is exactly the ordering a priority queue was providing. def zero_one_bfs(adj, src, n): # adj[u] = [(v, w), ...] with w in {0, 1} dist = [float('inf')] * n dist[src] = 0 dq = deque([src]) while dq: u = dq.popleft() for v, w in adj[u]: if dist[u] + w < dist[v]: dist[v] = dist[u] + w if w == 0: dq.appendleft(v) else: dq.append(v) return dist On a graph with edges 0-1 (weight 1), 0-2 (0), 2-3 (1), 1-3 (0), 3-4 (1) and 2-4 (1), this returns 0, 1, 0, 1, 1, matching Dijkstra exactly. Plain BFS returns 0, 1, 1, 2, 2, wrong for three of the five vertices, because it counts edges instead of summing weights. That contrast is the cleanest way to show what BFS actually optimises. The technique belongs to the label-correcting family, whose general form Bertsekas set out in 1993. One structural difference from ordinary BFS matters: a vertex can be relaxed more than once, so the guard is a distance comparison, not a visited check. The trap. Writing if v not in visited instead of if dist[u] + w < dist[v]. The visited check lets the first arrival win, and across a 0-weight edge the first arrival need not be the best. The code still runs and returns plausible numbers. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 9. Is the graph bipartite? The question. Can the vertices be split into two sets so that every edge crosses between them? Phrased in interviews as "split these people so no two enemies share a group", or "is this graph 2-colourable". Colour the source 0, colour every neighbour with the opposite colour, and fail if you ever meet a neighbour already carrying your own colour. def is_bipartite(adj, n): colour = [-1] * n for s in range(n): if colour[s] != -1: continue # a new component colour[s] = 0 q = deque([s]) while q: u = q.popleft() for v in adj[u]: if colour[v] == -1: colour[v] = 1 - colour[u] q.append(v) elif colour[v] == colour[u]: return False return True The clean explanation: the colour is the parity of the BFS layer. A conflict means an edge joins two vertices in the same layer, closing a cycle of odd length, and a graph is bipartite exactly when it has no odd cycle. The 4-cycle is bipartite, the 5-cycle is not, and the algorithm confirms both. The trap, and it fails more submissions than any other: the outer for s in range(n) loop. A disconnected graph needs BFS restarted from every uncoloured vertex, so a solution starting only at vertex 0 passes every connected test and fails the moment there are two components. Counting components and detecting cycles need the same loop. 10. Course schedule: BFS topological sort The question. Given n courses and a list of prerequisite pairs, can every course be taken? The follow-up asks for a valid order. This is cycle detection on a directed graph, and the BFS answer is Kahn's algorithm (1962): repeatedly take a vertex with no remaining prerequisites, remove it, decrement its successors. def find_order(n, prerequisites): adj = [[] for _ in range(n)] indeg = [0] * n for course, prereq in prerequisites: adj[prereq].append(course) indeg[course] += 1 q = deque(i for i in range(n) if indeg[i] == 0) order = [] while q: u = q.popleft() order.append(u) for v in adj[u]: indeg[v] -= 1 if indeg[v] == 0: q.append(v) return order if len(order) == n else [] # short == cycle With 6 courses and prerequisites 1←0, 2←0, 3←1, 3←2, 4←3, 5←4 this schedules all six as 0, 1, 2, 3, 4, 5. With the cyclic set 1←0, 2←1, 0←2 it schedules none: every vertex starts with in-degree 1, so the initial queue is empty. One test covers both cases, and it is the heart of the answer: if the output is shorter than n, the leftover vertices form a cycle. The trap. Getting the edge direction backwards. The pair [a, b] means "to take a, first take b", so the edge runs b → a and it is a's in-degree that rises. Reverse it and you get a valid topological order of the reversed graph: it looks right, passes the cycle check, and is wrong. Read the direction out loud before writing the loop. Broader treatment in topological sorting. The depth-first counterpart, covering cycle detection, topological sort, strongly connected components and bridges, is in DFS interview questions. 11. The complexity answers interviewers expect Half of a BFS interview is analysis. What to say, and why: Problem shapeTimeSpaceThe reason to give Graph, adjacency listO(V + E)O(V)Each vertex is enqueued once, each edge examined twice Graph, adjacency matrixO(V2)O(V)Finding one vertex's neighbours scans a whole row Grid, R × CO(R × C)O(R × C)V = RC and E < 2RC, so V + E is linear in cells Multi-source gridO(R × C)O(R × C)Unchanged: sources only seed the same single wavefront Word ladder, N words of length LO(N × L2 × 26)O(N × L)26L candidates per word, each O(L) to build and hash Bidirectional, branching b, depth dO(bd/2)O(bd/2)Two half-depth searches, so the exponent halves 0-1 BFSO(V + E)O(V)A deque replaces the heap, so no log factor Two points reward volunteering. The O(V) space is not incidental: BFS holds a whole layer, which on a wide graph is most of the vertex set. That is the real reason to prefer DFS on deep, narrow graphs, and a better answer than "DFS uses less memory", which is not always true. And the edge term is E directed but 2E undirected. Cormen, Leiserson, Rivest and Stein give the full analysis; Sedgewick and Wayne the clearest short one. BFS was published twice before it had a name: Moore in 1959 for the shortest path through a maze, Lee in 1961 for routing circuit boards. Lee's version is literally the grid BFS of sections 4 and 5, which is why grid pathfinding is still sometimes called Lee's algorithm. Once the edges carry different weights the queue becomes a priority queue; those problems are worked through in Dijkstra interview questions. 12. Mistakes that fail the interview Ordered by frequency, not severity. The first three account for most rejected solutions. - Marking visited on dequeue instead of enqueue. Distances stay correct so tests pass, but the queue grows to O(E), which on a dense graph is the difference between passing and timing out. - Using a list as a queue. list.pop(0) and JavaScript's shift() are O(n). Use collections.deque, or an index pointer into an array in a language without one. - Forgetting the outer loop over components. Bipartite checking, component counting and cycle detection all restart BFS from every unvisited vertex. Starting only at vertex 0 passes every connected test and fails the first disconnected one. - Reaching for BFS on a weighted graph. BFS minimises the number of edges, not total weight. If two edges differ in cost you need Dijkstra, or 0-1 BFS when the weights are only 0 and 1. - Reading the queue length inside the layer loop. In level order traversal, for _ in range(len(q)) must snapshot the length before the loop, or levels merge. - Not asking about the input. Directed? Connected? Self-loops or parallel edges? Are grid diagonals neighbours? Can the start equal the goal? Each changes the code, and Skiena's point holds: problems are won in the modelling, not the traversal. - Announcing O(V + E) without saying which representation. The bound belongs to the adjacency list. On a matrix the same code is O(V2). One habit beats all of the above. Before writing code, say out loud what the vertices are, what the edges are, and what a step costs. If every step costs the same, BFS is correct; if not, you have avoided the trap. McDowell makes the same point generally, and it bites hardest on graphs, where the graph is so often hidden. 13. Frequently asked questions How do I know a problem wants BFS and not DFS? + Look for the word fewest, or a synonym: minimum steps, shortest transformation, earliest minute, nearest exit. BFS answers those exactly, provided every step costs the same. If the question only asks about reachability or connected components, either traversal works, and BFS avoids deep recursion on large inputs. Why must I mark a vertex visited when I enqueue it? + Because between being enqueued and dequeued, a vertex can be discovered again by others in the same frontier. Marking on dequeue lets it be pushed once per incoming edge, so the queue holds O(E) entries instead of O(V). The distances still come out right, which is what makes the bug easy to miss. What is multi-source BFS and when do I need it? + You push every source into the queue before the loop starts, all at distance zero. BFS expands one shared wavefront, so each cell is reached first by whichever source is nearest. It costs the same as single-source BFS, O(V + E). Rotting oranges and nearest-exit problems are the standard examples. Can BFS ever handle weighted edges? + Only when every weight is 0 or 1. Then a double-ended queue, pushing to the front across a zero-weight edge and the back across a one-weight edge, gives the right answer in O(V + E) with no logarithmic factor. For any other weights BFS is simply wrong, because it minimises the number of edges rather than the total weight, and you need Dijkstra. How much faster is bidirectional BFS? + It halves the exponent rather than dividing by a constant, turning roughly b to the power d into 2 times b to the power d over 2. At branching factor 10 and depth 6 that is 1,111,111 vertices against about 2,222, a factor of 500. It needs predecessors to be as cheap as successors: free undirected, a reversed copy directed. What complexity should I state for a grid BFS? + O(R times C) for both time and space. The reason to give is that the grid is a graph with R times C vertices and fewer than 2 R C edges, so V plus E is linear in the cell count. The space is the visited set plus the queue, which can hold a large fraction of the grid at once. Do I need a visited set when running BFS on a tree? + No. A tree has no cycles and every node has one parent, so no node is reached twice and the set would never reject anything. Say why you are omitting it rather than omitting it silently: the same omission on a general graph is an infinite loop, and the interviewer cannot tell which you meant. 14. References The papers that introduced these techniques and the texts that analyse them, in chronological order. - Moore, E. F. (1959). “The shortest path through a maze.” Proceedings of an International Symposium on the Theory of Switching, Harvard University Press, 285–292. - Lee, C. Y. (1961). “An algorithm for path connections and its applications.” IRE Transactions on Electronic Computers, EC-10(3), 346–365. - Kahn, A. B. (1962). “Topological sorting of large networks.” Communications of the ACM, 5(11), 558–562. - Pohl, I. (1971). “Bi-directional search.” In Machine Intelligence 6, Edinburgh University Press, 127–140. - Bertsekas, D. P. (1993). “A simple and fast label correcting algorithm for shortest paths.” Networks, 23(8), 703–709. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Section 22.2. MIT Press. - Sedgewick, R. and Wayne, K. (2011). Algorithms, 4th edition, Section 4.1. Addison-Wesley. - McDowell, G. L. (2015). Cracking the Coding Interview, 6th edition. CareerCup. - Skiena, S. S. (2020). The Algorithm Design Manual, 3rd edition, Chapter 5. Springer. --- ## DFS Interview Questions Source: https://learngraphtheory.org/articles/dfs-interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What a DFS question is actually testing - 2. The template, recursive and iterative - 3. Cycle detection in a directed graph - 4. Cycle detection in an undirected graph - 5. Topological sort by post-order - 6. Clone graph - 7. All paths: DFS as backtracking - 8. Word search on a grid - 9. Strongly connected components - 10. Bridges and articulation points - 11. Complexity, and the recursion depth question - 12. Mistakes that fail the interview - 13. Frequently asked questions - 14. References 1. What a DFS question is actually testing DFS questions are not about traversal. Any candidate can walk a graph. What is being checked is whether you know the bookkeeping DFS makes available and BFS does not: the order things finish in, whether a vertex is still on the stack, and how far back a subtree can reach. That is the whole subject. Cycle detection, topological sort, strongly connected components, bridges and articulation points are all one traversal plus one extra array. If a question asks about ordering, dependencies, cycles, or what breaks if this is removed, it is a DFS question. If it asks for the fewest of anything, it is a BFS question. The eight below are the ones that recur, each with the problem, the solution, the follow-up, and the error that loses the offer. Every worked example was executed by script. 2. The template, recursive and iterative Recursive DFS is four lines, and you should be able to write it without thinking. def dfs(u, adj, seen): seen.add(u) for v in adj[u]: if v not in seen: dfs(v, adj, seen) The iterative version is where candidates slip, because the obvious translation is subtly different from the recursive one. def dfs_iter(src, adj): seen, stack = set(), [src] while stack: u = stack.pop() if u in seen: # a vertex can be pushed several times continue seen.add(u) for v in adj[u]: if v not in seen: stack.append(v) Two things to notice, and to say out loud. - Check seen on pop, not only on push. Unlike BFS, the same vertex can sit on the stack more than once, pushed by several neighbours before any is expanded. Skip the pop-time check and you get duplicate visits. - The visit order differs from the recursive version. Neighbours pushed in ascending order pop in descending order, so iterative DFS explores the last neighbour first. Push them reversed to match. A favourite gotcha. The harder point: the plain iterative version has no post-order. It knows when a vertex is discovered, never when its subtree finishes, and finish time is exactly what sections 5, 9 and 10 need. To recover it, push each vertex twice or carry a child index in the frame. Knowing recursion is not merely cosmetic here is worth saying. The technique is old: it is Trémaux's rule for threading a maze, recorded by Lucas in 1882. 3. Cycle detection in a directed graph The question. Does a directed graph contain a cycle? Phrased as deadlock detection, build dependency loops, or "can this course schedule be completed". The wrong answer is a single visited set: reaching a seen vertex does not mean a cycle, it may be a second route into a finished part of the graph. The right answer uses three colours: white undiscovered, grey discovered but still on the recursion stack, black finished. WHITE, GREY, BLACK = 0, 1, 2 def has_cycle(adj, n): colour = [WHITE] * n def visit(u): colour[u] = GREY for v in adj[u]: if colour[v] == GREY: # back edge: v is an ancestor return True if colour[v] == WHITE and visit(v): return True colour[u] = BLACK # only now is u finished return False return any(colour[s] == WHITE and visit(s) for s in range(n)) An edge into a grey vertex is a back edge, and a directed graph has a cycle if and only if a DFS finds a back edge. An edge into a black vertex is harmless. That equivalence, and the four-way classification of edges it belongs to, is the standard treatment in Cormen, Leiserson, Rivest and Stein. The running graph. Four edge types, no back edge, so no cycle. Add one arc and the back edge appears, with the cycle readable straight off the tree path. On the running graph, DFS from vertex 0 classifies its 8 arcs as 5 tree edges, 1 forward edge and 2 cross edges, with no back edge, so it is acyclic. Add the single arc 5 → 0 and exactly one back edge appears. The follow-up: print the cycle, not just a boolean. The back edge gives it to you: if it is u → v, walk parent pointers from u up to v and close the loop. Here the back edge is 5 → 0 and the cycle is 0 → 1 → 3 → 5 → 0. A parent array costs one line and turns a yes/no into the diagnosis a real build tool has to report. The trap. Setting colour[u] = BLACK in the wrong place, or not at all. Leave finished vertices grey and every second route into them looks like a cycle, so you report false positives on any DAG containing a diamond, as the running graph does. 4. Cycle detection in an undirected graph The question. Same problem, undirected graph. It looks like the previous question and it is not. Three colours are wrong here. Every undirected edge is traversable both ways, so stepping from u to v, the edge back to u looks like an edge into a grey vertex, and every edge reports a cycle. Instead, ignore the edge you arrived on. def has_cycle_undirected(adj, n): seen = [False] * n def visit(u, parent): seen[u] = True for v in adj[u]: if not seen[v]: if visit(v, u): return True elif v != parent: # a seen, non-parent neighbour return True return False return any(not seen[s] and visit(s, -1) for s in range(n)) Without the v != parent guard, a graph consisting of the single edge 0-1 reports a cycle. With it, a 4-vertex tree correctly reports none and a triangle correctly reports one. Those are the three test cases to check on the whiteboard, and checking them unprompted reads very well. The follow-up: parallel edges? Then v != parent is not enough: two distinct edges between u and v genuinely form a cycle of length 2, and the parent check swallows the second. Track the edge you came in on, not the vertex. See simple graphs vs multigraphs. The trap. A disconnected graph. The outer loop over every unvisited vertex is not optional, and a solution that starts only at vertex 0 passes every connected test case. 5. Topological sort by post-order The question. Order the vertices of a DAG so every arc points forward. The BFS answer is Kahn's in-degree peeling; the DFS answer is shorter and is what a DFS question wants. Run DFS, append each vertex when it finishes, reverse the list. That is the whole algorithm, and the reason is one sentence: a vertex finishes only after everything reachable from it, so it finishes later than its successors, and reversing puts it before them. def topological_sort(adj, n): colour = [0] * n # 0 white, 1 grey, 2 black order = [] def visit(u): colour[u] = 1 for v in adj[u]: if colour[v] == 1: raise ValueError("cycle") if colour[v] == 0: visit(v) colour[u] = 2 order.append(u) # post-order: AFTER the children for s in range(n): if colour[s] == 0: visit(s) return order[::-1] Finish order reversed is a topological order: no arc points backwards in it. On the running DAG the post-order is 5, 3, 1, 4, 2, 0; reversing gives 0, 2, 4, 1, 3, 5, and all eight arcs point forward in it. Say that it is a topological order, not the one: a DAG usually has many. The follow-up: how do you detect a cycle here? The grey check from section 3. That is the appeal: one traversal both orders the DAG and rejects a non-DAG, where Kahn's needs a separate count at the end. The DFS formulation is Tarjan's. The trap. Appending in pre-order, when the vertex is discovered rather than when it finishes. The result looks plausible, is wrong, and on small graphs often coincides with a valid order, so it survives casual testing. DFS Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 6. Clone graph The question. Given a reference to a node in a connected undirected graph, return a deep copy. The only difficulty is cycles: a naive recursive copy loops forever. The fix is a map from original node to its copy, which doubles as the visited set, and it must be written before recursing. def clone_graph(node, made=None): if node is None: return None if made is None: made = {} if node in made: return made[node] copy = Node(node.val) made[node] = copy # register BEFORE recursing for nb in node.neighbors: copy.neighbors.append(clone_graph(nb, made)) return copy Registering the copy before the recursive calls is the entire question. Do it after and a cycle sends you round again before the entry exists, recursing until the stack dies. It is the same shape as memoising any self-referential structure. The follow-up: iterative, or BFS? Both work, with an identical map. Say that the map, not the traversal order, is what makes it correct. O(V + E) either way. 7. All paths: DFS as backtracking The question. List every path from a source to a target in a DAG. Variants: all root-to-leaf paths, path sum, permutations. This is the family where DFS stops being a graph traversal and becomes backtracking, and the difference is one line: you undo your choice on the way out. def all_paths(adj, src, dst): out, path = [], [] def walk(u): path.append(u) if u == dst: out.append(path[:]) # COPY, not the live list else: for v in adj[u]: walk(v) path.pop() # the backtracking step walk(src) return out On the running DAG there are exactly 4 paths from 0 to 5: 0→1→3→5, 0→2→3→5, 0→2→4→5 and 0→3→5. Two details carry the answer. Append a copy, path[:], since path is mutated afterwards and the reference gives you a list of identical empty lists. And there is no visited set: you enumerate paths, not vertices, so a vertex legitimately appears in many paths. The path.pop() keeps state correct without one. The follow-up: complexity? Not O(V + E). A DAG can have exponentially many paths, so listing them is exponential in the output; the honest answer is O(V × 2V). Saying linear here reveals you have not thought about what the output is. Asked only how many paths exist, that is a different problem: count with dynamic programming over the topological order, in O(V + E). The trap. Adding a visited set because "DFS always has one". On a cyclic graph you do exclude vertices already on the current path, but that is the path, not a global set, and a global set silently returns a subset of the answers. 8. Word search on a grid The question. Given a grid of letters and a word, decide whether the word can be spelled by moving between orthogonally adjacent cells, using no cell twice. This is backtracking on an implicit grid graph, and the "no cell twice" clause is what forces the undo. def exist(board, word): R, C = len(board), len(board[0]) def walk(r, c, i): if i == len(word): return True if not (0 <= r < R and 0 <= c < C): return False if board[r][c] != word[i]: return False board[r][c] = '#' # mark, so the path cannot reuse it found = any(walk(r + dr, c + dc, i + 1) for dr, dc in ((1,0), (-1,0), (0,1), (0,-1))) board[r][c] = word[i] # UNDO on the way out return found return any(walk(r, c, 0) for r in range(R) for c in range(C)) The marker must be restored: a cell blocked by one failed attempt has to be available to a different start, and forgetting it gives a function that succeeds only when the first path tried happens to work. Overwriting the board instead of keeping a visited set is a legitimate trick, but say so, since it mutates the caller's input. The follow-up: complexity. O(R × C × 3L) for a word of length L: every cell is a possible start, and after the first step you never go back the way you came, so each later step has at most 3 choices, not 4. That 3 is the detail that signals you thought about it. 9. Strongly connected components The question. Partition a directed graph into maximal sets of mutually reachable vertices. It appears as "find circular dependencies", or as preprocessing before a DAG algorithm. Two DFS answers exist and you should know which you are writing. Kosaraju-Sharir is two passes and far easier to get right under pressure. DFS the graph recording finish order, then DFS the reversed graph taking vertices in decreasing finish order; each tree of the second pass is one component. def kosaraju(adj, radj, n): seen, order = [False] * n, [] def pass1(u): seen[u] = True for v in adj[u]: if not seen[v]: pass1(v) order.append(u) # finish order for s in range(n): if not seen[s]: pass1(s) comp, c = [-1] * n, 0 def pass2(u): comp[u] = c for v in radj[u]: if comp[v] == -1: pass2(v) for u in reversed(order): # decreasing finish time if comp[u] == -1: pass2(u); c += 1 return comp, c On a graph of two triangles, 0→1→2→0 and 3→4→5→3, joined by the single arc 2→3, this returns exactly two components, {0,1,2} and {3,4,5}. Reachability confirms it: 0 reaches 3, and 3 does not reach 0. Tarjan's algorithm does it in one pass with a stack and a low-link value: faster in practice, much easier to botch on a whiteboard. Both are O(V + E), and both have a visualizer here. Tarjan's 1972 paper gave the single-pass method; the two-pass version is credited to Kosaraju and was first published by Sharir in 1981. The follow-up: why does the reversed graph work? Reversing every arc leaves the components unchanged, since mutual reachability is symmetric. The vertex with the highest finish time lies in a source component of the condensation, and reversal turns a source into a sink, so a DFS started there cannot leave it. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 10. Bridges and articulation points The question. Which edges, if removed, disconnect the graph? Which vertices? Framed as single points of failure, or critical connections in a cluster. This is the deepest standard DFS question, and it is one idea: alongside each vertex's discovery time, track low[u], the smallest discovery time reachable from u's subtree using at most one non-tree edge. def bridges(adj, n): disc, low = [-1] * n, [-1] * n out, clock = [], 0 def visit(u, parent): nonlocal clock disc[u] = low[u] = clock; clock += 1 for v in adj[u]: if v == parent: parent = -2 # skip ONE copy of the parent edge continue if disc[v] == -1: visit(v, u) low[u] = min(low[u], low[v]) if low[v] > disc[u]: out.append((u, v)) # nothing below v reaches u or above else: low[u] = min(low[u], disc[v]) for s in range(n): if disc[s] == -1: visit(s, -1) return out Two triangles joined by one edge. Inside each triangle every vertex can reach back to the triangle's entry point, so low collapses; across the join it cannot, and that is the bridge. low[v] > disc[u] says the subtree below v has no way back to u or above, so u-v is the only route and removing it splits the graph. On the two-triangle graph the discovery times run 0 to 5 and the low values are 0, 0, 0, 3, 3, 3. The only bridge is 2-3; the articulation points are 2 and 3. Brute force agrees: removing that edge, or either vertex, leaves 2 components, and no other single removal disconnects anything. Articulation points use the same traversal, two rules: a non-root u qualifies if some child v has low[v] >= disc[u], and the root qualifies if it has more than one DFS child. Note >= against the > for bridges. That one character separates the two answers, and mixing them up is the commonest error here. The trap. The parent check. if v == parent: continue without the one-shot guard is wrong on a multigraph: two parallel edges to the parent mean the pair is not a bridge, and skipping both hides it. Track the edge index, or skip only the first occurrence as above. The algorithm is Hopcroft and Tarjan's, 1973, and there is a visualizer for it. 11. Complexity, and the recursion depth question Every algorithm above is one traversal, so the time bound barely moves. Space is where the interesting question lives. ProblemTimeSpaceThe reason to give DFS, adjacency listO(V + E)O(V)Each vertex visited once, each edge examined once per direction Directed cycle detectionO(V + E)O(V)One colour array over the same traversal Topological sortO(V + E)O(V)Post-order list plus the recursion stack Kosaraju-Sharir SCCO(V + E)O(V + E)Two traversals, and the reversed graph is a second copy Bridges, articulation pointsO(V + E)O(V)Two integer arrays, disc and low Word search, word length LO(R × C × 3L)O(L)Every cell a start; 3 onward choices after the first step All pathsO(V × 2V)O(V)The output itself can be exponential The recursion depth question is asked in almost every DFS interview, so have the answer ready. DFS recurses as deep as the longest path it follows, which on a path graph is V. CPython's default limit is 1000, so a few thousand vertices in a line crashes it, and a 1000 by 1000 grid of land cells can recurse a million deep. The fix is the iterative version from section 2, not sys.setrecursionlimit, which only turns a clean exception into a real stack overflow. Say that explicitly. The O(V) space is that stack, and the honest comparison with BFS is that DFS holds one root-to-leaf path while BFS holds a whole layer: neither is uniformly smaller, it depends on whether the graph is deep or wide. Aho, Hopcroft and Ullman give the aggregate analysis; Sedgewick and Wayne the shortest clear treatment. 12. Mistakes that fail the interview Ordered by frequency; the first three account for most rejected solutions. - One visited set for directed cycle detection. Seeing a vertex again is not a cycle. Grey (on the stack) must be distinct from black (finished), or every DAG with two routes to a vertex reports one. - Using three colours for undirected cycle detection. The reverse error. Every edge looks like a back edge unless you skip the one you arrived on. - Recursing on input that can be large. A million-cell grid recurses a million deep. Offer the iterative version before being asked. - Building the topological order in pre-order. It must be post-order, appended when the vertex finishes, then reversed. Pre-order produces something that looks like an answer and is not. - Forgetting to undo in backtracking. The path.pop(), or restoring the grid cell. Without it the first failed branch poisons every later one. - Storing the live list instead of a copy. out.append(path) gives a list of references to one mutated list. It must be path[:]. - Confusing > and >= in the low-link test. low[v] > disc[u] is a bridge, >= is an articulation point. One character apart. - Omitting the outer loop over components. Cycle detection, component counting and SCC all need DFS restarted from every unvisited vertex. - Not asking about the input. Directed or undirected? Connected? Self-loops or parallel edges? Each changes the code, and Skiena's point holds: problems are won in the modelling. The habit that prevents most of these: before writing, name the extra array. Colour, parent, finish order, or low-link. DFS questions differ from each other by that array, not by the traversal, and naming it first makes the rest mechanical. McDowell makes the same argument generally; it is unusually literal here. 13. Frequently asked questions When should I reach for DFS instead of BFS? + When the question is about ordering, dependencies, cycles, or what breaks if something is removed. All of those need to know when a vertex finishes, or whether it is still on the stack, and only DFS gives you that. If it asks for the fewest of anything, use BFS: DFS finds a path, not the shortest one. Why do I need three colours for directed cycle detection? + Because a plain visited set cannot tell an ancestor from a finished vertex. Grey means still on the recursion stack, so an edge into a grey vertex closes a loop and is a real cycle. Black means finished, and an edge into a black vertex is just a second route into an already-explored part of the graph, which is legal in a DAG. Why does reversing DFS post-order give a topological sort? + Because a vertex finishes only after everything reachable from it, so it always finishes later than its successors. Reversing finish order therefore places every vertex before everything it points to, which is the topological condition. Append in post-order, when the vertex finishes, not when it is discovered. What is a low-link value? + For a vertex u it is the smallest discovery time reachable from u's subtree using tree edges plus at most one non-tree edge. It answers "can anything below u get back above u without the edge to its parent". If not, that edge is a bridge. It is the one extra array that turns ordinary DFS into an algorithm for bridges, articulation points and Tarjan's strongly connected components. How deep can recursive DFS go before it breaks? + As deep as the longest path it follows, which on a path graph is the number of vertices. CPython's default limit is 1000, so a few thousand vertices in a line crashes it, and a 1000 by 1000 grid of land cells recurses a million deep. Rewrite it iteratively rather than raising the limit, which only turns a clean exception into a real stack overflow. Is iterative DFS the same as recursive DFS with a stack? + Not quite. The simple stack version visits neighbours in reverse order, so push them reversed to match, and it must check the visited set on pop as well as push, since a vertex can sit on the stack several times. More importantly it has no post-order, so topological sort, strongly connected components and low-link algorithms need a version that pushes each vertex twice or tracks a child index. Do I need a visited set when enumerating all paths? + No, and adding one is a common bug. You enumerate paths rather than vertices, so the same vertex legitimately appears in many paths, and a global visited set silently returns only some. What you need is the current path, undone with a pop on the way out. On a cyclic graph you exclude vertices already on that path, which is not a global set. 14. References The papers that introduced these techniques and the texts that analyse them, in chronological order. - Lucas, É. (1882). Récréations Mathématiques, Volume 1. Gauthier-Villars. (Records Trémaux's systematic maze-threading rule, the earliest description of depth-first search.) - Tarjan, R. E. (1972). “Depth-first search and linear graph algorithms.” SIAM Journal on Computing, 1(2), 146–160. - Hopcroft, J. and Tarjan, R. E. (1973). “Algorithm 447: efficient algorithms for graph manipulation.” Communications of the ACM, 16(6), 372–378. - Aho, A. V., Hopcroft, J. E. and Ullman, J. D. (1974). The Design and Analysis of Computer Algorithms. Addison-Wesley. - Tarjan, R. E. (1976). “Edge-disjoint spanning trees and depth-first search.” Acta Informatica, 6(2), 171–185. - Sharir, M. (1981). “A strong-connectivity algorithm and its applications in data flow analysis.” Computers & Mathematics with Applications, 7(1), 67–72. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Section 22.3. MIT Press. - Sedgewick, R. and Wayne, K. (2011). Algorithms, 4th edition, Sections 4.1–4.2. Addison-Wesley. - McDowell, G. L. (2015). Cracking the Coding Interview, 6th edition. CareerCup. - Skiena, S. S. (2020). The Algorithm Design Manual, 3rd edition, Chapter 5. Springer. --- ## Dijkstra Interview Questions Source: https://learngraphtheory.org/articles/dijkstra-interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What a Dijkstra question is actually testing - 2. The template, and lazy deletion - 3. Network delay time, and rebuilding the path - 4. Cheapest flights within K stops - 5. Path with minimum effort: replacing the plus - 6. Maximum-probability path - 7. Counting shortest paths - 8. The second shortest path - 9. Why non-negative weights are non-negotiable - 10. The complexity answers - 11. Mistakes that fail the interview - 12. Frequently asked questions - 13. References 1. What a Dijkstra question is actually testing Nobody is asked to recite Dijkstra's algorithm. What you get is a problem whose weights are not distances, and the interview is asking whether you can see that the shape of the algorithm still fits. That shape is: a priority queue ordered by a cost label, a relaxation rule that improves a neighbour's label, and a guarantee that once a vertex is popped its label is final. Change what "cost" means, change the comparison, and the same twelve lines solve minimum effort, maximum probability, cheapest flights and half a dozen other questions. The interview is testing whether you know which parts you are allowed to change and which part you are not. Dijkstra's original 1959 note is two pages long and the idea has not needed revising since. The eight problems below are the ones that recur, each with the solution, the follow-up, and the error that loses the offer. Every worked example was executed by script. 2. The template, and lazy deletion Write this without thinking. The comments mark the two lines that separate a correct implementation from a plausible one. import heapq def dijkstra(adj, src, n): # adj[u] = [(v, w), ...] with w >= 0 dist = [float('inf')] * n dist[src] = 0 heap = [(0, src)] while heap: d, u = heapq.heappop(heap) if d > dist[u]: # STALE entry: a better label was found continue # after this one was pushed. Skip it. for v, w in adj[u]: if d + w < dist[v]: dist[v] = d + w heapq.heappush(heap, (dist[v], v)) # push, never decrease-key return dist The if d > dist[u]: continue line is the whole answer to "how do you handle decrease-key". A binary heap has no efficient decrease-key, so instead of updating an entry you push a second one and ignore the obsolete pop. This is lazy deletion, and being able to name it is worth more than the code around it. The heap can therefore hold up to O(E) entries rather than O(V), which is why the bound is O((V + E) log V) and not O((V + E) log E): the logarithms differ by a constant factor since E < V2. Two more things to say aloud. A vertex is settled the moment it is popped with a current label, and its distance never changes afterwards; that is the invariant the greedy choice rests on. And you do not need a separate visited set, because the staleness check already rejects any second pop. 3. Network delay time, and rebuilding the path The question. Given a directed weighted graph and a source, how long until every vertex is reached? Return -1 if some vertex never is. Interview phrasings involve signal propagation, package delivery, or "when does the last server hear about it". This is plain Dijkstra plus one line: the answer is max(dist), and -1 when any entry is still infinite. The running graph. Vertex 1 is first labelled 4, then improved to 3 before it is settled, and four of the nine pushed entries are popped stale and skipped. On the running graph Dijkstra from 0 settles in the order 0, 2, 1, 3, 4, 5 and returns distances 0, 3, 1, 8, 10, 12. Watch vertex 1: the arc 0 → 1 labels it 4, then 0 → 2 → 1 improves it to 3 before it is ever popped. That is the algorithm working exactly as intended, and it is why you must not commit to a label at push time. The follow-up: return the path, not just the length. Keep a parent array, set parent[v] = u in the same branch that improves dist[v], then walk it backwards from the target and reverse. On this graph that gives 0 → 2 → 1 → 4 → 5, cost 12. Say "a shortest path" rather than "the": there are two of cost 12 here, as section 7 shows. The trap. Setting parent[v] outside the improvement branch, so it records the last vertex that tried rather than the one that succeeded. The distances stay right and the reconstructed path is wrong, which is the worst kind of bug to find in review. 4. Cheapest flights within K stops The question. Cheapest route from source to target using at most K intermediate stops. This is the question that catches people, because plain Dijkstra is wrong here. Its correctness rests on a vertex having one final label, but under a stop limit a vertex has a different best cost for each number of stops used, and a cheap route that burns too many hops can be worse than an expensive short one. Settling a vertex once throws away exactly the alternative you need. Two correct answers, and knowing both is the point. Widen the state. Keep Dijkstra but make the vertex a pair (node, stops_used). The label is now final per pair, so the invariant holds again. def cheapest(adj, src, dst, K, n): best = [[float('inf')] * (K + 2) for _ in range(n)] best[src][0] = 0 heap = [(0, src, 0)] # (cost, node, stops) while heap: c, u, k = heapq.heappop(heap) if u == dst: return c # first pop of dst is optimal if k > K or c > best[u][k]: continue for v, w in adj[u]: if c + w < best[v][k + 1]: best[v][k + 1] = c + w heapq.heappush(heap, (c + w, v, k + 1)) return -1 Or use Bellman-Ford, which is the cleaner answer. Relaxing every edge exactly K + 1 times, each round from a snapshot of the previous round, gives the cheapest route using at most K + 1 edges directly. That is Bellman's 1958 formulation, and it is O(K × E) with no heap at all. Offering it unprompted reads very well. The trap. The Bellman-Ford version must relax from a copy of the previous round's distances. Relaxing in place lets a single round propagate along several edges, which silently allows more than K stops and returns a too-cheap answer that looks plausible. Dijkstra Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Path with minimum effort: replacing the plus The question. Minimise the largest single edge on the route rather than the total. Phrasings: minimum effort path, swim in rising water, the maximum weight you must be able to carry. The insight is that Dijkstra never actually required addition. It requires that extending a path cannot improve its cost, so that a settled label stays final. max satisfies that just as well as +, so change one line: cand = max(d, w) # instead of d + w if cand < best[v]: best[v] = cand heapq.heappush(heap, (cand, v)) On the running graph the minimax values from vertex 0 are 0, 2, 1, 5, 5, 5, so the best bottleneck to vertex 5 is 5: brute-forcing every route confirms it. That it is a genuinely different objective is visible in the two cheapest routes, which both cost 12 but have largest arcs of 5 and 7. Minimising the total and minimising the largest arc are not the same question, and in general the bottleneck-optimal route need not be a shortest one at all. Same graph, same code, one changed line. Swapping + for max turns shortest path into widest path. The follow-up: what else can replace the plus? Any operation that is monotone, meaning extending a path never lowers its cost. max works, multiplication by probabilities in [0,1] works if you maximise, and ordinary addition of non-negative weights works. Subtraction does not, which is the same reason negative edges are forbidden. The trap. On a grid version, a union-find or binary-search-plus-BFS solution is also accepted and is sometimes faster. If you offer Dijkstra, be ready to say why you chose it: no parameter to binary search over, and one pass. 6. Maximum-probability path The question. Each edge has a success probability; find the route from source to target with the highest probability that every edge succeeds. Costs multiply rather than add, and you want the largest product, so invert the queue into a max-heap and relax with ×. Probabilities lie in [0,1], so extending a path can only shrink the product, which is exactly the monotonicity Dijkstra needs. cand = p * pw # instead of d + w if cand > best[v]: # > because we maximise best[v] = cand heapq.heappush(heap, (-cand, v)) # negate: heapq is a MIN-heap On a graph with a direct arc of 0.30 from 0 to 3, plus the two-arc route 0 → 1 → 3 at 0.9 and 0.8, the best probability is 0.72 along the two-arc route, beating the single arc. That is the sentence to say: more edges can be better here, which is never true for ordinary shortest paths with positive weights. The follow-up: why not take logarithms? You can, and it is a good answer. Since log(ab) = log a + log b, maximising a product of probabilities is minimising a sum of -log p, which are non-negative, so unmodified Dijkstra applies. Mention the caveat: floating-point log of a probability near zero loses precision, and an edge of probability 0 gives an infinity you must special-case. 7. Counting shortest paths The question. How many distinct shortest routes are there from source to target? Usually asked modulo 109+7. One extra array, and one extra branch. Alongside dist keep ways, the number of shortest routes to each vertex. When a relaxation improves a label, the count is replaced. When it ties, the count is added. if d + w < dist[v]: dist[v] = d + w ways[v] = ways[u] # strictly better: replace heapq.heappush(heap, (dist[v], v)) elif d + w == dist[v]: ways[v] = (ways[v] + ways[u]) % MOD # a tie: ADD On the running graph the counts are 1, 1, 1, 1, 2, 2. Brute force agrees: of the 9 routes from 0 to 5, two cost 12, namely 0→2→1→3→4→5 and 0→2→1→4→5. Note the shorter route in edges is not uniquely best, which is the sort of detail worth pointing at. The follow-up: is the tie branch safe? It is, but only because ways[u] is final when u is popped, and every relaxation happens from a popped vertex. Adding counts from a vertex that has not settled would double count. This is the clearest example of why "settled means final" is the invariant that matters, not the code. The trap. Forgetting that the elif must sit on the equality, not inside the improvement branch. Written as a single if d + w <= dist[v] the counts are replaced on ties instead of added, and the answer is 1 for every vertex. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 8. The second shortest path The question. Find the second shortest route from source to target. Clarify immediately whether "second" means strictly longer than the best, or simply the next route in a list where ties count separately. The two answers differ and interviewers ask this on purpose. The technique is to relax the settled-once rule: keep the two best labels per vertex and let a vertex be popped twice. best1 = [inf] * n; best2 = [inf] * n best1[src] = 0 heap = [(0, src)] while heap: d, u = heapq.heappop(heap) if d > best2[u]: continue # worse than both labels we keep for v, w in adj[u]: nd = d + w if nd < best1[v]: best1[v], nd = nd, best1[v] # demote the old best to candidate heapq.heappush(heap, (best1[v], v)) # the NEW best must propagate too if best1[v] < nd < best2[v]: # strictly worse than best best2[v] = nd heapq.heappush(heap, (nd, v)) On the running graph the distinct route costs from 0 to 5 are 12, 13, 14, 15, so the strictly-second best is 13. If instead ties count separately, the answer is 12 again, because two different routes achieve it. Ask before you code. The follow-up: generalise to K. Keep a list of the K best labels per vertex, or use Yen's algorithm for K shortest loopless paths, which is a genuinely different and much heavier problem. Saying "loopless changes it completely" is the right instinct: without that restriction a shortest walk can repeat a zero-weight cycle forever. 9. Why non-negative weights are non-negotiable Every interviewer asks this, and most candidates answer "because Dijkstra is greedy", which is true and explains nothing. The precise reason: the algorithm assumes that when a vertex is popped with the smallest label in the queue, no route still under construction can reach it more cheaply. A negative edge breaks that, because extending a path can reduce its cost. Have a concrete counterexample ready. Take four vertices with arcs 0→1 (1), 0→2 (2), 2→1 (−2) and 1→3 (1). One negative arc. The damage does not appear where the negative edge is, but one step downstream at vertex 3. Dijkstra returns 0, 0, 2, 2; the correct answer is 0, 0, 2, 1. The subtlety is worth stating precisely, because it is more interesting than the usual answer: vertex 1's label ends up correct. It is expanded while its label is still 1, and the later improvement to 0 does get written. But nothing re-relaxes 1 → 3 afterwards, so vertex 3 keeps 2 instead of 1. A candidate who says "the wrong value appears downstream of the negative edge, not at it" is clearly speaking from having tried it. The follow-up: what do you use instead? Bellman-Ford, which relaxes every edge V - 1 times in O(VE) and detects negative cycles on the V-th pass. If you need all pairs and have negative edges but no negative cycle, Johnson's algorithm reweights with one Bellman-Ford run so that every weight becomes non-negative, then runs Dijkstra from each vertex. Cormen, Leiserson, Rivest and Stein give the correctness proof for the greedy choice in full. The trap. "Just add a constant to every weight to make them positive." It does not work, and being able to say why in one sentence is a strong signal: adding c to every edge adds c × (number of edges) to a route, which penalises routes with more edges, so it changes which route is shortest. 10. The complexity answers Have the bound and the reason ready, and name the data structure. Saying "Dijkstra is O(E log V)" without naming the heap invites a follow-up you will then fail. Priority queueTimeThe reason to give Binary heap, lazy deletionO((V + E) log V)Up to E entries pushed, each pop and push is logarithmic Fibonacci heapO(E + V log V)Decrease-key is O(1) amortised, so edges cost no logarithm Unsorted arrayO(V2 + E)Scan for the minimum each round; best on dense graphs Space, any variantO(V + E)The graph, plus a heap that never exceeds E entries The heap-based bound is Johnson's 1977 result; the Fibonacci heap improvement is Fredman and Tarjan, 1987. The binary heap itself is Williams' 1964 construction. The Fibonacci variant is theoretically better and almost always slower in practice, because its constants are large, and saying that shows judgement rather than recitation. Sedgewick and Wayne give the shortest clear account of the indexed-priority-queue alternative, which does support decrease-key. Two numbers worth having. On a sparse graph with V = 105 and E = 5 × 105, the binary-heap bound works out to about 10 million operations against roughly 2.2 million for the Fibonacci heap: a real gap on paper that constants erase in practice. And on a dense graph where E approaches V2, the plain array at O(V2) beats the binary heap's O(V2 log V), which is the one case where the "naive" implementation is the right call. 11. Mistakes that fail the interview Ordered by frequency; the first three account for most rejected solutions. - Omitting the stale-entry check. Without if d > dist[u]: continue you re-expand vertices from obsolete labels. The answer usually still comes out right, and the running time degrades badly, which is why it survives testing. - Using Dijkstra where the state is not just the vertex. A stop limit, a fuel budget or a "at most K discounts" clause means the label is per (vertex, resource) pair. Settling the vertex alone discards the route you needed. - Reaching for Dijkstra on negative weights. Use Bellman-Ford, and never try to shift the weights positive. - Forgetting that heapq is a min-heap. Maximising anything means pushing the negated key, and forgetting the second negation on the way out is the classic silent bug. - Comparing tuples that contain an unorderable second element. heappush(heap, (dist, node_object)) raises as soon as two distances tie and Python falls through to comparing the objects. Push an index, or add a tiebreaking counter. - Setting the parent pointer outside the improvement branch. Distances stay correct and the reconstructed path is wrong. - Replacing counts on a tie instead of adding them. One <= where < and == belonged, and every shortest-path count becomes 1. - Quoting a bound without the data structure. Binary heap, Fibonacci heap and array give three different answers, and the interviewer wants to know you know that. - Not asking about the input. Directed? Weights non-negative? Can the target be unreachable? Are there parallel edges with different weights? Each changes the code, and Skiena's point holds: problems are won in the modelling. The habit that prevents most of these: before writing, say what the label means and why extending a route can never improve it. If you cannot say that sentence, the problem is not a Dijkstra problem, and you have just saved yourself twenty minutes. McDowell makes the same argument for interview problems generally. 12. Frequently asked questions Why can Dijkstra not handle negative weights? + Because it assumes that once a vertex is popped with the smallest label in the queue, no route still being built can reach it more cheaply. A negative edge breaks that, since extending a path can reduce its cost. On the four-vertex example with arcs 0 to 1 of weight 1, 0 to 2 of weight 2, 2 to 1 of weight minus 2 and 1 to 3 of weight 1, Dijkstra returns 0, 0, 2, 2 where the truth is 0, 0, 2, 1. Use Bellman-Ford instead. What is lazy deletion and why do I need it? + A binary heap has no efficient decrease-key, so instead of updating a vertex's entry you push a second one with the better label and ignore the obsolete entry when it surfaces. The guard is one line: if the popped distance exceeds the current best for that vertex, skip it. The cost is that the heap can hold up to E entries rather than V, which is why the bound is O((V + E) log V). Can I use Dijkstra when the path has a limit on the number of edges? + Not as written, because a vertex no longer has a single final label: its best cost differs for each number of hops used. Either widen the state so the queue holds pairs of vertex and hops used, which restores the invariant, or use Bellman-Ford and relax every edge exactly K plus one times from a snapshot of the previous round. The second is usually the cleaner answer. What can I replace the addition with? + Anything monotone, meaning extending a route can never lower its cost. Using max instead of plus solves bottleneck or minimum-effort problems. Multiplying probabilities in the range zero to one and maximising works too, and can equivalently be done by minimising the sum of negative logarithms. Subtraction is exactly what fails, which is the same reason negative edges are forbidden. How do I count the number of shortest paths? + Carry a second array holding the number of shortest routes to each vertex. When a relaxation strictly improves a label, replace that count with the predecessor's. When it ties the existing label exactly, add the predecessor's count instead. It is correct only because a vertex's count is final when it is popped, and every relaxation runs from a popped vertex. Dijkstra or A* in an interview? + A* is Dijkstra with a heuristic added to the priority, the 1968 formulation of Hart, Nilsson and Raphael, and it reduces to Dijkstra when that heuristic is zero. Reach for it only when there is a single target and a genuine admissible heuristic, such as straight-line distance on a map or grid. Without one there is nothing to guide the search, and offering A* on an abstract graph signals that you are pattern matching rather than thinking. Which heap should I say I would use? + A binary heap with lazy deletion, giving O((V + E) log V), because it is what every standard library provides and its constants are small. Mention that a Fibonacci heap improves the bound to O(E + V log V) but is slower in practice, and that on a dense graph a plain array scan at O(V squared) beats both. Naming the trade-off matters more than naming the fastest. 13. References The papers that introduced these techniques and the texts that analyse them, in chronological order. - Bellman, R. (1958). “On a routing problem.” Quarterly of Applied Mathematics, 16(1), 87–90. - Dijkstra, E. W. (1959). “A note on two problems in connexion with graphs.” Numerische Mathematik, 1, 269–271. - Williams, J. W. J. (1964). “Algorithm 232: Heapsort.” Communications of the ACM, 7(6), 347–348. - Hart, P. E., Nilsson, N. J. and Raphael, B. (1968). “A formal basis for the heuristic determination of minimum cost paths.” IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107. - Johnson, D. B. (1977). “Efficient algorithms for shortest paths in sparse networks.” Journal of the ACM, 24(1), 1–13. - Fredman, M. L. and Tarjan, R. E. (1987). “Fibonacci heaps and their uses in improved network optimization algorithms.” Journal of the ACM, 34(3), 596–615. - Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2009). Introduction to Algorithms, 3rd edition, Section 24.3. MIT Press. - Sedgewick, R. and Wayne, K. (2011). Algorithms, 4th edition, Section 4.4. Addison-Wesley. - McDowell, G. L. (2015). Cracking the Coding Interview, 6th edition. CareerCup. - Skiena, S. S. (2020). The Algorithm Design Manual, 3rd edition, Chapter 8. Springer. --- ## Finite and Infinite Graphs Explained Source: https://learngraphtheory.org/articles/finite-and-infinite-graphs.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. The definition never said finite - 2. How big: countable, uncountable, locally finite - 3. What finiteness was silently buying you - 4. A one-line counterexample to the handshaking corollary - 5. König's infinity lemma: the first bridge - 6. De Bruijn and Erdős: colouring transfers - 7. Rays, double rays and ends - 8. Spanning trees and the axiom of choice - 9. The Rado graph: one graph to rule them all - 10. When infinite is easier - 11. Infinite graphs in computing - 12. Common mistakes - 13. Glossary - 14. Frequently asked questions - 15. References 1. The definition never said finite Go back to the definition in the guide to vertices and edges and read it for what it actually says: G = (V, E) with E ⊆ [V]² A set of vertices and a set of 2-element subsets of it. Nothing there mentions size. V may be the four landmasses of Königsberg, the integers, the reals, or the set of all finite binary strings, and the definition does not flinch. An infinite graph is simply a graph whose vertex set is infinite, and the whole subject of this article is what quietly stops working when it is. This is not a modern refinement. The first book ever written on graph theory is Dénes König's Theorie der endlichen und unendlichen Graphen of 1936, whose title translates as Theory of Finite and Infinite Graphs. Infinite graphs were in the subject from its first textbook, and Diestel's Graph Theory still devotes a full chapter to them. The important move is to notice that "finite" is a hypothesis you have been using without seeing it. Most textbook statements are of the form "let G be a finite graph", and most of the standard proof techniques quietly need it: - Induction on the number of vertices needs a number of vertices to induct on. - Extremal arguments ("take the vertex of maximum degree", "take the longest path") need the maximum or longest to exist. - Counting arguments need the counts to be finite numbers rather than cardinals. - Termination of an algorithm needs the search to run out of graph. Remove finiteness and each of those has to be replaced by something else. What replaces them, in almost every case, is compactness: the idea that a statement about an infinite object can sometimes be assembled from statements about all of its finite pieces. Sections 5 and 6 are the two classical theorems that do exactly that. 2. How big: countable, uncountable, locally finite "Infinite" is not one size, and three separate measurements matter. Conflating them is the first source of confusion. Four standard infinite graphs. The first three are locally finite; the star is not, and that single difference decides which theorems apply to it. MeasurementWhat it asksWhy it matters Cardinality of VCountable (ℵ0) or uncountable?Countable graphs can be enumerated v1, v2, …, which is what most constructions rely on Cardinality of EHow many edges?A countable graph has at most countably many edges, so |E| ≤ ℵ0 follows from |V| = ℵ0 Local finitenessIs every degree finite?The single most important dividing line: it is the hypothesis of König's lemma A graph is locally finite when every vertex has finite degree. It is a genuinely independent condition from countability, and the two combine in all four ways: - The double ray, an infinite path running in both directions, is countable and locally finite: every degree is 2. - The infinite square grid on ℤ² is countable and locally finite: every degree is 4. This is the graph a spreadsheet, a cellular automaton or a tiled game world actually lives on. - The infinite star, one centre joined to countably many leaves, is countable but not locally finite: the centre has infinite degree. - The complete graph on an uncountable vertex set is uncountable and not locally finite. - An uncountable collection of disjoint double rays is uncountable yet locally finite: there are uncountably many vertices, and every one of them still has degree 2. This is the fourth combination, and it shows that neither condition implies anything about the other. One consequence is worth stating because it trips people up: a locally finite graph can still be infinite, and an infinite graph can have every degree small. Local finiteness bounds each vertex individually; it says nothing about the size of the graph. 3. What finiteness was silently buying you Here is the honest inventory. These are standard results and standard techniques, and each fails on infinite graphs for a specific reason. Finite factInfinite statusCounterexample or reason A tree with at least 2 vertices has at least 2 leavesFailsThe ray has exactly one leaf; the double ray has none Some vertex has maximum degreeFailsDegrees 1, 2, 3, … with no vertex attaining a maximum The number of odd-degree vertices is evenFailsThe ray has exactly one, see section 4 Induction on |V|UnavailableThere is no number of vertices to induct on; transfinite induction needs a well-ordering BFS terminatesFailsIt enumerates forever; it is a semi-decision procedure, see section 11 A connected graph has a spanning treeHolds, but needs choiceSee section 8 k-colourabilityTransfers from finite subgraphsDe Bruijn and Erdős, section 6 An infinite connected graph contains a rayHolds if locally finiteKönig's infinity lemma, section 5; false without local finiteness Ramsey-type resultsOften easierThe infinite version has a clean proof and no bounds to optimise, section 10 Two rows deserve immediate comment because they are the ones people find most surprising. "Some vertex has maximum degree" fails, and it fails for the most mundane reason in analysis: an infinite set of natural numbers need not contain a largest element. Build a graph whose vertices have degrees 1, 2, 3 and so on without bound, and the quantity Δ(G) is simply not defined. Every extremal argument that begins "let v be a vertex of maximum degree" has therefore quietly assumed finiteness, or at least a bound. The infinite star breaks König's lemma, which is why local finiteness appears in its statement. The star is infinite and connected, yet its longest path has length 2: leaf, centre, leaf. There is no ray anywhere in it. One vertex of infinite degree is enough to destroy the conclusion, which tells you the hypothesis is doing real work rather than being technical hygiene. 4. A one-line counterexample to the handshaking corollary The handshaking lemma is the oldest theorem in the subject, and its corollary is the one everybody remembers: the number of vertices of odd degree is even. On infinite graphs it is false, and the counterexample takes one line to draw. Take the ray: vertices v0, v1, v2, … with an edge between consecutive ones. v0 —— v1 —— v2 —— v3 —— v4 —— ⋯ deg(v0) = 1 odd deg(vi) = 2 even, for every i ≥ 1 odd-degree vertices: exactly one. The finite corollary says the count must be even. It is worth being precise about what survives, because the answer is more interesting than a flat "it fails". The equation ∑ deg(v) = 2|E| does not become false; it becomes vacuous. Both sides are cardinal numbers, and for an infinite cardinal κ we have 2κ = κ, so the identity holds trivially and tells you nothing. What actually breaks is the parity argument built on it. That argument splits a finite sum into odd and even parts and concludes that the odd part has an even number of terms, and it is the finiteness of the sum, not the equation, that makes the step legal. The lesson generalises: when a finite theorem fails on infinite graphs, it is usually not the statement that breaks but the proof technique, and the technique is usually counting. Finite and Infinite Graphs Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. König's infinity lemma: the first bridge If finiteness is what most proofs rely on, the interesting question is what can replace it. The first and most useful answer is a lemma König published in 1927, nine years before his textbook. König's infinity lemma. Every infinite, connected, locally finite graph contains a ray, that is, an infinite path v0, v1, v2, … with no repeated vertex. The proof is a pigeonhole argument run infinitely often, and it is worth seeing because the shape recurs throughout infinite combinatorics. Finitely many branches cannot share out infinitely many vertices without one of them getting infinitely many. Repeat that step forever and the choices spell out a ray. Start at any vertex v0. The graph is infinite and connected, so infinitely many vertices are reachable from it. The graph is locally finite, so v0 has only finitely many neighbours. Deleting v0 splits those infinitely many remaining vertices among finitely many pieces, so by the pigeonhole principle at least one piece is infinite. Step into it, and you are in exactly the situation you started in. Repeating forever produces v0, v1, v2, …, and because each step moves into a region you have not yet visited, no vertex repeats. Both hypotheses are doing work, and dropping either one kills the conclusion: - Drop local finiteness and the infinite star is a counterexample: infinite, connected, and its longest path has 3 vertices. - Drop infiniteness and there is nothing to prove. The lemma reaches well beyond graph theory. In its tree form, "an infinite finitely branching tree has an infinite branch", it is the combinatorial core of compactness arguments in logic, and it is what lets you conclude that a computation with infinitely many finite states must have an infinite run. It is the same idea that underlies the compactness theorem for propositional logic, which is where the next section comes from. 6. De Bruijn and Erdős: colouring transfers The second great bridge takes a property that is checkable on finite pieces and lifts it to the whole infinite graph. De Bruijn–Erdős theorem (1951). For a finite k, an infinite graph is k-colourable if and only if every one of its finite subgraphs is k-colourable. The "only if" direction is trivial: a colouring of the whole graph restricts to every subgraph. The content is the other direction, and it is genuinely surprising. It says that colourability, which is a global constraint over infinitely many vertices simultaneously, is entirely determined by what happens on finite windows. Nothing new can go wrong "at infinity". Two qualifications that matter, and that popular accounts usually skip: - The number of colours must be finite. The theorem is about a fixed finite k. It does not say that a graph whose finite subgraphs are all colourable with finitely many colours is finitely colourable. - It needs a choice principle. The standard proofs go through Tychonoff's theorem for compact Hausdorff spaces, Zorn's lemma, or the ultrafilter lemma. In Zermelo-Fraenkel set theory without any choice, the theorem is not provable. De Bruijn and Erdős worked with the axiom of choice available, as almost everyone does. The practical reading for anyone modelling an unbounded system: if your constraint can be expressed as a colouring with a fixed finite palette, you can verify it on finite fragments and conclude it for the whole. That is exactly the licence a compactness argument gives you, and it is why finite model checking can sometimes say something about unbounded runs. 7. Rays, double rays and ends Finite graph theory has no vocabulary for "what the graph looks like far away", because a finite graph has no far away. Infinite graph theory needs one, and the standard construction is due to Halin. - A ray is a one-way infinite path v0, v1, v2, …. König's lemma says every infinite connected locally finite graph has one. - A double ray is a two-way infinite path, indexed by all of ℤ. It is an infinite tree with no leaves at all, which is exactly what no finite tree can be. - An end is an equivalence class of rays, where two rays count as equivalent when no finite set of vertices separates them. Ends are the formal answer to "how many distinct directions does this graph run off in". The notion is easier to feel through examples than to define: GraphEndsReading The ray1It runs off in a single direction The double ray2Two directions; cutting any finite piece leaves two infinite halves The infinite grid ℤ²1Removing any finite patch still leaves one connected infinite region, so all rays are equivalent The infinite binary treeUncountably manyOne per infinite branch, and the branches correspond to infinite binary strings The grid entry is the instructive one. Intuitively a plane runs off in every direction, so you might expect many ends, but the definition asks whether rays can be separated by removing finitely many vertices, and in a grid they cannot: go far enough out and you can always walk around any finite hole. One end is the right answer, and it is the definition, not the picture, that decides. 8. Spanning trees and the axiom of choice Every finite connected graph has a spanning tree, and the proof is a two-line greedy argument: keep deleting an edge that lies on a cycle until none is left. It terminates because there are finitely many edges. The infinite statement is still true, but it is true for a different and much deeper reason: Every connected graph has a spanning tree. For infinite graphs this requires the axiom of choice, and the statement is in fact equivalent to it. The usual proof applies Zorn's lemma to the family of acyclic subgraphs ordered by inclusion, which is a choice principle in disguise. That the implication runs both ways, so that "every connected graph has a spanning tree" is not merely a consequence of choice but as strong as it, is a genuinely striking result: an innocuous-looking statement about graphs turns out to be one of the many equivalent forms of a set-theoretic axiom. The practical significance is small and the conceptual significance is large. No algorithm you write will be affected, because the graphs a program touches are finite or at least computably presented. But it marks precisely where infinite graph theory stops being combinatorics and starts being set theory, and it explains why textbooks on infinite graphs are careful to say which choice principles they assume. The same pattern recurs elsewhere. Several statements that are routine for finite graphs turn out, in their infinite versions, to depend on choice or to be independent of the base axioms, which is why De Bruijn–Erdős also carries a choice hypothesis. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 9. The Rado graph: one graph to rule them all Infinite graphs are not merely finite graphs with more of everything. Some of them behave in ways that have no finite analogue at all, and the cleanest example is the Rado graph. Say a countable graph has the extension property if, for every pair of disjoint finite sets of vertices U and W, there exists a vertex joined to every vertex in U and to none in W. Then: Uniqueness. Any two countable graphs with the extension property are isomorphic. There is, up to isomorphism, exactly one such graph: the Rado graph, also called the random graph R. And the reason it is called the random graph is the second half of the story. Erdős and Rényi showed that if you build a countably infinite graph by deciding each possible edge independently with probability 1/2, the result satisfies the extension property with probability 1. So: Toss a fair coin for every pair of vertices in a countably infinite set. With probability 1 the graph you get is the Rado graph. Do it again, differently. You get the Rado graph again, up to isomorphism. There is essentially one countably infinite random graph. Nothing remotely like this happens for finite graphs, where random graphs on n vertices are wildly various and the interesting questions are all about which properties hold with high probability. The Rado graph also contains every finite graph and every countable graph as an induced subgraph, which makes it universal for countable graphs. The point for a working reader is not the construction itself but the warning it carries: intuitions calibrated on finite graphs are not just quantitatively but qualitatively wrong at infinity. "Random" stops meaning "varied" and starts meaning "canonical". 10. When infinite is easier It would be reasonable to assume infinite graphs are uniformly harder. Sometimes the opposite is true, and Ramsey theory is the standard illustration. Finite RamseyInfinite Ramsey StatementFor every k there is an N such that any 2-colouring of the edges of KN has a monochromatic KkAny 2-colouring of the edges of the complete graph on a countably infinite vertex set has an infinite monochromatic complete subgraph ProofHarder, and the bounds are the subject of a large literatureA short pigeonhole argument Open problemsEven R(5,5) is unknownThe statement is clean and settled Ramsey proved both in his 1930 paper. The infinite version is easier precisely because it does not ask for a bound: you never have to say how far out the monochromatic structure appears, only that it appears. Extracting the finite version from the infinite one is possible, by a compactness argument of the same family as König's lemma, but the bounds it yields are terrible, which is why finite Ramsey theory is a separate and much harder subject. The general moral: infinite statements are often cleaner because they are not quantitative. Whenever a finite theorem is hard because of its bounds, its infinite analogue may be much simpler and may still tell you something useful. 11. Infinite graphs in computing Infinite graphs are not a purely mathematical indulgence. They appear constantly in computer science, usually implicitly, and always in the same disguised form: a graph you never build. - State spaces. The configuration graph of a Turing machine, a program with unbounded integers, or a protocol with unbounded queues is infinite. Model checking such a system is exactly the study of an infinite graph presented by a finite description. - Lattices and grids. Pathfinding on an unbounded tile map, cellular automata and image lattices all live on ℤ², an infinite locally finite graph. - Game trees. The full tree of a game with no move limit is infinite; search algorithms explore a finite prefix of it. - Cayley graphs. An infinite group with a chosen generating set gives an infinite vertex-transitive graph, and geometric group theory studies groups precisely through these graphs and their ends. - Recursively defined structures. The graph of all finite strings under "append one character" is an infinite tree, and it is the object König's lemma is usually applied to in computability. The engineering consequence is a single sharp distinction: On an infinite locally finite graph, search becomes semi-decidable. A breadth first search from s will find a path to t if one exists, and will run forever if none does. Local finiteness is what makes each level finite, so that the search reaches every distance in finite time. It can confirm reachability but never refute it. That asymmetry is why BFS rather than DFS is the right choice on an infinite graph: BFS explores in order of distance and will reach any reachable vertex in finite time, whereas DFS can descend a single infinite branch and never come back. It is the same reason iterative deepening exists. To get termination you need something extra, such as local finiteness plus a bound, a monotone measure that decreases, or a finite abstraction of the state space. 12. Common mistakes - Assuming a maximum or minimum exists. "Take the vertex of maximum degree" and "take the longest path" both presuppose finiteness or an explicit bound. On an infinite graph they may denote nothing at all. - Using the odd-degree corollary. The ray has exactly one odd-degree vertex, as section 4 shows. - Inducting on the number of vertices. There is no such number. Transfinite induction is available but needs a well-ordering, which is another appeal to choice. - Forgetting local finiteness in König's lemma. The infinite star is infinite, connected, and contains no ray. The hypothesis is not decoration. - Expecting termination. Search on an infinite graph is semi-decidable, so a "no path" answer never arrives. Bound the search explicitly, or work with a finite abstraction. - Running DFS on an infinite graph. A single infinite branch swallows it. Use BFS or iterative deepening. - Applying finite counting results. Anything proved by counting vertices, edges or incidences needs re-examination; anything proved by compactness usually survives. - Treating "infinite" as one thing. Countable versus uncountable and locally finite versus not are independent, and almost every theorem depends on which combination you have. 13. Glossary TermMeaning Infinite graphA graph whose vertex set is infinite; the definition G = (V, E) is unchanged Countable graph|V| = ℵ0, so the vertices can be listed v1, v2, … Locally finiteEvery vertex has finite degree; independent of how large the graph is RayA one-way infinite path with no repeated vertex Double rayA two-way infinite path, indexed by the integers EndAn equivalence class of rays, where two rays are equivalent if no finite vertex set separates them König's infinity lemmaEvery infinite connected locally finite graph contains a ray De Bruijn–Erdős theoremFor finite k, k-colourability of an infinite graph follows from k-colourability of all its finite subgraphs Rado graphThe unique countable graph with the extension property; the countable random graph Extension propertyFor disjoint finite U and W there is a vertex joined to all of U and none of W Compactness argumentDeducing a property of an infinite graph from the same property on all its finite subgraphs Semi-decidableA yes-answer arrives in finite time; a no-answer may never arrive 14. Frequently asked questions What is an infinite graph? A graph whose vertex set is infinite. The definition G = (V, E) with E a set of 2-element subsets of V says nothing about size, so no change to the definition is needed. What changes is which theorems and proof techniques still apply: induction on the number of vertices, extremal arguments that pick a maximum, and counting arguments all rely on finiteness, while compactness arguments such as König's infinity lemma survive. What does locally finite mean, and why does it matter so much? A graph is locally finite when every vertex has finite degree, which is independent of whether the graph itself is finite. It matters because it is the hypothesis of König's infinity lemma: every infinite, connected, locally finite graph contains a ray. Drop it and the conclusion fails immediately, since the infinite star is infinite and connected yet its longest path has only three vertices. Most infinite graphs that arise in computing, such as the integer grid and state spaces with finitely many moves per state, are locally finite. Does the handshaking lemma work on infinite graphs? Not usefully. The equation itself becomes vacuous: both sides are infinite cardinals, and doubling an infinite cardinal changes nothing, so it holds trivially and carries no information. The corollary everyone actually uses, that the number of odd-degree vertices is even, is simply false. The one-way infinite path has exactly one vertex of degree 1 and all others of degree 2, so it has exactly one odd-degree vertex. Can BFS or DFS run on an infinite graph? BFS can, in a limited sense: on a locally finite graph it reaches every vertex at distance d in finite time, so it will find a path to the target if one exists. It will not terminate if no path exists, which makes reachability semi-decidable rather than decidable. DFS is worse, because it can descend a single infinite branch and never return, so it may miss a target that lies one step off the start. Use BFS or iterative deepening, and bound the search explicitly if you need an answer either way. Is there really only one countably infinite random graph? Up to isomorphism, yes. Build a countably infinite graph by deciding each possible edge independently with probability one half, and with probability 1 the result has the extension property. Any two countable graphs with that property are isomorphic, so almost every such random construction produces the same graph, the Rado graph. It is also universal: every finite and every countable graph appears in it as an induced subgraph. Nothing comparable happens for finite random graphs. Do infinite graphs matter in practice, or are they purely theoretical? They appear constantly, always as a graph you never build. The configuration space of a program with unbounded integers, the tile map of an unbounded game world, a game tree with no move limit, and the tree of all finite strings are all infinite graphs given by finite descriptions. Model checking, termination analysis and search over unbounded state spaces are, formally, infinite graph problems. The practical consequence is that search becomes semi-decidable, so algorithms need an explicit bound or a finite abstraction to guarantee an answer. 15. References The definitions, theorems and attributions above come from these sources, listed in chronological order. - König, D. (1927). "Über eine Schlussweise aus dem Endlichen ins Unendliche." Acta Litterarum ac Scientiarum Regiae Universitatis Hungaricae Francisco-Josephinae, Sectio Scientiarum Mathematicarum (Szeged) 3, 121 to 130. The infinity lemma. - Ramsey, F. P. (1930). "On a Problem of Formal Logic." Proceedings of the London Mathematical Society s2-30, 264 to 286. Contains both the finite and the infinite Ramsey theorems. - König, D. (1936). Theorie der endlichen und unendlichen Graphen. Leipzig: Akademische Verlagsgesellschaft. The first book on graph theory, and its title already names both cases. - de Bruijn, N. G. and Erdős, P. (1951). "A Colour Problem for Infinite Graphs and a Problem in the Theory of Relations." Indagationes Mathematicae 13, 369 to 373. - Erdős, P. and Rényi, A. (1963). "Asymmetric Graphs." Acta Mathematica Academiae Scientiarum Hungaricae 14, 295 to 315. Contains the observation that the countable random graph is determined up to isomorphism. - Halin, R. (1964). "Über unendliche Wege in Graphen." Mathematische Annalen 157, 125 to 137. The theory of ends of graphs. - Rado, R. (1964). "Universal Graphs and Universal Functions." Acta Arithmetica 9, 331 to 340. The explicit construction of the universal countable graph. - Bollobás, B. (1998). Modern Graph Theory. Graduate Texts in Mathematics 184. New York: Springer. - Cameron, P. J. (1997). "The Random Graph." In R. L. Graham and J. Nešetřil (eds), The Mathematics of Paul Erdős II, 333 to 351. Berlin: Springer. A survey of the Rado graph and its properties. - West, D. B. (2001). Introduction to Graph Theory, 2nd edition. Upper Saddle River: Prentice Hall. - Bondy, J. A. and Murty, U. S. R. (2008). Graph Theory. Graduate Texts in Mathematics 244. London: Springer. - Diestel, R. (2017). Graph Theory, 5th edition. Graduate Texts in Mathematics 173. Berlin: Springer. Chapter 8 is devoted to infinite graphs, rays and ends. Build the finite piece and watch the pattern Lay out a long path or a patch of the integer grid and run a traversal across it. Everything an algorithm can ever see of an infinite graph is a finite fragment like this one. Open the visualizer Related Articles Vertices and Edges Explained Read more → Simple Graphs vs Multigraphs Read more → BFS vs DFS Read more → --- ## BFS vs DFS: Key Differences Source: https://learngraphtheory.org/articles/bfs-vs-dfs.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Deep Dive into Breadth-First Search (BFS) - 2. Deep Dive into Depth-First Search (DFS) - 3. BFS vs. DFS: The Ultimate Comparison - 4. Real-World Applications: When to use which? - 5. Advanced Variations - Conclusion - Frequently Asked Questions (FAQ) Graph traversal is the foundational mechanism of almost all graph theory algorithms. It is the systematic process of visiting, checking, analyzing, and often updating each vertex (node) in a graph. Whether you are finding the shortest route on Google Maps, recommending friends on a social network, crawling the internet for a search engine, or solving a complex puzzle algorithmically, you are utilizing graph traversal. At the heart of this domain lie two legendary algorithms: Breadth-First Search (BFS) and Depth-First Search (DFS). While both guarantee that every reachable node in a connected component will be visited exactly once, the order and strategy by which they visit nodes lead to drastically different applications, time/space complexities, and behavioral patterns. In this comprehensive guide, we will dissect both algorithms, analyze their data structures, review implementation code, compare their complexities, and finally look at real-world scenarios to help you confidently answer the ultimate question: "When should I use BFS, and when should I use DFS?" 1. Deep Dive into Breadth-First Search (BFS) Breadth-First Search explores the graph in concentric "waves." It starts at a chosen root node (or arbitrary starting node) and visits all immediate neighbors first. Only after completely exhausting all nodes at a distance of exactly 1 edge away does it move on to nodes at distance 2, and so forth. The Water Drop Analogy: Think of BFS like a drop of water creating ripples in a still pond. The ripple expands outward uniformly in all directions. It does not shoot off in one straight line; it covers area equally by radius. The Mechanics of BFS To ensure that nodes are visited in this strictly layer-by-layer fashion, BFS relies on a Queue (First-In, First-Out, or FIFO) data structure to keep track of which nodes to visit next. The standard BFS algorithm follows these steps: - Initialize a Queue and a boolean array (or Hash Set) to keep track of visited nodes. - Enqueue the starting node and immediately mark it as visited. - Begin a loop that continues as long as the Queue is not empty. - Dequeue the front node. This is your current node. Process it as needed. - Iterate over every adjacent neighbor of this current node. - If a neighbor has not been visited, mark it as visited and enqueue it. - Repeat until the queue is empty. BFS Implementation Example Below is a clean implementation of BFS in Python using an adjacency list representation and the highly efficient collections.deque. from collections import deque def breadth_first_search(graph, start_node): # Initialize a queue and a set to keep track of visited nodes queue = deque([start_node]) visited = {start_node} # Store the traversal order traversal_order = [] while queue: # Dequeue a vertex from the front of the queue current_node = queue.popleft() traversal_order.append(current_node) # Get all adjacent vertices of the dequeued vertex for neighbor in graph[current_node]: if neighbor not in visited: # Mark as visited IMMEDIATELY before enqueuing # to prevent duplicate entries in the queue visited.add(neighbor) queue.append(neighbor) return traversal_order # Example Graph (Adjacency List) graph = { 'A': ['B', 'C'], 'B': ['A', 'D', 'E'], 'C': ['A', 'F'], 'D': ['B'], 'E': ['B', 'F'], 'F': ['C', 'E'] } print(breadth_first_search(graph, 'A')) # Output: ['A', 'B', 'C', 'D', 'E', 'F'] BFS on the example graph. It finishes each ring of nodes before moving outward, so the queue empties in the order A, B, C, D, E, F. Stop Reading, Start Visualizing Reading code is good, but watching the algorithm execute in real-time builds true intuition. Watch how the Queue manages the BFS wave on our platform. Launch Interactive BFS Visualizer 2. Deep Dive into Depth-First Search (DFS) Depth-First Search takes a drastically different, more aggressive approach. Instead of exploring broadly, it explores deeply. It starts at a root node and explores as far as possible along a singular branch until it hits a "dead end" (a node with no unvisited neighbors). Once stuck, it retreats or backtracks up the path just enough to find a new, unexplored branch to dive down. Run it yourself in the DFS visualizer. DFS is also the engine behind topological sorting. The Maze Runner Analogy: Think of DFS like navigating a physical labyrinth. You place your hand on the right wall and keep walking forward. You keep taking turns until you hit a dead end, at which point you turn around, walk back to the last intersection, and take the unexplored path. The Mechanics of DFS Because DFS needs to remember where it came from so it can backtrack, it utilizes a Stack (Last-In, First-Out, or LIFO) data structure. Most commonly, developers implement DFS recursively, elegantly utilizing the CPU's native call stack rather than creating a manual stack object. The standard Recursive DFS algorithm follows these steps: - Start at the current node and immediately mark it as visited. - Process the node (e.g., print its value). - Iterate over all its adjacent neighbors. - If a neighbor has not been visited, recursively call the DFS function on that neighbor. - When the loop finishes (no unvisited neighbors remain), the function returns, automatically backtracking to the previous caller on the stack. DFS Implementation Example Below is an elegant recursive implementation of DFS in Python. def depth_first_search_recursive(graph, current_node, visited=None, traversal_order=None): # Initialize collections on the first call if visited is None: visited = set() if traversal_order is None: traversal_order = [] # Mark the current node as visited and record it visited.add(current_node) traversal_order.append(current_node) # Recursively visit all unvisited neighbors for neighbor in graph[current_node]: if neighbor not in visited: depth_first_search_recursive(graph, neighbor, visited, traversal_order) return traversal_order # Example Graph (Adjacency List) graph = { 'A': ['B', 'C'], 'B': ['A', 'D', 'E'], 'C': ['A', 'F'], 'D': ['B'], 'E': ['B', 'F'], 'F': ['C', 'E'] } print(depth_first_search_recursive(graph, 'A')) # Output: ['A', 'B', 'D', 'E', 'F', 'C'] Notice the difference in output compared to BFS. DFS aggressively pursued the path A to B to D before backtracking. DFS on the same graph. It dives down A, B, D, hits a dead end, backtracks, and continues, giving the order A, B, D, E, F, C. Visualize the Backtracking Magic Recursive call stacks can be hard to map in your head. Watch how DFS dives deep and visually backtracks along the edges when it gets stuck. Launch Interactive DFS Visualizer BFS vs DFS Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three Both bounds assume the graph is stored as an adjacency list. On an adjacency matrix the same two traversals are Θ(n2), because finding one vertex’s neighbours means scanning a whole row; see graph representation for why the storage choice, not the pseudocode, sets the complexity. 3. BFS vs. DFS: The Ultimate Comparison In technical interviews and real-world system design, choosing the wrong traversal algorithm can lead to catastrophic memory exhaustion or highly inefficient execution times. Let's break down the technical differences. The single design choice that separates the two algorithms: BFS pulls the oldest node from a queue, DFS pulls the newest node from a stack. Metric Breadth-First Search (BFS) Depth-First Search (DFS) Data Structure Queue (FIFO) - Iterative approach Stack (LIFO) - Often Recursive approach Time Complexity O(V + E)Must visit every Vertex (V) and check every Edge (E). O(V + E)Also visits every Vertex and explores every Edge. Space Complexity O(W) where W is the maximum width of the graph.Can be O(V) in worst-case dense graphs. O(H) where H is the maximum depth/height of the graph.Can be O(V) if the graph is a single long line. Shortest Path Guaranty Yes. For unweighted graphs, BFS is guaranteed to find the shortest path. No. DFS might find a highly convoluted, massive path before finding the short, direct one. Memory Footprint High if the tree/graph is very wide (e.g., social network). The queue must hold all nodes in the current level. High if the tree/graph is very deep. The call stack must hold all ancestors of the current node. Optimality Optimal for finding targets close to the source. Optimal for finding targets far from the source or exploring all possibilities. Understanding the Space Complexity Trade-off While both algorithms have a time complexity of O(V + E), their space complexities drastically differ depending on the topology of the graph. - Wide Graphs: If a graph has a massive branching factor (e.g., an HTML DOM tree, or a Twitter follower graph), the BFS Queue will balloon in size because it must store an entire "level" of the graph at once. In these scenarios, DFS is highly memory efficient. - Deep Graphs: If a graph is extremely deep (e.g., a massive blockchain or a deep decision tree), DFS might exceed the maximum recursion depth or consume too much stack memory. In these scenarios, BFS might be preferable, though iterative deepening DFS is often the actual solution used in production. If you are preparing for interviews specifically, the eight BFS problems that keep recurring, with their follow-ups and traps, are worked through in BFS interview questions. 4. Real-World Applications: When to use which? The choice between BFS and DFS usually comes down to what you are trying to extract from the graph structure. Use Breadth-First Search when: - Finding the Shortest Path (Unweighted Graphs): Because BFS explores level by level, the first time it encounters the target node, it has definitively taken the fewest number of edges to get there. GPS routing algorithms often use variations of BFS (like Dijkstra's) for this reason. - Target is likely close to the source: If you are searching a family tree for an immediate cousin, BFS will find them instantly. DFS might explore an entirely different branch of the family back to the 16th century before checking the cousin. - Peer-to-Peer Networks: Protocols like BitTorrent use BFS to find neighbor nodes. "Find all peers 1 hop away, then 2 hops away." - Web Crawlers: Search engines like Google initially used BFS to crawl the web. They want to index the most prominent links on the main page before diving deeply into obscure, nested sub-pages. - Social Network "Degrees of Separation": Features like LinkedIn's "2nd-degree connection" rely entirely on BFS limits. Use Depth-First Search when: - Checking for Cycles: DFS is uniquely equipped to find back-edges (edges pointing back to an ancestor). If a back-edge exists, the graph has a cycle. This is critical for preventing infinite loops in routers and detecting deadlocks in operating systems. - Topological Sorting: In build systems (like Make, Webpack, or npm), tasks have dependencies. DFS is used to linearly order these tasks so that every task is executed only after its dependencies are met. - Memory is heavily constrained (and the graph is wide): As discussed, DFS only needs to store the path from the root to the current node, making it incredibly memory-efficient for wide, massive graphs like game-state trees (e.g., Chess engines). - Solving Puzzles and Mazes: DFS is the algorithm of choice for solving Sudoku, N-Queens, or generating/solving physical mazes. It aggressively explores a possible solution, and if it fails, it simply backtracks and tries the next configuration. - Finding Strongly Connected Components: Algorithms like Tarjan's or Kosaraju's rely heavily on DFS to group nodes into strongly connected clusters. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack For DFS specifically, the eight problems that keep recurring in interviews are worked through in DFS interview questions. 5. Advanced Variations In modern computer science, raw BFS and raw DFS are often augmented to handle massive scales. - Bidirectional BFS: Instead of running BFS from the source to the target, you run two simultaneous BFS waves - one from the source and one from the target. When the waves intersect, you've found the shortest path. This drastically reduces the search space area. - Iterative Deepening DFS (IDDFS): This combines the space-efficiency of DFS with the shortest-path guarantee of BFS. It runs a DFS but sets a strict depth limit. If the target isn't found, it increments the depth limit and starts over. It's heavily used in AI and game tree analysis. Conclusion Breadth-First Search and Depth-First Search are the undisputed cornerstones of graph theory. BFS is your expansive, level-by-level wave, perfect for shortest paths and broad network analysis. DFS is your aggressive, memory-efficient maze runner, perfect for cycle detection, topological sorting, and deep puzzle solving. Mastering both algorithms is non-negotiable for passing technical interviews at top tech companies and for designing robust, scalable software architectures. Frequently Asked Questions When should I choose BFS over DFS? Choose Breadth-First Search (BFS) when you need to find the shortest path in an unweighted graph, or when you are searching for a target node known to be close to the starting point. When is DFS more advantageous than BFS? Depth-First Search (DFS) is better for exploring all possibilities (e.g., maze solving), topological sorting, cycle detection, or when you have memory constraints, as DFS uses less memory on deep graphs. How do the data structures used by BFS and DFS differ? BFS uses a Queue (First-In, First-Out) to explore level by level. DFS uses a Stack (Last-In, First-Out), which is often implemented implicitly using recursion. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Shortest Path Algorithms Source: https://learngraphtheory.org/articles/shortest-path-algorithms.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Introduction to the Shortest Path Problem - 2. Dijkstra's Algorithm: The Greedy Workhorse - 3. Bellman-Ford Algorithm: Handling the Negative - 4. Floyd-Warshall Algorithm: The All-Pairs Solution - 5. Comprehensive Comparison - 6. Frequently Asked Questions (FAQ) 1. Introduction to the Shortest Path Problem In the vast field of computer science, few problems are as universally applicable as the "Shortest Path Problem". Whether it is Google Maps calculating the fastest route home in heavy traffic, an internet router determining how to forward a data packet, or an AI figuring out the optimal sequence of moves to win a game, the underlying mathematics are identical. At its core, the problem asks: Given a graph composed of vertices (locations) and edges (roads connecting them), where each edge has an assigned "weight" (distance, time, or cost), what is the path from a starting vertex to a destination vertex that minimizes the total weight? However, the real world is messy. Some roads have tolls, some networks have high latency, and in abstract systems like financial arbitrage, edge weights can actually be negative. Because of these varying constraints, computer scientists have developed a specialized suite of algorithms. The three most vital algorithms you must master are Dijkstra, Bellman-Ford, and Floyd-Warshall. 2. Dijkstra's Algorithm: The Greedy Workhorse Conceived by the legendary Dutch computer scientist Edsger W. Dijkstra in 1956, this algorithm is the undisputed king of routing. It is remarkably efficient, conceptually elegant, and forms the foundation of modern GPS systems and the OSPF internet routing protocol. Deep dive: the full guide to Dijkstra's algorithm works through a complete example, or run it yourself in the Dijkstra visualizer. The Strategy: Dijkstra's algorithm uses a "Greedy" approach. It always chooses the unvisited vertex that is closest to the source, explores its neighbors, and updates their distances if a shorter path is found. How it works internally The algorithm maintains a list of "tentative" distances for every node. Initially, the distance to the starting node is 0, and all other distances are set to infinity. It utilizes a Priority Queue (often a Min-Heap) to constantly fetch the node with the current lowest tentative distance. - Assign a tentative distance value to every node, 0 for our initial node and to infinity for all other nodes. - Set the initial node as current and mark all other nodes unvisited. - For the current node, consider all of its unvisited neighbors and calculate their tentative distances through the current node. Compare the newly calculated tentative distance to the current assigned value and assign the smaller one. - When we are done considering all of the unvisited neighbors of the current node, mark the current node as visited. A visited node will never be checked again. - Select the unvisited node that is marked with the smallest tentative distance, set it as the new "current node", and go back to step 3. Implementation in Python import heapq def dijkstra(graph, start): # Initialize distances and priority queue distances = {vertex: float('infinity') for vertex in graph} distances[start] = 0 pq = [(0, start)] while pq: current_distance, current_vertex = heapq.heappop(pq) # Nodes can get added to the priority queue multiple times. # We only process a vertex the first time we remove it. if current_distance > distances[current_vertex]: continue for neighbor, weight in graph[current_vertex].items(): distance = current_distance + weight # If a shorter path is found, update it and push to queue if distance < distances[neighbor]: distances[neighbor] = distance heapq.heappush(pq, (distance, neighbor)) return distances # Example Adjacency Dictionary with edge weights graph = { 'A': {'B': 4, 'C': 2}, 'B': {'C': 5, 'D': 10}, 'C': {'E': 3}, 'D': {'F': 11}, 'E': {'D': 4}, 'F': {} } print(dijkstra(graph, 'A')) # Output: {'A': 0, 'B': 4, 'C': 2, 'D': 9, 'E': 5, 'F': 20} Dijkstra on the example graph. Notice D is reached through C and E (2 + 3 + 4 = 9), not through the tempting direct edge B to D of weight 10. The Fatal Flaw: Negative Weights Dijkstra's greedy nature makes it incredibly fast, running in O((V + E) log V) time. However, this greed comes with a fatal flaw: It assumes that adding an edge to a path can never make the path shorter. Therefore, if a graph contains negative edge weights (representing a profit, or time-travel, or energy regeneration), Dijkstra will fail silently and return incorrect results. Shortest Path Algorithms Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. Bellman-Ford Algorithm: Handling the Negative When you cannot guarantee that all edge weights are positive, the Bellman-Ford algorithm steps in. While Dijkstra is a greedy algorithm, Bellman-Ford uses a Dynamic Programming approach. It is slower, but far more robust. Deep dive: the full guide to the Bellman-Ford algorithm covers negative cycle detection in detail, or step through it in the Bellman-Ford visualizer. Instead of cleverly picking the closest node, Bellman-Ford takes a sledgehammer approach: It simply "relaxes" (updates the distances of) every single edge in the graph. And it does this V - 1 times (where V is the total number of vertices). Because the longest possible path without a cycle in a graph can only be V - 1 edges long, repeating the process this many times guarantees that the absolute shortest paths have rippled outward to every node. A single negative edge is enough to fool Dijkstra. Because Bellman-Ford relaxes every edge repeatedly, it recovers the true shortest distance. Detecting Negative Cycles Bellman-Ford's greatest superpower is its ability to detect Negative Weight Cycles. If a graph has a loop where the total sum of weights is negative, you could infinitely loop through that cycle to achieve a distance of negative infinity. Shortest paths are mathematically undefined in such graphs. Bellman-Ford detects this by running the edge-relaxation process one final time (the V-th time). If any distance is still capable of being reduced, a negative cycle exists. Implementation in Python def bellman_ford(graph, num_vertices, start): distances = {vertex: float('infinity') for vertex in range(num_vertices)} distances[start] = 0 # Relax all edges V - 1 times for _ in range(num_vertices - 1): for u, v, weight in graph: if distances[u] != float('infinity') and distances[u] + weight < distances[v]: distances[v] = distances[u] + weight # Check for negative-weight cycles for u, v, weight in graph: if distances[u] != float('infinity') and distances[u] + weight < distances[v]: print("Graph contains a negative weight cycle!") return None return distances # Graph represented as an Edge List: (source, destination, weight) edges = [ (0, 1, -1), (0, 2, 4), (1, 2, 3), (1, 3, 2), (1, 4, 2), (3, 2, 5), (3, 1, 1), (4, 3, -3) ] print(bellman_ford(edges, 5, 0)) The time complexity is a hefty O(V * E), making it unsuitable for massive networks like social media graphs, but perfect for specialized systems like financial currency arbitrage detection. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 4. Floyd-Warshall Algorithm: The All-Pairs Solution Both Dijkstra and Bellman-Ford are Single-Source shortest path algorithms. They calculate the distance from one specific starting node to all other nodes. But what if you are designing a routing table or a flight matrix, and you need to know the shortest path from every node to every other node simultaneously? Deep dive: the full guide to the Floyd-Warshall algorithm walks the distance matrix pass by pass, or watch it fill in the Floyd-Warshall calculator. You could run Dijkstra V times, but the Floyd-Warshall Algorithm provides an incredibly elegant, pure Dynamic Programming alternative. It computes the shortest path between all pairs of vertices in a single, compact block of logic. The Mechanics Floyd-Warshall uses a 2D adjacency matrix. It iterates through every possible intermediate vertex k. For every pair of vertices (i, j), it asks a simple question: "Is it shorter to go directly from i to j, or is it shorter to go from i to k, and then from k to j?" The single comparison at the heart of Floyd-Warshall, repeated for every intermediate node k and every pair (i, j). Implementation in Python def floyd_warshall(graph, V): # Initialize the distance matrix dist = [[float('infinity') for _ in range(V)] for _ in range(V)] # Set the diagonal to 0, and initialize given edges for i in range(V): dist[i][i] = 0 for u, v, w in graph: dist[u][v] = w # Core logic: check every intermediate vertex k for k in range(V): for i in range(V): for j in range(V): # Update distance if path through k is shorter dist[i][j] = min(dist[i][j], dist[i][k] + dist[k][j]) return dist The beauty of Floyd-Warshall lies in its simplicity, consisting of just three tightly nested loops. However, this results in a rigid time complexity of O(V³). It is exclusively used for dense graphs or graphs with a relatively small number of vertices. 5. Comprehensive Comparison Algorithm Time Complexity Handles Negative Weights? Best Use Case Dijkstra O((V + E) log V) No (Fails silently) GPS navigation, standard network routing, massive sparse graphs. Bellman-Ford O(V * E) Yes (And detects cycles) Financial arbitrage, systems with penalties/costs. Floyd-Warshall O(V³) Yes Flight network matrices, dense graphs, computing all pairs. Frequently Asked Questions How do Dijkstra's and Bellman-Ford algorithms differ? Dijkstra's is a greedy algorithm that runs in O((V + E) log V) but fails on negative weights. Bellman-Ford runs in O(VE), but can handle negative weights and detect negative cycles. What is the Floyd-Warshall algorithm used for? It is an all-pairs shortest path algorithm using dynamic programming, finding the shortest paths between all pairs of nodes in O(V³) time. How does the A* algorithm optimize shortest path searches? A* optimizes the search by using heuristics to estimate distance to the goal, focusing the search direction towards the target rather than exploring blindly in all directions. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Minimum Spanning Trees (MST) Source: https://learngraphtheory.org/articles/minimum-spanning-trees.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is a Minimum Spanning Tree? - 2. Prim's Algorithm: The Node-Centric Approach - 3. Kruskal's Algorithm: The Edge-Centric Approach - 4. The Secret Sauce: Union-Find Disjoint Sets - 5. Prim vs. Kruskal: Which one to choose? - 6. Real World Applications - Frequently Asked Questions (FAQ) 1. What is a Minimum Spanning Tree (MST)? Imagine you are tasked with designing a new telecommunications network for a city. You have a map with several neighborhoods (nodes) and the possible routes where you can lay fiber-optic cables (edges). Laying cable is expensive, and the cost varies depending on the terrain (edge weights). Your goal is to ensure that every neighborhood is connected to the network, meaning there is a path from any neighborhood to any other neighborhood. However, to save money, you want the total cost of laying the cables to be as small as possible. You don't need redundant connections; as long as everything is connected, you are satisfied. This exact scenario is the textbook definition of the Minimum Spanning Tree (MST) problem. Let's break down the terminology: - Tree: A connected graph with absolutely no cycles. If you have V vertices, a tree will always have exactly V - 1 edges. - Spanning: It covers (spans) every single vertex in the original graph. - Minimum: Among all the possible spanning trees that could be drawn over the graph, this one has the smallest total edge weight. It is important to note that a graph might have multiple Minimum Spanning Trees if several edges share the same weights, but the minimum total weight will always be unique. To solve this problem, computer science relies heavily on two legendary Greedy Algorithms: Prim's Algorithm and Kruskal's Algorithm. The MST (green) links all six nodes for a total weight of 13. Both Prim's and Kruskal's algorithms find this same tree. 2. Prim's Algorithm: The Node-Centric Approach Prim's Algorithm was originally designed in 1930 by mathematician Vojtěch Jarník, and later rediscovered and popularized by Robert Prim in 1957. Prim's takes a node-centric approach, slowly growing the tree outward from a single starting vertex, much like a mold spreading on a piece of bread. Deep dive: the full guide to Prim's algorithm covers the priority queue implementation, or grow the tree yourself in the Prim visualizer. The Strategy: Start at any arbitrary node. Look at all the edges connecting the nodes in your current growing tree to nodes outside the tree. Pick the edge with the lowest weight. Add that edge and the new node to your tree. Repeat until all nodes are included. The Mechanics To efficiently find the lowest weight edge connecting the tree to the outside world, Prim's algorithm heavily utilizes a Priority Queue (Min-Heap), making it very similar in structure to Dijkstra's Shortest Path Algorithm. - Initialize a boolean array to keep track of nodes already included in the MST. - Initialize a Priority Queue to store tuples of (edge_weight, destination_node). - Pick a random starting node, mark it as included, and push all its edges into the Priority Queue. - While the Priority Queue is not empty and the MST doesn't have V - 1 edges: - Pop the edge with the minimum weight. - If the destination node is already in the MST, ignore it (to prevent cycles). - Otherwise, add the edge to the MST, mark the new node as included, and push all edges radiating from this new node into the Priority Queue. Prim's Algorithm in Python import heapq def prims_algorithm(graph, start_node): # graph is represented as an adjacency list: # graph[u] = [(weight, v), ...] mst = [] visited = set([start_node]) # Initialize the priority queue with edges from the start node edges = [ (weight, start_node, to_node) for weight, to_node in graph[start_node] ] heapq.heapify(edges) total_cost = 0 while edges: weight, frm, to = heapq.heappop(edges) # If the destination is not visited, it's a safe edge if to not in visited: visited.add(to) mst.append((frm, to, weight)) total_cost += weight # Push all edges originating from the newly visited node for next_weight, next_to in graph[to]: if next_to not in visited: heapq.heappush(edges, (next_weight, to, next_to)) return mst, total_cost Prim's grows one contiguous tree from a start node. The badges show the order nodes are pulled in: A, C, B, D, E, F. Visualize Prim's Algorithm Growing Watch how the Priority Queue evaluates the "frontier" edges in real-time. Seeing the tree grow node by node is the best way to internalize the greedy choice. Launch Interactive Prim's Visualizer Minimum Spanning Trees (MST) Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. Kruskal's Algorithm: The Edge-Centric Approach Joseph Kruskal published his algorithm in 1956. Unlike Prim, which grows a single continuous tree from a root, Kruskal takes an edge-centric approach. It looks at the graph as a whole, focusing entirely on the edges rather than the nodes. Deep dive: the full guide to Kruskal's algorithm works through the edge sort and cycle checks, or run it in the Kruskal visualizer. The Strategy: Throw all the edges in a pile and sort them from smallest weight to largest weight. Pick up the smallest edge. If adding this edge to your MST does not create a cycle, keep it. If it creates a cycle, throw it away. Repeat until you have V - 1 edges. Initially, Kruskal's algorithm treats every node as its own separate tree. As it adds edges, it merges these small trees into larger forests, until eventually, there is only one massive tree spanning the entire graph. 4. The Secret Sauce: Union-Find Disjoint Sets The entire logic of Kruskal's algorithm hinges on one crucial step: "If adding this edge does not create a cycle." Deep dive: the full guide to Union-Find explains path compression and union by rank with code. How can we efficiently determine if connecting Node A and Node B will create a cycle? Running a full DFS every time we want to add an edge would be painfully slow. This is where the brilliant Union-Find Disjoint Set data structure comes to the rescue. A Union-Find data structure keeps track of elements partitioned into a number of disjoint (non-overlapping) subsets. It supports two primary operations in near constant O(1) time: - Find: Determine which set a particular element belongs to. (Usually by finding the "root" or "representative" of the set). - Union: Join two subsets into a single subset. When evaluating an edge between Node A and Node B, we simply call Find(A) and Find(B). If they return the same root, they are already in the same connected component, meaning adding an edge between them would create a cycle. If they return different roots, it is safe to add the edge, and we call Union(A, B). Kruskal's Algorithm in Python class UnionFind: def __init__(self, size): # Initially, every node is its own parent (root) self.parent = [i for i in range(size)] self.rank = [0] * size def find(self, i): # Path compression optimization if self.parent[i] == i: return i self.parent[i] = self.find(self.parent[i]) return self.parent[i] def union(self, i, j): root_i = self.find(i) root_j = self.find(j) if root_i != root_j: # Union by rank optimization if self.rank[root_i] < self.rank[root_j]: self.parent[root_i] = root_j elif self.rank[root_i] > self.rank[root_j]: self.parent[root_j] = root_i else: self.parent[root_j] = root_i self.rank[root_i] += 1 return True return False def kruskals_algorithm(vertices_count, edges): # edges is a list of tuples: [(weight, u, v), ...] # Step 1: Sort all edges in non-decreasing order of their weight edges.sort() uf = UnionFind(vertices_count) mst = [] total_cost = 0 # Step 2: Iterate through the sorted edges for edge in edges: weight, u, v = edge # If including this edge does not cause a cycle, include it if uf.union(u, v): mst.append((u, v, weight)) total_cost += weight # Optimization: Stop early if we have V-1 edges if len(mst) == vertices_count - 1: break return mst, total_cost Kruskal's works on the edges directly: sort them, then add each one unless it would close a cycle (like A-B here). It reaches the same weight-13 tree. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Prim vs. Kruskal: Which one to choose? While both algorithms are guaranteed to find the exact same minimum spanning tree weight, their performance varies significantly based on the topology of the graph being processed. There is a third option too: Boruvka's algorithm builds the MST from many fronts at once and parallelizes better than either. Metric Prim's Algorithm Kruskal's Algorithm Data Structure Priority Queue (Min-Heap) Disjoint Set (Union-Find) Time Complexity O(E log V) using a Binary Heap. O(E log E) or O(E log V) heavily dominated by sorting the edges. Graph Density Excellent for Dense Graphs. Since it only looks at adjacent edges of visited nodes, it performs much better when the number of edges E is close to V². Excellent for Sparse Graphs. Since the first step is sorting all edges, having fewer edges makes the sort blazingly fast. Implementation Complexity Can be slightly more complex due to managing the Priority Queue and tracking visited states. Very straightforward to implement, provided you have a pre-written Union-Find class. Growth Pattern Grows a single, contiguous tree. Grows a forest of disjoint trees that eventually merge together. 6. Real World Applications of MSTs The Minimum Spanning Tree is not just a theoretical construct; it is actively used in various engineering disciplines to minimize costs and optimize routing. - Network Design: Telecommunications, electrical grids, water supply networks, and computer networks use MST algorithms to ensure all nodes are connected with the minimum amount of physical wire or pipe. - Approximation Algorithms: MSTs are frequently used as a stepping stone to solve harder NP-Hard problems, such as the Traveling Salesperson Problem (TSP) approximation. - Cluster Analysis: In machine learning and data mining, Single-Linkage Clustering relies on Kruskal's algorithm to group data points based on their shortest distance to other clusters. - Image Segmentation: MST algorithms can be used to partition an image into distinct regions or objects based on pixel similarity. Frequently Asked Questions What is a Minimum Spanning Tree (MST)? An MST is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. How does Union-Find prevent cycles in Kruskal's algorithm? Union-Find tracks connected components. Before adding an edge between u and v, Kruskal's checks if they belong to the same set. If they do, adding the edge would create a cycle, so it is skipped. Can a graph have more than one MST? Yes, a graph can have multiple MSTs if there are multiple edges with the same weight. If all edge weights in the graph are unique, there is only one unique MST. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Traveling Salesperson Problem (TSP) Source: https://learngraphtheory.org/articles/traveling-salesperson-problem.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is the Traveling Salesperson Problem? - 2. The Curse of Dimensionality: Combinatorial Explosion - 3. P vs NP and NP-Hardness - 4. Exact Algorithms: Held-Karp and Branch-and-Bound - 5. Heuristics and Approximations: Good Enough, Fast Enough - 6. Real-World Applications of TSP - 7. Frequently Asked Questions (FAQ) 1. What is the Traveling Salesperson Problem? The Traveling Salesperson Problem (TSP) is a classic algorithmic problem in the field of computer science and operations research. The problem statement is deceptively simple to understand: "Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?" In the language of Graph Theory, the TSP is formulated as follows: Given a complete, weighted graph (where vertices represent cities, edges represent roads, and weights represent distances), find a Hamiltonian Cycle of minimum total weight. A Hamiltonian Cycle is a closed loop that visits every vertex in a graph exactly once. While calculating the distance of a single route involves simple addition, finding the absolute shortest route among all possible routes is what makes this problem notoriously difficult. The faint lines are all the possible roads between cities (a complete graph). TSP picks the bold loop: the shortest tour that visits all seven cities once and returns Home. 2. The Curse of Dimensionality: Combinatorial Explosion Why can't we just use a computer to check every single route, calculate the total distance for each, and then pick the shortest one? This approach is called Brute Force. The issue with Brute Force is a mathematical phenomenon known as combinatorial explosion. The number of possible routes scales factorially with the number of cities. - If you have 5 cities, there are (5 - 1)! / 2 = 12 possible routes. A human can calculate this on paper in a few minutes. - If you have 10 cities, there are 181,440 possible routes. A modern computer can calculate this in a fraction of a millisecond. - If you have 20 cities, there are 60,822,550,204,416,000 possible routes. This would take a standard computer a few years to compute. - If you have 61 cities, there are more possible routes than there are atoms in the observable universe. Because the number of possible solutions grows at an astronomical O(n!) rate, brute-forcing the TSP is physically impossible for any significant number of cities, regardless of how fast supercomputers become. The number of distinct tours is (n-1)!/2. Note the logarithmic axis: each step of five cities adds several orders of magnitude. Traveling Salesperson Problem (TSP) Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. P vs NP and NP-Hardness The Traveling Salesperson Problem is classified as NP-Hard. Understanding what this means requires a brief look at computational complexity theory, specifically the P vs NP problem. - P (Polynomial Time): Problems that can be solved relatively quickly (in polynomial time) by a computer. Examples include sorting an array or finding the shortest path between just two points (Dijkstra's algorithm). - NP (Nondeterministic Polynomial Time): Problems where, if you are given a potential solution, you can verify if that solution is correct quickly (in polynomial time). The decision version of TSP ("Does a route exist that is shorter than length L?") is in NP-Complete. The optimization version ("What is the absolute shortest route?") is NP-Hard. This implies that computer scientists strongly believe there is no algorithm that can find the exact shortest route for all cases of TSP in polynomial time. If anyone ever discovers a fast exact algorithm for TSP, they would prove that P = NP, winning a $1,000,000 Millennium Prize and fundamentally altering the foundation of computer security and mathematics. 4. Exact Algorithms: Holding Out for the Perfect Route Despite the NP-Hard classification, researchers have developed clever algorithms that find the exact optimal solution much faster than brute force, though they still struggle as the number of cities grows large. The Held-Karp Algorithm (Dynamic Programming) Developed in 1962, the Held-Karp algorithm uses Dynamic Programming to solve the TSP exactly in O(n^2 * 2^n) time. While 2^n is still exponential (and therefore very slow for large inputs), it is drastically faster than the O(n!) time of brute force. Held-Karp works by breaking the problem down into smaller overlapping subproblems. It calculates the shortest path from the start to a subset of cities, stores that result in memory (memoization), and reuses it to build the paths to larger subsets. However, because it must store results for every subset of cities, it requires O(n * 2^n) memory, which quickly becomes the limiting factor before processing power does. Held-Karp in Python The state is a pair: the set of cities already visited, and the city you are standing in. There are 2^n such sets and n cities to stand in, which is where the O(n^2 * 2^n) comes from. Slow, but a colossal improvement on checking every permutation. from itertools import combinations def held_karp(dist): """Exact TSP by dynamic programming. Returns (length, tour).""" n = len(dist) # best[(subset, j)] = cheapest way to start at city 0, visit exactly the # cities in subset, and finish standing in city j. best = {} parent = {} for j in range(1, n): best[(frozenset([j]), j)] = dist[0][j] for size in range(2, n): for subset in map(frozenset, combinations(range(1, n), size)): for j in subset: prev = subset - {j} cost, who = min( (best[(prev, k)] + dist[k][j], k) for k in prev ) best[(subset, j)] = cost parent[(subset, j)] = who full = frozenset(range(1, n)) length, last = min((best[(full, j)] + dist[j][0], j) for j in range(1, n)) # Walk the parent pointers back to rebuild the tour tour, subset, j = [0], full, last while j is not None: tour.append(j) j, subset = parent.get((subset, j)), subset - {j} tour.append(0) return length, list(reversed(tour)) # Five cities at (12,7), (4,13), (8,5), (13,2), (17,4), distances rounded. dist = [ [0, 10, 4, 5, 6], [10, 0, 9, 14, 16], [4, 9, 0, 6, 9], [5, 14, 6, 0, 4], [6, 16, 9, 4, 0], ] print(held_karp(dist)) # (35, [0, 4, 3, 2, 1, 0]) -> A E D C B A, the optimal loop # For n = 5 this fills 80 states instead of checking 24 permutations. The gap # widens fast: at n = 20 it is about 10 million states against 10^17 tours. Branch and Bound Branch and Bound is another exact approach. It systematically enumerates candidate solutions using a tree structure. Before exploring a branch of the tree (a partial route), it calculates a "bound" (an optimistic estimate of the shortest possible route that could result from this branch). If this bound is already worse than the best route found so far, the algorithm "prunes" the branch, entirely skipping the evaluation of millions of useless routes. Visualize TSP Solvers Watch how different algorithms approach the same map of cities. See the brute force algorithm struggle, while heuristics find near-perfect routes in seconds. Open the TSP Visualizer Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Heuristics and Approximations: Good Enough, Fast Enough In the real world, companies like FedEx or Amazon do not need the absolute mathematically perfect route. A route that is 99% optimal but can be calculated in 5 seconds is infinitely more valuable than a 100% optimal route that takes 5,000 years to compute. This is where heuristics come in. Nearest Neighbor (Greedy Algorithm) The simplest heuristic is the Nearest Neighbor algorithm: Start at a random city, go to the nearest unvisited city, and repeat until you return home. It is incredibly fast O(n^2), but it can result in a "greedy trap," where taking short hops early on forces the salesperson to make massive cross-country jumps at the very end. It typically yields routes 25% longer than the optimal. Nearest Neighbor in Python It is about ten lines and runs in O(n^2). On the five-city instance from the previous section it lands 14% off the optimum, and that gap is the entire trade: an instant answer in exchange for giving up the guarantee. def nearest_neighbor(dist, start=0): """Greedy TSP tour: always hop to the closest city not yet visited.""" n = len(dist) unvisited = set(range(n)) - {start} tour = [start] current = start while unvisited: nxt = min(unvisited, key=lambda j: dist[current][j]) tour.append(nxt) unvisited.remove(nxt) current = nxt tour.append(start) # close the loop return tour def tour_length(dist, tour): return sum(dist[tour[i]][tour[i + 1]] for i in range(len(tour) - 1)) # The same five cities used for Held-Karp above. dist = [ [0, 10, 4, 5, 6], [10, 0, 9, 14, 16], [4, 9, 0, 6, 9], [5, 14, 6, 0, 4], [6, 16, 9, 4, 0], ] tour = nearest_neighbor(dist) print(tour, tour_length(dist, tour)) # [0, 2, 3, 4, 1, 0] 40 -> A C D E B A # The exact answer is 35, so this tour is 14% too long. Nearest Neighbor took # the three cheap edges around A first, stranded city B, and then had to pay # 16 and 10 to get home. Cheap early choices are exactly how it goes wrong. 2-Opt Optimization 2-Opt is a local search algorithm. It takes an existing, possibly messy route, and attempts to untangle it. It looks at two edges (roads) in the route. If the roads cross each other, 2-Opt deletes those two edges and reconnects the cities in a way that removes the crossing. By repeatedly removing crossings, 2-Opt can take a poorly optimized route and turn it into a highly efficient one. 2-Opt in one move: two crossing edges are swapped so the path no longer overlaps itself, cutting the tour from about 1313 to about 1016. Ant Colony Optimization (ACO) Inspired by nature, ACO simulates a colony of ants searching for food. Simulated "ants" wander the graph randomly. When an ant completes a tour, it lays down "pheromone" on the edges it traversed. The shorter the tour, the stronger the pheromone it leaves. Over time, subsequent ants are probabilistically drawn to edges with stronger pheromones. Through this decentralized positive feedback loop, the colony converges on highly optimized routes. 6. Real-World Applications of TSP The mathematics of TSP apply far beyond a salesperson driving a car. - Logistics and Delivery: The most obvious application. Companies routing delivery trucks, school buses, or garbage collection rely on massive TSP solvers to save millions of dollars in fuel and time. - Manufacturing (Drilling Printed Circuit Boards): To manufacture a motherboard, a robotic drill must drill tens of thousands of tiny holes. Treating the holes as "cities", solving the TSP minimizes the distance the drill head must move, drastically speeding up production. - Astronomy: Telescopes capturing images of different stars or galaxies must point at various locations in the sky. Formulating the observation schedule as a TSP minimizes the time the telescope spends re-orienting itself. - DNA Sequencing: In bioinformatics, when piecing together fragmented DNA strands, the "cities" are DNA fragments, and the "distance" is a measure of how poorly they match. Solving a variation of TSP helps find the most likely sequence of the complete genome. Frequently Asked Questions What is the Traveling Salesperson Problem (TSP)? TSP is the problem of finding the shortest possible route that visits a set of cities exactly once and returns to the starting city. What makes TSP so difficult to solve exactly? TSP is NP-hard. The number of possible routes grows factorially, O(n!), leading to a combinatorial explosion. For 20 cities, there are already trillions of routes to search. What is the 2-Opt heuristic? 2-Opt is a local search heuristic that improves a TSP route by repeatedly taking two edges that cross each other, deleting them, and reconnecting them to untangle the route. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Vehicle Routing Problem (VRP) Source: https://learngraphtheory.org/articles/vehicle-routing-problem.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is the Vehicle Routing Problem? - 2. VRP vs. TSP: Understanding the Difference - 3. Common Variations of VRP (CVRP, VRPTW) - 4. How is the VRP Solved? - 5. The Clarke and Wright Savings Algorithm - 6. Real-World Applications - 7. Frequently Asked Questions (FAQ) 1. What is the Vehicle Routing Problem? The Vehicle Routing Problem (VRP) is a combinatorial optimization and integer programming problem that asks: "What is the optimal set of routes for a fleet of vehicles to traverse in order to deliver to a given set of customers?" Introduced in 1959 by George Dantzig and John Ramser, the VRP is a central problem in the fields of transportation, distribution, and logistics. The goal is typically to minimize the total route cost (which could mean minimizing distance, time, fuel, or the number of vehicles used) while ensuring that every customer is visited exactly once. In a standard VRP: - There is a central Depot where all vehicles start and end their journeys. - There is a Fleet of vehicles, all assumed to be identical in the base problem. - There is a set of Customers, each located at a specific node on a graph. 2. VRP vs. TSP: Understanding the Difference If you are familiar with graph theory, the VRP sounds suspiciously like the Traveling Salesperson Problem (TSP). In fact, the VRP is a generalization of the TSP. The core difference is the number of actors: - TSP: You have one salesperson who must visit every city and return home. - VRP: You have multiple vehicles. You must first cluster the cities (assigning a specific group of cities to Vehicle A, another group to Vehicle B), and then solve a mini-TSP for each vehicle. Because the TSP is already NP-Hard (computationally intractable for exact solutions at large scales), the VRP is also NP-Hard, but significantly more complex due to the added clustering dimension. TSP is one loop through every stop. VRP first clusters the stops, then runs a route for each vehicle out of a shared depot. Vehicle Routing Problem (VRP) Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. Common Variations of VRP The basic VRP assumes infinite vehicle capacity and no time constraints. In the real world, a FedEx truck cannot carry infinite packages. This has led to the creation of several highly practical variations of the VRP. Capacitated Vehicle Routing Problem (CVRP) In the CVRP, every vehicle has a maximum carrying capacity (e.g., a truck can hold 500 lbs), and every customer has a specific demand (e.g., Customer A needs 50 lbs of goods). A vehicle's route must end, and it must return to the depot, before the sum of the demands on its route exceeds its capacity. This is the most common variation studied in operations research. In the CVRP, the blue route fills the truck to its 100-unit limit and must head back to the depot, so a second route is needed for the remaining demand. Vehicle Routing Problem with Time Windows (VRPTW) In the VRPTW, customers require deliveries within specific time frames. For example, a restaurant might need its ingredients delivered between 8:00 AM and 10:00 AM. If a truck arrives at 7:30 AM, it must wait. If it arrives at 10:15 AM, the delivery fails. This adds severe timing constraints to the routing logic. Other Notable Variations - VRPPD (Pickup and Delivery): Vehicles don't just drop off goods; they pick them up from customers and deliver them to other customers (like Uber or DoorDash). - MDVRP (Multi-Depot VRP): A company has multiple warehouses (depots). The algorithm must decide not only the routes but also which depot serves which customer. - SDVRP (Split Delivery VRP): A customer's demand is so large that it can be split and fulfilled by multiple different vehicles. Interactive VRP Simulator Watch a fleet of vehicles navigate a city grid. Add capacity constraints and watch how the algorithm forces trucks back to the depot when they run out of space. Launch the VRP Visualizer 4. How is the VRP Solved? Because the VRP is NP-Hard, calculating the absolute perfect set of routes for hundreds of customers is practically impossible. Instead, logistics companies rely on a two-phase approach to find "near-optimal" routes very quickly. Phase 1: Cluster First, Route Second The most intuitive human approach is also used by algorithms. First, group the customers into clusters based on geographical proximity and vehicle capacity. Once you have a cluster of customers assigned to a specific truck, you treat that cluster as a standalone Traveling Salesperson Problem and solve for the shortest path. Phase 2: Metaheuristics Once an initial set of routes is established, algorithms attempt to continuously improve them. These are known as metaheuristics. - Simulated Annealing: The algorithm occasionally accepts "worse" routes in the short term to escape local optimums, slowly "cooling down" to settle on a globally efficient route. - Genetic Algorithms: Takes multiple good routing solutions, "breeds" them together by swapping sections of routes, and keeps the "fittest" (shortest) offspring. - Tabu Search: An aggressive local search that keeps a memory (a tabu list) of recently evaluated routes to prevent the algorithm from getting stuck in infinite loops. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. The Clarke and Wright Savings Algorithm If you want to write a VRP solver, the best place to start is the Clarke and Wright Savings Algorithm, developed in 1964. It remains one of the most famous and widely implemented heuristics for the Capacitated VRP (CVRP). The logic is based on the concept of "savings." - Start with the worst case: Imagine you have 10 customers. You send 10 separate trucks out from the depot. Truck 1 goes to Customer 1 and returns. Truck 2 goes to Customer 2 and returns, etc. - Calculate Savings: If you combine the routes (Depot -> Customer 1 -> Customer 2 -> Depot), how much distance do you save compared to sending two separate trucks? You calculate this "savings" value for every possible pair of customers. - Sort and Merge: You sort the pairs by the highest savings. You then aggressively merge routes together, starting with the biggest savings, provided that the combined route does not exceed the vehicle's maximum capacity. The savings for merging A and B is d(Depot,A) + d(Depot,B) − d(A,B). Clarke-Wright merges the pairs with the largest savings first, as long as capacity allows. Clarke-Wright in Python The whole heuristic is a sort followed by a greedy merge loop, and the capacity check inside that loop is the single line that separates a VRP from a TSP. def clarke_wright(dist, demand, capacity): """Savings heuristic for the capacitated VRP. dist[0] is the depot.""" n = len(dist) # Step 1: one out-and-back trip per customer routes = {c: [c] for c in range(1, n)} load = {c: demand[c] for c in range(1, n)} def route_of(c): return next((k for k, r in routes.items() if c in r), None) # Step 2: how much is saved by serving i and j on one trip instead of two savings = sorted( ((dist[0][i] + dist[0][j] - dist[i][j], i, j) for i in range(1, n) for j in range(i + 1, n)), reverse=True, ) # Step 3: merge greedily, best saving first for _, i, j in savings: ri, rj = route_of(i), route_of(j) if ri is None or rj is None or ri == rj: continue if load[ri] + load[rj] > capacity: continue # the constraint that makes this VRP and not TSP a, b = routes[ri], routes[rj] # i and j each have to sit at an end of their own route if a[-1] == i and b[0] == j: merged = a + b elif b[-1] == j and a[0] == i: merged = b + a elif a[0] == i and b[0] == j: merged = a[::-1] + b elif a[-1] == i and b[-1] == j: merged = a + b[::-1] else: continue routes[ri], load[ri] = merged, load[ri] + load[rj] del routes[rj], load[rj] return [[0] + r + [0] for r in routes.values()] # A depot at (0,0) and five customers scattered around it. dist = [ [0, 5, 4, 5, 4, 5], [5, 0, 3, 5, 9, 6], [4, 3, 0, 7, 8, 3], [5, 5, 7, 0, 8, 10], [4, 9, 8, 8, 0, 8], [5, 6, 3, 10, 8, 0], ] demand = [0, 4, 3, 5, 3, 4] # depot first, then one figure per customer routes = clarke_wright(dist, demand, capacity=10) print(routes) # [[0, 1, 2, 0], [0, 3, 4, 0], [0, 5, 0]] # loads 7, 8 and 4, all within the 10-unit vehicle capacity # Total distance 39. Sending one van per customer would cost 46, so the # savings merges cut 15% off. Customer 5 stays on its own trip only because # adding it to the first route would need 11 units of capacity. This algorithm runs in O(n^2 log n) time and produces highly efficient routes almost instantly, serving as the baseline upon which metaheuristics can improve. 6. Real-World Applications A 5% improvement in VRP optimization translates to millions of dollars in savings for major corporations. - Package Delivery (UPS/FedEx): UPS famously uses an optimization system called ORION (On-Road Integrated Optimization and Navigation). By heavily penalizing left-hand turns in their VRP calculations (which waste gas while waiting at traffic lights), UPS saves over 10 million gallons of fuel a year. - Ridesharing (Uber/Lyft): UberPool is a massive, real-time Dynamic VRP with Pickup and Delivery. The algorithm must continuously re-cluster and re-route vehicles as new ride requests appear dynamically. - Waste Management: Garbage trucks must route through every street in a city (a subset of VRP known as the Capacitated Arc Routing Problem), ensuring the trucks return to the dump before hitting their weight limits. Frequently Asked Questions What is the difference between TSP and the Vehicle Routing Problem (VRP)? TSP asks for a single route visiting all cities. VRP generalizes this to finding optimal routes for a fleet of multiple vehicles starting from a central depot to service customers. What is Capacitated Vehicle Routing (CVRP)? CVRP is a VRP variant where each vehicle has a maximum carrying capacity. Vehicles must return to the depot to reload once they reach their capacity limit. How do modern delivery companies solve VRP? Because VRP is NP-hard, companies use metaheuristics (like Tabu Search, Genetic Algorithms, or Simulated Annealing) combined with specialized routing software to calculate routes in minutes. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Max-Flow Min-Cut Theorem & Network Flow Source: https://learngraphtheory.org/articles/network-flow-max-flow-min-cut.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is a Flow Network? - 2. The Three Rules of Network Flow - 3. The Ford-Fulkerson Algorithm (Augmenting Paths) - 4. The Secret Weapon: Residual Graphs - 5. The Max-Flow Min-Cut Theorem (Symmetry in Math) - 6. Improving Ford-Fulkerson: The Edmonds-Karp Algorithm - 7. Real-World Applications (Bipartite Matching) 1. What is a Flow Network? Imagine a complex city water system. You have a main water treatment plant pushing water out, and a main reservoir where all the water eventually ends up. Between them lies a massive network of pipes of varying sizes. Some pipes are massive water mains (high capacity), while others are small residential pipes (low capacity). The question is: What is the maximum amount of water you can pump from the plant to the reservoir per second without bursting any pipes? In graph theory, this is modeled as a Flow Network. A flow network is a directed graph where: - Source (s): The starting node where the "flow" originates (the water plant). - Sink (t): The destination node where all the "flow" ends up (the reservoir). - Edges: The connections between nodes (the pipes). - Capacity (c): The maximum amount of flow an edge can handle (the width of the pipe). A flow network from source S to sink T. Each edge is labeled flow / capacity; saturated edges (flow = capacity) are drawn in bold blue. This assignment achieves the maximum flow of 19. 2. The Three Rules of Network Flow To mathematically define a valid flow through this network, we must follow three absolute rules: - Capacity Constraint: The flow on any edge cannot exceed its capacity. If a pipe can hold 10 gallons per second, you can't push 11 gallons through it. Mathematically: 0 ≤ f(u,v) ≤ c(u,v). - Conservation of Flow: For every node in the graph (except the Source and the Sink), the total flow entering the node must exactly equal the total flow leaving the node. Nodes don't magically generate or consume water. What goes in must come out. - Skew Symmetry (Optional but helpful context): The flow from node U to V is the negative of the flow from V to U. If 5 units flow from U to V, then -5 units flow from V to U. The goal of the Maximum Flow Problem is to find a valid assignment of flow to every edge that maximizes the total flow leaving the Source s (which, due to conservation of flow, will perfectly equal the total flow arriving at the Sink t). 3. The Ford-Fulkerson Algorithm (Augmenting Paths) To solve the maximum flow problem, we use the Ford-Fulkerson algorithm, developed in 1956. The logic behind it is delightfully intuitive. The algorithm works like this: - Start with a flow of 0 on all edges. - Find a path from the Source to the Sink where every edge in the path has available, unused capacity. This is called an Augmenting Path. - Find the edge on this path with the smallest available capacity. This is the "bottleneck" edge. - Push an amount of flow equal to the bottleneck capacity along the entire path. - Repeat steps 2-4 until no more augmenting paths can be found. When you can no longer find a path from the Source to the Sink that can accept more flow, you have found the Maximum Flow. But wait, there's a catch! Max-Flow Min-Cut Theorem & Network Flow Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 4. The Secret Weapon: Residual Graphs If you implement Ford-Fulkerson exactly as described above, you might get the wrong answer. Why? Because you might make a "bad" greedy choice early on, sending flow down a pipe that blocks a much better route later. To fix this, Ford-Fulkerson utilizes a brilliant concept called the Residual Graph. The residual graph allows the algorithm to "undo" bad decisions. Whenever you push X units of flow forward along an edge from U to V, you must add a "backward edge" in the residual graph from V to U with a capacity of X. This backward edge represents your ability to "push back" or cancel out the flow you just sent. If a future augmenting path utilizes one of these backward edges, it is effectively redirecting the water you previously sent down a different, more optimal pipe. You must always search for your augmenting paths in the Residual Graph, not the original graph. A greedy augmenting path can get stuck (left). The residual graph adds a backward edge v→u, letting the algorithm push flow back and redirect it to reach the true maximum (right). Interactive Flow Networks Watch the Ford-Fulkerson algorithm dynamically build the residual graph and find augmenting paths. See how "pushing back" flow allows the algorithm to correct early greedy mistakes. Launch Max Flow Visualizer 5. The Max-Flow Min-Cut Theorem This brings us to one of the most beautiful and profound theorems in all of graph theory: The Max-Flow Min-Cut Theorem. Imagine you want to completely sabotage the city's water system. You want to sever a set of pipes such that absolutely zero water can reach the reservoir from the treatment plant. Naturally, you want to do this with the least amount of effort, meaning you want to sever pipes whose total combined capacity is as small as possible. This is called a Cut. An s-t cut partitions the graph's nodes into two sets: one containing the Source (s) and one containing the Sink (t). The capacity of the cut is the sum of the capacities of all edges going from the source set to the sink set. The Minimum Cut is the cut with the smallest possible total capacity. The theorem states an incredible equivalence: The maximum amount of flow you can push through a network is EXACTLY EQUAL to the capacity of the minimum cut. They are two sides of the exact same coin. The bottleneck that restricts your flow is exactly the same bottleneck you would target to sever the network. By solving for Max-Flow (using Ford-Fulkerson), you are simultaneously finding the exact value of the Min-Cut. The dashed line is a minimum cut, splitting the nodes into {S, A, B, D} and {C, T}. It severs edges A→C and D→T, whose capacities sum to 9 + 10 = 19, exactly matching the maximum flow. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 6. Improving Ford-Fulkerson: The Edmonds-Karp Algorithm Ford-Fulkerson has a flaw: it doesn't tell you how to find the augmenting path. If you just use an arbitrary Depth First Search (DFS), and your graph has very specific edge weights, the algorithm can run incredibly slowly. In fact, with irrational edge capacities, plain Ford-Fulkerson might never even terminate! In 1972, Jack Edmonds and Richard Karp published a simple but brilliant modification: Always find the shortest augmenting path using Breadth-First Search (BFS). By forcing the algorithm to find augmenting paths with the fewest number of edges (ignoring capacity), the Edmonds-Karp Algorithm guarantees a polynomial time complexity of O(V * E^2), entirely independent of the actual capacity values on the edges. Edmonds-Karp in Python The implementation below is Ford-Fulkerson with one BFS at its heart, and it returns the minimum cut alongside the flow. That second return value is the theorem made concrete: once no augmenting path remains, the vertices still reachable from the source are exactly the source side of the cut. from collections import deque def edmonds_karp(capacity, source, sink): """Max flow by Ford-Fulkerson, always augmenting along the shortest path. capacity[u][v] is the capacity of the edge u -> v. Returns the max flow value together with the minimum cut that certifies it. """ # Residual graph: every edge gets a back edge starting at zero residual = {u: dict() for u in capacity} for u in capacity: for v, c in capacity[u].items(): residual[u][v] = residual[u].get(v, 0) + c residual.setdefault(v, dict()).setdefault(u, 0) max_flow = 0 while True: # BFS, not DFS: this is the whole Edmonds-Karp idea. Choosing the # shortest augmenting path is what bounds the work at O(V * E^2). parent = {source: None} queue = deque([source]) while queue and sink not in parent: u = queue.popleft() for v, c in residual[u].items(): if v not in parent and c > 0: parent[v] = u queue.append(v) if sink not in parent: break # no augmenting path left, the flow is maximum # Push the bottleneck capacity along the path we just found bottleneck, v = float("inf"), sink while parent[v] is not None: bottleneck = min(bottleneck, residual[parent[v]][v]) v = parent[v] v = sink while parent[v] is not None: residual[parent[v]][v] -= bottleneck residual[v][parent[v]] += bottleneck v = parent[v] max_flow += bottleneck # The min cut falls out for free: everything still reachable from the # source in the residual graph is on the source side of the cut. seen, stack = {source}, [source] while stack: u = stack.pop() for v, c in residual[u].items(): if v not in seen and c > 0: seen.add(v) stack.append(v) cut = [(u, v, c) for u in capacity for v, c in capacity[u].items() if u in seen and v not in seen] return max_flow, sorted(seen), cut capacity = { 's': {'v1': 16, 'v2': 13}, 'v1': {'v3': 12}, 'v2': {'v1': 4, 'v4': 14}, 'v3': {'v2': 9, 't': 20}, 'v4': {'v3': 7, 't': 4}, 't': {}, } flow, source_side, cut = edmonds_karp(capacity, 's', 't') print(flow) # 23 print(source_side) # ['s', 'v1', 'v2', 'v4'] print(cut) # [('v1', 'v3', 12), ('v4', 'v3', 7), ('v4', 't', 4)] print(sum(c for _, _, c in cut)) # 23 # The cut capacity equals the max flow, exactly as the theorem promises. Those # three edges are the bottleneck: widening anything else changes nothing. 7. Real-World Applications Network flow algorithms aren't just for plumbing. They solve incredibly complex assignment and routing problems in software engineering and operations research. Maximum Bipartite Matching Imagine you have 5 job applicants and 5 available jobs. Each applicant is only qualified for certain jobs. How do you assign the maximum number of people to a job they are qualified for? You can turn this into a Max-Flow problem! Create a "Source" node and connect it to all applicants with capacity 1. Connect the applicants to the jobs they are qualified for with capacity 1. Connect all jobs to a "Sink" node with capacity 1. Run Ford-Fulkerson. The maximum flow will perfectly equal the maximum number of people you can successfully employ. Image Segmentation in Computer Vision In computer vision, separating an object (the foreground) from its background is a classic problem. By representing pixels as a graph, where edges represent the color similarity between neighboring pixels, the problem of cutting the foreground from the background can be mapped perfectly to the Minimum-Cut problem. Sports Elimination During a baseball season, can you mathematically prove that a team is eliminated from the playoffs, even if they win all their remaining games? By creating a flow network representing the remaining games between all other teams, you can use Max-Flow to prove whether a scenario exists where the team could still win the division. Frequently Asked Questions What is the Max-Flow Min-Cut Theorem? The theorem states that in a flow network, the maximum amount of flow passing from a source to a sink is equal to the total weight of the edges in a minimum cut that separates them. What is the difference between Ford-Fulkerson and Edmonds-Karp? Ford-Fulkerson is a method that uses DFS or BFS to find augmenting paths, running in O(E * f) time. Edmonds-Karp is an implementation of Ford-Fulkerson that strictly uses BFS, guaranteeing a polynomial time complexity of O(V * E²). What is a residual graph in network flow? A residual graph represents the remaining capacity of edges in the network. It tracks both forward capacity (remaining allowable flow) and backward capacity (flow that can be redirected or canceled). Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Graph Coloring Problem Source: https://learngraphtheory.org/articles/graph-coloring-problem.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is the Graph Coloring Problem? - 2. The Chromatic Number (and Bipartite Graphs) - 3. The Famous Four Color Theorem - 4. How to Color a Graph (Algorithms) - 5. Real-World Applications (Scheduling, Compilers) - 6. Why Sudoku is Just a Graph Coloring Puzzle - 7. Frequently Asked Questions (FAQ) 1. What is the Graph Coloring Problem? Graph coloring is exactly what it sounds like: assigning colors to certain elements of a graph subject to specific constraints. By far the most common type of graph coloring is Vertex Coloring. The rules of Vertex Coloring are remarkably simple: - You must assign a color to every vertex (node) in the graph. - No two adjacent vertices can share the same color. If there is an edge connecting Node A to Node B, they must be different colors. - The goal is to use the absolute minimum number of colors possible. While the rules are simple enough for a child to understand, finding the minimum number of colors for a large, complex graph is incredibly difficult. In fact, finding the exact minimum is an NP-Complete problem, meaning there is no known fast algorithm to solve it perfectly for all graphs. 2. The Chromatic Number (and Bipartite Graphs) The absolute minimum number of colors needed to color a specific graph is called its Chromatic Number, usually denoted by the Greek letter Chi χ(G). Let's look at a few examples to understand chromatic numbers: - A Graph with no edges: You can color every node the same color. χ = 1. - A Triangle (Complete Graph K3): Every node connects to every other node. Node A is Red, Node B must be Blue, and Node C must be Green. χ = 3. - A Star Graph: One central node connected to many outer nodes. The center is Red. Because the outer nodes don't connect to each other, they can all be Blue. χ = 2. Bipartite Graphs Any graph that can be colored using exactly two colors (χ = 2) is called a Bipartite Graph. This is a massive concept in graph theory. If a graph is bipartite, you can split its vertices into two distinct sets (like "Red nodes" and "Blue nodes") where edges only ever cross between the sets, never within them. If a graph contains a cycle with an odd number of nodes (like a triangle), it can never be bipartite. A coloring is proper only when no edge joins two nodes of the same color. This graph contains a triangle, so it cannot be done with fewer than 3 colors: χ = 3. Graph Coloring Problem Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. The Famous Four Color Theorem Historically, graph coloring originated from mapmaking. In 1852, Francis Guthrie was trying to color a map of the counties of England and noticed that he only needed four colors to ensure that no two bordering counties shared a color. He asked his brother, a mathematician, if this was a universal rule. This became the Four Color Problem: Given any separation of a plane into contiguous regions, producing a figure called a map, no more than four colors are required to color the regions of the map so that no two adjacent regions have the same color. To view a map as a graph, you simply treat each country as a Node, and draw an Edge between two countries if they share a border. The theorem states that any planar graph (a graph that can be drawn on a flat piece of paper without any edges crossing) has a chromatic number of at most 4. It took over a century to prove. In 1976, Kenneth Appel and Wolfgang Haken finally proved the Four Color Theorem using a computer program to check 1,936 specific configurations. It was the first major mathematical theorem to be proved using a computer, causing massive philosophical debates among mathematicians at the time. A map becomes a graph by turning regions into nodes and shared borders into edges. This arrangement genuinely needs all four colors, and the Four Color Theorem says no flat map ever needs more. Interactive Graph Coloring Try to manually color a complex graph using the fewest colors possible, or watch the Greedy Algorithm do it instantly. Can you find a graph that requires 5 colors? Launch the Coloring Visualizer 4. How to Color a Graph (Algorithms) Because finding the perfect chromatic number is NP-Complete, we rarely try to find the perfect solution in software engineering. Instead, we use heuristics to find a good solution very quickly. The Greedy Algorithm The simplest approach is the Greedy Algorithm. It doesn't guarantee the absolute minimum number of colors, but it guarantees it will never use more colors than the maximum degree of a vertex plus one (d + 1). - Order the vertices of the graph (V1, V2, ... Vn). - Assign the first available color (Color 1) to the first vertex (V1). - For every subsequent vertex, look at all its previously colored neighbors. - Assign the current vertex the lowest numbered color that is not currently used by any of its neighbors. - If all previously used colors are taken by neighbors, introduce a new color. The Catch: The number of colors the Greedy Algorithm uses heavily depends on the order of the vertices. If you order them poorly, it uses too many colors. This led to the Welsh-Powell Algorithm, which simply says: Sort the vertices in descending order based on their degree (number of connections) before running the Greedy Algorithm. This small tweak yields drastically better results. Greedy Coloring in Python That sensitivity to ordering is easy to see for yourself. The graph below is bipartite, so two colors are always enough, but the greedy algorithm only finds that answer when the vertices arrive in a helpful order. def greedy_coloring(graph, order): """Color vertices in the given order, always taking the lowest free color.""" color = {} for v in order: used = {color[n] for n in graph[v] if n in color} c = 0 while c in used: c += 1 color[v] = c return color # A bipartite graph: every edge joins an 'A' vertex to a 'B' vertex, # so two colors are always sufficient. graph = { 'A1': ['B2', 'B3'], 'A2': ['B1', 'B3'], 'A3': ['B1', 'B2'], 'B1': ['A2', 'A3'], 'B2': ['A1', 'A3'], 'B3': ['A1', 'A2'], } interleaved = ['A1', 'B1', 'A2', 'B2', 'A3', 'B3'] print(greedy_coloring(graph, interleaved)) # {'A1': 0, 'B1': 0, 'A2': 1, 'B2': 1, 'A3': 2, 'B3': 2} -> 3 colors side_by_side = ['A1', 'A2', 'A3', 'B1', 'B2', 'B3'] print(greedy_coloring(graph, side_by_side)) # {'A1': 0, 'A2': 0, 'A3': 0, 'B1': 1, 'B2': 1, 'B3': 1} -> 2 colors, optimal # Same graph, same algorithm, one extra color purely because of vertex order. # Welsh-Powell is the standard fix: sort by descending degree before coloring. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Real-World Applications Graph coloring isn't just about making pretty maps. It solves massive logistical problems. 1. Scheduling Conflicts (Exams and Taxis) Imagine you are scheduling final exams for a university. You have hundreds of classes, and many students take multiple classes. If two classes share a student, their exams cannot be scheduled at the same time. - Nodes: The classes. - Edges: Connect two classes if they share at least one student. - Colors: The time slots. By coloring the graph, you find the minimum number of time slots needed to schedule all exams without any student having a conflict. 2. Compiler Register Allocation When a compiler translates your code (like C++ or Rust) into machine code, it has to assign your variables to the CPU's hardware registers. CPUs only have a tiny number of registers (e.g., 16 or 32). If two variables are used at the same time in your code, they cannot be stored in the same register. Compilers build an "interference graph" where variables are nodes, and edges connect variables whose lifespans overlap. Coloring the graph assigns variables to registers. If the chromatic number is higher than the available CPU registers, the compiler is forced to "spill" variables to the slower RAM. 3. Frequency Assignment Cell phone towers transmit signals on specific frequencies. If two towers are too close to each other, they cannot use the same frequency or they will cause interference. By representing towers as nodes and drawing edges between towers that overlap geographically, telecom companies use graph coloring to assign frequencies (colors) to towers using the minimum amount of bandwidth possible. 6. Why Sudoku is Just a Graph Coloring Puzzle If you've ever played Sudoku, you've been solving a Graph Coloring problem. In a standard 9x9 Sudoku board, there are 81 squares. You must fill them with numbers 1-9 such that no row, column, or 3x3 block contains a duplicate number. To convert this to a graph: - Create 81 Nodes (one for each square). - Draw an Edge between any two nodes if they share the same row, the same column, or the same 3x3 block. - The pre-filled numbers are nodes that are already "colored." - You have exactly 9 Colors (the numbers 1 through 9) to color the rest of the graph. Writing a Sudoku solver using a Graph Coloring algorithm with backtracking is a classic computer science assignment and highlights the incredible versatility of graph theory. A 4x4 example: filling a cell is coloring a node, and every cell in the same row, column, or box is a neighbour it cannot match. A full 9x9 board is the same idea with 9 colors. Frequently Asked Questions What is vertex coloring in graph theory? Vertex coloring is the assignment of colors to each vertex in a graph such that no two adjacent vertices (connected by an edge) share the same color. What is the chromatic number of a graph? The chromatic number (χ) is the minimum number of colors needed to color a graph properly. For example, a bipartite graph has a chromatic number of 2. What is the complexity of solving the graph coloring problem? Finding the chromatic number of a general graph is NP-complete, meaning there is no known polynomial-time algorithm to solve it exactly. Heuristics like the Greedy algorithm are used in practice. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## A* Search Algorithm: Step-by-Step Guide + Visualizer Source: https://learngraphtheory.org/articles/a-star-search-algorithm.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is the A* Algorithm? - 2. The Problem with Dijkstra's Algorithm - 3. The Secret Sauce: Heuristics (f = g + h) - 4. Choosing the Right Heuristic (Manhattan vs Euclidean) - 5. Step-by-Step Execution of A* - 6. Real-World Applications in Game Dev & AI - 7. Frequently Asked Questions (FAQ) 1. What is the A* Algorithm? The A* (pronounced "A-Star") algorithm is a graph traversal and path search algorithm that is widely used in computer science due to its completeness, optimality, and optimal efficiency. Invented in 1968 by Peter Hart, Nils Nilsson, and Bertram Raphael of the Stanford Research Institute, it was originally designed to help the Shakey robot navigate through rooms. Today, A* is the absolute industry standard for routing and pathfinding. Whether a character in a strategy game is walking across a grid, or a GPS is calculating the fastest drive across a country, A* (or a variant of it) is likely doing the heavy lifting. This article stays on the grid, where A* is easiest to see. If you want the theory behind it, the companion guide on the A* search algorithm in AI covers the 1968 Hart, Nilsson and Raphael paper, admissible versus consistent heuristics, the optimality proof and the memory bounded variants. 2. The Problem with Dijkstra's Algorithm To understand why A* is brilliant, we must first understand what it improves upon. Before A*, the gold standard for finding the shortest path was Dijkstra's Algorithm. Dijkstra's algorithm is guaranteed to find the shortest path. However, it is fundamentally "blind." When you ask Dijkstra to find a path from New York to Los Angeles, it will explore roads leading to Boston, Miami, and Chicago just as eagerly as it explores roads heading west. It searches outward in a perfect circle (or sphere) in all directions equally until it accidentally bumps into the target. In a large map, exploring every possible direction is incredibly slow and wastes massive amounts of processing power. We need an algorithm that is "smart" enough to know which direction the target is in, so it can prioritize exploring paths that head towards the goal. Both reach the goal along the same shortest path, but Dijkstra expands every node closer than the goal, while A* only expands nodes that head toward it, exploring far fewer cells. A* Search Algorithm Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. The Secret Sauce: Heuristics (f = g + h) A* solves the "blindness" problem by introducing a Heuristic. A heuristic is an educated guess. In pathfinding, it's a function that estimates how far a node is from the final destination. A* assigns a score to every node it discovers. The node with the lowest score is the one it explores next. The core equation of A* is: f(n) = g(n) + h(n) - n is the current node being evaluated on the graph. - g(n) is the Exact Cost. It is the known distance from the starting node to node n. (This is exactly what Dijkstra uses). - h(n) is the Heuristic Estimate. It is the estimated distance from node n to the final goal. - f(n) is the Total Cost. This is the sum of g and h. A* always prioritizes the node with the lowest f score. By adding the h(n) component, A* is pulled towards the goal like a magnet. It will ignore paths that go in the opposite direction of the goal, drastically reducing the search space compared to Dijkstra's algorithm. g is the real distance already travelled from the Start; h is the estimated distance still to go. A* sorts its frontier by their sum, f = g + h. 4. Choosing the Right Heuristic The magic of A* relies entirely on the accuracy of the heuristic function h(n). If your heuristic overestimates the distance to the goal, A* is no longer guaranteed to find the shortest path. If it underestimates, it becomes slower. Therefore, choosing the right heuristic for your specific game or map is critical. Manhattan Distance (For Grid Worlds without Diagonals) If your game uses a grid (like Pac-Man or a traditional roguelike) and characters can only move Up, Down, Left, or Right, you should use the Manhattan Distance. It calculates the total number of blocks horizontally and vertically between the current node and the target. function heuristic(node, goal) { return abs(node.x - goal.x) + abs(node.y - goal.y) } Euclidean Distance (For Open Worlds or Any-Angle Movement) If characters can move in any direction (or if you are routing on a continuous map), you should use Euclidean Distance. This is the straight-line "as the crow flies" distance between two points, calculated using the Pythagorean theorem. function heuristic(node, goal) { return sqrt((node.x - goal.x)^2 + (node.y - goal.y)^2) } Chebyshev Distance (For Grid Worlds with Diagonals) If your game uses a grid but allows diagonal movement (where moving diagonally costs the same as moving straight), you use the Chebyshev distance. It takes the maximum of the horizontal or vertical differences. function heuristic(node, goal) { return max(abs(node.x - goal.x), abs(node.y - goal.y)) } The same node-to-goal gap gives a different estimate depending on how movement is allowed: Manhattan (7) for 4-way grids, Euclidean (5) for any-angle, Chebyshev (4) for 8-way grids. Visualize the Difference Watch Dijkstra and A* race to solve the same maze. Notice how Dijkstra floods the entire map, while A* creates a laser-focused path directly toward the exit. Launch A* Visualizer Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Step-by-Step Execution of A* Here is exactly how the algorithm maintains its state and processes nodes during execution: - Initialization: Create two sets: an OPEN set (nodes to be evaluated) and a CLOSED set (nodes already evaluated). Add the starting node to the OPEN set. - Loop Start: Find the node in the OPEN set with the lowest f cost. Let's call this the Current node. - Check Goal: If the Current node is the target goal, you are done! Follow the parent pointers backward to reconstruct the path. - Move to Closed: Remove the Current node from the OPEN set and add it to the CLOSED set. - Expand Neighbors: Look at all adjacent neighbors of the Current node. For each neighbor: - If the neighbor is an obstacle (wall) or is in the CLOSED set, ignore it. - Calculate the neighbor's g cost (Current g + distance to neighbor). - If the neighbor is not in the OPEN set, calculate its h and f costs, set its parent to Current, and add it to the OPEN set. - If the neighbor is already in the OPEN set, check if this new path to the neighbor is better (has a lower g cost). If it is, update its parent and g cost. - Repeat: Loop back to Step 2. If the OPEN set becomes empty and the goal hasn't been reached, there is no valid path. 6. Real-World Applications in Game Dev & AI A* is the backbone of movement in digital spaces. - Real-Time Strategy (RTS) Games: When you drag-select 50 units in StarCraft or Age of Empires and click a point on the map, A* calculates 50 individual paths, routing them around buildings, rivers, and each other. (Often utilizing specialized variants like Flow Fields or Hierarchical A*). - RPG and Stealth Games: NPCs use A* on a "NavMesh" (Navigation Mesh - a simplified graph of walkable surfaces) to chase the player, patrol corridors, or find cover. - Robotics: Automated warehouse robots (like those used by Amazon) use A* to navigate factory floors without colliding with shelving units or other robots. - GPS Navigation: While standard A* is too slow for continental maps, modified versions (like ALT - A* with Landmarks and Triangle inequality) are used to quickly calculate optimal driving routes. Frequently Asked Questions How does the A* algorithm differ from Dijkstra's? A* uses a heuristic function (estimating the distance to the goal) to guide its path search, whereas Dijkstra's algorithm is blind and explores in all directions equally. This makes A* significantly faster and more targeted. What is an admissible heuristic in A*? An admissible heuristic is one that never overestimates the actual cost to reach the goal. Admissibility guarantees that A* will find the mathematically shortest path. When should I choose Manhattan vs. Euclidean distance? Use Manhattan distance for grids where movement is restricted to 4 directions (up, down, left, right). Use Euclidean distance for continuous environments allowing movement in any angle. Related Articles A* Search Algorithm in AI Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## A* Search Algorithm in AI: The Complete Guide Source: https://learngraphtheory.org/articles/a-star-algorithm-in-ai.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. What is the A* search algorithm? - 2. Where A* came from: Hart, Nilsson and Raphael (1968) - 3. The evaluation function f(n) = g(n) + h(n) - 4. How A* runs, step by step - 5. A worked example on a weighted graph - 6. Admissible and consistent heuristics - 7. Why A* is optimal, and when it is not - 8. Time and space complexity - 9. How to design a good heuristic - 10. A* vs Dijkstra, UCS, greedy best first and BFS - 11. Variants of A* worth knowing - 12. Common mistakes in A* implementations - 13. Where A* is actually used - 14. Frequently asked questions 1. What is the A* search algorithm? A* (pronounced "A star") is a best first graph search algorithm that finds a minimum cost path from a start node to a goal node. In artificial intelligence it is classified as an informed search: unlike breadth first search or uniform cost search, which only know what has already been paid, A* also uses an estimate of what still lies ahead. Every node A* keeps on its frontier carries three numbers: - g(n): the exact cost of the best path found so far from the start node to n. - h(n): the heuristic, an estimate of the cheapest cost from n to a goal. - f(n) = g(n) + h(n): the estimated cost of the cheapest complete solution that passes through n. The algorithm repeatedly expands the frontier node with the smallest f. That single rule produces the property A* is famous for: if the heuristic never overestimates the real remaining cost, the first goal node A* removes from the frontier lies on an optimal path. Nothing in that definition mentions grids or video games. A* is defined on any weighted graph with non negative edge costs. The graph can be a road network, a tile map, the state space of a sliding tile puzzle, a robot's configuration space, or the set of partial plans inside an automated planner. Grid pathfinding is simply the picture that is easiest to draw, and it is covered in the companion guide on A* pathfinding on grids. A* is less a separate invention than a dial. Set h to zero and you get Dijkstra, which is optimal but explores twice as much. Drop g and you get greedy best first search, which explores almost nothing and returns a path costing 46 where 22 was available. Keep both and you get the informed middle. 2. Where A* came from: Hart, Nilsson and Raphael (1968) Peter E. Hart, Nils J. Nilsson and Bertram Raphael, working at the Stanford Research Institute, published "A Formal Basis for the Heuristic Determination of Minimum Cost Paths" in IEEE Transactions on Systems Science and Cybernetics, volume 4, issue 2, pages 100 to 107, July 1968. The practical motivation was Shakey, the SRI mobile robot, which had to plan its own route through a suite of rooms rather than follow a scripted track. Two contributions from that paper still define the subject: - It separated the search procedure from the knowledge inside it. The same loop becomes a different algorithm depending on which h you hand it. Search strategy became a parameter rather than a design decision. - It proved the guarantees. If h never overestimates the true remaining cost, the procedure returns a minimum cost path. A companion result showed that no other algorithm with access to the same heuristic information can expand fewer nodes and still guarantee optimality. The name is a piece of notation that stuck. Nilsson had described an earlier family of procedures labelled A1, A2 and so on. The star marks the member of that family that uses an admissible heuristic and is therefore the optimal one. It is not an abbreviation and it does not stand for anything. A 1972 note in the SIGART Newsletter corrected the original optimality argument. The 1968 proof leaned on what the authors called the consistency assumption, and the correction made explicit what has to change when a heuristic is admissible but not consistent: already expanded nodes must be allowed to be reopened. That distinction is section 6 below, and it is still the single most common source of silently wrong A* implementations more than fifty years later. If a citation is what brought you here: Hart, P. E., Nilsson, N. J., and Raphael, B. (1968). "A Formal Basis for the Heuristic Determination of Minimum Cost Paths." IEEE Transactions on Systems Science and Cybernetics 4(2), 100 to 107. 3. The evaluation function f(n) = g(n) + h(n) Everything A* does follows from how it ranks its frontier. The evaluation function splits the estimate of a solution into a part that is known exactly and a part that is guessed: f(n) = g(n) + h(n) g(n) cost already paid exact, measured along the path actually taken h(n) cost still to pay estimated by the heuristic function f(n) total estimate the value the priority queue sorts on The value of that design is that one component controls the entire behaviour of the search: - h(n) = 0 everywhere. Then f = g and A* degenerates into uniform cost search, which on a graph with non negative weights is exactly Dijkstra's algorithm. Correct, but uninformed: it expands every node cheaper than the goal, in every direction. - h(n) = h*(n), the true remaining cost. A* walks straight down an optimal path and expands almost nothing else. This is the theoretical ceiling, and it is unavailable in practice because computing h* is the very problem you were trying to solve. - Ignore g and rank on h alone. That is greedy best first search. It is fast, it is often badly wrong, and it carries no optimality guarantee. - Reweight: f = g + w × h with w > 1. Weighted A* trades a bounded amount of solution quality for a large amount of speed. The path it returns is never worse than w times the optimal cost. Read that list as a scale of how much you trust the estimate. A* sits in the middle and spends effort in proportion to how little the heuristic knows: a sharp heuristic produces a search that looks like a straight line, a weak one produces a search that looks like Dijkstra. 4. How A* runs, step by step The frontier is a priority queue, traditionally called OPEN, ordered by f. A map of best known g values and a map of parent pointers complete the state. function a_star(start, goal, h): open = priority queue ordered by f # the frontier g = map with default infinity # best known cost from start parent = map # for path reconstruction g[start] = 0 open.push(start, h(start)) while open is not empty: current = open.pop_min() # node with the smallest f if current == goal: return reconstruct(parent, goal) # goal test on pop, not on generate for (next, cost) in edges(current): tentative = g[current] + cost if tentative < g[next]: # a cheaper route to next g[next] = tentative parent[next] = current open.push_or_update(next, tentative + h(next)) return failure # the goal is unreachable Four implementation details carry most of the subtlety: - The goal test happens when a node is popped, not when it is generated. A goal can be generated early along an expensive route. Returning at that moment gives you a path, just not the cheapest one. The worked example below contains a case where testing on generation costs 7 units out of 13. - Relaxation is guarded by tentative < g[next], not by membership of a closed set. Written this way, the loop reopens an already expanded node automatically when a cheaper route to it turns up later, which is what makes it correct for any admissible heuristic. - Most priority queues have no decrease key operation. Production code usually pushes a duplicate entry with the better f and discards stale entries as they come out. This is called lazy deletion: it costs a little memory and saves a lot of bookkeeping. - The path is recovered backwards. Follow parent pointers from the goal to the start and reverse the list. A* never stores paths, only one parent per node. 5. A worked example on a weighted graph Grid animations hide the arithmetic. Here is a small weighted graph, the kind used in AI courses, traced by hand from start to finish. Six nodes: S (start), A, B, C, D and G (goal). Edge costs are road distances, and the heuristic h is a straight line distance to G, which is what makes it admissible: a straight line is never longer than a road. Edges (undirected, cost) Heuristic h(n) S-A 4 A-C 5 S = 11 C = 5 S-B 3 A-D 12 A = 9 D = 3 S-G 20 B-D 7 B = 8 G = 0 C-G 6 D-G 3 The example graph. The expensive direct edge S to G exists precisely to show why the goal test has to wait until a node is popped. A* pops the node with the smallest f, breaking ties in favour of the smaller h, which is the standard tie break because it prefers nodes nearer the goal: StepPoppedghfFrontier after the step 1S01111B 11, A 13, G 20 2B3811A 13, D 13, G 20 3D10313G 13 (improved from 20), A 13 4G13013goal popped, search ends The answer is S → B → D → G at a cost of 13. Three things in that trace are worth pausing on: - The goal was generated in step 1 with f = 20 and ignored. An implementation that returns as soon as it generates the goal would have reported a path costing 20, which is 54 percent worse than optimal. A* keeps it on the frontier and only accepts it once nothing cheaper remains. - Node C was never even generated. Dijkstra on the same graph expands every node with distance below 13, which includes C at 9. A* never touches it because the frontier never gets that far. - Node A was generated but never expanded. Its f of 13 ties with the goal, and since a tie at f = C* can be resolved either way, A never became worth opening. That is exactly the boundary case the optimal efficiency theorem talks about. A* Search in AI Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 6. Admissible and consistent heuristics Two properties of h are worth distinguishing carefully, because they are often used as though they were the same thing and they are not. Admissibility A heuristic is admissible if it never overestimates: h(n) <= h*(n) for every node n where h*(n) is the true cheapest cost from n to a goal Admissibility is optimism. The heuristic is allowed to be wildly wrong as long as it errs on the low side. h = 0 is admissible, which is why Dijkstra is a special case of A* rather than a rival to it. Consistency, also called monotonicity A heuristic is consistent if it satisfies a triangle inequality along every edge: h(n) <= cost(n, n') + h(n') for every edge n to n' h(goal) = 0 In words: taking one step can never reduce your estimate by more than the step actually cost. Consistency is the stronger property. Every consistent heuristic is admissible; the reverse is false. The practical consequence is precise. When h is consistent, f never decreases along a path, so the first time A* pops a node, its g value is already optimal. That is what licenses the familiar optimisation of putting expanded nodes into a closed set and never looking at them again. When h is only admissible, that licence is withdrawn. A cheaper route to an already expanded node can still be discovered, and if the implementation refuses to reopen it, the final answer can be wrong. Here is a graph where exactly that happens: Every heuristic value here is admissible, yet an implementation that refuses to reopen closed nodes returns a path costing 13 when a path costing 12 exists. Edges (directed, cost) Heuristic h(n) True cost h*(n) S -> A 1 S = 0 S = 12 S -> B 2 A = 11 A = 11 A -> C 1 B = 5 B = 11 B -> C 1 C = 0 C = 10 C -> G 10 G = 0 G = 0 Every h(n) <= h*(n), so the heuristic is admissible. Edge A -> C breaks consistency: h(A) = 11 > cost(A,C) + h(C) = 1. Trace it: A* pops S (f = 0), generating A at f = 1 + 11 = 12 and B at f = 2 + 5 = 7. It pops B (f = 7) and generates C with g = 3, f = 3. It pops C (f = 3), closes it with g = 3, and generates G at f = 13. Only then does it pop A (f = 12) and discover a route to C costing just 2. If C is closed and closed means final, that discovery is thrown away and the algorithm returns 13 instead of the optimal 12. Two ways out, and you should pick one deliberately: prove your heuristic is consistent and keep the closed set optimisation, or write the relaxation as tentative < g[next] with no closed set check so reopening happens on its own. The reassuring news is that nearly every heuristic used in practice, including straight line distance, Manhattan distance on a uniform cost grid, and any heuristic derived from a relaxed problem, is consistent. That is precisely why this bug can survive years of use before a strange map exposes it. 7. Why A* is optimal, and when it is not The optimality argument is short enough to hold in your head, and worth knowing because it tells you exactly which assumption to check when A* misbehaves. Let C* be the cost of an optimal solution. Suppose A* is about to pop a goal node G2 reached along a suboptimal path, so g(G2) > C*. Since h(G2) = 0, its priority is f(G2) = g(G2) > C*. Now consider any optimal path from the start to a real optimal goal. At the moment before the pop, that path must have at least one node n sitting on the frontier, because the start was expanded and the goal was not. For that node, g(n) is the optimal cost to reach it and, because h is admissible, f(n) = g(n) + h(n) <= g(n) + h*(n) = C*. So f(n) <= C* < f(G2), and A* would have popped n first. The contradiction proves that the first goal popped is optimal. The same reasoning gives you the conditions under which the guarantee fails: - The heuristic overestimates somewhere. The inequality f(n) <= C* breaks, and A* can walk past the optimal path. This is not always a disaster, but the guarantee is gone and you should say so in the code. - Negative edge costs. A* assumes costs are non negative, as does Dijkstra. With negative edges the frontier ordering means nothing. Use Bellman Ford instead. - Costs that change during the search. Static A* has no answer for a door that closes mid plan. That is what D* Lite and lifelong planning A* exist for. - Infinite or unbounded graphs. Completeness needs a finite branching factor and edge costs bounded below by some positive epsilon. Without that, A* can chase an infinite sequence of ever cheaper steps and never terminate. One further result rounds out the theory. Dechter and Pearl proved in 1985 that A* is optimally efficient: no other optimal algorithm using the same heuristic information is guaranteed to expand fewer nodes. Every node with f(n) < C* must be expanded by any such algorithm, and A* expands exactly those, plus some subset of the nodes sitting exactly at f(n) = C*. There is no cleverer scheduling to be found; the only remaining lever is a better heuristic. 8. Time and space complexity The textbook figure is O(bd), where b is the branching factor and d is the depth of the optimal solution, and that number is honest but not very informative. The sharper statement is about the heuristic's error. The number of nodes A* expands grows exponentially unless the error of the heuristic is bounded logarithmically in the true cost: |h(n) - h*(n)| <= O(log h*(n)) Almost no heuristic used in practice meets that bar, so the honest reading is that A* is exponential in the worst case and that a good heuristic reduces the base of the exponent rather than removing it. On an explicit graph that you already hold in memory, the accounting is the same as Dijkstra's: O(E log V) with a binary heap, since every edge triggers at most one queue update. The interesting question is not the bound but how few of those V and E the search ever touches. Memory is the constraint that actually bites. A* keeps every generated node in memory, both the frontier and the explored set, so its space complexity matches its time complexity. Real systems run out of RAM long before they run out of patience. This single fact explains the entire family of variants in section 11: iterative deepening A*, simplified memory bounded A*, and beam style truncations all exist to trade optimality or repeated work for a frontier that fits. 9. How to design a good heuristic Since the algorithm is fixed and provably efficient, all of the engineering lives in h. There is a reliable recipe for producing one. Relax the problem Take the real problem, delete a constraint, and solve what is left exactly. The exact cost of a relaxed problem is always a lower bound on the real cost, so it is admissible by construction, and because a relaxed solution can be built one step at a time it is consistent as well. The 8 puzzle is the standard illustration. A tile may move to the adjacent blank square. Delete constraints and two classic heuristics fall out: - h1, misplaced tiles. Allow any tile to teleport anywhere in one move. Then the cost is simply the number of tiles not in their goal position. - h2, Manhattan distance. Allow a tile to move to any adjacent square, blank or not. Then the cost is the sum, over all tiles, of the horizontal plus vertical distance to that tile's goal square. Both are admissible. The difference in practice is dramatic: on random 8 puzzle instances requiring 12 moves, the classic comparison reported by Russell and Norvig has iterative deepening search generating on the order of 3.6 million nodes, A* with h1 generating a few hundred, and A* with h2 generating fewer than a hundred. At depth 24 the gap widens to roughly 39,000 nodes for h1 against roughly 1,600 for h2. Same algorithm, same code, different h. Prefer a dominant heuristic If h2(n) >= h1(n) for every node and both are admissible, then h2 dominates h1 and A* with h2 never expands more nodes. Bigger is better, right up to the ceiling of h*. This is why Manhattan distance beats misplaced tiles: it is always at least as large and never overestimates. Combine several heuristics If you have several admissible heuristics and none dominates the others, take their maximum. The maximum of admissible heuristics is admissible, and it dominates all of them. The cost is that you now evaluate several functions per node, so the win has to be paid for in evaluation time. Precompute with pattern databases For a fixed goal state, solve subproblems exhaustively in advance and store the exact costs in a table. A pattern database for a subset of the puzzle tiles gives a far stronger admissible bound than any formula. This is the same idea that makes landmark based routing work on continental road networks: precompute exact distances to a handful of landmarks, then use the triangle inequality to bound the rest. On grids, choose the metric that matches the moves Manhattan distance for 4 directional movement, Euclidean for any angle movement, Chebyshev or octile for 8 directional grids. Choosing a metric that allows more freedom than the movement rules do makes the heuristic inadmissible and quietly destroys the optimality guarantee. The grid pathfinding guide works through all three with worked numbers. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 10. A* vs Dijkstra, UCS, greedy best first and BFS Every one of these is the same loop with a different priority. That is the most useful way to remember them. AlgorithmFrontier priorityOptimal?Use it when Breadth first searchinsertion order (FIFO)Yes, if all edges cost the sameUnweighted graphs, fewest hops Uniform cost search / Dijkstraf = gYesWeighted graph, no useful estimate available, or many goals at once Greedy best first searchf = hNoYou need any solution fast and quality is negotiable A*f = g + hYes, if h is admissibleSingle goal, a decent estimate exists, the path must be shortest Weighted A*f = g + w × hWithin a factor of wA* is too slow and a bounded detour is acceptable The intuition behind the A* against Dijkstra comparison is geometric. Dijkstra expands everything with g(n) < C*, which on an open map is a circle around the start. A* expands everything with g(n) + h(n) < C*, which is an ellipse with the start and the goal as its focal points. The sharper the heuristic, the flatter that ellipse becomes, until it collapses onto the path itself. Both return the same optimal answer; they differ only in how much of the map they had to look at to be sure. For a broader map of the family, see the overview of shortest path algorithms, and for the traversal foundations underneath all of it, BFS vs DFS. 11. Variants of A* worth knowing Nearly every named variant is a response to one of two pressures: memory, or the need to answer before the optimum is proved. - Weighted A* multiplies the heuristic by w > 1. Solutions are guaranteed within a factor w of optimal, and in practice are usually far better than that bound while running an order of magnitude faster. - IDA* (iterative deepening A*) runs repeated depth first searches with an increasing f cutoff. Memory drops to the depth of the search, and the price is re-expanding nodes on every iteration. This is the standard tool for puzzles with enormous state spaces. - SMA* (simplified memory bounded A*) uses all the memory you give it and, when it runs out, drops the worst leaf while remembering its value in the parent. It remains optimal if the optimal path fits in memory. - ARA* and other anytime variants start with a large w, return a solution quickly, then keep improving it while time remains, tightening the bound as they go. - D* and D* Lite repair an existing plan when edge costs change, instead of replanning from scratch. This is what a robot discovering an unmapped obstacle needs. - Jump point search exploits the symmetry of uniform cost grids to skip whole runs of identical cells. Same answer as A*, often an order of magnitude fewer expansions, grids only. - Bidirectional A* searches forward from the start and backward from the goal at once. It is powerful and subtle, since the stopping condition and the heuristic consistency requirements both become harder. - ALT and contraction hierarchies are what production route planners actually run. Plain A* is too slow on continental road networks, so these precompute landmark distances or shortcut edges and then search a much smaller effective graph. 12. Common mistakes in A* implementations Most A* bugs do not crash. They return a path that is slightly too long, on some inputs, sometimes. These are the recurring causes: - Mismatched units. g measured in seconds and h measured in metres is the classic. If h is even slightly overscaled it stops being admissible and the guarantee is gone. Convert both to the same currency before adding them. - A closed set with an inconsistent heuristic. The failure mode from section 6. If you cannot prove consistency, allow reopening. - No tie breaking. When many nodes share the same f, an arbitrary order makes A* explore a wide plateau of equally promising cells. Breaking ties towards the smaller h costs nothing and keeps every guarantee intact. The other common trick, scaling h by 1 + epsilon, narrows the search dramatically on grids but makes the heuristic very slightly inadmissible, so use it knowing you have traded the proof for the speed. - Testing the goal on generation. Fast, tempting, and it silently returns suboptimal paths, as the worked example showed. - Floating point comparison. Euclidean heuristics produce irrational values, and strict equality checks on f behave unpredictably. Compare with a tolerance, or keep costs in integers. - Forgetting h(goal) = 0. A non zero estimate at the goal breaks consistency at the last edge and can distort the whole search. - Recomputing h on every queue operation. The heuristic is called constantly. Cache it on the node; profilers find this one over and over. - Mutable states used as dictionary keys. In puzzle search, a board mutated in place corrupts the visited set. Store an immutable encoding. 13. Where A* is actually used A* earns its place anywhere a best sequence of decisions has to be found in a space too large to enumerate: - Robotics and motion planning. The original Shakey application, still current: grid and lattice planners for mobile robots, warehouse fleets and autonomous vehicles, usually with D* Lite handling replanning as the world changes. - Game AI. Units routing across navigation meshes and tile maps, typically with hierarchical decomposition on top so that a long path is planned coarsely first and refined locally. - Route planning. Journey planners run A* derivatives with landmark heuristics or contraction hierarchies, which is how a continental route is returned in milliseconds. - Puzzle and state space search. Sliding tile puzzles, Rubik's cube solvers and similar combinatorial problems, usually with IDA* and pattern databases because the state spaces are far too large for a frontier in memory. - Automated planning. Classical planners search the space of world states with heuristics derived automatically from a relaxed version of the planning problem, which is the same relaxation recipe from section 9 applied mechanically. - Bioinformatics. Multiple sequence alignment is a shortest path problem in a high dimensional lattice, and A* with an admissible bound is a standard exact method. - Speech and language decoding. A* search over a word lattice, where g is the score of the partial hypothesis and h bounds the best completion, remains a textbook decoding strategy. The pattern is always the same. Model the problem as a graph, find an admissible lower bound on the remaining cost, and let the priority queue do the rest. 14. Frequently asked questions Does A* always find the shortest path? Yes, provided three conditions hold: the heuristic is admissible (it never overestimates the remaining cost), edge costs are non negative, and the implementation tests for the goal when a node is popped rather than when it is generated. If the heuristic is admissible but not consistent, the implementation must also allow already expanded nodes to be reopened. Break any one of those conditions and A* still returns a path, but it is no longer guaranteed to be the cheapest one. What is the difference between A* and Dijkstra's algorithm? They are the same algorithm with different priorities. Dijkstra orders its frontier by g(n), the cost already paid, so it expands outwards in every direction like a growing circle. A* orders by f(n) = g(n) + h(n), so the region it expands is an ellipse stretched from the start towards the goal. Setting h(n) = 0 turns A* into Dijkstra exactly. Dijkstra is the better choice when no useful estimate exists or when you need distances to many destinations at once; A* is the better choice for a single destination with a decent estimate available. What is the difference between an admissible and a consistent heuristic? Admissible means h(n) is never greater than the true remaining cost h*(n). Consistent, also called monotone, is the stronger triangle inequality h(n) <= cost(n, n') + h(n') for every edge, with h(goal) = 0. Every consistent heuristic is admissible, but not the reverse. Consistency matters because it guarantees that the first time A* pops a node, its g value is already optimal, which is what makes a closed set safe. With a merely admissible heuristic, a cheaper route to a closed node can still appear later and the node must be reopened. What is the time and space complexity of A*? In the worst case both are O(b^d), exponential in the depth of the solution. The number of expansions stays polynomial only when the heuristic error is bounded logarithmically, which almost never happens in practice, so a better heuristic reduces the base of the exponent rather than removing it. On an explicit graph already in memory, A* behaves like Dijkstra at O(E log V) with a binary heap. Space is the practical limit: A* stores every node it generates, which is why memory bounded variants such as IDA* and SMA* exist. Why is it called A*, and is it an AI algorithm or a graph algorithm? The name comes from Nilsson's earlier family of procedures named A1, A2 and so on; the star marks the member of that family that uses an admissible heuristic and is therefore optimal. It is both an AI algorithm and a graph algorithm. It is taught in AI courses as the canonical informed search method, and it is a shortest path algorithm on weighted graphs. It is not machine learning: nothing is trained and no data is fitted. The knowledge lives entirely in the heuristic function you supply, although a learned model can perfectly well be used to supply one. Is A* complete, and what happens if no path exists? A* is complete on a graph with a finite branching factor whose edge costs are bounded below by some positive value: if a solution exists it will be found. When no path exists, the frontier empties after the entire reachable component has been expanded and the algorithm reports failure. That failure case is the expensive one, since proving that nothing exists means exploring everything reachable, which is a good reason to add a connectivity precheck on large maps. See the frontier move Run Dijkstra on your own graph in the interactive visualizer. It is A* with h set to zero, so what you watch flooding outwards is exactly the work a good heuristic removes. Launch the visualizer Related Articles A* Pathfinding on Grids Read more → Shortest Path Algorithms Read more → Dijkstra's Algorithm Read more → --- ## Graph Algorithms Time Complexity Source: https://learngraphtheory.org/articles/graph-algorithms-complexity.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Introduction to Computational Complexity in Graphs - 2. Graph Representation: The Silent Complexity Modifier - 3. Graph Traversal Algorithms (BFS & DFS) - 4. Shortest Path Algorithms - 5. Minimum Spanning Tree Algorithms - 6. Connectivity and Topological Sorting - 7. Network Flow Algorithms - 8. NP-Hard Graph Problems - 9. The Ultimate Complexity Cheat Sheet 1. Introduction to Computational Complexity in Graphs In computer science, algorithms are rarely judged solely on whether they produce the correct result. They are judged on how efficiently they produce that result. As data scales, whether you're analyzing a social network with billions of users or calculating logistics for global shipping routes, efficiency becomes the ultimate bottleneck. We measure efficiency using Big O Notation, which provides an upper bound on the time (execution speed) and space (memory consumption) an algorithm requires, relative to the size of its input. In graph theory, the input size is almost always defined by two parameters: - V (or |V|): The number of Vertices (nodes) in the graph. - E (or |E|): The number of Edges (connections) in the graph. A graph can be Sparse (where E is close to V) or Dense (where E is close to V²). The sparsity or density of a graph heavily dictates which algorithms, and which data structures, will perform optimally. Rule of Thumb: If an algorithm operates in O(V²) time, it might be acceptable for a graph with 1,000 vertices (1,000,000 operations). But for a graph with 1,000,000 vertices, it requires 1,000,000,000,000 operations, rendering it completely useless for real-time applications. Understanding asymptotic complexity is non-negotiable for system design. How the common complexity classes scale as the input grows. Constant and logarithmic curves stay flat, while quadratic, exponential, and factorial curves explode. 2. Graph Representation: The Silent Complexity Modifier Before analyzing any specific algorithm, we must discuss graph representation. How you store a graph in memory fundamentally alters the time and space complexity of every operation performed on it. The two most common representations are the Adjacency Matrix and the Adjacency List. Adjacency Matrix An adjacency matrix is a 2D array of size V × V. A cell at matrix[i][j] holds a boolean (or a weight) indicating if an edge exists from vertex i to vertex j. - Space Complexity: O(V²). This is extremely memory-intensive. For a graph with 100,000 nodes, the matrix requires 10 billion entries. - Edge Lookup (Are i and j connected?): O(1). Immediate verification. - Finding all neighbors of a vertex: O(V). You must iterate through the entire row for that vertex, checking all possible connections, even if the graph is sparse. Adjacency List An adjacency list is an array (or hash map) of lists. The index represents the vertex, and the list at that index contains all of its connected neighbors. - Space Complexity: O(V + E). This is optimal because it only stores the vertices and the actual edges that exist. - Edge Lookup: O(deg(V)), where deg(V) is the degree (number of neighbors) of the vertex. In the worst case (a dense graph), this is O(V), but in sparse graphs, it is practically O(1). - Finding all neighbors of a vertex: O(deg(V)). You only iterate over the actual neighbors, making traversals lightning fast. The same graph stored two ways. The matrix reserves a cell for every possible edge, while the list stores only the edges that exist. Conclusion: Unless dealing with a dense graph where you need constant-time edge lookups, the Adjacency List is the standard. For the rest of this article, assume all complexities are based on an Adjacency List representation unless stated otherwise. 3. Graph Traversal Algorithms Traversals form the backbone of almost all complex graph logic. They are used to visit every node and edge in a graph systematically. Breadth-First Search (BFS) BFS explores a graph layer by layer, starting from a source node and exploring all its immediate neighbors before moving to the next level. It uses a Queue (FIFO) data structure. - Time Complexity: O(V + E). Each vertex is enqueued and dequeued exactly once O(V), and every edge is examined once for directed graphs or twice for undirected graphs O(E). Thus, the total operations scale linearly with the size of the graph. - Space Complexity: O(V). In the worst case (like a star graph), the queue will hold all vertices except the root at the same time. The visited array also requires O(V) space. - Use Cases: Shortest path in unweighted graphs, peer-to-peer networks, web crawlers. Depth-First Search (DFS) DFS goes as deep as possible along a branch before backtracking. It relies heavily on recursion (Call Stack) or an explicit Stack (LIFO). - Time Complexity: O(V + E). Similar to BFS, every vertex is visited once, and every edge is examined once (or twice). The asymptotic time is identical to BFS. - Space Complexity: O(V). The maximum depth of the recursion stack can reach V if the graph is a single long path (e.g., a linked list structure). - Use Cases: Topological sorting, cycle detection, solving mazes, pathfinding in heavily constrained environments. // Example: DFS Time Complexity Breakdown void DFS(int v) { visited[v] = true; // O(1) per vertex -> Total: O(V) for (int u : adjList[v]) { // Iterates over edges of v if (!visited[u]) { DFS(u); } } // Total across all loops: O(E) } // Combined Time: O(V + E) BFS and DFS share the same O(V + E) time and O(V) space. The numbered badges show the order each algorithm visits the vertices. 4. Shortest Path Algorithms Shortest path algorithms are the most frequently used graph algorithms in the real world, heavily applied in mapping services, network routing protocols, and AI navigation. Dijkstra’s Algorithm Dijkstra computes the shortest path from a single source node to all other nodes in a graph with non-negative edge weights. It uses a Greedy approach and a Priority Queue. The complexity of Dijkstra is entirely dependent on the data structure used to implement the Priority Queue. - With a basic Array/List: Time: O(V²). Extracting the minimum takes O(V), done V times. Updating neighbors takes O(E) total. Best for dense graphs where E ≈ V². - With a Binary Min-Heap: Time: O((V + E) log V). Extracting the minimum takes O(log V), done V times. Updating a key (decrease-key) takes O(log V), done E times. Best for typical, sparse graphs. - With a Fibonacci Heap: Time: O(E + V log V). A Fibonacci Heap allows decrease-key operations to run in O(1) amortized time, leaving only the V extractions taking O(log V). This is the theoretical optimal complexity for Dijkstra. - Space Complexity: O(V). To store distances, predecessors, and the Priority Queue. Bellman-Ford Algorithm Unlike Dijkstra, Bellman-Ford can handle graphs with negative edge weights and detect negative weight cycles. It does this by repeatedly "relaxing" all edges. - Time Complexity: O(V × E). The algorithm loops V-1 times, and in each loop, it iterates over all E edges. This is significantly slower than Dijkstra, so it should only be used when negative weights are present. - Space Complexity: O(V). Requires an array to store the current shortest distances from the source. Floyd-Warshall Algorithm Floyd-Warshall is an All-Pairs Shortest Path algorithm. It finds the shortest paths between every pair of vertices in a weighted graph (even with negative weights, provided there are no negative cycles). It uses Dynamic Programming. - Time Complexity: O(V³). It features three nested loops, each iterating from 1 to V, updating the matrix of shortest paths. Because of this cubic complexity, it is only viable for small graphs (typically V < 500). - Space Complexity: O(V²). It maintains a 2D distance matrix. A* Search Algorithm A* is a targeted search algorithm used extensively in video game pathfinding (like NPC movement) and GPS routing. It is essentially Dijkstra's algorithm augmented with a Heuristic function h(n) that estimates the distance to the goal, steering the search direction. - Time Complexity: Worst-case O(b^d), where b is the branching factor (average number of edges per node) and d is the depth of the optimal solution. In the best case (perfect heuristic), it is O(d). If the heuristic is 0, A* degrades to Dijkstra's O((V + E) log V). - Space Complexity: O(b^d) in the worst case, as it must keep all generated nodes in memory (in the OPEN and CLOSED lists). The massive space requirement is usually the bottleneck for A*, leading to variants like Iterative Deepening A* (IDA*). Graph Algorithms Time Complexity Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 5. Minimum Spanning Tree Algorithms A Minimum Spanning Tree (MST) is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles, and with the minimum possible total edge weight. MSTs are crucial in network design, such as laying cables, electrical grids, and piping systems. Kruskal’s Algorithm Kruskal's takes a global greedy approach: sort all edges by weight, and continually pick the smallest edge that does not form a cycle. It relies on a Disjoint-Set (Union-Find) data structure to detect cycles. - Time Complexity: O(E log E) or O(E log V). Sorting the edges dominates the time, taking O(E log E). Since E ≤ V², log E ≤ 2 log V, meaning O(E log E) is asymptotically equivalent to O(E log V). The Union-Find operations take practically O(1) time, specifically O(α(V)) where α is the inverse Ackermann function. - Space Complexity: O(V + E) to store the graph and the Union-Find structure. Prim’s Algorithm Prim's algorithm builds the MST piece by piece, starting from an arbitrary node and greedily adding the cheapest edge that connects the growing tree to a new vertex outside the tree. - Time Complexity: Prim's complexity depends on the Priority Queue used, making its analysis identical to Dijkstra's algorithm. With a Binary Heap, it runs in O((V + E) log V). With a Fibonacci Heap, it runs in O(E + V log V). - Space Complexity: O(V) for the Priority Queue and tracking arrays. Kruskal vs. Prim: Generally, Kruskal's algorithm is preferred for sparse graphs (because sorting E elements is fast), while Prim's algorithm using a Fibonacci heap is mathematically superior for dense graphs. 6. Connectivity and Topological Sorting Analyzing the structure and dependencies within a graph requires specialized algorithms. These operate heavily on Directed Acyclic Graphs (DAGs) and complex directed networks. Topological Sort (Kahn's Algorithm & DFS-based) Topological sorting is a linear ordering of its vertices such that for every directed edge uv from vertex u to vertex v, u comes before v. It is primarily used for task scheduling, resolving package dependencies, and build systems. - Time Complexity: O(V + E). Both Kahn's Algorithm (which uses in-degree arrays and a queue) and the DFS-based approach visit every node and edge exactly once. - Space Complexity: O(V) for the queue/stack and arrays keeping track of visited status and in-degrees. Strongly Connected Components (Tarjan's and Kosaraju's) A Strongly Connected Component (SCC) in a directed graph is a maximal subset of vertices where every vertex is reachable from every other vertex in that subset. Identifying SCCs is vital in recommendation engines, circuit design, and ecosystem modeling. - Kosaraju's Algorithm: Involves two passes of DFS. First pass on the original graph, second pass on the transposed (reversed edges) graph. Time: O(V + E). Space: O(V). - Tarjan's Algorithm: Requires only a single pass of DFS, using an array to track "low link" values to identify component roots. Time: O(V + E). Space: O(V). Tarjan's is generally preferred in practice because performing a single DFS pass has better cache locality and less overhead than reversing the entire graph. 7. Network Flow Algorithms Network flow algorithms determine the maximum amount of flow that can pass from a source to a sink through a network of pipes with defined capacities. Applications include traffic routing, fluid dynamics, bipartite matching, and internet packet routing. Ford-Fulkerson Method The original method finds augmenting paths from source to sink using DFS and pushes flow until no paths remain. - Time Complexity: O(E × F), where F is the maximum flow of the network. The danger here is that if the capacities are very large or irrational numbers, Ford-Fulkerson can take extremely long or never terminate. This is pseudo-polynomial time. - Space Complexity: O(V) to maintain the visited array during DFS. Edmonds-Karp Algorithm An implementation of Ford-Fulkerson that uses BFS instead of DFS to find the shortest augmenting path (in terms of number of edges). This guarantees termination. - Time Complexity: O(V × E²). It can be proven that the maximum number of augmentations is bounded by O(V × E), and each BFS takes O(E). This is strongly polynomial and independent of the maximum flow F. - Space Complexity: O(V + E) for the BFS queue and storing the residual graph. Dinic's Algorithm Dinic's drastically optimizes flow calculation by building "Level Graphs" using BFS and then pushing multiple flows simultaneously using blocking flows via DFS. - Time Complexity: O(V² × E). In networks with unit capacities (like bipartite matching), the complexity drops remarkably to O(E × √V). It is the gold standard for competitive programming and practical network flow execution. - Space Complexity: O(V + E). Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 8. NP-Hard Graph Problems Some of the most famous problems in graph theory do not have any known polynomial-time O(N^k) solutions. These are classified as NP-Hard, meaning their complexity grows factorially or exponentially. The Traveling Salesperson Problem (TSP) Finding the shortest possible route that visits every node exactly once and returns to the origin. - Brute Force: O(V!) (Factorial time). Evaluating all permutations. Unusable for V > 15. - Dynamic Programming (Held-Karp): O(V² 2^V). Significantly better than factorial, but still exponential. Space complexity is heavy: O(V 2^V). Unusable for V > 30. Graph Coloring Problem Assigning a color to each vertex such that no two adjacent vertices share the same color, using the minimum number of colors. - Time Complexity: Checking if a graph can be colored with k colors is NP-Complete. Exact algorithms run in O(2^V) or worse. Modern solutions rely on heuristics, greedy approaches (like Welsh-Powell), or approximation algorithms rather than seeking perfect solutions. 9. The Ultimate Complexity Cheat Sheet Bookmark this page. Below is a comprehensive table summarizing the time and space complexity of every algorithm discussed. (Assuming Adjacency List representation and Binary Heaps where applicable). Algorithm Category Time Complexity (Worst) Space Complexity Breadth-First Search (BFS) Traversal O(V + E) O(V) Depth-First Search (DFS) Traversal O(V + E) O(V) Dijkstra's (Binary Heap) Shortest Path O((V + E) log V) O(V) Dijkstra's (Fibonacci Heap) Shortest Path O(E + V log V) O(V) Bellman-Ford Shortest Path (Negative) O(V × E) O(V) Floyd-Warshall All-Pairs Shortest Path O(V³) O(V²) A* Search Heuristic Search O(b^d) O(b^d) Kruskal's Algorithm Minimum Spanning Tree O(E log E) O(V + E) Prim's Algorithm Minimum Spanning Tree O((V + E) log V) O(V) Topological Sort DAG Processing O(V + E) O(V) Tarjan's / Kosaraju's Strongly Connected Comps. O(V + E) O(V) Ford-Fulkerson Network Flow O(E × F) O(V) Edmonds-Karp Network Flow O(V × E²) O(V + E) Dinic's Algorithm Network Flow O(V² × E) O(V + E) Traveling Salesperson (DP) NP-Hard O(V² 2^V) O(V 2^V) Frequently Asked Questions Why is graph representation important for complexity? The choice between an adjacency matrix and an adjacency list drastically changes the time and space complexity of algorithms. An adjacency list generally performs better for sparse graphs, while an adjacency matrix is better for dense graphs and quick edge lookups. What is the time complexity of Dijkstra's algorithm? Using a binary heap, Dijkstra's algorithm runs in O((V + E) log V) time. With an optimized Fibonacci heap, the time complexity drops to O(E + V log V), making it much faster for dense graphs. What is the difference in complexity between BFS and DFS? Both Breadth-First Search (BFS) and Depth-First Search (DFS) share the same worst-case time complexity of O(V + E) and space complexity of O(V), assuming an adjacency list representation. The difference lies in their traversal patterns and structural applications, rather than their asymptotic complexity. Why use Bellman-Ford if it's slower than Dijkstra's? Dijkstra's algorithm cannot handle graphs with negative edge weights; it will produce incorrect results. Bellman-Ford runs in O(V × E) but can safely route paths through negative edges and detect negative weight cycles, making it essential for certain financial or specialized routing models. Related Articles A* Search Algorithm: Pathfinding Read more → BFS vs DFS Deep Dive Read more → Network Flow & Max-Cut Read more → --- ## History of Graph Theory Source: https://learngraphtheory.org/articles/history-of-graph-theory.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. The Birth (1736): The Seven Bridges of Königsberg - 2. The 19th Century: Hamilton, Kirchhoff, and Trees - 3. The Century-Long Battle: The Four Color Theorem - 4. The Probabilistic Revolution: Erdős and Rényi - 5. The Modern Era: Algorithms, Networks, and Machine Learning - 6. Conclusion: A Branch that Connects Everything 1. The Birth (1736): The Seven Bridges of Königsberg Unlike many fields of mathematics that developed slowly over centuries from ancient Greek or Indian traditions, the exact birthplace and birthdate of graph theory are undisputed. It was born in 1736 from the mind of a single genius: Leonhard Euler. At the time, the Prussian city of Königsberg (now Kaliningrad, Russia) was situated on both sides of the Pregel River, and included two large islands. These four landmasses were connected to each other by exactly seven bridges. The citizens of Königsberg had created a local puzzle, a piece of urban folklore: Was it possible to take a walk through the city, crossing every single one of the seven bridges exactly once, and return to the starting point? Despite numerous attempts, no one could find such a path, nor could anyone definitively prove it was impossible. In 1735, the problem was brought to the attention of Euler, who was working at the St. Petersburg Academy. Euler's Abstraction Euler's genius lay not just in solving the problem, but in realizing what information was irrelevant. He recognized that the exact physical geography, the size of the islands, the length of the bridges, the distance between them, mattered not at all. The only thing that mattered was the connections. He abstracted the map: the four landmasses became points (what we now call vertices or nodes), and the seven bridges became lines connecting them (what we now call edges). In his seminal 1736 paper, "Solutio Problematis ad Geometriam Situs Pertinentis" (The solution of a problem relating to the geometry of position), Euler proved the walk was impossible. He reasoned that when you enter a landmass by a bridge, you must leave it by another bridge. Therefore, every landmass must have an even number of bridges connected to it (unless it is the starting or ending point). Since all four landmasses in Königsberg had an odd number of bridges connected to them, the path could not exist. By shifting focus from distance and measurement to connections and relative positions, Euler inadvertently founded not only Graph Theory but also laid the conceptual foundations for Topology. Euler's leap of abstraction: the city's four landmasses become vertices and its seven bridges become edges. Because every vertex ends up with an odd degree, no walk can cross all seven bridges exactly once. 2. The 19th Century: Hamilton, Kirchhoff, and Trees For roughly a century after Euler's paper, graph theory saw minimal development. It was viewed mostly as a collection of recreational puzzles rather than a serious branch of mathematics. However, in the 19th century, graph theory began to find applications in other scientific disciplines. Trees and Chemistry In 1857, the British mathematician Arthur Cayley utilized a specific type of graph, a connected graph with no cycles, which he called a tree, to count the number of isomers of alkanes in theoretical chemistry. It was also around this time, in 1878, that the mathematician James Joseph Sylvester first officially used the term "graph" in the mathematical sense, drawing an analogy between chemical bonds and mathematical connections. Electrical Circuits In 1847, the German physicist Gustav Kirchhoff applied graph theory concepts to electrical circuits. To calculate the voltage and current in each branch of a complex electrical network, Kirchhoff developed laws that fundamentally relied on finding a spanning tree of the circuit graph. This was one of the earliest instances of graph theory being applied to serious engineering problems. The Icosian Game and Hamiltonian Paths In 1857, Irish mathematician Sir William Rowan Hamilton invented the "Icosian game," a puzzle sold as a wooden dodecahedron with a peg at each vertex. The goal was to find a path that visited every vertex exactly once and returned to the start. While Euler studied paths that visited every edge exactly once (now called Eulerian paths), paths that visit every vertex exactly once are now forever known as Hamiltonian paths. Determining whether a Hamiltonian path exists remains, to this day, an intensely difficult computational problem (NP-complete). History of Graph Theory Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. The Century-Long Battle: The Four Color Theorem Perhaps no problem drove the development of graph theory more than the famous Four Color Conjecture. The problem was first proposed in 1852 by Francis Guthrie, while trying to color the map of counties of England. The conjecture simply states: Given any separation of a plane into contiguous regions (like a political map of countries), the regions can be colored using at most four colors so that no two adjacent regions have the same color. This problem translates perfectly into graph theory: if every region is a vertex, and an edge connects regions that share a border, can you color the vertices using only four colors such that no two connected vertices share a color? False Starts and Frustration The problem seems deceptively simple. In 1879, Alfred Kempe published a proof that was widely accepted. Eleven years later, in 1890, Percy Heawood found a fatal flaw in Kempe's proof (though Heawood was able to salvage the proof to prove the Five Color Theorem). In 1880, Peter Tait published a different proof; eleven years later, Julius Petersen found a flaw in that one as well. For decades, the brightest mathematical minds failed to prove the Four Color Conjecture, driving significant advancements in the study of planar graphs, vertex coloring, and chromatic polynomials. The Computer-Assisted Breakthrough Finally, in 1976, mathematicians Kenneth Appel and Wolfgang Haken provided a proof. However, it was highly controversial. Their proof reduced the infinite possible maps down to 1,936 reducible configurations (later refined to 633). To verify that every single configuration was colorable, they relied on over 1,000 hours of computer calculation. This was the first major mathematical theorem proven using a computer. It sparked a massive philosophical debate in the mathematical community. If a human cannot independently verify the steps, is it truly a mathematical proof? Today, computer-assisted proofs are widely accepted, but the Four Color Theorem remains a defining moment in the history of mathematics. Interactive Graph Coloring Want to try to color a complex map with only 3 or 4 colors? Use our interactive Graph Coloring visualizer to understand why vertex coloring is such a complex algorithmic challenge. Launch Coloring Visualizer 4. The Probabilistic Revolution: Erdős and Rényi For over two hundred years, graph theory focused on static, deterministic structures. If you had a specific graph, you asked specific questions about it. But in the late 1950s, the field experienced a monumental paradigm shift thanks to the brilliant Hungarian mathematicians Paul Erdős and Alfréd Rényi. They introduced the concept of Random Graphs, transforming graph theory from a purely structural discipline into a probabilistic and statistical science. The Erdős–Rényi Model They proposed a simple model: take n vertices. For every possible pair of vertices, flip a biased coin. With probability p, draw an edge between them. With probability 1 - p, do not. Instead of asking "does this graph have a Hamiltonian path?", mathematicians began asking, "what is the probability that a random graph has a Hamiltonian path?" Phase Transitions and the Giant Component Erdős and Rényi discovered something breathtaking. As you slowly increase the probability p of edges forming, properties don't appear gradually, they appear suddenly. They discovered sudden phase transitions, much like water suddenly freezing into ice at exactly zero degrees. For instance, if p is small, the graph is a scattered collection of small, disconnected components. But the moment p crosses a specific critical threshold, a "Giant Component" abruptly emerges, instantly connecting a massive fraction of all vertices into a single network. This mathematical discovery proved to be fundamental for understanding everything from the spread of infectious diseases to the resilience of the internet. The Erdős–Rényi phase transition. Below an average degree of 1 the graph is a scatter of tiny pieces; the instant the threshold is crossed, a single giant component abruptly connects most of the vertices. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. The Modern Era: Algorithms, Networks, and Machine Learning With the advent of the computer age in the latter half of the 20th century, graph theory exploded. It moved from abstract mathematics to the absolute core of computer science. The Algorithmic Boom (1950s - 1980s) As computers became capable of processing large datasets, researchers focused intensely on developing efficient algorithms to traverse and manipulate graphs. Edsger W. Dijkstra published his famous Shortest Path algorithm in 1959. Ford and Fulkerson published their Max-Flow algorithm in 1956. Robert Tarjan developed linear-time algorithms for finding strongly connected components and articulation points in the 1970s. Network Science and The Web (1990s - 2000s) As the internet grew, researchers realized that the Erdős-Rényi random graph model did not accurately describe real-world networks. Real networks, like the World Wide Web or human social networks, are not purely random. In 1999, Albert-László Barabási and Réka Albert introduced the Scale-Free Network model, characterized by the presence of highly connected "hubs." At the same time, Duncan Watts and Steven Strogatz formalized the "Small-World" phenomenon (the six degrees of separation). Perhaps the most famous modern application of graph theory was Larry Page and Sergey Brin's PageRank algorithm, which fundamentally treated the entire World Wide Web as a massive directed graph, using eigenvector centrality to rank the importance of websites and birth the Google search engine. Graph Neural Networks (2010s - Present) Today, graph theory has merged with artificial intelligence. Standard neural networks struggle with unstructured, asymmetric data. Graph Neural Networks (GNNs) were developed to allow machine learning models to directly process graph structures. GNNs are currently driving massive breakthroughs in drug discovery (by treating molecules as graphs of atoms), traffic prediction (by treating road networks as graphs), and recommendation systems (by modeling user-item interactions as bipartite graphs). Three centuries at a glance: from a recreational puzzle about bridges in 1736 to the Graph Neural Networks powering modern AI. 6. Conclusion: A Branch that Connects Everything The history of graph theory is a testament to the power of mathematical abstraction. What began as an effort to solve a trivial puzzle about bridges over a river in Prussia has evolved into the definitive mathematical language for describing complex relationships. From mapping the intricate wiring of the human brain (the connectome) to optimizing global supply chains, from finding the fastest route home on Google Maps to discovering the chemical structures of new antibiotics, graph theory remains the invisible architecture of our deeply connected world. Frequently Asked Questions Who is credited with founding graph theory? Swiss mathematician Leonhard Euler is considered the father of graph theory, starting with his historic paper solving the Königsberg bridge problem in 1736. What was the Seven Bridges of Königsberg puzzle? A puzzle asking if one could walk through the city of Königsberg crossing all seven of its bridges exactly once. Euler proved mathematically that this is impossible. Why is the Four Color Theorem historically significant? Proposed in 1852 and solved in 1976, it was the first major mathematical theorem to be proven using a computer program, sparking significant philosophical discussions on computerized proofs. Related Articles Vertices and Edges Explained Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Applications of Graph Theory Source: https://learngraphtheory.org/articles/applications-of-graph-theory.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Introduction: The World is Connected - 2. Search Engines: The PageRank Algorithm - 3. Social Networks: Mapping Human Connections - 4. GPS and Navigation Systems - 5. E-Commerce Recommendation Systems - 6. Machine Learning: Graph Neural Networks - 7. Frequently Asked Questions (FAQ) 1. Introduction: The World is Connected Graph theory is often perceived as an abstract branch of mathematics, born in 1736 when Leonhard Euler solved the famous Seven Bridges of Königsberg problem. However, today, it is arguably the most practically applicable field of discrete mathematics. At its core, a graph is simply a collection of dots (called nodes or vertices) connected by lines (called edges). While this sounds simple, this structure is uniquely capable of modeling complex relationships. If you can define entities and the relationships between them, you can model it as a graph. Because the real world is highly interconnected, traditional relational databases (tables with rows and columns) often struggle to capture the nuance of these connections. Graphs excel precisely where traditional databases fail. Let's explore the top 5 ways graph theory silently runs our digital lives. 2. Search Engines: The PageRank Algorithm In the late 1990s, search engines struggled to provide relevant results. They mostly relied on keyword density (how many times a word appeared on a page). This was easily manipulated, leading to poor user experiences. Then came Google. Larry Page and Sergey Brin realized that the World Wide Web is fundamentally a massive, directed graph. In this graph: - Nodes are individual web pages. - Edges are hyperlinks connecting one page to another. They invented the PageRank algorithm, which uses the structure of this graph to measure the importance of website pages. The core idea is simple but revolutionary: a page is considered important if other important pages link to it. A hyperlink from a highly authoritative site (like Wikipedia or BBC) carries much more "weight" than a link from an obscure personal blog. How it works: PageRank calculates the probability that a person randomly clicking on links will arrive at any particular page. It performs massive matrix multiplications across billions of nodes to achieve a steady-state probability for the entire web graph. While modern search algorithms are far more complex and incorporate thousands of machine learning signals, the graph-theoretic foundation of PageRank remains one of the most important inventions of the internet era. Applications of Graph Theory Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. Social Networks: Mapping Human Connections Companies like Facebook, LinkedIn, and X (formerly Twitter) are essentially massive graph databases. The entire premise of social media relies on modeling human relationships. In a social graph: - Nodes represent users, groups, pages, or locations. - Edges represent relationships like "friends with," "follows," "likes," or "lives in." Graph algorithms are used extensively to enhance user experience: "People You May Know" Have you ever wondered how LinkedIn accurately suggests colleagues, or Facebook suggests long-lost high school friends? They use graph algorithms to find triadic closures. If Node A is friends with Node B, and Node B is friends with Node C, the algorithm calculates the probability that Node A and Node C should also be friends based on their mutual connections. Community Detection Algorithms like the Louvain method or Girvan-Newman algorithm are used to identify clusters within the network. By analyzing edge density, social networks can group users into distinct communities (e.g., "tech enthusiasts," "local sports fans," "alumni") even if the users never explicitly declared those interests, allowing for highly targeted advertising. 4. GPS and Navigation Systems Perhaps the most direct and visual application of graph theory is in routing and navigation. Applications like Google Maps, Waze, and logistics software for companies like Amazon and FedEx rely entirely on graph algorithms. In a road network graph: - Nodes are intersections or specific addresses. - Edges are the roads connecting them. - Weights on the edges represent the time, distance, or cost to travel that segment. Finding the Shortest Path When you ask your GPS for directions home, it doesn't look at every possible route. It uses algorithms like Dijkstra's Algorithm or A* (A-Star) Search. A* is an optimized version of Dijkstra that uses a heuristic (like the straight-line distance to the destination) to "pull" the search in the right direction, ignoring roads that obviously lead away from the goal. Dynamic Edge Weights What makes modern navigation incredible is that the edge weights are not static. Waze and Google Maps constantly update the weights of the edges based on real-time traffic data, accidents, and road closures. If an edge weight (traffic time) suddenly spikes, the graph algorithm instantly recalculates the shortest path, offering you a detour. The same weighted-graph model scales up from one driver to an entire distribution network. Graph theory in supply chain optimization works one four-echelon network end to end: lead times as shortest paths, bottlenecks as a minimum cut, and the cheapest shipping plan as a minimum cost flow. See Shortest Path Algorithms in Action Curious how Dijkstra and A* actually navigate a grid? You don't need to guess. Watch the algorithms search through obstacles in real-time. Try the Pathfinding Visualizer Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. E-Commerce Recommendation Systems "Customers who bought this item also bought..." Whether you are on Amazon, Netflix, or Spotify, recommendation engines drive a massive percentage of engagement and revenue. While there are many ways to build these engines (like collaborative filtering), graph-based approaches are among the most powerful. These systems often use Bipartite Graphs. A bipartite graph has two distinct sets of nodes where edges only connect nodes from different sets. - Set A: Users - Set B: Products (or Movies, or Songs) - Edges: Represent interactions (e.g., User 1 "purchased" Product X, or User 2 "rated" Movie Y with 5 stars). By traversing this graph, algorithms can find users who have similar edge patterns to you. If the graph shows that you and User B have highly similar connections to a set of movies, the algorithm will find edges (movies) connected to User B that are not yet connected to you, and recommend them. 6. Machine Learning: Graph Neural Networks (GNNs) The cutting edge of artificial intelligence is currently intersecting with graph theory in the form of Graph Neural Networks (GNNs). Traditional neural networks (like CNNs for images or RNNs for text) expect data to be neatly formatted in grids or sequences. However, much of the world's data is unstructured and relational (like a molecular structure or a financial transaction network). GNNs are designed to operate directly on graph structures. Drug Discovery and Chemistry In a molecular graph, nodes are atoms and edges are chemical bonds. GNNs can "learn" the properties of a molecule by analyzing the graph structure. This allows pharmaceutical companies to rapidly screen millions of chemical compounds to predict which ones might be effective drugs, significantly accelerating the drug discovery process. Molecules are only one of the graphs biology runs on: graph theory in biology covers protein interaction networks, gene regulation motifs, genome assembly as an Eulerian path, and food web collapse. Fraud Detection Banks use graph databases to model financial transactions. A node is a bank account, and a directed edge is a transfer of money. Fraud rings often create complex webs of transactions moving money through hundreds of accounts to hide the source. Graph algorithms can easily detect these circular transaction patterns or unusually dense clusters of activity that traditional tabular databases would miss entirely. Security teams model their networks the same way: graph theory in cybersecurity turns hosts and privileges into an attack graph, then finds the easiest intrusion route and the cheapest set of controls that breaks every one of them. Frequently Asked Questions How does graph theory benefit social network analysis? It models users as nodes and friendships/connections as edges, enabling community detection, influencer identification, and recommendation algorithms (like "people you may know"). How is graph theory used in web search engines? Algorithms like Google PageRank treat web pages as nodes and hyperlinks as directed edges. By analyzing the link structure, they can measure page authority and rank search results. What role does graph theory play in routing and logistics? It represents delivery hubs and intersections as nodes and roads as weighted edges. Shortest path and network flow algorithms are used to optimize delivery routes and reduce transit times. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Graph Theory in Software Engineering Source: https://learngraphtheory.org/articles/software-engineering-concepts.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. Introduction: The Graphs You Already Use - 2. Version Control: Git is a Directed Acyclic Graph (DAG) - 3. Package Managers & Dependency Resolution (Topological Sort) - 4. Build Systems (Make, Bazel, Webpack) - 5. Memory Management: Garbage Collection via Graph Traversal - 6. Microservice Architectures and Tracing - 7. Frequently Asked Questions (FAQ) 1. Introduction: The Graphs You Already Use When software engineers hear "Graph Theory," many think of LeetCode whiteboard interviews or abstract pathfinding problems like Dijkstra's algorithm. However, graph theory is deeply woven into the fabric of daily software development. Every time you commit code, run `npm install`, compile an application, or rely on a programming language to free memory, you are invoking sophisticated graph algorithms. Understanding how these tools map to graph theory doesn't just satisfy intellectual curiosity; it makes you a vastly better engineer capable of debugging deep system failures. In this guide, we'll strip away the magic and look at practical software engineering concepts through the lens of nodes and edges. 2. Version Control: Git is a Directed Acyclic Graph (DAG) Git, the ubiquitous version control system created by Linus Torvalds, is not a linear timeline of changes. At its absolute core, Git is a Directed Acyclic Graph (DAG). Let's break down that term: - Directed: The connections (edges) have a specific direction. In Git, every commit points backward to its parent commit(s). - Acyclic: There are no loops. A commit cannot be its own parent, nor can it point to a future commit that points back to it. - Graph: It consists of nodes (commits) and edges (parent pointers). Branches and Merges in a DAG In Git, a "branch" is simply a movable pointer to a specific node in the graph. When you create a new commit, a new node is created pointing back to the current node, and the branch pointer moves forward. When you merge two branches, Git creates a new "Merge Commit." This is a special node that has two parent edges, pointing to the tips of the two branches you just merged. Because Git tracks the history as a graph, it can run graph traversal algorithms (like finding the Lowest Common Ancestor) to figure out exactly how to perform a 3-way merge automatically. Understanding that Git is a DAG clarifies complex commands: - git rebase: You are literally unplugging a sub-graph of nodes and attaching it to a different node in the graph. - git cherry-pick: You are duplicating a node and appending the copy to your current location in the graph. - git log --graph: This command explicitly draws the DAG in your terminal. Graph Theory in Software Engineering Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 3. Package Managers & Dependency Resolution Whenever you run npm install, pip install, or cargo build, you are asking a package manager to resolve a massive dependency graph. If Package A requires Package B, and Package B requires Package C, the package manager must install C before B, and B before A. This relationship is modeled as a Directed Acyclic Graph (DAG) where nodes are packages and directed edges represent dependencies ("A depends on B"). Topological Sorting To figure out the exact order in which to download and install packages, package managers use an algorithm called Topological Sorting. A Topological Sort takes a DAG and outputs a linear ordering of its nodes such that for every directed edge U -> V (U depends on V), node V comes before node U in the ordering. It ensures that no package is installed before the packages it relies upon are ready. Dependency Hell and Cyclic Dependencies Topological sorting only works on Acyclic graphs. If Package A depends on B, and Package B depends on A, you have a cycle. The graph traversal algorithm will detect this cycle and the package manager will throw an error, famously known as a "Cyclic Dependency." Furthermore, solving version conflicts (e.g., A needs C v1.0, but B needs C v2.0) transforms the graph problem into a boolean satisfiability problem (SAT), which is NP-Complete. Modern package managers use advanced SAT solvers over the dependency graph to find a valid resolution. The same dependency graph is also an attack surface, since a compromise anywhere upstream reaches everything that transitively depends on it. Graph theory in cybersecurity covers that reachability argument alongside attack graphs and minimum-cut segmentation. Interactive Topological Sort Visualize how a Build System uses Topological Sorting to figure out the exact order of compilation. Create your own dependency graphs and watch the algorithm unravel them. Launch Topological Sort Visualizer 4. Build Systems (Make, Bazel, Webpack) Just like package managers, Build Systems rely entirely on graph theory. When you type make, the system looks at a Makefile to understand the targets and their prerequisites. The build system constructs a Dependency Graph of files. If main.o depends on main.c and math.h, edges are drawn from main.o to the source files. Graph Traversal for Incremental Builds The true power of build graphs is incremental compilation. If you change a single file (e.g., math.h) in a project with 10,000 files, you don't want to recompile everything. The build system performs a graph traversal (like BFS or DFS) starting from the modified node (math.h) and following the directed edges backwards to find all the targets that depend on it. It only rebuilds the sub-graph that was "tainted" by the change, leaving the rest of the compiled artifacts untouched. This is how massive monorepo build tools like Google's Bazel achieve lightning-fast compilation times. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Memory Management: Garbage Collection If you use a language with automatic memory management (like Java, Python, JavaScript, or C#), you rely on a Garbage Collector (GC) to free memory that is no longer in use. But how does the computer know what memory is "safe" to delete? The answer is Graph Reachability. The "Mark and Sweep" Algorithm The heap memory of a running program is a massive, highly connected graph. The nodes are objects in memory, and the edges are references (pointers) from one object to another. To free memory safely, the Garbage Collector uses a graph algorithm known as Mark and Sweep: - The Roots: The GC identifies "Root" nodes. These are objects that are definitely active, such as global variables or variables in the current execution stack. - The Mark Phase (Graph Traversal): The GC starts at the Root nodes and performs a Depth-First Search (DFS) or Breadth-First Search (BFS) through memory. Every time it visits an object, it marks it as "reachable" (alive). - The Sweep Phase: Once the traversal is complete, the GC scans all of memory. Any object that is not marked as reachable is considered garbage because the program has no way to access it anymore. The GC deletes these unreachable nodes and reclaims the memory. Without graph traversal, modern high-level programming languages simply wouldn't work. 6. Microservice Architectures and Tracing As monolithic applications break down into microservices, the architecture of a company's backend becomes a vast, distributed graph. Each microservice is a Node, and an API call over the network is an Edge. Distributed Tracing When a user clicks "Checkout" on an e-commerce site, that single action might trigger a cascade of 50 different microservice calls (Auth Service -> Inventory Service -> Payment Gateway -> Notification Service). If the checkout process takes 5 seconds, how do you know which service is causing the bottleneck? Engineers use Distributed Tracing (like Jaeger or OpenTelemetry). These tools inject an ID into the initial request and pass it along the edges of the network. The result is a literal Graph visualization of the request's journey, allowing engineers to pinpoint exactly which node (service) introduced latency or threw an error. Network Resilience and Centrality By analyzing the microservice graph, Site Reliability Engineers (SREs) can use graph metrics like Degree Centrality or Betweenness Centrality to find single points of failure. If one service is a central hub that 80% of other services depend on, it becomes a critical target for high-availability scaling and caching strategies. Frequently Asked Questions How do version control systems like Git use graphs? Git represents repository history as a Directed Acyclic Graph (DAG), where commits are nodes that point to their parent commits, enabling branching and merging operations. What is a dependency graph in build tools? It is a directed graph where nodes represent source files or packages and edges represent imports/dependencies. Build tools run topological sorting to compile dependencies in the correct order. When are graph databases preferred over relational databases? Graph databases (e.g., Neo4j) are preferred when data relationships are highly interconnected, nested, or dynamic, such as in social networks, recommendation systems, or fraud detection. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Graph Algorithms for Coding Interviews Source: https://learngraphtheory.org/articles/coding-interviews.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Table of Contents - 1. How to Spot a Graph Problem in Disguise - 2. Pattern 1: Breadth-First Search (BFS) for Shortest Paths - 3. Pattern 2: Depth-First Search (DFS) for Connectivity - 4. Pattern 3: Topological Sorting for Dependencies - 5. Pattern 4: Union-Find for Connected Components - 6. Pattern 5: Dijkstra's for Weighted Shortest Path - 7. Frequently Asked Questions (FAQ) 1. How to Spot a Graph Problem in Disguise During an interview at a major tech company, you rarely receive a prompt that explicitly says, "Here is a directed acyclic graph, please traverse it." Instead, graph problems are usually disguised as real-world scenarios or grid-based puzzles. You should immediately start thinking about graph algorithms if the problem description involves any of the following themes: - Connections: "Find people who are friends of friends," "Determine if there is a flight between City A and City B." - Dependencies: "You have a list of tasks and prerequisites, what order should you complete them?" or "Compile a project with dependencies." - Grids and Mazes: "Find the shortest path out of this 2D array maze," "Count the number of islands in a grid." - Transformations: "Change one word into another word by changing a single letter at a time (Word Ladder)." Once you identify the problem as a graph problem, your next step is to figure out the best representation. Adjacency Lists (using Hash Maps or Lists of Lists) are generally the best choice for interview settings because they are memory efficient and fast to iterate over. Grids (2D arrays) act as implicit graphs, where each cell is a node and its adjacent cells are its edges. 2. Pattern 1: Breadth-First Search (BFS) for Shortest Paths Breadth-First Search is your go-to algorithm whenever a problem asks for the "shortest path" or the "minimum number of steps" in an unweighted graph. Practice it live in the BFS visualizer. BFS explores the graph level by level, radiating outward like ripples in a pond. Because it explores all nodes at a distance of 1 before exploring any nodes at a distance of 2, the first time it reaches the destination node, it is guaranteed to have found the shortest path. Key Identifying Keywords: Shortest path, minimum steps, closest, nearest, level-order traversal. Implementation Pattern (Python): BFS always requires a Queue (FIFO data structure) and a Visited Set to prevent infinite loops. from collections import deque def bfs_shortest_path(graph, start, target): queue = deque([(start, 0)]) # (current_node, distance) visited = set([start]) while queue: node, distance = queue.popleft() if node == target: return distance for neighbor in graph[node]: if neighbor not in visited: visited.add(neighbor) queue.append((neighbor, distance + 1)) return -1 # Path not found Classic LeetCode Problems: Word Ladder, Shortest Path in Binary Matrix, Rotting Oranges. Graph Algorithms for Coding Interviews Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three For BFS on its own, in more depth, see the eight worked problems in BFS interview questions. 3. Pattern 2: Depth-First Search (DFS) for Connectivity Depth-First Search explores a graph by going as deep as possible down one path before backtracking. While it is rarely used to find the shortest path, it is incredibly efficient for exploring the entire structure of a graph, finding components, or checking if a path exists at all. Practice it live in the DFS visualizer. DFS is highly favored in interviews because it can be implemented very concisely using recursion. Key Identifying Keywords: Connectivity, reachability, all paths, exploring regions, backtracking, deep search. Implementation Pattern (Python): DFS requires a Stack (LIFO), which is most commonly handled implicitly by the call stack via recursion. def dfs_traverse(graph, start, visited=None): if visited is None: visited = set() visited.add(start) # Process the node here for neighbor in graph[start]: if neighbor not in visited: dfs_traverse(graph, neighbor, visited) return visited Note: If you are doing backtracking (like finding all valid paths or permutations), you will need to remove the node from the `visited` set after the recursive call returns. Classic LeetCode Problems: Number of Islands, Clone Graph, Pacific Atlantic Water Flow. Visualize BFS vs DFS Understanding the visual difference between how BFS spreads and DFS plunges is crucial for knowing which to apply during an interview. Compare BFS & DFS Interactively 4. Pattern 3: Topological Sorting for Dependencies If a problem asks you to order items based on prerequisites, you need Topological Sorting. This algorithm only works on Directed Acyclic Graphs (DAGs). If there is a cycle (e.g., Task A requires Task B, and Task B requires Task A), a topological sort is impossible. Deep dive: the full guide to topological sorting covers both Kahn's algorithm and the DFS variant, and topological sort interview questions works through the patterns interviewers actually ask. Or try the topological sort visualizer. The most intuitive way to implement this in an interview is using Kahn's Algorithm, which relies on calculating the "in-degree" (number of incoming edges) of every node. Key Identifying Keywords: Prerequisites, dependencies, scheduling, ordering, compiling, resolving. Implementation Pattern (Python): from collections import deque, defaultdict def topological_sort(num_courses, prerequisites): # 1. Build Graph and In-Degree array graph = defaultdict(list) in_degree = {i: 0 for i in range(num_courses)} for course, prereq in prerequisites: graph[prereq].append(course) in_degree[course] += 1 # 2. Find all nodes with 0 in-degree (no prerequisites) queue = deque([node for node in in_degree if in_degree[node] == 0]) top_order = [] # 3. Process the queue while queue: node = queue.popleft() top_order.append(node) for neighbor in graph[node]: in_degree[neighbor] -= 1 if in_degree[neighbor] == 0: queue.append(neighbor) # 4. Check for cycles if len(top_order) == num_courses: return top_order return [] # Cycle detected Classic LeetCode Problems: Course Schedule I & II, Alien Dictionary. Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 5. Pattern 4: Union-Find (Disjoint Sets) for Connected Components Union-Find is a specialized data structure used specifically to track the partitioning of a set into disjoint sub-sets. It answers two questions blazingly fast (in nearly O(1) time): Deep dive: the full guide to Union-Find explains path compression and union by rank with code, and union-find interview questions applies it to the problems that come up in interviews. - Are Node A and Node B in the same component? - Can we merge the component containing Node A with the component containing Node B? It is the perfect tool for finding cycles in undirected graphs or grouping items dynamically. Key Identifying Keywords: Connected components, dynamic connectivity, grouping, redundant connection, merging sets. Implementation Pattern (Python): You must memorize the implementation of a Union-Find class, specifically remembering to include Path Compression in the `find` method to ensure optimal time complexity. class UnionFind: def __init__(self, size): self.parent = [i for i in range(size)] self.rank = [1] * size def find(self, x): # Path compression if self.parent[x] != x: self.parent[x] = self.find(self.parent[x]) return self.parent[x] def union(self, x, y): root_x = self.find(x) root_y = self.find(y) if root_x == root_y: return False # Already in same set (cycle detected) # Union by rank if self.rank[root_x] > self.rank[root_y]: self.parent[root_y] = root_x elif self.rank[root_x] < self.rank[root_y]: self.parent[root_x] = root_y else: self.parent[root_y] = root_x self.rank[root_x] += 1 return True Classic LeetCode Problems: Redundant Connection, Number of Connected Components in an Undirected Graph, Accounts Merge. 6. Pattern 5: Dijkstra's for Weighted Shortest Path If the problem asks for the shortest path, but the edges have weights (costs, times, distances), standard BFS will fail. You need Dijkstra's algorithm. Deep dive: the full guide to Dijkstra's algorithm, or run it in the Dijkstra visualizer. Dijkstra's is essentially BFS, but instead of a standard Queue, it uses a Priority Queue (Min-Heap) to ensure that you always explore the cheapest available path next. Key Identifying Keywords: Shortest path with costs, cheapest flight, minimum time, network delay. Implementation Pattern (Python): import heapq def dijkstra(graph, start, target): # Priority queue stores tuples of (total_cost, node) pq = [(0, start)] # Dictionary to keep track of the minimum cost to reach each node min_cost = {start: 0} while pq: current_cost, node = heapq.heappop(pq) # If we reached the target, this is guaranteed to be the shortest path if node == target: return current_cost # If we found a shorter path previously, ignore this older tuple if current_cost > min_cost.get(node, float('inf')): continue for neighbor, weight in graph[node]: new_cost = current_cost + weight # Only push to queue if we found a strictly better path if new_cost < min_cost.get(neighbor, float('inf')): min_cost[neighbor] = new_cost heapq.heappush(pq, (new_cost, neighbor)) return -1 Classic LeetCode Problems: Network Delay Time, Cheapest Flights Within K Stops (Note: Bellman-Ford is also good for this one), Path With Maximum Probability. Frequently Asked Questions What are the most common graph questions in coding interviews? The most common topics are finding connected components, cycle detection, topological sorting (e.g., Course Schedule), and shortest paths using Dijkstra's. What graph representations are expected in interviews? Adjacency Lists are the standard expectation because they are space-efficient O(V + E). Be comfortable converting input edge lists into Adjacency Lists. How do I identify a graph-based interview problem? If a problem describes entities with relationships, dependency networks, matrix traversals (like grid navigation), or transitive rules, it can usually be modeled as a graph. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → --- ## Graph Theory Interview Questions Source: https://learngraphtheory.org/articles/interview-questions.html Mohammed Islam Hadjoudj Expert Operations Research Engineer Comprehensive Table of Contents - 1. Introduction: The FAANG Graph Expectation - 2. Core Concepts & Space-Time Tradeoffs - 3. Pattern 1: The Implicit Grid Graph - 3.1 Number of Islands (Classic) - 3.2 Number of Distinct Islands (Advanced) - 4. Pattern 2: Traversal & State Management - 4.1 Clone Graph - 4.2 Pacific Atlantic Water Flow (Multi-Source) - 5. Pattern 3: Cycle Detection & Topological Sort - 5.1 Course Schedule (Kahn's Algorithm) - 5.2 Alien Dictionary (Hard) - 6. Pattern 4: Shortest Paths (Unweighted & Weighted) - 6.1 Word Ladder (BFS) - 6.2 Network Delay Time (Dijkstra's) - 7. Pattern 5: Disjoint Sets (Union-Find) - 7.1 Redundant Connection - 8. Execution Strategy for the 45-Minute Interview 1. Introduction: The FAANG Graph Expectation If you are reviewing standard algorithm texts like Introduction to Algorithms (CLRS) or Elements of Programming Interviews (EPI), you will quickly note that graph theory constitutes one of the largest and most theoretically dense chapters. In the context of coding interviews at top-tier companies, graph problems are frequently utilized as a critical filtering mechanism. Why do interviewers rely so heavily on graphs? - Multi-dimensional Evaluation: A single graph question assesses your understanding of fundamental data structures (Hash Maps, Queues, Stacks, Priority Queues), recursion, tree traversal (since trees are restricted graphs), and space-time complexity analysis. - Real-World Applicability: Graphs natively model networks. Whether it's routing internet traffic, calculating mutual friends on a social platform, resolving software dependencies (like npm or pip), or finding the fastest route on Google Maps, graphs are the underlying data layer. - Pattern Recognition: Interviewers want to see if you can abstract a verbose, scenario-based prompt into a standard mathematical model (vertices and edges) and apply a known pattern. "The secret to mastering graph interviews is realizing that there are rarely 'new' graph questions. There are only new disguises for about six core graph patterns.", Cracking the Coding Interview (CTCI) methodologies. 2. Core Concepts & Space-Time Tradeoffs Before diving into specific patterns, let's establish the foundational vocabulary and the critical tradeoffs you must discuss with your interviewer during the planning phase. Representing the Graph You generally have three ways to represent a graph in memory. Choosing the right one is often the first test. - Adjacency List (Most Common): A Hash Map (or Array of Arrays) where keys are vertices and values are lists of neighboring vertices. Time to find neighbors: O(1). Space: O(V + E). Use this 95% of the time. - Adjacency Matrix: A 2D array of size V x V where matrix[i][j] = 1 indicates an edge. Time to check a specific edge: O(1). Space: O(V^2). Only use this for very dense graphs where E ≈ V^2. - Edge List: A simple list of coordinate pairs [[u, v], [x, y]]. Often how the input is provided, but terrible for traversal. Always convert this to an Adjacency List before processing. The Golden Rule of Graph Traversal Unlike trees, graphs can have cycles. If you do not track which nodes you have already visited, your algorithm will enter an infinite loop resulting in a Stack Overflow or Time Limit Exceeded (TLE) error. Always maintain a visited Hash Set or Array. 3. Pattern 1: The Implicit Grid Graph Often, a problem will hand you a 2D matrix representing a map, a maze, or a board. The trick is to realize that the matrix itself is the graph. Each cell (r, c) is a vertex, and an edge exists between it and its immediate orthogonal neighbors: (r+1, c), (r-1, c), (r, c+1), (r, c-1). Pattern Recognition Clues: You are given a 2D grid/matrix. You are asked to find connected regions, the largest region, or traverse a maze. Algorithm: DFS or BFS starting from specific trigger cells. Modify the grid in-place to act as the visited set to achieve O(1) auxiliary space (if allowed by the interviewer). 3.1 Number of Islands (Classic) The Problem: Given an m x n 2D binary grid representing a map of '1's (land) and '0's (water), return the number of islands. An island is surrounded by water and is formed by connecting adjacent lands horizontally or vertically. The Approach: We iterate through every cell. When we hit a '1', it indicates a new island. We increment our counter, and then use DFS to "sink" the island (convert all connected '1's to '0's). This ensures we don't count the same island twice. def numIslands(grid): if not grid: return 0 rows, cols = len(grid), len(grid[0]) islands = 0 def dfs(r, c): # Base case: Out of bounds or water if r < 0 or c < 0 or r >= rows or c >= cols or grid[r][c] == '0': return # Sink the land (mark as visited) grid[r][c] = '0' # Explore all 4 directions dfs(r + 1, c) dfs(r - 1, c) dfs(r, c + 1) dfs(r, c - 1) for r in range(rows): for c in range(cols): if grid[r][c] == '1': islands += 1 dfs(r, c) return islands Complexity Analysis: - Time Complexity: O(M * N) where M is rows and N is columns. Every cell is visited at most a constant number of times. - Space Complexity: O(M * N) in the worst case (if the entire grid is one massive island, the call stack will go M*N deep). If modifying the input is forbidden, an external visited set requires O(M * N) space. 3.2 Number of Distinct Islands (Advanced Variation) The Problem: Similar to Number of Islands, but return the number of unique island shapes. Two islands are considered the same if one can be translated (not rotated or reflected) to equal the other. The Approach: We still use DFS to find islands. However, we need a way to "fingerprint" or serialize the shape of the island. We can do this by recording the traversal path relative to the starting cell (e.g., "Right, Down, Left, Up"). We store these string signatures in a Hash Set. The final answer is the size of the set. This is a classic example of combining a graph traversal pattern with string serialization to solve a harder constraint. Graph Theory Interview Questions Master every graph algorithm, not just one The Handbook, the Masterclass and the Interview Prep Kit: 55 algorithms, 109 slides and 50 interview problems, explained as clearly as this article. $12.99 from $10.99 See all three 4. Pattern 2: Traversal & State Management This pattern tests your raw ability to traverse an explicit graph (usually provided via an adjacency list or node references) while keeping careful track of what has been built or visited to handle cyclic dependencies natively. Pattern Recognition Clues: "Return a deep copy", "Find all nodes that can reach X". Algorithm: DFS or BFS paired heavily with a Hash Map to store states, mappings, or reachability caching (memoization). 4.1 Clone Graph The Problem: Given a reference of a node in a connected undirected graph, return a deep copy (clone) of the graph. Each node contains a value (int) and a list (List[Node]) of its neighbors. The Approach: The primary danger is infinite recursion due to cycles. We use a Hash Map where the key is the original node and the value is the newly cloned node. During DFS, if a node is already in the map, we simply return its clone. class Node: def __init__(self, val = 0, neighbors = None): self.val = val self.neighbors = neighbors if neighbors is not None else [] def cloneGraph(node): if not node: return None # Map original node -> cloned node old_to_new = {} def dfs(node): if node in old_to_new: return old_to_new[node] # Create clone and add to map BEFORE iterating neighbors copy = Node(node.val) old_to_new[node] = copy for nei in node.neighbors: copy.neighbors.append(dfs(nei)) return copy return dfs(node) Complexity Analysis: - Time Complexity: O(V + E). We visit every vertex and edge exactly once. - Space Complexity: O(V). The hash map and the recursion stack take space proportional to the number of vertices. 4.2 Pacific Atlantic Water Flow (Multi-Source BFS/DFS) The Problem: Given an m x n matrix of non-negative integers representing the height of each unit cell in a continent, the Pacific ocean touches the left and top edges, and the Atlantic ocean touches the right and bottom edges. Water can flow in 4 directions to neighboring cells with an equal or lower height. Find all grid coordinates where water can flow to both oceans. The Approach: The naive approach is to run a DFS from every single cell and check if it reaches both oceans (O((M*N)^2)). The optimal approach reverses the logic: Start from the oceans and flow uphill. Run a multi-source DFS from all cells touching the Pacific, marking reachable cells. Do the same for the Atlantic. The intersection of both reachable sets is the answer. 5. Pattern 3: Cycle Detection & Topological Sort Directed Acyclic Graphs (DAGs) are prevalent in scheduling, compilation, and dependency resolution. Whenever tasks have prerequisites (A must happen before B), you are dealing with a DAG. If there is a cycle, the tasks cannot be completed. Pattern Recognition Clues: "Prerequisites", "Scheduling", "Order of compilation", "Determine if possible to complete all tasks". Algorithm: Kahn's Algorithm (BFS with In-Degrees) or DFS with a 3-color state (Unvisited, Visiting, Visited) for Cycle Detection. 5.1 Course Schedule (Kahn's Algorithm) The Problem: There are numCourses courses labeled 0 to numCourses - 1. You are given an array prerequisites where [a, b] indicates you must take course b first if you want to take course a. Return true if you can finish all courses. The Approach (Kahn's Algorithm): We calculate the in-degree of every node (how many prerequisites it has). We add all nodes with an in-degree of 0 to a Queue. We pop nodes from the queue, logically "taking" the course, and decrement the in-degree of all its neighbors. If a neighbor's in-degree drops to 0, it is added to the queue. If we process all nodes, there is no cycle. from collections import deque, defaultdict def canFinish(numCourses, prerequisites): adj_list = defaultdict(list) in_degree = {i: 0 for i in range(numCourses)} # Build graph and in-degrees for crs, pre in prerequisites: adj_list[pre].append(crs) in_degree[crs] += 1 # Find all courses with no prerequisites queue = deque([crs for crs in in_degree if in_degree[crs] == 0]) courses_taken = 0 # Process courses while queue: current = queue.popleft() courses_taken += 1 for neighbor in adj_list[current]: in_degree[neighbor] -= 1 if in_degree[neighbor] == 0: queue.append(neighbor) return courses_taken == numCourses 5.2 Alien Dictionary (Hard) The Problem: There is a new alien language that uses the English alphabet. However, the order among the letters is unknown to you. You are given a list of strings words from the alien language's dictionary, where the strings in words are sorted lexicographically by the rules of this new language. Return a string of the unique letters in the new alien language sorted in lexicographically increasing order by the new language's rules. The Approach: This is considered one of the hardest interview questions, but it reduces to a standard Topological Sort. 1. Compare adjacent words to find the first differing character. This establishes a directed edge (e.g., if "wrt" comes before "wrf", then 't' -> 'f'). 2. Build the adjacency list and in-degree map for all unique characters. 3. Run Kahn's Algorithm. If there's a cycle (e.g., a -> b and b -> a), return an empty string. Otherwise, the order in which characters are popped from the queue is the valid alien alphabet. 6. Pattern 4: Shortest Paths (Unweighted & Weighted) Shortest path problems are split entirely based on whether the edges have weights (costs, distances, times) or not. Mixing up the algorithms for these two types is an instant red flag in an interview. Pattern Recognition Clues: "Shortest path", "Minimum steps", "Fastest route", "Cheapest cost". Unweighted Edges Algorithm: Standard Breadth-First Search (BFS). BFS naturally radiates outward in concentric circles, guaranteeing the first time you reach the target is the shortest path. Weighted Edges Algorithm: Dijkstra's Algorithm (using a Min-Heap/Priority Queue). If negative weights exist (rare in interviews), use Bellman-Ford. 6.1 Word Ladder (BFS) The Problem: Given two words, beginWord and endWord, and a dictionary wordList, return the number of words in the shortest transformation sequence from beginWord to endWord. Every adjacent pair of words must differ by exactly one letter. The Approach: This is finding the shortest path in an unweighted graph. The words are nodes, and edges exist between words differing by one letter. from collections import deque def ladderLength(beginWord, endWord, wordList): word_set = set(wordList) if endWord not in word_set: return 0 queue = deque([(beginWord, 1)]) while queue: word, steps = queue.popleft() if word == endWord: return steps # Try changing every character for i in range(len(word)): for char in 'abcdefghijklmnopqrstuvwxyz': next_word = word[:i] + char + word[i+1:] if next_word in word_set: word_set.remove(next_word) # Mark as visited queue.append((next_word, steps + 1)) return 0 Complexity Analysis: - Time Complexity: O(M^2 * N), where M is word length and N is total words. For each word popped, we generate 26 * M new words, and string slicing takes O(M) time. - Space Complexity: O(M * N) to store the words in the queue and the set. 6.2 Network Delay Time (Dijkstra's Algorithm) The Problem: You are given a network of n nodes, labeled from 1 to n. You are also given times, a list of travel times as directed edges times[i] = (ui, vi, wi), where ui is the source, vi is the target, and wi is the time it takes for a signal to travel from source to target. We will send a signal from a given node k. Return the minimum time it takes for all the n nodes to receive the signal. The Approach: Because the edges have weights (times) that are non-negative, this is a textbook application of Dijkstra's Algorithm. We use a Min-Heap to always process the node with the current shortest known distance from the start node. import heapq from collections import defaultdict def networkDelayTime(times, n, k): edges = defaultdict(list) for u, v, w in times: edges[u].append((v, w)) min_heap = [(0, k)] # (distance, node) visited = set() t = 0 while min_heap: w1, n1 = heapq.heappop(min_heap) if n1 in visited: continue visited.add(n1) t = max(t, w1) for n2, w2 in edges[n1]: if n2 not in visited: heapq.heappush(min_heap, (w1 + w2, n2)) return t if len(visited) == n else -1 Complete pack All three resources, one price The Handbook, the Masterclass and the Interview Prep Kit together — the complete path from this article to interview-ready. $42.97 $29.99 Get the complete pack 7. Pattern 5: Disjoint Sets (Union-Find) The Union-Find data structure is incredibly elegant and heavily favored by interviewers at Google and Amazon. It is specifically designed to answer one question extremely fast: "Do these two nodes belong to the same connected component?" Pattern Recognition Clues: "Find connected components", "Detect cycle in an undirected graph", "Find minimum spanning tree", "Are A and B connected?". Algorithm: Disjoint Set Union (DSU) utilizing Path Compression and Union by Rank to achieve nearly O(1) time complexity for operations. 7.1 Redundant Connection The Problem: In this problem, a tree is an undirected graph that is connected and has no cycles. You are given a graph that started as a tree with n nodes, with one additional edge added. The added edge has two different vertices chosen from 1 to n, and was not an edge that already existed. Return an edge that can be removed so that the resulting graph is a tree of n nodes. The Approach: We iterate through the given edges. For each edge (u, v), we check if u and v are already in the same set using Union-Find. If they are, adding this edge creates a cycle, so this edge is the redundant one! If they aren't, we Union them together. class UnionFind: def __init__(self, size): self.parent = list(range(size + 1)) self.rank = [1] * (size + 1) def find(self, n): # Path compression if self.parent[n] != n: self.parent[n] = self.find(self.parent[n]) return self.parent[n] def union(self, n1, n2): p1, p2 = self.find(n1), self.find(n2) if p1 == p2: return False # Cycle detected # Union by rank if self.rank[p1] > self.rank[p2]: self.parent[p2] = p1 self.rank[p1] += self.rank[p2] else: self.parent[p1] = p2 self.rank[p2] += self.rank[p1] return True def findRedundantConnection(edges): uf = UnionFind(len(edges)) for u, v in edges: if not uf.union(u, v): return [u, v] Complexity Analysis: - Time Complexity: O(E * α(V)) where α is the Inverse Ackermann function. In practice, this is O(1) per operation, meaning overall time is linear with respect to the number of edges. - Space Complexity: O(V) to store the parent and rank arrays. 8. Execution Strategy for the 45-Minute Interview Having the technical knowledge is only half the battle. Executing flawlessly under pressure requires a strict behavioral framework. Follow this sequence when handed a graph problem: - Clarify the Graph Characteristics (Minutes 1-5): - Is it Directed or Undirected? - Are the edges Weighted or Unweighted? - Can there be cycles? - Can the graph be disconnected? (Crucial for knowing if you need an outer loop over all vertices). - State the Model (Minutes 5-10): Explicitly tell the interviewer, "I will model this as a directed, unweighted graph where the nodes are X and the edges are Y. Because we are looking for the shortest path, I will apply Breadth-First Search." - Analyze Complexity Before Coding (Minutes 10-15): Define V and E in terms of the problem's variables. State the time and space complexity of your proposed solution. Get the interviewer's nod of approval before touching the whiteboard/editor. - Implement Mechanically (Minutes 15-35): If you recognized the pattern, the implementation should be muscle memory. Always write out your visited set first to avoid forgetting it. - Dry Run (Minutes 35-45): Trace through your code with a small example. Manually update your queues and visited sets as you walk through line by line. This catches 90% of off-by-one errors. By studying these five foundational patterns, Implicit Grids, Traversal, Topological Sort, Shortest Paths, and Disjoint Sets, you transition from trying to memorize hundreds of LeetCode solutions to simply mapping new problems to known architectural blueprints. Good luck with your interview! Frequently Asked Questions How do you detect a cycle in a directed vs. undirected graph? For directed graphs, use DFS with a recursion stack (back edges). For undirected graphs, use DFS/BFS or Union-Find, checking if a visited node is not the parent of the current node. What is Topological Sort and when is it applicable? Topological sorting is a linear ordering of vertices such that for every directed edge u -> v, u comes before v. It is only applicable to Directed Acyclic Graphs (DAGs). What is the difference between Kruskal's and Prim's MST algorithms? Kruskal's is an edge-based greedy algorithm that sorts edges and adds them using Union-Find. Prim's is a node-based greedy algorithm that builds the tree node-by-node from a starting vertex. Related Articles Graph Theory Interview Questions Read more → Shortest Path Algorithms Read more → Network Flow & Max-Cut Read more → ---