A Guide to Algorithm Design: Paradigms, Methods, and Complexity Analysis

Transcription

1 A Guide to Algorithm Design: Paradigms, Methods, and Complexity Analysis Anne Benoit, Yves Robert, Frédéric Vivien To cite this version: Anne Benoit, Yves Robert, Frédéric Vivien. A Guide to Algorithm Design: Paradigms, Methods, and Complexity Analysis. Chapman Hall/CRC, pp.380, 2013, Applied Algorithms and Data Structures series, > <hal- HAL Id: hal Submitted on 23 Nov 2013 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Anne Benoit, Yves Robert, Frédéric Vivien A guide to algorithm design Paradigms, methods and complexity analysis CRC PRESS Boca Raton London New York Washington, D.C.

3 Contents Preface xv I Polynomial-time algorithms: Exercises 1 1 Introduction to complexity On the complexity to compute x n Naive method Binary method Factorization method Knuth s tree method Complexity results Asymptotic notations: O, o, Θ, and Ω Exercises Exercise 1.1: Longest balanced section Exercise 1.2: Find the star Exercise 1.3: Breaking boxes Exercise 1.4: Maximum of n integers Exercise 1.5: Maximum and minimum of n integers Exercise 1.6: Maximum and second maximum of n integers. 11 Exercise 1.7: Merging two sorted sets Exercise 1.8: The toolbox Exercise 1.9: Sorting a small number of objects Solutions to exercises Solution to Exercise 1.1: Longest balanced section Solution to Exercise 1.2: Find the star Solution to Exercise 1.3: Breaking boxes Solution to Exercise 1.4: Maximum of n integers Solution to Exercise 1.5: Maximum and minimum of n integers 20 Solution to Exercise 1.6: Maximum and second maximum of n integers Solution to Exercise 1.7: Merging two sorted sets Solution to Exercise 1.8: The toolbox Solution to Exercise 1.9: Sorting a small number of objects Bibliographical notes v

4 vi 2 Divide-and-conquer Strassen s algorithm Master theorem Solving recurrences Solving homogeneous recurrences Solving nonhomogeneous recurrences Solving the recurrence for Strassen s algorithm Exercises Exercise 2.1: Product of two polynomials Exercise 2.2: Toeplitz matrices Exercise 2.3: Maximum sum Exercise 2.4: Boolean matrices: The Four-Russians algorithm 41 Exercise 2.5: Matrix multiplication and inversion Solutions to exercises Solution to Exercise 2.1: Product of two polynomials Solution to Exercise 2.2: Toeplitz matrices Solution to Exercise 2.3: Maximum sum Solution to Exercise 2.4: Boolean matrices: The Four-Russians algorithm Solution to Exercise 2.5: Matrix multiplication and inversion Bibliographical notes Greedy algorithms Motivating example: The sports hall Designing greedy algorithms Graph coloring On coloring bipartite graphs Greedy algorithms to color general graphs Coloring interval graphs Theory of matroids Exercises Exercise 3.1: Interval cover Exercise 3.2: Memory usage Exercise 3.3: Scheduling dependent tasks on several machines 65 Exercise 3.4: Scheduling independent tasks with priorities.. 66 Exercise 3.5: Scheduling independent tasks with deadlines.. 66 Exercise 3.6: Edge matroids Exercise 3.7: Huffman code Solutions to exercises Solution to Exercise 3.1: Interval cover Solution to Exercise 3.2: Memory usage Solution to Exercise 3.3: Scheduling dependent tasks on several machines Solution to Exercise 3.4: Scheduling independent tasks with priorities

5 Solution to Exercise 3.5: Scheduling independent tasks with deadlines Solution to Exercise 3.6: Edge matroids Solution to Exercise 3.7: Huffman code Bibliographical notes Dynamic programming The coin changing problem The knapsack problem Designing dynamic-programming algorithms Exercises Exercise 4.1: Matrix chains Exercise 4.2: The library Exercise 4.3: Polygon triangulation Exercise 4.4: Square of ones Exercise 4.5: The wind band Exercise 4.6: Ski rental Exercise 4.7: Building set Solutions to exercises Solution to Exercise 4.1: Matrix chains Solution to Exercise 4.2: The library Solution to Exercise 4.3: Polygon triangulation Solution to Exercise 4.4: Square of ones Solution to Exercise 4.5: The wind band Solution to Exercise 4.6: Ski rental Solution to Exercise 4.7: Building set Bibliographical notes Amortized analysis Methods for amortized analysis Running examples Aggregate analysis Accounting method Potential method Exercises Exercise 5.1: Binary counter Exercise 5.2: Inserting and deleting Exercise 5.3: Stack Exercise 5.4: Deleting half the elements Exercise 5.5: Searching and inserting Exercise 5.6: Splay trees Exercise 5.7: Half perimeter of a polygon Solutions to exercises Solution to Exercise 5.1: Binary counter Solution to Exercise 5.2: Inserting and deleting vii

6 viii Solution to Exercise 5.3: Stack Solution to Exercise 5.4: Deleting half the elements Solution to Exercise 5.5: Searching and inserting Solution to Exercise 5.6: Splay trees Solution to Exercise 5.7: Half perimeter of a polygon Bibliographical notes II NP-completeness and beyond NP-completeness A practical approach to complexity theory Problem classes Problems in P Problems in NP NP-complete problems and reduction theory Polynomial reduction Cook s theorem Growing the class NPC of NP-complete problems Optimization problems versus decision problems Examples of NP-complete problems and reductions SAT CLIQUE VERTEX-COVER Scheduling problems Other famous NP-complete problems Importance of problem definition Strong NP-completeness Why does it matter? Bibliographical notes Exercises on NP-completeness Easy reductions Exercise 7.1: Wheel Exercise 7.2: Knights of the round table Exercise 7.3: Variants of CLIQUE Exercise 7.4: Path with vertex pairs Exercise 7.5: VERTEX-COVER with even degrees Exercise 7.6: Around 2-PARTITION About graph coloring Exercise 7.7: COLOR Exercise 7.8: 3-COLOR Exercise 7.9: 3-COLOR-PLAN Scheduling problems Exercise 7.10: Scheduling independent tasks with p processors 152

7 Exercise 7.11: Scheduling with two processors More involved reductions Exercise 7.12: Transitive subchain Exercise 7.13: INDEPENDENT SET Exercise 7.14: DOMINATING SET Exercise 7.15: Carpenter Exercise 7.16: k-center Exercise 7.17: Variants of 3-SAT Exercise 7.18: Variants of SAT PARTITION is NP-complete Exercise 7.19: SUBSET-SUM Exercise 7.20: NP-completeness of 2-PARTITION Solutions to exercises Solution to Exercise 7.1: Wheel Solution to Exercise 7.2: Knights of the round table Solution to Exercise 7.3: Variants of CLIQUE Solution to Exercise 7.4: Path with vertex pairs Solution to Exercise 7.5: VERTEX-COVER with even degrees 158 Solution to Exercise 7.6: Around 2-PARTITION Solution to Exercise 7.7: COLOR Solution to Exercise 7.8: 3-COLOR Solution to Exercise 7.9: 3-COLOR-PLAN Solution to Exercise 7.10: Scheduling independent tasks with p processors Solution to Exercise 7.11: Scheduling with two processors Solution to Exercise 7.12: Transitive subchain Solution to Exercise 7.13: INDEPENDENT SET Solution to Exercise 7.14: DOMINATING SET Solution to Exercise 7.15: Carpenter Solution to Exercise 7.16: k-center Solution to Exercise 7.17: Variants of 3-SAT Solution to Exercise 7.18: Variants of SAT Solution to Exercise 7.19: SUBSET-SUM Solution to Exercise 7.20: NP-completeness of 2-PARTITION Bibliographical notes Beyond NP-completeness Approximation results Approximation algorithms Vertex cover Traveling salesman problem (TSP) Bin packing PARTITION Polynomial problem instances Partitioning problems ix

8 x Assessing problem complexity Linear programming Formal definition Relaxation and rounding Randomized algorithms The algorithm Results Branch-and-bound and backtracking Backtracking: The n queens Branch-and-bound: The knapsack Graph algorithms Bibliographical notes Exercises going beyond NP-completeness Approximation results Exercise 9.1: Single machine scheduling Exercise 9.2: SUBSET-SUM Exercise 9.3: SET-COVER Exercise 9.4: VERTEX-COVER Exercise 9.5: Scheduling independent tasks in parallel Exercise 9.6: Point clustering Exercise 9.7: k-center Exercise 9.8: Knapsack Dealing with NP-complete problems Exercise 9.9: Mixed integer linear program for replica placement Exercise 9.10: A randomized algorithm for independent set. 218 Exercise 9.11: Branch-and-bound applied to MAX-SAT Solutions to exercises Solution to Exercise 9.1: Single machine scheduling Solution to Exercise 9.2: SUBSET-SUM Solution to Exercise 9.3: SET-COVER Solution to Exercise 9.4: VERTEX-COVER Solution to Exercise 9.5: Scheduling independent tasks in parallel Solution to Exercise 9.6: Point clustering Solution to Exercise 9.7: k-center Solution to Exercise 9.8: Knapsack Solution to Exercise 9.9: Mixed integer linear program for replica placement Solution to Exercise 9.10: A randomized algorithm for independent set Solution to Exercise 9.11: Branch-and-bound applied to MAX- SAT Bibliographical notes

9 III Reasoning on problem complexity Reasoning to assess a problem complexity Basic reasoning Polynomial instances NP-complete instances Set of problems with polynomial-time algorithms Set of NP-complete problems Numbers Graphs Chains-on-chains partitioning Optimal algorithms for homogeneous resources Dynamic-programming algorithm Binary search algorithm Improved algorithms Variants of the problem Communication costs Chain of heterogeneous resources Extension to a clique of heterogeneous resources NP-completeness Practical solutions Integer linear program Conclusion Replica placement in tree networks Access policies Motivation Impact of the policies on the existence of a solution Impact of the policies on the cost of a solution Complexity results Definitions MinNb problem MinCost problem Integer linear program Variants of the replica placement problem Enforcing a quality of service Power-aware replica placement Conclusion Packet routing MEDP: Maximum edge-disjoint paths Problem statement Naive greedy algorithm Short-requests-first greedy algorithm xi

10 xii Inapproximability result PRVP: Packet routing with variable-paths Problem statement Bounding optimal makespan via linear programming Routing algorithm Steady-state approach Conclusion Matrix product, or tiling the unit square Problem motivation NP-completeness A guaranteed heuristic The ColPeriSum(s) problem Performance guarantee Looking for a better solution Related problems Online scheduling Flow time optimization Competitive analysis Definition Method to establish a competitive analysis result Makespan optimization List scheduling algorithms Randomized optimization of makespan Conclusion References 349 Index 359

11 List of exercises 1.1 Longest balanced section Find the star Breaking boxes Maximum of n integers Maximum and minimum of n integers Maximum and second maximum of n integers Merging two sorted sets The toolbox Sorting a small number of objects Product of two polynomials Toeplitz matrices Maximum sum Boolean matrices: The Four-Russians algorithm Matrix multiplication and inversion Interval cover Memory usage Scheduling dependent tasks on several machines Scheduling independent tasks with priorities Scheduling independent tasks with deadlines Edge matroids Huffman code Matrix chains The library Polygon triangulation Square of ones The wind band Ski rental Building set Binary counter Inserting and deleting Stack Deleting half the elements Searching and inserting xiii

12 xiv 5.6 Splay trees Half perimeter of a polygon Wheel Knights of the round table Variants of CLIQUE Path with vertex pairs VERTEX-COVER with even degrees Around 2-PARTITION COLOR COLOR COLOR-PLAN Scheduling independent tasks with p processors Scheduling with two processors Transitive subchain INDEPENDENT SET DOMINATING SET Carpenter k-center Variants of 3-SAT Variants of SAT SUBSET-SUM NP-completeness of 2-PARTITION Single machine scheduling SUBSET-SUM SET-COVER VERTEX-COVER Scheduling independent tasks in parallel Point clustering k-center Knapsack Mixed integer linear program for replica placement A randomized algorithm for independent set Branch-and-bound applied to MAX-SAT