Algorithms and Complexity - ...

[ Pobierz całość w formacie PDF ]
Algorithms and Complexity
Herbert S. Wilf
University of Pennsylvania
Philadelphia, PA 19104-6395
Copyright Notice
Copyright 1994 by Herbert S. Wilf. This material may be reproduced for any educational purpose, multiple
copies may be made for classes, etc. Charges, if any, for reproduced copies must be just enough to recover
reasonable costs of reproduction. Reproduction for commercial purposes is prohibited. This cover page must
be included in all distributed copies.
Internet Edition, Summer, 1994
This edition of Algorithms and Complexity is available at the web site
.
It may be taken at no charge by all interested persons. Comments and corrections are welcome, and should
be sent to
wilf@math.upenn.edu
A Second Edition of this book was published in 2003 and can be purchased now. The Second Edition contains
solutions to most of the exercises.
CONTENTS
Chapter 0: What This Book Is About
0.1Background ......................................1
0.2 Hard vs. easy problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
0.3Apreview.......................................4
Chapter 1: Mathematical Preliminaries
1.1 Orders of magnitude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Positional number systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Manipulations with series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4 Recurrence relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5Counting ...................................... 21
1.6Graphs ....................................... 24
Chapter 2: Recursive Algorithms
2.1Introduction..................................... 30
2.2Quicksort ...................................... 31
2.3 Recursive graph algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4 Fast matrix multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.5 The discrete Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.6 Applications of the FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.7Areview....................................... 60
Chapter 3: The Network Flow Problem
3.1Introduction..................................... 63
3.2 Algorithms for the network ow problem . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.3 The algorithm of Ford and Fulkerson . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.4 The max-ow min-cut theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.5 The complexity of the Ford-Fulkerson algorithm . . . . . . . . . . . . . . . . . . . . . 70
3.6Layerednetworks................................... 72
3.7 The MPM Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.8 Applications of network ow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Chapter 4: Algorithms in the Theory of Numbers
4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.2 The greatest common divisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.3 The extended Euclidean algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4 Primality testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.5 Interlude: the ring of integers modulo
n
......................... 89
4.6 Pseudoprimality tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.7 Proof of goodness of the strong pseudoprimality test . . . . . . . . . . . . . . . . . . . . 94
4.8 Factoring and cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.9 Factoring large integers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.10 Proving primality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
iii
Chapter 5: NP-completeness
5.1Introduction.....................................104
5.2 Turing machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.3 Cook’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.4 Some other NP-complete problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.5Hafaloaf......................................119
5.6 Backtracking (I): independent sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.7 Backtracking (II): graph coloring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.8 Approximate algorithms for hard problems . . . . . . . . . . . . . . . . . . . . . . . . 128
iv
Preface
For the past several years mathematics majors in the computing track at the University of Pennsylvania
have taken a course in continuous algorithms (numerical analysis) in the junior year, and in discrete algo-
rithms in the senior year. This book has grown out of the senior course as I have been teaching it recently.
It has also been tried out on a large class of computer science and mathematics majors, including seniors
and graduate students, with good results.
Selection by the instructor of topics of interest will be very important, because normally I’ve found
that I can’t cover anywhere near all of this material in a semester. A reasonable choice for a rst try might
be to begin with Chapter 2 (recursive algorithms) which contains lots of motivation. Then, as new ideas
are needed in Chapter 2, one might delve into the appropriate sections of Chapter 1 to get the concepts
and techniques well in hand. After Chapter 2, Chapter 4, on number theory, discusses material that is
extremely attractive, and surprisingly pure and applicable at the same time. Chapter 5 would be next, since
the foundations would then all be in place. Finally, material from Chapter 3, which is rather independent
of the rest of the book, but is strongly connected to combinatorial algorithms in general, might be studied
as time permits.
Throughout the book there are opportunities to ask students to write programs and get them running.
These are not mentioned explicitly, with a few exceptions, but will be obvious when encountered. Students
should all have the experience of writing, debugging, and using a program that is nontrivially recursive,
for example. The concept of recursion is subtle and powerful, and is helped a lot by hands-on practice.
Any of the algorithms of Chapter 2 would be suitable for this purpose. The recursive graph algorithms are
particularly recommended since they are usually quite foreign to students’ previous experience and therefore
have great learning value.
In addition to the exercises that appear in this book, then, student assignments might consist of writing
occasional programs, as well as delivering reports in class on assigned readings. The latter might be found
among the references cited in the bibliographies in each chapter.
I am indebted rst of all to the students on whom I worked out these ideas, and second to a num-
ber of colleagues for their helpful advice and friendly criticism. Among the latter I will mention Richard
Brualdi, Daniel Kleitman, Albert Nijenhuis, Robert Tarjan and Alan Tucker. For the no-doubt-numerous
shortcomings that remain, I accept full responsibility.
This book was typeset in T
E
X. To the extent that it’s a delight to look at, thank T
E
X. For the deciencies
in its appearance, thank my limitations as a typesetter. It was, however, a pleasure for me to have had the
chance to typeset my own book. My thanks to the Computer Science department of the University of
Pennsylvania, and particularly to Aravind Joshi, for generously allowing me the use of T
E
X facilities.
Herbert S. Wilf
v
Chapter 0: What This Book Is About
0.1 Background
An algorithm is a method for solving a class of problems on a computer. The complexity of an algorithm
is the cost, measured in running time, or storage, or whatever units are relevant, of using the algorithm to
solve one of those problems.
This book is about algorithms and complexity, and so it is about methods for solving problems on
computers and the costs (usually the running time) of using those methods.
Computing takes time. Some problems take a very long time, others can be done quickly. Some problems
seem
to take a long time, and then someone discovers a faster way to do them (a ‘faster algorithm’). The
study of the amount of computational eort that is needed in order to perform certain kinds of computations
is the study of computational
complexity
.
Naturally, we would expect that a computing problem for which millions of bits of input data are
required would probably take longer than another problem that needs only a few items of input. So the time
complexity of a calculation is measured by expressing the running time of the calculation
as a function of
some measure of the amount of data that is needed to describe the problem to the computer.
For instance, think about this statement: ‘I just bought a matrix inversion program, and it can invert
an
n × n
matrix in just 1
.
2
n
3
minutes.’ We see here a typical description of the complexity of a certain
algorithm. The running time of the program is being given as a function of the size of the input matrix.
A faster program for the same job might run in 0
.
8
n
3
minutes for an
n × n
matrix. If someone were
to make a really important discovery (see section 2.4), then maybe we could actually lower the exponent,
instead of merely shaving the multiplicative constant. Thus, a program that would invert an
n × n
matrix
in only 7
n
2
.
8
minutes would represent a striking improvement of the state of the art.
For the purposes of this book, a computation that is guaranteed to take at most
cn
3
time for input of
size
n
will be thought of as an ‘easy’ computation. One that needs at most
n
10
time is also easy. If a certain
calculation on an
n × n
matrix were to require 2
n
minutes, then that would be a ‘hard’ problem. Naturally
some of the computations that we are calling ‘easy’ may take a very long time to run, but still, from our
present point of view the important distinction to maintain will be the polynomial time guarantee or lack of
it.
The general rule is that if the running time is at most a polynomial function of the amount of input
data, then the calculation is an easy one, otherwise it’s hard.
Many problems in computer science are known to be easy. To convince someone that a problem is easy,
it is enough to describe a fast method for solving that problem. To convince someone that a problem is
hard is hard, because you will have to prove to them that it is
impossible
to nd a fast way of doing the
calculation. It will
not
be enough to point to a particular algorithm and to lament its slowness. After all,
that
algorithm may be slow, but maybe there’s a faster way.
Matrix inversion is easy. The familiar Gaussian elimination method can invert an
n ×n
matrix in time
at most
cn
3
.
To give an example of a hard computational problem we have to go far aeld. One interesting one is
called the ‘tiling problem.’ Suppose* we are given innitely many identical oor tiles, each shaped like a
regular hexagon. Then we can tile the whole plane with them,
i.e.
, we can cover the plane with no empty
spaces left over. This can also be done if the tiles are identical rectangles, but not if they are regular
pentagons.
In Fig. 0.1 we show a tiling of the plane by identical rectangles, and in Fig. 0.2 is a tiling by regular
hexagons.
That raises a number of theoretical and computational questions. One computational question is this.
Suppose we are given a certain polygon, not necessarily regular and not necessarily convex, and suppose we
have innitely many identical tiles in that shape. Can we or can we not succeed in tiling the whole plane?
That elegant question has been
proved
* to be computationally unsolvable. In other words, not only do
we not know of any fast way to solve that problem on a computer, it has been
proved
that there isn’t
any
* See, for instance, Martin Gardner’s article in
Scientic American
, January 1977, pp. 110-121.
* R. Berger, The undecidability of the domino problem,
Memoirs Amer. Math. Soc.
66
(1966), Amer.
[ Pobierz całość w formacie PDF ]