Dynamic Programming

We began our study of algorithmic techniques with greedy algorithms, which in some sense form the most natural approach to algorithm design. Faced with a new computational problem, we‘ve seen that it‘s not hard to propose multiple possible greedy algorithms; the challenge is then to determine whether any of these algorithms provides a correct solution to the problem in all cases.

6.1 Weighted Interval Scheduling: A Recursive Procedure

We have seen that a particular greedy algorithm produces an optimal solution to the Interval Scheduling Problem, where the goal is to accept as large a set of nonoverlapping intervals as possible. The weighted Interval Scheduling Problem is a strictly more general version, in which each interval has a certain value (or weight), and we want to accept a set of maximum value.

Designing a Recursive Algorithm

Since the original Interval Scheduling Problem is simply the special case in which all values are equal to 1, we know already that most greedy algorithms will not solve this problem optimally. But even the algorithm that worked before (repeatedly choosing the interval that ends earliest) is no longer optimal in this more general setting.

Indeed, no natural greedy algorithm is known for this problem, which is what motivates our switch to dynamic programming. As discussed above, we will begin our introduction to dynamic programming with a recursive type of algorithm for this problem, and then in the next section we‘ll move to a more iterative method that is closer to the style we use in the rest of this chapter.

We use the notation from our discussion of Interval Scheduling. We have requests labeled , with each request specifying a start time and a finish time . Each interval now also has a value, or weight . Two intervals are compatible if they do not overlap. The goal of our current problem is to select a subset of mutually compatible intervals, so as to maximize the sum of the values of the selected intervals,

Let‘s suppose that the requests are sorted in order of nondecreasing finish time: . We‘ll say a request comes before a request if . This will be the natural left-to-right order in which we‘ll consider intervals. To help in talking about this order, we define , for an interval , to be the largest index such that intervals and are disjoint. In other words, is the leftmost interval that ends before begins. We define if no request is disjoint from .

Now, given an instance of the Weighted Interval Scheduling Problem, let‘s consider an optimal solution , ignoring for now that we have no idea what it is. Here‘s something completely obvious that we can say about : either interval (the last one) belongs to , or it doesn‘t. Suppose we explore both sides of this dichotomy a little further. If , then clearly no interval indexed strictly between and can belong to , because by the definition of , we know that intervals all overlap interval . Moreover, if , then must include an optimal solution to the problem consisting of requests - for if it didn‘t, we could replace ‘s choice of requests from with a better one, with no danger of overlapping request .

On the other hand, if , then is simply equal to the optimal solution to the problem consisting of requests . This is by completely analogous reasoning: we‘re assuming that does not include request ; so if it does not choose the optimal set of requests from , we could replace it with a better one.

All this suggests that finding the optimal solution on intervals involves looking at the optimal solutions of smaller problems of the form . Thus, for any value of between and , let denote the optimal solution to the problem consisting of requests , and let denote the value of this solution. (We define , based on the convention that this is the optimum over an empty set of intervals.) The optimal solution we‘re seeking is precisely , with value . For the optimal solution on , our reasoning above (generalizing from the case in which ) says that either , in which case , or , in which case . Since these are precisely the two possible choices ( or ), we can further say that.

(1)

And how do we decide whether belongs t the optimal solution . This too is easy: it belongs to the optimal solution if and only if the first of the options above is at least as good as the second; in other words,

Request belongs to an optimal solution on the set if and only if

(2)

These facts form the first crucial component on which a dynamic programming solution is based: a recurrence equation that expresses the optimal solution (or its value) in terms of the optimal solutions to smaller subproblems.

Despite the simple reasoning that led to this point, (1) is already a significant development. It directly gives us a recursive algorithm to compute , assuming that we have already sorted the requests by finishing time and computed the values of for each .

If then

Return

Else

Return

Endif

The correctness of the algorithm follows directly by induction on :

correctly computes for each .

Proof. By definition . Now, take some , and suppose by way of induction that correctly computes for all . By the induction hypothesis, we know that and ; and hence from (1) it follows that

Unfortunately, if we really implemented the algorithm as just written, it would take exponential time to run in the worst case.

Memoizing the Recursion

In fact, though, we‘re not so far from having a polynomial-time algorithm. A fundamental observation, which forms the second crucial component of a dynamic programming solution, is that our recursive algorithm is really only solving different subproblems: . The fact that it runs in exponential time as written is simply due to the spectacular redundancy in the number of times it issues each of these calls.

How could we eliminate all this redundancy? We could store the value of in a globally accessible place the first time we compute it and then simply use this precomputed value in place of all future recursive calls. This technique of saving values that have already been computed is referred to as memoization.

We implement the above strategy in the more “intelligent” procedure . This procedure will make use of an array ; will start with the value “empty”, but will hold the value of as soon as it is first determined. To determine , we invoke .