1 .1 . The Structure of Production Planning Models.
Production planning or smoothing problems are g en erally tackled by form ulating a mathematical model o f the production and commercial environment. This has been discussed in Chapter 2 , but the e sse n tia l p oints are worth re ite ra tin g here.
The production environment i s u s u a lly modelled over some fixe d time in t e r v a l, perhaps a y e a r, which is subdivided into time periods o f, s a y , a month or q u a rter. Key a ttrib u te s of the production/inventory/ w orkforce system, fo r example, production le v e ls , sa le s and stocks are then considered to change from one time period to the next and v a ria tio n w ith in each time period is ignored. The model is then manipulated and "so lve d " so as to provide the best i n i t i a l d ecisio n s (f o r example, f i r s t time period production le v e l s ) , which optim ise some a ttrib u te of in t e r e s t , fo r example, expected p r o f it .
The atte n tio n of th is chapter is p rim a rily d irected towards dynamic programming so lutio ns to models in which the sto c h a stic elements are im portant i . e . those models which d ir e c t ly involve u n ce rtain ty in some fu tu re a ttrib u te s of the p ro cess. In th is case atte n tio n is r e s t r ic t e d to the maximisation o f expected p r o f it . M inimisation o f expected loss o r cost can be handled in e x a c tly the same way: ju s t co n sid er maximising minus the expected c o s t.
1 .2 . The B a s ic Elem ents o f Dynamic Prograimring
The form o f dynamic programming applied here w ill be that re le v a n t to f in it e time horizon models as opposed to "steady s ta te " models where discounted expected p r o fit or average expected p ro fit per time period is maximised over an in f in it e horizon. The optim isation problem i s subdivided into subproblems p ertaining to each time period. Each subproblem i s concerned with the maximisation of expected revenue from the time period in question to the time horizon. The o p tim isatio n must be performed fo r each possible s ta te of the system a t the s t a r t o f the time p eriod, and the subproblems are solved back wards in the sense th at the f i r s t subproblem is that of maximising the
expected la s t time period p r o f it , the second subproblem is th a t of maximising the expected p r o fit in the la s t two time periods and so on, u n til the la s t subproblem which is th at of maximising the to ta l
expected p r o fit accrued in a l l time p eriods. At each stage use is made o f the re s u lts o f the previous stage. The optim isation in the fin a l subproblem i s , o f course, only performed fo r the i n i t i a l s ta te of the system. T h is provides the optimal so lutio n to such models where decisions have to be implemented over time.
1 .3 . Advantages and Disadvantages of Dynamic Programming
Although dynamic programming techniques th e o re tic a lly provide r e lia b ly optimal s o lu tio n s , com putationally they s u ffe r from the "curse of d im e n sio n ality ". This is now explained. The state of the system a t the s t a r t o f any time period is known as the state space. This is u su ally
-46-
ch aracte rise d by an m-dimensional ve c to r, l . e . a point in F m (m- dimenslonal Euclidean sp ace). The o p tim isatio n fo r each subproblem
(except the la s t ) must be performed fo r each possible value of the sta te v a ria b le a t the s t a r t of the f i r s t time period of the subproblem. In p ra c tic e , in dynamic programming approaches, the domain of possible sta te v a ria b le values is covered by a grid and optim isation performed fo r each ve rte x of the g rid . I f N grid p oints are taken fo r each
sta te space dimension then the subproblem in vo lve s some Nm optim isatio ns. Thus the computational complexity of the dynamic programming approach explodes exp on entially w ith the number of s t a t e space dimensions. This is a severe lim ita tio n of the method which u s u a lly r e s t r ic t s i t s a p p lica tio n to models w ith four or le s s s t a t e space dimensions. An approximate approach, which is fe a s ib le i f the number of state space dimensions is sm a ll, is to assume some parameterized functional form fo r the maximum expected future revenue as a function o f the present s ta te va ria b le value and to perform the o p tim isatio n over the parameters of the fu n ctio n .
Thus dynamic programming is unable to handle the general m u lti- commodity production smoothing problem. M u ltip le commodities must be aggregated to be amenable to th is approach. However i f the number of s ta te space dimensions is sm a ll, dynamic programming may be e f f ic ie n t . When the sta te space has only one dimension i t may, indeed, be more e f f ic ie n t than any other method. Moreover i t s computational complexity expands only lin e a r ly w ith the number o f time periods in the model. This compares favourably with other approaches, fo r example, lin e a r programming, where the computational com plexity would expand approximately as the cube of the number of time p eriods.
1 .4 . The Contents o f the Chapter
The p rin c ip le o f dynamic programming is treated more form ally in Section 2, f i r s t in a f a i r l y general way and then applied to a simple production/inventory model. This model 1s tackle d 1n d e ta il in Section 3 , where i t is re s tric te d to one state space dimension and an e f f ic ie n t algorithm fo r it s so lutio n is d erived. In Section 4 the m ulti-dim ensional ve rsio n is discussed together with the d if f ic u lt i e s th at i t presents. The chapter ends with a d iscu ssio n of the outcome of th is in v e stig a tio n in to dynamic programming techniques.
-48-
2 . THE DYNAMIC PROGRAMMING APPROACH
2 .1 . A General Formulation
Suppose that the production/inventory system is modelled as a Markov process. C haracterize the state of the system or s ta te va ria b le a t the end o f time period t by the random vector q^. At the s t a r t of the t th time period suppose th a t contro ls x t are applied and random in p u t y t re a lis e d . In a p p lica tio n s the s ta te va ria b le might represent sto ck le v e ls , the co n tro ls:p ro d u ctio n targets and the random input: demand.
Suppose that the revenue accrued in the t th time p e rio d , V^, is some prescribed functio n of qt l , x t and y