Semidefinite relaxations with the method of moments of dynamical programs under discrete variables

Texto completo

(1)Semidefinite Relaxations with the Method of Moments of Dynamical Programs under Discrete Variables. by. Camilo Ortiz. A thesis presented to the department of Ingeniería Industrial as requisite for obtaining the degree of Maestría en Ingeniería Industrial. Director: René Meziat. Universidad de los Andes Bogotá, Colombia January 13, 2008.

(2) Abstract This work proposes the use of a semidefinite relaxation for solving non-linear, non-convex dynamical programs under discrete constraints in the state variables and the control variables. The principal theoretical features of the methodology are outlined, funding its results in previous findings where the Method of Moments is used as a tool for obtaining convex relaxations of discrete programs which includes polynomial constraints and cost functions. In sum, I intend to apply the method of moments for generating a semidefinite relaxation to dynamic programming problems with discrete variables in order to obtain optimal solutions more efficiently. A three step relaxation is proposed: a polynomial, a convex and finally a semidefinite relaxation. Two benchmark problems are worked out to illustrate this proposal: one problem refers to a simple model of the classical inventory problem under discrete constraints; and the other is a specific situation in chess involving knights and rooks. Keywords: dynamic programming, integer programming, semidefinite relaxations, method of moments, chess, inventory problem..

(3) Contents 1 Introduction 1.1 Inventory Problem Under Discrete Constraints . . . . . . . . . . . 1.2 The Knight Problem . . . . . . . . . . . . . . . . . . . . . . . . .. 1 2 3. 2 Analysis of the Methodology 2.1 Polynomial Relaxation of the Discrete Variables . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Algebraic Polynomial Relaxations . . . . 2.1.2 Trigonometric Polynomial Relaxations . 2.1.3 Polynomially Restricted Model . . . . . 2.2 Convex Relaxation of the Polynomial Model . . 2.2.1 Introducing the Probability Moments . . 2.2.2 Construction of the Convex Constraints 2.2.3 Convex Model . . . . . . . . . . . . . . . 2.3 Semidefinite Relaxation Using Hankel and Toeplitz Matrices . . . . . . . . . . . . . . . . . 2.3.1 Using Hankel Matrices . . . . . . . . . . 2.3.2 Using Toeplitz Matrices . . . . . . . . . 2.3.3 Resulting Models . . . . . . . . . . . . .. 7 . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. 9 9 10 10 11 11 12 15. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. 17 17 20 21. 3 Relaxation of the Benchmark Problems 3.1 Semidefinite Relaxation of the Inventory Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Polynomial Relaxation . . . . . . . . . . . . . . . . 3.1.2 Convex Model . . . . . . . . . . . . . . . . . . . . . 3.1.3 Semidefinite Relaxation . . . . . . . . . . . . . . . 3.2 Semidefinite Relaxation of the Knight Problem . . . . . . . 3.2.1 Reduction of the Dimension in the Knight’s Motion 3.2.2 Polynomial Relaxation . . . . . . . . . . . . . . . . 3.2.3 Convex Model . . . . . . . . . . . . . . . . . . . . . 3.2.4 Semidefinite Relaxation . . . . . . . . . . . . . . .. 23 . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. 23 23 24 24 25 25 27 28 29.

(4) ii 4 Instances of the Benchmark Problems 4.1 Solving the Inventory Problem . . . . . 4.1.1 Situation with 12 periods . . . . 4.1.2 Situation with 24 periods . . . . 4.2 Solving the Knight Problem . . . . . . 4.2.1 A Case with Multiple Solutions 4.2.2 A Case with an Unique Solution. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. 30 31 31 32 32 32 34. 5 Conclusions. 36. Bibliography. 38. A Numerical Results A.1 Inventory Problem . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Knight Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 41 41 43.

(5) Chapter 1 Introduction Dynamic Programming is a field in global optimization which involves a great variety of models, even if we restrict to the non-stochastic ones. The conventional general form of a dynamical program will be used, based on the definition given in [1] with the following model: min gN (xN ) +. N −1 X. gt (xt , ut ). (1.1). t=0. s.t.. x 0 = s0 xt+1 = ft (xt , ut ) ∀t ∈ {0, . . . , N − 1}. (1.2). where the cost function in (1.1) depends on the control variables (ut ), the state variables (xt ) and the dynamic system defined in (1.2). The characterization of the variables ut and xt is a key factor for determining the methodology to be used. Here, more specifically, we are mainly interested in the analysis of discrete control variables in Rn , i.e. where we include the following restrictions ut ∈ Ωt = {ct,1 , . . . , ct,kt } ⊆ Rn , ∀t ∈ {0, . . . , N − 1}.. (1.3). Nevertheless, our approach also considers some cases where there are also discrete finite state variables as: xt ∈ Σt = {st,1 , . . . , st,lt } ⊆ Rn , ∀t ∈ {0, . . . , N }.. (1.4). Even if we only include discrete control variables as defined in (1.3) and accept continuous state variables (i.e., (1.4) not considered), we can realize that these are problems which usually have no trivial solution. The presence of discrete variables.

(6) §1.1. 2. in optimization problems has always been a great challenge. As a result, to solve this type of dynamical programs I use an approach which involves the well known method of moments. This approach has been proposed for global and integer optimization in [8, 10, 11, 12, 13, 14, 16]. There are other proposals that use this method in non-linear optimal control [4, 6, 7, 15, 17, 22, 27, 28, 29, 31]. The methodology presented here attempts to avoid the finite discrete variables of these problems, by relaxing the restrictions made in the control variables (1.3) and the state variables (1.4). The relaxations are made by a three step reformulation of the model. The first step relaxes the discrete restrictions into polynomial constraints using Lagrange polynomials with Ωt and Σt as zeros for each polynomial. The second step is made using the method of moments, changing the power of each variable with the corresponding moment of some probability distribution supported in Ωt and Σt respectively. It is vital to emphasize the fact that an exact semidefinite relaxation of the problem ((1.1) and (1.2)) is attained. The use of moments of probability measures supported in one dimensional sets (i.e. contained in R) assures this. Although the approach involves theoretical parameters (measures of probability), these can be finally modeled with a third step as a semidefinite program (SDP) with the use of known results using Hankel and Toeplitz matrices (see [3, 9, 10, 18, 19]). In order to understand the importance of this methodology that uses the method of moments and the semidefinite optimization in dynamic programming problems, two benchmark problems with discrete variables justifying this approach are presented.. 1.1 Inventory Problem Under Discrete Constraints Consider the classical inventory problem modeled as follows: min aN sN +. N −1 X. (at st + bt pt + ct p2t ). (1.5). t=0. s.t.. s0 = sinit st+1 = st + pt − dt+1 ∀t ∈ {0, . . . , N − 1} pt ∈ {r1 , . . . , rl } ⊆ R ∀t ∈ {0, . . . , N − 1}. (1.6). where dt is the estimated demand for the period t, sinit is the initial stock in the inventory, st is the stock for the period t, pt is the production made in period t,.

(7) §1.2. 3. and r1 , . . . , rl are the l possible production quantities for each period. Letters at , bt and ct are the coefficients of the polynomial cost function (1.5). Notice that (1.5) and (1.6) fits as a particular case of (1.1) and (1.2). Each term can be related as follows: General Form. Inventory Problem. ut −→ pt xt −→ st ft (xt , ut ) −→ st + pt − dt+1 N −1 X. gN (xN ) −→ aN sN N −1 X gt (xt , ut ) −→ (at st + bt pt + ct p2t ).. t=0. t=0. 1.2 The Knight Problem In order to fully appreciate the ways in which this methodology can be applied, our research group1 developed a very interesting problem involving the game of chess. The aim of this problem is to find the right trajectory which takes a knight on a chessboard from one corner: (1, 1), to its opposite: (n, n), eluding the attack of several rooks located arbitrarily on the chess board (see Figures 1.1 and 1.2)2 .The mathematical formulation of this problem is: min x2N − 2nxN + yN2 − 2nyN + 2n2 = ||(xN , yN ) − (n, n)||2 s.t.. x0 = 1 y0 = 1 xt+1 = xt + ut yt+1 = yt + vt xt ∈ {1, . . . , n} \ PX yt ∈ {1, . . . , n} \ PY (ut , vt ) ∈ MK. ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1}.. (1.7). (1.8). Here N is the number of steps taken by the knight, (xt , yt ) are the coordinates 1. Grupo de Investigación Métodos Numéricos-Optimización of the Laboratorio de Matemáticas Aplicadas at the Universidad de los Andes. 2 Notice that the usual chessboard has n = 8. This problem is defined for arbitrary board size n × n..

(8) §1.2. 4. 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0s0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 M0Z0Z0Z0 8. a. b. c. d. e. f. g. h. Figure 1.1: Knight Problem.. 0Z0Z0Z0Z Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0s0Z0 4 0Z0M0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 8 7. a. b. c. d. e. f. g. h. Figure 1.2: The allowed moves of the knight in chess..

(9) §1.2. 5. of the knight in each step, (ut , vt ) are the moves made by the knight in each step which necessarily belongs to the set MK = {(2, 1), (1, 2), (−1, 2), (−2, 1), (−2, −1), (−1, −2), (1, −2), (2, −1)}. (1.9) As you can see, MK restricts every ut and vt into a two dimensional, finite, discrete set in R2 . In (1.8), the initial position (1, 1) is guaranteed by the first two constraints and the minimization of the cost function (1.7) assures, if feasible, the final destination of the knight in (n, n). The dynamical system is defined in the constraints xt+1 = xt + ut ∀t ∈ {0, . . . , N − 1} yt+1 = yt + vt ∀t ∈ {0, . . . , N − 1}. (1.10). and the rooks are eluded with the constraints xt ∈ {1, . . . , n} \ PX ∀t ∈ {0, . . . , N } yt ∈ {1, . . . , n} \ PY ∀t ∈ {0, . . . , N }. (1.11). where PX and PY are defined as follows: if the positions of the rooks are PT = {(px,1 , py,1 ), . . . , (px,M , py,M )}. (1.12). PX = {px,1 , . . . , px,M } and. (1.13). = {py,1 , . . . , py,M }.. (1.14). we define:. PY. Again, the formulation ((1.7) and (1.8)) fits in the general form of a discrete dynamical program as defined in (1.1) and (1.2). Each term can be related like this: General Form. Knight Problem ! ut ut −→ vt ! xt xt −→ yt ! ! xt ut ft (xt , ut ) −→ + yt vt gN (xN ) −→ x2N − 2nxN + yN2 − 2nyN + 2n2. N −1 X t=0. gt (xt , ut ) −→ 0.

(10) §1.2. 6. In this problem the control variables (ut , vt ) as well as the state variables (xt , yt ) are restricted to be finite discrete, as seen in (1.8). With these two benchmark problems: the inventory problem and the knight problem, I will illustrate the use of this methodology. Both of them involve a polynomial cost function and discrete control variables..

(11) Chapter 2 Analysis of the Methodology This work is concentrated in relaxing non-convex dynamical programs as given in the general form in (1.1) and (1.2) where at least the control variables are finite discrete. Moreover, its focused in functions gt that can be expressed as the sum of one-dimensional polynomials respect to only one component of the control variable, and ft where each component can be expressed as one-dimensional polynomials respect to only one component of the control variable. So if we have gt expressed as follows gt (xt , ut ) = gt,1 (xt , ut ) + . . . + gt,n (xt , ut ). (2.1). and the componentwise definitions .  ft,1 (xt , ut )   .. , ft (xt , ut ) =  .   ft,n (xt , ut )   ut,1  .  .  ut =   .  and ut,n   xt,1  .   xt =  ..   xt,n. (2.2). (2.3). (2.4).

(12) §2.1. 8. we have then that for each r = 1, . . . , n and each t = 0, . . . , N − 1: gt,r (xt , ut ) =. Jt,r X. αt,r,i (xt )uit,r = αt,r (xt ) · FJt,r (ut,r ),. (2.5). βt,r,i (xt )uit,r = βt,r (xt ) · FLt,r (ut,r ). (2.6). i=0 Lt,r. ft,r (xt , ut ) =. X i=0. where Fn (u) = (1, u1 , . . . , un )0 . One-dimensional trigonometric polynomials are considered as well, i.e. when either gt , ft or both are expressed for each r = 1, . . . , n and each t = 0, . . . , N − 1 as: ˆ. gt,r (xt , ut ) =. Jt,r X. α̂t,r,i (xt )eijut,r = α̂t,r (xt ) · GJ¯t,r ,Jˆt,r (ut,r ),. (2.7). β̂t,r,i (xt )eijut,r = β̂t,r (xt ) · GL̄t,r ,L̂t,r (ut,r ). (2.8). i=J¯t,r. ft,r (xt , ut ) =. L̂t,r X i=L̄t,r. √ where GM,N (u) = (eM ju , . . . , eN ju )0 and j = −1. The relaxation will result into a semidefinite program with the same minimum. Indeed, a convex, exact relaxation will be obtained providing us with information about the solution of the original problem ((1.1) and (1.2)) whenever it has the polynomial structure shown in (2.5) and (2.6) or in (2.7) and (2.8). Also, we have to remember that we worked finite discrete control variables as in (1.3) and possible finite discrete state variables as in (1.4). To attain this aim we follow a recent proposal of J.B. Lasserre in [11] to relax integer polynomial programs and recent proposals given in [4, 5, 7, 10, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 28, 30] to analyze optimal control problems with moments. As indicated in the introduction, the relaxation is made in three steps: a polynomial relaxation made with Lagrange polynomials; a convex relaxation made with the method of moments; and a semidefinite relaxation made with the theory of Hankel and Toeplitz matrices in semidefinite programming. This chapter will present the theoretical facts of this relaxation applied to the general form of a dynamical program given in (1.1) and (1.2) including the polynomial structure shown above and the discrete constraints given in (1.3) and (1.4)..

(13) §2.1. 9. 2.1 Polynomial Relaxation of the Discrete Variables The first reformulation of the general form is applied on the discrete constraints (1.3) and (1.4). Depending on the type of problem, the relaxation is made with algebraic polynomials or trigonometric polynomials. This relaxations are made with the same principles of Lagrange polynomials.. 2.1.1 Algebraic Polynomial Relaxations The main idea of this relaxation is to define algebraic polynomials that have their zeros in the same values as in (1.3) and (1.4). To apply this relaxation it is necessary that each component of ut and xt is separately discrete constrained, i.e. for every r = 1, . . . , n, each component ut,r is restricted: ut,r ∈ {ct,r,1 , . . . , ct,r,kt,r } = Ωt,r , ∀t ∈ {0, . . . , N }. (2.9). and each component xt,r : xt,r ∈ {st,r,1 , . . . , st,r,lt,r } = Σt,r , ∀t ∈ {0, . . . , N }.. (2.10). Then, for each component of the control variables we have the polynomials1 : kt,r Y (ut,r − ct,r,i )2 = γu,t,j · F2kt,r (ut,r ), ∀t ∈ {0, . . . , N − 1} pt,r (ut,r ) =. (2.11). i=1. and for the state variables2 : lt,r Y qt,r (xt,r ) = (xt,r − st,r,i )2 = γx,t,j · F2lt,r (xt,r ), ∀t ∈ {0, . . . , N }. (2.12). i=1. As a result, the discrete restrictions defined in (1.3) and (1.4) can by replaced by the componentwise polynomial constraints pt,r (ut,r ) = 0, ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N − 1}. (2.13). qt,r (xt,r ) = 0, ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N }.. (2.14). and 1 2. γu,t,j is the vector of coefficients of the polynomial pt,r . γx,t,j is the vector of coefficients of the polynomial qt,r ..

(14) §2.1. 10. As the polynomials are constructed as positive Lagrange polynomials based on the sets Ωt,r and Σt,r , the restrictions in (1.3) and (1.4) are equivalent to the polynomial restrictions defined in (2.13) and (2.14). On the other hand, the reason of having positive Lagrange polynomials3 is for further steps which require this characteristic.. 2.1.2 Trigonometric Polynomial Relaxations The other way of relaxing the discrete constraints in (1.3) and (1.4) is with trigonometric polynomials. In this case we also need the discrete restrictions to be applied for each component as in (2.9) and(2.10). Then the trigonometric polynomials for every r = 1, . . . , n are: for every t = 0, . . . , N − 1 −jkt,r ut,r. pt,r (ut,r ) = e. kt,r Y (ejut,r − ejct,r,i )2 = γu,t,j · G−kt,r ,kt,r (ut,r ). (2.15). i=1. and for every t = 0, . . . , N −jlt,r xt,r. qt,r (xt,r ) = e. lt,r Y (ejxt,r − ejst,r,i )2 = γx,t,j · G−lt,r ,lt,r (xt,r ). (2.16). i=1. √ where j = −1. Using this time the polynomials (2.15) and (2.16), we can again replace the discrete restrictions (1.3) and (1.4) with the componentwise polynomial restrictions defined in (2.13) and (2.14). This is a similar construction of positive Lagrange polynomials based on the sets Ωt,r and Σt,r , only here we use trigonometric polynomials. As a result, the restrictions in (1.3) and (1.4) are still equivalent to the polynomial restrictions defined in (2.13) and (2.14) using the trigonometric polynomials defined in (2.15) and (2.16) as the zeros are defined in the sets Ωt,r and Σt,r . Once more, we use positive trigonometric polynomials which are needed for further steps of the relaxation.. 2.1.3 Polynomially Restricted Model We have seen that the polynomial relaxations (algebraic or trigonometric) are exact, i.e. they are equivalent to the discrete constraints and it is illustrated in 3. The polynomials are positive because like in definition (2.11) each factor of the product is squared..

(15) §2.2. 11. the following equations: Ωt,r = {ut,r ∈ R|pt,r (ut,r ) = 0}, ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N } (2.17) Σt,r = {xt,r ∈ R|qt,r (xt,r ) = 0}, ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N }. (2.18) Then the model that finally arises after this first step of the relaxation is: min gN (xN ) +. N −1 X. gt (xt , ut ). (2.19). t=0. s.t.. x0 = xt+1 = pt,r (ut,r ) = qt,r (xt,r ) =. s0 ft (xt , ut ) ∀t ∈ {0, . . . , N − 1} 0 ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N − 1} 0 ∀r ∈ {1, . . . , n}, ∀t ∈ {0, . . . , N }. (2.20). when we have that both, the control variables and the state variables, are discrete. If we only have discrete variables in the control. the model would be without the last constraint in (2.20).. 2.2 Convex Relaxation of the Polynomial Model Now I will explain the second step of the relaxation which consists in changing the model recently presented in (2.19) and (2.20) to a relaxation that will be convex at least respect to the control variable. For this step I will concentrate on the control variables, their discrete restriction sets Ωt,r and their defined polynomials pt,r .. 2.2.1 Introducing the Probability Moments As pt,r (u) ≥ 0, we can easily characterize the set of measures of probability supported in Ωt,r in the following way: P(Ωt,r ) = {µ ∈ P(R)|hpt,r , µi = 0}. (2.21). where P(R) stands for the set of all probability measures in R, P(Ωt,r ) stands for R the set of all probability measures supported in Ωt,r and hpt,r , µi = R pt,r (u)dµ(u). This bracket is the expected value of pt,r respect to the probability distribution µ..

(16) §2.2. 12. The algebraic moments respect to the probability distribution µ ∈ P(R) are obtained with the usual definition: Z mi = ui dµ(u). (2.22) R. With this moments we construct the vectors of moments of length kt,r + 1 to fit the vectors of coefficients γu,t,r of the polynomials pt,r like this:   m0  .  .  mt,r =  (2.23)  . . m2kt,r The trigonometric moments respect to the probability distribution µ ∈ P(I) where I = [−π, π) are obtained by the following definition: Z (2.24) mi = eiju dµ(u). I. √ where j = −1. The vectors of moments will also have length kt,r + 1 to fit the vectors of coefficients γx,t,r of the polynomials pt,r and are defined as follows:   m−kt,r   ..   .     (2.25) mt,r =  1  .   .   ..   mkt,r With the vector of moments defined in (2.23) and (2.25) is how a convex formulation of the problem respect to the control variable will be attained. The main idea is to characterize the set of probabilities P(Ωt,r ) as P(Ωt,r ) = {µ ∈ P(R)|γu,t,r · mt,r = 0}. (2.26). where mt,r is defined as (2.23) or (2.25) depending on the type of polynomial.. 2.2.2 Construction of the Convex Constraints In order to understand the equivalence of the polynomial constraints to the following convex constraints, I introduce the following results..

(17) §2.2. 13. Lemma 1. Given an algebraic polynomial f and a discrete set of real values Ω = {c1 , . . . , ck }, the integer program min. f (u). s.t.. u∈Ω. u. (2.27). is equivalent to the following polynomial program min. f (u). s.t.. p(u) = 0. u. where p(u) = program. Qk. 2 i=1 (u−ci ). (2.28). which in turn is equivalent to the following semidefinite. min. a·m. s.t.. γ·m=0 Z mi = ui dµ(u). m. (2.29). R. µ ∈ P(R) where m = (m0 , . . . , m2k )0 , p(u) = γ · F2k (u), a = (a1 , . . . , aJ )0 is the vector of coefficients of f and J is the degree of f . Moreover, every solution m∗ of the convex program (2.29) is composed of the algebraic moments of a probability measure supported in the set of solutions u∗ of (2.27). Proof. Equivalence between (2.27) and (2.28) is trivial. To settle the equivalence between (2.28) and (2.29), we must use some results of probability in optimization proposed in [8, 10, 11, 12, 18]. Thus, the polynomial program (2.28) can be relaxed into the abstract optimization problem in measures: min. hf, µi. s.t.. hp, µi = 0. µ. (2.30). µ ∈ P(R) R where hf, µi = R f (s)dµ(s) is the expected value of f respect to the probability distribution µ and P(R) is the family of probability distributions supported in R. Let G be the set of global optima of program (2.27), then whatever µ ∈ P(G) satisfies hp, µi = 0 and becomes a minimum of (2.30). On the other hand, if µ∗ is R an optimum for (2.30) and c ∈ R with µ∗ (c) > 0. As hp, µ∗ i = R p(s)dµ∗ (s) = 0.

(18) §2.2. 14. we have that c ∈ Ω and then support(µ∗ ) ⊆ Ω. If there is c0 ∈ Ω such that f (c) > f (c0 ) there would be µ∗∗ feasible in (2.30) such that hf, µ∗ i > hf, µ∗∗ i contradicting µ∗ optimality. Then c must be a global optimum of (2.27) and then support(µ∗ ) ⊆ G. In this way (2.30) is an exact relaxation of (2.27). Now (2.30) can be easily transformed into (2.29) provided that every measure µ be finitely supported, which is definitely the case given the constraint hp, µi = 0. Lemma 2. Given a trigonometric polynomial f and a discrete set of real values Ω = {c1 , . . . , ck }, the integer program min. f (u). s.t.. u∈Ω. u. (2.31). is equivalent to the following polynomial program min. f (u). s.t.. p(u) = 0. u. (2.32). √ Q where p(u) = ki=1 (eju − ejci )2 and j = −1, which in turn is equivalent to the following semidefinite program min m. s.t.. Jˆ X. ai m i. (2.33). l=J¯. γ·m=0 Z mi = eiju dµ(u) I. µ ∈ P(I) where m = (m−k , . . . , 1, . . . , mk )0 , p(u) = γ · F2k (u), aJ¯, . . . , aJˆ are the coefficients ¯ Jˆ are the lowest and highest degree of f . Moreover, every solution m∗ of f and J, of the convex program (2.33) is composed of the algebraic moments of a probability measure supported in the set of solutions u∗ of (2.31). Proof. This proof is equivalent to the proof of the Lemma 1 using trigonometric polynomials. Using Lemmas 1 and 2 depending if we are using algebraic or trigonometric polynomials respectively, two outcomes arise. The first one is the change of the control variables with the moment variables and the second one is changing the polynomials restrictions for the moments restrictions..

(19) §2.2. 15. The change of variables is made on each power of a component ut,r of the control variable that appears in the problem with a moment variable in the following way4 : for the algebraic case uit,r −→ mt,r i , ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}. (2.34). and for the trigonometric case ejiut,r −→ mt,r i , ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}.. (2.35). On the other hand, the new moments restrictions for every r = 1, . . . , n and every t = 0, N − 1 are: for the algebraic case γu,t,r · mt,r = 0 Z t,r mi = uit,r dµt,r (ut,r ). (2.36) (2.37). R. µt,r ∈ P(R). (2.38). t,r 0 where mt,r = (mt,r 0 , . . . , m2kt,r ) ; and for the trigonometric case. γu,t,r · mt,r = 0 Z t,r eijut,r dµt,r (ut,r ) mi =. (2.39) (2.40). I. µt,r ∈ P(I). (2.41). t,r 0 t,r is the where mt,r = (mt,r −kt,r , . . . , 1, . . . , mkt,r ) . Thus, with these constraints m vector of moments of a probability distribution supported in Ωt,r , i.e. a probability distribution µ ∈ P(Ωt,r ).. 2.2.3 Convex Model Using this results, by first changing the variables as illustrated in (2.34) or (2.35) and then replacing the restriction of the polynomial model defined (2.20) with (2.36,2.37,2.38) or (2.39,2.40,2.41) depending on the type of polynomials, the convex model that ends up demarcating is: Here we have mt,r as the algebraic moment defined in (2.22) or in trigonometric moment i (2.24) differentiating for each t and r. 4.

(20) §2.2. 16. for the algebraic polynomials min gN (xN ) +. Jt,r N −1 X n X X. αt,r,i (xt,r )mt,r i. (2.42). t=0 r=1 i=0. s.t.. x 0 = s0 PLt,r t,r xt+1,r = ∀r ∈ {1, . . . , n} i=0 βt,r,i (xt,r )mi ∀t ∈ {0, . . . , N − 1} t,r γu,t,r · m = 0 ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} R i t,r mi = R ut,r dµt,r (ut,r ) ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} µt,r ∈ P(R) ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}. (2.43). 0 where mt,r = (m0 , . . . , mt,r 2kt,r ) ; and for the trigonometric case. min gN (xN ) +. Jˆt,r N −1 X n X X. α̂t,r,i (xt,r )mt,r i. (2.44). t=0 r=1 i=J¯t,r. s.t.. x 0 = s0 PL̂t,r ∀r ∈ {1, . . . , n} xt+1,r = β (x )mt,r i i=L̄t,r t,r,i t,r γu,t,r · mt,r = 0 mt,r i. =. R I. eijut,r dµt,r (ut,r ). µt,r ∈ P(I). ∀t ∈ {0, . . . , N − 1} ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}. (2.45). t,r 0 where mt,r = (mt,r −kt,r , . . . , 1, . . . , mkt,r ) . As the set of probability distributions is a convex5 set and the control variables are replaced by the moment variables of this measures, we have in (2.42,2.43) as well as in (2.44,2.45) a convex formulation respect to the new control variables mt,r .. 5. The set of probability distributions is convex as it is supported on finite values by the second restriction of (2.43) or (2.45)..

(21) §2.3. 17. The procedure for obtaining a convex formulation for the state variable starting from the discrete constraints is made equivalently. For now I will continue to explain the methodology concentrated only on the control variables, but have in mind that these same steps are valid for the state variables. Later when I apply the methodology to a benchmark problem I will explain how to merge the two convex relaxations.. 2.3 Semidefinite Relaxation Using Hankel and Toeplitz Matrices The models presented in (2.42,2.43) and (2.44,2.45) are useful in the sense that they are convex respect to the control variables but they are still theoretical models based on probability measures. It is necessary to bring down this models into applicable programs which can be computationally solved. This last (third) step does this by relaxing the convex models into semidefinite programs. The semidefinite programming 6 is an area in optimization which nowadays has active research in the software for solving its problems.. 2.3.1 Using Hankel Matrices By using a classical characterization of the vector of algebraic moments mt,r of probability measures supported in R, we have characterized the vector mt,r as the moments of probability distribution supported in Ωt,r by including the linear constraint γu,t,r · mt,r = 0. In this way the following set k. t,r t,r {mt,r ∈ R2kt,r +1 : (mt,r = 0, mt,r 0 = 1} i+j )i,j=0 ≥ 0, γu,t,r · m. (2.46). represents the set of vectors mt,r that are composed of algebraic moments of probability distributions supported in Ωt,r . Here we used the Hankel matrix defined as   t,r · · · mt,r m0 mt,r 1 kt,r   t,r t,r t,r   m · · · m m 1 2 k +1 t,r kt,r t,r . (2.47) Kt,r = (mi+j )i,j=0 =  . . . ..  .  . . . . . .   t,r mt,r mt,r kt,r mkt,r +1 · · · 2kt,r 6. The semidefinite programming is based on restrictions characterized by positive semidefinite matrices..

(22) §2.3. 18. In (2.46) the characterization of algebraic moments of probability measures on the real line is made through positive semidefinite Hankel matrices (See [3, 9, 10, 18, 19]). With the sets defined in (2.46) we replace the theoretical constraints defined in (2.36), (2.37) and (2.38). In this way the following semidefinite program is defined7 Jt,r N −1 X n X X min gN (xN ) + αt,r,i (xt,r )mt,r (2.48) i t=0 r=1 i=0. s.t.. Kt,r. x 0 = s0 PLt,r t,r xt+1,r = ∀r ∈ {1, . . . , n} i=0 βt,r,i (xt,r )mi ∀t ∈ {0, . . . , N − 1} γu,t,r · mt,r = 0 ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} t,r m0 = 1 ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} t,r kt,r = (mi+j )i,j=0 ≥ 0 ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}. (2.49). The following results will justify the equivalence of the formulation given in (2.48) and (2.49) with the original dynamical program defined in (1.1) and (1.2) using the characterization of the functions ft and gt mentioned at the beginning of this chapter. Theorem 1. The problem defined in (2.48) and (2.49) is an exact relaxation ∗ of (1.1) and (1.2). Moreover, the set of solutions mt,r of (2.48) and (2.49) is composed of the moments of probability distributions µ∗t,r supported on the set of solutions u∗t,r of (1.1) and (1.2). ∗. Proof. Let mt,r be a solution of the dynamical program (2.48) and (2.49). As the Hamiltonian of this problem Ht (xt , mt,1 , . . . , mt,n , λt+1 ) is convex in the controls mt,1 , . . . , mt,n (fixing xt and λt ), we have by The Pontryagin Minimum Principle 7. For our purpose we need that 2kt,r ≥ Jt,r , Lt,r . If that is not the case, kt,r must be replaced by max{kt,r , dJt,r /2e, dLt,r /2e} which determines the size of the Hankel matrices..

(23) §2.3. 19. ([1]) that ∗. ∗. mt,1 , . . . , mt,n ∈ arg. Ht (x∗t , mt,1 , . . . , mt,n , λt+1 ). min. mt,1 ,...,mt,n. (2.50). γu,t,r · mt,r = 0. s.t. Kt,r ≥ 0 mt,r 0 = 1 where t,1. t,n. Ht (xt , m , . . . , m , λt+1 ) =. Jt,r n X X. αt,r,i (xt,r )mt,r i. (2.51). r=1 i=0. +λ0t+1. Lt,r n X X. βt,r,i (xt,r )mt,r i. r=1 i=0. and λN = OgN (x∗N ). (2.52) ∗. λt = λ0t+1 (Oxt βt (x∗t ) · mt,r ) + Oxt αt (x∗t ) · mt,r. ∗. (2.53). ∀t ∈ {0, . . . , N − 1}. From the Lemma 1 and the Hankel matrix characterization of the moments, we have that (2.50) is a semidefinite program in the control variables mt,r , so it is equivalent to the integer program: min. gt (xt , ut ) + λ0t+1 ft (xt , ut ). s.t.. ut ∈ Ωt. ut. (2.54). whose objective function is the Hamiltonian Ht (xt , ut , λt+1 ) = gt (xt , ut ) + λ0t+1 ft (xt , ut ).. (2.55). This means that mt,1 , . . . , mt,n are composed of the algebraic moments of a probability distribution supported in the global optimum set of the polynomial gt (xt , ut ) + λ0t+1 ft (xt , ut ) in the discrete sets Ωt,r . ∗ Thus, we can trace back the solution mt,r to a convex combination of admisible controls u∗t,r of the original dynamical program given in (1.1) and (1.2). In this way we conclude that the problem expressed in moments in (2.48) and (2.49) is an exact relaxation of the dynamical program in (1.1) and (1.2)..

(24) §2.3. 20. Corollary 2. If the dynamical program given in (1.1) and (1.2) has only one ∗ solution, the relaxation given in (2.48) and (2.49) also has a unique solution mt,r with the form: ∗ (2.56) mt,r = (1, u∗t,r , . . . , (u∗t,r )2kt,r )0 where u∗t,r is the unique solution of (1.1) and (1.2).. 2.3.2 Using Toeplitz Matrices Now I will characterize the vector of trigonometric moments mt,r of probability measures supported in Ωt,r in a similar way as the algebraic ones in (2.46). In this way the following set k. t,r t,r = 0, mt,r {mt,r ∈ R2kt,r +1 : (mt,r 0 = 1} i−j )i,j=0 ≥ 0, γu,t,r · m. (2.57). represents the set of vectors mt,r that are composed of trigonometric moments of probability distributions supported in Ωt,r . In this case we used the Toeplitz matrix defined as  t,r  t,r m0 mt,r · · · m −1 −kt,r  t,r  t,r t,r  m1 m0 · · · m−kt,r +1  t,r kt,r  . (2.58) Tt,r = (mi−j )i,j=0 =  . .. .. ...  . . .  .  t,r mt,r kt,r mkt,r −1 · · ·. mt,r 0. The characterization given in (2.57) of trigonometric moments of probability measures on the real line interval I = [−π, π) is through positive semidefinite Toeplitz matrices (See [3, 9, 10, 18, 19]). As with the Hankel matrices we will use (2.57) to replace the theoretical constraints defined in (2.39), (2.40) and (2.41). In this way, the following semidefinite program is defined8 min gN (xN ) +. Jˆt,r N −1 X n X X. α̂t,r,i (xt,r )mt,r i. (2.59). t=0 r=1 i=J¯t,r. For our purpose we need that kt,r ≥ Jˆt,r , L̂t,r and −kt,r ≤ J¯t,r , L̄t,r . If that is not the case, kt,r must be replaced by max{kt,r , |Jˆt,r |, |L̂t,r |, |J¯t,r |, |L̄t,r |} which determines the size of the Toeplitz matrices. 8.

(25) §2.3. s.t.. 21. x 0 = s0 PL̂t,r β (x )mt,r ∀r ∈ {1, . . . , n} xt+1,r = i i=L̄t,r t,r,i t,r γu,t,r · m. t,r. ∀t ∈ {0, . . . , N − 1} ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} (2.60) ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1} ∀r ∈ {1, . . . , n} ∀t ∈ {0, . . . , N − 1}. = 0. mt,r = 1 0 k. t,r Tt,r = (mt,r i−j )i,j=0 ≥ 0. Next I will show the results concerning the equivalence of the trigonometric formulation given in (2.59) and (2.60) with the original dynamical program defined in (1.1) and (1.2) where the characterization of the functions ft and gt are as mentioned at the beginning of this chapter. Theorem 3. The problem defined in (2.59) and (2.60) is an exact relaxation ∗ of (1.1) and (1.2). Moreover, the set of solutions mt,r of (2.59) and (2.60) is composed of the moments of probability distributions µ∗t,r supported on the set of solutions u∗t,r of (1.1) and (1.2). Proof. This is an analog proof to the one made for Theorem 1 using Toeplitz matrices. Corollary 4. If the dynamical program given in (1.1) and (1.2) has only one ∗ solution, the relaxation given in (2.59) and (2.60) also has a unique solution mt,r with the form: ∗. ∗. ∗. ∗. mt,r = (e−jkt,r ut,r , . . . , 1, ejut,r , . . . , ejkt,r ut,r )0. (2.61). where u∗t,r is the unique solution of (1.1) and (1.2).. 2.3.3 Resulting Models With the models presented in (2.48) and (2.49) for the algebraic relaxation and in (2.59) and (2.60) for the trigonometric one, the three step methodology ends by obtaining an equivalent, semidefinite relaxation of the general form of a dynamical program with discrete variables described in (1.1) and (1.2). Keep in mind that this chapter only covered the exact relaxations made for the control variables..

(26) §2.3. 22. The treatment of the discrete state variables is made exactly the same, only that the resulting model will not necessarily be equivalent to the original one. The models obtained are definitely convex respect to the control variables, but if we apply the relaxation to the state variable the model itself may not be convex. So, in case the resulting model after relaxing the state variables is convex, we can say the relaxation is exact when applied for every discrete variable in the dynamical program. On the other hand, Corollary 2 and Corollary 4 highlight the case when the original model has only one solution, in which case the optimal vector of moments provides every power (algebraic or trigonometric) of that solution. In case the problem has multiple solutions, for example u∗t,r1 , . . . , u∗t,rl different optima9 , we would have that the first optimal moment of the respective variable in the relaxed model would obtain a solution as follows ∗. mt,r = λ1 u∗t,r1 + . . . + λl u∗t,rl 1. (2.62). P where li=1 λi = 1 and λi ≥ 0 for i = 1, . . . , l. In other words, the solution in moments is a convex combination of the possible solutions of the original dynamical model. In sum, with the relaxation we would not have a solution of the original dynamical discrete program but a convex combination of the multiple solutions if that is the case10 . This happens either for the algebraic moments as well as for the trigonometric moments and it can be taken as an advantage because it can tell you when there is more than one solution in the problem. Nevertheless, knowing the possible values (if the possibilities are finite of course) of the discrete variables like Ωt,r , makes possible to find which are the values λ1 , . . . , λl that compose this convex combination by solving a system of equations. So it is possible to reverse the process so that one can see the discrete solutions involved in the dynamical program.. 9. We know they are only finite possible solutions as the variables are finitely discrete restricted. Notice that if there is no solution in the original problem there would not be any solution in the relaxed model either. 10.

(27) Chapter 3 Relaxation of the Benchmark Problems In this chapter I will apply the relaxation described previously to benchmark problems introduced at the beginning. The first one, the inventory problem, involves discrete variables only in the control, while the other one, the knight problem, involves discrete control and state variables.. 3.1 Semidefinite Relaxation of the Inventory Problem Here I will present the semidefinite relaxation of the discrete inventory problem given in (1.5) and (1.6).. 3.1.1 Polynomial Relaxation The first step of the relaxation is obtained with the positive polynomial l Y q(pt ) = (pt − ri )2 = γ · F2l (pt ). (3.1). i=1. which nulls in each possible production quantity ri and has the vector of coefficients γ. Here the polynomial defined from the discrete constraints is the same for every period t. We use algebraic polynomials because the objective function also involves algebraic polynomials and will make the change of variable of the next step achievable..

(28) §3.1. 24. Now we can define the polynomially constrained model: min aN sN +. N −1 X. (at st + bt pt + ct p2t ). (3.2). t=0. s.t.. s0 = sinit st+1 = st + pt − dt+1 ∀t ∈ {0, . . . , N − 1} . q(pt ) = 0 ∀t ∈ {0, . . . , N − 1}. (3.3). 3.1.2 Convex Model From the polynomial relaxation in (3.2) and (3.3), the convex model of the inventory problem made in the second step of the relaxation takes the form: min aN sN +. N −1 X. (at st + bt mt1 + ct mt2 ). (3.4). t=0. s.t.. s0 = st+1 = γ · mt = mti = µt ∈. sinit st + mt1 − dt+1 0 R i p dµt (pt ) R t P(R). ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N. − 1} − 1} − 1} − 1}.. (3.5). Here the change of variables made from the polynomial model (3.2) and (3.3) can be illustrated as (3.6) pit −→ mti , ∀t ∈ {0, . . . , N − 1}.. 3.1.3 Semidefinite Relaxation Now I apply the third step of the relaxation, in this case we use Hankel matrices as it is going to be applied to algebraic polynomial functions. The resulting semidefinite program is min aN sN +. N −1 X. (at st + bt mt1 + ct mt2 ). (3.7). t=0. s.t.. s0 = st+1 = γ · mt = Kt = (mti+k )li,k=0 ≥ mt0 =. sinit st + mt1 − dt+1 0 0 1. ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N ∀t ∈ {0, . . . , N. − 1} − 1} . − 1} − 1}. (3.8).

(29) §3.2. 25. This model ends in fact being convex itself and not only respect to the control variables pt . We can see this by the linear objective function (3.7) and the convex semidefinite restrictions presented in (3.8). Now we have the new convex formulation of the Inventory Problem which will be solved by semidefinite programming algorithms. Notice that for every step it has to be checked that the Hankel matrix Kt is positive semidefinite.. 3.2 Semidefinite Relaxation of the Knight Problem The knight problem described in (1.7) and (1.8), is a special case as it involves discrete control and state variables. In addition, it has a complex discrete constraint on the control characterized with the set MK defined in (1.9). In fact, a reformulation of the original model must be made before applying the relaxation because this set (MK ) does not itself satisfies the condition of being componentwise discrete separable1 as in (2.9). Following, I will present the reformulation of the original model and after that, the application of the three step relaxation proposed.. 3.2.1 Reduction of the Dimension in the Knight’s Motion The control variables ut and vt of the knight problem can be viewed as the movement of the knight. The movement of the knight on the plane (board) can be reduced to a one dimensional movement by using complex-polar coordinates in the control variables. The radius can be easily deduced as r= 1. √. 12 + 22 =. √. 5. (3.9). Another way of seeing this is by acknowledging the analogy between the set in the general form Ωt and the set in the knight problem MK . Not satisfying the componentwise discrete separable condition is as saying that it is not possible in MK to find separable Ωt,r sets for every control variable ut,r which in the knight problem are the variables ut and vt ..

(30) §3.2. 26. and the eight possible angles can be calculated as: θ1 θ2 θ3 θ4 θ5 θ6 θ7 θ8. = = = = = = = =. tan−1 ( 12 ) tan−1 (2) π − tan−1 (2) π − tan−1 ( 12 ) −π + tan−1 ( 12 ) −π + tan−1 (2) − tan−1 (2) − tan−1 ( 12 ).. (3.10). In Figure 3.1 we can see that every move can be defined in complex-polar representation as rejθi (3.11) √ for all i = 1, . . . , 8 where j = −1. In this way, the two-dimensional moves are reduced to one dimensional moves described by the eight angles listed in (3.10).. 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0M0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 8. a. b. c. d. e. f. g. h. Figure 3.1: Symmetry of the possible moves of the knight. Thus, we can express the discrete control constraint given in (1.8) ((ut , vt ) ∈ MK ) in terms of an angle φt as: √ √ 5 jφt (e + e−jφt ), ∀t ∈ {0, . . . , N − 1} ut = 5 cos φt = 2 √ √ 5 jφt vt = 5 sin φt = (e − e−jφt ), ∀t ∈ {0, . . . , N − 1} (3.12) 2.

(31) §3.2. 27. with the additional restriction of φt ∈ {θ1 , . . . , θ8 } .. (3.13). The variable φt is treated as a one-dimensional discrete control in the interval I = [−π, π) by using trigonometric polynomials and trigonometric moments. With this new variable we have the following model which will be used instead of the one defined in (1.7) and (1.8): min x2N − 2nxN + yN2 − 2nyN + 2n2 = ||(xN , yN ) − (n, n)||2 s.t.. x0 = 1 y0 = 1 √ xt+1 = xt + 25 (ejφt + e−jφt ) √ yt+1 = yt + 25 (ejφt − e−jφt ) xt ∈ {1, . . . , n} \ PX yt ∈ {1, . . . , n} \ PY φt ∈ {θ1 , . . . , θ8 }. ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1}. (3.14). (3.15). 3.2.2 Polynomial Relaxation Having this new dynamical program in (3.14) and (3.15) with the dimensional reduction in the control, I will present the polynomially constrained relaxation of the knight problem. This is obtained with the following positive polynomials nx Y p(xt ) = (xt − ai )2 = γx · F2nx (xt ), i=1 ny. q(yt ) =. Y (yt − bi )2 = γy · F2ny (yt ) and i=1 −8jφt. r(φt ) = e. 8 Y (ejφt − ejθi )2 = γφ · G−8,8 (φt ). i=1. where {a1 , . . . , anx } = {1, . . . , n} \ PX and {b1 , . . . , bny } = {1, . . . , n} \ PY . Here γx is the vector of coefficients of the algebraic polynomial p, γy is the vector of coefficients of the algebraic polynomial q and γφ is the vector of coefficients of the trigonometric polynomial r. In this case we use algebraic polynomials for the state variables xt and yt as they present algebraic polynomials in the objective function (3.14) and trigonometric polynomials for the control variables φt as the dynamic system in (3.15) contains complex trigonometric exponents..

(32) §3.2. 28. Then, the polynomially constrained relaxation for the knight problem is: min x2N − 2nxN + yN2 − 2nyN + 2n2 s.t.. x0 = 1 y0 = 1 √ xt+1 = xt + 25 (ejφt + e−jφt ) √ yt+1 = yt + 25 (ejφt − e−jφt ) p(xt ) = 0 q(yt ) = 0 r(φt ) = 0. ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1}. (3.16). (3.17). 3.2.3 Convex Model From the polynomial relaxation in (3.16) and (3.17), the convex model of the knight problem follows as:. s.t.. m01 ρ01 mt+1 1 t+1 ρ1 γx · mt γy · ρt γφ · η t mti µt ρti νt ηit υt. N N N 2 min mN 2 − 2nm1 + ρ2 − 2nρ1 + n. (3.18). = = = = = = = = ∈ = ∈ = ∈. (3.19). 1 1 √ √ t + 25 η1t mt1 + 25 η−1 √ √ t − 25 η1t ρt1 + 25 η−1 0 0 0 R i x dµt (xt ) R t P(R) R i y dνt (yt ) R t P(R) R jiφ e t dυt (φt ) R P(I). ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1}. √ where j = −1 and I = [−π, π). The change of variables made in this (second) step of the relaxation can be illustrated as follows: xit −→ mti , ∀t ∈ {0, . . . , N }. (3.20). yti −→ ρti , ∀t ∈ {0, . . . , N }. (3.21). ejiφt −→ ηit , ∀t ∈ {0, . . . , N − 1}.. (3.22).

(33) §3.2. 29. 3.2.4 Semidefinite Relaxation Finally, I apply the third step of the relaxation to the model (3.18) and (3.19).In this case we use Hankel matrices for the algebraic moments mt and ρt , and Toeplitz matrices for the trigonometric moments η t . Therefore, the resulting semidefinite program is N N N 2 min mN 2 − 2nm1 + ρ2 − 2nρ1 + n. s.t.. m01 ρ01 mt+1 1 t+1 ρ1 γx · mt γy · ρt γφ · η t Ht mt0 Kt ρt0 Tt η0t. = = = = = = = = = = = = =. 1 1 √ √ t mt1 + 25 η−1 + 25 η1t √ √ t − 25 η1t ρt1 + 25 η−1 0 0 0 x (mti+k )ni,k=0 ≥0 1 ny (ρti+k )i,k=0 ≥0 1 t (ηi−k )8i,k=0 ≥ 0 1. ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N } ∀t ∈ {0, . . . , N − 1} ∀t ∈ {0, . . . , N − 1}.. (3.23). (3.24). This model also ends being convex itself and not only respect to the control variables φt . This can be seen by the linear objective function (3.23) and the convex semidefinite restrictions presented in (3.24). A convex model is needed this time as we applied the relaxation not only to the control variables but to the state variables (xt and yt ) as well. We have now the new convex formulation of the Knight Problem which will be solved by semidefinite programming algorithms. Notice that for every step it has to be checked that three matrices (Ht , Kt and Tt ) are positive semidefinite, where Ht and Kt are Hankel matrices and Tt are Toeplitz matrices..

(34) Chapter 4 Instances of the Benchmark Problems The final goal is to solve arbitrary problems formulated as in (3.7)and (3.8) for the Inventory problem, and (3.23)and (3.24) for the Knight problem. For solving these problems, it is important to have in mind the facts mentioned in Section 2.3.3 and illustrated in (2.62) about the solutions obtained in moments so that they are computed into relevant solutions of the original discrete problems. The positive semidefinite programs are solved with packages implemented straightforward for this type of problems. Currently, there is active research on the theory and algorithms for semidefinite programming. Some important concepts and tools have been developed around it. For instance, consider the tremendous development on interior point methods and the introduction of self-concordant barrier functions. It is not my purpose to explain the essentials of semidefinite programming here; for an introductory review on this subject, see [2]. What one has to have in mind is that these algorithms have polynomial time complexity which is the main motivation of this approach as we treat discrete constrained problems. When you have a non-convex problem in optimization, the time complexity is exponential in general. These are problems of global optimization. In the compilation presented by Neumaier [26] the best algorithm known today for solving these problems directly is based in Monte-Carlo algorithm which is probabilistic. In addition, the best deterministic algorithm is based on ’Branch and Bound’. Therefore, if we realize that in the worst case Branch and Bound takes exponential time complexity, we can say that there is no deterministic algorithm for global optimization problems that takes polinomial time complexity..

(35) §4.1. 31. The semidefinite programming solver I used is a software package called SeDuMi1 from the Advanced Optimization Lab of McMaster University. This is a component for solving semidefinite programs in MATLABr . In this case, before calling the semidefinite algorithms in this package and obtaining the solution of these problems, it is necessary to develop in MATLABr the algorithms for obtaining the correct form of the formulation, which includes the Hankel and/or the Toeplitz matrices characterizations. Bellow I will show the results obtained for some instances of the inventory and the knight problem. These instances can have unique or multiple solutions in their original formulation.. 4.1 Solving the Inventory Problem Here two situations of the Inventory Problem as modeled in (3.7) and (3.8) are solved.. 4.1.1 Situation with 12 periods The first instance of the inventory problem has 12 periods (N −1 = 12), 4 possible production quantities (l = 4) and the following values: at bt ct {r1 , . . . , rl }. = = = =. 1 ∀t ∈ {0, . . . , N } 1 ∀t ∈ {0, . . . , N − 1} 1 ∀t ∈ {0, . . . , N − 1} {0, 1, 2, 3}.. (4.1). In other words this instance would have an objective function min sN +. N −1 X. (st + pt + p2t ). (4.2). t=0. if we use the notation used in the general form of the discrete inventory problem presented in (1.5) and (1.6). The solution obtained here is illustrated in Figure 4.1 and the specific values for each period are shown in Table A.1. 1. See http://sedumi.mcmaster.ca for more information..

(36) §4.2. 32. Figure 4.1: The solution of a 12 period instance of the inventory problem.. 4.1.2 Situation with 24 periods The second instance of the inventory problem has 24 periods (N − 1 = 24), 6 possible production quantities (l = 6) and the following values: at bt ct {r1 , . . . , r4 }. = = = =. 1 ∀t ∈ {0, . . . , N } 1 ∀t ∈ {0, . . . , N − 1} 1 ∀t ∈ {0, . . . , N − 1} {0, 1, 2, 3, 4, 5}.. (4.3). So, the objective function in this case would be the same as the previous instance given in (4.2). The solution obtained here is illustrated in Figure 4.2 and the specific values for each period are shown in Table A.2.. 4.2 Solving the Knight Problem Finally I present two other examples that solve the relaxation model of the knight problem presented in (3.23) and (3.24). The first example is a situation with multiple solutions and the second one with an unique solution.. 4.2.1 A Case with Multiple Solutions A remarkable feature of the method proposed here is that it allows us to find multiple solutions in discrete dynamical programs. The first instance for the.

(37) §4.2. 33. Figure 4.2: The solution of a 24 period instance of the inventory problem. knight problem has six possible steps (N = 6) and the board size is 8 × 8 (n = 8). There are two rooks as illustrated in Figure 4.3. The results of the two solutions. 0Z0Z0Z0M 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0Z0ZrZ 3 Z0Z0Z0Z0 2 0Z0Z0ZrZ 1 Z0Z0Z0Z0 8. a. b. c. d. e. f. g. h. Figure 4.3: The knight problem has multiple possible solutions when n = 8, N = 6 and the two rooks shown above. illustrated are shown in Tables A.3 and A.4. In this case the first moment variables mt1 and ρt1 are the ones forced in the first steps so that we can separate each solution. The solution obtained without forcing the variables gave similar results in the objective function (3.23) (with 0.1086 and 0.2813). It is expected that the.

(38) §4.2. 34. results are made from a convex combination of optimal solutions but this does not explain this outcome. Three possible problems that could influence in some way these results may explain this. 1. In this model there is a strong complexity associated due to the interdependency between the moment variables in restrictions like the third and fourth in (3.24). 2. The formulation of the trigonometric polynomial can not be exact. This is because the nature of the angles treated (3.10) are not computationally precise. So, in the strong interdependency mentioned, we can see that these small numerical errors in the formulation of the trigonometric moment η t will alter the fulfillment of the restrictions involved, which will lead to a direct impact on the algebraic moments mt and ρt . 3. The not so exact nor robust packages involved in semidefinite programming are still on development. There are many parameters to have in count so that a desired behavior is obtained. Therefore, we cannot easily obtain a solution by the use of the convex solution found because it is not clear and neither exact the possible options for each step of η1t influencing the other first moments mt1 and ρt1 . These three statements will also make difficult the work on the unique solution models.. 4.2.2 A Case with an Unique Solution The other instance of the knight problem is also with six time steps (N = 6) and the board size 8 × 8 (n = 8). In this case there are three rooks as illustrated in Figure 4.4. The results are shown in Table A.5. Notice that we obtain only one solution. This time the solution obtained is definitely more exact than the ones obtained on the multiple solution instance with an objective function near zero (actually 0.0044)..

(39) §4.2. 35. 0Z0Z0Z0M 7 Z0Z0Z0Z0 6 0Z0Z0ZrZ 5 Z0Z0Z0Z0 4 0Z0ZrZ0Z 3 Z0Z0Z0Z0 2 0ZrZ0Z0Z 1 Z0Z0Z0Z0 8. a. b. c. d. e. f. g. h. Figure 4.4: The knight problem has an unique solution when n = 8, N = 6 and the three rooks shown above..

(40) Chapter 5 Conclusions This work involves two very interesting applications of the method of moments to dynamical programs with discrete variables. The theory involved in the relaxation of the general definition given in (1.1) and (1.2) opens many possibilities to solve finite discrete dynamical programs in Operations Research. There are many problems that fit into the general definition including inventory problems, optimal routing problems, portfolio selection problems and different permutation problems. It is remarkable that the Knight Problem involves both algebraic and trigonometric moments. This proposal is restricted to state of the art solvers for semidefinite programming. Although in theory, the models suggested ((2.48) and (2.49) or (2.59) and (2.60)) seem to work, numerically the models need exact formulation so that the constraints involving dependency between the moments fit. This can be solved with various manipulations of the implementation and a good use of the parameters of the algorithms of semidefinite programming. I expect that in the future there will be more robust algorithms which generate more reliable solutions so that more problems of semidefinite programming have practical and usable results. The results presented here were obtained using SeDuMi package from the Advanced Optimization Lab of McMaster University. Here we treated variables which are separable into one dimensional components including their discrete constraints. When the problem involves variables in several dimensions we must use a different approach by following an iterative scheme. In this case we reduced the two dimensional control variables in the Knight Problem to one dimensional variables, but this is not always possible to accomplish..

(41) §5.0. 37. With this work, the reader has only a glimpse of the complexity involved in global optimization. While we stand in a moment where there is no proof of efficient solutions to every hard problem NP (Non-deterministic Polynomial time), this type of approaches that engage into special cases like the dynamical programs with discrete constrains treated here is a good option to follow. This way each specific need is worked so that an alternate usable solution is obtained.. Acknowledgments This work was supported by the following research grants: 1. Grupo de Investigación Métodos Numéricos-Optimización - Laboratorio de Matemáticas Aplicadas - Universidad de los Andes. 2. Proyecto conjunto Aula Cimne-Uniandes, 2003-2008. 3. Matching Grant LMGP-0311-1110522121, Sun Microsystems, 2005. 4. Departamento de Ingeniería Industrial, Universidad de los Andes. 5. Proyecto Semilla de René Meziat, Facultad de Ciencias, 2007..

(42) Bibliography [1] D.P. Bertsekas. Dynamic Programming and Optimal Control, volume 1. Athena Scientific, 1995. [2] S. Boyd and L. Vanderberghe. Semidefinite programing. SIAM Review, 1995. [3] R. Curto and L.A. Fialkow. Recursiveness, positivity, and truncated moment problems. Houston Journal of Mathematics, 17(4):603, 1991. [4] J. Egozcue, R. Meziat, and P. Pedregal. From a non-linear non-convex formulation to a linear convex formulation of non-convex variational problems. Journal of Applied Mathematics and Optimization, 47:27–44, 2003. [5] I.A. Fotiou, P. Rostalski, P.A. Parrilo, and M. Morari. Parametric optimization and optimal control using algebraic geometry methods. Int. J. Control, 79(11), 2006. [6] K. Gatermann and P.A. Parrilo. Symmetry groups, semidefinite programs, and sums of squares. Journal of Pure and Appl. Algebra, 192(1-3):95–128, 2004. [7] D. Henrion and A. Garulli, editors. Positive Polynomials in Control, volume 312 of Lecture Notes in Control and Information Sciences. Springer, New York, 2005. [8] D. Henrion and J.B. Lasserre. GloptiPoly: Global optimization over polynomials with Matlab and SeDuMi. ACM Transactions on Mathematical Software, 29(2):165–194, June 2003. [9] I. Iohvidov. Hankel and Toeplitz Matrices and Forms. Springer, Sep. 1982. [10] J.B. Lasserre. Global optimization with polynomials and the problem of moments. SIAM J. Optim., 11(3):796–817, 2001..

(43) §5.0. 39. [11] J.B. Lasserre. Polynomials nonnegative on a grid and discrete optimization. Trans. Amer. Math. Soc., 354:631–649, 2002. [12] J.B. Lasserre. Semidefinite programming vs. LP relaxations for polynomial programming. J. Math. Oper. Res., 27(2):347–360, 2002. [13] J.B. Lasserre. SOS approximations of polynomials nonnegative on a real algebraic set. SIAM J. Optim., 16:610–208, 2005. [14] J.B. Lasserre. A sum of squares approximation of nonnegative polynomials. SIAM J. Optim., 16:751–765, 2006. [15] J.B. Lasserre. A semidefinite programming approach to the generalized problem of moments. Math. Programm., 112(1):65–92, 2008. [16] J.B. Lasserre and T. Prieto-Rumeau. SDP vs. LP relaxations for the moment approach in some performance evaluation problems. Stoch. Models, 20:439– 456, 2004. [17] P.A. Parrilo M.C. Mazzaro and R.S. Sánchez Peña. Robust identification toolbox. Latin American Applied Research, 34(2):91–100, 2004. [18] R. Meziat. Analysis of non convex polynomial programs by the method of moments. In C.A. Floudas and P.M. Pardalos, editors, Frontiers in Global Optimization, volume 74 of Nonconvex Optimization and its Applications, pages 353–372. Kluwer, 2003. [19] R. Meziat. The method of moments in global optimization. Journal of Mathematical Sciences, 116(3):3303–3324, July 2003. [20] R. Meziat. Two dimensional nonconvex variational problems. In Q. Liqun, T. Koklay, and Y. Xiaoqi, editors, Optimization and Control with Applications, volume 96 of Applied Optimization, pages 393–405. Springer, 2005. [21] R. Meziat, J. Egozcue, and P. Pedregal. The method of moments for nonconvex variational problems. In P. Pardalos and N. Hadjisavvas, editors, Advances in Convex Analysis and Global Optimization, volume 54 of Nonconvex Optimization and its Applications Series, pages 371–382. Kluwer, 2001. [22] R. Meziat, D. Patiño, and P. Pedregal. An alternative approach for non linear optimal control problems based on the method of moments. Journal of Computational Optimization, 38(1):147–171, 2007..

(44) §5.0. 40. [23] R. Meziat and C. Rivera. Analysis of one dimensional non convex variational problems with restrictions on the derivative. Preprint. [24] R. Meziat and T. Roubicek. Coarse-convex-compactification approach to numerical solution of nonconvex variational problems. Preprint. [25] R. Meziat and J. Villalobos. Analysis of microstructures and phase transitions in one dimensional elasticity by convex optimization. Journal of Structural and Multidisciplinary Optimization, 32(6):507–519, 2006. [26] A. Neumaier. Complete search in continuous global optimization and constraint satisfaction. In A. Iserles, editor, Acta Numerica 2004. Cambridge University Press, Universität Wien Strudlhofgasse 4, A-1090 Wien, Austria, November 2003. [27] P.A. Parrilo. Semidefinite programming relaxations for semialgebraic problems. Math. Program. Ser. B, 96(2):293–320, May 2003. [28] P.A. Parrilo and S. Lall. Semidefinite programming relaxations and algebraic optimization in control. Eur. J. Control, 9(2-3):307–321, 2003. [29] P.A. Parrilo and B. Sturmfels. Minimizing polynomial functions. In Algorithmic and Quantitative Real Algebraic Geometry, volume 60 of DIMACS Series in Discrete Mathematics and Theoretical Computer Science, pages 83– 99. AMS, Providence, 2001. [30] S. Prajna, A. Papachristodoulou, P. Seiler, and P.A. Parrilo. SOSTOOLS and its control applications. In Positive Polynomials in Control, volume 312 of Lecture Notes in Control and Information Sciences, pages 273–292. Springer, Berlin, 2005. [31] P.A Parrilo S. Prajna and A. Rantzer. Nonlinear control synthesis by convex optimization. IEEE Transactions on Automatic Control, 49(2), 2004..

(45) Appendix A Numerical Results A.1 Inventory Problem Step. Demand. Production. Stock. t. dt. mt∗ 1. s∗t. 1.0028 1.9973 1.9969 2.0024 2.9987 3.0004 3.0007 3.0008 2.0001 1.9976 2.0031 2.9992. 0.0000 1.0028 2.0001 1.9970 2.9994 4.9981 6.9985 6.9992 4.0000 2.0001 1.9977 3.0008 4.0000 0.0000. 0 1 2 3 4 5 6 7 8 9 10 11 12 13. 1 2 1 1 1 3 6 4 2 1 2 4. Table A.1: The solution obtained for each period of the inventory problem when l = 4 and N − 1 = 12 as illustrated in Figure 4.1..

(46) §A.2. 42. Step. Demand. Production. Stock. t. dt. mt∗ 1. s∗t. 4.013 4.0131 3.9968 3.9968 3.9967 3.9967 3.0522 3.0265 3.0034 2.9939 2.9942 2.9987 3.0035 2.9997 2.9993 3.003 3.0086 3.015 3.0214 2.9964 2.1964 2.0707 2.0179 1.9965. 0 4.013 6.0261 4.0229 7.0197 7.0165 8.0132 3.0654 5.0919 5.0953 6.0892 8.0834 10.0821 10.0855 7.0852 8.0845 10.0875 11.0961 11.1111 12.1325 5.1289 2.3253 3.396 4.4139 5.4104 0.4104. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25. 2 6 1 4 3 8 1 3 2 1 1 3 6 2 1 2 3 2 10 5 1 1 1 5. Table A.2: The solution obtained for each period of the inventory problem for l = 6 and N − 1 = 24 as illustrated in Figure 4.2..

(47) §A.2. 43. A.2 Knight Problem Step. x-coordinate. y-coordinate. t. mt∗ 1. ρt∗ 1. 1 2 3 4 5 6. 2.0006 2.9999 3.9992 4.9965 6.0032 7.9913. 2.9996 4.9999 7.0001 4.9997 6.9957 7.9608. Table A.3: One of the solutions obtained for each step of the knight problem illustrated in Figure 4.3. There are multiple possible solutions when n = 8 and N = 6.. Step. x-coordinate. y-coordinate. t. mt∗ 1. ρt∗ 1. 1 2 3 4 5 6. 1.9960 1.0013 2.0007 3.9990 5.9971 7.9953. 3.0012 5.0024 7.0014 7.9919 6.9935 7.9251. Table A.4: Another solution obtained for the knight problem illustrated in Figure 4.3. There are multiple possible solutions when n = 8 and N = 6..

(48) §A.2. 44. Step. x-coordinate. y-coordinate. t. mt∗ 1. ρt∗ 1. 1 2 3 4 5 6. 2.0001 1.0000 2.0000 3.9996 5.9990 7.9900. 2.9997 4.9995 6.9993 7.9998 6.9989 7.9919. Table A.5: This is the solution obtained for the knight problem when n = 8 and N = 6 and the play illustrated in Figure 4.4. Here there is only one solution..

(49)