Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces / Antonio Falcó and Anthony Nouy

Texto completo

(1)Noname manuscript No. (will be inserted by the editor). Proper Generalized Decomposition for Nonlinear Convex Problems in Tensor Banach Spaces Antonio Falcó · Anthony Nouy. the date of receipt and acceptance should be inserted later. Abstract. Tensor-based methods are receiving a growing interest in scientic comput-. ing for the numerical solution of problems dened in high dimensional tensor product spaces. A family of methods called Proper Generalized Decompositions methods have been recently introduced for the a priori construction of tensor approximations of the solution of such problems. In this paper, we give a mathematical analysis of a family of progressive and updated Proper Generalized Decompositions for a particular class of problems associated with the minimization of a convex functional over a reexive tensor Banach space.. Mathematics Subject Classication (2000) Keywords. 65K10,49M29. Nonlinear Convex Problems, Tensor Banach spaces, Proper Generalized. Decomposition, High-Dimensional Problems, Greedy Algorithms. 1 Introduction Tensor-based methods are receiving a growing interest in scientic computing for the numerical solution of problems dened in high dimensional tensor product spaces, such as partial dierential equations arising from stochastic calculus (e.g. Fokker-Planck. This work is supported by the ANR (French National Research Agency, grants ANR-2010COSI-006 and ANR-2010-BLAN-0904) and by the PRCEU-UCH30/10 grant of the Universidad CEU Cardenal Herrera. A. Falcó Departamento de Ciencias, Físicas, Matemáticas y de la Computación, Universidad CEU Cardenal Herrera, San Bartolomé 55 46115 Alfara del Patriarca (Valencia), Spain. E-mail: [email protected] A. Nouy LUNAM Université, GeM, UMR CNRS 6183, Ecole Centrale Nantes, Université de Nantes 1 rue de la Noë, BP 92101, 44321 Nantes Cedex 3, France. E-mail: [email protected].

(2) 2. Antonio Falcó, Anthony Nouy. equations) or quantum mechanics (e.g. Schrödinger equation), stochastic parametric partial dierential equations in uncertainty quantication with functional approaches, and many mechanical or physical models involving extra parameters (for parametric analyses) among others. For such problems, classical approximation methods based on the a priori selection of approximation bases suer from the so called curse of dimensionality associated with the exponential (or factorial) increase in the dimension of approximation spaces. Tensor-based methods consist in approximating the solution. u ∈ V of a problem, where V is a tensor space generated by d vector spaces Vj n 1 e.g. Vj = R j ) , using separated representations of the form. u ≈ um =. m X. (1). wi. (d ). ⊗ . . . ⊗ wi ,. (j ). wi. (assume. ∈ Vj. (1). i=1. where. (j ). ⊗ represents the Kronecker product. The functions wi. are not a priori selected. but are chosen in an optimal way regarding some properties of. u.. A rst family of numerical methods based on classical constructions of tensor approximations [18, 23, 35] have been recently investigated for the solution of highdimensional partial dierential equations [3, 19, 21, 22, 28]. They are based on the systematic use of tensor approximations inside classical iterative solvers. Another family of methods, called Proper Generalized Decomposition (PGD) methods [9, 16, 27, 33, 34], have been introduced for the direct construction of representations of type (1). PGD methods introduce alternative denitions of tensor approximations, not based on natural best approximation problems, for the approximation to be computable without a priori information on the solution. u.. The particular structure of approximation sets. allows the interpretation of PGDs as generalizations of Proper Orthogonal Decomposition (or Singular Value Decomposition, or Karhunen-Loève Decomposition) for the a priori construction of a separated representation. um. of the solution. They can also. be interpreted as a priori model reduction techniques in the sense that they provide a way for the a priori construction of optimal reduced bases for the representation of the solution. Several denitions of PGDs have been proposed. Basic PGDs are based on a progressive construction of the sequence elementary tensor. (k ). ⊗dk=1 wm. um ,. where at each step, an additional. is added to the previously computed decomposition. um−1. [2, 26, 30]. Progressive denitions of PGDs can thus be considered as Greedy algorithms [37] for constructing separated representations [1, 6]. A possible improvement of these progressive decompositions consists in introducing some updating steps in order to capture an approximation of the optimal decomposition, which would be obtained by dening the whole set of functions simultaneously (and not progressively). For many applications, these updating strategies allow recovering good convergence properties of separated representations [31, 33, 34].. In [6], convergence results are given for the progressive Proper Generalized Decomposition in the case of the high-dimensional Laplacian problem. In [16], convergence is proved in the more general setting of linear elliptic variational problems in tensor Hilbert spaces. The progressive PGD is interpreted as a generalized singular value decomposition with respect to the metric induced by the operator, which is not neces-. 1 More precisely, V is the closure with respect to a norm k · k of the algebraic tensor space nN o N. V=. a. d j=1. Vj = span. d j=1. v (j) : v (j) ∈ Vj and 1 ≤ j ≤ d.

(3) PGD for Nonlinear Convex Problems. 3. sarily a crossnorm on the tensor product space.. In this paper, we propose a theoretical analysis of progressive and updated Proper Generalized Decompositions for a class of problems associated with the minimization. J,. of an elliptic and dierentiable functional. J (u) = min J (v), v∈V. where. V. is a reexive tensor Banach space. In this context, progressive PGDs consist. in dening a sequence of approximations. um ∈ V. um = um−1 + zm , where. S1. dened by. zm ∈ S1. is a tensor subset with suitable properties (e.g. rank-one tensors subset,. Tucker tensors subset, ...), and where. zm. is an optimal correction in. S1. of. um−1 ,. dened by. J (um−1 + zm ) = min J (um−1 + z) z∈S1. Updated progressive PGDs consist in correcting successive approximations by using the information generated in the previous steps. At step optimal correction. zm ∈ S 1. of. um−1 ,. m,. after having computed an. a linear (or ane) subspace. Um ⊂ V. such that. um−1 + zm ∈ Um. is generated from the previously computed information, and the. next approximation. um. is dened by. J (um ) = min J (v) ≤ J (um−1 + zm ) v∈Um. The outline of the paper is as follows. In section 2, we briey recall some classical properties of tensor Banach spaces. In particular, we introduce some assumptions on the weak topology of the tensor Banach space in order for the (updated) progressive PGDs to be well dened (properties of subsets. S1 ). In section 3, we introduce a class of. convex minimization problems on Banach spaces in an abstract setting. In section 4, we introduce and analyze the progressive PGD (with or without updates) and we provide some general convergence results. While working on this paper, the authors became aware of the work [7], which provides a convergence proof for the purely progressive PGD when working on tensor Hilbert spaces. The present paper can be seen as an extension of the results of [7] to the more general framework of tensor Banach spaces and to a larger family of PGDs, including updating strategies and a general selection. S1 . In section 5, we present some classical examples of applications of Lp tensor spaces (generalizing the multidip mensional singular value decomposition to L spaces), solution of p-Laplacian problem, of tensor subsets. the present results: best approximation in. and solution of elliptic variational problems (involving inequalities or equalities).. 2 Tensor Banach spaces Banach spaces. Nd. j =1 Vj generated from Vj (1 ≤ j ≤ d) equipped with norms k·kj . As underlying eld we choose. We rst consider the denition of the algebraic tensor space a.

(4) 4. Antonio Falcó, Anthony Nouy. R,. but the results hold also for. C.. Nd. j =1 Vj refers to the `algebraic'. The sux `a' in a. nature. By denition, all elements of. V :=. a. d O. Vj. j =1 are. nite. v=. linear combinations of elementary tensors. Nd. (j ) j =1 v. . . v (j ) ∈ Vj .. A typical representation format is the Tucker or tensor subspace format. u=. X. ai. I = I1 × . . . × Id. (j ). bij ,. (2). j =1. i∈I where. d O. is a multi-index set with. Ij = {1, . . . , rj }, rj ≤ dim(Vj ),. (j ). bij ∈ Vj (ij ∈ Ij ) are linearly independent (usually orthonormal) vectors, and ai ∈ R. Here, ij are the components of i = (i1 , . . . , id ). The data size is determined by the numbers rj collected in the tuple r := (r1 , . . . , rd ). The set of all tensors representable by (2) with xed r is. Tr (V) :=. v∈V:. Uj ⊂ Vj such that Nd v ∈ U := a j =1 Uj .. there are subspaces. dim(Uj ) = rj. and. (3). To simplify the notations, the set of rank-one tensors (elementary tensors) will be denoted by. o. n. R1 (V) := T(1,...,1) (V) = ⊗dk=1 w(k) : w(k) ∈ Vk . By denition, we then have. V = span R1 (V).. We also introduce the set of rank-m. tensors dened by. Rm (V) :=. (m X. ) zi : zi ∈ R1 (V). .. i=1 We say that. V. and a norm. norm. k·k,. Vk·k is a Banach tensor space if there exists an algebraic tensor space k·k on V such that Vk·k is the completion of V with respect to the. i.e.. Vk·k :=. d O k·k. Vj =. a. j =1. j =1 If. Vk·k. is a Hilbert space, we say that. Vk·k. is a. k·k. Od. Vj. .. Hilbert tensor space.. 2.1 Topological properties of Tensor Banach spaces. span R1 (V) is dense in Vk·k . Since R1 (V) ⊂ Tr (V) for all r ≥ (1, 1, . . . , 1), span Tr (V) is also dense in Vk·k . Nd. Observe that then. Any norm. k·k. on a. Od j =1 is called a. j =1 Vj satisfying. v (j ) =. crossnorm.. Yd j =1. kv (j ) kj. for all. v (j ) ∈ Vj (1 ≤ j ≤ d). (4).

(5) PGD for Nonlinear Convex Problems. Remark 1. 5. k. Eq. (4) implies the inequality. Nd. j =1 v. (j ). k.. Qd. j =1 kv. (j ). kj. which is equiv-. alent to the continuity of the tensor product mapping. d. O. :. ×.  . Vj , k·kj. . −→  a. j =1. given by. . ⊗ (v (1) , . . . , v (d) ). equipped with norm. . k·k. j =1. k·k. is called a. Denition 1 v∈V=. a. Vj , k·k ,. (5). = ⊗dj=1 v (j ) ,. where. (X, k · k). denotes a vector space. X. k · k.. is a crossnorm on a. Od. . j =1. k·k is denoted ∗ j =1 Vj , i.e.. As usual, the dual norm to. ∗. d O. ∗. ϕ(j ). by. k·k∗ .. If. k·k. is a crossnorm and also. Nd =. Yd j =1. kϕ(j ) k∗j. for all. ϕ(j ) ∈ Vj∗ (1 ≤ j ≤ d) ,. (6). reasonable crossnorm. Now, we introduce the following norm. Let. Nd. Vj. j =1 Vj ,. be Banach spaces with norms we dene the norm. k·k∨. k·kj. for. 1 ≤ j ≤ d.. Then for. by.     ϕ(1) ⊗ ϕ(2) ⊗ . . . ⊗ ϕ(d) (v) : 0 6= ϕ(j ) ∈ Vj∗ , 1 ≤ j ≤ d . kvk∨ := sup Qd   kϕ(j ) k∗ j =1. (7). j. vm ∈ V is weakly convergent if limm→∞ hϕ, vm i exists ϕ ∈ V ∗ . We say that (vm )m∈N converges weakly to v ∈ V if limm→∞ hϕ, vm i = hϕ, vi for all ϕ ∈ V ∗ . In this case, we write vm * v . We recall that a sequence. for all. Denition 2. A subset. M ⊂V. is called. weakly closed. if. vm ∈ M. and. vm * v. implies. v ∈ M. Note that `weakly closed' is stronger than `closed', i.e.,. M. weakly closed. ⇒ M. closed. The following proposition has been proved in [15].. Proposition 1 Let Vk·k be a Banach tensor space with a norm satisfying k · k & k·k∨ on V. Then the set Tr (V) is weakly closed. 2.2 Examples. 2.2.1 The Bochner spaces Our rst example, the Bochner spaces, are a generalization of the concept of. Lp -spaces. to functions whose values lie in a Banach space which is not necessarily the space. C. k · kX .. I ⊂ Rs. and µ a nite I (e.g. a probability measure). Let us consider the Bochner space Lpµ (I ; X ), 1 ≤ p < ∞, dened by. Let. X. be a Banach space endowed with a norm. Let. measure on with. R or. Lpµ (I ; X ) =. . Z v:I→X: I. . kv (x)kpX dµ(x) < ∞ ,.

(6) 6. Antonio Falcó, Anthony Nouy. and endowed with the norm. Z kvk∆p = I. kv (x)kpX dµ(x). We now introduce the tensor product space the space. Lpµ (I ; X ). can be identied with. 1/p. Vk·k∆p = X ⊗k·k∆p Lpµ (I ). For 1 ≤ p < ∞, Vk·k∆p. (see Section 7, Chapter 1 in [11]).. Moreover, the following proposition can be proved (see Proposition 7.1 in [11]):. Proposition 2 For 1 ≤ p < ∞, the norm k·k∆p satises k·k∆p & k·k∨ on X⊗a Lpµ (I ). By Propositions 2 and 1, we then conclude:. Corollary 1 For 1 ≤ p < ∞, the set Tr (X ⊗a Lpµ (I )) is weakly closed in Lpµ (I ; X ). In particular, for K ⊂ Rk , we have that Tr (Lpν (K ) ⊗a Lpµ (I )) and R1 (Lpν (K ) ⊗a Lpµ (I )) are weakly closed sets in Lpν⊗µ (K × I ) . 2.2.2 The Sobolev spaces ΩP = Ω1 × . . . × Ωd ⊂ Rd , with Ωk ⊂ R. Let α ∈ Nd denote a d |α| = k=1 αk . Dα (u) = ∂xα11 . . . ∂xαdd (u) denotes a partial derivative of order |α|. For a xed 1 ≤ p < ∞, we introduce the Sobolev space. Let. multi-index and of. u(x1 , . . . , xd ). H m,p (Ω ) = {u ∈ Lp (Ω ) : Dα (u) ∈ Lp (Ω ), 0 ≤ |α| ≤ m} equipped with the norm. X. kvkm,p =. kDα (v)kLp (Ω ). 0≤|α|≤m We let. Vk = H m,p (Ωk ),. endowed with norms. kwkm,p;k =. m X. k · km,p;k. dened by. k∂xj k (w)kLp (Ωk ) .. j =0 Then we have the following equality. H m,p (Ω ) =. d O k·km,p. H m,p (Ωj ) .. j =1 A rst result is the following.. Proposition 3 For 1 < p < ∞, m ≥ 0 and Ω = Ω1 × . . . × Ωd , the set  R1  a. d O.  n. o. H m,p (Ωj )  = ⊗dk=1 w(k) : w(k) ∈ H m,p (Ωk ) ,. j =1. is weakly closed in (H m,p (Ω ), k · km,p ). To prove the above proposition we need the following two lemmas..

(7) PGD for Nonlinear Convex Problems. 7. Lemma 1 Assume 1 < p < ∞ and Ω = Ω1 ×. . .×Ωd . Then the set R1 is weakly closed in Lp (Ω ). Proof. Let. (j ). {vn }n∈N , with vn = ⊗dj=1 vn , be a sequence in R1. weakly converges to an element. Lp (Ω ),. and also the sequences. N d. p j =1 L (Ωj ). a. N d. p j =1 L (Ωj ). a. . . that. p. v ∈ L (Ω ). Then the sequence {vn }n∈N is bounded in (j ) {vn }n∈N ⊂ Lp (Ωj ) for each j ∈ {1, 2, . . . , d}. Then, (j ). j ∈ {1, 2, . . . , d}, we can extract a subsequence, namely {vnk }k∈N , that weakly (j ) converges to some v ∈ Lp (Ωj ). Weak convergence in Lp (Ωj ) implies the convergence. for each. in distributional sense, that is, the subsequence. (j ). {vnk }k∈N. converges to. D (Ω ).. D0 (Ωj ). d ⊗j =1 v (j ) in. v (j ). (j ) d From Proposition 6.2.3 of [4], we have that {⊗j =1 vnk }k∈N converges to 0. in. u t. By uniqueness of the limit, we obtain the desired result.. Lemma 2 Assume 1 < p < ∞, m ≥ 1 and Ω = Ω1 × . . . × Ωd . For any measurable functions wk : Ωk → R such that ⊗dk=1 wk 6= 0, we have ⊗dk=1 wk ∈ H m,p (Ω ) if and only if wk ∈ H m,p (Ωk ) for all k ∈ {1 . . . d}. Proof. Suppose that. wk ∈ H m,p (Ωk ). d Y. X. k ⊗dk=1 wk km,p =. for all. k ∈ {1 . . . d}.. Since. k∂xαkk (wk )kLp (Ωk ). 0≤|α|≤m k=1. X. d Y. α∈{0,...,m}d. k=1. ≤ d Y. =. . m X.  k=1. k∂xαkk (wk )kLp (Ωk ).  k∂xj k (wk )kLp (Ωk ) . =. j =0. ⊗dk=1 wk ∈ H m,p (Ω ). d m,p Conversely, if ⊗k=1 wk ∈ H (Ω ),. d Y. kwk km,p;k ,. k=1. we have. then. X. k ⊗dk=1 wk km,p =. kDα (⊗dk=1 wk )kLp (Ω ) < ∞. 0≤|α|≤m which implies that. α = (0, . . . , 0),. kDα (⊗dk=1 wk )kLp (Ω ) < ∞. for all. α. such that. 0 ≤ |α| ≤ m.. Taking. we obtain. k ⊗dk=1 wk kLp (Ω ) =. d Y. kwk kLp (Ωk ) < ∞. k=1. kwk kLp (Ωk ) < ∞ for all k. Now, for k ∈ {1 . . . d}, taking α = (. . . , 0, j, 0, . . .) αk = j , with 1 ≤ j ≤ m, and αl = 0 for l 6= k, we obtain. and therefore such that. kDα (⊗dl=1 wl )kLp (Ω ) = k∂xj k wk kLp (Ωk ). Y. kwl kLp (Ωl ). l6=k. k∂xj k wk kp,Ωk < ∞ k ∈ {1 . . . d}. u t. and then. for all. j ∈ {1, . . . , m}.. Therefore. wk ∈ H m,p (Ωk ). for all.

(8) 8. Antonio Falcó, Anthony Nouy. Proof of Proposition 3 m ≥ 1,. For. m = 0 the proposition follows from Lemma 1. Now, assume. and let us consider a sequence.  {zn }n∈N ⊂ R1  a. d O.  H. m,p. (Ωj ) . j =1. z ∈ H m,p (Ω ).. that weakly converges to an element.  R1  a. d O. . Since. . H m,p (Ωj )  ⊂ R1  a. j =1 we have closed in. z ∈ H. z ∈ R1.  Lp (Ωj )  ,. j =1. N d. p j =1 L (Ωj ). a. d O. . because, from Lemma 1, the latter set is weakly. p. L (Ω ). Therefore, there exist wk ∈ Lp (Ωk ) such that z = ⊗dk=1 wk . Since (Ω ),from Lemma 2, wk∈ H m,p (Ωk ) for 1 ≤ k ≤ d, and therefore z =. m,p. ⊗dk=1 wk ∈ R1. a. Nd. j =1 H. m,p. (Ωj ). .. u t. From Proposition 5.18 and Example 5.19 in [15] it follows the following statement.. Proposition 4 The set Tr. N d a. j =1 H. m,2. (Ωj ). . is weakly closed in H m,2 (Ω ).. 3 Optimization of functionals over Banach spaces Let. V. be a reexive Banach space, endowed with a norm. dual space of. V. and we denote by. ∗. h·, ·i : V × V → R. k · k.. We denote by. V∗. the. the duality pairing. We consider. the optimization problem. J (u) = min J (v ). (π ). v∈V. J :V →R. where. is a given functional.. 3.1 Some useful results on minimization of functionals over Banach spaces In the sequel, we will introduce approximations of (π ) by considering an optimization on subsets. M ⊂V,. i.e.. J (u) = min J (v ). (8). v∈M. We here recall classical theorems for the existence of a minimizer (see e.g. [14]).. Denition 3. We say that a map. J : V −→ R. is weakly sequentially lower semicon-. M ⊂ V if for all v ∈ M and J (v ) ≤ lim inf m→∞ J (vm ) (respectively,. tinuous (respectively, weakly sequentially continuous) in for all. vm ∈ M. such that. vm * v ,. it holds. J (v ) = limm→∞ J (vm )). J 0 is strongly continuous ∗ when for any sequence vn * v in V it holds that J (vn ) → J (v ) in V . Recall that the convergence in norm implies the weak convergence. Thus, J weakly sequentially (lower semi)continuous in M ⇒ J (lower semi)continuous in M. It can be If. J 0 : V −→ V ∗. exists as Gateaux derivative, we say that. 0. 0. shown (see Proposition 41.8 and Corollary 41.9 in [39]) the following result..

(9) PGD for Nonlinear Convex Problems. 9. Proposition 5 Let V be a reexive Banach space and let J : V → R be a functional, then the following statements hold. (a) If J is a convex and lower semicontinuous functional, then J is weakly sequentially lower semicontinuous. (b) If J 0 : V −→ V ∗ exists on V as Gateaux derivative and is strongly continuous (or compact), then J is weakly sequentially continuous. Finally, we have the following two useful theorems.. Theorem 1 Assume V is a reexive Banach space, and assume M ⊂ V is bounded and weakly closed. If J : M → R ∪ {∞} is weakly sequentially lower semicontinuous, then problem (8) has a solution. Proof. α = inf v∈M J (v ) and {vn } ⊂ M be a minimizing sequence. Since A is {vn }n∈N is a bounded sequence in a reexive Banach space and therefore, there exists a subsequence {vnk }k∈N that converges weakly to an element u ∈ V . Since M is weakly closed, u ∈ M and since J is weakly sequentially lower semicontinuous, J (u) ≤ lim inf k→∞ J (vnk ) = α. Therefore, J (u) = α and u is solution of the minimization problem. u t Let. bounded,. We now remove the assumption that on. M. is bounded by adding a coercivity condition. J.. Theorem 2 Assume V is a reexive Banach space, and M ⊂ V is weakly closed. If J : M → R ∪ {∞} is weakly sequentially lower semicontinuous and coercive on M , i.e. limkvk→∞ J (v ) = +∞, then problem (8) has a solution. Proof. Pick an element. J (v ) ≤ J (v0 )}.. Since. J. v0 ∈ M. such that. is coercive,. M0. J (v0 ) 6= ∞. is weakly sequentially lower semicontinuous, is then equivalent to. and dene. is bounded. Since. J (u) = minv∈M0 J (v ),. M0. M. M0 = {v ∈ M : J. is weakly closed and. is weakly closed. The initial problem. which admits a solution from Theorem 1.. u t. 3.2 Convex optimization in Banach spaces From now on, we will assume that the functional (A1) (A2). J J. J. satises the following assumptions.. is Fréchet dierentiable, with Fréchet dierential is elliptic, i.e. there exist. α>0. and. s>1. J 0 : V → V ∗. v, w ∈ V ;. such that for all. hJ 0 (v ) − J 0 (w), v − wi ≥ αkv − wks In the following,. s. will be called the ellipticity exponent of. (9). J.. Lemma 3 Under assumptions (A1)-(A2), we have (a) For all v, w ∈ V, J (v ) − J (w) ≥ hJ 0 (w), v − wi +. (b) J is strictly convex. (c) J is bounded from below and coercive.. α kv − wks . s. (10).

(10) 10. Antonio Falcó, Anthony Nouy. Proof. (a) For all. v, w ∈ V ,. Z. 1. J (v ) − J (w) = 0. d J (w + t(v − w))dt = dt. Z. = hJ 0 (w), v − wi +. 1. hJ 0 (w + t(v − w)), v − widt. 0. α kt(v − w)ks dt t. 0. α kv − wks s. = hJ 0 (w), v − wi + (b) From (a), we have for. 1. hJ 0 (w + t(v − w)) − J 0 (w), v − widt. 0 Z 1. ≥ hJ 0 (w), v − wi +. Z. v 6= w, J (v ) − J (w) > hJ 0 (w), v − wi. (c) Still from (a), we have for all. v ∈V,. J (v ) ≥ J (0) + hJ 0 (0), vi +. α α kvks ≥ J (0) − kJ 0 (0)kkvk + kvks s s. which gives the coercivity and the fact that. J. is bounded from below.. u t The above properties yield the following classical result (see for example Theorem 7.4.4 in [10]).. Theorem 3 Under assumptions (A1)-(A2), the problem (π ) admits a unique solution. u ∈ V which is equivalently characterized by. hJ 0 (u), vi = 0. Proof. ∀v ∈ V. We here only give a sketch of proof of this very classical result.. and a fortiori lower semicontinuous. Since. J. (11). J. is continuous. is convex and lower semicontinuous, it is. also weakly sequentially lower semicontinuous (Proposition 5(a)). The existence of a solution then follows from Theorem 2. The uniqueness is given by the strict convexity of. J , and the equivalence between (π ) and (11) classically follows from the dierentiability of J . u t. Lemma 4 Assume that J satises (A1)-(A2). If {vm } ⊂ V is a sequence such that J (vm ) −→ J (u), where u is the solution of (π ), then vm → u, i.e. m→∞. ku − vm k −→ 0 m→∞. Proof. By the ellipticity property (10) of. J,. we have. J (vm ) − J (u) ≥ hJ 0 (u), vm − ui + Therefore,. which ends the proof.. α α ku − vm ks = ku − vm ks . s s. α ku − vm ks ≤ J (vm ) − J (u) −→ 0, m→∞ s u t. (12).

(11) PGD for Nonlinear Convex Problems. 11. 4 Progressive Proper Generalized Decompositions in Tensor Banach Spaces 4.1 Denition of progressive Proper Generalized Decompositions We now consider the minimization problem (π ) of functional. V = Vk·k .. Banach space. Assume that we have a functional. S1. (A1)-(A2) and a weakly closed subset (B1). with 0 ∈ S1 , v ∈ S1 we have λv ∈ S1 span S1 is dense in Vk·k .. such that. S1 ⊂ V,. (B2) for each (B3). Vk·k. in. J on a reexive tensor J : Vk·k −→ R satisfying. λ ∈ R,. for all. and. Using the notation introduced in Section 2.2 we give the following examples.. Example 1. Consider. Vk·k = Lpµ (I ; X ). and. S1 = Tr (X ⊗a Lpµ (I )).. Example 2. Consider. Vk·k = H m,2 (Ω ). and. S1 = Tr. Example 3. Consider. Vk·k = H m,p (Ω ). and. S1 = R1. The set. S1. N d a. m,2 (Ωj ) j =1 H. N d a. j =1 H. m,p. . .. (Ωj ) .. can be used to characterize the solution of problem (π ) as shown by the. following result.. Lemma 5 Assume that J satises (A1)-(A2) and let u∗ ∈ Vk·k satisfy J (u∗ ) = min J (u∗ + z). z∈S1. (13). Then u∗ solves (π ). Proof. For all. γ ∈ R+. and. z ∈ S1 , J (u∗ + γz) ≥ J (u∗ ). and therefore. hJ 0 (u∗ ), zi = lim. γ&0. holds for all. z ∈ S1 .. 1 γ. (J (u∗ + γz) − J (u∗ )) ≥ 0. From (B2), we have. hJ 0 (u∗ ), zi = 0. ∀z ∈ S1 ,. From (B3), we then obtain. hJ 0 (u∗ ), vi = 0 and the lemma follows from Theorem 3. In the following, we denote by. Sm. Sm =. ∀v ∈ Vk·k ,. u t. the set. (m X. ) zi : zi ∈ S1. i=1 The next two lemmas will be useful to dene a progressive Proper Generalized Decomposition..

(12) 12. Antonio Falcó, Anthony Nouy. Lemma 6 For each v ∈ Vk·k , the set v + S1 = {v + w : w ∈ S1 }. is weakly closed in Vk·k . Proof S1 u t. Assume that. v + wn * w for some {wn }n≥1 ⊂ S1 , then wn * w − v and since w − v ∈ S1 . In consequence w ∈ v + S1 and the lemma follows.. is weakly closed,. Lemma 7 (Existence of a S1 -minimizer) Assume that J : Vk·k −→ R satises (A1)-(A2). Then for any v ∈ Vk·k , the following problem admits a solution: J (w ∗ ) =. Proof. Fréchet dierentiability of. we have that. J. J. min J (w). w∈v+S1. implies that. J. is continuous and since. J. is convex,. is weakly sequentially lower semicontinuous by Proposition 5. Moreover,. J. is coercive on. in. Vk·k .. Vk·k. by Lemma 3(c). By Lemma 6,. v + S1. is a weakly closed subset. Then, the existence of a minimizer follows from Theorem 2.. u t. Denition 4 (Progressive PGDs) Assume that J : Vk·k −→ R satises (A1)-(A2), and let. u ∈ Vk·k. satisfy. J (u) = min J (v).. (14). v∈Vk·k. {um }m≥1 over S1 of u, um ∈ Vk·k from um−1 ∈ Vk·k ∈ S1 ⊂ V such that. We dene a progressive Proper Generalized Decomposition follows. We let. u0 = 0. and for. m ≥ 1,. we construct. we show below. We rst nd an element. ẑm. J (um−1 + ẑm ) = min J (um−1 + z) z∈S1. Next before to update by. (c ) (l ). c, l. and. r,. as. (∗).. m to m +1, we can choose one of the following strategies denoted. respectively:. zm = ẑm . Dene um = um−1 + zm , update m to m + 1 and goto (∗). zm = ẑm . Construct a closed subspace U(um−1 + zm ) in Vk·k such um−1 + zm ∈ U(um−1 + zm ). Then, dene um by Let. Let. J (um ) = update. (r ). as. m. to. m+1. (∗). U(ẑm ). min. v∈U(um−1 +zm ). J (v ),. and goto. Construct a closed subspace. in. Vk·k. such that. ẑm ∈ U(ẑm ),. by. J (z m ) = Then, dene. um = um−1 + zm ,. min. z∈U(ẑm ). J (um−1 + z).. such that. J (um ) = J (um−1 + zm ) = update. m. to. m+1. that. and goto. (∗).. min. v∈um−1 +U(ẑm ). J (v ),. and dene. zm.

(13) PGD for Nonlinear Convex Problems Strategies of type. (l ). and. (r ). 13. updates.. are called. Proper Generalized Decomposition. {um }m≥1. of. Observe that to each progressive. u we can assign a sequence of symbols. (perhaps nite), that we will denote by. α(u) = α1 α2 · · · αk · · · where. αk ∈ {c, l, r} for all k = 1, 2, . . . . That means that uk αk = c, or with an update strategy of type l or r. update if. was obtained without if. αk = l. or. αk = r. respectively. In particular, the progressive PGD dened in [7] coincides with a PGD where. αk = c. for all. k ≥ 1.. Such a decomposition is called a. while a decomposition such that. progressive PGD.. αk = l. or. αk = r. purely progressive PGD, k is called an updated. for some. Remark 2. The update αm = l can be dened with several updates at each iteration. (0) (p ) m ⊂ Vk·k dened by um = um−1 + ẑm , we introduce a sequence {um }dp=1. Letting. (p+1). J (um. )=. min. J (v ). (p). v∈U(um ) with let. (p ). U(um ). being a closed linear subspace of. (d m ). um = um. Vk·k. which contains. (p ). um. . We nally. .. v in the algebraic Uj,min (v) ⊂ Vj are given by the intersecNd v ∈ a j =1 Uj . It can be shown [15] that. In [15] it was introduced the following denition. For a given tensor space. V,. the minimal subspaces. tion of all subspaces. a. Uj ⊂ Vj. satisfying. Nd. j =1 Uj,min (v) is a nite dimensional subspace of. Example 4 (Illustrations of updates) αm = l. subspace. U (v m ) = . For a given. vm = ẑm if αm = r) there are U(vm ). Among others, we have. or. V.. vm ∈ Vk·k. (e.g.. vm = um−1 + zm. if. several possible choices for dening a linear. Nd. j =1 Uj,min (vm ) . In the case of αm = l, all subspaces U(um−1 + zm ) um ∈ V for all m ≥ 1. Pm Assume that vm = i=1 αi zi for some {z1 , . . . , zm } ⊂ Vk·k , αi ∈ R, 1 ≤ i ≤ m. a. are nite dimensional and we have that Then we can dene. U(vm ) = span {z1 , . . . , zm }. In the context of Greedy algorithms for computing best approximations, an up-. αm = r. date of type. by using an orthonormal basis of. U(vm ). corresponds to an. orthogonal Greedy algorithm.. . Assume. vm ∈ V. Fix k ∈ {1, 2, . . . , d}. By using. we can write. N. j6=k. (j ). wi. vm =. . for. Pm. (k ). i=1 wi. ⊗. i = 1, . . . , m.. U(vm ) =.  m X . i=1. (k ). vi. N. j6=k. (j ). wi. . a. Nd. j =1 Vj. ∼ Vk ⊗a =. N a. . j6=k. for some elementary tensors. Vj , (k ). wi. ⊗. Then we can dene the linear subspace.     O (j ) ( k ) ⊗ wi  : vi ∈ Vk , 1 ≤ i ≤ m .  j6=k. The minimization on U(vm ) corresponds to an update of functions along dimension k (functions in the Banach space Vk ). Following the remark 2, several updates could be dened by choosing a sequence of updated dimensions..

(14) 14. Antonio Falcó, Anthony Nouy. 4.2 On the convergence of the progressive PGDs Now, we study the convergence of progressive PGDs. Recall that. ẑm ∈ S1. is a solution. of. J (um−1 + ẑm ) = min J (um−1 + z), z∈S1. For. αm = c,. we have. zm = ẑm. um = um−1 + zm ,. and. so that. J (um ) = J (um−1 + zm ) = J (um−1 + ẑm ) αm = l, we have zm = ẑm um−1 + zm , so that. For of. and. um. is obtained by an update (or several updates). J (um ) ≤ J (um−1 + zm ) = J (um−1 + ẑm ) Otherwise, for. ẑm ,. αm = r,. we have. um = um−1 + zm. with. zm. obtained by an update of. such that. J (um ) = J (um−1 + zm ) ≤ J (um−1 + ẑm ) We begin with the following Lemma.. Lemma 8 Assume that J satises (A1)-(A2), and let u ∈ Vk·k satisfy (14). Then {J (um )}m≥1 , where {um }m≥1 is a progressive Proper Generalized Decomposition over S1 of u, is a non increasing sequence: J (um ) ≤ J (um−1 ) for all m ≥ 1.. Moreover, if J (um ) = J (um−1 ), um−1 is the solution of (π ). Proof. By denition, we have. J (um ) ≤ J (um−1 + zm ) ≤ J (um−1 + ẑm ) ≤ J (um−1 + z) In particular, since. 0 ∈ S1. J (um ) = J (um−1 ),. we have. by assumption (B1), we have. ∀z ∈ S1. J (um ) ≤ J (um−1 ).. If. J (um−1 ) = min J (um−1 + z) z∈S1. and by Lemma 5, we have that. Remark 3. If. um−1. J (um ) = J (um−1 ). solves (π ).. holds for some. u t m > 1,. that is. um−1. is the solution. α(u) = 1 ≤ k ≤ m − 1. Otherwise, {J (um )}m∈N is a N numbers and α(u) ∈ {c, l, r} .. of (π ), then the updated PGD is described by a nite sequence of symbols. α1 α2 · · · αm−1 ,. where. αk ∈ {c, l, r}. strictly decreasing sequence of real. Denition 5 symbols. α ∈ {c, l, r}. αα···α··· . Let. for. Then. α∞ ∈ {c, l, r}N. denotes the innite sequence of. From now on, we will distinguish two convergence studies, one with a weak continuity assumption on functional on. J. J,. the other one without weak continuity assumption. but with an additional Lipschitz continuity assumption on the dierential. J 0..

(15) PGD for Nonlinear Convex Problems. 15. 4.2.1 A rst approach for weakly sequentially continuous functionals Here, we introduce the following assumption. (A3) The map. J : Vk·k −→ R. is weakly sequentially continuous.. Theorem 4 Assume that J satises (A1)-(A3), and let u ∈ Vk·k satisfy (14). Then every progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, converges in Vk·k to u, that is, lim ku − um k = 0. m→∞. Proof. From Lemma 8, {J (um )} is a non increasing sequence. If there exists m such J (um ) = J (um−1 ), from Lemma 8, we have um = u, which ends the proof. Let us now suppose that J (um ) < J (um−1 ) for all m. J (um ) is a strictly decreasing sequence which is bounded below by J (u). Then, there exists that. J ∗ = lim J (um ) ≥ J (u). m→∞. If. J. ∗. = J (u), Lemma 4 allows to conclude that um → u. Therefore, it remains to J ∗ = J (u). Since J is coercive, the sequence {um }m∈N is bounded in Vk·k .. prove that. Then, there exists a subsequence Since. J. {umk }k∈N. that weakly converges to some. u∗ ∈ V .. is weakly sequentially continuous, we have. J ∗ = lim J (umk ) = J (u∗ ). k→∞. By denition of the PGD, we have for all. z ∈ S1 ,. J (um(k+1) ) ≤ J (umk +1 ) ≤ J (umk + z) Taking the limit with. k,. and using the weak sequential continuity of. ∗. ∗. J (u ) ≤ J (u + z ) and by Lemma 5, we obtain proof.. u∗ = u. J,. we obtain. ∀z ∈ S1 ,. and a fortiori. J (u∗ ) = J (u),. that concludes the. u t. 4.2.2 A second approach for a class of functionals with Lipschitz continuous derivative on bounded sets Now, assume that assumption (A3) is replaced by (A3). ∗ J 0 : Vk·k −→ Vk·k is Lipschitz continuous on bounded set in Vk·k , there exists a constant CA > 0 such that. sets, i.e. for. A. a bounded. kJ 0 (v) − J 0 (w)k ≤ CA kv − wk for all. (15). v, w ∈ A.. The next ve lemmas will give some useful properties of the sequence. {zm }m≥1 .. Lemma 9 Assume that J satises (A1)-(A2), and let u ∈ Vk·k satisfy (14). If {um }m≥1 is a progressive Proper Generalized Decomposition over S1 of u, then hJ 0 (um−1 + zm ), zm i = 0,. hold for all m ≥ 1..

(16) 16. Antonio Falcó, Anthony Nouy. Proof. zm = λm wm , with λm ∈ R+ and kwm k = 1. In the cases αm = c (purely progressive PGD) and αm = l, we have J (zm ) = J (ẑm ) = minz∈S1 J (um−1 + z). Let. From assumption (B2), we obtain. J (um−1 + λm wm ) ≤ J (um−1 + λwm ) for all. z). λ ∈ R. This inequality is also true for αm = r since J (zm ) = minz∈U(ẑm ) J (um−1 + U(ẑm ) is a linear space. Taking λ = λm ± γ , with γ ∈ R+ , we obtain for all. and. cases. 0≤ Taking the limit. 1 γ. (J (um−1 + λm wm ± γwm ) − J (um−1 + λm wm )) .. γ & 0,. we obtain. 0 ≤ ±hJ 0 (um−1 + λm wm ), wm i. and therefore. hJ 0 (um−1 + λm wm ), wm i = 0, which ends the proof.. u t. Lemma 10 Assume that J satises (A1)-(A2), and let u ∈ Vk·k satisfy (14). Then the corrections {zm }m≥1 of a progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, satisfy ∞ X. kzm ks < ∞,. (16). m=1. for the ellipticity constant s > 1 and thus, lim kzm k = 0.. (17). m→∞. Proof. By the ellipticity property (10), there exist. s>1. and. α>0. J (um−1 ) − J (um−1 + zm ) ≥ −hJ 0 (um−1 + zm ), zm i + Using Lemma 9 and. J (um ) ≤ J (um−1 + zm ),. m,. and using. α kzm ks . s. we then obtain. J (um−1 ) − J (um ) ≥ Now, summing on. such that. α kzm ks s. limm→∞ J (um ) = J ∗ < ∞,. (18) we obtain. ∞ ∞ X α X (J (um−1 ) − J (um )) = J (0) − J ∗ < +∞. kzm ks ≤ s m=1. which implies proves (17).. m=1. limm→∞ kzm ks = 0.. The continuity of the map. x 7→ x1/s. at. x = 0. u t. Lemma 11 Assume that J satises (A1)-(A3), and let u ∈ Vk·k satisfy (14). Then for every progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, there exists C > 0 such that for m ≥ 1, |hJ 0 (um−1 ), zi| 6 Ckzm kkzk,. holds for all z ∈ S1 ..

(17) PGD for Nonlinear Convex Problems. 17. Proof. Since J (um ) converges and since J is coercive, {um }m≥1 is a bounded sequence. kzm k → 0 as m → ∞ (Lemma 10), {zm }m≥1 is also a bounded sequence. Let a > 0 such that supm kum k + supm kzm k ≤ a and let CB be the Lipschitz continuity 0 constant of J on the bounded set B = {v ∈ Vk·k : kvk ≤ a}. Then. Since. −hJ 0 (um−1 ), zi = hJ 0 (um−1 + z) − J 0 (um−1 ), zi − hJ 0 (um−1 + z), zi ≤ CB kzk2 − hJ 0 (um−1 + z), zi z ∈ A = {z ∈ S1 : kzk ≤ supm kzm k}. zm ) ≤ J (um−1 + z) for all z ∈ S1 , we have. for all. By convexity of. J. and since. J (um−1 +. hJ 0 (um−1 + z), zm − zi ≤ J (um−1 + zm ) − J (um−1 + z) ≤ 0 z ∈ A,. Therefore, for all. we have. 0. −hJ (um−1 ), zi ≤ CB kzk2 − hJ 0 (um−1 + z), zm i ≤ CB kzk2 − hJ 0 (um−1 + z) − J 0 (um−1 + zm ), zm i 2. ≤ CB kzk + CB kz − zm kkzm k. . 2. (Choice of. ≤ CB kzk + kzkkzm k + kzm k z = wkzm k ∈ A,. Let. with. kwk = 1.. |hJ (um−1 ), wi| ≤ 3CB kzm k w = z/kzk,. with. z ∈ S1 ,. B). . Then. 0. Taking. 2. (Lemma 9). and. ∀w ∈ {w ∈ S1 : kwk = 1}. C = 3 CB > 0. we obtain. |hJ 0 (um−1 ), zi| ≤ Ckzm kkzk. ∀z ∈ S1. u t Since. Vk·k. ∗∗ ,V∗ h·, ·iVk·k k·k. is reexive, we can identify. with. ∗ ,V h·, ·iVk·k (i.e. k·k. ∗∗ Vk·k. with. Vk·k. and the duality pairing. weak and weak-∗ topologies coincide on. ∗ ). Vk·k. Lemma 12 Assume that J satises (A1)-(A3), and let u ∈ Vk·k satisfy (14). Then for every progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, the se∗ quence {J 0 (um )}m∈N weakly-∗ converges to 0 in Vk·k , that is, limm→∞ hJ 0 (um ), zi = 0 for all z in a dense subset of Vk·k . Proof. The sequence. {um }m∈N. being bounded, and since. bounded sets, we have that there exists a constant. J0. C>0. is Lipschitz continuous on. such that. kJ 0 (um )k = kJ 0 (um ) − J 0 (u)k ≤ Cku − um k ∗ ∗ {J 0 (um )} ⊂ Vk·k is a bounded sequence. Since Vk·k is also re0 exive, from any subsequence of {J (um )}m∈N , we can extract a further subsequence 0 ∗ {J (umk )}k∈N that weakly-∗ converges to an element ϕ ∈ Vk·k . By using Lemma 11, we have for all z ∈ S1 , |hJ 0 (umk ), zi| ≤ Ckzmk +1 kkzk. That proves that. Taking the limit with. k,. and using Lemma 10, we obtain. hϕ, zi = 0. ∀z ∈ S1 ,. ϕ = 0. Since from any subsequence of {J 0 (um )}m∈N we can extract a further subsequence that weakly-∗ same limit 0, then the whole sequence converges to 0. u t. By using assumption (B3), we conclude that the initial sequence converges to the.

(18) 18. Antonio Falcó, Anthony Nouy. Lemma 13 Assume that J satises (A1)-(A3), and let u ∈ Vk·k satisfy (14). Let us consider a progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, such that for the ellipticity constant s of J and α(u) = α1 · · · αm · · · , one of the two following conditions hold: (a) s > 1 and there exists a subsequence {αmk }k∈N such that αmk = l for all k ≥ 1. (b) 1 < s ≤ 2 and there exists k ≥ 1 such that α(u) = α1 · · · αk−1 α∞ where α ∈ {c, r}.. Then, there exists a subsequence {umk }k∈N such that hJ 0 (umk ), umk i → 0.. Proof. αm = l for some m ≥ 1, the U(um−1 + zm ) ⊂ Vk·k . 0 The global minimum is attained and unique, and it is characterized by J (um ), v = 0 for all v ∈ U(um−1 + zm ). Thus, under condition (a), there exists a subsequence such 0 that hJ (umk ), umk i = 0 for all k ≥ 1. Now, we consider that statement (b) holds. ∞ Without loss of generality we may assume that α(u) = α where α ∈ {c, r}. In both Pm z . Thus, we have cases, um = k=1 k um. First, assume that condition (a) holds. Recall that if. is obtained by the minimization of. |hJ 0 (um ), um i| ≤. m X. J. on the closed subspace. |hJ 0 (um ), zk i|. k=1 m X. ≤C. kzm+1 kkzk k. (By Lemma 11).. k=1 Let. s∗ > 1. be such that. 1/s∗ + 1/s = 1.. By Holder's inequality, we have. m X. 1/s∗. 0. |hJ (um ), um i| ≤ Ckzm+1 km. !1/s kzk k. s. k=1. . = C mkzm+1 k. s∗. 1/s∗. m X. !1/s kzk k. s. .. (19). k=1. P∞. s. k=1 kzk k. s. < ∞. Then there exists a subsequence such that mk kzmk +1 k → 0. For 1 < s ≤ 2, we have s ≤ s∗ . Since limk→∞ kzk k = 0, we have ∗ ∗ kzk ks ≤ kzk ks for k large enough, and therefore we also have mk kzmk +1 ks → 0, which from (19) ends the proof of the lemma. u t. From Lemma 10, we have. Theorem 5 Assume that J satises (A1)-(A3), and let u ∈ Vk·k satisfy (14). Let us consider a progressive Proper Generalized Decomposition {um }m≥1 over S1 of u, such that for the ellipticity constant s of J and α(u) satisfy one of the following conditions: (a) s > 1 and there exists a subsequence {αmk }k∈N such that αmk = l for all k ≥ 1. (b) 1 < s ≤ 2 and there exists k ≥ 1 such that α(u) = α1 · · · αk−1 α∞ where α ∈ {c, r}.. (c) s > 1 and α(u) is nite. Then {um }m≥1 , converges in Vk·k to u, that is, lim ku − um k = 0.. m→∞.

(19) PGD for Nonlinear Convex Problems. Proof. 19. From the same argument used in the proof of Theorem 4, if (c) holds we have. um = u. for some. m.. Thus, the theorem follows in this case. Otherwise,. {J (um )}. is. strictly decreasing and there exists. J ∗ = lim J (um ) ≥ J (u). m→∞. If. J ∗ = J (u),. Lemma 4 allows to conclude that. fore, it remains to prove that. J ∗ = J (u).. {um }. strongly converges to. By the convexity of. J,. u.. There-. we have. J (um ) − J (u) ≤ hJ 0 (um ), um − ui = hJ 0 (um ), um i − hJ 0 (um ), ui By Lemmas 12 and 13, we have that there exists a subsequence. hJ 0 (umk ), umk i → 0. and. hJ 0 (umk ), ui → 0,. {umk }k∈N. such that. and therefore. J ∗ − J (u) = lim J (umk ) − J (u) ≤ 0 k→∞. Since we already had. J ∗ ≥ J (u),. this yields. J ∗ = J (u ),. which ends the proof.. u t. 5 Examples 5.1 On the Singular Value Decomposition in. Lp. spaces for. p≥2. V is said to be smooth if for any linearly independent elements x, y ∈ V , φ(t) = kx − tyk is dierentiable. A Banach space is said to be uniformly. A Banach space the function. smooth if its modulus of smoothness. ρ(τ ) =. sup. kx + τ yk + kx − τ yk. 2. x,y∈V kxk=kyk=1. −1 ,. τ > 0,. satises the condition. lim. τ →0. ρ(τ ) = 0. τ. In uniformly smooth spaces, and only in such spaces, the norm is uniformly Fréchet dierentiable. It can be shown that the. Lp -spaces for 1 < p < ∞ are uniformly smooth. (see Corollary 6.12 in [29]). Following section 2.2.1, we introduce the tensor product of Lebesgue spaces. Lpµ (I1 × I2 ) = Lpµ1 (I1 ) ⊗∆p Lpµ2 (I2 ) = Lpµ1 (I1 , Lpµ2 (I2 )), with. p ≥ 2,. and. µ = µ1 ⊗ µ2. a nite product measure. Recall that. Z kvk∆p = Let. R. u. |v(x)|p dµ(x). 1/p. I1 ×I2. p be a given function in Lµ (I1 ×I2 ). We introduce the functional. dened by. J (v ) =. 1 p. kv − ukp∆p .. J : Lpµ (I1 ×I2 ) →.

(20) 20 Let. Antonio Falcó, Anthony Nouy. G : Lpµ (I1 × I2 ) → R. be the functional given by the. well known (see for example page 170 in [20]) that. G. p-norm G(v) = kvk∆p .. It is. is Fréchet dierentiable, with. 1−p ∈ Lqµ (I1 × I2 ), G0 (v) = v |v|p−2 kvk∆ p. q such that 1/q +1/p = 1. We denote by C k the set of k-times Fréchet dierentiable p ∞ functionals from Lµ (I1 × I2 ) to R. Then, if p is an even integer, G ∈ C . Otherwise, when p is not an even integer, the following statements hold (see [5] and 13.13 in [25]):. with. (a) If. p. G is (p − 1)-times. is an integer,. dierentiable with Lipschitzian highest Fréchet. derivative. (b) Otherwise,. G is [p]-times Fréchet dierentiable with highest derivative being Höldep − [p].. rian of order (c). G. has no higher Hölder Fréchet dierentiability properties.. As a consequence we obtain that. G ∈ C2. for all. p ≥ 2, and the functional J is also J 0 (v) = G(v − u)p−1 G0 (v − u),. Fréchet dierentiable with Fréchet derivative given by that is,. hJ 0 (v), wi =. Z. (v − u) |v − u|p−2 w dµ.. I1 ×I2 Thus, where. J V. F : V −→ W, v ∈ V , then it is also 0 1 that J ∈ C , and as a. satises assumption (A1). It is well-known that if a functional and. W. are Banach spaces, is Fréchet dierentiable at. locally Lipschitz continuous at consequence. J0. v ∈ V.. Thus, if. p ≥ 2,. we have. satises (A3).. Finally, in order to prove the convergence of the (updated) progressive PGD for each. u ∈ Lpµ (I1 × I2 ). have to verify that (A2) that for all. S1 = T(r1 ,r2 ) (Lpµ1 (I1 ) ⊗a Lpµ2 (I2 )), where (r1 , r2 ) ∈ N2 , we on J is satised. Since there exists a constant αp > 0 such. over. s, t ∈ R,. (|s|p−2 s − |t|p−2 t)(s − t) ≥ αp |s − t|p (see for example (7.1) in [8]), then, for all. v, w ∈ Lpµ (I1 × I2 ),. hJ 0 (v) − J 0 (w), v − wi ≥ αp kv − wkp , which proves the ellipticity property of. J,. and assumption (A2) holds.. From Theorem 5, we conclude that the (updated) progressive Proper Generalized Decomposition converges. . p ≥ 2 if conditions (a) or (c) of Theorem p = 2 if condition (b) of Theorem 5 holds.. for all for. 5 hold.. Let us detail the application of the progressive PGD over. S1 = T(r1 ,r2 ) (Lpµ1 (I1 ) ⊗a Lpµ2 (I2 )). r1 = r2 = r. The v ∈ Lpµ1 (I1 ) ⊗a Lpµ2 (I2 ), there exist two minimal subspaces Uj,min (v), j = 1, 2, with dim U1,min (v) = dim U2,min (v) and such that v ∈ U1,min (v) ⊗a U2,min (v). In consequence, for a xed r ∈ N and for. We claim that in dimension. d = 2,. we can only consider the case. claim follows from the fact that (see [15]) for each. u ∈ Lpµ (I1 × I2 ) \ Lpµ1 (I1 ) ⊗a Lpµ2 (I2 ).

(21) PGD for Nonlinear Convex Problems we let. u1. 21. such that. J (u1 ) = Then there exist two bases. J (z ).. min. p z∈T(r,r) (Lp µ1 (I1 )⊗a Lµ2 (I2 )). (j ). (j ). {u1 , . . . , ur } ⊂ Lpµj (Ij ). that. r X r X. u1 =. (1). Uj,min (v),. of. for. j = 1, 2,. such. (2). σk,l uk ⊗ ul ,. k=1 l=1 and. u − u1 ∈ /. Lpµ1 (I1 ) ⊗a. Lpµ2 (I2 ). Proceeding inductively we can write um =. mr X mr X. (1). (2). σk,l uk ⊗ ul .. k=1 l=1. m, an example of update of type αm = r would consist in updating the {σk,l : k, l ∈ {(m− 1)r +1, . . . , mr}}. An example of update of type αm = l would consist in updating the whole set of coecients {σk,l : k, l ∈ {1, . . . , mr}}. In the case p = 2 and when we take orthonormal bases, it corresponds to the 2 classical SVD decomposition in the Hilbert space Lµ (I1 × I2 ). In this case we have At step. coecients. um =. mr X. (1). σj uj. (2). ⊗ uj .. j =1 where. (j ). (j ). σj = |hu, u1 ⊗ u2 i|,. for. 1 ≤ j ≤ mr.. In this sense, the progressive PGD can be interpreted as a SVD decomposition of a. u (l ). Lp -space. where. p ≥ 2.. Let us recall that for. p > 2,. function. in a. of type. is required for applying Theorem 5 (at least for a subsequence of iterates).. an update strategy. The above results can be naturally extended to tensor product of Lebesgue spaces,. d a ⊗k=1. Lpµk (Ik ). k·k∆p. with. d>2. and. S1 = R1. N d a. p k=1 Lµk (Ik ). . ,. leading to a gen-. eralization of multidimensional singular value decomposition introduced in [16] for the case of Hilbert tensor spaces.. 5.2 Nonlinear Laplacian We here present an example taken from [8]. We refer to section 2.2.2 for the introduction to the properties of Sobolev spaces. Let. Vk·k = H01,p (Ω ), support in. Ω). in. Ω = Ω1 × . . . × Ωd . Cc∞ (Ω ) (functions. which is the closure of. H 1,p (Ω ). with respect to the norm in. with the norm. kvk =. d X. Given some in. C ∞ (Ω ). H 1,p (Ω ).. p > 2,. we let. with compact. We equip. H01,p (Ω ). !1/p k∂xk (v)kpLp (Ω ). k=1. k · k1,p on H 1,p (Ω ) introduced J : Vk·k → R dened by. which is equivalent to the norm then introduce the functional. J (v ) =. 1 p. kvkp − hf , vi,. in section 2.2.2. We.

(22) 22. Antonio Falcó, Anthony Nouy. with. ∗ f ∈ Vk·k .. Its Fréchet dierential is. J 0 (v) = A(v) − f where. A(v) = −. d X ∂ k=1. A. is called the. p-Laplacian.. ∂v ∂xk. ∂xk. ∂v ∂xk. !. Assumptions (A1)-(A3) on the functional are satised (see. [8]). Assumption (B3) on the set. R1. N d. m,p (Ωj ) j =1 H0. a. can be easily proved from Proposition 3 that the set. 1,p closed in (H0 (Ω ), k it is also weakly. p−2. · k1,p ). Since the norm k · k 1,p closed in (H0 (Ω ), k · k).. R1. . is also satised. Indeed, it. N d a. 1,p. j =1 H0. is equivalent to. (Ωj ). k · k1,p. . on. is weakly. H01,p (Ω ),. Then, from Theorem 5, the progressive PGD converges if there exists a subsequence of updates of type. (l ).. 5.3 Linear elliptic variational problems on Hilbert spaces. Let. Vk·k = V1 ⊗a . . . ⊗a Vd. k·k. be a tensor product of Hilbert spaces. We consider the. following problem. J (u) = min J (v), v∈K. where. K ⊂ Vk·k , a : Vk·k × Vk·k → R. J (v ) =. 1 a(v, v) − `(v) 2. is a coercive continuous symmetric bilinear. form,. a(v, v) ≥ αkvk2. ∀v ∈ Vk·k ,. a(v, w) ≤ βkvkkwk ` : Vk·k → R. ∀v, w ∈ Vk·k ,. is a continuous linear form,. `(v) ≤ γkvk. ∀v ∈ Vk·k .. Case where K is a closed and convex subset of Vk·k .. The solution. u. is equivalently. characterized by the variational inequality. a(u, v − u) ≥ `(v − u). ∀v ∈ K. In order to apply the results of the present paper, we have to recast the problem as. Vk·k . We introduce a convex and Fréchet dierentiable ∗ j : Vk·k → R with Fréchet dierential j 0 : Vk·k → Vk·k , such that j (v) = 0 0 if v ∈ K and j (v) > 0 if v ∈ / K . We further assume that j is Lipschitz on bounded sets. −1 We let j (v) = j (v), with > 0, and introduce the following penalized problem. an optimization problem in functional. J (u ) = min J (v), v∈Vk·k. J (v) = J (v) + j (v).

(23) PGD for Nonlinear Convex Problems As. → 0, j. 23. tends to the indicator function of set. Assumptions (A1)-(A2) are veried since ferential. ∗ J0 : Vk·k → Vk·k. J. K. and. u → u. (see e.g. [17]).. is Fréchet dierentiable with Fréchet dif-. dened by. hJ0 (v), zi = a(v, z) − `(z) + hj0 (v), zi, and. J. is elliptic since. hJ0 (v) − J0 (w), v − wi = a(v − w, v − w) + hj0 (v) − j0 (w), v − wi ≥ αkv − wk2 Assumption (A3) comes from the continuity of. Case where K = Vk·k .. If. K = Vk·k ,. a. and. `. and from the properties of. j0.. we recover the classical case of linear elliptic. variational problems on Hilbert spaces analyzed in [16]. In this case, the bilinear form. a denes a norm kvka = J is here equal to. p. a(v, v) on Vk·k , equivalent to the norm k·k. The functional J (v ) =. 1 1 ku − vk2a − kuk2a 2 2. The progressive PGD can be interpreted as a generalized Eckart-Young decomposition (generalized singular value decomposition) with respect to this non usual metric, and dened progressively by. ku − um k2a = min ku − um−1 − zk2a z∈S1. We have. J (um−1 ) − J (um ) =. 1 1 2 kzm k2a := σm 2 2. and. ku − um k2a = kuk2a −. m X k=1. where. σm. σk2 −→ 0 m→∞. can be interpreted as the dominant singular value of. (u − um−1 ) ∈ Vk·k .. The PGD method has been successfully applied to this class of problems in dierent contexts: separation of spatial coordinates for the solution of Poisson equation in high dimension [2, 6], separation of physical variables and random parameters for the solution of parameterized stochastic partial dierential equations [30].. 6 Conclusion In this paper, we have considered the solution of a class of convex optimization problems in tensor Banach spaces with a family of methods called progressive Proper Generalized Decomposition (PGD) that consist in constructing a sequence of approximations by successively correcting approximations with optimal elements in a given subset of tensors. We have proved the convergence of a large class of PGD algorithms (including update strategies) under quite general assumptions on the convex functional and on the subset of tensors considered in the successive approximations. The resulting succession of approximations has been interpreted as a generalization of a multidimensional singular value decomposition (SVD). Some possible applications have been considered. Further theoretical investigations are still necessary for a better understanding of the dierent variants of PGD methods and the introduction of more ecient algorithms.

(24) 24. Antonio Falcó, Anthony Nouy. for their construction (e.g. alternated direction algorithms). The analysis of algorithms for the solution of successive approximation problems on tensor subsets is still an open problem. In the case of dimension. d = 2,. further analyses would be required in order. to better characterize the PGD as a direct extension of SVD when considering more general norms.. References 1. A. Ammar, F. Chinesta, and A. Falcó. On the convergence of a greedy rank-one update algorithm for a class of linear systems. Archives of Computational Methods in Engineering, 17(4):473486, 2010. 2. A. Ammar, B. Mokdad, F. Chinesta, and R. Keunings. A new family of solvers for some classes of multidimensional partial dierential equations encountered in kinetic theory modelling of complex uids. Journal of Non-Newtonian Fluid Mechanics, 139(3):153 176, 2006. 3. J. Ballani and L. Grasedyck. A projection method to solve linear systems in tensor format. Technical Report 309, RWTH Aachen, 2010. 4. P. Blanchard and E. Brüning. Mathematical Methods in Physics: Distributions, Hilbert Space Operators, and Variational Methods. Birkhäuser, 2003. 5. R. Bonic and J. Frampton. Dierentiable functions on certain Banach spaces, Bull. Am. Math. Soc., 71(2):393395, 1965. 6. C. Le Bris, T. Lelievre, and Y. Maday. Results and questions on a nonlinear approximation approach for solving high-dimensional partial dierential equations. Constructive Approximation, 30(3):621651, 2009. 7. E. Cances, V. Ehrlacher, and T. Lelievre. Convergence of a greedy algorithm for highdimensional convex nonlinear problems. arXiv, arXiv(1004.0095v1 [math.FA]), April 2010. 8. C. Canuto and K. Urban. Adaptive optimization of convex functionals in banach spaces. SIAM J. Numer. Anal., 42(5):2043-2075, 2005. 9. F. Chinesta, A. Ammar, and E. Cueto. Recent advances in the use of the Proper Generalized Decomposition for solving multidimensional models. Archives of Computational Methods in Engineering, 17(4):373391, 2010. 10. P. G. Ciarlet. Introduction to Numerical Linear Algebra and Optimization. Cambridge Texts in applied Mathematics, Cambridge University Press, 1989. 11. A. Defant, and K. Floret. Tensor Norms and Operator ideals. North-Holland, 1993. 12. L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl., 21(4):12531278, 2000. 13. V. de Silva and L.-H. Lim. Tensor rank and ill-posedness of the best low-rank approximation problem. SIAM Journal of Matrix Analysis & Appl., 30(3):10841127, 2008. 14. I. Ekeland and R. Teman. Convex Analysis and Variational Problems. Classics in Applied Mathematics. SIAM, 1999. 15. A. Falcó and W. Hackbusch. On minimal subspaces in tensor representations Preprint 70/2010 Max Planck Institute for Mathematics in the Sciences, 2010. 16. A. Falcó and A. Nouy. A Proper Generalized Decomposition for the solution of elliptic problems in abstract form by using a functional Eckart-Young approach. J. Math. Anal. Appl., 376: 469480, 2011. 17. R. Glowinski. Numerical Methods for Nonlinear Variational Problems. Springer-Verlag, New York, 1984. 18. W. Hackbusch, and S. Kühn A new scheme for the tensor representation. J. Fourier Anal. Appl. 15 (2009) 706722. 19. W. Hackbusch, B. N. Khoromskij, S. A. Sauter, and E. E. Tyrtyshnikov. Use of tensor formats in elliptic eigenvalue problems. Technical Report Research Report No.2010-78, Max Planck Institute for Mathematics in the Sciences, 2008. 20. R. Holmes. Geometric Functional Analysis and its Applications. Springer-Verlag, New York, 1975. 21. B. N. Khoromskij, and I. Oseledets DMRG+QTT approach to high-dimensional quantum molecular dynamics. Preprint 69/2010 MPI MiS, Leipzig, 2010. 22. B. N. Khoromskij and C. Schwab. Tensor-structured galerkin approximation of parametric and stochastic elliptic pdes. Technical Report Research Report No. 2010-04, ETH, 2010..

(25) PGD for Nonlinear Convex Problems. 25. 23. T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455500, 2009. 24. T. G. Kolda. Orthogonal tensor decompositions. SIAM J. Matrix Analysis & Applications, 23(1):243255, 2001. 25. A. Kriegl and P.W. Michor. The Convenient Setting of Global Analysis Mathematical Surveys and Monographs Volume 53, American Mathematical Society, 1997. 26. P. Ladevèze. Nonlinear Computational Structural Mechanics - New Approaches and NonIncremental Methods of Calculation. Springer Verlag, 1999. 27. P. Ladevèze, J.-C. Passieux and D. Neron. The LATIN multiscale computational method and the Proper Generalized Decomposition Computer Methods in Applied Mechanics and Engineering, 199:1287-1296, 2010. 28. H. G. Matthies and E. Zander. Sparse representations in stochastic mechanics. In Manolis Papadrakakis, George Stefanou, and Vissarion Papadopoulos, editors, Computational Methods in Stochastic Dynamics, volume 22 of Computational Methods in Applied Sciences, pages 247265. Springer Netherlands, 2011. 29. T. Morrison Functional Analysis. An Introduction to Banach Space Theory. John Wiley and Sons, 2001 30. A. Nouy. A generalized spectral decomposition technique to solve a class of linear stochastic partial dierential equations. Computer Methods in Applied Mechanics and Engineering, 196(45-48):45214537, 2007. 31. A. Nouy. Generalized spectral decomposition method for solving stochastic nite element equations: invariant subspace problem and dedicated algorithms. Computer Methods in Applied Mechanics and Engineering, 197:47184736, 2008. 32. A. Nouy. Recent developments in spectral stochastic methods for the numerical solution of stochastic partial dierential equations Archives of Computational Methods in Engineering, 16:251-285, 2009. 33. A. Nouy. Proper Generalized Decompositions and separated representations for the numerical solution of high dimensional stochastic problems. Archives of Computational Methods in Engineering, 17:403434, 2010. 34. A. Nouy. A priori model reduction through Proper Generalized Decomposition for solving time-dependent partial dierential equations. Computer Methods in Applied Mechanics and Engineering, 199:1603-1626, 2010. 35. I. Oseledets. Compact matrix form for the d-dimensional tensor decomposition Submitted to SIAM J. Sci. Comp., available as preprint 2009-1 at IMN Ras Moscow, 2009. 36. R. A. Ryan. Intoduction to tensor products of Banach spaces. Springer, 2002. 37. V. Temlyakov Greedy Approximation Acta Numerica,17:235-409, 2008. 38. A. Uschmajew. Well-posedness of convex maximization problems on stiefel manifolds and orthogonal tensor product approximations. Numerische Mathematik, 115:309331 2010. 39. E. Zeidler. Nonlinear Functional Analysis and its Applications III. Variational Methods and Optimization. Springer-Verlag, 1985..

(26)