Proximal Point Methods for Multiobjective Optimization in Riemannian Manifolds

Transcrição

1 Universidade Federal de Goiás IME - Instituto de Matemática e Estatística Programa de Pós Graduação em Matemática Lucas Vidal de Meireles Proximal Point Methods for Multiobjective Optimization in Riemannian Manifolds Doctoral Thesis Funded by CAPES Goiânia 2019

2

3 Lucas Vidal de Meireles Proximal Point Methods for Multiobjective Optimization in Riemannian Manifolds Tese apresentada ao Programa de Pós-Graduação do Instituto de Matemática e Estatística da Universidade Federal de Goiás, como requisito parcial para obtenção do título de Doutor em Matemática. Área de concentração: Otimização Orientador: Prof. Dr. Glaydston de Carvalho Bento Goiânia

4 Ficha de identificação da obra elaborada pelo autor, através do Programa de Geração Automática do Sistema de Bibliotecas da UFG. Meireles, Lucas Vidal de Proximal Point Methods for Multiobjective Optimization in Riemannian Manifolds [manuscrito] / Lucas Vidal de Meireles f. Orientador: Prof. Dr. Glaydston de Carvalho Bento. Tese (Doutorado) - Universidade Federal de Goiás, Instituto de Matemática e Estatística (IME), Programa de Pós-Graduação em Matemática, Goiânia, Bibliografia. 1. Multiobjective Optimization. 2. Optimality Conditions. 3. Proximal Point Method. 4. Approximate Solution. 5. Riemannian Manifolds. I. Bento, Glaydston de Carvalho, orient. II. Título. CDU 517

5

6

7 Dedicado a: À minha Família 7

8 Agradecimentos Primeiramente gostaria de agradecer à CAPES pelo apoio financeiro, pois sem o auxílio seria inviável a realização do doutorado. Ao meu orientador Prof. Dr. Glaydston de Carvalho Bento, pelos ensinamentos e pela sua grande paciência, compreensão, incentivo, apoio, amizade e confiança com que pude contar na realização deste trabalho. Ao Prof. Dr. João Xavier da Cruz Neto, pelos ensinamentos (durante toda minha formação acadêmica) e pela sua grande paciência, compreensão, incentivo, apoio, amizade e confiança com que pude contar na realização deste trabalho. Aos professores Paulo Sérgio Marques dos Santos (que sempre acompanha minha formação fornecendo valiosas contruições, aprendizados, compreensão, puxões de orelhas), Orizon Pereira Ferreira, Luis Roman Lucambio Perez e Paulo Roberto Oliveira por terem aceito participar da banca de defesa desta tese de doutorado, pelo tempo que disponibilizaram à leitura da mesma e pelas consideráveis observações e sugestões. Aos professores do IME-UFG pelo apoio, em especial aos que integram o grupo de Otimização, e a todos os funcionários do IME-UFG. Aos meus amigos do Piauí, em especial Valdinês, Yuri, Gilson e Samara e amigos que acabei por conquistar em Goiânia, Pedro, Bruno, Fabrícia, Fabiana, Tatiana, e toda galerinha que sempre estiveram próximo durante o doutorado. E agradeço a todos que me ajudaram direta e (ou) indiretamente para a conclusão deste trabalho. Aos familiares, em especial um agradecimento ao amor da minha vida, Maria José Vidal Ramos, mãe te amo muito. Sou um filho super feliz por sempre está ao meu lado, nos momentos de muito choro e alguns de alegria, por sempre apoiar minhas decisões. Agradeço também ao meus dois pais, Regiomar Pinto e Manuel Vicente por todo apoio e incentivo. Não posso deixar de agradecer aos meus irmãos Iracema Vidal Ramos e Tiago Vidal Ramos, por todo apoio e carinho. 8

9 Abstract In this work, two different proximal-type methods are investigated in the Riemannian context, namely, an exact and an inexact version. Two strategies were used to analyze these methods. For the exact version, we used a direct approach by investigating the regularized problem, not considering any convexity assumption over the constraint sets, that determine the vectorial improvement steps, which replaces the classical approach via scalarization. To study the inexact version, a definition of the approximate Pareto efficient solution is introduced. For the convex case on Hadamard manifolds, full convergence of both methods to a weak Pareto optimal point is obtained. Keywords: Multiobjective Optimization, Optimality Conditions, Proximal Point Method, Approximate Solution, Riemannian Manifolds. 9

10 Resumo Neste trabalho, dois diferentes métodos tipo-proximal são investigados no contexto Riemanniano, uma versão exata e uma inexata. Duas estratégias foram usadas a fim de analizar estes métodos. Para a versão exata, usamos uma abordagem direta investigando o problema regularizado, não considerando qualquer hipótese de convexidade sobre os conjuntos de restrições, que fornece os passos de melhoria vetorial do subproblema proximal, o qual substitui a abordagem clássica via escalarização. Para estudar a versão inexata introduzimos uma definição de solução Pareto eficiente aproximada. No caso convexo, sobre variedades Hadamard, convergência total de ambos os métodos para um ponto Pareto fraco ótimo é obtido. Palavras-chave: Otimização Multiobjetivo, Condições de Otimalidade, Método do Ponto Proximal, Solução Aproximada, Variedades Riemanniana. 10

11 Contents 1 Introduction 2 2 Preliminaries Notation and terminology on Riemannian manifolds Nonsmooth analysis on Riemannian manifolds Multiobjective optimization Proximal Point Method for Locally Lipschitz Functions in Multiobjective Optimization of Hadamard Manifolds Proximal Point Method Convergence Analysis Full Convergence An Inexact Proximal Point Method for Multiobjective Optimization on Riemannian Manifolds Inexact Proximal Point Algorithm Convergence Analysis Full Convergence Future works and Conclusions Proximal Point Method for Multiobjective Problem with Vector-Valued Bregman Distance Computing the Center of Mass in Stiefel Manifolds via Proximal Point Method Conclusions

12 Chapter 1 Introduction In various applications, such as engineering, economy, statistics, industry, agriculture, and design problems, several objective functions should be simultaneously optimized; see, for instance, [33,38,65] and the references therein. This characterizes the so-called multiobjective (or multicriteria) optimization problem. Historically, multicriteria equilibrium first studied for economic applications, in the context of utility theory and welfare economics. On a differentiable manifold M, a multiobjective optimization problem can be formulated as follows: min F (x) = (F 1 (x),..., F m (x)) s.t. x C, where F : M R m is the objective function and C is a nonempty subset of M. During the last few decades, several authors have proposed extensions of the proximal point method for multiobjective problems. As far as we know, the first direct generalization, in the linear context, was presented in the seminal work of Bonnel et al. [18], which motivated others studies, such as that of Ceng and Yao [21], Chen et al. [24], Villacorta and Oliveira [61], Gregório and Oliveira [34], Rocha et al. [53], Choung [26], and Sousa Júnior et al. [13, 15], among others. To the best of the author s knowledge, a proximal-type method for solving the multiobjective problem in the Riemannian context was first proposed by Tang and Huang [60] (also see Bento et al. [14]). It is worth mentioning that in [18], [60], and [14], scalarization techniques were used; thus, the methods were limited to convex vector problems, either from the classical point of view or in terms of the Riemannian metric. It is known that each constraint set of the subproblems forces the algorithm to be a descent process. A motivation to consider this vector improving process, which is essential for justifying the process at a behavioral level, where a risk-averse agent agrees to change only if the change improves in every respect (all components of the vector), is given in Bento et al. [9]. In this approach, the convexity of the subproblems plays an important role. In this study, we focus on the particular case when the objective function is locally Lipschitz and C M is a nonempty and closed set. Locally Lipschitz functions have always been given special attention either in the Euclidean setting [39, 49, 55, 58] or in the Riemannian context [10, 12, 35, 48]. Recently, focusing on this relevant class of functions, Bento et al. [6], 2

13 studied a constrained multiobjective optimization problem and extended the proximal point method proposed by Bonnel et al. [17] in the finite dimensional multiobjective setting. Based on this, a new approach for convergence analysis of the method in which the scalarized problem was replaced by a necessary condition for weakly Pareto. In this context, based on [6], we aim to go much further. In this work, our goal is to generalize the proximal point method presented in [6] in the Riemannian setting. The key point to reach our goal was to obtain conditions of Fritz John type, Theorem 2.3.7, given in terms of the Mordukhovich (limiting) subdifferential, as the necessary conditions for weak Pareto optimal solutions (see Minami [46] for a similar result in the linear context), for which we explore the tools provided in Ledyaev [41]. This allowed us to replace the classic approach via scalarization by a purely vectorial approach without a convexity assumption on the constraint sets that determine the vector improvement steps because the objective function is locally Lipschitz (not necessarily convex). The partial convergence result has been ensured, i.e., that each accumulation point (if any) of any sequence generated by the proposed proximal method is a Pareto critical point. Moreover, under some additional assumptions, we show the full convergence of the generated sequence. In most practical cases, due to computational errors or stopping rules, the optimization algorithms for nonlinear programming yields only approximate solutions as opposed to exact solutions. The concept of approximate solution can be considered as an adequate compromise with a given prescribed error. Another reason that the study of approximate solutions is favored is that many real-world optimization problems are not completely and adequately modeled before the procedure for their solutions begins. In such circumstances, an approximate solution becomes more relevant than the exact solution. Hence, it is interesting to have a theoretical analysis of the notion of an approximate solution, for more discussion see [17, 43, 64, 66]. It is worth mentioning that in the middle of the nineteen-eighties, Loridan [43] derived some properties of ɛ-efficient points solution for vector minimization problems and used Ekeland s variational principle [30] to establish the ɛ-pareto optimality and ɛ-quasi Pareto optimality. Several authors have been interested in ɛ-optimal solutions for such problems, see, for example, [17, 27, 36, 64, 66], and references therein. Based on [43], we introduce a version of the definition of approximate Pareto efficient solutions on Riemannian manifolds, see Definition We also establish a necessary condition for this type of Pareto efficient solutions in our context, see Theorem 2.3.8, which is similar to [27, Theorem 3.7] for linear settings. For the exact case, we recovered Theorem already proposed. Furthermore, we discuss an inexact version of the proximal point algorithm, in which case the sequence generated in (4.1) is necessarily an approximate solution of the vectorial proximal optimization subproblem given by min F k (x) := F (x) + λ k 2 d2 (x, x k )ς k s.t. x Ω k, where Ω k = {x C F (x) F (x k )}. Generally, an additional hypothesis has to be made for the error term for inexact algorithms. In some cases, it required that this term is summable [21, 54, 63], in others, it is sufficient to demand that the exogenous error term 3

14 is constant [14, 18]. For our analysis, we choose the first approach. Such a condition is essential in Theorem for demonstrating convergence to a weakly Pareto point. It is worth pointing out that under mild assumptions the accumulation points, if any, of each generated sequence, are Pareto critical points, see Theorem Furthermore, we also show full convergence of any generated sequence to a weak Pareto point, see Theorem As a result of this work, we have written two scientific papers, one of which is already published, [4], and the other is under review, [5]. Both of the aforementioned papers have been submitted to important indexed journals of international circulation in my area of research. In addition, we have two scientific articles in preparation, which are discussed in the last chapter of this thesis. This thesis is organized as follows. In Chapter 2, definitions and auxiliary results about Riemannian geometry are presented as well as nonsmooth analysis in this context. Moreover, the nonsmooth multiobjective optimization problem, the necessary conditions for weak Pareto optimal solutions are introduced. Further, we formulate a definition of the approximate Pareto solution and optimality conditions for characterizing these solutions for our problem proposal. In Chapter 3, A Riemannian proximal point algorithm for obtaining the solution of a multiobjective optimization problem is stated, the well-definedness of the generated sequence is established when the objective function is locally Lipschitz, and partial convergence is proved, i.e., every accumulation point of the generated sequence, if any, is a Pareto critical point of the objective function. Assuming that the objective function is convex, and the manifold is Hadamard, a full convergence result is obtained. In Chapter 4, We present and analyze an inexact version of proximal point algorithm, its well-definedness, and partial and full convergence is demonstrated. Finally, in Chapter 5, we present proposals for future research and conclusions. 4

15 Chapter 2 Preliminaries In this chapter, we introduce some notations about Riemannian geometry and some results related to nonsmooth analysis in this setting and multiobjective optimization, which will be used throughout the Chapters 3 and Notation and terminology on Riemannian manifolds In this section, we introduce some notations about Riemannian geometry, which can be found in any introductory book on Riemannian geometry, such as in Sakai [56], Do Carmo [20]. From now on, let M be a n-dimensional connected manifold. We denote by T x M the n-dimensional tangent space of M at x, and by T M = x M T x M tangent bundle of M. Suppose that M be endowed with a Riemannian metric, is a Riemannian manifold. Recall that the metric can be used to define the length of piecewise smooth curves γ : [a, b] M joining x to y, i.e., such that γ(a) = x and γ(b) = y, by l(γ) = b a γ (t) dt, and, moreover, by minimizing this functional length over the set of all such curves, we obtain a Riemannian distance d(x, y), which induces the original topology on M, and, given a not empty set U M, we can define the distance function associated to U, which is given by: M x d U (x) = inf{d(x, y) : y U}. A Riemannian manifold is complete iff geodesics are defined for any values of t. Hopf-Rinow Theorem asserts that, if this is the case then any pair of points, say p and q, in M can be joined by a (not necessarily unique) minimal geodesic segment. Moreover, (M, d) is a complete metric space and bounded and closed subsets are compact. From the completeness of the Riemannian manifold M, for each p M, the exponential map exp p : T p M M can be defined by exp p v = γ v (1, p), where γ v (, p) : R : M denotes the unique geodesic such that γ v (0, p) = p and γ v(0, p) = v. 5

16 We denote by R the curvature tensor defined by R(X, Y ) = X Y Z Y X Z [X,Y ] Z with X, Y, Z X (M) where [X, Y ] = Y X XY Moreover, the sectional curvature with respect to X and Y is given K(X, Y ) = R(X, Y )Y, X /( X 2 Y 2 X, Y 2 ) where X = X, Y 1/2. If K(X, Y ) 0 for all X and Y, then M is called a Riemannian manifold of nonpositive curvature and we use the short notation K 0. Theorem Let M be a complete, n-dimensional, simply connected Riemannian manifold with nonpositive sectional curvature. Then M is diffeomorphic to the Euclidean space R n. More precisely, at any point x M, the exponential map exp x is a diffeomorphism. Proof. See [20, Lemma 3.2 p. 149] or [56, Theorem 4.1 p. 221]. A complete simply connected Riemannian manifold of nonpositive sectional curvature is called a Hadamard manifold. Thus Theorem states that if M is a Hadamard manifold then M has the same topology and differential structure of the Euclidean space R n. Furthermore, there are known some similar geometrical properties of the Euclidean space R n such as, given two points there exists an unique geodesic segment that joins them. Hence, if M is a Hadamard manifold the lenght of the geodesic segment γ joining x to y is equals d(x, y). Moreover, the exponential map exp x v = γ v (1, x) is a diffeomorphism. Let y M and exp 1 y : M T y M be the inverse of the exponential map. Note that d(y, x) = exp 1 x y, the function d 2 y : M R defined by d 2 y(x) = d 2 (y, x) is C and grad d 2 y(x) = 2 exp 1 y x. A geodesic triangle (x 1, x 2, x 3 ) of a Riemannian manifold is the set consisting of three distinct points x 1, x 2, x 3 called the vertices and three minimizing geodesic segments γ i+1 joining x i+1 to x i+2 called the sides, where i = 0, 1, 2. Theorem Let M be a Hadamard manifold (x 1, x 2, x 3 ) a geodesic triangle. Denote by γ i+1 : [0, l i+1 ] M geodesic segments joining x i+1 to x i+2 and consider l i = L(γ i ), θ i = (γ i+1(0), γ i(l i )) where i 1, 2, 3(mod 3). Then θ 1 + θ 2 + θ 3 π; As l i = d(x i+1, x i+2 ), we can rewrite (2.1) as follows l 2 i+1 + l 2 i+2 2l i+1 l i+2 cos θ i l 2 i. (2.1) d 2 (x i, x i+2 ) + d 2 (x i, x i+1 ) d(x i, x i+2 )d(x i, x i+1 ) cos θ i d 2 (x i+1, x i+2 ), where θ i = (exp 1 x i x i+1, exp 1 x i x i+2 ). Since d(x i, x i+2 )d(x i, x i+1 ) cos θ i = exp 1 x i x i+1, exp 1 x i x i+2, then we have d 2 (x i, x i+2 ) + d 2 (x i, x i+1 ) exp 1 x i x i+1, exp 1 x i x i+2 d 2 (x i+1, x i+2 ). (2.2) Proof. See [56, Proposition 4.5 p. 223]. 6

17 It is known that (M, d) is a complete metric space. The next two results are valid in the present context and will be useful in the following this work. Firstly we introduce the Lipschitz notion, a function f on a Riemannian manifold to be Lipschitz with rank L when, for any x, y M, f(x) f(y) Ld(x, y). We say that f is locally Lipschitz at x M provided that f is Lipschitz in a neighborhood of x. Theorem Let C M be a not empty set and f : M R be a Lipschitz function on M with constant L. If x is a minimizer for the constrained minimization problem, min f(x), x C, (2.3) and τ L, then x is also a minimizer for the unconstrained minimization problem min{f(x) + τd C (x)}, x M. (2.4) If τ > L and C is a closed set, then the converse assertion is also true: any minimizer x for the unconstrained problem (2.4) is also a minimizer for the constrained problem (2.3). In particular, x C. Proof. See [62, Theorem 3.2.1]. We denote the extended real line by R = R {+ }. For a function f : M R the domain of f is defined by dom(f) = {x M : f(x) < + }. We say f is proper when dom(f). Lemma (Ekeland s Variational Principle) Let f : M R be a proper lower semicontinuous function that is bounded above. If x dom(f) and δ > 0 are such that then there exists z dom(f) such that (a) d(z, x) δ; f( x) inf{f(x) : x M} + δ, (b) f(x) + δd(x, z) > f(z), whenever x z. Proof. See Ekeland [30, 31]. 2.2 Nonsmooth analysis on Riemannian manifolds The next, is provided some elements of nonsmooth analysis on Riemannian manifolds. 7

18 Definition Let f : M R be a proper and lower semicontinuous function. Fréchet subdifferential of f at x dom(f) is defined by The f(x) := {dh(x) : h C 1 (M) and f h attains a local minimum at x}, where dh(x), differential of h at x, is a element in the dual space of T x M. The (limiting) subdifferential and singular subdifferential of f at x M is defined by and f(x) := {v T x M : (x k, v k ) Gr( f), (x k, v k ) (x, v), f(x k ) f(x)}, f(x) := {v T x M : (x k, v k ) Gr( f), (x k, t k v k ) (x, v), f(x k ) f(x) and t k 0 + }, respectively, where Gr( f) := {(y, u) T M : u f(y)}. It follows directly from Definition that f(x) f(x). It should further be noted that f(x) may be empty; however, if f attains a local minimum at x, then 0 f(x). A necessary (but not sufficient) condition for x M to be a minimizer of f is that 0 f(x). We present below some properties of this subdifferentials. Proposition Let M be a Riemannian manifold. If f : M R is lower semicontinuous and proper then {x dom(f) : f(x) } is dense in dom(f). Proof. See [2, Proposition 4.17]. Theorem Let f 1,..., f m : M R be lower semicontinuous functions. Then, for any x M, either ( m f i )(x) f i (x) and or there exist vi ( m f i )(x) f i (x), f i (x), i = 1,..., m not all zero such that 0 = vi. Proof. See [41, Theorem 4.13]. 8

19 Theorem Let f i : M R, i = 1, 2,..., m, be lower semicontinuous functions. Then either: there exist vi f i (x), i = 1, 2,..., m, not all zero such that 0 = vi, or there exist µ i 0 with m µ i = 1 such that max(f 1,..., f m )(x) µ i (f i )(x) + (f i )(x) k {i µ i 0} k {i µ i =0} and max(f 1,..., f m )(x) (f i )(x). Proof. See [41, Theorem 4.16]. Theorem Let M be a locally convex Riemannian manifold. Let f : M R be a lower semicontinuous function. Then the following are equivalent: (i) f is locally Lipschitz at x M; (ii) f is bounded in a neighborhood of x; (iii) f is bounded in a neighborhood of x; (iv) f(x) is bounded; (v) f(x) = {0}. Proof. See [41, Corollary 5.3]. The next two results establish that under locally Lipschitz property the limiting subdifferential is not empty. To the best of the authors knowledge there is not prove explicitly in the literature of proof such result, and for a question of completeness, we present your proofs. Proposition Let f : M R be a locally Lipschitz function and consider a bounded sequence (x k ) k dom( f). If (v k ) k is a sequence such that v k f(x k ), for each k N, then (v k ) k is also bounded. Proof. Taking a subsequence if necessary, we can assume that (x k ) k converges to x. Since f is locally Lipschitz, there is a neighborhood U k of x k and L k > 0 such that f(p) f(q) L k d(p, q) whenever p, q U k. By Theorem 2.2.5, v k L k. Define A := {x k } {lim x k }. Note that A k N U k (i.e., k N U k is a open cover for A) and A is a compact set (this follows from the Hopf-Rinow theorem). Hence, there exists a finite set I := {r 1,..., r s } N such that A I U k. Therefore, there exists L := max r I L r such that v k L, for all k N, which concludes the proof. 9

20 Proposition If f : M R is a locally Lipschitz function, then the limiting subdifferential is not empty. Proof. It is known that the domain of the Fréchet subdifferential of a lower semicontinuous function f defined on a Riemannian manifold M is dense in dom(f), by Proposition Thus, because f(x) f(x), we can deduce that f is nonempty in a dense subset of dom(f). Let us now prove that dom( f) = dom(f). Given an arbitrary point x dom(f), using again Proposition there exists a sequence (x k ) k dom( f) converging to x. Thus, there is some v k f(x k ), for all k. Since f is locally Lipschitz and (x k ) k converges to x, it follows that f(x k ) converges to f(x). Besides, Proposition implies that there exists a subsequence of (v k ) k converging to some v. By Definition 2.2.1, v f(x). The desired result follows from the arbitrariness of x. For a general closed subset C M and a point x C, the Fréchet normal cone and the (limiting) normal cone of C at x were defined in [41] by N C (x) = δ C (x); N C (x) = δ C (x), respectively, where δ C : M R is the indicator function of C. As remarked by Li et al. [42], when C is convex, i.e., for any x, y C there exists a minimizing geodesic of M connecting x, y that is contained in C, then N C (x) = N C (x). Proposition Let M be a Riemannian manifold and let C be a closed subset of M. Then, for any x C, ) N C (x) = cone ( dc (x), ) where cone ( dc (x) = {ty y d C (x), t 0}. Proof. See [41, Proposition 3.5]. The next result is the joining of previous results and we present your proof by completeness. Proposition Let C M be a nonempty and closed set. If x C, then where B x := {ξ T x M : ξ 1}. d C (x) B x N C (x), (2.5) Proof. Take x C. It is known that the distance function, d C (x), is locally Lipschitz with constant L = 1. From Theorem 2.2.5, we have that a Lipschitz function f of rank L near x satisfies ξ L for every ξ f(x). Thus, d C (x) B x. On the other hand, it follows from Proposition that N C (x) = cone d C (x). This implies that d C (x) N C (x) and the proof is completed. In the particular case that C is a convex set, it follows from [42, Lemma 5.2 and Theorem 5.3] that the expression in (2.5) takes the form d C (x) = B x N C (x). (2.6) 10

21 2.3 Multiobjective optimization To conclude this chapter we present some elements and results, in particular, optimality condition, of multiobjective programming useful in the later chapters. Let I := {1,..., m}, R m + = {y R m : y i 0, i I} and R m ++ = {y R m : y i > 0, i I}. For y, z R m, y z (y z) means that z y R m + (z y R m ++). A sequence (x k ) in the space R m is said to be decreasing if x l x k whenever k < l. A point x is called the infimum of (x k ) when x x k, for all k 0 and there is no x such that x x and x x k, for all k 0. In this case, we denote x = inf k x k. Definition Let A R m and y R m. (i) The set A (y R m +) is called a section of A at y. (ii) The set A is said to be R m +-complete if any decreasing sequence of A is bounded by an element of A, i.e., whenever (x k ) A is decreasing sequence, then there exists x A such that x x k for all k 0. Lemma If A R m closed and lower bounded set, then A is R m +-complete. Proof. See [44, Lemma 3.5]. Now, motivated by the algorithms studied in the next two chapter, let h i, i {1, 2,..., m} =: I, g j, j {1, 2,..., m} =: J, be real-valued functions on a finite dimensional Riemannian manifold M and Ω := {x C g j (x) 0, j = 1, 2,..., m}. We consider the problem of finding a weakly Pareto solution of H := (h 1,..., h m ) in Ω, i.e., a point x Ω such that does not exist another decision point x Ω with h i (x) < h i ( x) for all i = 1,..., m. We denote this constrained problem as and the set of all its weakly Pareto solutions by min H(x) s.t. x Ω, (2.7) S ω (H, Ω) := argmin ω {H(x) x Ω}. Next, we define a notion of approximate Pareto solution Definition Let ɛ R m +. A point x Ω is an ɛ-quasi-weakly Pareto solution of the problem in (2.7) if there does not exist another point x Ω such that where d is Riemannian distance. h i (x) + ɛ i d(x, x ) < h i (x ), i = 1, 2,..., m, We denote the set of ɛ-quasi-weakly Pareto solutions H in Ω by S ω ɛq(h, Ω). Remark

22 i) Note that Definition is a natural extension of that introduced in the linear setting by Loridan in [43]. See also Chuong et al. [27] and references therein; ii) It is easy to see that, even in the linear context, in general, S ω (H, Ω) S ω ɛq(h, Ω) and S ω (H, Ω) = S ω ɛq(h, Ω) for ɛ = 0. Theorem Consider the multiobjective problem (2.7). Then, argmin{h(x) x Ω} is not empty iff H(Ω) has a section R m +-complete. Proof. See [44, Theorem 3.4]. Necessary conditions for weakly Pareto and ɛ-quasi-weakly Pareto solutions are provided, which will play important roles in the subsequent analysis of the proposed methods in next chapters. From now on, let us assume that h i, i I, g j, j J of problem (2.7) are locally Lipschitz functions Lemma If x is a weak Pareto solution of H in Ω, then for any δ > 0 and x C, φ δ (x) = max,j J {h i(x) h i ( x) + δ, g j (x)} > 0. Proof. Suppose the contrary, that is, for some δ > 0 and x C, φ δ (x) 0. Then, we have g j (x) 0, for every j J, and so x Ω. On the other hand, h i (x) < h i (x) + δ h i ( x), for every i I, which contradicts the weakly Pareto-optimality of x. Hence, we obtain the conclusion of Lemma Theorem Let h i, g j : C R, i I, j J, be locally Lipschitz functions, and C M be a not empty and closed set. If x S ω (H, Ω), then there exist real numbers such that ν i 0, i I, µ j 0, j J, τ > 0, (a) ν i + j J µ j = 1; (b) 0 ν i h i ( x) + j J µ j g j ( x) + τ d C ( x). Proof. Let {δ l } R ++ be a sequence converging to 0, and for each l, the following function is defined: φ l (x) = max,j J {h i(x) h i ( x) + δ l, g j (x)}. 12

23 It should be noted that φ l ( x) = δ l and φ l (x) > 0 for all x C, by Lemma Hence, φ l ( x) = δ l inf x C φ l(x) + δ l. As h i, i I, g j, j J, are locally Lipschitz, φ l is locally Lipschitz continuous, since the maximum of locally Lipschitz functions is also a locally Lispchitz function. By Ekeland s Variational Principle, there exists x l C such that and d( x, x l ) δ l, (2.8) φ l (x) + δ l d(x, x l ) > φ l (x l ), wherenever x x l, that is, x l is a global minimal solution for ψ l ( ) = φ l ( ) + δ l d(, x l ) over C. Let L be the local Lipschitz constant of ψ l ( ) at x l. From Theorem 2.1.3, x l is a local minimal of the function ψ l ( ) + τd C ( ) when τ L. By the necessary condition whereby x l is a minimizer for this function and by the sum rule for limiting subdifferentials, see Theorem 2.2.3, we have 0 (ψ l ( ) + τd C ( )) (x l ) φ l (x l ) + δ l (d(, x l ))(x l ) + τ d C (x l ). This implies that there exist z l max{h i (x) h i ( x) + δ l, g j (x)} x=xl, for each l N, satisfying 0 z l + δ l (d(, x l ))(x l ) + τ d C (x l ). (2.9) Moreover, joining the maximum rule for limiting subdifferentials Theorem with Theorem item (v), there exist non-negative real numbers νi, l µ l j such that νi l + µ l j = 1, (2.10) j J z l {i ν l i 0} ν l i h i (x l ) + {j µ l j 0} µ l j g j (x l ). (2.11) Therefore, combining (2.9) and (2.11) yields 0 νi h l i (x l ) + µ l j g j (x l ) + δl (d(, x l ))(x l ) + τ d C (x l ). {i ν i 0} l {j µ l j 0} Hence, there are sequences (u l i) l, (vj) l l, ( w l ) l, (w l ) l in T xl M with u l i h i (x l ), vj l g j (x l ), w l δ l (d(, x l ))(x l ), w l τ d C (x l ) such that 0 = νiu l l i + µ l jvj l + w l + w l. (2.12) {i ν i 0} l {j µ l j 0} 13

24 It should be noted that in view of (2.10), (ν l i) l and (µ l j) l are bounded. Moreover, as h i, g j and d(, x l ) are locally Lipschitz, by Theorem the sequences (u l i) l, (v l j) l, and ( w l ) l are also bounded. In particular, from (2.12), the sequence (w l ) l is also bounded. Making a refinement, if necessary, it may be assumed that the sequences (ν l i) l, (µ l j) l, (u l i) l, (v l j) l, ( w l ) l, and (w l ) l converge to ν i, µ j, u i, v j, w, and w, respectively. It is easy to see that ν i 0, µ j 0 which proves the first part and (a). and ν i + µ j = 1, j J For the proof of the second part, as (x l ) converges to x (this follows from (2.8)), it may be assumed that (x l, w l ) T U x, where U x is a neighborhood of x such that T U x U x R n. Letting l tend to infinity in (2.12), we obtain 0 = ν i u i + j J µ j v j + w, where is addition the indices j which µ j = 0 and ν j = 0. By the upper semicontinuity of the limiting subdifferential, we have u l i h i (x l ) with u l i u i implies that u i h i ( x) vi l g j (x l ) with vj l v j implies that v j g j ( x) w l τ d C (x l ) with w l w implies that w τ d C ( x). It follows that 0 ν i h i ( x) + j J µ j g j ( x) + τ d C ( x), and the desired result is proved. Define x M φ(x) := max{h i (x) h i (x ) + ɛ i d(x, x ), g j (x) }. (2.13) j J The next result presents a necessary condition for ɛ-quasi-(weakly) Pareto solutions associated to the problem in (2.7). For a version of this result in the linear setting, see [27]. Theorem Let x Sɛq(H, ω Ω). Then, there exist τ > 0 and λ i 0, i I, µ j 0, j J with m λ i + m j=1 µ m = 1, such that 0 λ i h i (x ) + j=1 µ j g j (x ) + 14 λ i ɛ i B x + τ d C (x ). (2.14)

25 Proof. From the definition of φ in (2.13), it is easy to see that φ(x ) = 0 and φ(x) 0, for all x C and, hence, φ(x ) = inf x C φ(x). Since h i, i {1, 2,..., m}, g j, j {1, 2,..., m} are locally Lipschitz functions, then φ( ) is also a locally Lipschitz function. Let us suppose that L is a Lipschitz constant of φ( ) at x. From Theorem follows that x is a local minimal for φ( ) + τd C ( ), whenever τ L. From the first-order optimality condition and using the sum rule for the limiting subdifferential, see Theorem 2.2.3, we obtain 0 φ(x ) + τ d C (x ). (2.15) By the definition of φ in (2.13), Theorem and Theorem applied to φ implies that there exist non-negative real numbers λ i, µ j with i {1, 2,..., m} and j {1, 2,..., m} such that m λ i + m j=1 µ j = 1 and φ(x ) λ i (h i ( ) + ɛ i d(, x )) (x ) + µ j g j (x ). {i λ i 0} Hence, using again Theorem 2.2.3, we get φ(x ) λ i h i ( ) + {i λ i 0} {i λ i 0} λ i ɛ i B x + {j µ j 0} {j µ j 0} µ j g j (x ), (2.16) and the desired result follows by combining (2.15) with (2.16). Remark From the last result is possible retrieves the necessary condition for weakly Pareto solutions, Theorem 2.3.7, which was introduced in the previous theorem. In the sequel, we define convex function and Pareto critical point in the framework. Definition Let F : M R m and C M a not empty and convex set. F is said to be convex on C iff for every x, y C and γ : [0, 1] M, the geodesic segment joining x and y, the following holds: Remark Note that: F (γ(t)) (1 t)f (x) + tf (y), t [0, 1]. (2.17) 1. When m = 1, the classic definition of convexity is recovered; 2. When C = M, the convexity notion presented above retrieves the definition introduced in [11]; 3. The expression (2.17) means every coordinate function of F is convex. In this case, the limiting subdifferential coincide with subdifferential of convex analysis. 15

26 Definition Let Ω M be a not empty, closed and convex set. A point x Ω is said to be Pareto critical of H in Ω iff for any y inj( x) Ω, there are an index i {1,..., m} and u h i ( x) such that u, exp 1 x y 0, where inj( x) is the injectivity radius of x, i.e., the domain where the inverse of exponential map in x is well defined. Remark If x is not a Pareto critical point of H in Ω, then there exists y Ω such that for all i {1, 2,..., m} and u i h i (ˆx), u i, exp 1 x y < 0. Note that, when Ω = M, the Pareto critical point notion presented above retrieves the Pareto critical point definition introduced in [6]. Similarly to what has been observed in [6], it is possible to verify that criticality is a necessary but not sufficient condition for optimality and the equivalence holds when F is convex (see next chapter). From now on, always that for appearing the inverse of exponential map stays implicit a injectivity domain. Note that, when M is a Hadamard manifold such domain is all M. 16

27 Chapter 3 Proximal Point Method for Locally Lipschitz Functions in Multiobjective Optimization of Hadamard Manifolds In this chapter, a proximal point algorithm for nonsmooth multiobjective optimization in the Riemannian context is proposed. Through this condition of optimality, introduced in chapter 2, has been possible to replace the classic approach, via scalarization, by a purely vectorial. Our main convergence result ensures that each accumulation point (if any) of any sequence generated by the method is a Pareto critical point. Moreover, when the problem is convex on a Hadamard manifold, full convergence of the method for a weak Pareto solution is obtained. It is worth noticing that the present chapter generalizes some results given in [7]. Throughout this section, it is assumed that C M is a nonempty, closed, and convex set. Consider the following multiobjective programming problem min F (x) = (F 1 (x),..., F m (x)) s.t. x C, (3.1) where F i : C R are locally Lipschitz functions, i = 1, 2,..., m, and m 2. We now make the following assumption on the map F : H1 : F 0. There is no loss of generality in considering the assumption H1 since the vector functions F ( ) and e F ( ) := (e F 1( ),..., e Fm( ) ) have the same set of Pareto critical points, where e α denotes the exponential map valued at α R. This fact was first observed by Huang and Yang in [37] and, in the Riemannian context, it can be verified in a similar way. 3.1 Proximal Point Method The proximal point algorithm, for obtaining a Pareto critical point of F in C, is now considered. Throughout the rest of this chapter, let (λ k ) k be a sequence of positive real 17

28 numbers such that 0 < a λ k b and let (ς k ) R m ++ be a sequence such that ς k = 1 for all k 0. The method generates a sequence (x k ) k C as follows: Algorthm 1 Initialization: Choose x 0 C. Stopping rule: Given x k, if x k is a Pareto critical point, then STOP. Iterative step: Take, as next iterate, x k+1 C such that x k+1 argmin ω {F (x) + λ k 2 d2 (x, x k )ς k x Ω k }, (3.2) where Ω k = {x C F (x) F (x k )}. Note that when m = 1 the Algorithm 1 coincide with the classic proximal method point. It is now proved that Algorithm 1 is well defined. Proposition Algorithm 1 is well defined. Proof. The starting point x 0 C is chosen in the initialization step. Assuming that the algorithm has reached iteration k, it is next shown that the (k + 1)-th iteration exists. Let F k (x) := F (x) + λ k 2 d 2 (x, x k )ς k. It should be noted that x k Ω k, which implies that F k (Ω k ) is not empty. It is straightforward to verify that F k (x) 0, because is a sum of functions with this property, and F k (Ω k ) is closed. Indeed, observe that F k is coercive function, i.e., lim d(x,x k ) F k (x) = +. Now, let (y l ) be a sequence in F k (Ω k ) such that y l converges to y. Since y l F k (Ω k ) there exist z l Ω k satisfying y l = F k (z l ) for any l. If (z l ) l N is unbounded then there is a subsequence (z l j ) j of (z l ) l N such that d(z l j, x k ) goes to infinity as j goes to infinity. As F k is coercive function it has been (F k (z l j )) +, as j goes to infinity, on the other hand F k (z l ) y because y l = F k (z l ) and y l converges to y that is contradiction. Hence, (z l ) l N is a bounded sequence. Assume that z l converges to z, take subsequence if necessary, by Hopf-Rinow Theorem, then by closeness of Ω k is follows that z Ω k. Applying F k and using uniqueness of the limit it holds y F k (Ω k ). Thus, F k (Ω k ) is closed. As the R m + has the property that all sequences decreasing, lower bound converges to its infimum, it follows from Lemma that F k (Ω k ) is R m +-complete. Thus, by Theorem 2.3.5, the set argmin ω {F k (x) x Ω k } is not empty. 18

29 3.2 Convergence Analysis The next result follows immediately from Theorem and plays an important role in convergence analysis. Proposition For all k N, there exist α k, β k R m +, u k i, w k T x km, and τ k R ++ such that (αi k + βi k )u k i λ k 1 αi k ς k 1 i exp 1 x k 1 + τ x k k w k = 0, (3.3) where w k d C (x k ) B x k N C (x k ), u k i F i (x k ) and (αk i + β k i ) = 1, k N. Proof. For every k, consider the functions G k 1 (x) := F (x) F (x k 1 ) and F k 1 (x) := F (x) + λ k 1 2 d2 (x, x k 1 )ς k 1. As F and d 2 (, x k 1 ) are locally Lipschitz, the coordinate functions (G k 1 ) i ( ) = F i ( ) F i (x k 1 ) and (F k 1 ) i ( ) = F i ( ) + λ k 1 d 2 (, x k 1 )ς k 1 2 i, i I, of G k 1 and F k 1, respectively, are also locally Lipschitz. Hence, since x k is weak Pareto solution for min F k 1 (x) s.t. G k 1 (x) 0, by Theorem 2.3.7, there exist real non-negative numbers α k i, β k i, with i in I, and a positive number τ k for every k such that (αk i + β k i ) = 1 and 0 (α k i + β k i ) F i (x k ) λ k 1 αi k ς k 1 i exp 1 x k 1 + τ x k k d C (x k ). Thus, there exist u k i F i (x k ) and w k d C (x k ) satisfying that 0 = (αi k + βi k )u k i λ k 1 αi k ς k 1 i exp 1 x k 1 + τ x k k w k, which proves the desired result. As a consequence of Proposition 3.2.1, the following two corollaries are obtained: Corollary If there exists k N such that x k+1 = x k, then x k is a Pareto critical point of F. Proof. Let it be assumed that x k+1 = x k for any k N. By Proposition 3.2.1, there exist α k+1, β k+1 R m +, u k+1 i, w k+1 T x k+1m, and τ k+1 R ++ satisfying (α k+1 i + β k+1 i )u k+1 i + τ k+1 w k+1 = 0, 19

30 whence (αk+1 i + β k+1 i )u k+1 i N C (x k+1 ). If x k is not a Pareto critical point of F, there is y C such that u k+1 i, exp 1 y < 0. Therefore, as x k+1 (αk+1 i + β k+1 i ) = 1, we obtain contradicting (α k+1 i + β k+1 i )u k+1 i, exp 1 x k+1 y < 0, (α k+1 i + β k+1 i )u k+1 i N C (x k ). Hence, x k is a Pareto critical point of F. Corollary If there exists k 0 N such that α k 0 = 0. Then, x k 0 is a Pareto critical point of F. Proof. If there exists k 0 N such that α k 0 = 0, then by (3.3), we have β k 0 i u k 0 i + τ k0 w k 0 = 0, where w k 0 B x k 0 N C (x k 0 ), u k 0 i F i (x k 0 ), and βk 0 i = 1 with β k 0 R m +. Suppose that x k 0 is not a Pareto critical point of F. Then, there exists y C such that for all i I and u k 0 i F i (x k 0 ), u k 0 i, exp 1 y < 0. As x k 0 βk 0 R m + and βk 0 i = 1, it holds β k 0 i u k 0 i which contradicts the fact that βk 0 i u k 0 i of F., exp 1 x k 0 y < 0, N C (x k 0 ). Thus, x k 0 is a Pareto critical point Therefore, if Algorithm 1 terminates after a finite number of iterations, it terminates at a Pareto critical point. This leads to the assumption that (x k ) k is an infinite sequence for convergence analysis; hence, x k+1 x k and α k 0 for all k N, in view of Corollaries and 3.2.3, respectively. In the sequel, our goal is to prove the following theorem: Theorem Let λ k and ς k be as above with 0 < c ςi k for all k N and i I. Then, every accumulation point of (x k ), if any, is a Pareto critical point of F. Proof. It follows immediately from the definition of x k that for every k N, there exists i := i(k) N such that F i (x k ) + λ k 1 2 d2 (x k, x k 1 )ς k 1 i F i (x k 1 ). Therefore, using the lower boundedness assumption of the sequences (λ k ) k and (ς k ) k, after some algebra we have ac 2 d2 (x k, x k 1 ) F i (x k 1 ) F i (x k ) F (x k 1 ) F (x k ) (3.4) 20

31 Since (F (x k )) k is non-increasing and F 0, the right-hand side of (3.4) converges to 0 as k +. Hence, d(x k, x k 1 ) 0 as k +. Then, exp 1 x k x k 1 converges to 0. Let now x be an accumulation point of (x k ) k. Then, there exists a subset K of N such that the sequence (x k ) k K converges to x. Applying Proposition for the sequence (x k ) k K, we have that there exist sequences (u k i ) F i (x k ), i I, k K (α k ) k K, (β k ) k K R m +, (w k ) k K T x km, and (τ k ) k K R ++ satisfying (αi k + βi k )u k i λ k 1 αi k ς k 1 i exp 1 x k 1 + τ x k k w k = 0, (3.5) where w k B x k N C (x k ) and (αk i + β k i ) = 1. As (x k ) k K converges to x, (x k ) is bounded. Since F is locally Lipschitz, it follows from the conditions above that the sequences (u k i ), (α k ), (β k ), and (w k ) are bounded. Thus, (3.5) implies that (τ k ) is also bounded. In this case, it can be assumed without loss of generality that (u k i ), (α k ), (β k ), (w k ), and (τ k ) (or suitable subsequences thereof) converge to u i, α, β, w, and τ, respectively. Moreover, the sequence (λ k 1 α k, ς k 1 ) is bounded; thus, by (3.5) we have (α i + β i )u i + τw = 0 in the limit. Due u i F i (x) and w N C (x), it follows from the previous identity that (α i + β i )u i N C (x). (3.6) Let it be assumed toward a contradiction that x is not Pareto critical of F. Then, there exists y C such that for all i I, u i, exp 1 x y < 0. Therefore, (α i + β i )u i, expx 1 y < 0 because α, β R m + and, besides, (α i + β i ) = 1. This contradicts (4.6), and the desired result is proved. 3.3 Full Convergence In order to prove the full convergence of the Algorithm 1, Theorem 3.3.4, we need some preliminaries and assumptions. Hereafter, we assume that F is convex, M is Hadamard manifold. 21

32 Next result establishes the relationship between weak Pareto solution and Pareto critical point mentioned in Remark Proposition Let F : M R m be a convex map. A point x C is a Pareto critical point of F in C if and only if it is a weak Pareto solution of F in C. Proof. It is first assumed that x is a Pareto critical point of F in C. Let it now be assumed toward a contradiction that x is not a weak Pareto solution of F in C. Then, there exists x C such that F i ( x) < F i (x), i = 1,..., m. Let γ : [0, 1] M, γ(t) = exp x (t exp 1 x x), be a geodesic segment joining x to x. Since C is convex γ([0, 1]) C and as F is convex, for all i I and u i F i (x), it holds that u i, exp 1 x x F i ( x) F i (x) < 0. However, this contradicts the fact that x is a Pareto critical point of F in C; thus, the first part of the proof is completed. Let us now to assume that x is a weak Pareto solution of F and that x is not a Pareto critical point of F in C. Then, there exists y C such that for all i I and u i F i (x), u i, exp 1 x y < 0. Hence, F i (x; exp 1 x y) = max u i, exp 1 u i x y < 0, i = 0,..., m. F i (x) In this case, for each i I and arbitrary u i F i (x), there exists δ i > 0 such that F i (exp x (t exp 1 x y)) F i (x) + t u i, exp 1 x y, t (0, δ i ). Due arbitrariness of i and u i, exp 1 x y < 0, we obtain F (exp x (t exp 1 x y)) F (x), t (0, δ), δ = min 1 i m δ i, contradicting the fact that x is a weak Pareto solution of F, and the proof is completed. The following assumption on the map F and the initial iterate x 0 is now made, which is also made in various studies on proximal algorithms (e.g., Bonnel et al. [18] and Ceng and Yao [21]). H2 : The set (F (x 0 ) R m +) F (C) is R m +-complete, that is, for all sequences {a k } C, with a 0 = x 0, such that F (a k+1 ) F (a k ) for all k N, there exists a C such that F (a) F (a k ) for all k N. Let U := {x C : F (x) F (x k ), k N}. (3.7) By H2, U is not empty and U Ω k for any k N. 22

33 Lemma Let M be a Hadamard manifold. It is assumed that F : M R m is convex and U is as in (3.7). Then, for all x U, holds: d 2 (x k, x) d 2 (x k+1, x k ) + d 2 (x k+1, x). Proof. Take x belongs to U and, by considering the geodesic triangle (x k x k+1 x), define θ = (exp 1 x k, exp 1 x k+1 k+1 x). By the law of cosines, we have d 2 (x k+1, x k ) + d 2 (x k+1, x) 2d(x k, x k+1 )d(x k+1, x) cos θ d 2 (x k, x). Since exp 1 x k+1 x k, exp 1 x k+1 x = d(x k, x k+1 )d(x k+1, x) cos θ, the above expression becomes d 2 (x k+1, x k ) + d 2 (x k+1, x) 2 exp 1 x k+1 x k, exp 1 x k+1 x d 2 (x k, x). (3.8) By Proposition 3.2.1, for any k 0, there exist α k+1 i, β k+1 i R +, i I, τ k+1 R ++, u k+1 F (x k+1 ), (u k+1 i F i (x k+1 )), and w k+1 B x k+1 N C (x k+1 ) such that (α k+1 i + β k+1 i )u k+1 i ( λ k α k+1 i ςi k ) exp 1 x k+1 x k + τ k+1 w k+1 = 0, which it follows ( λ k α k+1 i ςi k So applying exp 1 x k+1 x and denoting σ k = exp 1 x k+1 x k, exp 1 x k+1 x = 1 σ k ) exp 1 x k = (α k+1 x k+1 i + β k+1 i )u k+1 i + τ k+1 w k+1. [ ( λ k (α k+1 i + β k+1 Combining this expression with (4.7) yields d 2 (x k, x) d 2 (x k+1, x k ) + d 2 (x k+1, x) 2 σ k [ (α k+1 i + β k+1 i ) u k+1 i α k+1 i ςi k i ) u k+1 i ) we obtain, exp 1 x k+1 x + τ k+1 w k+1, exp 1 x k+1 x, exp 1 x k+1 x + τ k+1 w k+1, exp 1 x k+1 x ]. ]. 23

34 Using the characterization of the normal cone, i.e., w k+1, exp 1 x k+1 x 0, we have d 2 (x k, x) d 2 (x k+1, x k ) + d 2 (x k+1, x) 2 σ k (α k+1 because 2τ k+1 /σ k > 0. As F is convex, x U, and u k+1 i Therefore, the lemma is proven. u k+1 i, exp 1 x 0. x k+1 i + β k+1 i ) u k+1 i F i (x k+1 ), one has, exp 1 x k+1 x (3.9) It is known that (M, d) is a complete metric space. A sequence (z k ) k M is said to be Fejér convergent to the not empty set W M if for all y W and k N. d(z k+1, y) d(z k, y), (3.10) The lemma below shows two important properties of Fejér convergence, which can be found in [32], as question of completeness we present its demonstration. Lemma Let (M, d) be a finite-dimensional complete metric space. If (z k ) k M is Fejér convergent to a nonempty set W M, then (z k ) k is bounded. Furthermore, if a accumulation point z of (z k ) k belongs to W, then lim k + z k = z. Proof. Take y W. The inequality (3.10) implies d(z k, y) d(z 0, y) for all k, therefore (z k ) k is bounded. Now let (z k j ) j be a subsequence of (z k ) k such that lim j + z k j = z. As z W, by (3.10), the sequence of positive numbers (d(z k, z)) k is decreasing and it has a subsequence, namely (d(z k j, z)) j, which converges to 0. Then the whole sequence converges to 0, i.e., 0 = lim k + d(z k, z) implying lim k + z k = z. Theorem Let M be a Hadamard manifold. Let F be convex and U be as in (3.7). Then, the sequence (x k ) k converges to a weak Pareto solution of F. Proof. By Lemma 3.3.2, (x k ) k is Fejér convergent to U. Thus, applying Lemma 3.3.3, for W = U and z k = x k for all k 0, it follows that (x k ) k is bounded, and by Hopf Rinow s Theorem, this sequence has an accumulation point x M. It follows from the definition of the iterative step in (3.2) that F (x k+1 ) F (x k ) for all k 0. Consequently, x U; hence, the entire sequence (x k ) k converges to x, by Lemma again. In view of Theorem 3.2.4, x is Pareto critical. The proof is completed by Proposition Theorem Under condition Theorem Assume that (x k ) k converges to x. Then ( ) k 1 1 [Fi α l+1 i + β l+1 i (x k ) F i (x) ] b k 2k d2 (x 0, x), l=0 where (αk+1 i + β k+1 i ) = 1, for any k. 24

35 Proof. Using (3.9) with x = x one has (α k+1 i + β k+1 i ) u k+1 i, exp 1 x k+1 x λ k 2 α k+1 i ςi k As F is convex function, after some algebra, it follows (α k+1 i + β k+1 i ) [ F i (x k+1 ) F i (x) ] λ k 2 b 2 [ d 2 (x k, x) d 2 (x k+1, x k ) d 2 (x k+1, x) ]. α k+1 i ςi k [ d 2 (x k, x) d 2 (x k+1, x) ] [ d 2 (x k, x) d 2 (x k+1, x) ]. By Summing the above expression from l = 0,..., k 1, one derives k 1 (α l+1 i + β l+1 i ) [ F i (x l+1 ) F i (x) ] b 2 l=0 k 1 [ d 2 (x l, x) d 2 (x l+1, x) ]. Since F i (x k ) F i (x k 1 ) for all i I and using telescopic sum, we have Hence k 1 (α l+1 i + β l+1 i ) [ F i (x k ) F i (x) ] b 2 l=0 ( k 1 l=0 l=0 [ d 2 (x 0, x) d 2 (x k, x) ]. ) [Fi α l+1 i + β l+1 i (x k ) F i (x) ] b 2 d2 (x 0, x). Now, dividing both sides of this last inequality by k ( ) k 1 1 [Fi α l+1 i + β l+1 i (x k ) F i (x) ] b k 2k d2 (x 0, x). l=0 25