Download or read book New Primitives for Convex Optimization and Graph Algorithms written by Arun Jambulapati and published by . This book was released on 2022 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Iterative methods, especially those from convex optimization, form the basis of much of the modern algorithmic landscape. The success of such methods relies on their generality: methods such as gradient descent and Newton's method typically converge to high-quality minimizers with only minimal assumptions placed on the objective. However, in many real-world settings the theoretical guarantees obtained by such algorithms are often insufficient to see use in practice. This thesis addresses this issue by developing methods for convex optimization and graph algorithms by exploiting problem-specific structure. Part I gives a state-of-the-art algorithm for solving Laplacian linear systems, as well as a faster algorithm for minimum-cost flow. Our results are achieved through novel combinations of classical iterative methods from convex optimization with graph-based data structures and preconditioners. Part II gives new algorithms for several generic classes of structured convex optimization problems. We give near-optimal methods for minimizing convex functions admitting a ball-optimization oracle and minimizing the maximum of N convex functions, as well as new algorithms for projection-efficient and composite convex minimization. Our results are achieved through a more refined understanding of classical accelerated gradient methods, and give new algorithms for a variety of important machine learning tasks such as logistic regression and hard margin SVMs. Part III discusses advancements in algorithms for the discrete optimal transport problem, a task which has seen enormous interest in recent years due to new applications in deep learning. We give simple and parallel algorithms for approximating discrete optimal transport, and additionally demonstrate that these algorithms can be implemented in space-bounded and streaming settings. By leveraging our machinery further, we also give improved pass-complexity bounds for graph optimization problems (such as bipartite matching and transshipment) in the semi-streaming model.
Download or read book Lectures on Modern Convex Optimization written by Aharon Ben-Tal and published by SIAM. This book was released on 2001-01-01 with total page 504 pages. Available in PDF, EPUB and Kindle. Book excerpt: Here is a book devoted to well-structured and thus efficiently solvable convex optimization problems, with emphasis on conic quadratic and semidefinite programming. The authors present the basic theory underlying these problems as well as their numerous applications in engineering, including synthesis of filters, Lyapunov stability analysis, and structural design. The authors also discuss the complexity issues and provide an overview of the basic theory of state-of-the-art polynomial time interior point methods for linear, conic quadratic, and semidefinite programming. The book's focus on well-structured convex problems in conic form allows for unified theoretical and algorithmical treatment of a wide spectrum of important optimization problems arising in applications.
Download or read book Interior point Polynomial Algorithms in Convex Programming written by Yurii Nesterov and published by SIAM. This book was released on 1994-01-01 with total page 414 pages. Available in PDF, EPUB and Kindle. Book excerpt: Specialists working in the areas of optimization, mathematical programming, or control theory will find this book invaluable for studying interior-point methods for linear and quadratic programming, polynomial-time methods for nonlinear convex programming, and efficient computational methods for control problems and variational inequalities. A background in linear algebra and mathematical programming is necessary to understand the book. The detailed proofs and lack of "numerical examples" might suggest that the book is of limited value to the reader interested in the practical aspects of convex optimization, but nothing could be further from the truth. An entire chapter is devoted to potential reduction methods precisely because of their great efficiency in practice.
Download or read book New Primitives for Tackling Graph Problems and Their Applications in Parallel Computing written by Peilin Zhong and published by . This book was released on 2021 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: These algorithms are built on a series of new graph algorithmic primitives which may be of independent interests.
Download or read book Advances in Convex Analysis and Global Optimization written by Nicolas Hadjisavvas and published by Springer Science & Business Media. This book was released on 2013-12-01 with total page 601 pages. Available in PDF, EPUB and Kindle. Book excerpt: There has been much recent progress in global optimization algo rithms for nonconvex continuous and discrete problems from both a theoretical and a practical perspective. Convex analysis plays a fun damental role in the analysis and development of global optimization algorithms. This is due essentially to the fact that virtually all noncon vex optimization problems can be described using differences of convex functions and differences of convex sets. A conference on Convex Analysis and Global Optimization was held during June 5 -9, 2000 at Pythagorion, Samos, Greece. The conference was honoring the memory of C. Caratheodory (1873-1950) and was en dorsed by the Mathematical Programming Society (MPS) and by the Society for Industrial and Applied Mathematics (SIAM) Activity Group in Optimization. The conference was sponsored by the European Union (through the EPEAEK program), the Department of Mathematics of the Aegean University and the Center for Applied Optimization of the University of Florida, by the General Secretariat of Research and Tech nology of Greece, by the Ministry of Education of Greece, and several local Greek government agencies and companies. This volume contains a selective collection of refereed papers based on invited and contribut ing talks presented at this conference. The two themes of convexity and global optimization pervade this book. The conference provided a forum for researchers working on different aspects of convexity and global opti mization to present their recent discoveries, and to interact with people working on complementary aspects of mathematical programming.
Download or read book Large Scale Convex Optimization written by Ernest K. Ryu and published by Cambridge University Press. This book was released on 2022-11-30 with total page 319 pages. Available in PDF, EPUB and Kindle. Book excerpt: A unified analysis of first-order optimization methods, including parallel-distributed algorithms, using monotone operators.
Download or read book Lectures on Modern Convex Optimization written by Aharon Ben-Tal and published by . This book was released on 2001 with total page 488 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Algorithms for Convex Optimization with Applications to Data Science written by Scott Roy and published by . This book was released on 2017 with total page 124 pages. Available in PDF, EPUB and Kindle. Book excerpt: Convex optimization is more popular than ever, with extensive applications in statistics, machine learning, and engineering. Nesterov introduced optimal first-order methods for large scale convex optimization in the 1980s, and extremely fast interior point methods for small-to-medium scale convex optimization emerged in the 1990s. Today there is little reason to prefer modelling with linear programming over convex programming for computational reasons. Nonetheless, there is room to improve the already sophisticated algorithms for convex optimization. The thesis makes three primary contributions to convex optimization. First, the thesis develops new, near optimal barriers for generalized power cones. This is relevant because the performance of interior point methods depends on representing convex sets with “small parameter” barriers. Second, the thesis introduces an intuitive, first-order method that achieves the best theoretical convergence rate and has better performance in practice than Nesterov’s method. The thesis concludes with a framework for reformulating a convex program by interchanging the objective function and a constraint function. The approach is illustrated on several examples.
Download or read book Parallel Algorithms in Computational Science and Engineering written by Ananth Grama and published by Springer Nature. This book was released on 2020-07-06 with total page 421 pages. Available in PDF, EPUB and Kindle. Book excerpt: This contributed volume highlights two areas of fundamental interest in high-performance computing: core algorithms for important kernels and computationally demanding applications. The first few chapters explore algorithms, numerical techniques, and their parallel formulations for a variety of kernels that arise in applications. The rest of the volume focuses on state-of-the-art applications from diverse domains. By structuring the volume around these two areas, it presents a comprehensive view of the application landscape for high-performance computing, while also enabling readers to develop new applications using the kernels. Readers will learn how to choose the most suitable parallel algorithms for any given application, ensuring that theory and practicality are clearly connected. Applications using these techniques are illustrated in detail, including: Computational materials science and engineering Computational cardiovascular analysis Multiscale analysis of wind turbines and turbomachinery Weather forecasting Machine learning techniques Parallel Algorithms in Computational Science and Engineering will be an ideal reference for applied mathematicians, engineers, computer scientists, and other researchers who utilize high-performance computing in their work.
Download or read book Faster Algorithms for Convex and Combinatorial Optimization written by Yin Tat Lee and published by . This book was released on 2016 with total page 458 pages. Available in PDF, EPUB and Kindle. Book excerpt: In this thesis, we revisit three algorithmic techniques: sparsification, cutting and collapsing. We use them to obtain the following results on convex and combinatorial optimization: --Linear Programming: We obtain the first improvement to the running time for linear programming in 25 years. The convergence rate of this randomized algorithm nearly matches the universal barrier for interior point methods. As a corollary, we obtain the first ... time randomized algorithm for solving the maximum flow problem on directed graphs with m edges and n vertices. This improves upon the previous fastest running time of achieved over 15 years ago by Goldberg and Rao. --Maximum Flow Problem: We obtain one of the first almost-linear time randomized algorithms for approximating the maximum flow in undirected graphs. As a corollary, we improve the running time of a wide range of algorithms that use the computation of maximum flows as a subroutine. --Non-Smooth Convex Optimization: We obtain the first nearly-cubic time randomized algorithm for convex problems under the black box model. As a corollary, this implies a polynomially faster algorithm for three fundamental problems in computer science: submodular function minimization, matroid intersection, and semidefinite programming. --Graph Sparsification: We obtain the first almost-linear time randomized algorithm for spectrally approximating any graph by one with just a linear number of edges. This sparse graph approximately preserves all cut values of the original graph and is useful for solving a wide range of combinatorial problems. This algorithm improves all previous linear-sized constructions, which required at least quadratic time. --Numerical Linear Algebra: Multigrid is an efficient method for solving large-scale linear systems which arise from graphs in low dimensions. It has been used extensively for 30 years in scientific computing. Unlike the previous approaches that make assumptions on the graphs, we give the first generalization of the multigrid that provably solves Laplacian systems of any graphs in nearly-linear expected time.
Download or read book Acceleration and New Analysis of Convex Optimization Algorithms written by Lewis Liu and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Recent years have witnessed a resurgence of the Frank-Wolfe (FW) algorithm, also known as conditional gradient methods, in sparse optimization and large-scale machine learning problems with smooth convex objectives. Compared to projected or proximal gradient methods, such projection-free method saves the computational cost of orthogonal projections onto the constraint set. Meanwhile, FW also gives solutions with sparse structure. Despite of these promising properties, FW does not enjoy the optimal convergence rates achieved by projection-based accelerated methods. On the other hand, FW algorithm is affine-covariant, and enjoys accelerated convergence rates when the constraint set is strongly convex. However, these results rely on norm-dependent assumptions, usually incurring non-affine invariant bounds, in contradiction with FW's affine-covariant property. In this work, we introduce new structural assumptions on the problem (such as the directional smoothness) and derive an affine in- variant, norm-independent analysis of Frank-Wolfe. Based on our analysis, we pro- pose an affine invariant backtracking line-search. Interestingly, we show that typical back-tracking line-search techniques using smoothness of the objective function surprisingly converge to an affine invariant stepsize, despite using affine-dependent norms in the computation of stepsizes. This indicates that we do not necessarily need to know the structure of sets in advance to enjoy the affine-invariant accelerated rate. Additionally, we provide a promising direction to accelerate FW over strongly convex sets using duality gap techniques and a new version of smoothness. In another line of research, we study algorithms beyond first-order methods. Quasi-Newton techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi- Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees.
Download or read book Algorithms and Data Structures written by Frank Dehne and published by Springer. This book was released on 2003-05-15 with total page 496 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book constitutes the refereed proceedings of the 7th International Workshop on Algorithms and Data Structures, WADS 2001, held in Providence, RI, USA in August 2001. The 40 revised full papers presented were carefully reviewed and selected from a total of 89 submissions. Among the topics addressed are multiobjective optimization, computational graph theory, approximation, optimization, combinatorics, scheduling, Varanoi diagrams, packings, multi-party computation, polygons, searching, etc.
Download or read book New Graph Algorithms Via Polyhedral Techniques written by Jakub Tarnawski and published by . This book was released on 2019 with total page 130 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mots-clés de l'auteur: graph algorithms ; linear programming ; traveling salesman problem ; perfect matching ; derandomization ; parallel algorithms ; approximation algorithms ; combinatorial optimization ; discrete optimization ; theoretical computer science.
Download or read book Algorithms for Convex Optimization written by Nisheeth K. Vishnoi and published by Cambridge University Press. This book was released on 2021-10-07 with total page 314 pages. Available in PDF, EPUB and Kindle. Book excerpt: In the last few years, Algorithms for Convex Optimization have revolutionized algorithm design, both for discrete and continuous optimization problems. For problems like maximum flow, maximum matching, and submodular function minimization, the fastest algorithms involve essential methods such as gradient descent, mirror descent, interior point methods, and ellipsoid methods. The goal of this self-contained book is to enable researchers and professionals in computer science, data science, and machine learning to gain an in-depth understanding of these algorithms. The text emphasizes how to derive key algorithms for convex optimization from first principles and how to establish precise running time bounds. This modern text explains the success of these algorithms in problems of discrete optimization, as well as how these methods have significantly pushed the state of the art of convex optimization itself.
Download or read book Non convex Optimization for Machine Learning written by Prateek Jain and published by Foundations and Trends in Machine Learning. This book was released on 2017-12-04 with total page 218 pages. Available in PDF, EPUB and Kindle. Book excerpt: Non-convex Optimization for Machine Learning takes an in-depth look at the basics of non-convex optimization with applications to machine learning. It introduces the rich literature in this area, as well as equips the reader with the tools and techniques needed to apply and analyze simple but powerful procedures for non-convex problems. Non-convex Optimization for Machine Learning is as self-contained as possible while not losing focus of the main topic of non-convex optimization techniques. The monograph initiates the discussion with entire chapters devoted to presenting a tutorial-like treatment of basic concepts in convex analysis and optimization, as well as their non-convex counterparts. The monograph concludes with a look at four interesting applications in the areas of machine learning and signal processing, and exploring how the non-convex optimization techniques introduced earlier can be used to solve these problems. The monograph also contains, for each of the topics discussed, exercises and figures designed to engage the reader, as well as extensive bibliographic notes pointing towards classical works and recent advances. Non-convex Optimization for Machine Learning can be used for a semester-length course on the basics of non-convex optimization with applications to machine learning. On the other hand, it is also possible to cherry pick individual portions, such the chapter on sparse recovery, or the EM algorithm, for inclusion in a broader course. Several courses such as those in machine learning, optimization, and signal processing may benefit from the inclusion of such topics.
Download or read book Planning Algorithms written by Steven M. LaValle and published by Cambridge University Press. This book was released on 2006-05-29 with total page 844 pages. Available in PDF, EPUB and Kindle. Book excerpt: Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. Written for computer scientists and engineers with interests in artificial intelligence, robotics, or control theory, this is the only book on this topic that tightly integrates a vast body of literature from several fields into a coherent source for teaching and reference in a wide variety of applications. Difficult mathematical material is explained through hundreds of examples and illustrations.
Download or read book Future Trends of HPC in a Disruptive Scenario written by L. Grandinetti and published by IOS Press. This book was released on 2019-09-27 with total page 286 pages. Available in PDF, EPUB and Kindle. Book excerpt: The realization that the use of components off the shelf (COTS) could reduce costs sparked the evolution of the massive parallel computing systems available today. The main problem with such systems is the development of suitable operating systems, algorithms and application software that can utilise the potential processing power of large numbers of processors. As a result, systems comprising millions of processors are still limited in the applications they can efficiently solve. Two alternative paradigms that may offer a solution to this problem are Quantum Computers (QC) and Brain Inspired Computers (BIC). This book presents papers from the 14th edition of the biennial international conference on High Performance Computing - From Clouds and Big Data to Exascale and Beyond, held in Cetraro, Italy, from 2 - 6 July 2018. It is divided into 4 sections covering data science, quantum computing, high-performance computing, and applications. The papers presented during the workshop covered a wide spectrum of topics on new developments in the rapidly evolving supercomputing field – including QC and BIC – and a selection of contributions presented at the workshop are included in this volume. In addition, two papers presented at a workshop on Brain Inspired Computing in 2017 and an overview of work related to data science executed by a number of universities in the USA, parts of which were presented at the 2018 and previous workshops, are also included. The book will be of interest to all those whose work involves high-performance computing.
