Download or read book Structure Exploiting Numerical Algorithms for Optimal Control written by Isak Nielsen and published by Linköping University Electronic Press. This book was released on 2017-04-20 with total page 202 pages. Available in PDF, EPUB and Kindle. Book excerpt: Numerical algorithms for efficiently solving optimal control problems are important for commonly used advanced control strategies, such as model predictive control (MPC), but can also be useful for advanced estimation techniques, such as moving horizon estimation (MHE). In MPC, the control input is computed by solving a constrained finite-time optimal control (CFTOC) problem on-line, and in MHE the estimated states are obtained by solving an optimization problem that often can be formulated as a CFTOC problem. Common types of optimization methods for solving CFTOC problems are interior-point (IP) methods, sequential quadratic programming (SQP) methods and active-set (AS) methods. In these types of methods, the main computational effort is often the computation of the second-order search directions. This boils down to solving a sequence of systems of equations that correspond to unconstrained finite-time optimal control (UFTOC) problems. Hence, high-performing second-order methods for CFTOC problems rely on efficient numerical algorithms for solving UFTOC problems. Developing such algorithms is one of the main focuses in this thesis. When the solution to a CFTOC problem is computed using an AS type method, the aforementioned system of equations is only changed by a low-rank modification between two AS iterations. In this thesis, it is shown how to exploit these structured modifications while still exploiting structure in the UFTOC problem using the Riccati recursion. Furthermore, direct (non-iterative) parallel algorithms for computing the search directions in IP, SQP and AS methods are proposed in the thesis. These algorithms exploit, and retain, the sparse structure of the UFTOC problem such that no dense system of equations needs to be solved serially as in many other algorithms. The proposed algorithms can be applied recursively to obtain logarithmic computational complexity growth in the prediction horizon length. For the case with linear MPC problems, an alternative approach to solving the CFTOC problem on-line is to use multiparametric quadratic programming (mp-QP), where the corresponding CFTOC problem can be solved explicitly off-line. This is referred to as explicit MPC. One of the main limitations with mp-QP is the amount of memory that is required to store the parametric solution. In this thesis, an algorithm for decreasing the required amount of memory is proposed. The aim is to make mp-QP and explicit MPC more useful in practical applications, such as embedded systems with limited memory resources. The proposed algorithm exploits the structure from the QP problem in the parametric solution in order to reduce the memory footprint of general mp-QP solutions, and in particular, of explicit MPC solutions. The algorithm can be used directly in mp-QP solvers, or as a post-processing step to an existing solution.

Download or read book Exploiting Problem Structure in Pattern Search Methods for Unconstrained Optimization written by Christopher John Price and published by . This book was released on 2004 with total page 14 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Optimization Techniques Exploiting Problem Structure written by Ajit R. Shenoy and published by . This book was released on 1997 with total page 398 pages. Available in PDF, EPUB and Kindle. Book excerpt: Author's abstract: The research presented in this dissertation investigates the use of all-at-once methods applied to aerodynamic design. All-at-once schemes are usually based on the assumption of sufficient continuity in the constraints and objectives, and this assumption can be troublesome in the presence of shock discontinuities. Special treatment has to be considered for such problems and we study several approaches. Our all-at-once methods are based on the Sequential Quadratic Programming method, and are designed to exploit the structure inherent in a given problem. The first method is a Reduced Hessian formulation which projects the optimization problem to a lower dimension design space. The second method exploits the sparse structure in a given problem which can yield significant savings in terms of computational effort as well as storage requirements. An underlying theme in all our applications is that careful analysis of the given problem can often lead to an efficient implementation of these all-at-once methods. Chapter 2 describes a nozzle design problem involving one-dimensional transonic flow. An initial formulation as an optimal control problem allows us to solve the problem as as two-point boundary problem which provides useful insight into the nature of the problem. Using the Reduced Hessian formulation for this problem, we find that a conventional CFD method based on shock capturing produces poor performance. The numerical difficulties caused by the presence of the shock can be alleviated by reformulating the constraints so that the shock can be treated explicitly. This amounts to using a shock fitting technique. In Chapter 3, we study variants of a simplified temperature control problem. The control problem is solved using a sparse SQP scheme. We show that for problems where the underlying infinite-dimensional problem is well-posed, the optimizer performs well, whereas it fails to produce good results for problems where the underlying infinite-dimensional problem is ill-posed. A transonic airfoil design problem is studied in Chapter 4, using the Reduced SQP formulation. We propose a scheme for performing the optimization subtasks that is based on an Euler Implicit time integration scheme. The motivation is to preserve the solution-finding structure used in the analysis algorithm. Preliminary results obtained using this method are promising. Numerical results have been presented for all the problems described.

Download or read book Convex Optimization Algorithms and Statistical Bounds for Learning Structured Models written by Amin Jalali and published by . This book was released on 2016 with total page 178 pages. Available in PDF, EPUB and Kindle. Book excerpt: Design and analysis of tractable methods for estimation of structured models from massive high-dimensional datasets has been a topic of research in statistics, machine learning and engineering for many years. Regularization, the act of simultaneously optimizing a data fidelity term and a structure-promoting term, is a widely used approach in different machine learning and signal processing tasks. Appropriate regularizers, with efficient optimization techniques, can help in exploiting the prior structural information on the underlying model. This dissertation is focused on exploring new structures, devising efficient convex relaxations for exploiting them, and studying the statistical performance of such estimators. We address three problems under this framework on which we elaborate below. In many applications, we aim to reconstruct models that are known to have more than one structure at the same time. Having a rich literature on exploiting common structures like sparsity and low rank at hand, one could pose similar questions about simultaneously structured models with several low-dimensional structures. Using the respective known convex penalties for the involved structures, we show that multi-objective optimization with these penalties can do no better, order-wise, than exploiting only one of the present structures. This suggests that to fully exploit the multiple structures, we need an entirely new convex relaxation, not one that combines the convex relaxations for each structure. This work, while applicable for general structures, yields interesting results for the case of sparse and low-rank matrices which arise in applications such as sparse phase retrieval and quadratic compressed sensing. We then turn our attention to the design and efficient optimization of convex penalties for structured learning. We introduce a general class of semidefinite representable penalties, called variational Gram functions (VGF), and provide a list of optimization tools for solving regularized estimation problems involving VGFs. Exploiting the variational structure in VGFs, as well as the variational structure in many common loss functions, enables us to devise efficient optimization techniques as well as to provide guarantees on the solutions of many regularized loss minimization problems. Finally, we explore the statistical and computational trade-offs in the community detection problem. We study recovery regimes and algorithms for community detection in sparse graphs generated under a heterogeneous stochastic block model in its most general form. In this quest, we were able to expand the applicability of semidefinite programs (in exact community detection) to some new and important network configurations, which provides us with a better understanding of the ability of semidefinite programs in reaching statistical identifiability limits.

Download or read book Numerical Nonsmooth Optimization written by Adil M. Bagirov and published by Springer Nature. This book was released on 2020-02-28 with total page 696 pages. Available in PDF, EPUB and Kindle. Book excerpt: Solving nonsmooth optimization (NSO) problems is critical in many practical applications and real-world modeling systems. The aim of this book is to survey various numerical methods for solving NSO problems and to provide an overview of the latest developments in the field. Experts from around the world share their perspectives on specific aspects of numerical NSO. The book is divided into four parts, the first of which considers general methods including subgradient, bundle and gradient sampling methods. In turn, the second focuses on methods that exploit the problem’s special structure, e.g. algorithms for nonsmooth DC programming, VU decomposition techniques, and algorithms for minimax and piecewise differentiable problems. The third part considers methods for special problems like multiobjective and mixed integer NSO, and problems involving inexact data, while the last part highlights the latest advancements in derivative-free NSO. Given its scope, the book is ideal for students attending courses on numerical nonsmooth optimization, for lecturers who teach optimization courses, and for practitioners who apply nonsmooth optimization methods in engineering, artificial intelligence, machine learning, and business. Furthermore, it can serve as a reference text for experts dealing with nonsmooth optimization.

Download or read book Exploiting structure in non convex quadratic optimization and gas network planning under uncertainty written by Jonas Schweiger and published by Logos Verlag Berlin GmbH. This book was released on 2017 with total page 206 pages. Available in PDF, EPUB and Kindle. Book excerpt: The amazing success of computational mathematical optimization over the last decades has been driven more by insights into mathematical structures than by the advance of computing technology. In this vein, Jonas Schweiger addresses applications, where nonconvexity in the model and uncertainty in the data pose principal difficulties. In the first part, he contributes strong relaxations for non-convex problems such as the non-convex quadratic programming and the Pooling Problem. In the second part, he contributes a robust model for gas transport network extension and a custom decomposition approach. All results are backed by extensive computational studies.

Download or read book Sparsity and Structure Exploiting Diagonally Dominant Relaxation of the OPF Problem written by Laith Mubaslat and published by . This book was released on 2020 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: "The Optimal Power Flow (OPF) is an optimization problem which tackles both the economy and physics of power systems operation. Due to high non-linearity in the power flow equations, the OPF problem is non-convex. Consequently, optimally solving for the OPF problem at a reasonable computational time presents a serious challenge. Several approaches were presented to solve the OPF problem. These include local solvers, heuristic methods and the approximation of non-linear equations. However, these approaches either do not bound the true value of the objective function or are lacking in the trade-off they provide between solution time and quality. As an alternative, convex relaxation techniques could be used to address this challenge. A convex relaxation is obtained by means of finding a convex representation of the problem’s feasible space. As a natural byproduct of the convexity of the resulting problem, a wide array of convex optimization techniques could be utilized. Furthermore, the solution obtained presents a lower bound on the global solution of the original non-convex problem. Several factors influence the tightness and scalability of convex relaxations. Those include the number and type of constraints used in the relaxation of the original non-convex problem. Most relaxations of the optimal power flow problem are based on second order conic or positive semidefinite type of constraints. Alternatively, in this dissertation we address the utilization of the linearly representable diagonally dominant cone in relaxing the optimal power flow problem. First, we investigate the diagonally-dominant-sum-of-squares relaxation of the problem. We evaluate the reasons behind its poor optimality gaps and scalability issue. We demonstrate that diagonal dominance could be utilized in creating a similar, yet tighter relaxation. The relaxation we propose is based on the semidefinite relaxation of the problem. This dissertation then follows to improve the tractability of the aforementioned relaxation. We achieve that by an investigation into the optimal exploitation of the sparsity and structure of the OPF problem. Several methods exist for the exploitation of sparsity in semidefininte programming. Specifically, chordal decomposition has been applied with great success to improve the tractability of the semidefinite relaxation of the optimal power flow problem. Accordingly, we investigate the utilization of chordal decomposition in improving the diagonal dominance based relaxation proposed in this thesis. We find that the direct exploitation of sparsity requires a number of linear inequalities that scales linearly with the size of the problem. Alternatively, chordal decomposition introduces equality and inequality constraints into the problem which needlessly increases its computational demand. We prove the direct exploitation of sparsity to be more beneficial in the case of a relaxation similar to that of this dissertation. Additionally, we exploit the structure of the problem in further reducing the number of linear inequalities by half. We further suggest two more relaxations based on the empirical results of the improved relaxation proposed"--

Download or read book Integrated Methods for Optimization written by John N. Hooker and published by Springer Science & Business Media. This book was released on 2011-11-13 with total page 655 pages. Available in PDF, EPUB and Kindle. Book excerpt: The first edition of Integrated Methods for Optimization was published in January 2007. Because the book covers a rapidly developing field, the time is right for a second edition. The book provides a unified treatment of optimization methods. It brings ideas from mathematical programming (MP), constraint programming (CP), and global optimization (GO)into a single volume. There is no reason these must be learned as separate fields, as they normally are, and there are three reasons they should be studied together. (1) There is much in common among them intellectually, and to a large degree they can be understood as special cases of a single underlying solution technology. (2) A growing literature reports how they can be profitably integrated to formulate and solve a wide range of problems. (3) Several software packages now incorporate techniques from two or more of these fields. The book provides a unique resource for graduate students and practitioners who want a well-rounded background in optimization methods within a single course of study. Engineering students are a particularly large potential audience, because engineering optimization problems often benefit from a combined approach—particularly where design, scheduling, or logistics are involved. The text is also of value to those studying operations research, because their educational programs rarely cover CP, and to those studying computer science and artificial intelligence (AI), because their curric ula typically omit MP and GO. The text is also useful for practitioners in any of these areas who want to learn about another, because it provides a more concise and accessible treatment than other texts. The book can cover so wide a range of material because it focuses on ideas that arerelevant to the methods used in general-purpose optimization and constraint solvers. The book focuses on ideas behind the methods that have proved useful in general-purpose optimization and constraint solvers, as well as integrated solvers of the present and foreseeable future. The second edition updates results in this area and includes several major new topics: Background material in linear, nonlinear, and dynamic programming. Network flow theory, due to its importance in filtering algorithms. A chapter on generalized duality theory that more explicitly develops a unifying primal-dual algorithmic structure for optimization methods. An extensive survey of search methods from both MP and AI, using the primal-dual framework as an organizing principle. Coverage of several additional global constraints used in CP solvers. The book continues to focus on exact as opposed to heuristic methods. It is possible to bring heuristic methods into the unifying scheme described in the book, and the new edition will retain the brief discussion of how this might be done.

Download or read book Optimization with PDE Constraints written by Michael Hinze and published by Springer Science & Business Media. This book was released on 2008-10-16 with total page 279 pages. Available in PDF, EPUB and Kindle. Book excerpt: Solving optimization problems subject to constraints given in terms of partial d- ferential equations (PDEs) with additional constraints on the controls and/or states is one of the most challenging problems in the context of industrial, medical and economical applications, where the transition from model-based numerical si- lations to model-based design and optimal control is crucial. For the treatment of such optimization problems the interaction of optimization techniques and num- ical simulation plays a central role. After proper discretization, the number of op- 3 10 timization variables varies between 10 and 10 . It is only very recently that the enormous advances in computing power have made it possible to attack problems of this size. However, in order to accomplish this task it is crucial to utilize and f- ther explore the speci?c mathematical structure of optimization problems with PDE constraints, and to develop new mathematical approaches concerning mathematical analysis, structure exploiting algorithms, and discretization, with a special focus on prototype applications. The present book provides a modern introduction to the rapidly developing ma- ematical ?eld of optimization with PDE constraints. The ?rst chapter introduces to the analytical background and optimality theory for optimization problems with PDEs. Optimization problems with PDE-constraints are posed in in?nite dim- sional spaces. Therefore, functional analytic techniques, function space theory, as well as existence- and uniqueness results for the underlying PDE are essential to study the existence of optimal solutions and to derive optimality conditions.

Download or read book Structural Optimization written by A. Borkowski and published by Springer Science & Business Media. This book was released on 1990-01-31 with total page 422 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book 40 Algorithms Every Data Scientist Should Know written by Jürgen Weichenberger and published by BPB Publications. This book was released on 2024-09-07 with total page 655 pages. Available in PDF, EPUB and Kindle. Book excerpt: DESCRIPTION Mastering AI and ML algorithms is essential for data scientists. This book covers a wide range of techniques, from supervised and unsupervised learning to deep learning and reinforcement learning. This book is a compass to the most important algorithms that every data scientist should have at their disposal when building a new AI/ML application. This book offers a thorough introduction to AI and ML, covering key concepts, data structures, and various algorithms like linear regression, decision trees, and neural networks. It explores learning techniques like supervised, unsupervised, and semi-supervised learning and applies them to real-world scenarios such as natural language processing and computer vision. With clear explanations, code examples, and detailed descriptions of 40 algorithms, including their mathematical foundations and practical applications, this resource is ideal for both beginners and experienced professionals looking to deepen their understanding of AI and ML. The final part of the book gives an outlook for more state-of-the-art algorithms that will have the potential to change the world of AI and ML fundamentals. KEY FEATURES ● Covers a wide range of AI and ML algorithms, from foundational concepts to advanced techniques. ● Includes real-world examples and code snippets to illustrate the application of algorithms. ● Explains complex topics in a clear and accessible manner, making it suitable for learners of all levels. WHAT YOU WILL LEARN ● Differences between supervised, unsupervised, and reinforcement learning. ● Gain expertise in data cleaning, feature engineering, and handling different data formats. ● Learn to implement and apply algorithms such as linear regression, decision trees, neural networks, and support vector machines. ● Creating intelligent systems and solving real-world problems. ● Learn to approach AI and ML challenges with a structured and analytical mindset. WHO THIS BOOK IS FOR This book is ideal for data scientists, ML engineers, and anyone interested in entering the world of AI. TABLE OF CONTENTS 1. Fundamentals 2. Typical Data Structures 3. 40 AI/ML Algorithms Overview 4. Basic Supervised Learning Algorithms 5. Advanced Supervised Learning Algorithms 6. Basic Unsupervised Learning Algorithms 7. Advanced Unsupervised Learning Algorithms 8. Basic Reinforcement Learning Algorithms 9. Advanced Reinforcement Learning Algorithms 10. Basic Semi-Supervised Learning Algorithms 11. Advanced Semi-Supervised Learning Algorithms 12. Natural Language Processing 13. Computer Vision 14. Large-Scale Algorithms 15. Outlook into the Future: Quantum Machine Learning

Download or read book New Primitives for Convex Optimization and Graph Algorithms written by Arun Jambulapati and published by . This book was released on 2022 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Iterative methods, especially those from convex optimization, form the basis of much of the modern algorithmic landscape. The success of such methods relies on their generality: methods such as gradient descent and Newton's method typically converge to high-quality minimizers with only minimal assumptions placed on the objective. However, in many real-world settings the theoretical guarantees obtained by such algorithms are often insufficient to see use in practice. This thesis addresses this issue by developing methods for convex optimization and graph algorithms by exploiting problem-specific structure. Part I gives a state-of-the-art algorithm for solving Laplacian linear systems, as well as a faster algorithm for minimum-cost flow. Our results are achieved through novel combinations of classical iterative methods from convex optimization with graph-based data structures and preconditioners. Part II gives new algorithms for several generic classes of structured convex optimization problems. We give near-optimal methods for minimizing convex functions admitting a ball-optimization oracle and minimizing the maximum of N convex functions, as well as new algorithms for projection-efficient and composite convex minimization. Our results are achieved through a more refined understanding of classical accelerated gradient methods, and give new algorithms for a variety of important machine learning tasks such as logistic regression and hard margin SVMs. Part III discusses advancements in algorithms for the discrete optimal transport problem, a task which has seen enormous interest in recent years due to new applications in deep learning. We give simple and parallel algorithms for approximating discrete optimal transport, and additionally demonstrate that these algorithms can be implemented in space-bounded and streaming settings. By leveraging our machinery further, we also give improved pass-complexity bounds for graph optimization problems (such as bipartite matching and transshipment) in the semi-streaming model.

Download or read book Exploiting Problem Structure in Distributed Constraint Optimisation with Complex Local Problems written by David A. Burke and published by . This book was released on 2008 with total page 214 pages. Available in PDF, EPUB and Kindle. Book excerpt: In today{u2019}s world, networks are ubiquitous, e.g. supply chain networks, computational grids, telecom networks and social networks. In many situations, the individual entities or {u2018}agents{u2019} that make up these networks need to coordinate their actions in order to make some group decision. Distributed Constraint Optimisation (DisCOP) considers algorithms explicitly designed to handle such problems, searching for globally optimal solutions while balancing communication load with processing time. However, most research on DisCOP algorithms only considers simplified problems where each agent has a single variable, i.e. only one decision to make. This is justified by two problem reformulations, by which any DisCOP with complex local problems (multiple variables per agent) can be transformed to give exactly one variable per agent. The restriction to a single variable has been an impediment to practical applications of DisCOP, since few problems naturally fit into that framework. Furthermore, there has been no research showing whether the standard reformulations are actually effective. In this dissertation, we address this issue. We evaluate the standard reformulation techniques and show that one of them is rarely competitive. We demonstrate that explicitly considering the structure of DisCOPs with complex local problems in the design of algorithms allows problems to be solved more efficiently. In particular, we show the benefits of distinguishing between the public (between agents) and private (within one agent) search spaces. Furthermore, we identify the public variables (those involved in inter-agent constraints) as a critical factor affecting how DisCOPs with complex local problems are solved. From this, we propose a number of novel techniques based on interchangeability, symmetry, relaxation, aggregation and domain reduction. These methods exploit the problem structure and act on the public variables to enable more efficient solving of Dis-COPs with complex local problems, thus greatly extending the range of problems that can be solved using DisCOP algorithms.

Download or read book Convex Optimization written by Stephen P. Boyd and published by Cambridge University Press. This book was released on 2004-03-08 with total page 744 pages. Available in PDF, EPUB and Kindle. Book excerpt: Convex optimization problems arise frequently in many different fields. This book provides a comprehensive introduction to the subject, and shows in detail how such problems can be solved numerically with great efficiency. The book begins with the basic elements of convex sets and functions, and then describes various classes of convex optimization problems. Duality and approximation techniques are then covered, as are statistical estimation techniques. Various geometrical problems are then presented, and there is detailed discussion of unconstrained and constrained minimization problems, and interior-point methods. The focus of the book is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. It contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance and economics.

Download or read book Exploiting Structure in Large scale Optimization for Machine Learning written by Cho-Jui Hsieh and published by . This book was released on 2015 with total page 288 pages. Available in PDF, EPUB and Kindle. Book excerpt: With an immense growth of data, there is a great need for solving large-scale machine learning problems. Classical optimization algorithms usually cannot scale up due to huge amount of data and/or model parameters. In this thesis, we will show that the scalability issues can often be resolved by exploiting three types of structure in machine learning problems: problem structure, model structure, and data distribution. This central idea can be applied to many machine learning problems. In this thesis, we will describe in detail how to exploit structure for kernel classification and regression, matrix factorization for recommender systems, and structure learning for graphical models. We further provide comprehensive theoretical analysis for the proposed algorithms to show both local and global convergent rate for a family of in-exact first-order and second-order optimization methods.

Download or read book Handbook of Graph Theory Combinatorial Optimization and Algorithms written by Krishnaiyan "KT" Thulasiraman and published by CRC Press. This book was released on 2016-01-05 with total page 1217 pages. Available in PDF, EPUB and Kindle. Book excerpt: The fusion between graph theory and combinatorial optimization has led to theoretically profound and practically useful algorithms, yet there is no book that currently covers both areas together. Handbook of Graph Theory, Combinatorial Optimization, and Algorithms is the first to present a unified, comprehensive treatment of both graph theory and c

Download or read book Practical Optimization Methods written by M. Asghar Bhatti and published by Springer Science & Business Media. This book was released on 2000-06-22 with total page 736 pages. Available in PDF, EPUB and Kindle. Book excerpt: This introductory textbook adopts a practical and intuitive approach, rather than emphasizing mathematical rigor. Computationally oriented books in this area generally present algorithms alone, and expect readers to perform computations by hand, and are often written in traditional computer languages, such as Basic, Fortran or Pascal. This book, on the other hand, is the first text to use Mathematica to develop a thorough understanding of optimization algorithms, fully exploiting Mathematica's symbolic, numerical and graphic capabilities.