Download or read book Nonhomogeneous Markov Deccision Processes Linked by Side Constraints written by Christopher L. Morse and published by . This book was released on 1997 with total page 28 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Constrained Markov Decision Processes written by Eitan Altman and published by CRC Press. This book was released on 1999-03-30 with total page 260 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction. The book is then divided into three sections that build upon each other. The first part explains the theory for the finite state space. The author characterizes the set of achievable expected occupation measures as well as performance vectors, and identifies simple classes of policies among which optimal policies exist. This allows the reduction of the original dynamic into a linear program. A Lagranian approach is then used to derive the dual linear program using dynamic programming techniques. In the second part, these results are extended to the infinite state space and action spaces. The author provides two frameworks: the case where costs are bounded below and the contracting framework. The third part builds upon the results of the first two parts and examines asymptotical results of the convergence of both the value and the policies in the time horizon and in the discount factor. Finally, several state truncation algorithms that enable the approximation of the solution of the original control problem via finite linear programs are given.
Download or read book Markov Decision Processes with Policy Constraints written by John Nafeh and published by . This book was released on 1976 with total page 338 pages. Available in PDF, EPUB and Kindle. Book excerpt: This work is concerned with Markov Decision Processes with policy constraints. The selection of an optimum stationary policy for such processes, in the absence of policy constraints, is a problem which has received a great deal of attention, and has been satisfactorily solved. Relatively little attention has been given to the case when policy constraints are present or to the formulation of such constraints. Optimum policy sensitivity analysis is also a subject in which little has been achieved. Towards those ends, this work makes three major contributions. First, policy constraints are formulated and categorized. Secondly, a computationally efficient iterative algorithm is developed for selecting the optimum policy for completely ergodic, infinite time horizon Markov Decision Processes with policy constraints for both the risk-indifferent and risk-sensitive cases. Finally, the sensitivity of optimum policies to the policy constraints is analyzed by using the algorithm to compute the value of removing a constraint or a group of constraints. (Author).
Download or read book Constrained Markov Decision Processes written by E. Altman and published by . This book was released on 1995 with total page 115 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Stochastic Equipment Replacement with Budget Constraints written by Christopher L. Morse and published by . This book was released on 1997 with total page 220 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book A Stopping Rule for Forecast Horizon In Nonhomogeneous Markov Decision Procesess written by James Bean, Wallace Hopp, Izak Duenyas and published by . This book was released on 1991 with total page 26 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Continuous Time Markov Decision Processes written by Xianping Guo and published by Springer Science & Business Media. This book was released on 2009-09-18 with total page 240 pages. Available in PDF, EPUB and Kindle. Book excerpt: Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.
Download or read book Continuous Time Markov Decision Processes written by Alexey Piunovskiy and published by Springer Nature. This book was released on 2020-11-09 with total page 605 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a systematic and rigorous treatment of continuous-time Markov decision processes, covering both theory and possible applications to queueing systems, epidemiology, finance, and other fields. Unlike most books on the subject, much attention is paid to problems with functional constraints and the realizability of strategies. Three major methods of investigations are presented, based on dynamic programming, linear programming, and reduction to discrete-time problems. Although the main focus is on models with total (discounted or undiscounted) cost criteria, models with average cost criteria and with impulsive controls are also discussed in depth. The book is self-contained. A separate chapter is devoted to Markov pure jump processes and the appendices collect the requisite background on real analysis and applied probability. All the statements in the main text are proved in detail. Researchers and graduate students in applied probability, operational research, statistics and engineering will find this monograph interesting, useful and valuable.
Download or read book Examples In Markov Decision Processes written by Alexey B Piunovskiy and published by World Scientific. This book was released on 2012-09-21 with total page 308 pages. Available in PDF, EPUB and Kindle. Book excerpt: This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was to collect them together in one reference book which should be considered as a complement to existing monographs on Markov decision processes.The book is self-contained and unified in presentation.The main theoretical statements and constructions are provided, and particular examples can be read independently of others. Examples in Markov Decision Processes is an essential source of reference for mathematicians and all those who apply the optimal control theory to practical purposes. When studying or using mathematical methods, the researcher must understand what can happen if some of the conditions imposed in rigorous theorems are not satisfied. Many examples confirming the importance of such conditions were published in different journal articles which are often difficult to find. This book brings together examples based upon such sources, along with several new ones. In addition, it indicates the areas where Markov decision processes can be used. Active researchers can refer to this book on applicability of mathematical methods and theorems. It is also suitable reading for graduate and research students where they will better understand the theory.
Download or read book Denumerable state nonhomogeneous markov decision processes written by JAMES C. BEAN and published by . This book was released on 1989 with total page 20 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Competitive Markov Decision Processes written by Jerzy Filar and published by Springer. This book was released on 1996-11-15 with total page 394 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book is intended as a text covering the central concepts and techniques of Competitive Markov Decision Processes. It is an attempt to present a rig orous treatment that combines two significant research topics: Stochastic Games and Markov Decision Processes, which have been studied exten sively, and at times quite independently, by mathematicians, operations researchers, engineers, and economists. Since Markov decision processes can be viewed as a special noncompeti tive case of stochastic games, we introduce the new terminology Competi tive Markov Decision Processes that emphasizes the importance of the link between these two topics and of the properties of the underlying Markov processes. The book is designed to be used either in a classroom or for self-study by a mathematically mature reader. In the Introduction (Chapter 1) we outline a number of advanced undergraduate and graduate courses for which this book could usefully serve as a text. A characteristic feature of competitive Markov decision processes - and one that inspired our long-standing interest - is that they can serve as an "orchestra" containing the "instruments" of much of modern applied (and at times even pure) mathematics. They constitute a topic where the instruments of linear algebra, applied probability, mathematical program ming, analysis, and even algebraic geometry can be "played" sometimes solo and sometimes in harmony to produce either beautifully simple or equally beautiful, but baroque, melodies, that is, theorems.
Download or read book Constrained Markov Decision Chains written by Stanford University. Department of Operations Research and published by . This book was released on 1971 with total page 16 pages. Available in PDF, EPUB and Kindle. Book excerpt: Consider a finite state and action discrete time parameter Markov decision chains. The objective is to provide an algorithm for finding a policy that minimizes the long run expected average cost when there are linear side conditions on the limit points of the expected state-action frequencies. This problem has been solved previously only for the case where every deterministic stationary policy has at most one ergodic class. The note removes that restriction by applying the Dantzig-Wolfe decomposition principle. (Author).
Download or read book A NEW OPTIMALITY CRITERION FOR NON HOMOGENOUS MARKOV DECISION PROCESSES written by Wallace J. Hopp, James C. Bean, Robert L. Smith and published by . This book was released on 1986 with total page 18 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Markovian Decision Processes written by Hisashi Mine and published by Elsevier Publishing Company. This book was released on 1970 with total page 166 pages. Available in PDF, EPUB and Kindle. Book excerpt: Markovian decision processes with discounting; Markovian decision processes with no discouting; Dynamic programming viewpoint of markovian decision processes; Semi-markovian decision processes; Generalized markovian decision processes; The principle of contraction mappings in markovian decision processes.
Download or read book Markov Decision Processes with Their Applications written by Qiying Hu and published by Springer. This book was released on 2010-11-19 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Put together by two top researchers in the Far East, this text examines Markov Decision Processes - also called stochastic dynamic programming - and their applications in the optimal control of discrete event systems, optimal replacement, and optimal allocations in sequential online auctions. This dynamic new book offers fresh applications of MDPs in areas such as the control of discrete event systems and the optimal allocations in sequential online auctions.
Download or read book rolling horizon procedures in nonhomogeneous markov decision processes written by jeffrey m. alden and published by . This book was released on 1987 with total page 22 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book On Constrained Optimization of the Klimov Network and Related Markov Decision Processes written by A. M. Makowski and published by . This book was released on 1991 with total page 17 pages. Available in PDF, EPUB and Kindle. Book excerpt:
