EBookClubs

Read Books & Download eBooks Full Online

EBookClubs

Read Books & Download eBooks Full Online

Book Development and Evaluation of an Arterial Adaptive Traffic Signal Control System Using Reinforcement Learning

Download or read book Development and Evaluation of an Arterial Adaptive Traffic Signal Control System Using Reinforcement Learning written by Yuanchang Xie and published by . This book was released on 2010 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation develops and evaluates a new adaptive traffic signal control system for arterials. This control system is based on reinforcement learning, which is an important research area in distributed artificial intelligence and has been extensively used in many applications including real-time control. In this dissertation, a systematic comparison between the reinforcement learning control methods and existing adaptive traffic control methods is first presented from the theoretical perspective. This comparison shows both the connections between them and the benefits of using reinforcement learning. A Neural-Fuzzy Actor-Critic Reinforcement Learning (NFACRL) method is then introduced for traffic signal control. NFACRL integrates fuzzy logic and neural networks into reinforcement learning and can better handle the curse of dimensionality and generalization problems associated with ordinary reinforcement learning methods. This NFACRL method is first applied to isolated intersection control. Two different implementation schemes are considered. The first scheme uses a fixed phase sequence and variable cycle length, while the second one optimizes phase sequence in real time and is not constrained to the concept of cycle. Both schemes are further extended for arterial control, with each intersection being controlled by one NFACRL controller. Different strategies used for coordinating reinforcement learning controllers are reviewed, and a simple but robust method is adopted for coordinating traffic signals along the arterial. The proposed NFACRL control system is tested at both isolated intersection and arterial levels based on VISSIM simulation. The testing is conducted under different traffic volume scenarios using real-world traffic data collected during morning, noon, and afternoon peak periods. The performance of the NFACRL control system is compared with that of the optimized pre-timed and actuated control. Testing results based on VISSIM simulation show that the proposed NFACRL control has very promising performance. It outperforms optimized pre-timed and actuated control in most cases for both isolated intersection and arterial control. At the end of this dissertation, issues on how to further improve the NFACRL method and implement it in real world are discussed.

Book Development and Evaluation of a Multi agent Based Neuro fuzzy Arterial Traffic Signal Control System

Download or read book Development and Evaluation of a Multi agent Based Neuro fuzzy Arterial Traffic Signal Control System written by Yunlong Zhang and published by . This book was released on 2007 with total page 126 pages. Available in PDF, EPUB and Kindle. Book excerpt: Arterial traffic signal control is a very important aspect of traffic management system. Efficient arterial traffic signal control strategy can reduce delay, stops, congestion, and pollution and save travel time. Commonly used pre-timed or traffic actuated signal control do not have the capability to fully respond to real-time traffic demand and pattern changes. Although some of the well-known adaptive control systems have shown advantageous over the traditional per-timed and actuated control strategies, their centralized architecture makes the maintenance, expansion, and upgrade difficult and costly.

Book Improving Traffic Safety and Efficiency by Adaptive Signal Control Based on Deep Reinforcement Learning

Download or read book Improving Traffic Safety and Efficiency by Adaptive Signal Control Based on Deep Reinforcement Learning written by Yaobang Gong and published by . This book was released on 2020 with total page 126 pages. Available in PDF, EPUB and Kindle. Book excerpt: As one of the most important Active Traffic Management strategies, Adaptive Traffic Signal Control (ATSC) helps improve traffic operation of signalized arterials and urban roads by adjusting the signal timing to accommodate real-time traffic conditions. Recently, with the rapid development of artificial intelligence, many researchers have employed deep reinforcement learning (DRL) algorithms to develop ATSCs. However, most of them are not practice-ready. The reasons are two-fold: first, they are not developed based on real-world traffic dynamics and most of them require the complete information of the entire traffic system. Second, their impact on traffic safety is always a concern by researchers and practitioners but remains unclear. Aiming at making the DRL-based ATSC more implementable, existing traffic detection systems on arterials were reviewed and investigated to provide high-quality data feeds to ATSCs. Specifically, a machine-learning frameworks were developed to improve the quality of and pedestrian and bicyclist’s count data. Then, to evaluate the effectiveness of DRL-based ATSC on the real-world traffic dynamics, a decentralized network-level ATSC using multi-agent DRL was developed and evaluated in a simulated real-world network. The evaluation results confirmed that the proposed ATSC outperforms the actuated traffic signals in the field in terms of travel time reduction. To address the potential safety issue of DRL based ATSC, an ATSC algorithm optimizing simultaneously both traffic efficiency and safety was proposed based on multi-objective DRL. The developed ATSC was tested in a simulated real-world intersection and it successfully improved traffic safety without deteriorating efficiency. In conclusion, the proposed ATSCs are capable of effectively controlling real-world traffic and benefiting both traffic efficiency and safety.

Book Data driven Adaptive Traffic Signal Control Via Deep Reinforcement Learning

Download or read book Data driven Adaptive Traffic Signal Control Via Deep Reinforcement Learning written by Tian Tan and published by . This book was released on 2020 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Adaptive traffic signal control (ATSC) system serves a significant role for relieving urban traffic congestion. The system is capable of adjusting signal phases and timings of all traffic lights simultaneously according to real-time traffic sensor data, resulting in a better overall traffic management and an improved traffic condition on road. In recent years, deep reinforcement learning (DRL), one powerful paradigm in artificial intelligence (AI) for sequential decision-making, has drawn great attention from transportation researchers. The following three properties of DRL make it very attractive and ideal for the next generation ATSC system: (1) model-free: DRL reasons about the optimal control strategies directly from data without making additional assumptions on the underlying traffic distributions and traffic flows. Compared with traditional traffic optimization methods, DRL avoids the cumbersome formulation of traffic dynamics and modeling; (2) self-learning: DRL self-learns the signal control knowledge from traffic data with minimal human expertise; (3) simple data requirement: by using large nonlinear neural networks as function approximators, DRL has enough representation power to map directly from simple traffic measurements, e.g. queue length and waiting time, to signal control policies. This thesis focuses on building data-driven and adaptive controllers via deep reinforcement learning for large-scale traffic signal control systems. In particular, the thesis first proposes a hierarchical decentralized-to-centralized DRL framework for large-scale ATSC to better coordinate multiple signalized intersections in the traffic system. Second, the thesis introduces efficient DRL with efficient exploration for ATSC to greatly improve sample complexity of DRL algorithms, making them more suitable for real-world control systems. Furthermore, the thesis combines multi-agent system with efficient DRL to solve large-scale ATSC problems that have multiple intersections. Finally, the thesis presents several algorithmic extensions to handle complex topology and heterogeneous intersections in real-world traffic networks. To gauge the performance of the presented DRL algorithms, various experiments have been conducted and included in the thesis both on small-scale and on large-scale simulated traffic networks. The empirical results have demonstrated that the proposed DRL algorithms outperform both rule-based control policy and commonly-used off-the-shelf DRL algorithms by a significant margin. Moreover, the proposed efficient MARL algorithms have achieved the state-of-the-art performance with improved sample-complexity for large-scale ATSC.

Book Development and Evaluation of Model based Adaptive Signal Control for Congested Arterial Traffic

Download or read book Development and Evaluation of Model based Adaptive Signal Control for Congested Arterial Traffic written by Gang Liu and published by . This book was released on 2015 with total page 139 pages. Available in PDF, EPUB and Kindle. Book excerpt: Under congested conditions, the road traffic states of different arterial links will interact with each other; therefore, it is necessary to understand the behavior of traffic corridors and to investigate corridor-wide traffic coordinated control strategies. In order to achieve this, traffic flow models are applied in signal control to predict future traffic states. Optimization tools are used to search for the best sequence of future control decisions, based on predictions by traffic flow models. A number of model-based adaptive control strategies have been presented in the literature and have been proved effective in practice. However, most studies have modeled the traffic dynamic either at a link-based level or at an individual movement-based level. Moreover, the efficiency of corridor-wide coordination algorithms for congested large-scale networks still needs to be further improved. A hierarchical control structure is developed to divide the complex control problem into different control layers: the highest level optimizes the cycle length, the mid layer optimizes the offsets, and the Model Predictive Control (MPC) procedure is implemented in the lowest layer to optimize the split. In addition, there is an extra multi-modal priority control layer to provide priority for different travel modes. Firstly, MPC is applied to optimize the signal timing plans for arterial traffic. The objectives are to increase the throughput. A hybrid urban traffic flow model is proposed to provide relatively accurate predictions of the traffic state dynamic, which is capable of simulating queue evolutions among different lane groups in a specific link. Secondly, this study expands the dynamic queue concept to the corridor-wide coordination problem. The ideal offset and boundary offsets to avoid spillback and starvation are found based on the shockwave profiles at each signalized intersection. A new multi-objective optimization model based on the preemptive goal programming is proposed to find the optimal offset. Thirdly, the priority control problem is formulated into a multi-objective optimization model, which is solved with a Non-dominated Sorting Genetic Algorithm. Pareto-optimal front results are presented to evaluate the trade-off among different objectives and the most appropriate solution is chosen with high-level information. Performance of the new adaptive controller is verified with software-in-the-loop simulation. The applied simulation environment contains VISSIM with the ASC/3 module as the simulation environment and the control system as the solver. The simulation test bed includes two arterial corridors in Edmonton, Alberta. The simulation network was well calibrated and validated. The simulation results show that the proposed adaptive control methods outperform actuated control in increasing throughput, decreasing delay, and preventing queue spillback.

Book Reinforcement Learning Based Traffic Signal Control for Signalized Intersections

Download or read book Reinforcement Learning Based Traffic Signal Control for Signalized Intersections written by Dunhao Zhong and published by . This book was released on 2021 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Vehicles have become an indispensable means of transportation to ensure people's travel and living materials. However, with the increasing number of vehicles, traffic congestion has become severe and caused a lot of social wealth loss. Therefore, improving the efficiency of transport management is one of the focuses of current academic circles. Among the research in transport management, traffic signal control (TSC) is an effective way to alleviate traffic congestion at signalized intersections. Existing works have successfully applied reinforcement learning (RL) techniques to achieve a higher TSC efficiency. However, previous work remains several challenges in RL-based TSC methods. First, existing studies used a single scaled reward to frame multiple objectives. Nevertheless, the single scaled reward has lower scalability to assess the controller's performance on different objectives, resulting in higher volatility on different traffic criteria. Second, adaptive traffic signal control provides dynamic traffic timing plans according to unforeseeable traffic conditions. Such characteristic prohibits applying the existing eco-driving strategies whose strategies are generated based on foreseeable and prefixed traffic timing plans. To address the challenges, in this thesis, we propose to design a new RL-TSC framework along with an eco-driving strategy to improve the TSC's efficiency on multiple objectives and further smooth the traffic flows. Moreover, to achieve effective management of the system-wide traffic flows, current researches tend to focus on the design of collaborative traffic signal control methods. However, the existing collaboration-based methods often ignore the impact of transmission delay for exchanging traffic flow information on the system. Inspired by the state-of-the-art max-pressure control in the traffic signal control area, we propose a new efficient RL-based cooperative TSC scheme by improving the reward and state representation based on the max-pressure control method and developing an agent that can address the data transmission delay issue by decreasing the discrepancy between the real-time and delayed traffic conditions. To evaluate the performance of our proposed work more accurately, in addition to the synthetic scenario, we also conducted an experiment based on the real-world traffic data recorded in the City of Toronto. We demonstrate that our method surpassed the performance of the previous traffic signal control methods through comprehensive experiments.

Book Adaptive Traffic Signal Control Using Deep Reinforcement Learning for Network Traffic Incidents

Download or read book Adaptive Traffic Signal Control Using Deep Reinforcement Learning for Network Traffic Incidents written by Tianxin Li (M.S. in Engineering) and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Traffic signal control is an essential aspect of urban mobility that significantly impacts the efficiency and safety of transportation networks. Traditional traffic signal control systems rely on fixed-time or actuated signal timings, which may not adapt to the dynamic traffic demands and congestion patterns. Therefore, researchers and practitioners have increasingly turned to reinforcement learning (RL) techniques as a promising approach to improve the performance of traffic signal control. This dissertation investigates the application of RL algorithms to traffic signal control, aiming to optimize traffic flow and reduce congestion. The study develops a simulation model of a signalized intersection and trains RL agents to learn how to adjust signal timings based on real-time traffic conditions. The RL agents are designed to learn from experience and adapt to changing traffic patterns, thereby improving the efficiency of traffic flow, even for scenarios in which traffic incidents occur in the network. In this dissertation, the potential benefits of using RL algorithms to optimize traffic signal control in scenarios with and without traffic incidents were explored. To achieve this, an incident generation module was developed using the open-source traffic signal performance simulation framework that relies on the SUMO software. This module includes emergency response vehicles to mimic the realistic impact of traffic incidents and generates incidents randomly in the network. By exposing the RL agent to this environment, it can learn from the experience and optimize traffic signal control to reduce system delay. The study began with a single intersection scenario, where the DQN algorithm was modeled to form the RL agent traffic signal controller. To improve the training process and model performance, experience replay and target network were implemented to solve the limitations of DQN. Hyperparameter tuning was conducted to find the best parameter combination for the training process, and the results showed that DQN outperformed other controllers in terms of the system-wise and intersection-wise queue distribution and vehicle delay. The study was then extended to a small corridor with 2 intersections and a grid network (2x2 intersection), and the incident generation module was used to expose the RL agent to different traffic scenarios. Again, hyperparameter tuning was conducted, and the DQN model outperformed other controllers in terms of reducing congestion and improving the system performance. The robustness of the DQN performance was also tested with different demands, and the microsimulation results showed that the DQN performance was consistent. Overall, this study highlights the potential of RL algorithms to optimize traffic signal control in scenarios with and without traffic incidents. The incident generation module developed in this study provides a realistic environment for the RL agent to learn and adapt, leading to improved system performance and reduced congestion. In addition, hyperparameter tuning is essential to lay down a solid foundation for the RL training process

Book Deep Reinforcement Learning Approach to Multimodal Adaptive Traffic Signal Control

Download or read book Deep Reinforcement Learning Approach to Multimodal Adaptive Traffic Signal Control written by Soheil Mohamad Alizadeh Shabestary and published by . This book was released on 2019 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: With perpetually increasing demand for transportation as a result of continued urbanization and population growth, it is essential to increase the existing transportation infrastructure. Optimizing traffic signals in real time, although is one of the primary tools to increase the efficiency of our urban transportation networks, is a difficult task, due to the non-linearity and stochasticity of the traffic system. Deriving a simple model of the intersection in order to design an appropriate adaptive controller is extremely challenging, and traffic signal control falls under the challenging category of sequential decision-making processes. One of the best approaches to resolving issues around adaptive traffic signal control is reinforcement learning (RL), which is model-free and suitable for sequential decision-making problems. Conventional discrete RL algorithms suffer from the curse of dimensionality, slow training, and lack of generalization. Therefore, we focus on developing continuous RL-based (CRL) traffic signal controller that addresses these issues. Also, we propose a more advanced deep RL-based (DRL) traffic signal controller that can handle high-dimensional sensory inputs from newer traffic sensors such as radars and the emerging technology of Connected Vehicles. DRL traffic signal controller directly operates with highly-detailed sensory information and eliminates the need for traffic experts to extract concise state features from the raw data (e.g., queue lengths), a process that is both case-specific and limiting. Furthermore, DRL extracts what it needs from the more detailed inputs automatically and improves control performance. Finally, we introduce two multimodal RL-based traffic signal controllers (MCRL and MiND) that simultaneously optimize the delay for both transit and regular traffic, as public transit is the more sustainable mode of transportation in busy cities and downtown cores. The proposed controllers are tested using Paramics traffic microsimulator, and the results show the superiority of both CRL and DRL over other state-of-practice and state-of-the-art traffic signal controllers. In addition to the advantages of MiND, such as its multimodal capabilities, significantly faster convergence, smaller model, and elimination of the feature extraction process, our experimental results show significant improvements in travel times for both transit and regular traffic at the intersection level compared to the base cases.

Book Design and Evaluation of Real Time Adaptive Traffic Signal Control Algorithims PHD

Download or read book Design and Evaluation of Real Time Adaptive Traffic Signal Control Algorithims PHD written by Steven Gebhart Shelby and published by . This book was released on 2001 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Book Adaptive Traffic Signal Control Using Approximate Dynamic Programming

Download or read book Adaptive Traffic Signal Control Using Approximate Dynamic Programming written by C. Cai and published by . This book was released on 2010 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This thesis presents a study on an adaptive traffic signal controller for real-time operation. An approximate dynamic programming (ADP) algorithm is developed for controlling traffic signals at isolated intersection and in distributed traffic networks. This approach is derived from the premise that classic dynamic programming is computationally difficult to solve, and approximation is the second-best option for establishing sequential decision-making for complex process. The proposed ADP algorithm substantially reduces computational burden by using a linear approximation function to replace the exact value function of dynamic programming solution. Machine-learning techniques are used to improve the approximation progressively. Not knowing the ideal response for the approximation to learn from, we use the paradigm of unsupervised learning, and reinforcement learning in particular. Temporal-difference learning and perturbation learning are investigated as appropriate candidates in the family of unsupervised learning. We find in computer simulation that the proposed method achieves substantial reduction in vehicle delays in comparison with optimised fixed-time plans, and is competitive against other adaptive methods in computational efficiency and effectiveness in managing varying traffic. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised. The proposed ADP algorithm is in compliance with a range of discrete systems of resolution from 0.5 to 5 seconds per temporal step. This study demonstrates the readiness of the proposed approach for real-time operations at isolated intersections and the potentials for distributed network control.

Book Arterial Control and Integration

Download or read book Arterial Control and Integration written by G. Scott Rutherford and published by . This book was released on 1990 with total page 66 pages. Available in PDF, EPUB and Kindle. Book excerpt: