Download or read book Large scale Simulation for Embodied Perception and Robot Learning written by Fei Xia (Researcher in computer vision) and published by . This book was released on 2021 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Being able to perceive and interact with complex human environments is an important yet challenging problem in robotics for decades. Learning active perception and sensorimotor control by interacting with the physical world is cumbersome as existing algorithms are too slow to learn in real-time, and robots are fragile and costly. This has given rise to learning in simulation, and to make progress on this problem, efficient simulation infrastructure needs to be developed to support interactive and long-horizon tasks, and sample-efficient learning algorithms need to be developed to solve these tasks. In this dissertation, I present two lines of work contributing to these topics. The first line of work is to create large-scale, realistic, and interactive simulation environments, including Gibson Environment and iGibson. Gibson Environment is proposed for learning real-world perception for active agents. Gibson Environment is built from the real world and reflects its semantic complexity. It has a neural network-based renderer and a mechanism named ``Goggle" to ensure no need to further domain adaptation before deployment of results in the real world. Gibson Environment significantly improves pixel-level realism over existing simulation environments. To build upon Gibson Environment and improve the physical realism of the simulation, I propose iGibson, a simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes. The simulated scenes are replicas of 3D scanned real-world homes, aligning the distribution of objects and layout to those of the real world. Novel long horizon problems including interactive navigation and mobile manipulation can be defined in this environment, and I show evidence that solutions can be transferred to the real world. The second line of work studies reinforcement learning (RL) for long-horizon robotics problems enabled by the interactive simulation environments. First, I introduce the interactive navigation problem and associated metrics. I leverage model-free RL algorithms to solve the proposed interactive navigation problems. Second, to solve challenging tasks in fully interactive simulation environments and improve sample efficiency of RL, I propose ReLMoGen, a framework to integrate motion generation into RL. I propose to lift the action space from joint control signals to motion generation subgoals. By lifting the action space and leveraging sampling-based motion planners, I can efficiently use RL to solve complex long-horizon tasks that existing RL methods cannot solve in the original action space.

Download or read book Robot Learning from Interactions with Physics realistic Environment Constructing Big Task Platform for Training AI Agents written by Xu Xie and published by . This book was released on 2021 with total page 124 pages. Available in PDF, EPUB and Kindle. Book excerpt: Robot learning from interactions is a crucial topic in the joint field of computer vision, robotics, and machine learning. Interactions are ubiquitous in daily life, concrete instances comprise object-object, robot-object, and robot-robot interactions. Learning from interactions to an intelligent robot system is important because it helps the robot to generate a sense of physics, meanwhile planning and acting reasonably. To achieve this purpose, one primary challenge that remains in the community is the absence of dataset that can be leveraged to study the diverse categories of interactions. To create those datasets, the interaction data should be realistic such that it reflects the underlying physical process. Further, we argue that learning interactions through simulations is a promising approach to synthesize and scale up diverse forms of interactions. This dissertation focuses on robot learning from interactions in Mixed Reality (MR) as well as leveraging the state-of-the-art physical simulation to construct virtual environments to afford Big Tasks. There are four major contributions along this pathway: 1. Robot learning object manipulation skills from human demonstrations. Instead of directly learning from a robot-object manipulation dataset that is hard to generalize, we alternatively seek an approach to create a human-object manipulation dataset and let the robot learn from the demonstration. We claim that the key attribute of building such dataset embodies the realistic hand-object interaction that involves a setup that can faithfully capture the fine-grained raw motion signals. This leads us to develop a tactile glove system and collect informative spatial-temporal sensory data during hand manipulations. An event parsing pipeline is proposed upon the hand interactions that are transferable to the robot's end and learn the manipulation skill. 2. A virtual testbed to construct rich interactive tasks. The major limitation of collecting real-world interaction data can be summarized as three folds: i) a specific setup is needed to trace one form of interaction, ii) amount of efforts need to spend on data cleaning and labeling, and iii) a single dataset is not capable to capture different modalities of interactions at the same time. To overcome those issues, we propose and develop a virtual testbed, VRGym platform, for realistic human-robot interactive tasks (Big Tasks). In VRGym, the pipelines we developed are able to synthesize diverse photo-realistic 3D scenes that incorporate various forms of interactions through physics-based simulation. Given available rich interactions, we expect to grow a general-purpose agent from the interactive tasks and advance the research areas of robotics, machine learning as well as cognitive science. 3. Robot learning from imperfect demonstrations --- small data. In the area of learning from demonstration, interacting with objects, one essential element is the creation of expert demonstrations. However, non-trivial efforts are needed when collecting those demonstrations and a large portion of them contains failure cases. We develop the demonstration setup for learning objects grasping skills upon VRGym platform with VR human interfaces. Human performers interact with the virtual scene by teleoperating the virtual robot arm. At the same time, the demonstration is evaluated through physics simulation such that even a perfect task plan may fail during the execution. Given the sparsity of demonstrations, we think the failed ones are valuable in addition to the perfect demonstration. This enlightens us to exploit the implicit characteristics of small data in the presence of imperfect demonstrations. 4. A game platform for large-scale social interactions. Social interactions are another important branch that goes beyond physical only interactions. To develop a general-purpose agent, it has to properly infer other agents motion or intentions and applies socially acceptable behaviors when interacting in the scene. Inspired by those facts, we leverage a popular computer game platform, Grand Theft Auto (GTA), to automatically construct fruitful realistic social interactions in the simulated urban scenarios. The city transportation system, including vehicles and pedestrians, can be fully controlled by the developed modding scripts. The GTA platform is a supplement to VRGym that extends robot learning from interactions to a larger scale. We utilize it to synthesize multi-vehicle driving scenarios and study the problem of trajectories prediction as to the basis of intentions inference. We highlight the safety aspect by predicting collision-free trajectories that accord with the social norm for vehicle driving.

Download or read book A Concise Introduction to Models and Methods for Automated Planning written by Hector Radanovic and published by Springer Nature. This book was released on 2022-05-31 with total page 132 pages. Available in PDF, EPUB and Kindle. Book excerpt: Planning is the model-based approach to autonomous behavior where the agent behavior is derived automatically from a model of the actions, sensors, and goals. The main challenges in planning are computational as all models, whether featuring uncertainty and feedback or not, are intractable in the worst case when represented in compact form. In this book, we look at a variety of models used in AI planning, and at the methods that have been developed for solving them. The goal is to provide a modern and coherent view of planning that is precise, concise, and mostly self-contained, without being shallow. For this, we make no attempt at covering the whole variety of planning approaches, ideas, and applications, and focus on the essentials. The target audience of the book are students and researchers interested in autonomous behavior and planning from an AI, engineering, or cognitive science perspective. Table of Contents: Preface / Planning and Autonomous Behavior / Classical Planning: Full Information and Deterministic Actions / Classical Planning: Variations and Extensions / Beyond Classical Planning: Transformations / Planning with Sensing: Logical Models / MDP Planning: Stochastic Actions and Full Feedback / POMDP Planning: Stochastic Actions and Partial Feedback / Discussion / Bibliography / Author's Biography

Download or read book Simulating Real world Human Activities with VirtualCity written by Jordan Ren and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Embodied environments act as a tool that enables various control tasks to be learned. Within these simulators, having realistic rendering and physics ensures that the sim2real gap for tasks isn't too large. Current embodied environments focus mainly on small-scale or low-level tasks, without the capability to learn large-scale diverse tasks, and often lack the realism for a small sim2real gap. To address the shortcomings of current simulators, we propose VirtualCity, a large-scale embodied environment that enables the learning of high-level planning tasks with photo-realistic rendering and realistic physics. To interact with VirtualCity, we provide a user-friendly Python API that allows the modification, control, and observation of the environment and its agents within. Building this realistic environment brings us closer to adapting models trained in simulation to solve real-world tasks.

Download or read book Embodiment in Socially Interactive Robots written by Eric Deng and published by Foundations and Trends (R) in Robotics. This book was released on 2019-01-30 with total page 118 pages. Available in PDF, EPUB and Kindle. Book excerpt: Socially interactive robots provide entertainment, information, and/or assistance; this last category is typically encompassed by socially assistive robotics. In all cases, such robots can achieve their primary functions without performing functional physical work. This monograph reviews the existing work that explores the role of physical embodiment in socially interactive robots. This class consists of robots that are not only capable of engaging in social interaction with humans, but are using primarily their social capabilities to perform their desired functions. This monograph explores the embodiment hypothesis that physical embodiment has a measurable effect on performance and perception of social interactions in socially interactive robotics. It presents a thorough review of existing work and analyzes existing results and approaches to embodiment to determine the current state of the embodiment hypothesis. This monograph is a comprehensive and in depth overview of embodiment in socially interactive robots that is a starting point for researchers and students beginning their own research in the area.

Download or read book Photorealistic Sensor Simulation for Perception driven Robotics Using Virtual Reality written by Winter Joseph Guerra and published by . This book was released on 2019 with total page 112 pages. Available in PDF, EPUB and Kindle. Book excerpt: In recent years, intensive research has centered around using small, perception-driven robotic systems (e.g. quadrotor vehicles) for complex tasks at operational speeds. Although much progress has been made towards that end in the fields of online-planning, fast-perception, and agile-control, most robotic systems are still confined to controlled laboratory settings due to cost, safety, and repeatability. In this thesis, we introduce a few novel contributions that we believe could assist the greater robotics community to bring their robotics systems out of the lab and into the real world. First, we introduce FlightGoggles, a photorealistic sensor simulator for perception-driven robotic vehicles. FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry and provides the ability to combine synthetic exteroceptive measurements generated in silico in real time and vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in flight in a motion-capture facility. Second, we present The Blackbird Dataset, a large-scale dataset for UAV perception in aggressive flight containing over 10 hours of flight data across 171 flights at velocities up to 13.8ms−1 in 5 environments with some dynamic elements. Third, we introduce a virtual reality framework for FlightGoggles that enables multi-agent or robot-human interaction in a safe manner by superimposing position data from multiple motion capture spaces into a unified virtual reality environment. Fourth, we propose an extension of FlightGoggles using augmented reality for aircraft-in-the-loop experiments that aims to aid sim-to-real transfer from simulated to real-world cameras. Lastly, we study applications of FlightGoggles in the greater robotics community through the AlphaPilot autonomous drone racing challenge and survey approaches and results from the top AlphaPilot teams, which may be of independent interest.

Download or read book The Embodied Brain Computational Mechanisms of Integrated Sensorimotor Interactions with a Dynamic Environment written by Mario Senden and published by Frontiers Media SA. This book was released on 2020-07-31 with total page 216 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Toward Quantifying the Simulation to Reality Difference for Autonomous Applications Reliant on Image based Perception written by Asher Elmquist and published by . This book was released on 2022 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Modeling and simulation of robotics requires the accurate modeling of cameras, which are widely relied on for perception, allowing robots to safely navigate their surroundings. The ability to simulate cameras within a complex, 3D virtual environment allows for inexpensive data generation with automatically labeled data, and in combination with other simulation tools, allows for closed-loop evaluation of an autonomy stack. These capabilities lead to inexpensive testing, removal of testing risk, and massive-scale evaluation of specific and safety-critical scenarios. These capabilities also enable inexpensive dataset generation for training existing algorithms for new environments, or developing new algorithms to improve perception capabilities. However, all these capabilities are predicated on the ability to accurately use simulation as a stand-in for reality, the failure of which has become known as the simulation-to-reality gap. Quantification of this gap is essential in understanding if, or to what degree, the simulation is sufficiently accurate. However, this quantification poses significant challenges due to the complexity of the environments and tasks for which simulation can be used. This dissertation presents a novel methodology which quantifies the difference between simulation and reality in a manner relevant for simulation of robotics. To accomplish this quantification, a perspective is proposed that defines what it means for simulation to predict reality and how sensor simulation should be judged. The proposed methodology leverages the target application to measure the difference in behavior when an evaluated algorithm is deployed in simulation and reality. This results in highly relevant quantification which can be understood as a measure of usefulness or predictiveness. The evaluated behavior is compared between simulation and reality for regions of similar content in the available data. By automatically comparing regions of similar content, the proposed methodology eases the challenge of validation by removing the need to obtain paired simulated and real data. The methodology can be used to measure improvements made in simulation and can be used for validation by determining a threshold for the simulation-to-reality difference, below which simulation should be considered validated for a specific use case. The validation and quantification research is enabled by a sensor simulation framework and autonomy research testbed (ART) developed as part of this research. The simulation framework allows for the modeling and simulation of cameras alongside a multi-physics simulation engine, to enable the simulation of autonomous robots and vehicles. While other simulation frameworks exist, this sensor simulation framework allows for the implementation of high-fidelity sensors as an application requires and additionally allows the fidelity to be adjusted when efficiency is required. The high-fidelity modeling leverages real-time hardware-accelerated ray-tracing techniques, physically-based rendering, and sampling of indirect light through global illumination. This framework is focused on allowing camera component models to be easily added as task-specific phenomena dictate. ART, which includes a simple autonomy stack and physical vehicle, complements the simulation framework by allowing real-world testing and data collection to be compared with simulation. This enables direct research into the simulation-to-reality differences using the same autonomy stack in simulation and reality. ART and the sensor simulation framework are free and open source, allowing other researchers to leverage the capabilities provided as part of this dissertation.

Download or read book Digital Human Modeling and Applications in Health Safety Ergonomics and Risk Management Human Body Motion and Behavior written by Vincent G. Duffy and published by Springer Nature. This book was released on 2021-07-03 with total page 377 pages. Available in PDF, EPUB and Kindle. Book excerpt: This two-volume set LNCS 12777 and 12778 constitutes the thoroughly refereed proceedings of the 12th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management, DHM 2021, which was held virtually as part of the 23rd HCI International Conference, HCII 2021, in July 2021. The total of 1276 papers and 241 posters included in the 39 HCII 2021 proceedings volumes was carefully reviewed and selected from 5222 submissions. DHM 2021 includes a total of 56 papers; they were organized in topical sections named: Part I, Human Body, Motion and Behavior: Ergonomics, human factors and occupational health; human body and motion modeling; and language, communication and behavior modeling. Part II, AI, Product and Service: Rethinking healthcare; artificial intelligence applications and ethical issues; and digital human modeling in product and service design.

Download or read book RoboCup 2013 Robot World Cup XVII written by Sven Behnke and published by Springer. This book was released on 2014-07-16 with total page 701 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book includes the thoroughly refereed post-conference proceedings of the 17th Annual RoboCup International Symposium, held in Eindhoven, The Netherlands, in June 2013. The 20 revised papers presented together with 11 champion team papers, 3 best paper awards, 11 oral presentations, and 19 special track on open-source hard- and software papers were carefully reviewed and selected from 78 submissions. The papers present current research and educational activities within the fields of robotics and artificial intelligence with a special focus to robot hardware and software, perception and action, robotic cognition and learning, multi-robot systems, human-robot interaction, education and edutainment, and applications.

Download or read book Radical Embodied Cognitive Science written by Anthony Chemero and published by MIT Press. This book was released on 2011-08-19 with total page 269 pages. Available in PDF, EPUB and Kindle. Book excerpt: A proposal for a new way to do cognitive science argues that cognition should be described in terms of agent-environment dynamics rather than computation and representation. While philosophers of mind have been arguing over the status of mental representations in cognitive science, cognitive scientists have been quietly engaged in studying perception, action, and cognition without explaining them in terms of mental representation. In this book, Anthony Chemero describes this nonrepresentational approach (which he terms radical embodied cognitive science), puts it in historical and conceptual context, and applies it to traditional problems in the philosophy of mind. Radical embodied cognitive science is a direct descendant of the American naturalist psychology of William James and John Dewey, and follows them in viewing perception and cognition to be understandable only in terms of action in the environment. Chemero argues that cognition should be described in terms of agent-environment dynamics rather than in terms of computation and representation. After outlining this orientation to cognition, Chemero proposes a methodology: dynamical systems theory, which would explain things dynamically and without reference to representation. He also advances a background theory: Gibsonian ecological psychology, “shored up” and clarified. Chemero then looks at some traditional philosophical problems (reductionism, epistemological skepticism, metaphysical realism, consciousness) through the lens of radical embodied cognitive science and concludes that the comparative ease with which it resolves these problems, combined with its empirical promise, makes this approach to cognitive science a rewarding one. “Jerry Fodor is my favorite philosopher,” Chemero writes in his preface, adding, “I think that Jerry Fodor is wrong about nearly everything.” With this book, Chemero explains nonrepresentational, dynamical, ecological cognitive science as clearly and as rigorously as Jerry Fodor explained computational cognitive science in his classic work The Language of Thought.

Download or read book Trust in Human Robot Interaction written by Chang S. Nam and published by Academic Press. This book was released on 2020-11-17 with total page 616 pages. Available in PDF, EPUB and Kindle. Book excerpt: Trust in Human-Robot Interaction addresses the gamut of factors that influence trust of robotic systems. The book presents the theory, fundamentals, techniques and diverse applications of the behavioral, cognitive and neural mechanisms of trust in human-robot interaction, covering topics like individual differences, transparency, communication, physical design, privacy and ethics. - Presents a repository of the open questions and challenges in trust in HRI - Includes contributions from many disciplines participating in HRI research, including psychology, neuroscience, sociology, engineering and computer science - Examines human information processing as a foundation for understanding HRI - Details the methods and techniques used to test and quantify trust in HRI

Download or read book Cognition and Multi Agent Interaction written by Ron Sun and published by Cambridge University Press. This book was released on 2006 with total page 456 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book explores the intersection between individual cognitive modeling and modeling of multi-agent interaction.

Download or read book Neural Computation in Embodied Closed Loop Systems for the Generation of Complex Behavior From Biology to Technology written by Poramate Manoonpong and published by Frontiers Media SA. This book was released on 2018-10-11 with total page 278 pages. Available in PDF, EPUB and Kindle. Book excerpt: How can neural and morphological computations be effectively combined and realized in embodied closed-loop systems (e.g., robots) such that they can become more like living creatures in their level of performance? Understanding this will lead to new technologies and a variety of applications. To tackle this research question, here, we bring together experts from different fields (including Biology, Computational Neuroscience, Robotics, and Artificial Intelligence) to share their recent findings and ideas and to update our research community. This eBook collects 17 cutting edge research articles, covering neural and morphological computations as well as the transfer of results to real world applications, like prosthesis and orthosis control and neuromorphic hardware implementation.

Download or read book Robotics written by Nicholas Roy and published by MIT Press. This book was released on 2013-07-05 with total page 501 pages. Available in PDF, EPUB and Kindle. Book excerpt: Robotics: Science and Systems VIII spans a wide spectrum of robotics, bringing together contributions from researchers working on the mathematical foundations of robotics, robotics applications, and analysis of robotics systems.

Download or read book Computer Vision ECCV 2022 written by Shai Avidan and published by Springer Nature. This book was released on 2022-10-22 with total page 785 pages. Available in PDF, EPUB and Kindle. Book excerpt: The 39-volume set, comprising the LNCS books 13661 until 13699, constitutes the refereed proceedings of the 17th European Conference on Computer Vision, ECCV 2022, held in Tel Aviv, Israel, during October 23–27, 2022. The 1645 papers presented in these proceedings were carefully reviewed and selected from a total of 5804 submissions. The papers deal with topics such as computer vision; machine learning; deep neural networks; reinforcement learning; object recognition; image classification; image processing; object detection; semantic segmentation; human pose estimation; 3d reconstruction; stereo vision; computational photography; neural networks; image coding; image reconstruction; object recognition; motion estimation.

Download or read book Biologically Inspired Cognitive Architectures 2010 written by BICA Society. Annual Meeting and published by IOS Press. This book was released on 2010 with total page 264 pages. Available in PDF, EPUB and Kindle. Book excerpt: "This book presents the proceedings of the First International Conference on Biologically Inspired Cognitive Architectures (BICA 2010), which is also the First Annual Meeting of the BICA Society. A cognitive architecture is a computational framework for the design of intelligent, even conscious, agents. It may draw inspiration from many sources, such as pure mathematics, physics or abstract theories of cognition. A biologically inspired cognitive architecture (BICA) is one which incorporates formal mechanisms from computational models of human and animal cognition, which currently provide the only physical examples with the robustness, flexibility, scalability and consciousness that artificial intelligence aspires to achieve. The BICA approach has several different goals: the broad aim of creating intelligent software systems without focusing on any one area of application; attempting to accurately simulate human behavior or gain an understanding of how the human mind works, either for purely scientific reasons or for applications in a variety of domains; understanding how the brain works at a neuronal and sub-neuronal level; or designing artificial systems which can perform the cognitive tasks important to practical applications in human society, and which at present only humans are capable of. The papers presented in this volume reflect the cross-disciplinarity and integrative nature of the BICA approach and will be of interest to anyone developing their own approach to cognitive architectures. Many insights can be found here for inspiration or to import into one's own architecture, directly or in modified form."--Publisher description.