Download or read book Distributed Query Processing Over Fluctuating Streams written by Roland Kotto Kombi and published by . This book was released on 2018 with total page 136 pages. Available in PDF, EPUB and Kindle. Book excerpt: In a Big Data context, stream processing has become a very active research domain. In order to manage ephemeral data (Velocity) arriving at important rates (Volume), some specific solutions, denoted data stream management systems (DSMSs),have been developed. DSMSs take as inputs some queries, called continuous queries,defined on a set of data streams. Acontinuous query generates new results as long as new data arrive in input. In many application domains, data streams haveinput rates and distribution of values which change over time. These variations may impact significantly processingrequirements for each continuous query.This thesis takes place in the ANR project Socioplug (ANR-13-INFR-0003). In this context, we consider a collaborative platformfor stream processing. Each user can submit multiple continuous queries and contributes to the execution support of theplatform. However, as each processing unit supporting treatments has limited resources in terms of CPU and memory, asignificant increase in input rate may cause the congestion of the system. The problem is then how to adjust dynamicallyresource usage to processing requirements for each continuous query ? It raises several challenges : i) how to detect a need ofreconfiguration ? ii) when reconfiguring the system to avoid its congestion at runtime ?In this work, we are interested by the different processing steps involved in the treatment of a continuous query over adistributed infrastructure. From this global analysis, we extract mechanisms enabling dynamic adaptation of resource usage foreach continuous query. We focus on automatic parallelization, or auto-parallelization, of operators composing the executionplan of a continuous query. We suggest an original approach based on the monitoring of operators and an estimation ofprocessing requirements in near future. Thus, we can increase (scale-out), or decrease (scale-in) the parallelism degree ofoperators in a proactive many such as resource usage fits to processing requirements dynamically. Compared to a staticconfiguration defined by an expert, we show that it is possible to avoid the congestion of the system in many cases or to delay itin most critical cases. Moreover, we show that resource usage can be reduced significantly while delivering equivalentthroughput and result quality. We suggest also to combine this approach with complementary mechanisms for dynamic adaptation of continuous queries at runtime. These differents approaches have been implemented within a widely used DSMS and have been tested over multiple and reproductible micro-benchmarks.
Download or read book Relevant Query Answering over Streaming and Distributed Data written by Shima Zahmatkesh and published by Springer Nature. This book was released on 2020-01-21 with total page 128 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book examines the problem of relevant query answering over the Web and provides a comprehensive overview of relevant query answering over streaming and distributed data. In recent years, Web applications that combine highly dynamic data streams with data distributed over the Web to provide relevant answers have attracted increasing attention. Answering in a timely fashion, i.e., reactively, is one of the most important performance indicators, especially when the distributed data is evolving. The book proposes a solution that retains a local replica of the distributed data and offers various maintenance policies to refresh the replica over time. A limited refresh budget guarantees the reactiveness of the system. Focusing on stream processing and Semantic Web, it appeals to scientists and graduate students in the field.
Download or read book Dynamic Optimization and Migration of Continuous Queries Over Data Streams written by Yali Zhu and published by . This book was released on 2006 with total page 319 pages. Available in PDF, EPUB and Kindle. Book excerpt: Abstract: Continuous queries process real-time streaming data and output results in streams for a wide range of applications. Due to the fluctuating stream characteristics, a streaming database system needs to dynamically adapt query execution. This dissertation proposes novel solutions to continuous query adaptation in three core areas, namely dynamic query optimization, dynamic plan migration and partitioned query adaptation. Runtime query optimization needs to efficiently generate plans that satisfy both CPU and memory resource constraints. Existing work focus on minimizing intermediate query results, which decreases memory and CPU usages simultaneously. However, doing so cannot assure that both resource constraints are being satisfied, because memory and CPU can be either positively or negatively correlated. This part of the dissertation proposes efficient optimization strategies that utilize both types of correlations to search the entire query plan space in polynomial time when a typical exhaustive search would take at least exponential time. Extensive experimental evaluations have demonstrated the effectiveness of the proposed strategies. Dynamic plan migration is concerned with on-the-fly transition from one continuous plan to a semantically equivalent yet more efficient plan. It is a must to guarantee the continuation and repeatability of dynamic query optimization. However, this research area has been largely neglected in the current literature. The second part of this dissertation proposes migration STRategies that dynamically migrate continuous queries while guaranteeing the integrity of the query results, meaning there are no missing, duplicate or incorrect results. The extensive experimental evaluations show that the proposed strategies vary significantly in terms of output rates and memory usages given distinct system configurations and stream workloads. Partitioned query processing is effective to process continuous queries with large stateful operators in a distributed system. Dynamic load redistribution is necessary to balance uneven workload across machines due to changing stream properties. However, existing solutions generally assume static query plans without runtime query optimization. This part of the dissertation evaluates the benefits of applying query optimization in partitioned query processing and shows dramatic performance improvement of more than 300%. Several load balancing strategies are then proposed to consider the heterogeneity of plan shapes across machines caused by dynamic query optimization. The effectiveness of the proposed strategies is analyzed through extensive experiments using a cluster.
Download or read book Adaptive Query Processing written by Amol Deshpande and published by Now Publishers Inc. This book was released on 2007 with total page 156 pages. Available in PDF, EPUB and Kindle. Book excerpt: Adaptive Query Processing surveys the fundamental issues, techniques, costs, and benefits of adaptive query processing. It begins with a broad overview of the field, identifying the dimensions of adaptive techniques. It then looks at the spectrum of approaches available to adapt query execution at runtime - primarily in a non-streaming context. The emphasis is on simplifying and abstracting the key concepts of each technique, rather than reproducing the full details available in the papers. The authors identify the strengths and limitations of the different techniques, demonstrate when they are most useful, and suggest possible avenues of future research. Adaptive Query Processing serves as a valuable reference for students of databases, providing a thorough survey of the area. Database researchers will benefit from a more complete point of view, including a number of approaches which they may not have focused on within the scope of their own research.
Download or read book Managing Erratic Data Streams in the Distributed Environments written by Shanzhong Zhu and published by . This book was released on 2006 with total page 272 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Development of Distributed Systems from Design to Application and Maintenance written by Bessis, Nik and published by IGI Global. This book was released on 2012-12-31 with total page 387 pages. Available in PDF, EPUB and Kindle. Book excerpt: "This book is a collection of research on the strategies used in the design and development of distributed systems applications"--Provided by publisher.
Download or read book Advanced Query Processing written by Barbara Catania and published by Springer Science & Business Media. This book was released on 2012-07-28 with total page 355 pages. Available in PDF, EPUB and Kindle. Book excerpt: This research book presents key developments, directions, and challenges concerning advanced query processing for both traditional and non-traditional data. A special emphasis is devoted to approximation and adaptivity issues as well as to the integration of heterogeneous data sources. The book will prove useful as a reference book for senior undergraduate or graduate courses on advanced data management issues, which have a special focus on query processing and data integration. It is aimed for technologists, managers, and developers who want to know more about emerging trends in advanced query processing.
Download or read book Parallel and Distributed Processing and Applications written by Yi Pan and published by Springer. This book was released on 2005-10-19 with total page 1182 pages. Available in PDF, EPUB and Kindle. Book excerpt: Welcome to the proceedings of ISPA 2005 which was held in the city of Nanjing. Parallel computing has become a mainstream research area in computer science and the ISPA conference has become one of the premier forums for the presentation of new and exciting research on all aspects of parallel computing. We are pleased to present the proceedings for the 3rd International Symposium on Parallel and Distributed Processing and Applications (ISPA 2005), which comprises a collection of excellent technical papers, and keynote speeches. The papers accepted cover a wide range of exciting topics, including architectures, software, networking, and applications. The conference continues to grow and this year a record total of 968 manuscripts (including workshop submissions) were submitted for consideration by the Program Committee or workshops. From the 645 papers submitted to the main conference, the Program Committee selected only 90 long papers and 19 short papers in the program. Eight workshops complemented the outstanding paper sessions.
Download or read book UbiComp 2003 Ubiquitous Computing written by Anind K. Dey and published by Springer Science & Business Media. This book was released on 2003-09-29 with total page 385 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book constitutes the refereed proceedings of the 5th International Conference on Ubiquitous Computing, UbiComp 2003, held in Seattle, WA, USA in Ocotber 2003. The 16 revised full papers and 11 technical note papers presented were carefully reviewed and selected from a total of 153 submissions. The papers are organized in topical sections on location and space, modeling and inference, context awareness, new devices and technologies, domestic environments and healthcare, social aspects and privacy, and new interfaces.
Download or read book Data Management in Grids written by Jean-Marc Pierson and published by Springer Science & Business Media. This book was released on 2005 with total page 151 pages. Available in PDF, EPUB and Kindle. Book excerpt: "The LNCS series reports state-of-the-art results in computer science research, development, and education, at a high level and in both printed and electronic form. Enjoying tight cooperation with the R & D community, with numerous individuals, as well as with prestigious organizations and societies, LNCS has grown into the most comprehensive computer science research forum available." "More recently, several color-cover sublines have been added featuring, beyond a collection of papers, various added-value components; these sublines include: tutorials (textbook-like monographs or collections of lectures given at advanced courses); state-of-the-art surveys (offering complete and mediated coverage of a topic); and hot topics (introducing emergent topics to the broader community)." "In parallel to the printed book, each new volume is published electronically in LNCS Online."--BOOK JACKET.
Download or read book Query Processing on Data Streams written by Bernhard Stegmaier and published by . This book was released on 2006 with total page 200 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Principles of Distributed Database Systems written by M. Tamer Özsu and published by Springer Nature. This book was released on 2019-12-02 with total page 674 pages. Available in PDF, EPUB and Kindle. Book excerpt: The fourth edition of this classic textbook provides major updates. This edition has completely new chapters on Big Data Platforms (distributed storage systems, MapReduce, Spark, data stream processing, graph analytics) and on NoSQL, NewSQL and polystore systems. It also includes an updated web data management chapter that includes RDF and semantic web discussion, an integrated database integration chapter focusing both on schema integration and querying over these systems. The peer-to-peer computing chapter has been updated with a discussion of blockchains. The chapters that describe classical distributed and parallel database technology have all been updated. The new edition covers the breadth and depth of the field from a modern viewpoint. Graduate students, as well as senior undergraduate students studying computer science and other related fields will use this book as a primary textbook. Researchers working in computer science will also find this textbook useful. This textbook has a companion web site that includes background information on relational database fundamentals, query processing, transaction management, and computer networks for those who might need this background. The web site also includes all the figures and presentation slides as well as solutions to exercises (restricted to instructors).
Download or read book Shared Query Processing in Data Streaming Systems written by Saileshwar Krishnamurthy and published by . This book was released on 2006 with total page 432 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Grid and Cooperative Computing GCC 2004 written by Hai Jin and published by Springer. This book was released on 2004-09-29 with total page 1047 pages. Available in PDF, EPUB and Kindle. Book excerpt: Welcome to the proceedings of GCC2004 and the city of Wuhan. Grid computing has become a mainstream research area in computer science and the GCC conference has become one of the premier forums for presentation of new and exciting research in all aspectsofgridandcooperativecomputing. Theprogramcommitteeispleasedtopresent the proceedings of the 3rd International Conference on Grid and Cooperative Comp- ing (GCC2004), which comprises a collection of excellent technical papers, posters, workshops, and keynote speeches. The papers accepted cover a wide range of exciting topics, including resource grid and service grid, information grid and knowledge grid, grid monitoring,managementand organizationtools, grid portal, grid service, Web s- vices and their QoS, service orchestration, grid middleware and toolkits, software glue technologies, grid security, innovative grid applications, advanced resource reservation andscheduling,performanceevaluationandmodeling,computer-supportedcooperative work, P2P computing, automatic computing, and meta-information management. The conference continues to grow and this year a record total of 581 manuscripts (including workshop submissions) were submitted for consideration. Expecting this growth, the size of the program committee was increased from 50 members for GCC 2003 for 70 in GCC 2004. Relevant differences from previous editions of the conf- ence: it is worth mentioning a signi?cant increase in the number of papers submitted by authors from outside China; and the acceptance rate was much lower than for p- vious GCC conferences. From the 427 papers submitted to the main conference, the program committee selected only 96 regular papers for oral presentation and 62 short papers for poster presentation in the program.
Download or read book Proceedings 2003 VLDB Conference written by VLDB and published by Morgan Kaufmann. This book was released on 2003-12-02 with total page 1185 pages. Available in PDF, EPUB and Kindle. Book excerpt: Proceedings of the 29th Annual International Conference on Very Large Data Bases held in Berlin, Germany on September 9-12, 2003. Organized by the VLDB Endowment, VLDB is the premier international conference on database technology.
Download or read book Spatio Temporal Data Streams written by Zdravko Galić and published by Springer. This book was released on 2016-08-26 with total page 116 pages. Available in PDF, EPUB and Kindle. Book excerpt: This SpringerBrief presents the fundamental concepts of a specialized class of data stream, spatio-temporal data streams, and demonstrates their distributed processing using Big Data frameworks and platforms. It explores a consistent framework which facilitates a thorough understanding of all different facets of the technology, from basic definitions to state-of-the-art techniques. Key topics include spatio-temporal continuous queries, distributed stream processing, SQL-like language embedding, and trajectory stream clustering. Over the course of the book, the reader will become familiar with spatio-temporal data streams management and data flow processing, which enables the analysis of huge volumes of location-aware continuous data streams. Applications range from mobile object tracking and real-time intelligent transportation systems to traffic monitoring and complex event processing. Spatio-Temporal Data Streams is a valuable resource for researchers studying spatio-temporal data streams and Big Data analytics, as well as data engineers and data scientists solving data management and analytics problems associated with this class of data.
Download or read book Handbook of Research on Machine Learning Innovations and Trends written by Hassanien, Aboul Ella and published by IGI Global. This book was released on 2017-04-03 with total page 1269 pages. Available in PDF, EPUB and Kindle. Book excerpt: Continuous improvements in technological applications have allowed more opportunities to develop automated systems. This not only leads to higher success in smart data analysis, but it increases the overall probability of technological progression. The Handbook of Research on Machine Learning Innovations and Trends is a key resource on the latest advances and research regarding the vast range of advanced systems and applications involved in machine intelligence. Highlighting multidisciplinary studies on decision theory, intelligent search, and multi-agent systems, this publication is an ideal reference source for professionals and researchers working in the field of machine learning and its applications.
