Download or read book Modern Data Architectures with Python written by Brian Lipp and published by Packt Publishing Ltd. This book was released on 2023-09-29 with total page 318 pages. Available in PDF, EPUB and Kindle. Book excerpt: Build scalable and reliable data ecosystems using Data Mesh, Databricks Spark, and Kafka Key Features Develop modern data skills used in emerging technologies Learn pragmatic design methodologies such as Data Mesh and data lakehouses Gain a deeper understanding of data governance Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionModern Data Architectures with Python will teach you how to seamlessly incorporate your machine learning and data science work streams into your open data platforms. You’ll learn how to take your data and create open lakehouses that work with any technology using tried-and-true techniques, including the medallion architecture and Delta Lake. Starting with the fundamentals, this book will help you build pipelines on Databricks, an open data platform, using SQL and Python. You’ll gain an understanding of notebooks and applications written in Python using standard software engineering tools such as git, pre-commit, Jenkins, and Github. Next, you’ll delve into streaming and batch-based data processing using Apache Spark and Confluent Kafka. As you advance, you’ll learn how to deploy your resources using infrastructure as code and how to automate your workflows and code development. Since any data platform's ability to handle and work with AI and ML is a vital component, you’ll also explore the basics of ML and how to work with modern MLOps tooling. Finally, you’ll get hands-on experience with Apache Spark, one of the key data technologies in today’s market. By the end of this book, you’ll have amassed a wealth of practical and theoretical knowledge to build, manage, orchestrate, and architect your data ecosystems.What you will learn Understand data patterns including delta architecture Discover how to increase performance with Spark internals Find out how to design critical data diagrams Explore MLOps with tools such as AutoML and MLflow Get to grips with building data products in a data mesh Discover data governance and build confidence in your data Introduce data visualizations and dashboards into your data practice Who this book is forThis book is for developers, analytics engineers, and managers looking to further develop a data ecosystem within their organization. While they’re not prerequisites, basic knowledge of Python and prior experience with data will help you to read and follow along with the examples.
Download or read book Modern Big Data Architectures written by Dominik Ryzko and published by John Wiley & Sons. This book was released on 2020-03-31 with total page 208 pages. Available in PDF, EPUB and Kindle. Book excerpt: Provides an up-to-date analysis of big data and multi-agent systems The term Big Data refers to the cases, where data sets are too large or too complex for traditional data-processing software. With the spread of new concepts such as Edge Computing or the Internet of Things, production, processing and consumption of this data becomes more and more distributed. As a result, applications increasingly require multiple agents that can work together. A multi-agent system (MAS) is a self-organized computer system that comprises multiple intelligent agents interacting to solve problems that are beyond the capacities of individual agents. Modern Big Data Architectures examines modern concepts and architecture for Big Data processing and analytics. This unique, up-to-date volume provides joint analysis of big data and multi-agent systems, with emphasis on distributed, intelligent processing of very large data sets. Each chapter contains practical examples and detailed solutions suitable for a wide variety of applications. The author, an internationally-recognized expert in Big Data and distributed Artificial Intelligence, demonstrates how base concepts such as agent, actor, and micro-service have reached a point of convergence—enabling next generation systems to be built by incorporating the best aspects of the field. This book: Illustrates how data sets are produced and how they can be utilized in various areas of industry and science Explains how to apply common computational models and state-of-the-art architectures to process Big Data tasks Discusses current and emerging Big Data applications of Artificial Intelligence Modern Big Data Architectures: A Multi-Agent Systems Perspective is a timely and important resource for data science professionals and students involved in Big Data analytics, and machine and artificial learning.
Download or read book Data Management at Scale written by Piethein Strengholt and published by "O'Reilly Media, Inc.". This book was released on 2020-07-29 with total page 404 pages. Available in PDF, EPUB and Kindle. Book excerpt: As data management and integration continue to evolve rapidly, storing all your data in one place, such as a data warehouse, is no longer scalable. In the very near future, data will need to be distributed and available for several technological solutions. With this practical book, you’ll learnhow to migrate your enterprise from a complex and tightly coupled data landscape to a more flexible architecture ready for the modern world of data consumption. Executives, data architects, analytics teams, and compliance and governance staff will learn how to build a modern scalable data landscape using the Scaled Architecture, which you can introduce incrementally without a large upfront investment. Author Piethein Strengholt provides blueprints, principles, observations, best practices, and patterns to get you up to speed. Examine data management trends, including technological developments, regulatory requirements, and privacy concerns Go deep into the Scaled Architecture and learn how the pieces fit together Explore data governance and data security, master data management, self-service data marketplaces, and the importance of metadata
Download or read book Data Analysis with Python written by David Taieb and published by Packt Publishing Ltd. This book was released on 2018-12-31 with total page 491 pages. Available in PDF, EPUB and Kindle. Book excerpt: Learn a modern approach to data analysis using Python to harness the power of programming and AI across your data. Detailed case studies bring this modern approach to life across visual data, social media, graph algorithms, and time series analysis. Key FeaturesBridge your data analysis with the power of programming, complex algorithms, and AIUse Python and its extensive libraries to power your way to new levels of data insightWork with AI algorithms, TensorFlow, graph algorithms, NLP, and financial time seriesExplore this modern approach across with key industry case studies and hands-on projectsBook Description Data Analysis with Python offers a modern approach to data analysis so that you can work with the latest and most powerful Python tools, AI techniques, and open source libraries. Industry expert David Taieb shows you how to bridge data science with the power of programming and algorithms in Python. You'll be working with complex algorithms, and cutting-edge AI in your data analysis. Learn how to analyze data with hands-on examples using Python-based tools and Jupyter Notebook. You'll find the right balance of theory and practice, with extensive code files that you can integrate right into your own data projects. Explore the power of this approach to data analysis by then working with it across key industry case studies. Four fascinating and full projects connect you to the most critical data analysis challenges you’re likely to meet in today. The first of these is an image recognition application with TensorFlow – embracing the importance today of AI in your data analysis. The second industry project analyses social media trends, exploring big data issues and AI approaches to natural language processing. The third case study is a financial portfolio analysis application that engages you with time series analysis - pivotal to many data science applications today. The fourth industry use case dives you into graph algorithms and the power of programming in modern data science. You'll wrap up with a thoughtful look at the future of data science and how it will harness the power of algorithms and artificial intelligence. What you will learnA new toolset that has been carefully crafted to meet for your data analysis challengesFull and detailed case studies of the toolset across several of today’s key industry contextsBecome super productive with a new toolset across Python and Jupyter NotebookLook into the future of data science and which directions to develop your skills nextWho this book is for This book is for developers wanting to bridge the gap between them and data scientists. Introducing PixieDust from its creator, the book is a great desk companion for the accomplished Data Scientist. Some fluency in data interpretation and visualization is assumed. It will be helpful to have some knowledge of Python, using Python libraries, and some proficiency in web development.
Download or read book Modern Data Mining with Python written by Dushyant Singh Sengar and published by BPB Publications. This book was released on 2024-02-26 with total page 471 pages. Available in PDF, EPUB and Kindle. Book excerpt: Data miner’s survival kit for explainable, effective, and efficient algorithms enabling responsible decision-making KEY FEATURES ● Accessible, and case-based exploration of the most effective data mining techniques in Python. ● An indispensable guide for utilizing AI potential responsibly. ● Actionable insights on modeling techniques, deployment technologies, business needs, and the art of data science, for risk mitigation and better business outcomes. DESCRIPTION "Modern Data Mining with Python" is a guidebook for responsibly implementing data mining techniques that involve collecting, storing, and analyzing large amounts of structured and unstructured data to extract useful insights and patterns. Enter into the world of data mining and machine learning. Use insights from various data sources, from social media to credit card transactions. Master statistical tools, explore data trends, and patterns. Understand decision trees and artificial neural networks (ANNs). Manage high-dimensional data with dimensionality reduction. Explore binary classification with logistic regression. Spot concealed patterns with unsupervised learning. Analyze text with recurrent neural networks (RNNs) and visuals with convolutional neural networks (CNNs). Ensure model compliance with regulatory standards. After reading this book, readers will be equipped with the skills and knowledge necessary to use Python for data mining and analysis in an industry set-up. They will be able to analyze and implement algorithms on large structured and unstructured datasets. WHAT YOU WILL LEARN ● Explore the data mining spectrum ranging from data exploration and statistics. ● Gain hands-on experience applying modern algorithms to real-world problems in the financial industry. ● Develop an understanding of various risks associated with model usage in regulated industries. ● Gain knowledge about best practices and regulatory guidelines to mitigate model usage-related risk in key banking areas. ● Develop and deploy risk-mitigated algorithms on self-serve ModelOps platforms. WHO THIS BOOK IS FOR This book is for a wide range of early career professionals and students interested in data mining or data science with a financial services industry focus. Senior industry professionals, and educators, trying to implement data mining algorithms can benefit as well. TABLE OF CONTENTS 1. Understanding Data Mining in a Nutshell 2. Basic Statistics and Exploratory Data Analysis 3. Digging into Linear Regression 4. Exploring Logistic Regression 5. Decision Trees with Bagging and Boosting 6. Support Vector Machines and K-Nearest Neighbors 7. Putting Dimensionality Reduction into Action 8. Beginning with Unsupervised Models 9. Structured Data Classification using Artificial Neural Networks 10. Language Modeling with Recurrent Neural Networks 11. Image Processing with Convolutional Neural Networks 12. Understanding Model Risk Management for Data Mining Models 13. Adopting ModelOps to Manage Model Risk
Download or read book Flow Architectures written by James Urquhart and published by "O'Reilly Media, Inc.". This book was released on 2021-01-06 with total page 280 pages. Available in PDF, EPUB and Kindle. Book excerpt: Software development today is embracing events and streaming data, which optimizes not only how technology interacts but also how businesses integrate with one another to meet customer needs. This phenomenon, called flow, consists of patterns and standards that determine which activity and related data is communicated between parties over the internet. This book explores critical implications of that evolution: What happens when events and data streams help you discover new activity sources to enhance existing businesses or drive new markets? What technologies and architectural patterns can position your company for opportunities enabled by flow? James Urquhart, global field CTO at VMware, guides enterprise architects, software developers, and product managers through the process. Learn the benefits of flow dynamics when businesses, governments, and other institutions integrate via events and data streams Understand the value chain for flow integration through Wardley mapping visualization and promise theory modeling Walk through basic concepts behind today's event-driven systems marketplace Learn how today's integration patterns will influence the real-time events flow in the future Explore why companies should architect and build software today to take advantage of flow in coming years
Download or read book Data Engineering with Python written by Paul Crickard and published by Packt Publishing Ltd. This book was released on 2020-10-23 with total page 357 pages. Available in PDF, EPUB and Kindle. Book excerpt: Build, monitor, and manage real-time data pipelines to create data engineering infrastructure efficiently using open-source Apache projects Key Features Become well-versed in data architectures, data preparation, and data optimization skills with the help of practical examples Design data models and learn how to extract, transform, and load (ETL) data using Python Schedule, automate, and monitor complex data pipelines in production Book DescriptionData engineering provides the foundation for data science and analytics, and forms an important part of all businesses. This book will help you to explore various tools and methods that are used for understanding the data engineering process using Python. The book will show you how to tackle challenges commonly faced in different aspects of data engineering. You’ll start with an introduction to the basics of data engineering, along with the technologies and frameworks required to build data pipelines to work with large datasets. You’ll learn how to transform and clean data and perform analytics to get the most out of your data. As you advance, you'll discover how to work with big data of varying complexity and production databases, and build data pipelines. Using real-world examples, you’ll build architectures on which you’ll learn how to deploy data pipelines. By the end of this Python book, you’ll have gained a clear understanding of data modeling techniques, and will be able to confidently build data engineering pipelines for tracking data, running quality checks, and making necessary changes in production.What you will learn Understand how data engineering supports data science workflows Discover how to extract data from files and databases and then clean, transform, and enrich it Configure processors for handling different file formats as well as both relational and NoSQL databases Find out how to implement a data pipeline and dashboard to visualize results Use staging and validation to check data before landing in the warehouse Build real-time pipelines with staging areas that perform validation and handle failures Get to grips with deploying pipelines in the production environment Who this book is for This book is for data analysts, ETL developers, and anyone looking to get started with or transition to the field of data engineering or refresh their knowledge of data engineering using Python. This book will also be useful for students planning to build a career in data engineering or IT professionals preparing for a transition. No previous knowledge of data engineering is required.
Download or read book Architecture Patterns with Python written by Harry Percival and published by O'Reilly Media. This book was released on 2020-03-05 with total page 304 pages. Available in PDF, EPUB and Kindle. Book excerpt: As Python continues to grow in popularity, projects are becoming larger and more complex. Many Python developers are now taking an interest in high-level software design patterns such as hexagonal/clean architecture, event-driven architecture, and the strategic patterns prescribed by domain-driven design (DDD). But translating those patterns into Python isn’t always straightforward. With this hands-on guide, Harry Percival and Bob Gregory from MADE.com introduce proven architectural design patterns to help Python developers manage application complexity—and get the most value out of their test suites. Each pattern is illustrated with concrete examples in beautiful, idiomatic Python, avoiding some of the verbosity of Java and C# syntax. Patterns include: Dependency inversion and its links to ports and adapters (hexagonal/clean architecture) Domain-driven design’s distinction between entities, value objects, and aggregates Repository and Unit of Work patterns for persistent storage Events, commands, and the message bus Command-query responsibility segregation (CQRS) Event-driven architecture and reactive microservices
Download or read book Scalable Big Data Architecture written by Bahaaldine Azarmi and published by Apress. This book was released on 2015-12-31 with total page 147 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book highlights the different types of data architecture and illustrates the many possibilities hidden behind the term "Big Data", from the usage of No-SQL databases to the deployment of stream analytics architecture, machine learning, and governance. Scalable Big Data Architecture covers real-world, concrete industry use cases that leverage complex distributed applications , which involve web applications, RESTful API, and high throughput of large amount of data stored in highly scalable No-SQL data stores such as Couchbase and Elasticsearch. This book demonstrates how data processing can be done at scale from the usage of NoSQL datastores to the combination of Big Data distribution. When the data processing is too complex and involves different processing topology like long running jobs, stream processing, multiple data sources correlation, and machine learning, it’s often necessary to delegate the load to Hadoop or Spark and use the No-SQL to serve processed data in real time. This book shows you how to choose a relevant combination of big data technologies available within the Hadoop ecosystem. It focuses on processing long jobs, architecture, stream data patterns, log analysis, and real time analytics. Every pattern is illustrated with practical examples, which use the different open sourceprojects such as Logstash, Spark, Kafka, and so on. Traditional data infrastructures are built for digesting and rendering data synthesis and analytics from large amount of data. This book helps you to understand why you should consider using machine learning algorithms early on in the project, before being overwhelmed by constraints imposed by dealing with the high throughput of Big data. Scalable Big Data Architecture is for developers, data architects, and data scientists looking for a better understanding of how to choose the most relevant pattern for a Big Data project and which tools to integrate into that pattern.
Download or read book Advanced Analytics in Power BI with R and Python written by Ryan Wade and published by Apress. This book was released on 2020-09-05 with total page 330 pages. Available in PDF, EPUB and Kindle. Book excerpt: This easy-to-follow guide provides R and Python recipes to help you learn and apply the top languages in the field of data analytics to your work in Microsoft Power BI. Data analytics expert and author Ryan Wade shows you how to use R and Python to perform tasks that are extremely hard to do, if not impossible, using native Power BI tools without Power BI Premium capacity. For example, you will learn to score Power BI data using custom data science models, including powerful models from Microsoft Cognitive Services. The R and Python languages are powerful complements to Power BI. They enable advanced data transformation techniques that are difficult to perform in Power BI in its default configuration, but become easier through the application of data wrangling features that languages such as R and Python support. If you are a BI developer, business analyst, data analyst, or a data scientist who wants to push Power BI and transform it from being just a business intelligence tool into an advanced data analytics tool, then this is the book to help you to do that. What You Will Learn Create advanced data visualizations through R using the ggplot2 package Ingest data using R and Python to overcome the limitations of Power Query Apply machine learning models to your data using R and Python Incorporate advanced AI in Power BI via Microsoft Cognitive Services, IBM Watson, and pre-trained models in SQL Server Machine Learning Services Perform string manipulations not otherwise possible in Power BI using R and Python Who This Book Is For Power users, data analysts, and data scientists who want to go beyond Power BI’s built-in functionality to create advanced visualizations, transform data in ways not otherwise supported, and automate data ingestion from sources such as SQL Server and Excel in a more succinct way
Download or read book Deciphering Data Architectures written by James Serra and published by "O'Reilly Media, Inc.". This book was released on 2024-02-06 with total page 262 pages. Available in PDF, EPUB and Kindle. Book excerpt: Data fabric, data lakehouse, and data mesh have recently appeared as viable alternatives to the modern data warehouse. These new architectures have solid benefits, but they're also surrounded by a lot of hyperbole and confusion. This practical book provides a guided tour of these architectures to help data professionals understand the pros and cons of each. James Serra, big data and data warehousing solution architect at Microsoft, examines common data architecture concepts, including how data warehouses have had to evolve to work with data lake features. You'll learn what data lakehouses can help you achieve, as well as how to distinguish data mesh hype from reality. Best of all, you'll be able to determine the most appropriate data architecture for your needs. With this book, you'll: Gain a working understanding of several data architectures Learn the strengths and weaknesses of each approach Distinguish data architecture theory from reality Pick the best architecture for your use case Understand the differences between data warehouses and data lakes Learn common data architecture concepts to help you build better solutions Explore the historical evolution and characteristics of data architectures Learn essentials of running an architecture design session, team organization, and project success factors Free from product discussions, this book will serve as a timeless resource for years to come.
Download or read book Databricks Certified Associate Developer for Apache Spark Using Python written by Saba Shah and published by Packt Publishing Ltd. This book was released on 2024-06-14 with total page 274 pages. Available in PDF, EPUB and Kindle. Book excerpt: Learn the concepts and exercises needed to get certified as a Databricks Associate Developer for Apache Spark 3.0 and validate your skills as a Spark expert with an industry-recognized credential Key Features Understand the fundamentals of Apache Spark to help you design robust and fast Spark applications Delve into various data manipulation components for each phase of your data engineering project Prepare for the certification exam with sample questions and mock exams, and get closer to your goal Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionWith extensive data being collected every second, computing power cannot keep up with this pace of rapid growth. To make use of all the data, Spark has become a de facto standard for big data processing. Migrating data processing to Spark will not only help you save resources that will allow you to focus on your business, but also enable you to modernize your workloads by leveraging the capabilities of Spark and the modern technology stack for creating new business opportunities. This book is a comprehensive guide that lets you explore the core components of Apache Spark, its architecture, and its optimization. You’ll become familiar with the Spark dataframe API and its components needed for data manipulation. Next, you’ll find out what Spark streaming is and why it’s important for modern data stacks, before learning about machine learning in Spark and its different use cases. What’s more, you’ll discover sample questions at the end of each section along with two mock exams to help you prepare for the certification exam. By the end of this book, you’ll know what to expect in the exam and how to pass it with enough understanding of Spark and its tools. You’ll also be able to apply this knowledge in a real-world setting and take your skillset to the next level.What you will learn Create and manipulate SQL queries in Spark Build complex Spark functions using Spark UDFs Architect big data apps with Spark fundamentals for optimal design Apply techniques to manipulate and optimize big data applications Build real-time or near-real-time applications using Spark Streaming Work with Apache Spark for machine learning applications Who this book is for This book is for you if you’re a professional looking to venture into the world of big data and data engineering, a data professional who wants to endorse your knowledge of Spark, or a student. Although working knowledge of Python is required, no prior Spark knowledge is needed. Additionally, experience with Pyspark will be beneficial.
Download or read book Python and R for the Modern Data Scientist written by Rick J. Scavetta and published by "O'Reilly Media, Inc.". This book was released on 2021-06-22 with total page 198 pages. Available in PDF, EPUB and Kindle. Book excerpt: Success in data science depends on the flexible and appropriate use of tools. That includes Python and R, two of the foundational programming languages in the field. This book guides data scientists from the Python and R communities along the path to becoming bilingual. By recognizing the strengths of both languages, you'll discover new ways to accomplish data science tasks and expand your skill set. Authors Rick Scavetta and Boyan Angelov explain the parallel structures of these languages and highlight where each one excels, whether it's their linguistic features or the powers of their open source ecosystems. You'll learn how to use Python and R together in real-world settings and broaden your job opportunities as a bilingual data scientist. Learn Python and R from the perspective of your current language Understand the strengths and weaknesses of each language Identify use cases where one language is better suited than the other Understand the modern open source ecosystem available for both, including packages, frameworks, and workflows Learn how to integrate R and Python in a single workflow Follow a case study that demonstrates ways to use these languages together
Download or read book Cassandra The Definitive Guide written by Jeff Carpenter and published by "O'Reilly Media, Inc.". This book was released on 2016-06-29 with total page 369 pages. Available in PDF, EPUB and Kindle. Book excerpt: Imagine what you could do if scalability wasn't a problem. With this hands-on guide, you’ll learn how the Cassandra database management system handles hundreds of terabytes of data while remaining highly available across multiple data centers. This expanded second edition—updated for Cassandra 3.0—provides the technical details and practical examples you need to put this database to work in a production environment. Authors Jeff Carpenter and Eben Hewitt demonstrate the advantages of Cassandra’s non-relational design, with special attention to data modeling. If you’re a developer, DBA, or application architect looking to solve a database scaling issue or future-proof your application, this guide helps you harness Cassandra’s speed and flexibility. Understand Cassandra’s distributed and decentralized structure Use the Cassandra Query Language (CQL) and cqlsh—the CQL shell Create a working data model and compare it with an equivalent relational model Develop sample applications using client drivers for languages including Java, Python, and Node.js Explore cluster topology and learn how nodes exchange data Maintain a high level of performance in your cluster Deploy Cassandra on site, in the Cloud, or with Docker Integrate Cassandra with Spark, Hadoop, Elasticsearch, Solr, and Lucene
Download or read book Advanced Data Analytics Using Python written by Sayan Mukhopadhyay and published by Apress. This book was released on 2022-11-26 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Understand advanced data analytics concepts such as time series and principal component analysis with ETL, supervised learning, and PySpark using Python. This book covers architectural patterns in data analytics, text and image classification, optimization techniques, natural language processing, and computer vision in the cloud environment. Generic design patterns in Python programming is clearly explained, emphasizing architectural practices such as hot potato anti-patterns. You'll review recent advances in databases such as Neo4j, Elasticsearch, and MongoDB. You'll then study feature engineering in images and texts with implementing business logic and see how to build machine learning and deep learning models using transfer learning. Advanced Analytics with Python, 2nd edition features a chapter on clustering with a neural network, regularization techniques, and algorithmic design patterns in data analytics with reinforcement learning. Finally, the recommender system in PySpark explains how to optimize models for a specific application. What You'll Learn Build intelligent systems for enterprise Review time series analysis, classifications, regression, and clustering Explore supervised learning, unsupervised learning, reinforcement learning, and transfer learning Use cloud platforms like GCP and AWS in data analytics Understand Covers design patterns in Python Who This Book Is For Data scientists and software developers interested in the field of data analytics.
Download or read book Data Mesh written by Zhamak Dehghani and published by "O'Reilly Media, Inc.". This book was released on 2022-03-08 with total page 387 pages. Available in PDF, EPUB and Kindle. Book excerpt: Many enterprises are investing in a next-generation data lake, hoping to democratize data at scale to provide business insights and ultimately make automated intelligent decisions. In this practical book, author Zhamak Dehghani reveals that, despite the time, money, and effort poured into them, data warehouses and data lakes fail when applied at the scale and speed of today's organizations. A distributed data mesh is a better choice. Dehghani guides architects, technical leaders, and decision makers on their journey from monolithic big data architecture to a sociotechnical paradigm that draws from modern distributed architecture. A data mesh considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure, treats data as a product, and introduces a federated and computational model of data governance. This book shows you why and how. Examine the current data landscape from the perspective of business and organizational needs, environmental challenges, and existing architectures Analyze the landscape's underlying characteristics and failure modes Get a complete introduction to data mesh principles and its constituents Learn how to design a data mesh architecture Move beyond a monolithic data lake to a distributed data mesh.
Download or read book Fast Python written by Tiago Antao and published by Simon and Schuster. This book was released on 2023-07-04 with total page 302 pages. Available in PDF, EPUB and Kindle. Book excerpt: Master Python techniques and libraries to reduce run times, efficiently handle huge datasets, and optimize execution for complex machine learning applications. Fast Python is a toolbox of techniques for high performance Python including: Writing efficient pure-Python code Optimizing the NumPy and pandas libraries Rewriting critical code in Cython Designing persistent data structures Tailoring code for different architectures Implementing Python GPU computing Fast Python is your guide to optimizing every part of your Python-based data analysis process, from the pure Python code you write to managing the resources of modern hardware and GPUs. You'll learn to rewrite inefficient data structures, improve underperforming code with multithreading, and simplify your datasets without sacrificing accuracy. Written for experienced practitioners, this book dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working. About the Technology Face it. Slow code will kill a big data project. Fast pure-Python code, optimized libraries, and fully utilized multiprocessor hardware are the price of entry for machine learning and large-scale data analysis. What you need are reliable solutions that respond faster to computing requirements while using less resources, and saving money. About the Book Fast Python is a toolbox of techniques for speeding up Python, with an emphasis on big data applications. Following the clear examples and precisely articulated details, you’ll learn how to use common libraries like NumPy and pandas in more performant ways and transform data for efficient storage and I/O. More importantly, Fast Python takes a holistic approach to performance, so you’ll see how to optimize the whole system, from code to architecture. What’s Inside Rewriting critical code in Cython Designing persistent data structures Tailoring code for different architectures Implementing Python GPU computing About the Reader For intermediate Python programmers familiar with the basics of concurrency. About the Author Tiago Antão is one of the co-authors of Biopython, a major bioinformatics package written in Python. Table of Contents: PART 1 - FOUNDATIONAL APPROACHES 1 An urgent need for efficiency in data processing 2 Extracting maximum performance from built-in features 3 Concurrency, parallelism, and asynchronous processing 4 High-performance NumPy PART 2 - HARDWARE 5 Re-implementing critical code with Cython 6 Memory hierarchy, storage, and networking PART 3 - APPLICATIONS AND LIBRARIES FOR MODERN DATA PROCESSING 7 High-performance pandas and Apache Arrow 8 Storing big data PART 4 - ADVANCED TOPICS 9 Data analysis using GPU computing 10 Analyzing big data with Dask