Fast Python: High Performance Techniques For Large Datasets
- 5h 29m
- Tiago Rodrigues Antao
- Manning Publications
- 2022
Master Python techniques and libraries to reduce run times, efficiently handle huge datasets, and optimize execution for complex machine learning applications.
Fast Python is a toolbox of techniques for high performance Python including:
- Writing efficient pure-Python code
- Optimizing the NumPy and pandas librari
- Rewriting critical code in Cython
- Designing persistent data structures
- Tailoring code for different architectures
- Implementing Python GPU computing
Fast Python is your guide to optimizing every part of your Python-based data analysis process, from the pure Python code you write to managing the resources of modern hardware and GPUs. You'll learn to rewrite inefficient data structures, improve underperforming code with multithreading, and simplify your datasets without sacrificing accuracy.
Written for experienced practitioners, this book dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working.
about the technology
Face it. Slow code will kill a big data project. Fast pure-Python code, optimized libraries, and fully utilized multiprocessor hardware are the price of entry for machine learning and large-scale data analysis. What you need are reliable solutions that respond faster to computing requirements while using less resources, and saving money.
about the book
Fast Python is a toolbox of techniques for speeding up Python, with an emphasis on big data applications. Following the clear examples and precisely articulated details, you’ll learn how to use common libraries like NumPy and pandas in more performant ways and transform data for efficient storage and I/O. More importantly, Fast Python takes a holistic approach to performance, so you’ll see how to optimize the whole system, from code to architecture_._
About the Author
Tiago Antao works in the field of genetics, analyzing very large datasets and implementing complex algorithms to process the data. He leverages Python with all its libraries to do scientific computing and data engineering tasks. He is one of the co-authors of Biopython, a major bioinformatics package written in Python. He holds a BE in informatics and a PhD in bioinformatics.
In this Book
-
Preface
-
About This Book
-
About The Cover Illustration
-
An Urgent Need for Efficiency in Data Processing
-
Extracting Maximum Performance from Built-In Features
-
Concurrency, Parallelism, and Asynchronous Processing
-
High-Performance NumPy
-
Re-Implementing Critical Code with Cython
-
Memory Hierarchy, Storage, and Networking
-
High-Performance Pandas and Apache Arrow
-
Storing Big Data
-
Data Analysis Using GPU Computing
-
Analyzing Big Data with Dask
-
Appendix A. Setting Up the Environment
-
Appendix B. Using Numba to Generate Efficient Low-Level Code
