Thinking in Pandas: How to Use the Python Data Analysis Library the Right Way

Understand and implement big data analysis solutions in pandas with an emphasis on performance. This book strengthens your intuition for working with pandas, the Python data analysis library, by exploring its underlying implementation and data structures. Thinking in Pandas introduces the topic of big data and demonstrates concepts by looking at exciting and impactful projects that pandas helped to solve. From there, you will learn to assess your own projects by size and type to see if pandas is the appropriate library for your needs. Author Hannah Stepanek explains how to load and normalize data in pandas efficiently, and reviews some of the most commonly used loaders and several of their most powerful options. You will then learn how to access and transform data efficiently, what methods to avoid, and when to employ more advanced performance techniques. You will also go over basic data access and munging in pandas and the intuitive dictionary syntax. Choosing the right DataFrame format, working with multi-level DataFrames, and how pandas might be improved upon in the future are also covered.By the end of the book, you will have a solid understanding of how the pandas library works under the hood. Get ready to make confident decisions in your own projects by utilizing pandas—the right way.

What You Will Learn

• Understand the underlying data structure of pandas and why it performs the way it does under certain circumstances
• Discover how to use pandas to extract, transform, and load data correctly with an emphasis on performance
• Choose the right DataFrame so that the data analysis is simple and efficient.
• Improve performance of pandas operations with other Python libraries

Who This Book Is For

Software engineers with basic programming skills in Python keen on using pandas for a big data analysis project. Python software developers interested in big data.

About the Author

Hannah Stepanek is a software developer with a passion for performance and is an open source advocate. She has over seven years of industry experience programming in Python and spent about two of those years implementing a data analysis project using pandas.

Hannah was born and raised in Corvallis, OR, and graduated from Oregon State University with a major in Electrical Computer Engineering. She enjoys engaging with the software community, often giving talks at local meetups as well as larger conferences. In early 2019, she spoke at PyCon US about the pandas library and at OpenCon Cascadia about the benefits of open source software. In her spare time she enjoys riding her horse Sophie and playing board games.