SQL on Big Data: Technology, Architecture, and Innovation

Learn various commercial and open source products that perform SQL on Big Data platforms. You will understand the architectures of the various SQL engines being used and how the tools work internally in terms of execution, data movement, latency, scalability, performance, and system requirements.

This book consolidates in one place solutions to the challenges associated with the requirements of speed, scalability, and the variety of operations needed for data integration and SQL operations. After discussing the history of the how and why of SQL on Big Data, the book provides in-depth insight into the products, architectures, and innovations happening in this rapidly evolving space.

SQL on Big Data discusses in detail the innovations happening, the capabilities on the horizon, and how they solve the issues of performance and scalability and the ability to handle different data types. The book covers how SQL on Big Data engines are permeating the OLTP, OLAP, and Operational analytics space and the rapidly evolving HTAP systems.

You will learn the details of:

• Batch Architectures―an understanding of the internals and how the existing Hive engine is built and how it is evolving continually to support new features and provide lower latency on queries
• Interactive Architectures―an understanding of how SQL engines are architected to support low latency on large data sets
• Streaming Architectures―an understanding of how SQL engines are architected to support queries on data in motion using in-memory and lock-free data structures
• Operational Architectures―an understanding of how SQL engines are architected for transactional and operational systems to support transactions on Big Data platforms
• Innovative Architectures―an exploration of the rapidly evolving newer SQL engines on Big Data with innovative ideas and concepts

About the Author

Sumit Pal is a big data and data science consultant working with multiple clients and advising them on their data architectures and big data solutions as well as providing hands on coding with Spark, Scala, Java and Python. He is a big data, visualization and data science consultant, and a software architect and big data enthusiast and builds end-to-end data-driven analytic systems. He has more than 22 years of experience in the software industry in various roles spanning companies from startups to enterprises.

Sumit has worked for Microsoft (SQL server development team), Oracle (OLAP development team) and Verizon (big data analytics team) in a career spanning 22 years. He has extensive experience in building scalable systems across the stack from middle-tier, data tier to visualization for analytics applications, using big data, and NoSQL DB. Sumit has deep expertise in database Internals, data warehouses, dimensional modeling, data science with Java and Python, and SQL. Sumit started his career with the SQL server development team at Microsoft and then as a core engineer for Oracle Corporation on their OLAP team. He has also worked at Verizon as an associate director of big data architecture, where he strategized, managed, architected, developed platforms solutions and analytics of machine learning applications. Sumit served as chief-architect with modeln leapfrogrx, architected the middle tier platform of an open source engine (Mondrian) j2ee, solved some complex dimensional modelling, and performance optimization problems.

Sumit has MS and BS in Computer Science.