Processing Data: Introducing Apache Spark

Apache Spark    |    Intermediate
  • 13 videos | 1h 44m 10s
  • Includes Assessment
  • Earns a Badge
Rating 4.7 of 57 users Rating 4.7 of 57 users (57)
Apache Spark is a powerful distributed data processing engine that can handle petabytes of data by chunking that data and dividing across a cluster of resources. In this course, explore Spark's structured streaming engine, including components like PySpark shell. Begin by downloading and installing Apache Spark. Then create a Spark cluster and run a job from the PySpark shell. Monitor an application and job runs from the Spark web user interface. Then, set up a streaming environment, reading and manipulating the contents of files that are added to a folder in real-time. Finally, run apps on both Spark standalone and local modes.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Describe how apache hadoop and spark work
    Recall the architecture and features of apache spark
    Recognize the use cases of spark in general and specifically, its structured streaming engine
    Install and configure apache spark
    Create a spark cluster with a master and worker
    Run a job on the pyspark shell and view its details from the spark web user interface (ui)
  • Execute spark commands and monitor jobs with the spark web ui
    Configure a spark cluster using the spark-env.sh file
    Set up an environment to stream files, and build an app to process files in real-time
    Execute apps on a spark standalone cluster
    Distinguish between spark standalone and local deployment modes
    Summarize the key concepts covered in this course

IN THIS COURSE

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Data Loading in Snowflake: Fundamentals of Stages
Rating 4.7 of 50 users Rating 4.7 of 50 users (50)
Book Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications, 1st Edition
Course Processing Data: Integrating Kafka with Python & Using Consumer Groups
Rating 4.7 of 31 users Rating 4.7 of 31 users (31)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course MySQL: Querying Data
Rating 4.7 of 150 users Rating 4.7 of 150 users (150)
Course Using Apache Spark for AI Development
Rating 4.2 of 20 users Rating 4.2 of 20 users (20)
Course Data Engineering on Microsoft Azure: Databricks
Rating 4.6 of 63 users Rating 4.6 of 63 users (63)