Streaming Data Architectures: An Introduction to Streaming Data in Spark

Data Science    |    Intermediate
  • 9 videos | 50m 9s
  • Includes Assessment
  • Earns a Badge
Rating 4.2 of 32 users Rating 4.2 of 32 users (32)
Learn the fundamentals of streaming data with Apache Spark. During this course, you will discover the differences between batch and streaming data. Observe the types of streaming data sources. Learn about how to process streaming data, transform the stream, and materialize the results. Decouple a streaming application from the data sources with a message transport. Next, learn about techniques used in Spark 1.x to work with streaming data and how it contrasts with processing batch data; how structured streaming in Spark 2.x is able to ease the task of stream processing for the app developer; and how streaming processing works in both Spark 1.x and 2.x. Finally, learn how triggers can be set up to periodically process streaming data; and the key aspects of working with structured streaming in Spark

WHAT YOU WILL LEARN

  • Recognize the differences between batch and streaming data and the types of streaming data sources
    List the steps in involved in processing streaming data, the transformation of streams, and the materialization of the results of the transformation
    Describe how the use of a message transport decouples a streaming application from the sources of streaming data
    Describe the techniques used in spark 1.x to work with streaming data and how it contrasts with processing batch data
  • Recall how structured streaming in spark 2.x is able to ease the task of stream processing for the app developer
    Compare how streaming processing works in both spark 1.x and 2.x
    Recognize how triggers can be set up to periodically process streaming data and describe the various output modes available to publish the results of stream processing
    Recognize the key aspects of working with structured streaming in spark

IN THIS COURSE

  • 2m 29s
  • 7m 18s
    Upon completion of this video, you will be able to recognize the differences between batch and streaming data, and the types of streaming data sources. FREE ACCESS
  • Locked
    3.  The Stream Processing Model
    3m 57s
    Upon completion of this video, you will be able to list the steps involved in processing streaming data, the transformation of streams, and the materialization of the results of the transformation. FREE ACCESS
  • Locked
    4.  The Message Transport
    4m 57s
    Upon completion of this video, you will be able to describe how the use of a message transport decouples a streaming application from the sources of streaming data. FREE ACCESS
  • Locked
    5.  Stream Processing with RDDs
    6m 19s
    After completing this video, you will be able to describe the techniques used in Spark 1.x to work with streaming data and how it contrasts with processing batch data. FREE ACCESS
  • Locked
    6.  Structured Streaming for Continuous Applications
    6m 3s
    After completing this video, you will be able to recall how structured streaming in Spark 2.x eases the task of stream processing for the app developer. FREE ACCESS
  • Locked
    7.  Streaming vs Structured Streaming
    4m 37s
    In this video, you will compare how streaming processing works in Spark 1.x and 2.x. FREE ACCESS
  • Locked
    8.  Triggers and Output Modes
    7m 21s
    Upon completion of this video, you will be able to recognize how triggers can be set up to periodically process streaming data and describe the various output modes available to publish the results of stream processing. FREE ACCESS
  • Locked
    9.  Exercise: Working with Streaming Data
    7m 7s
    After completing this video, you will be able to recognize the key aspects of working with structured streaming in Spark. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Processing Data: Integrating Kafka with Apache Spark
Rating 4.7 of 29 users Rating 4.7 of 29 users (29)
Course Spark for High-speed Big Data Analytics
Rating 4.5 of 62 users Rating 4.5 of 62 users (62)
Book Spark in Action, Second Edition

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Implementing Governance Strategies
Rating 4.6 of 48 users Rating 4.6 of 48 users (48)
Course Conditional Statements & Loops: If-else Control Structures in Python
Rating 4.4 of 621 users Rating 4.4 of 621 users (621)
Course Generative AI and Its Impact to Everyday Business
Rating 4.6 of 893 users Rating 4.6 of 893 users (893)