Apache Kafka: Integration with Spark

Kafka 2.1    |    Intermediate
  • 10 videos | 1h 2m 59s
  • Includes Assessment
  • Earns a Badge
Rating 4.6 of 19 users Rating 4.6 of 19 users (19)
Apache Kafka can easily integrate with Apache Spark to allow processing of the data entered into Kafka. Discover how to integrate Kafka with Spark.

WHAT YOU WILL LEARN

  • Install and configure the spark streaming package for kafka
    Read data into spark from kafka
    Read data in parallel into spark from kafka
    Write data back to kafka from spark
    Write data back to kafka from spark in parallel
  • Create a direct stream to access kafka data from spark
    Use locationstrategies and consumerstrategies to improve performance
    Use an rdd in cases where batch processing would be a better solution
    Use offsets to handle exactly-once semantics
    Use kafka and spark to split words from sentences

IN THIS COURSE

  • 3m 59s
    Find out how to install and configure the Spark Streaming package for Kafka. FREE ACCESS
  • 6m 25s
    Find out how to read data into Spark from Kafka. FREE ACCESS
  • Locked
    3.  Reading Data from Kafka in Parallel
    3m 13s
    In this video, learn how to read data in parallel into Spark from Kafka. FREE ACCESS
  • Locked
    4.  Writing Data to Kafka
    6m 1s
    In this video, you will learn how to write data from Spark back to Kafka. FREE ACCESS
  • Locked
    5.  Writing Data to Kafka in Parallel
    8m 34s
    In this video, find out how to write data back to Kafka from Spark in parallel. FREE ACCESS
  • Locked
    6.  Creating a Direct Stream
    5m 43s
    During this video, you will learn how to create a direct stream to access Kafka data from Spark. FREE ACCESS
  • Locked
    7.  Planning for Location and Consumer Strategies
    4m 26s
    In this video, you will learn how to use Location Strategies and Consumer Strategies to improve performance. FREE ACCESS
  • Locked
    8.  Using an RDD
    5m 20s
    In this video, you will learn how to use an RDD in cases where batch processing would be more efficient. FREE ACCESS
  • Locked
    9.  Managing Offsets
    12m 28s
    Learn how to use offsets to handle semantics that occur only once. FREE ACCESS
  • Locked
    10.  Creating a Kafka-Spark Word-splitting Application
    6m 50s
    In this video, you will use Kafka and Spark to split sentences into words. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Book Kafka Streams in Action: Event-Driven Applications and Microservices, Second Edition
Course Spark for High-speed Big Data Analytics
Rating 4.5 of 62 users Rating 4.5 of 62 users (62)
Journey Data Infrastructure with Apache Cassandra
Rating 5.0 of 1 users Rating 5.0 of 1 users (1)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Book Official Google Cloud Certified Professional Data Engineer Study Guide
Course Apache Spark Getting Started
Rating 4.5 of 137 users Rating 4.5 of 137 users (137)
Course MongoDB Aggregation
Rating 4.4 of 28 users Rating 4.4 of 28 users (28)