Apache Kafka: intermediate Kafka 2.1
Expertise:
Technology:
- 6 Courses | 5h 6m 55s
- 3 Books | 12h 24m
- Includes Lab
- 5 Courses | 7h 7m 47s
- 3 Books | 15h 46m
- 1 Audiobook | 7h 3m 42s
- 2 Courses | 1h 53m 18s
- 3 Books | 12h 24m
- Includes Lab
Apache Kafka is an open-source stream processing platform for providing a unified, high-throughput, low-latency and real-time data feeds.
GETTING STARTED
Processing Data: Getting Started with Apache Kafka
-
1m 20s
-
10m 22s
COURSES INCLUDED
Apache Kafka: Concepts & Components
Apache Kafka is a distributed platform that lets you publish, store, and process streams of records. Here we look at the basic concepts of Kafka, and examine its main components.
8 videos |
33m
Assessment
Badge
Apache Kafka: Deployment & Operating Tasks
To get started with Apache Kafka, you must first deploy it. Let's examine the specification requirements, deployment to various platforms, and some of the main operating tasks you can perform in Kafka.
9 videos |
53m
Assessment
Badge
Apache Kafka: Operations
To maintain peak performance of your Kafka deployment, you must monitor and manage the servers. Learn how to use the tools available for monitoring Kafka, as well as how to perform tuning for Kafka.
9 videos |
36m
Assessment
Badge
Apache Kafka: Development
Apache Kafka comes with a set of APIs for consumers and producers to write to and read from logs. Explore both the producer and consumer APIs, data serialization/deserialization techniques, and strategies for testing Kafka.
18 videos |
1h 14m
Assessment
Badge
Apache Kafka: Integration with Spark
Apache Kafka can easily integrate with Apache Spark to allow processing of the data entered into Kafka. Discover how to integrate Kafka with Spark.
10 videos |
1h 2m
Assessment
Badge
Apache Kafka: Integration with Storm
Apache Kafka can easily integrate with Apache Storm to allow processing of the data entered into Kafka. Discover how to integrate Kafka with Storm.
8 videos |
46m
Assessment
Badge
SHOW MORE
FREE ACCESS
COURSES INCLUDED
Processing Data: Getting Started with Apache Kafka
Apache Kafka is a popular event streaming platform used by Fortune 100 companies for both real-time and batch data processing. In this course, you will explore the characteristics of event streaming and how Kafka architecture allows for scalable streaming data. Install Kafka and create some topics, which are essentially channels of communication between apps and data. Set up and work with multiple topics for durable storage. Create multiple brokers and cluster of nodes to handle messages and store their replicas. Then, monitor the settings and logs for those brokers. Finally, see how topic partitions and replicas provide redundancy and maintain high availability.
11 videos |
1h 31m
Assessment
Badge
Processing Data: Integrating Kafka with Python & Using Consumer Groups
Producers and consumers are applications that write events to and read events from Kafka. In this course, you will focus on integrating Python applications with a Kafka environment, implementing consumer groups, and tweaking Kafka configurations. Begin by connecting to Kafka from Python. You will produce to and consume messages from a Kafka topic using Python. Next, discover how to tweak Kafka broker configurations. You will place limits on the size of messages and disable deletion of topics. Then, publish messages to partitioned topics and explore the use of partitioning algorithms to determine the placement of messages on partitions. Explore consumer groups, which allow a set of consumers to process messages published to partitioned Kafka topics in parallel - without any duplication of effort. Finally, learn different ways to optimize Kafka's performance, using configurations for brokers and topics, as well as producer and consumer apps.
12 videos |
1h 24m
Assessment
Badge
Processing Data: Introducing Apache Spark
Apache Spark is a powerful distributed data processing engine that can handle petabytes of data by chunking that data and dividing across a cluster of resources. In this course, explore Spark's structured streaming engine, including components like PySpark shell. Begin by downloading and installing Apache Spark. Then create a Spark cluster and run a job from the PySpark shell. Monitor an application and job runs from the Spark web user interface. Then, set up a streaming environment, reading and manipulating the contents of files that are added to a folder in real-time. Finally, run apps on both Spark standalone and local modes.
13 videos |
1h 44m
Assessment
Badge
Processing Data: Integrating Kafka with Apache Spark
Flexible and Intuitive, DataFrames are a popular data structure in data analytics. In this course, build Spark applications that process data streamed to Kafka topics using DataFrames. Begin by setting up a simple Spark app that streams in messages from a Kafka topic, processes and transforms them, and publishes them to an output sink. Next, leverage the Spark DataFrame application programming interface by performing selections, projections, and aggregations on data streamed in from Kafka, while also exploring the use of SQL queries for those transformations. Finally, you will perform windowing operations - both tumbling windows, where the windows do not overlap, and sliding windows, where there is some overlapping of data.
12 videos |
1h 45m
Assessment
Badge
Processing Data: Using Kafka with Cassandra & Confluent
Apache Cassandra is a trusted open-source NoSQL distributed database that easily integrates with Apache Kafka as part of an ETL pipeline. This course focuses on that integration of Kafka, Spark and Cassandra and explores a managed version of Kafka with the Confluent data streaming platform. Begin by integrating Kafka with Apache Cassandra as part of an ETL pipeline involving a Spark application. Discover Apache Cassandra and learn the steps involved in linking Spark with this wide-column database. Next, examine the various features of the Confluent platform and find out how easy it is to set up and work with a Kafka environment. After completing this course, you will be prepared to implement and manage steam processing systems in your organization.
7 videos |
41m
Assessment
Badge
SHOW MORE
FREE ACCESS
COURSES INCLUDED
Apache Kafka: Clustering
What makes Apache Kafka so powerful and fault-tolerant is its clustering capabilities. Discover how to create and manage clusters in Kafka.
10 videos |
54m
Assessment
Badge
Apache Kafka: Real-time Applications
A major feature of Apache Kafka is building real-time applications that react to data streams. Discover how to create real-time applications in Kafka.
9 videos |
58m
Assessment
Badge
EARN A DIGITAL BADGE WHEN YOU COMPLETE THESE COURSES
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.
Digital badges are yours to keep, forever.BOOKS INCLUDED
Book
Kafka in ActionStarting with an overview of Kafka's core concepts, you'll immediately learn how to set up and execute basic data movement tasks and how to produce and consume streams of events. Advancing quickly, you'll soon be ready to use Kafka in your day-to-day workflow, and start digging into even more advanced Kafka topics.
4h 23m
By Dave Klein, Dylan Scott, Viktor Gamov
Book
Big Data SMACK: A Guide to Apache Spark, Mesos, Akka, Cassandra, and KafkaExplaining each of the full-stack technologies and, more importantly, how to best integrate them, this book provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples in every situation.
3h 56m
By Isaac Ruiz, Raul Estrada
Book
Kafka Streams in Action: Real-time Apps and Microservices with the Kafka Streams APIThis book teaches you everything you need to know to implement stream processing on data flowing into your Kafka platform, allowing you to focus on getting more from your data without sacrificing time or effort.
4h 5m
By William P. Bejeck Jr.
BOOKS INCLUDED
Book
Kafka in ActionStarting with an overview of Kafka's core concepts, you'll immediately learn how to set up and execute basic data movement tasks and how to produce and consume streams of events. Advancing quickly, you'll soon be ready to use Kafka in your day-to-day workflow, and start digging into even more advanced Kafka topics.
4h 23m
By Dave Klein, Dylan Scott, Viktor Gamov
Book
Kafka Troubleshooting in Production: Stabilizing Kafka Clusters in the Cloud and On-premisesThis book provides Kafka administrators, site reliability engineers, and DataOps and DevOps practitioners with a list of real production issues that can occur in Kafka clusters and how to solve them.
3h 15m
By Elad Eldor
Book
Kafka Streams in Action: Event-Driven Applications and Microservices, Second EditionEverything you need to implement stream processing on Apache KafkaⓇ using Kafka Streams and the kqsIDB event streaming database.
8h 8m
By William P. Bejeck Jr.
BOOKS INCLUDED
Book
Kafka in ActionStarting with an overview of Kafka's core concepts, you'll immediately learn how to set up and execute basic data movement tasks and how to produce and consume streams of events. Advancing quickly, you'll soon be ready to use Kafka in your day-to-day workflow, and start digging into even more advanced Kafka topics.
4h 23m
By Dave Klein, Dylan Scott, Viktor Gamov
Book
Big Data SMACK: A Guide to Apache Spark, Mesos, Akka, Cassandra, and KafkaExplaining each of the full-stack technologies and, more importantly, how to best integrate them, this book provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples in every situation.
3h 56m
By Isaac Ruiz, Raul Estrada
Book
Kafka Streams in Action: Real-time Apps and Microservices with the Kafka Streams APIThis book teaches you everything you need to know to implement stream processing on data flowing into your Kafka platform, allowing you to focus on getting more from your data without sacrificing time or effort.
4h 5m
By William P. Bejeck Jr.
AUDIOBOOKS INCLUDED
Audiobook
Kafka in ActionMaster the wicked-fast Apache Kafka streaming platform through hands-on examples and real-world projects.
7h 3m 42s
By Dave Klein, Dylan Scott, Viktor Gamov
SKILL BENCHMARKS INCLUDED
Data Infrastructures with Apache Kafka Competency (Intermediate Level)
The Data Infrastructures with Apache Kafka Competency (Intermediate Level) benchmark measures your ability to recognize the basics, use cases, and core concepts of Apache Kafka. You will be evaluated on your skills in building Python applications that produce messages for and consume messages from Kafka topics, creating and using consumer groups, and developing Apache Spark applications that can manipulate streaming data. A learner who scores high on this benchmark demonstrates that they have the ability to work with Apache Kafka with minimal supervision.
18m
| 18 questions
Data Infrastructures with Apache Kafka Proficiency (Advanced Level)
The Data Infrastructures with Apache Kafka Proficiency (Advanced Level) benchmark measures your ability to build Apache Spark applications that process streaming data published to Kafka. You will be evaluated on your skills in applying transformations based on aggregations and window operations, setting up and managing a Kafka environment using Confluent, and defining an ETL pipeline involving Kafka, Spark, and Cassandra. A learner who scores high on this benchmark demonstrates that they have the skills to work on Kafka without any supervision.
16m
| 16 questions
YOU MIGHT ALSO LIKE
Rating 4.7 of 31 users
(31)