Faust: Getting Started with Stream Processing

IN THIS COURSE

• 1.  Course Overview
  2m 25s
  In this video, you’ll learn more about your instructor and the course. In this course, you’ll learn the basics of stream processing and how it differs from batch processing. You’ll also learn the components that make up the architecture of a stream processing system. You’ll then learn about the Faust stream processing library, which uses native Python code for transformations on streaming data. You’ll also learn concepts of producers, consumers, and topics in Apache Kafka. FREE ACCESS
• 2.  Batch Processing Characteristics
  8m 27s
  In this video, you’ll learn about Batch Data. You’ll discover Batch Data is easier to process. Data you're processing is referred to as Batch Data when the dataset is bounded and a fixed size. Batch Data is when you know all the records you're going to process upfront before processing begins. The dataset is bounded, which means it's not being constantly added to. Batch Data is typically stored in traditional storage systems. FREE ACCESS
• 3.  Stream Processing Characteristics
  9m 31s
  In this video, you’ll learn about Streaming Data. You’ll discover Streaming Data is an unbounded dataset of infinite size. Streaming Data is an infinite dataset that never ends. This data is constantly being updated. A stream is real-time data. With Streaming Data, new items come in and are being constantly added to your dataset, while processing is ongoing. This is the most important and relevant difference between Batch Data and Streaming Data. FREE ACCESS
• 4.  Stream Processing Architectural Components
  8m 54s
  In this video, you’ll learn more about the typical architecture of a Stream processing system. You’ll learn about general architecture and how streaming data is processed by the different frameworks that we have to support streams. You’ll discover streaming data is received and processed in real-time and this stream can be received from multiple different streaming sources. You’ll learn the data you receive in the stream does not have a reliable source of truth. FREE ACCESS
• 5.  Element-wise Transformations and Aggregations
  8m 45s
  In this video, you’ll learn about the different types of stream processing applied to input elements. You’ll learn there are different ways to process streaming elements. You’ll discover the simplest kind of transformations applied input data are element-wise transformations. Element-wise transformations look at one event or one entity in the input stream at a time. You’ll perform all transformations on that entity. Stream processing systems make decisions based on the value of that event. FREE ACCESS
• 6.  Faust Stream Processing Application Characteristics
  11m 25s
  In this video, you’ll learn about the technology that you'll use for Stream processing. You’ll learn Faust is a Stream processing system. Faust is simple, intuitive, performant, and fast. It’s completely native Python-based. Because Faust is built in Python and uses Python APIs makes it very easy to perform Stream processing. Everything you perform on input streams involves code in basic straightforward Python. You’ll learn Faust Stream processing integrates with the Python AsyncIO module. FREE ACCESS
• 7.  The Core Characteristics of Apache Kafka
  8m 29s
  In this video, you’ll learn about Apache Kafka. You’ll discover Faust integrates with Apache Kafka and reads messages from Kafka topics. To work with Faust, you’ll need an understanding Apache Kafka. Here, you’ll learn what exactly Apache Kafka is and how it works. Apache Kafka is an open-source Stream processing software that provides a high throughput low-latency platform for real-time data feeds. Apache Kafka can perform Stream processing operations using Kafka streams. FREE ACCESS
• 8.  Installing Apache Kafka and the Faust Library
  8m 12s
  In this video, you’ll watch a demo on installing Apache Kafka. You’ll learn Faust integrates closely with Kafka to read input streams of messages. In this demo, you’ll also download and install Faust specific Python libraries. First, you’ll make sure you have Python installed on your machine, with a Python version greater than or equal to 3.6. FREE ACCESS
• 9.  Using Producers, Consumers, & Topics in Apache Kafka
  10m 12s
  In this video, you’ll watch a demo. You’ll learn how Apache Kafka works. You’ll understand producers, consumers, and topics in Kafka. You’ll learn to use the console scripts in Kafka to set up producers to send messages to a particular topic. You’ll also learn how to create topics, list topics, and then use consumers to read messages from our topic. FREE ACCESS
• 10.  Implementing a Simple Faust Application
  10m 10s
  In this video, you’ll watch a demo. In this demo, you’ll learn to write your first Faust program in Python, to process an input data stream. You’ll need to have ZooKeeper and the Kafka server up and running. First, make sure your terminal windows with ZooKeeper and Kafka server look like the ones onscreen. Now, you’re ready to write code in Faust. You’ll write code within your current working directory. This working directory is Projects/Skillsoft. FREE ACCESS
• 11.  Testing a Faust Application
  9m 28s
  In this video, you’ll watch a demo. In this demo, you’ll open the terminal window where you'll run the code for your first Faust application. You’ll learn a Faust worker starts a single instance of your Faust application. You've instantiated your Faust application in your code. If you want to run one instance of this application, you'll start a single worker. A single worker typically runs on one of your cluster machines. FREE ACCESS
• 12.  Using the 'faust' Command to Run Workers
  8m 22s
  In this video, you’ll watch a demo. In this demo, you’ll go back to your Faust streaming application and tweak the code. Everything else remains the same. You’ll import Faust and instantiate the Faust application on line 4. Then, you’ll create a new topic called messages. You’ll look at the process_messages code, which you’ve decorated using the app.agent decorator. This is the code that will process your streaming messages. FREE ACCESS
• 13.  Course Summary
  2m 10s
  In this video, you’ll review what you’ve learned in the course. In this course, you’ve learned the basic characteristics of batch data and batch processing. You also learned about the types of transformations you can perform on streaming data. You learned to use Faust for stream processing a pure Python library that can be installed and run on a cluster of machines. You learned Faust has built-in support for reading messages from Kafka topics. FREE ACCESS