Faust: Performing Operations & Maintaining State Using Tables

Faust 1.10    |    Expert
  • 15 videos | 1h 37m 43s
  • Includes Assessment
  • Earns a Badge
Rating 3.5 of 2 users Rating 3.5 of 2 users (2)
Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Perform group-by operations on streams
    Perform group-by and items operations on streams
    Access raw events and buffer events before performing operations
    Enumerate entities in a stream with a single agent and with multiple agents
    Forward messages to destination topics
    Filter stream records
    Perform word count using tables
  • Save table state to an embedded rocksdb database
    Perform grouping operations and understand table sharding
    Compute aggregations on streaming data
    Perform multiple grouping operations with the right agent structure
    Process streaming elements using multiple worker instances
    Recall the different kinds of sinks that can be used with a faust agent
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 2m 12s
    In this video, you’ll learn more about the instructor and the course. In this course, you’ll learn more about the operations supported by streams in Faust. You will discover how the groupBy operation allows you to designate a key that is used to repartition an input stream and create a new topic in Kafka. You’ll also learn how to use tables in Faust for stateful stream processing.   FREE ACCESS
  • 7m 55s
    In this video, you’ll watch a demo that explores some of the operations you can perform on input streams using Faust stream processing. You’ll work within a new folder, using your current working directory. You’ll learn more about how these operations work, and how data from an input stream is sent to agents for processing. FREE ACCESS
  • Locked
    3.  Performing Group-by and Items Operations in Faust
    8m 50s
    In this video, you’ll watch a demo that shows how you can use the items operation on a stream to iterate over both the key as well as the value for all of the messages in the input stream. You’ll use the same setup for this stream processing application. You’ll follow onscreen directions and see the key variable contains the message key. And you’ll discover the loan record is the actual message value. FREE ACCESS
  • Locked
    4.  Accessing Raw Events and Buffering Events in Faust
    8m 14s
    In this video, you will watch a demo. Here, you’ll learn that within Faust, if you need access to this metadata, it's possible to get access using the events operation. Here, you’ll learn to the events operation that will allow you to iterate over raw event values including access to the original message payload and message metadata. In this demo, you'll use a subset of the metadata information that is accessible for every event. FREE ACCESS
  • Locked
    5.  Enumerating with Single and Multiple Agent Instances
    5m 6s
    In this video, you will watch a demo. In this demo, you will learn another operation that can be used to perform an input stream in Faust. You'll follow directions to learn how to get an index associated with every element of a string using enumerate. You will set up the application and set up a topic to listen to. FREE ACCESS
  • Locked
    6.  Forwarding Messages to Destination Topics via Faust
    6m 5s
    In this video, you will watch a demo. In this demo, you will learn that the stream processing you want to perform on messages you receive from a topic is to simply forward those messages to a new different topic. Following onscreen directions, you’ll learn this is possible using the through operation in Faust. Onscreen, you’ll see an async for loop, which iterates over the loan records in your stream. FREE ACCESS
  • Locked
    7.  Filtering Stream Records Using Faust
    3m 10s
    In this video, you will watch a demo. In this demo, you’ll learn how to filter out a record before you perform stream processing. You’ll learn how to use the filter operation. The filter operation operates on the stream before you process the stream, which allows you to filter records before repartitioning the stream. Filtering omits records that you don’t want. FREE ACCESS
  • Locked
    8.  Performing Word Count Using Faust Tables
    11m 18s
    In this video, you will watch a demo. In this demonstration, you’ll explore how stream processing can be more useful if you have the ability to maintain state. You’ll learn how to maintain state using tables. First, you’ll look at our current working directory because it’s changed. You’ll open a folder, then a subfolder following the onscreen directions. Then, you’ll perform a basic hello world operation for stateful stream processing the word count. FREE ACCESS
  • Locked
    9.  Saving Table State to RocksDB Using Faust
    7m 2s
    In this video, you will watch a demo. You’ll use the current application under the tables folder. You’ll discover every Faust application has a data directory. In this demo, you’ll follow onscreen directions to utilize a data directory called word-count-data and its subfolder, called tables. You’ll learn that any data associated with a worker instance will be stored on the same hardware where your worker instance runs. Following onscreen directions, you will run code. FREE ACCESS
  • Locked
    10.  Grouping Operations and Table Sharding in Faust
    5m 53s
    In this video, you will watch a demo. In this demonstration, you will move beyond word count and return to the earlier loan processing application. You’ll following onscreen directions to utilize tables to count the number of applicants with the different kinds of degrees. You’ll learn a grouping and counting operation using tables. You'll also learn how tables and grouping go together when working in a production environment. FREE ACCESS
  • Locked
    11.  Computing Aggregations on Streaming Data Using Faust
    7m 5s
    In this video, you will watch a demo. In this demonstration, you will learn how to compute the average income for loan applicants based on their degree. You’ll follow directions to compute an average for applicants with high school degrees, with college degrees, and then graduate degrees. You’ll set up a model of a Faust application and the loan details topic to reach real listen for a publish messages. FREE ACCESS
  • Locked
    12.  Performing Multiple Grouping Operations in Faust
    5m 29s
    In this video, you’ll watch a demo. First, you’ll review how grouping works in Faust. You’ll learn that when you perform a grouping operation, your stream is repartitioned and the key that you group by is associated with every message in your input stream. You’ll discover Faust ensures all messages with the same key are sent for processing to the same worker instance. Following directions, you’ll look at three different workers running stream processing code. FREE ACCESS
  • Locked
    13.  Processing with Multiple Workers in Faust
    5m 24s
    In this video, you’ll watch a demo. In this demo, you'll see how to run multiple worker instances to process your streaming data. You'll bring up these workers on the same local machine. You’ll use a grouping operation to see how all messages with the same key are always sent to the same worker instance. FREE ACCESS
  • Locked
    14.  Working with Sinks in Faust
    11m 47s
    In this video, you’ll watch a demo. In this demo, you'll learn how you can use sinks in your stream processing. First, you’ll learn what a sink is in Faust. These are a part of your stream processing in Faust. They are used to perform additional actions after an agent has completed processing an event. A sink is used when you've completed processing your event.   FREE ACCESS
  • Locked
    15.  Course Summary
    2m 13s
    In this video, you’ll summarize the course. In this course, you’ve learned how to perform stateful transformation on input streams using force tables and explores the range of operations that you can perform on streams, such as grouping, filtering, and aggregations. You learned about the wide range of operations supported by streams and Faust. And you discovered how the group by operation allows you to designate a key that’s used to repartition or input stream. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Processing Data: Integrating Kafka with Python & Using Consumer Groups
Rating 4.7 of 31 users Rating 4.7 of 31 users (31)
Course Continuous Data Pipelines and Dynamic Tables in Snowflake
Rating 5.0 of 1 users Rating 5.0 of 1 users (1)
Book Kafka Streams in Action: Event-Driven Applications and Microservices, Second Edition

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course CompTIA Network+: Network Services & Data Center Architectures
Rating 4.5 of 82 users Rating 4.5 of 82 users (82)
Course Developing Apps with Tkinter: Menu Bars, Seaborn Visualization, & MySQL DBs
Rating 4.8 of 6 users Rating 4.8 of 6 users (6)
Course Faust: Stream Processing Using Windowing Operations
Rating 3.7 of 6 users Rating 3.7 of 6 users (6)