Continuous Data: Ingesting Continuous Data in Snowflake

Snowflake    |    Intermediate
  • 7 videos | 48m 20s
  • Includes Assessment
  • Earns a Badge
Rating 4.8 of 33 users Rating 4.8 of 33 users (33)
Data is generally processed using a batch or stream methodology depending on how much time between data generation and processing is acceptable. The Snowflake feature Snowpipes processes data in micro-batches which fall in between these two scenarios. In this course, you will cover the implementation of Snowpipes when data is sourced from an internal Snowflake stage. You will kick things off by looking at data ingestion options in Snowflake from a theoretical standpoint, including the differences between bulk data loading and Snowpipes. Then, you get hands-on to set up the infrastructure for data ingestion: an internal stage for CSV data, a destination table for a data load, and a pipe to carry out the load in micro-batches. Next, you will ingest the data into the destination table and explore how this process can be monitored by tracking the pipe status. Finally, you will implement a Snowflake task to trigger a Snowpipe at regular time intervals.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Outline how batch, stream, and micro-batch processing works
    Recognize how the snowpipe service works in snowflake
    Set up the stage, file format, and data for loading continuous data
  • Create a snowpipe and view its metadata
    Load data into a table automatically using a snowflake pipe
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 2m 10s
  • 7m 50s
  • Locked
    3.  Continuous Data Ingestion with Snowpipes
    9m 5s
  • Locked
    4.  Setting Up a Table and Stage for Data Ingestion
    11m 3s
  • Locked
    5.  Creating a Snowflake Pipe
    6m 39s
  • Locked
    6.  Ingesting Data with a Snowpipe
    9m 47s
  • Locked
    7.  Course Summary
    1m 47s

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Data Transformation Using the Snowpark API
Rating 5.0 of 1 users Rating 5.0 of 1 users (1)
Course Snowflake Performance: Clustering and Search Optimization
Rating 4.0 of 2 users Rating 4.0 of 2 users (2)
Course Snowflake Performance: Query Acceleration and Caching
Rating 4.7 of 3 users Rating 4.7 of 3 users (3)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Queries in Snowflake: Getting Started with Performance Optimizations
Rating 4.6 of 51 users Rating 4.6 of 51 users (51)
Course Advanced Analytics: Performing Analytics Using Snowflake
Rating 4.6 of 48 users Rating 4.6 of 48 users (48)
Course Data Loading in Snowflake: Using External Stages
Rating 4.6 of 32 users Rating 4.6 of 32 users (32)