Microsoft Fabric: Spark & the Capacity Metrics App for Lakehouses
Microsoft Fabric 2024
| Expert
- 15 videos | 1h 57m 51s
- Includes Assessment
- Earns a Badge
Spark is a key technology on Fabric Lakehouses, and a fundamental part of the DP-600 test curriculum. In this course, you'll learn how Apache Spark integrates with Microsoft Fabric to handle large-scale data processing through distributed computing. First, learn about Spark pools and study the role of the T-SQL endpoint. Create Fabric shortcuts, set up storage accounts, enable hierarchical namespaces, and use Shared Access Signatures (SAS) to link these sources and build Delta tables from the connected data. Next, create notebooks with Apache Spark in Microsoft Fabric, run PySpark and SparkSQL commands, monitor resource usage and learn how to associate lakehouses with notebooks. Finally, explore the Microsoft Fabric Capacity Metrics App, tracking capacity units (CUs), managing SKUs, and handling overages and throttling. Complete the course by installing the app, entering your Fabric capacity ID, and using charts to analyze utilization metrics. This course is part of a series that prepares learners for Exam DP-600: Implementing Analytics Solutions Using Microsoft Fabric.
WHAT YOU WILL LEARN
-
Discover the key concepts covered in this course
Outline key attractions, features and terms related to apache spark and spark on fabric
Outline the role of the t-sql endpoint and semantic models in working with data lakehouses
Define fabric shortcuts, enumerate their types and outline the role they play in lakehouses
Create an adls gen2 storage account from azure and connect to it from fabric via a shortcut
Create delta tables based on data connected via a shortcut and study how updates propagate through to the delta table
Create onelake shortcuts with an amazon s3 bucket as the underlying data source
Analyze spark pools, starter pools, spark environments and run spark on fabric in high concurrency mode -
Create a notebook, associate a lakehouse with it, and then run various pyspark and sparksql commands
Perform grouping and aggregation operations in pyspark and sparksql
Write dataframes out to managed delta tables, as well as to parquet and json files
Analyze fabric capacities, skus, and capacity units (cus) and outline responses to overages, from overage protection to background rejection
Install the microsoft fabric capacity metrics app from microsoft appsource and enter our fabric capacity id to complete the process
Analyze utilization and other usage metrics, as well as throttling and overages in the fabric capacity metrics app
Summarize the key concepts covered in this course
IN THIS COURSE
-
2m 14sIn this video, you will discover the key concepts covered in this course. FREE ACCESS
-
5m 55sAfter completing this video, you will be able to outline key attractions, features and terms related to Apache Spark and Spark on Fabric. FREE ACCESS
-
3. The T-SQL Endpoint & Semantic Models7m 21sIn this video, find out how to outline the role of the T-SQL endpoint and semantic models in working with data lakehouses. FREE ACCESS
-
4. Data Lakehouse Shortcuts5m 6sDuring this video, discover how to define Fabric shortcuts, enumerate their types and outline the role they play in lakehouses. FREE ACCESS
-
5. Creating a Shortcut in Fabric11m 17sIn this video, you will learn how to create an ADLS Gen2 storage account from Azure and connect to it from Fabric via a shortcut. FREE ACCESS
-
6. Creating Dynamic Shortcuts in Fabric8m 16sLearn how to create delta tables based on data connected via a shortcut and study how updates propagate through to the delta table. FREE ACCESS
-
7. Creating OneLake Shortcuts for Amazon S3 Buckets12m 53sUpon completion of this video, you will be able to create OneLake shortcuts with an Amazon S3 bucket as the underlying data source. FREE ACCESS
-
8. Apache Spark in Microsoft Fabric6m 32sFind out how to analyze Spark pools, starter pools, Spark environments and run Spark on Fabric in high concurrency mode. FREE ACCESS
-
9. Working with Lakehouses, Notebooks, & Spark Commands10m 42sIn this video, learn how to create a notebook, associate a lakehouse with it, and then run various PySpark and SparkSQL commands. FREE ACCESS
-
10. Grouping & Aggregating in SparkSQL & PySpark8m 7sDuring this video, discover how to perform grouping and aggregation operations in PySpark and SparkSQL. FREE ACCESS
-
11. Parquet, JSON Files, & Delta Tables Dataframes9m 40sAfter completing this video, you will be able to write dataframes out to managed Delta tables, as well as to parquet and JSON files. FREE ACCESS
-
12. The Fabric Capacity Metrics App7m 27sLearn how to analyze Fabric capacities, SKUs, and capacity units (CUs) and outline responses to overages, from overage protection to background rejection. FREE ACCESS
-
13. Installing the Microsoft Fabric Capacity Metrics App8m 9sIn this video, you will learn how to install the Microsoft Fabric Capacity Metrics App from Microsoft AppSource and enter our Fabric capacity ID to complete the process. FREE ACCESS
-
14. Using the Microsoft Fabric Capacity Metrics App11m 44sFind out how to analyze utilization and other usage metrics, as well as throttling and overages in the Fabric Capacity Metrics App. FREE ACCESS
-
15. Course Summary2m 27sIn this video, we will summarize the key concepts covered in this course. FREE ACCESS
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.
Digital badges are yours to keep, forever.
