Data Lakes on AWS
Amazon Web Services
| Intermediate
- 12 videos | 1h 9m 14s
- Includes Assessment
- Earns a Badge
This course discusses the transition of data warehousing to cloud-based solutions using the AWS (Amazon Web Services) cloud platform. In 11 videos, the course explores how data lakes store data using a flat structure, and the data are tagged, making it easy to search and query. You will learn how to build a data lake on the AWS cloud by storing data in S3 (simple storage service) buckets. You will learn to set up your data lake architecture lake using AWS Glue, a fully managed ETL (extract, transform, load) service. You will learn to configure and run Glue crawlers, and you will examine how crawlers merge data stored in an S3 folder path; and to use S3 to generate metadata tables in Glue. Learners will use Athena, Amazon's interactive query service as a simple way to analyze data in S3 using standard SQL. Finally, you will examine how to merge the data crawled by our CSV (comma separated values) crawler into a single table.
WHAT YOU WILL LEARN
-
Configure a custom role with specific permissions on aws
Create an s3 bucket and upload files
Recognize the different operations that can be performed using the aws glue console
Create metadata tables in glue using the web console
Perform queries on the glue data catalog using athena
Perform data crawling on s3 to automatically detect schemas -
Execute queries on data in crawled tables
Perform crawling operations with multiple files in the same path
Merge data stored in multiple files in the same folder path
Merge data when files have the exact same schema
Recall the roles and features of the different aws services used in the data lake architecture
IN THIS COURSE
-
1m 37s
-
7m 11sDuring this video, you will learn how to configure a custom role with specific permissions on Amazon Web Services. FREE ACCESS
-
3. Upload Data to S35m 41sIn this video, you will learn how to create an S3 bucket and upload files. FREE ACCESS
-
4. Explore the Glue Web Console3m 17sUpon completion of this video, you will be able to recognize the different operations that can be performed using the AWS Glue console. FREE ACCESS
-
5. Manually Create Glue Tables6m 18sLearn how to create metadata tables in Glue using the web console. FREE ACCESS
-
6. Query the Data Lake Using Amazon Athena6m 11sDuring this video, you will learn how to perform queries on the Glue data catalog using Athena. FREE ACCESS
-
7. Configure and Run Glue Crawlers9m 25sTo find out how to automatically detect schemas when performing data crawling on S3, consult a reliable source. FREE ACCESS
-
8. Access Data in Crawled Tables3m 55sFind out how to execute queries on data in tables that have been crawled. FREE ACCESS
-
9. Crawl Multiple CSV Files in the Same Folder Path6m 54sLearn how to perform crawling operations with multiple files in the same directory. FREE ACCESS
-
10. Merge Data in Multiple Files in the Same Folder Path6m 59sDuring this video, you will learn how to merge data stored in multiple files in the same directory. FREE ACCESS
-
11. Work with Files Having the Exact Same Schema6m 47sIn this video, find out how to merge data when files have the same schema. FREE ACCESS
-
12. Exercise: Data Lakes on AWS with S3 and Glue5mUpon completion of this video, you will be able to recall the roles and features of the different AWS services used in the data lake architecture. FREE ACCESS
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.
Digital badges are yours to keep, forever.
