Programming with MapReduce

Apache Hadoop    |    Beginner
  • 16 videos | 1h 6m 43s
  • Includes Assessment
  • Earns a Badge
Rating 4.4 of 28 users Rating 4.4 of 28 users (28)
You must have a good understanding of MapReduce to be able to program with it. Here we look at MapReduce in detail, and demonstrate the basics of programming in MapReduce.

WHAT YOU WILL LEARN

  • Understand the components of a mapreduce job and the steps to create them. explain each job component and it's function in hadoop mapreduce. understand the steps in creating map reduce job components.
    Understand a conceptual example of the mapreduce process and how each piece fits into the overall mapreduce algorithm
    Learn how to use java to write hadoop mapreduce jobs. understand which additional jar to use, and the functionality of the classes within them
    Know how to create and execute hadoop mapreduce jobs. illustrate how to compile and run mapreduce programs.
    Understand the programmatic steps in a hadoop mapreduce job. know how the jobclient, jobtracker, and tasktracker work, and their interaction with the hadoop distributed file system (hdfs).
    Understand the concept of mapreduce chaining. describe how mapreduce jobs may have several steps with the last mapreduce output will be used as input for the next mapreduce job.
    Understanding pre-compile, compile, and run commands. discover different techniques to package and run mapreduce jobs.
    Understand how mapreduce stores and reads big data. demonstrate how mapreduce and hadoop handle data with the hdfs over a distributed processing system.
  • Understand how persistence in the hdfs compares to other file storage systems. learn the specifics of reading and writing data in the hdfs, and it's redundancy across the cluster.
    Overview of how jobs run in mapreduce. introduction to the unit testing process. understand tools and techniques in unit testing.
    Understand how to view mapreduce job status and how to review and understand log files. learn about how log files are handled by different distributions of hadoop.
    Understand scenarios where a mapreduce job would need to be terminated. learn how to use the "-list" and "-kill" commands.
    Overview of junit, and unit testing techniques using junit. understand test cases using junit. overview of junit configuration scripts.
    Explain cloudera mrunit. compare unit testing with mrunit and without mrunit. understand the unit testing process, and unit testing files.
    Understand how to use a dummy cluster for unit and integration testing. learn the basics of a mini-hdfs and a mini-mapreduce cluster.
    Understand the basics of the hadoop localjobrunner. understand that the localjobrunner is a mini version of the mapreduce execution engine. explain how the localjobrunner can run in a debugger and can step through code in mappers and reducers.

IN THIS COURSE

  • 4m 24s
    In this video, you will understand the components of a MapReduce job and the steps to create them. Explain each job component and its function in Hadoop MapReduce. Understand the steps in creating map reduce job components. FREE ACCESS
  • 3m 34s
    In this video, you will understand a conceptual example of the MapReduce process and how each piece fits into the overall MapReduce algorithm. FREE ACCESS
  • Locked
    3.  Hadoop MapReduce and Java
    3m 38s
    In this video, you will learn how to use Java to write Hadoop MapReduce jobs. You will understand which additional JAR to use, and the functionality of the classes within them. FREE ACCESS
  • Locked
    4.  Executing Hadoop MapReduce Jobs
    7m
    In this video, you will learn how to create and execute Hadoop MapReduce jobs. You will also learn how to compile and run MapReduce programs. FREE ACCESS
  • Locked
    5.  Job Flow Progress and Monitoring
    3m 57s
    In this video, you will understand the programmatic steps in a Hadoop MapReduce job. You will know how the JobClient, JobTracker, and TaskTracker work, and their interaction with the Hadoop Distributed File System (HDFS). FREE ACCESS
  • Locked
    6.  Chaining MapReduce Jobs
    3m 15s
    In this video, you will understand the concept of MapReduce chaining. Describe how MapReduce jobs may have several steps with the last MapReduce output being used as input for the next MapReduce Job. FREE ACCESS
  • Locked
    7.  Executing Commands
    7m 37s
    Learn how to understand pre-compile, compile, and run commands. Discover different techniques to package and run MapReduce Jobs. FREE ACCESS
  • Locked
    8.  How Data is Processed, Persisted, and Read on the HDFS
    3m 23s
    In this video, you will understand how MapReduce stores and reads Big Data. You will also see how MapReduce and Hadoop handle data with the HDFS over a distributed processing system. FREE ACCESS
  • Locked
    9.  Understanding Persistence and Types of Big Data
    3m 21s
    In this video, you will understand how persistence in the HDFS compares to other file storage systems. Learn the specifics of reading and writing data in the HDFS, and its redundancy across the cluster. FREE ACCESS
  • Locked
    10.  Understanding High-Level Processing
    3m 46s
    In this video, you will get an overview of how jobs run in MapReduce. You will also learn about the unit testing process and understand tools and techniques in unit testing. FREE ACCESS
  • Locked
    11.  Viewing Job Status and Log Files
    5m 36s
    In this video, you will understand how to view MapReduce job status and how to review and understand log files. Learn about how log files are handled by different distributions of Hadoop. FREE ACCESS
  • Locked
    12.  Killing MapReduce Jobs
    3m 28s
    In this video, you will understand scenarios where a MapReduce job would need to be terminated. Learn how to use the "-list" and "-kill" commands. FREE ACCESS
  • Locked
    13.  Using JUnit
    3m 21s
    In this video, learn how to do an overview of JUnit, and unit testing techniques using JUnit. Understand test cases using JUnit. Overview of JUnit configuration scripts. FREE ACCESS
  • Locked
    14.  Introduction to MRUnit
    3m 12s
    During this video, you will learn how to explain Cloudera MRUnit. Compare unit testing with MRUnit and without MRUnit. Understand the unit testing process, and unit testing files. FREE ACCESS
  • Locked
    15.  Using a Mini MapReduce (and HDFS) Cluster
    3m 37s
    In this video, you will understand how to use a dummy cluster for unit and integration testing. Learn the basics of a mini-HDFS and a mini-MapReduce cluster. FREE ACCESS
  • Locked
    16.  Using the Hadoop LocalJobRunner
    3m 35s
    In this video, you will understand the basics of the Hadoop LocalJobRunner. Understand that the LocalJobRunner is a mini version of the MapReduce execution engine. Explain how the LocalJobRunner can run in a debugger and can step through code in mappers and reducers. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Journey Raspberry Pi Basics
Rating 2.0 of 1 users Rating 2.0 of 1 users (1)
Course Hadoop & MapReduce Getting Started
Rating 4.7 of 54 users Rating 4.7 of 54 users (54)
Course Getting Started with Java: The Fundamentals of Java Programming
Rating 4.5 of 526 users Rating 4.5 of 526 users (526)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Python - Introduction to NumPy for Multi-dimensional Data
Rating 4.5 of 543 users Rating 4.5 of 543 users (543)
Course CloudOps Solutioning Strategies
Rating 4.6 of 58 users Rating 4.6 of 58 users (58)
Course Be Productive by Working Efficiently
Rating 4.5 of 176 users Rating 4.5 of 176 users (176)