Final Exam: Big Data Infrastructures

WHAT YOU WILL LEARN

• 

  Recognize the need for big data
  

  name and describe the role of the main layers of big data analytics from the bottom to the top
  

  define the role of the data processing layer and specify how information captured in the previous layer is processed
  

  describe spark and how it offers open-source scalable massively parallel in-memory solutions for analytics applications
  

  list the main characteristics of spark such as loading behavior, file formats, parallelism, cache, data skews
  

  name most important performance optimization techniques such as file format selection, level of parallelism and api selection
  

  describe the concept of big data and the history behind it
  

  identify the sources that are capable of generating big data
  

  define the big 7 characteristics that define big data
  

  describe the subcomponents of hadoop such as mapreduce and hdfs
• 

  describe the difference between horizontal and vertical scaling
  

  name and describe the features of storage systems such as hdfs, s3 and object stores, elastic search and apache solr, kudu, cockroachdb
  

  describe the rewarding role of nosql databases in horizontal distribution of large, structured and unstructured data
  

  specify when to use nosql and when to use sql database
  

  specify use cases, benefits and challenges of popular key-value data stores
  

  describe graph database use cases and specify why the relationship between data is as important as the data itself in a graph database
  

  specify the shortcoming of distributed systems and why these shortcomings make big data even more important
  

  describe what horizontal scaling is and specify how it eliminates the need for adding more memory to existing machines by using clusters (aka, sharding )
  

  name and describe the four types of big data analytics (i.e. prescriptive, predictive, diagnostic, descriptive)
  

  describe the challenges in the current data analytics models and system designs such as scalability, consistency, reliability, efficiency, and maintainability