Big Data Hadoop Training

Topics: 

• Introduction to MapReduce 
• Learning the working procedure of MapReduce 
• Understanding Map and reduce concepts 
• Stages in MapReduce 
• The terminology used in MR such as Shuffle, Sort, Combiners, Partitions, Output Format, Input Format and Output Format. 

Learning Outcome: Upon the completion of this chapter you learn the procedure to write a word count program, knowledge of MapReduce Combiner, writing a custom practitioner, deploying unit tests, how to use a local job runner, what is a tool runner, data set joining etc. 

Topics: 

• Overview of Hadoop Hive 
• Understanding the architecture of Hadoop 
• Comparison between Hive, RDBMS, and Pig 
• Creation of database
• working with Hive Query Language
• Different Hive Tables 
• Group by and other clauses,
• Storing the Hive Results,
• HCatalog, and  Hive tables, 
• Hive partitioning, and Buckets

Learning outcome: By the completion of this module you will learn the process to create a database in Hive, Hive table creation, Database dropping and customization to a table, Writing Hive queries to pull data, Hive Table Partitioning and Group by clause. 

Topics: 

• The index in Hive 
• Hive Map side join 
• User-defined functions in Hive 
• Working with complex data types
• overview of Impala
• Difference between Impala and Hive
• Architecture of Impala 

Learning Outcome: This chapter will give you complete knowledge of Hive queries, joining table, sequence table deployment, writing indexes, data storage in a different table.  

Topics: 

• Introduction to Apache Pig 
• Pig features 
• Schema and various data types in Hive
• Tuples and Fields 
• Available functions in Pig, and Hive Bags 

Learning outcome: By the completion of this chapter you will gain knowledge to work with Pig, loading of data, storing the data into files, restricting data to 4 rows, working with Filter By, Group By, Split, Distinct, Cross in Hive. 

Topics: 

• Introduction to Apache Sqoop 
• Importing and exporting data 
• Sqoop Limitations 
• Performance improvement with Sqoop 
• Flume overview 
• Flume Architecture 
• What is CAP theorem and Hbase 

Learning Outcome: Upon the completion of this module you will be able to generate sequence numbers, Consume twitter data using Sqoop, Hive table creation with AVRO, Table creation in HBase, AVRO with Pig, Scan and enable table, Deploying disable.   

Topics: 

• Introduction to Spark 
• Procedure to write Spark applications with Scala 
• Overview of object-oriented programming 
• A detailed study of Scala
• Scala Uses
• Executing Scala code 
• Multiple classes of Scala such as Getters, Extending Objects, Abstract, Constructors,
• Setters,  Overriding Methods.  
• Scala and Java interoperability
• Bobsrockets package
• Anonymous functions, and functional programming
• comparison between Mutable and immutable collections
• control Structures in Scala
• Scala REPL, Lazy Values
• Directed Acyclic Graph (DAG),
• Spark in Hadoop ecosystem and Spark UI 
• Developing Spark application using SBT/Eclipse

Learning Outcome: Upon the completion of this module you will gain knowledge to write Spark applications using Scala, Scala ability for Spark real-time analytics operation.  

Topics: 

• Introduction to Apache Spark 
• Features of Spark 
• Spark components Comparison
• between Spark and Hadoop 
• Introduction t Scala and RDD
• Integrating HDFS with Spark

Learning Outcome: Upon the completion of this chapter, you will learn the importance of RDD in Spark and how it makes big data processes faster. 

Topics: 

• Introduction to Spark SQL 
• Importance of SQL in Spark
• Spark SQL JSON support 
• Structured data processing 
• Working with parquet files and XML data 
• Procedure to read JDBC file 
• Writing Data frame to HIve 
• Hive context creation 
• Role of Spark Dataframe
• Overview of schema manual inferring,  
• JDBC table reading
• working with CSV files 
• Data transformation from DataFrame to JDBC
• Shared accumulators, and variables. 
• User-defined functions in Spark SQL 
• Query and Transform data in data frames   
• Configuration of Hive on Spark as an execution engine 
• Dataframe benefits 

Learning Outcome: After finishing this chapter you will gain knowledge to use data frames to query and transform data and get an overview of advantages that arise out of using data frames.  

Topics: 

• Overview of Spark MLlib
• Introduction to different algorithms 
• Graph processing analysis in Spark 
• Understanding Spark interactive algorithm 
• ML algorithms supported by MLlib,
• Introduction to Machine learning 
• Introduction to accumulators,
• Overview of Decision Tree, Logistic Regression,
• Linear Regression. Building a Recommendation Engine 
• K-means clustering techniques 

Learning Outcome: Upon the completion of this module you will gain hands-on experience in building a recommendation engine. 

Topics: 

• Introduction to Kafka 
• Use of Kafka 
• Kafka workflow,
• Kafka architecture
• Basic operations,
• Configuring Kafka cluster
• Integration of  Apache
• Kafka and Apache Flume Producing and consuming messages  
• Kafka monitoring tools. 

Learning Outcome: Upon the completion of this module, you will gain hands-on exposure in the configuration of  Single Node Multi Broker Cluster, Single Node Single Broker Cluster, and integration of Apache Flume and Kafka.   

Topics: 

• Introduction to Spark Streaming 
• Working with Spark streaming 
• Spark Streaming Architecture 
• Data processing using Spark streaming 
• Requesting count and DStream
• Features of Spark Streaming
• Working with advanced data sources
• Sliding window and multi-batch operations Spark Streaming features Discretized Streams (DStreams),
• Spark Streaming workflow
• Output Operations on DStreams,
• important Windowed Operators
• Windowed Operators and their use
• Stateful Operators.important

Windowed Operators Learning Outcome: After finishing this module you will learn to execute Twitter sentiment analysis, Kafka-Spark Streaming, streaming using Netcat server, and Spark-Flume Streaming. 

Topics: 

• Setting up 4 node cluster 
• Running MapReduce code 
• Running MapReduce jobs 
• Working with cloud manager setup 

Learning Outcome: By the completion of this chapter you will gain hands-on expertise in building a multi-node Hadoop cluster and working knowledge of cloud managers.

Topics: 

• Introduction to Hadoop configuration 
• Various parameters to be followed in the configuration process 
• Importance of Hadoop configuration file 
• Hadoop environment setup 
• MapReduce parameters
• HDFS parameters The process to include and exclude 
• Data node directory structures
• Overview of the File system image Understanding Edit log

Learning Outcome: In this chapter, you will gain hands-on exposure in executing performance tuning in MapReduce. 

Topics: 

• Basics of checkpoint procedure
• Failure of Name node 
• Procedure to recover failed node 
• Metadata and Data backup,
• Safe Mode, Different problems and solutions Adding and removing nodes 

Learning Outcome: Upon the completion of this chapter, you will learn the process to recover the MapReduce File system, Hadoop cluster monitoring, Usage of job scheduler to schedule jobs, Fair Scheduler and process to its configuration, FIFO schedule and MapReduce job submission flow. 

At HKR, we provide practice tests at the end of the course along with Big Data Hadoop Interview Questions, community questions and answers, Sample Resumes, etc to crack the interview.

We at HKR trainings provide the learners with practice Mock Interview Sessions and excellent Job Support at the end of the course.