| Outcomes
By the end of the course, learners will be able to:
Course Contributors:
|
Level: | 01 |
| Duration: | 22 Hours | |
| Pre-requisites: | Python (Level 1), SQL (Level 1) – For Spark SQL | |
| What’s next: | Data Management |
Course
PySpark: Level 01
One of the most valuable technology skills is the ability to analyze huge data sets, and one of the best technology for this task is Apache Spark. Top technology companies like Google, Facebook, Netflix, Airbnb, Amazon, NASA, and more are all using Spark to solve their big data problems. Spark is a tool for doing parallel computation with large datasets and it integrates well with Python. PySpark is the Python package that can make this happen. Apache Spark is a lightning-fast cluster computing designed for fast computation. It was built on top of Hadoop MapReduce and it extends the MapReduce model to efficiently use more types of computations which includes Interactive Queries and Stream Processing. This is a brief tutorial that explains the basics of Spark Core programming.This course has been designed to enhance your Spark skills. At the end of this course, you will be able to understand Spark DataFrames, DataFrame Operations and important concepts such as Partitioning of data.