When will I receive my Course Certificate?

If you complete the course successfully, your electronic Course Certificate will be added to your Accomplishments page - from there, you can print your Course Certificate or add it to your LinkedIn profile.

Why can’t I audit this course?

This course is currently available only to learners who have paid or received financial aid, when available.

Is financial aid available?

Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

Databricks Associate Developer: Apache Spark with Python

Databricks Associate Developer: Apache Spark with Python

Instructor: Packt - Course Instructors

8 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

2 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

8 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

2 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

What you'll learn

Create and manipulate SQL queries in Apache Spark
Build complex Spark functions using user-defined functions (UDFs)
Develop real-time applications using Spark Streaming

Details to know

Shareable certificate

Add to your LinkedIn profile

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

There are 8 modules in this course

This course equips you with essential skills for working with Apache Spark using Python, preparing you for Databricks' certification exam. Apache Spark is a powerful open-source engine for processing large-scale data, and mastering it is a key asset in the data engineering and big data domain.

Throughout the course, learners will gain hands-on experience with Spark's core components, including data processing, streaming, and machine learning. Practical examples and exercises will build confidence and ensure you're ready for real-world challenges. What sets this course apart is its strong focus on practical skills and real-world applications of Apache Spark. You'll not only learn the theory but also apply your knowledge in hands-on projects that reinforce the concepts. This course is ideal for aspiring data engineers, analysts, or scientists who want to achieve Databricks certification. A solid understanding of Python is required, and familiarity with Pyspark is beneficial, but not mandatory.

In this section, we introduce the Spark certification exam structure, review question types, and outline a step-by-step preparation strategy to enhance exam readiness.

What's included

2 videos1 reading1 assignment

In this section, we explore Apache Spark's architecture, components, and applications, focusing on its role in big data processing, machine learning, and real-time analytics.

What's included

1 video7 readings1 assignment

1 video Total 1 minute

Understanding Apache Spark and Its Applications - Overview Video 1 minute

7 readings Total 130 minutes

Introduction 20 minutes
Spark Core 20 minutes
Spark MLlib 20 minutes
Big Data Processing 20 minutes
Real-time Streaming 20 minutes
Data Engineers 10 minutes
Data Scientists 20 minutes

1 assignment Total 10 minutes

Exploring Apache Spark Fundamentals 10 minutes

In this section, we will learn Spark's architecture, execution hierarchy, and key operations for efficient big data processing.

What's included

1 video4 readings1 assignment

In this section, we explore PySpark DataFrame operations, focusing on data manipulation, viewing, and aggregation techniques for efficient big data processing.

What's included

1 video5 readings1 assignment

1 video Total 1 minute

Spark DataFrames and Their Operations - Overview Video 1 minute

5 readings Total 150 minutes

Introduction 30 minutes
Viewing Columns of Data 30 minutes
Converting a PySpark DataFrame to a Pandas DataFrame 30 minutes
Changing the Case of a Column 30 minutes
Dropping Null Values from a DataFrame 30 minutes

1 assignment Total 10 minutes

Spark DataFrame Fundamentals 10 minutes

In this section, we explore advanced Spark operations, including groupBy, join optimizations, and AQE, to enhance performance and scalability in data processing workflows.

What's included

1 video10 readings1 assignment

1 video Total 1 minute

Advanced Operations and Optimizations in Spark - Overview Video 1 minute

10 readings Total 240 minutes

Introduction 20 minutes
Joining DataFrames in Spark 30 minutes
Use case 30 minutes
Reading and Writing Data 30 minutes
Reading and Writing Delta Files 20 minutes
Catalyst Optimizer 20 minutes
Data-based Optimizations in Apache Spark 20 minutes
Managing Data Spills in Apache Spark 20 minutes
Shuffle Joins 20 minutes
Optimizing Wide Transformations 30 minutes

1 assignment Total 10 minutes

Mastering Spark Execution and Data Management 10 minutes

In this section, we explore Spark SQL for structured data processing, covering query implementation, data analysis, and integration with external systems.

What's included

4 readings1 assignment

In this section, we explore real-time data processing with Spark, focusing on Structured Streaming, streaming architectures, and joins for dynamic data handling.

What's included

1 video4 readings1 assignment

In this section, we will learn Spark ML workflows, scalable data analysis, and model evaluation techniques for real-world applications.

What's included

10 readings1 assignment

10 readings Total 190 minutes

Introduction 20 minutes
Unsupervised Learning 10 minutes
ML with Spark 10 minutes
Spark MLlib 10 minutes
ML Life Cycle 30 minutes
Handling Categorical Variables 20 minutes
Assembling the Vector 20 minutes
Cross-validation 20 minutes
Model Monitoring and Management 20 minutes
Model Iteration and Improvement 30 minutes

1 assignment Total 10 minutes

Machine Learning with Spark ML 10 minutes

Instructor

Packt - Course Instructors

Packt

1,471 Courses 392,127 learners

Offered by

Packt

Why people choose Coursera for their career

Felipe M.

Learner since 2018

"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."

Jennifer J.

Learner since 2020

"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."

Larry W.

Learner since 2021

"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

Open new doors with Coursera Plus

Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription

Learn more

Advance your career with an online degree

Earn a degree from world-class universities - 100% online

Explore degrees

Join over 3,400 global companies that choose Coursera for Business

Upskill your employees to excel in the digital economy

Learn more

Frequently asked questions

Yes, you can preview the first video and view the syllabus before you enroll. You must purchase the course to access content not included in the preview.

If you decide to enroll in the course before the session start date, you will have access to all of the lecture videos and readings for the course. You’ll be able to submit assignments once the session starts.

Once you enroll and your session begins, you will have access to all videos and other resources, including reading items and the course discussion forum. You’ll be able to view and submit practice assessments, and complete required graded assignments to earn a grade and a Course Certificate.