Deep-Dive In Deltalake Using Pyspark In Databricks

Free Download Deep-Dive In Deltalake Using Pyspark In Databricks
Published 9/2023
MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHz
Language: English | Size: 1.88 GB | Duration: 3h 53m
Unlock the Power of Delta Lake: Master Databricks and Revolutionize Data Management in this Comprehensive Course

What you’ll learn
Understand the power of Delta table in Apache Spark
Build Delta Lakehouse using Databricks
Explore advanced features of Delta Lake, such as schema evolution and time travel
Learn to use Databricks effectively for data processing and analysis
Understand End to End use of Delta tables in Apache Spark
Hands on practice with Delta Lake
Learn how to leverage the power of Delta Lake with a Spark Environment!
Requirements
Basic Databricks and PySpark Knowlege needed
Must have experienced in SQL and Python
Willing to learn new skills
Description
This is an immersive course that provides a comprehensive understanding of Delta Lake, a powerful open-source storage layer for big data processing, and how to leverage it using Databricks. With hands-on experience and a step-by-step approach, this course explores the core concepts, architecture, and best practices of Delta Lake. Throughout the course, you will gain valuable insights into data lakes, data ingestion, data management, and data quality. You will learn the advanced capabilities of Delta Lake, including schema evolution, transactional writes, asset management, and time travel. Moreover, this course covers how to integrate Delta Lake with Databricks, a cloud-based platform for data engineering and analytics. You will witness the seamless integration of Delta Lake with Databricks, empowering you to perform analytics, data engineering, and machine learning projects efficiently using these technologies. To enhance your learning experience, this course also includes an end-to-end project where you will apply the acquired knowledge to build a real-world data solution. You will design a data pipeline, perform data ingestion, transform data using Delta Lake, conduct analytics, and visualize the results. This hands-on project will solidify your understanding and provide you with practical skills applicable to various data-driven projects. By the end of this course, you will be equipped with the expertise to leverage the power of Delta Lake using Databricks and successfully implement scalable and reliable data solutions. Whether you are a data engineer, data scientist, or data analyst, this course offers immense value in advancing your big data skills and accelerating your career in the field of data engineering and analytics.
Overview
Section 1: Introduction
Lecture 1 Introduction and Syllabus
Lecture 2 Architecture and Introduction of Delta-Lake
Lecture 3 How Delta table differs from Normal tables
Lecture 4 Create a Delta table
Lecture 5 Generate Column in Delta table
Lecture 6 Read a Delta table
Lecture 7 Write to a Delta table
Lecture 8 ReplaceWhere while writing to Delta table
Lecture 9 Delete a Delta table
Lecture 10 Update a Delta table
Lecture 11 Upsert or Merge Statment in Delta tables
Lecture 12 Understand transactional logs in _delta_log folder
Lecture 13 How to go to time travel in Delta table using History
Lecture 14 Restore Delta table to previous version
Lecture 15 How to add constraint in Delta table
Lecture 16 How to add user meta data information in a Delta table
Lecture 17 Schema evolution and enforcement in Delta table
Lecture 18 Shallow and Deep Clone of Delta table
Lecture 19 Addition on Deep and Shallow Clone of Delta table
Lecture 20 How to enable Change Data Feed in Delta table
Lecture 21 Reduce small file issue using optimize
Data Engineer who wants to switch to Azure Big Data Engineer,Beginner Apache Spark Developer,Data Analyst
Homepage

www.udemy.com/course/deep-dive-in-deltalake-using-pyspark-in-databricks/