Udacity – Data Streaming Nanodegree 2021-8

Published on: 2023-01-19 00:27:05

Categories: 28

Descriptions

The ultimate goal of the Data Streaming Nanodegree program is to provide students with the latest skills to process data in real-time by building fluency in modern data engineering tools, such as Apache Spark,Kafka, Spark Streaming, and Kafka Streaming. A graduate of this program will be able to: Understand the components of data streaming systems. Ingest data in real-time using Apache Kafka and Spark and run analysis. Use the Faust Stream Processing Python library to build a real-time stream-based application. Compile real-time data and run live analytics, as well as draw insights from reports generated by the streaming console. Learn about the Kafka ecosystem, and the types of problems each solution is designed to solve. Use the Confluent Kafka Python library for simple topic management, production, and consumption. Explain the components of Spark Streaming (architecture and API), integrate Apache Spark Structured Streaming and Apache Kafka, manipulate data using Spark, and read DataFrames in the Spark Streaming Console.

This program is comprised of 2 courses and 2 projects. Each project you build will be an opportunity to demonstrate what you’ve learned in the course, and will demonstrate to potential employers that you have skills in these areas. The goal of this course is to demonstrate knowledge of the tools taught throughout, including Kafka Consumers, Producers, & Topics; Kafka Connect Sources and Sinks, Kafka REST Proxy for producing data over REST, Data Schemas with JSON and Apache Avro/Schema Registry, Stream Processing with the Faust Python Library, and Stream Processing with KSQL.

What you’ll learn

Describe and explain streaming data stores and
stream processing
Describe and explain real-world usages of stream processing
Describe and explain Kafka architecture
Describe and explain Kafka topics and configuration
Utilize Confluent Kafka Python to create topics and configuration
Describe and explain Kafka producers, consumers, and configuration
Describe and explain common scenarios for stream
processing, and where you would use stream versus batch
Describe and explain common stream processing strategies
Describe and explain how time and windowing works in stream processing

Who this course is for

This program is intended for software engineers looking to build real-time data processing proficiency, as well as data engineers looking to enhance their existing skill set with the next advancement in data engineering.

Specificatoin of Data Streaming Nanodegree

Publisher : Udacity
Teacher : Ben Goldberg , David Drummond , Judit Lantos , Sean Murdock
Language : English
Level : Advance
Number of Course : 19
Duration : Approximately 2 months to complete Suggested pace of 2-5 hours/week

Content of Data Streaming Nanodegree

Welcome to the Data Streaming Nanodegree Program
Lesson 01: Data Streaming Nanodegree Program Introduction
Lesson 02: Introduction to Data Streaming
Lesson 03: Nanodegree Career Services

Data Ingestion with Kafka & Kafka Streaming
Lesson 01: Introduction to Stream Processing
Lesson 02: Apache Kafka
Lesson 03: Data Schemas and Apache Avro
Lesson 04: Kafka Connect and REST Proxy
Lesson 05: Stream Processing Fundamentals
Lesson 06: Stream Processing with Faust
Lesson 07: KSQL
Lesson 08: Optimizing Public Transportation
Lesson 09: Optimize Your GitHub Profile

Apache Spark and Spark Streaming
Lesson 01: The Power of Spark
Lesson 02: Data Wrangling with Spark
Lesson 03: Intro to Spark Streaming
Lesson 04: Structured Streaming APIs
Lesson 05: Integration of Spark Streaming and Kafka
Lesson 06: SF Crime Statistics with Spark Streaming
Lesson 07: Take 30 Min to Improve your LinkedIn

Requirements

Intermediate SQL, and Python. And experience with ETL.
Basic familiarity with traditional batch processing and traditional service architectures is desired, but not required.