Skip to content

The Internals of Spark Structured Streaming (Apache Spark 3.3.1)

Welcome to The Internals of Spark Structured Streaming online book! 🤙

I'm Jacek Laskowski, an IT freelancer specializing in Apache Spark (incl. Spark SQL and Spark Structured Streaming), Delta Lake and Apache Kafka (incl. Kafka Streams and ksqlDB) (with brief forays into a wider data engineering space, e.g. Trino, Dask and dbt, mostly during Warsaw Data Engineering meetups).

I'm very excited to have you here and hope you will enjoy exploring the internals of Spark Structured Streaming as much as I have.

Flannery O'Connor

I write to discover what I know.

"The Internals Of" series

I'm also writing other online books in the "The Internals Of" series. Please visit "The Internals Of" Online Books home page.

Expect text and code snippets from a variety of public sources. Attribution follows.

Now, let's take a deep dive into Spark Structured Streaming 🔥

Last update: 2022-12-28