The Internals of Spark Core (Apache Spark 3.5.2)¶

Welcome to The Internals of Spark Core online book! 🤙

I'm Jacek Laskowski, a Freelance Data(bricks) Engineer specializing in Apache Spark (incl. Spark SQL and Spark Structured Streaming), Delta Lake, Databricks, and Apache Kafka (incl. Kafka Streams) with brief forays into a wider data engineering space (e.g., Trino, Dask and dbt, mostly during Warsaw Data Engineering meetups).

I'm very excited to have you here and hope you will enjoy exploring the internals of Spark Core as much as I have.

Expect text and code snippets from a variety of public sources. Attribution follows.

Now, let's take a deep dive into Spark Core 🔥

Last update: 2024-09-14