daemon.py is a Python module in pyspark package.
from pyspark import daemon
Top-Level Code Environment
If the module is executed in the top-level code environment (e.g.,
python -m), its
__name__ is set to the string
Sometimes "top-level code" is called an entry point to the application.
Learn more in the __main__ — Top-level code environment.
When executed in the top-level code environment,
daemon.py calls manager function.
manager runs until it is stopped (e.g.,
manager creates a new process group (
manager creates a listening socket on the loopback interface (possibly using IPv6 based on
SPARK_PREFER_IPV6 environment variable).
SPARK_REUSE_WORKER environment variable (
manager launches a worker process (in a child process using
Launching Worker Process¶
worker( sock: socket, authenticated: Bool) -> Optional[int]
worker is called by a worker process after the
worker runs a worker.