Apache Spark Architecture and Components1 / 5
A Spark executor has 4 cores and 16 GB of memory. How many tasks can run concurrently on this executor?
CorrectIncorrect
Alex
Concurrent tasks per executor = spark.executor.cores / spark.task.cpus. With the default spark.task.cpus=1, each core runs one task, so 4 cores = 4 concurrent tasks. Increasing spark.task.cpus reduces concurrency but gives each task more CPU—useful for multi-threaded ML algorithms or GPU workloads. Memory is shared: each task's working memory ≈ (executor memory × memory fraction) / concurrent tasks. Distractor analysis: The option “The number of concurrent tasks is capped…” is wrong—spark.default.parallelism controls total partitions, not per-executor concurrency. The option “Concurrency is limited by…” is wrong—spark.scheduler.maxRegisteredResourcesWaitingTime is unrelated to task slot count. The option “Up to 4 tasks can run concurrently…” is wrong—memoryFraction affects spill thresholds but doesn't cap task concurrency. Ref: spark.apache.org/docs/latest/configuration.html#scheduling
Sourcedatabricks.com
Follow-up answers are available in the app. Create a free account — no credit card required.
Question 1 of 5
Create a free account