Partitions in Apache Spark

Partitions in Apache Spark

One of the most important things to learn about Spark is that it's not magic. The framework still adheres to the rules of computer science. What I mean by this is that you can still do plenty of unoptimized workflows and see poor performance. Understanding how Spark works under the hood, from even a cursory level, can help in writing better Spark applications. 

Read More