Introducción Arquitectura Apache Spark (Shuffling)

In scenario #3, what was the last action taken in the conclusion of stage #1?. The new dataset was written to disk as shuffle files. Each partition of data (bag of D&B candies) was reduced to a distinct set of colors. Each partition of data was read into each executor by their corresponding tasks. Each task executed a push operation to move their respective partitions into the next stage. None of the above.

Which key factors relate to the performance hit associated with wide transformations?. The pre-processing of each partition to conform to a data structure suitable to the corresponding wide transformation. The writing of shuffle data to disk. The reading of shuffle data from disk. The reading of shuffle data from one executor into another. The post-processing of each partition to conclude the shuffle operation. All are corrects.

For the following question, identify which operations are narrow and which are wide. wide. narrow.

Given the correct strategy, the shuffle operation for any wide transformations can be avoided. True. False.

Because of the performance cost associated with wide transformations, namely the cost of their respective shuffle operations, wide transformations should be avoided altogether. True. False.

One can assist the Catalyst Optimizer’s optimization processes by executing multiple wide transformations back-to-back. True. False.