Certified Associate Developer for Apache Spark Practice Test

Certified Associate Developer for Apache Spark Practice Test practice questions, page 11 of 14, with answer review and explanations.

Question 51

The code block shown below contains an error. The code block intended to return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId. Identify the error. Code block: StoresDF.join(employeesDF, "inner", "storeID")

  1. The key column storeID needs to be wrapped in the col() operation.
  2. The key column storeID needs to be in a list like ["storeID"].
  3. The key column storeID needs to be specified in an expression of both DataFrame columns like storesDF.storeId == employeesDF.storeId.
  4. There is no DataFrame.join() operation – DataFrame.merge() should be used instead.
  5. The column key is the second parameter to join() and the type of join in the third parameter to join() – the second and third arguments should be switched.

Question 52

Which of the following operations can perform an outer join on two DataFrames?

  1. DataFrame.crossJoin()
  2. Standalone join() function
  3. DataFrame.outerJoin()
  4. DataFrame.join()
  5. DataFrame.merge()

Question 53

Which of the following pairs of arguments cannot be used in DataFrame.join() to perform an inner join on two DataFrames, named and aliased with "a" and "b" respectively, to specify two key columns?

  1. on = [a.column1 == b.column1, a.column2 == b.column2]
  2. on = [col("column1"), col("column2")]
  3. on = [col("a.column1") == col("b.column1"), col("a.column2") == col("b.column2")]
  4. All of these options can be used to perform an inner join with two key columns.
  5. on = ["column1", "column2"]

Question 54

The below code block contains a logical error resulting in inefficiency. The code block is intended to efficiently perform a broadcast join of DataFrame storesDF and the much larger DataFrame employeesDF using key column storeId. Identify the logical error. Code block: storesDF.join(broadcast(employeesDF), "storeId")

  1. The larger DataFrame employeesDF is being broadcasted rather than the smaller DataFrame storesDF.
  2. There is never a need to call the broadcast() operation in Apache Spark 3.
  3. The entire line of code should be wrapped in broadcast() rather than just DataFrame employeesDF.
  4. The broadcast() operation will only perform a broadcast join if the Spark property spark.sql.autoBroadcastJoinThreshold is manually set.
  5. Only one of the DataFrames is being broadcasted rather than both of the DataFrames.

Question 55

The code block shown below contains an error. The code block is intended to return a new DataFrame that is the result of a cross join between DataFrame storesDF and DataFrame employeesDF. Identify the error. Code block: storesDF.join(employeesDF, "cross")

  1. A cross join is not implemented by the DataFrame.join() operations – the standalone CrossJoin() operation should be used instead.
  2. There is no direct cross join in Spark, but it can be implemented by performing an outer join on all columns of both DataFrames.
  3. A cross join is not implemented by the DataFrame.join()operation – the DataFrame.crossJoin()operation should be used instead.
  4. There is no key column specified – the key column "storeId" should be the second argument.
  5. A cross join is not implemented by the DataFrame.join() operations – the standalone join() operation should be used instead.