Certified Associate Developer for Apache Spark Practice Test

Certified Associate Developer for Apache Spark Practice Test practice questions, page 6 of 14, with answer review and explanations.

Question 26

Which of the following code blocks returns a new DataFrame where column productCategories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF? A sample of storesDF is displayed below:

  1. storesDF.withColumn("productCategories", explode(col("productCategories")))
  2. storesDF.withColumn("productCategories", split(col("productCategories")))
  3. storesDF.withColumn("productCategories", col("productCategories").explode())
  4. storesDF.withColumn("productCategories", col("productCategories").split())
  5. storesDF.withColumn("productCategories", explode("productCategories"))

Question 27

Which of the following code blocks returns a new DataFrame with column storeDescription where the pattern "Description: " has been removed from the beginning of column storeDescription in DataFrame storesDF? A sample of DataFrame storesDF is below:

  1. storesDF.withColumn("storeDescription", regexp_replace(col("storeDescription"), "^Description: "))
  2. storesDF.withColumn("storeDescription", col("storeDescription").regexp_replace("^Description: ", ""))
  3. storesDF.withColumn("storeDescription", regexp_extract(col("storeDescription"), "^Description: ", ""))
  4. storesDF.withColumn("storeDescription", regexp_replace("storeDescription", "^Description: ", ""))
  5. storesDF.withColumn("storeDescription", regexp_replace(col("storeDescription"), "^Description: ", ""))

Question 28

Which of the following code blocks returns a new DataFrame where column division from DataFrame storesDF has been replaced and renamed to column state and column managerName from DataFrame storesDF has been replaced and renamed to column managerFullName?

  1. (storesDF.withColumnRenamed(["division", "state"], ["managerName", "managerFullName"])
  2. (storesDF.withColumn("state", col("division")) .withColumn("managerFullName", col("managerName")))
  3. (storesDF.withColumn("state", "division") .withColumn("managerFullName", "managerName"))
  4. (storesDF.withColumnRenamed("state", "division") .withColumnRenamed("managerFullName", "managerName"))
  5. (storesDF.withColumnRenamed("division", "state") .withColumnRenamed("managerName", "managerFullName"))

Question 29

The code block shown contains an error. The code block is intended to return a new DataFrame where column sqft from DataFrame storesDF has had its missing values replaced with the value 30,000. Identify the error. A sample of DataFrame storesDF is displayed below: Code block: storesDF.na.fill(30000, col("sqft"))

  1. The argument to the subset parameter of fill() should be a string column name or a list of string column names rather than a Column object.
  2. The na.fill() operation does not work and should be replaced by the dropna() operation.
  3. he argument to the subset parameter of fill() should be a the numerical position of the column rather than a Column object.
  4. The na.fill() operation does not work and should be replaced by the nafill() operation.
  5. The na.fill() operation does not work and should be replaced by the fillna() operation.

Question 30

Which of the following operations fails to return a DataFrame with no duplicate rows?

  1. DataFrame.dropDuplicates()
  2. DataFrame.distinct()
  3. DataFrame.drop_duplicates()
  4. DataFrame.drop_duplicates(subset = None)
  5. DataFrame.drop_duplicates(subset = "all")