Databricks Associate-Developer-Apache-Spark-3.5 試験概要:

Databricks Associate-Developer-Apache-Spark-3.5 試験シラバストピック:

Databricks Certified Associate Developer for Apache Spark 3.5 - Python 認定 Associate-Developer-Apache-Spark-3.5 試験問題：

問題 #1
30 of 55.
A data engineer is working on a num_df DataFrame and has a Python UDF defined as:
def cube_func(val):
return val * val * val
Which code fragment registers and uses this UDF as a Spark SQL function to work with the DataFrame num_df?

A. spark.createDataFrame(cube_func("num")).show()
B. num_df.register("cube_func").select("num").show()
C. num_df.select(cube_func("num")).show()
D. spark.udf.register("cube_func", cube_func)
num_df.selectExpr("cube_func(num)").show()

問題 #2
44 of 55.
A data engineer is working on a real-time analytics pipeline using Spark Structured Streaming.
They want the system to process incoming data in micro-batches at a fixed interval of 5 seconds.
Which code snippet fulfills this requirement?

A. query = df.writeStream \
.outputMode("append") \
.trigger(processingTime="5 seconds") \
.start()
B. query = df.writeStream \
.outputMode("append") \
.trigger(continuous="5 seconds") \
.start()
C. query = df.writeStream \
.outputMode("append") \
.start()
D. query = df.writeStream \
.outputMode("append") \
.trigger(once=True) \
.start()

問題 #3
A data scientist wants each record in the DataFrame to contain:
The first attempt at the code does read the text files but each record contains a single line. This code is shown below:

The entire contents of a file
The full file path
The issue: reading line-by-line rather than full text per file.
Code:
corpus = spark.read.text("/datasets/raw_txt/*") \
.select('*', '_metadata.file_path')
Which change will ensure one record per file?
Options:

A. Add the option wholetext=False to the text() function
B. Add the option wholetext=True to the text() function
C. Add the option lineSep=", " to the text() function
D. Add the option lineSep='\n' to the text() function

問題 #4
A data engineer wants to create a Streaming DataFrame that reads from a Kafka topic called feed.

Which code fragment should be inserted in line 5 to meet the requirement?
Code context:
spark \
.readStream \
.format("kafka") \
.option("kafka.bootstrap.servers", "host1:port1,host2:port2") \
.[LINE 5] \
.load()
Options:

A. .option("kafka.topic", "feed")
B. .option("topic", "feed")
C. .option("subscribe", "feed")
D. .option("subscribe.topic", "feed")

問題 #5
48 of 55.
A data engineer needs to join multiple DataFrames and has written the following code:
from pyspark.sql.functions import broadcast
data1 = [(1, "A"), (2, "B")]
data2 = [(1, "X"), (2, "Y")]
data3 = [(1, "M"), (2, "N")]
df1 = spark.createDataFrame(data1, ["id", "val1"])
df2 = spark.createDataFrame(data2, ["id", "val2"])
df3 = spark.createDataFrame(data3, ["id", "val3"])
df_joined = df1.join(broadcast(df2), "id", "inner") \
.join(broadcast(df3), "id", "inner")
What will be the output of this code?

A. The code will result in an error because broadcast() must be called before the joins, not inline.
B. The code will work correctly and perform two broadcast joins simultaneously to join df1 with df2, and then the result with df3.
C. The code will fail because the second join condition (df2.id == df3.id) is incorrect.
D. The code will fail because only one broadcast join can be performed at a time.

解説：