Databricks Associate-Developer-Apache-Spark-3.5 Exam Syllabus Topics:

Databricks Certified Associate Developer for Apache Spark 3.5 - Python Sample Questions:

Question #1
A data engineer noticed improved performance after upgrading from Spark 3.0 to Spark 3.5. The engineer found that Adaptive Query Execution (AQE) was enabled.
Which operation is AQE implementing to improve performance?

A. Improving the performance of single-stage Spark jobs
B. Optimizing the layout of Delta files on disk
C. Collecting persistent table statistics and storing them in the metastore for future use
D. Dynamically switching join strategies

Question #2
A data engineer is working on a Streaming DataFrame streaming_df with the given streaming data:

Which operation is supported with streamingdf ?

A. streaming_df. select (countDistinct ("Name") )
B. streaming_df.filter (col("count") < 30).show()
C. streaming_df.groupby("Id") .count ()
D. streaming_df.orderBy("timestamp").limit(4)

Question #3
A Data Analyst is working on the DataFrame sensor_df, which contains two columns:
Which code fragment returns a DataFrame that splits the record column into separate columns and has one array item per row?
A)

B)

C)

D)

A. exploded_df = exploded_df.select(
"record_datetime",
"record_exploded.sensor_id",
"record_exploded.status",
"record_exploded.health"
)
exploded_df = sensor_df.withColumn("record_exploded", explode("record"))
B. exploded_df = exploded_df.select(
"record_datetime",
"record_exploded.sensor_id",
"record_exploded.status",
"record_exploded.health"
)
exploded_df = sensor_df.withColumn("record_exploded", explode("record"))
C. exploded_df = sensor_df.withColumn("record_exploded", explode("record")) exploded_df = exploded_df.select("record_datetime", "sensor_id", "status", "health")
D. exploded_df = exploded_df.select("record_datetime", "record_exploded")

Question #4
An engineer has two DataFrames: df1 (small) and df2 (large). A broadcast join is used:
python
CopyEdit
from pyspark.sql.functions import broadcast
result = df2.join(broadcast(df1), on='id', how='inner')
What is the purpose of using broadcast() in this scenario?
Options:

A. It filters the id values before performing the join.
B. It increases the partition size for df1 and df2.
C. It reduces the number of shuffle operations by replicating the smaller DataFrame to all nodes.
D. It ensures that the join happens only when the id values are identical.

Question #5
A developer wants to test Spark Connect with an existing Spark application.
What are the two alternative ways the developer can start a local Spark Connect server without changing their existing application code? (Choose 2 answers)

A. Execute their pyspark shell with the option --remote "sc://localhost"
B. Execute their pyspark shell with the option --remote "https://localhost"
C. Set the environment variable SPARK_REMOTE="sc://localhost" before starting the pyspark shell
D. Ensure the Spark property spark.connect.grpc.binding.port is set to 15002 in the application code
E. Add .remote("sc://localhost") to their SparkSession.builder calls in their Spark code

Solutions: