Prepared with AI assistance by Pass-IT. Vendor sources support exam facts; the exercises and study recommendations are ours.
The Databricks Certified Associate Developer for Apache Spark certification covers Spark architecture and practical DataFrame tasks using Python. The current assessment includes 45 scored multiple-choice questions in 90 minutes; additional unscored questions may appear. Check the official certification page for the guide and exam details, verified on September 9, 2026.
For DataFrame practice, start by writing the output you expect from a small input. This makes you decide what a row represents before choosing an API. A transformation can execute successfully while answering the wrong business question.
Work out the result first
This original exercise uses a static batch of events. Each event has an identifier and a user. The source owner confirms that the second appearance of e1 is a replay of the same event with identical data.
| event_id | user_id |
|---|---|
| e1 | u1 |
| e1 | u1 |
| e2 | u1 |
| e3 | u2 |
Your task is to count distinct business events for each user. Before writing code, check that both columns have the expected types and decide how missing identifiers should be handled. A missing identifier cannot safely establish whether two records represent the same event.
The proposed answer removes the replay using events.dropDuplicates(["event_id"]), then groups the retained rows by user_id and counts them. Spark’s dropDuplicates documentation explains that the method compares either the specified subset or all columns.
Three events remain: e1, e2, and e3. The expected output is:
| user_id | event_count |
|---|---|
| u1 | 2 |
| u2 | 1 |
Deduplicating on user_id would discard a legitimate event because u1 produced both e1 and e2. Comparing all columns works for this particular input because the replay is identical, but an added delivery timestamp could make that choice retain both deliveries. Choose the key from the event’s meaning.
Change the requirement
Now suppose e1 appears with a different user_id. Treat that as a conflict to investigate. Deduplication alone does not establish which row is newest or authoritative. You need a defined conflict policy and, if recency matters, a trustworthy ordering field.
Verify the retained row count of three and the user counts of two and one for the original input. This batch exercise does not define a streaming solution: streaming also needs decisions about state and late arrivals. Review SQL, architecture, and the remaining guide topics separately.
Continue with original Pass-IT practice questions and explanations to examine why plausible alternatives lose legitimate events.