Databricks Data Engineer Associate

Key Topics to Master:

• Databricks Data Intelligence Platform architecture and the separation of control plane from compute
• Delta Lake ACID transactions and time travel as the basis for rollback and audit trails
• Unity Catalog as the governance layer over data and AI assets
• All-purpose clusters against job clusters against SQL warehouses
• Cost models for each compute service and when serverless is the better fit
• Cluster startup behaviour and concurrency limits
• Choosing compute for ad hoc analyst SQL against scheduled ETL

Key Topics to Master:

• COPY INTO for incremental loading from ADLS, S3, or GCS into Unity Catalog tables
• Auto Loader with schema enforcement and schema evolution
• Auto Loader file discovery modes, including directory listing and file notification
• Lakeflow Connect standard connectors against managed connectors
• JDBC, ODBC, and REST clients in notebooks, orchestrated by Lakeflow Jobs
• Ingesting semi-structured JSON and nested data into Delta tables
• Choosing between Auto Loader, Lakeflow Connect, and partner connectors for a stated requirement
• Databricks audit log delivery format and latency behaviour

Key Topics to Master:

• Reading bronze tables with PySpark or SQL, handling nulls, standardizing types, writing silver tables
• Inner, left, broadcast, multi-key, and cross joins, plus union and union all
• Adding, dropping, splitting, and renaming columns, filtering rows, exploding arrays
• Deduplication and aggregates including count, approximate count distinct, mean, and summary
• spark.sql.shuffle.partitions and spark.default.parallelism
• spark.executor.memory, spark.driver.memory, and spark.sql.autoBroadcastJoinThreshold
• Building gold layer materialized views, views, streaming tables, and tables in Unity Catalog
• Data quality checks and validation rules on silver and gold datasets

Key Topics to Master:

• Retries and conditional tasks including branching and looping
• Notebook, SQL query, dashboard, and pipeline task types
• Task dependencies expressed through the DAG-based task graph
• Scheduled triggers, file arrival triggers, and table update triggers
• Choosing time-based triggers against data-driven triggers
• Job schedules and how they interact with upstream pipeline availability

Key Topics to Master:

• Databricks Git Folders, formerly Databricks Repos, including creating and switching branches
• Committing, pushing, and opening pull requests through Databricks Git integration
• Declarative Automation Bundles for packaging jobs, pipelines, and workspace assets
• Bundle variables and target overrides for environment-specific configuration
• Promoting one codebase across dev, test, and prod targets
• Databricks CLI commands to validate, deploy, and manage bundles
• Where bundle deployment fits inside an automated CI/CD workflow

Key Topics to Master:

• Comparing current job execution time against historical baselines in run history
• Interpreting job statuses and DAG task graphs to find upstream blockers
• Tracking pipeline run times and failure rates
• Recognising data skew, shuffling, and disk spilling in stage-level Spark UI metrics
• Adaptive query execution with skew join handling
• Liquid Clustering and predictive optimization
• Diagnosing cluster startup failures, library conflicts, and out-of-memory errors

Key Topics to Master:

• Managed against external tables in Unity Catalog, and converting between them
• Creating, modifying, and deleting tables under Unity Catalog governance
• GRANT, REVOKE, and DENY applied through the UI and through SQL
• Principals: users, groups, and service principals
• Levels of the Unity Catalog security hierarchy and where a privilege should sit
• Column-level masking to hide sensitive fields from specific groups
• Row-level security to restrict visible rows by user group
• Unity Catalog ABAC policies for centrally controlled filtering and masking

• 1Download the Databricks Certified Data Engineer Associate exam guide dated May 4, 2026 and read all seven section outlines before touching any course.
• 2Write a two-column mapping of old to new product names: Repos to Git Folders, Asset Bundles to Declarative Automation Bundles, Delta Live Tables to Lakeflow Spark Declarative Pipelines.
• 3Sign up for Databricks Free Edition and create a workspace so you have somewhere to run every exercise in the following weeks.
• 4Work through the Databricks Academy self-paced course Get Started with Data Governance on Databricks.
• 5Create one all-purpose cluster, one job cluster, and one SQL warehouse, then note the startup time and cost model of each.

• 1Create a managed table and an external table in Unity Catalog, convert between them, and record what changes about the underlying files.
• 2Run DESCRIBE HISTORY on a Delta table, then use time travel to read a version from before an intentional bad write.
• 3Practise GRANT, REVOKE, and DENY at catalog, schema, and table level for a user, a group, and a service principal.
• 4Apply a column mask and a row filter to one table, then query it as a member of a restricted group.
• 5Read the Databricks documentation page on Unity Catalog ABAC policies and note how it differs from per-object grants.

• 1Take the Databricks Academy self-paced course Data Ingestion with Lakeflow Connect.
• 2Load the same file set twice with COPY INTO and confirm it does not duplicate rows on the second run.
• 3Set up Auto Loader in directory listing mode, then in file notification mode, and write down when each is preferable.
• 4Trigger a schema change mid-stream and observe Auto Loader schema evolution and the checkpoint behaviour.
• 5Ingest a nested JSON file and flatten it into a Delta table using explode.

• 1Compare Lakeflow Connect standard connectors against managed connectors in the Databricks documentation and list what each handles for you.
• 2Build one JDBC ingestion notebook and schedule it with Lakeflow Jobs so ingestion and orchestration are linked in your head.
• 3For five stated requirements varying by volume, frequency, and data type, choose between Auto Loader, Lakeflow Connect, and a partner connector and justify each choice in one sentence.
• 4Read the Databricks documentation on audit log delivery and note the file format, the latency, and whether files can be overwritten.

• 1Build a bronze table from raw files, then a silver table that removes nulls, casts types, and deduplicates.
• 2Write the same transformation twice, once in SQL and once in PySpark, since the exam presents code in both.
• 3Practise inner, left, broadcast, multi-key, and cross joins on tables you built, and confirm the row counts you predicted.
• 4Use approx_count_distinct and summary on a wide table and compare the result and runtime against exact aggregates.
• 5Add data quality validation rules to the silver write and test them with deliberately bad input.

• 1Create a materialized view, a view, a streaming table, and a standard table over the same silver data, then list the refresh and cost differences.
• 2Set spark.sql.shuffle.partitions to three different values on one shuffle-heavy job and record the runtime each time.
• 3Change spark.sql.autoBroadcastJoinThreshold and confirm through the Spark UI whether the plan switched to a broadcast join.
• 4Read the Databricks documentation on spark.executor.memory and spark.driver.memory and note which failures point to each.
• 5Take the Databricks Academy self-paced course Build Data Pipelines with Lakeflow Spark Declarative Pipelines.

• 1Take the Databricks Academy self-paced course Deploy Workloads with Lakeflow Jobs.
• 2Build a job with four tasks in a DAG, including a notebook task, a SQL query task, and a pipeline task.
• 3Add a conditional branch and a retry policy, then make one task fail on purpose and watch the run history.
• 4Configure a file arrival trigger and a table update trigger, and write one sentence on when each beats a cron schedule.
• 5Use the run history view to compare execution times across five runs and identify the slowest task.

• 1Take the Databricks Academy self-paced course DevOps Essentials for Data Engineering.
• 2Connect a Git provider, create a Databricks Git Folder, branch, commit, push, and open a pull request from the workspace UI.
• 3Install the Databricks CLI and run bundle validate and bundle deploy against a dev target.
• 4Add bundle variables and a prod target override, deploy the same codebase to two targets, and diff the resulting jobs.
• 5Package a Lakeflow Job and a pipeline into a single Declarative Automation Bundle.

• 1Create a deliberately skewed join by concentrating keys, then find the skew in Spark UI stage metrics by comparing median and maximum shuffle read.
• 2Fix that skew twice, once by enabling adaptive query execution skew join handling and once by salting the key, and compare the runtimes.
• 3Force a disk spill by shrinking executor memory, then identify the spill in the Spark UI.
• 4Enable Liquid Clustering on one table and read the Databricks documentation on predictive optimization.
• 5Reproduce a library conflict on a cluster and read the driver log to find the failure line.

• 1Re-read the exam guide and score yourself out of five on every individual objective, not just each section.
• 2Spend revision time in proportion to the published weights: Transformation and Modeling at 22 percent and Ingestion at 21 percent together are 43 percent of the exam.
• 3Work the five retired sample questions printed in the exam guide and check that you can explain why each distractor is wrong.
• 4Rebuild one end-to-end pipeline from raw file to gold table, orchestrated by a job and deployed by a bundle, without looking anything up.

• 1Sit at least two full 45-question practice sets under a strict 90-minute clock, which gives you two minutes per question.
• 2Practise with no documentation open at all, because the exam guide states that no test aides are allowed.
• 3Register in Kryterion Webassessor and run the Kryterion system check on the exact machine and webcam you will use.
• 4Clear your desk and do one practice set in the room and lighting you will actually test in.
• 5Review every question you missed and trace it back to a specific exam guide objective.

• 1Rehearse the three lowest-scoring objectives from your week 10 self-scoring and nothing else.
• 2Re-read the Governance and Security objectives, which are 15 percent of the exam and the easiest marks to lose cheaply.
• 3Confirm your government-issued photo ID matches the name on your Webassessor registration exactly.
• 4Take the exam early in the day and keep the following two weeks free in case the 14-day retake wait applies.