Data Engineer with Databricks Expertise

Job Summary

We are seeking a highly skilled Data Engineer with Databricks certifications and

extensive practical experience working with the Databricks Lakehouse platform and its

supporting capabilities. The ideal candidate will possess a robust understanding of AWS

services relevant to data engineering and demonstrate proficiency in data pipeline

development, programming, data modeling, and governance.

Key Responsibilities

• Design and implement data pipelines using AWS and Databricks.

• Manage and orchestrate data workflows, ensuring eEicient data ingestion

processes.

• Utilize Delta Lake for data storage and management, ensuring secure data

sharing across platforms with Delta Sharing.

• Implement data governance and access control using Unity Catalog.

• Apply the Medallion architecture (Bronze, Silver, Gold layers) to projects.

• Develop and manage ETL pipelines using AWS Glue.

• Leverage Amazon S3 for data storage and management.

• Utilize AWS Lambda and Step Functions for serverless computing in data

processing.

• Apply data warehousing solutions and best practices in data transformation.

• Monitor and implement data quality checks and governance practices.

• Collaborate with data scientists, analysts, and other stakeholders in an Agile

environment.

• Deliver large-scale data processing and analytics projects within cloud

environments.

Required Skills and Qualifications

• Databricks certifications with practical experience in the Databricks Lakehouse

platform.

• Proficient in AWS services: Glue, S3, Lambda, Step Functions, and data

warehousing solutions.

• Strong programming skills in Python, Scala, or Java.

• Proficient in SQL for querying and manipulating data.

• Experience in data modeling for Lakehouse architecture.

• Knowledge of ETL best practices and data transformation techniques.

• Understanding of data governance frameworks and practices.

• Proven track record of delivering data engineering projects, particularly in cloud

environments.

• Experience with large-scale data processing and analytics projects.

• Ability to work in Agile teams and collaborate eEectively with various

stakeholders.

Preferred Qualifications

• Advanced AWS certifications.

• Experience with additional cloud platforms (e.g., Azure).

• Familiarity with other data engineering tools and frameworks.