Databricks Data Engineer | Senior Data Engineer

We are Hiring: Databricks Data Engineer | Senior Data Engineer | Migration / Technical Architect

We are looking for skilled Databricks professionals to join our data engineering and migration teams across multiple experience levels.

Experience

Locations:

Work Mode: Work From office

Notice Period : Immediate to 30 days

Key Responsibilities

• Assess legacy databases, ETL jobs, data warehouses, and reporting dependencies to identify migration, redesign, or retirement opportunities.
• Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, SQL, and Delta Lake.
• Implement data ingestion and transformation using Unity Catalog, Lakeflow, Auto Loader, Databricks Workflows, and Databricks SQL.
• Migrate legacy data platforms and workloads from Oracle, SQL Server, SSIS, Informatica, Teradata, Snowflake, Hadoop, and other data platforms to Databricks.
• Optimize Spark and SQL workloads through performance tuning, partitioning, liquid clustering, Photon, and efficient compute utilization.
• Develop reusable, metadata-driven components to automate ingestion, conversion, validation, reconciliation, and deployment.
• Perform schema validation, record-count checks, business-rule testing, regression testing, parallel runs, and cutover readiness assessments.
• Implement production-ready logging, monitoring, restartability, security, and data quality controls.
• Follow CI/CD, DevOps, data governance, and deployment best practices.
• For senior and architect-level roles, lead migration workstreams, review solution designs, mentor engineers, and resolve complex technical challenges.

Mandatory Skills

• Strong hands-on experience in Databricks, PySpark, Python, and SQL.
• Practical knowledge of Delta Lake, Unity Catalog, Lakeflow, Auto Loader, and Databricks Workflows.
• Experience in data engineering, ETL/ELT development, data modelling, and production data pipelines.
• Knowledge of incremental loads, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD).
• Hands-on experience with data validation, reconciliation, testing, and performance optimization.
• Understanding of CI/CD, workspace configuration, data governance, and production support.
• Ability to understand business requirements, data consumption patterns, and the software development lifecycle (SDLC).