Lead I - Data Engineering

Overview We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in building scalable data solutions on the Azure ecosystem. The ideal candidate will have strong hands-on expertise in Databricks, Spark (Java), modern data lake architectures, and end-to-end ETL development.

Key Responsibilities • Design, develop, and maintain scalable data pipelines using Azure and Databricks. • Build efficient ingestion and transformation workflows using Databricks with Pyspark and Delta Lake • Develop and optimize ETL pipelines using Java and SQL. • Implement best practices for data modeling, data quality, governance, and pipeline performance. • Work with relational and lakehouse databases including PostgreSQL and Delta Lake. • Collaborate with cross-functional teams and support production workloads, troubleshooting, and optimization.

Required Skills & Experience • 5+ years of hands-on data engineering experience. • Strong working knowledge of the Azure ecosystem, especially: o Azure Databricks o Azure Data Lake Storage (ADLS) o Azure Functions o Azure Data Factory (optional but preferred) o Expertise in Apache Spark with Java (PySpark). • Advanced SQL skills including query optimization. • Strong understanding of ETL/ELT methodologies and data warehousing concepts. • Experience working with Delta Lake (schema evolution, ACID, time travel). • Experience with PostgreSQL or similar databases. • Strong problem-solving skills and attention to detail. • Ability to communicate effectively with technical and non-technical teams. • Proactive, self-driven, and able to work in fast-paced environments. Preferred Skills • Experience with CI/CD tools (Azure DevOps, Git). • Knowledge of infrastructure-as-code tools (Terraform). • Exposure to streaming platforms (Kafka, Event Hub). • Familiarity with data governance tools such as Purview.

Mandatory Skills

Skills: Azure Cloud,Java,Pyspark