Data Engineer

We are seeking a Databricks Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions on the Databricks Lakehouse Platform. You will be responsible for transforming raw data into high-value datasets that power analytics, AI/ML models, and business insights. Were looking for a dynamic, self-motivated, and proactive leader who thrives in a fast- paced environment and can foster a high-performing culture.

• Develop scalable ETL/ELT pipelines using PySpark, SQL, and Delta Lake.
• Build and maintain batch and stream data pipelines using Databricks Workflows.
• Design and implement Lakehouse architecture and optimize Delta Lake tables.
• Integrate data from multiple sources including APIs, databases, cloud storage, and third-party platforms.
• Ensure data quality and reliability using Delta Live Tables, expectations, and monitoring tools.
• Work with CI/CD pipelines using Git, Databricks Repos, and DevOps tools.
• Implement governance, security, and compliance standards (Unity Catalog, lineage, RBAC).

Must have skills

• Strong knowledge of data modeling, data warehousing concepts, and Lakehouse fundamentals.
• Strong experience with Databricks, PySpark, Apache Spark, and SQL.
• Hands-on with Delta Lake, Databricks Workflows, Unity Catalog.
• Experience building scalable pipelines on Azure, AWS, or GCP.
• Solid understanding of distributed systems and performance optimization.
• Experience with streaming frameworks (Structured Streaming, Kafka, Auto Loader).
• Passion for problem-solving both technical and business-oriented
• User-centric mindset focused on building delightful software experiences
• Lead by example: implement best practices and maintain high-quality code standards

Nice to have skills

• Databricks certification (Data Engineer Associate / Professional).
• Experience with ML workflows and feature engineering.
• Hands-on with Databricks Photon, serverless SQL, and optimization techniques.
• Familiarity with BI tools (Power BI, Tableau, Looker)