Data Engineering Lead

You will be responsible for architecting, building, and optimizing scalable data pipelines and platforms to drive critical business decisions. Your expertise in PySpark, ETL frameworks, SQL Stored Procedures, and Data Modeling will be crucial for performance tuning across large datasets. - Lead the design and development of scalable data pipelines and ETL workflows using PySpark and SQL - Develop and maintain Stored Procedures for batch processes and data transformations - Design and implement robust data models (Star, Snowflake, and normalized models) for data warehouse and lake environments - Collaborate with BI, Analytics, and Product teams to understand data requirements and ensure data quality, lineage, and integrity - Optimize data workflows for performance, cost, and scalability across distributed systems - Manage data ingestion from structured and semi-structured sources (e.g., APIs, files, databases, streams) - Support CI/CD and DevOps practices for data pipeline deployments - Collaborate with cross-functional teams on data governance, access controls, and compliance - Guide and mentor junior data engineers; perform code reviews and technical planning **Qualifications Required:** - 5+ years of experience in data engineering and data platform development - Experience with Azure ADlS Synapse or Microsoft Fabric - Expertise in PySpark/Spark SQL for large-scale distributed data processing - Understanding of ETL/ELT architectures, data ingestion, and transformation strategies - Familiarity with data modeling techniques and tools - Proven track record in performance tuning of queries, jobs, and pipelines - Experience with tools like Airflow or equivalent for orchestration and transformations The company is looking for a Data Engineering Lead with a strong background in PySpark, ETL frameworks, SQL Stored Procedures, and Data Modeling to optimize data pipelines and platforms for critical business decisions.,