Job
Description
Job Description We are seeking a highly skilled Azure Data Engineer with strong expertise in Data Architecture , PySpark/Python, Azure Databricks, and data streaming solutions . The ideal candidate will have hands-on experience in designing and implementing large-scale data pipelines, along with solid knowledge of data governance and data modeling . Key Responsibilities Design, develop, and optimize PySpark/Python-based data streaming jobs on Azure Databricks . Build scalable and efficient data pipelines for batch and real-time processing. Implement data governance policies, ensuring data quality, security, and compliance. Develop and maintain data models (dimensional, relational, NoSQL) to support analytics and reporting. Collaborate with cross-functional teams (data scientists, analysts, and business stakeholders) to deliver data solutions. Troubleshoot performance bottlenecks and optimize Spark jobs for efficiency. Ensure best practices in CI/CD, automation, and monitoring of data workflows. Mentor junior engineers and lead technical discussions (for senior/managerial roles). Mandatory Skills & Experience 5+ years of relevant experience as a Data Engineer/Analyst/Architect (8+ years for Manager/Lead positions). Expert-level proficiency in PySpark/Python and Azure Databricks (must have worked on real production projects ). Strong experience in building and optimizing streaming data pipelines (Kafka, Event Hubs, Delta Lake, etc.). 4+ years of hands-on experience in data governance & data modeling (ER, star schema, data vault, etc.). In-depth knowledge of Azure Data Factory, Synapse, ADLS, and SQL/NoSQL databases . Experience with Delta Lake, Databricks Workflows, and performance tuning . Familiarity with data security, metadata management, and lineage tracking . Excellent communication skills (must be able to articulate technical concepts clearly).
