Job
Description
We have 2 requirements : 1 - Azure Engineer Specialist Mandate Skills - ADB, Pyspark , ADF , Delta Lake 2 - Azure Lead Mandate Skills - ADF, ADB , Syapse , Python , Erwin Also looking for someone who is handling experience in team leading. Data Engineer utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle the distribution of data types and volumes in support of data architecture design. A Senior Data Engineer designs and oversees the entire data infrastructure, data products, and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost-effective. Key Responsibilities: Design and oversee the entire data architecture strategy. Mentor junior data architects to ensure skill development in alignment with the team strategy. Design and implement complex scalable, high-performance data architectures that meet business requirements. Model data for optimal reuse, interoperability, security, and accessibility. Develop and maintain data flow diagrams and data dictionaries. Collaborate with stakeholders to understand data needs and translate them into technical solutions. Ensure data accessibility through a performant, cost-effective consumption layer that supports use by citizen developers, data scientists, AI, and application integration. Ensure data quality, integrity, and security across all data systems. Qualification: Experience in Erwin, Azure Synapse, Azure Databricks, Azure DevOps, SQL, Power BI, Spark, Python, and R. Ability to drive business results by building optimal cost data landscapes. Familiarity with Azure AI/ML Services, Azure Analytics: Event Hub, Azure Stream Analytics, Scripting: Ansible Experience with machine learning and advanced analytics. Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes). Understanding of CI/CD pipelines and automated testing frameworks. Certifications such as AWS Certified Solutions Architect, IBM certified data architect, or similar are a plus.
