Lead Azure Big Data Engineer

As a Senior Azure Data Engineer, you will be responsible for designing, building, and optimizing scalable data pipelines for large-scale data processing using Python and Apache Spark. Your role will involve leveraging Azure Databricks for big data processing, working with PySpark to analyze large datasets, and utilizing Azure Data Factory for seamless data integration across cloud environments. Additionally, you will implement and maintain solutions using Azure Data Lake Storage for big data analytics and transformation. Your key responsibilities will include writing efficient and reusable Python code, working with core Azure data services like Azure SQL Database and Azure Synapse Analytics, and designing and maintaining complex SQL queries for managing relational databases. You will also collaborate with cross-functional teams to understand data requirements, ensure data management integrity and security, and provide mentorship to junior team members. To excel in this role, you should have proficiency in Python programming, hands-on experience with libraries such as Pandas and NumPy, and strong skills in SQL for querying and managing relational databases. In-depth knowledge of Azure technologies, including Azure Data Factory, Azure Cosmos DB, and Azure Blob Storage, is essential. Your problem-solving skills, analytical mindset, and excellent communication abilities will be valuable assets in this position. The ideal candidate for this role will hold a B.Tech degree in any branch or specialization, with 3-7 years of relevant experience. Architectural experience and familiarity with infrastructure as code tools like Terraform or Biceps will be considered a plus. If you are passionate about data engineering, cloud platforms, and working with complex data challenges, we encourage you to apply for this exciting opportunity.,