Azure Data Engineer

You will be responsible for developing and maintaining scalable data processing systems using Apache Spark and Azure Databricks. This includes implementing data integration from various sources such as RDBMS, ERP systems, and files. You will design and optimize SQL queries, stored procedures, and relational schemas. Additionally, you will build stream-processing systems using technologies like Apache Storm or Spark-Streaming, and utilize messaging systems like Kafka or RabbitMQ for data ingestion. Performance tuning of Spark jobs for optimal efficiency will be a key focus area. Collaboration with cross-functional teams to deliver high-quality data solutions is essential in this role. You will also lead and mentor a team of data engineers, fostering a culture of continuous improvement and Agile practices. Key skills required for this position include proficiency in Apache Spark and Azure Databricks, strong experience with Azure ecosystem and Python, as well as working knowledge of Pyspark (Nice-to-Have). Experience in data integration from varied sources, expertise in SQL optimization and stream-processing systems, familiarity with Kafka or RabbitMQ, and the ability to lead and mentor engineering teams are also crucial. A strong understanding of distributed computing principles is a must. To qualify for this role, you should hold a Bachelor's degree in Computer Science, Information Technology, or a related field.,