Data Engineer (AWS/ Azure)

As an experienced Data Engineer, you will be responsible for the following key responsibilities: - Leading development activities for the Data engineering team, ensuring efficient collaboration with other teams such as application management and product delivery. - Working closely with technical leads, product managers, and support teams to provide guidance and leadership in implementing tools and technologies for cost-efficient architecture and infrastructure. For AWS Data Engineer: - Utilize AWS services like S3, Glue, SNS, SQS, Lambda, Redshift, and RDS effectively. - Demonstrate proficiency in programming, particularly in Python. - Design complex SQL queries and optimize data retrieval processes. - Possess a strong understanding of Spark, Pyspark, Hadoop, Hive, and Spark-SQL. For Azure Data Engineer: - Develop and maintain optimal data pipelines. - Collect and manage large, intricate data sets to meet business requirements. - Identify and implement internal process enhancements, such as automation and infrastructure redesign for scalability. - Create data tools to support analytics and data science teams, enhancing product innovation. - Collaborate with stakeholders to address data-related technical issues and support infrastructure needs. Qualifications Required: For AWS Data Engineer: - Hands-on experience with AWS services mentioned above. - Proficiency in Python programming. - Strong skills in designing complex SQL queries and optimizing data fetching processes. - Knowledge of Spark, Pyspark, Hadoop, Hive, and Spark-SQL. For Azure Data Engineer: - Familiarity with Azure cloud services. - Experience in developing Big Data applications using Spark, Hive, Sqoop, Kafka, and Map Reduce. - Exposure to stream-processing systems like Spark-Streaming and Strom. This job offers the opportunity to work with cutting-edge technologies and collaborate with diverse teams to drive innovation in data engineering.,