Sr Data Engineer

As a Data Engineer at Everstream Analytics, you will be a key player in the development and maintenance of our data infrastructure. Working alongside a team of skilled engineers, you will be responsible for designing, developing, and optimizing Python-based data pipelines and products that support our cloud-native data platform. Your expertise will be crucial in utilizing various AWS services such as MSK (Kafka), Lambda, Glue, Spark, Athena, Lake Formation, Redshift, S3, and RDS to ensure the scalability, reliability, and efficiency of our data infrastructure. Your responsibilities will include architecting, developing, and owning data pipelines that manage large volumes of data from multiple sources while maintaining data quality, integrity, and availability. You will leverage your expertise in AWS data services to create scalable and cost-effective data solutions. Experience with relational databases like PostgreSQL on RDS, graph databases like Neo4j, stream processing tools such as Apache Kafka and Apache Spark, and proficiency in Python development will be essential for success in this role. Collaboration with Product Management, Data Science, and leadership teams to understand data requirements and deliver solutions that meet business needs will be a key aspect of your role. Additionally, you will be responsible for monitoring and optimizing data pipelines for scalability and efficiency, maintaining documentation for data engineering processes, and providing leadership within the data engineering team. The ideal candidate will have proven experience in designing and building cloud-native data platforms in a SaaS or PaaS environment, proficiency in AWS services, relational and graph database technologies, distributed system design, data warehousing, and stream processing. Strong programming skills in Python, problem-solving abilities, and the capability to work collaboratively with cross-functional teams are crucial. A degree in Computer Science, Data Engineering, or related field, or equivalent experience is preferred. This position is based at the Everstream Analytics office in Koregaon Park, Pune. Everstream Analytics, a company focused on revolutionizing the supply chain industry with disruptive technology, offers a dynamic work environment where growth and innovation are encouraged. If you are passionate about driving change and want to be part of a team that values resiliency, responsiveness, and critical thinking, consider joining Everstream Analytics to advance your career. Learn more about Everstream Analytics at www.everstream.ai.,