Senior Data Engineer

This is an on-site role based in Pune / Hyderabad. Below is a brief overview of what we’re looking for:

Strong Experience with Python and PySpark coding

Strong expertise with AWS Cloud

The required experience is 8+ years

Here is the job description as well:

100% onsite in the Pune / Hyderabad office.

Key things: Python, PySpark, AWS

Responsibilities

·       Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.

·       Collaborate with product and technology teams to design and validate the capabilities of the data platform

·       Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability

·       Provide technical support and usage guidance to the users of our platform’s services.

·       Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services.

Qualifications

·       Experience building and optimizing data pipelines in a distributed environment

·       Experience supporting and working with cross-functional teams

·       Proficiency working in a Linux environment

·       5+ years of advanced working knowledge of SQL, Python, and PySpark

·       Knowledge or hands-on experience with Palantir Foundry

·       Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline

·       Experience with platform monitoring and alert tools