Data Engineer

You will be responsible for developing and enhancing data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation. Collaborate with product and technology teams to design and validate the capabilities of the data platform. Identify, design, and implement process improvements, automating manual processes, optimizing for usability, and re-designing for greater scalability. Provide technical support and usage guidance to the users of our platforms services. Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services. You should have experience building and optimizing data pipelines in a distributed environment. Experience supporting and working with cross-functional teams. Proficiency working in a Linux environment. A minimum of 4 years of advanced working knowledge of SQL, Python, and PySpark is required. Knowledge of PySpark queries is a must. Knowledge of Palantir is also needed. Experience using tools such as Git/Bitbucket, Jenkins/CodeBuild, Code Pipeline. Experience with platform monitoring and alert tools.,