Data Engineer

You will be responsible for building and maintaining secure, scalable data pipelines using Databricks and Azure. This includes handling ingestion from various sources such as files, APIs, and streaming, performing data transformation, and ensuring quality validation. Collaboration with subsystem data science and product teams will be essential to prepare for machine learning readiness. Your technical expertise should cover working with Notebooks (SQL, Python), Delta Lake, Unity Catalog, ADLS/S3, job orchestration, APIs, structured logging, and Infrastructure as Code (IaC) tools like Terraform. Delivery practices like Trunk-based development, Test-Driven Development (TDD), Git, and Continuous Integration/Continuous Deployment (CI/CD) for notebooks and pipelines are expected. You should also be familiar with integrating different data formats such as JSON, CSV, XML, Parquet, and various databases including SQL, NoSQL, and graph databases. Strong communication skills are vital to justify decisions, document architecture, and align with enabling teams. In this role, you will have the opportunity to engage in Proximity Talks, where you can interact with other designers, engineers, and product experts to enhance your knowledge. Working with a world-class team will enable you to constantly challenge yourself and acquire new skills on a daily basis. This is a contract position based in Abu Dhabi, and if relocation from India is necessary, the company will cover travel and accommodation expenses on top of your salary. Proximity is a globally recognized technology, design, and consulting partner for leading Sports, Media, and Entertainment companies. Headquartered in San Francisco, with offices in Palo Alto, Dubai, Mumbai, and Bangalore, Proximity has been instrumental in creating and scaling high-impact products used by 370 million daily users, with a combined net worth of $45.7 billion among client companies. As part of the Proximity team, you will work alongside a diverse group of coders, designers, product managers, and experts who solve complex challenges and develop cutting-edge technology at scale. The rapidly growing team of Proxonauts offers you the opportunity to make a significant impact on the company's success. You will have the chance to collaborate with experienced leaders who have led multiple tech, product, and design teams. To learn more about us, you can watch our CEO, Hardik Jagda, share insights about Proximity, explore our values and meet our team members, visit our website, blog, and Studio Proximity, and get an insider perspective on Instagram by following @ProxWrks and @H.Jagda.,