Senior Data Engineer Palantir 64sqr Apex Llp

7.0 - 10.0 years

20 - 25 Lacs

bengaluru

Work from Office

Job Description: We are hiring a Senior Data Engineer with strong expertise in Palantir Foundry or Gotham to lead the design, development, and optimization of data engineering solutions across large-scale enterprise environments. The ideal candidate will have deep experience building secure, high-performance data pipelines using Python, ETL/ELT practices, distributed computing frameworks, and cloud platforms ( AWS ,Azure ,GCP)while working closely with DevOps, analytics, and business teams to enable data-driven decisions. Key Responsibilities: Architect, build, and optimize large-scale ETL/ELT pipelines using Palantir Foundry/Gotham . Integrate diverse data sources including SQL , NoSQL , APIs , and real-time streaming platforms (e.g., Kafka) into Palantirs ontology-driven models . Develop and customize ontologies , transforms , and operational workflows within Palantir to support business intelligence and analytics. Implement scalable, distributed processing using frameworks like Apache Spark , Hadoop , and other big data technologies for handling petabyte-scale datasets . Write efficient, production-grade Python scripts for data transformation, automation, workflow orchestration, and custom business logic. Optimize data workflows and platform performance through query tuning , caching , partitioning , and incremental data updates . Ensure robust data governance , lineage tracking , and enterprise-grade security (e.g., RBAC, encryption). Collaborate with DevOps and platform engineering teams to implement and maintain CI/CD pipelines and automated, scalable deployment processes . Maintain and version-control data assets and transformations using Git and DevOps best practices. Partner with data scientists, analysts, and stakeholders to translate complex business requirements into scalable data engineering solutions. Create and maintain comprehensive technical documentation , including data models, architecture diagrams, deployment workflows, and operational guides. Required Skills: Proven expertise in Palantir Foundry or Gotham , with deep knowledge of its ontology framework, code workbooks, and data pipeline architecture. Strong programming experience in Python and advanced SQL . Hands-on experience with ETL/ELT design , data transformation, and workflow automation. Proficiency with Apache Spark , Hadoop , or similar distributed data processing frameworks. Experience integrating data from relational and non-relational databases , streaming sources , and external APIs . Understanding of data modeling , ontology-driven design , and semantic layers . Solid grasp of CI/CD , DevOps practices , and deployment automation tools. Familiarity with cloud platforms (AWS, Azure, or GCP), containerization (Docker), and orchestration tools (Kubernetes, Airflow). Nice to Have: Experience with Foundry Ontology SDK , Object Explorer, and Python transforms in Palantir. Exposure to data cataloging , metadata management , and monitoring frameworks . Understanding of ML/AI pipeline integration within data platforms. Certifications in Palantir , cloud platforms , or big data technologies.

Posted Date not available

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.