86 Apache Iceberg Jobs - Page 3

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

Job Description Join our dynamic Digital Marketing Data Engineering team at Fanatics, where you&aposll play a critical role in shaping the big data ecosystem that powers our eCommerce and Digital Marketing platforms. As a full-time Staff Data Engineer, youll design, build, and optimize scalable data pipelines and architectures, ensuring seamless data flow and effective collection across cross-functional teams. You will also leverage your backend engineering skills to support API integrations and real-time data exchanges. What We&aposre Looking For BTech/MTech/BS/MS in Computer Science or a related field, or equivalent practical experience. 9+ years of software engineering experience, with a ...

About the Role We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data platforms that empower data-driven decision-making. This role requires deep technical expertise in modern data engineering frameworks, architectural patterns, and cloud-native solutions on AWS. You will be a key contributor to our data strategy, ensuring data quality, governance, and reliability while mentoring other engineers in the team. Key Responsibilities Design, develop, and own robust, scalable, and maintainable data pipelines (batch & real-time). Architect and implement Data Lake, Data Warehouse, and Lakehouse solutions using modern frameworks and architectural patterns. E...

About Role We are looking for an experienced Backend Developer with strong expertise in Python, FastAPI, Socket.IO, and Kubernetes to join our team in Bangalore. The ideal candidate has a proven track record of building and scaling production-grade systems handling 10k+ users, with deep knowledge of microservices, asynchronous programming, and distributed architectures. Responsibilities Design, develop, and maintain backend services and APIs using FastAPI and Python. Build and optimize real-time applications leveraging WebSockets (Socket.IO) and asyncio. Architect scalable microservices and ensure high availability in Kubernetes (K8s) environments. Implement message queues (Celery, RabbitMQ,...

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 7+ years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 46 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

The Platform Data Engineer will be responsible for designing and implementing robust data platform architectures, integrating diverse data technologies, and ensuring scalability, reliability, performance, and security across the platform. The role involves setting up and managing infrastructure for data pipelines, storage, and processing, developing internal tools to enhance platform usability, implementing monitoring and observability, collaborating with software engineering teams for seamless integration, and driving capacity planning and cost optimization initiatives.

When 5% of Indian households shop with us, it's important to build data-backed, resilient systems to manage millions of orders every day. We've done this - with zero downtime! ???? Sounds impossible Well, that's the kind of Engineering muscle that has helped Meesho become the e-commerce giant that it is today. We value speed over perfection, and see failures as opportunities to become better. We've taken steps to inculcate a strong Founder's Mindset across our engineering teams, making us grow and move fast. We place special emphasis on the continuous growth of each team member - and we do this with regular 1-1s and open communication. Tech Culture We have a unique tech culture where enginee...

As a Software Development Engineer at our company, you will have the opportunity to work in either Hyderabad, Telangana, India or Bengaluru, Karnataka, India. You should hold a Bachelor's degree in Computer Science, a related technical field, or have equivalent practical experience. With at least 8 years of experience in software development using general-purpose programming languages, you will be responsible for designing and building solutions for data migrations and developing software for Data Warehouse migration. Your expertise in programming languages such as Java, C/C++, Python, or Go will be crucial for this role. Additionally, experience in systems engineering with designing and bui...

We are looking for a highly skilled Scala Data Engineer to design, build, and optimize large-scale data platforms. The ideal candidate will have deep expertise in Scala, Spark, and SQL, with proven experience delivering scalable and high-performance data solutions in cloud-native environments. Key Responsibilities Design, build, and optimize scalable data pipelines using Apache Spark and Scala. Develop real-time streaming pipelines leveraging Kafka/Event Hubs. Own the design and architecture of data systems with a strong focus on performance, scalability, and reliability. Collaborate with cross-functional teams to deliver high-quality data products. Mentor junior engineers and enforce best p...

ROLE PROFILE: Are you ready to take your engineering career to the next level Join us as a Mid Lead Engineer and contribute to building state-of-the-art data platforms in AWS , leveraging Python and Spark . Be part of a dynamic team, driving innovation and scalability for data solutions in a supportive and hybrid work environment. ROLE SUMMARY: This role is ideal for an experienced data engineer looking to step into a leadership position while remaining hands-on with cutting-edge technologies. You will design, implement and optimize ETL workflows using Python and Spark, contributing to our robust data lakehouse architecture on AWS. Success in this role requires technical expertise, strong pr...

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorgan Chase within Asset and Wealth Management, you play a crucial role in an agile team dedicated to improving, developing, and providing reliable, cutting-edge technology solutions that are secure, stable, and scalable. As a key technical contributor, you are tasked with implementing essential technology solutions across diverse technical domains within various business functions to support the firm's strategic goals. Job responsibilities Executes creative software solutions, design, development, and technical troubleshooting with ability...

You have a fantastic opportunity to join us as a DAML Presales Engineer with a rich experience of 10-16 years. In this role, you will serve as a trusted advisor to our customers, guiding them towards effective DAML-based solutions integrated seamlessly with modern AWS data architectures. Your expertise in AWS data analytics platforms will be instrumental in solving intricate business challenges. Your main responsibilities will include collaborating closely with sales teams and AWS stakeholders to gain insights into client issues and crafting innovative DAML-based smart contract solutions that work harmoniously with AWS Data Lake and analytics services. You will be expected to deliver impactf...

You will be responsible for developing, deploying, monitoring, and maintaining ETL Jobs as well as all data engineering and pipeline activities. Your role will involve having a good understanding of DB activities and providing support in DB solutions. Additionally, you must possess proven expertise in SQL queries. Your key responsibilities will include designing and constructing various enterprise procedure constructs using any ETL tool, preferably PentahoDI. You will be expected to provide accurate work estimates, manage efforts across multiple lines of work, design and develop exception handling and data cleansing/standardization procedures, gather requirements from various stakeholders re...

As a DAML Presales Engineer with 10-16 years of experience, your primary role will be to act as a trusted advisor to customers, advocating for DAML-based solutions integrated with modern AWS data architectures. Your responsibilities will include collaborating with sales teams and AWS stakeholders to design smart contract solutions using DAML and AWS Data Lake and analytics services. You will be expected to deliver compelling demos, PoCs, and technical presentations showcasing DAML solutions and AWS data platforms like Glue, EMR, and Redshift. Your expertise in AWS data lake components such as Glue, EMR, and Redshift will be crucial in designing solutions that balance performance, cost, and r...

Job Overview: Responsible for developing, deploying, monitoring and maintaining ETL Jobs and all data engineering and pipeline activities. Having good knowledge of DB activities and support in DB solutions. Having proven expertise in SQL queries. Responsibilities and Duties Design and construction of various enterprise procedure constructs using any ETL tool preferably PentahoDI. Provide accurate work estimates and manager efforts across multiple lines of work Design and develop exception handling and data cleansing/standardization procedures Requirement gathering from various stakeholders related to ETL automation Designing and creating the data extraction, transformation, and load of data ...

About the Team As a part of the DoorDash organization, you will be joining a data-driven team that values timely, accurate, and reliable data to make informed business and product decisions. Data serves as the foundation of DoorDash's success, and the Data Engineering team is responsible for building database solutions tailored to various use cases such as reporting, product analytics, marketing optimization, and financial reporting. By implementing robust data structures and data warehouse architecture, this team plays a crucial role in facilitating decision-making processes at DoorDash. Additionally, the team focuses on enhancing the developer experience by developing tools that support th...

As a Lead database engine developer, you will play a crucial role in enhancing our database engine to operate at an exabyte scale. Our analytical database engine processes trillions of data points daily, enabling rapid queries with an impressive 60 ms response time at P50. Your technical expertise and leadership will be pivotal in ensuring that our system seamlessly manages exabytes of data on a daily basis. Your responsibilities will include developing and executing innovative technical strategies for our database engine that align with Newrelic's business objectives. You will focus on optimizing scalability and performance to handle exabyte-scale data while maintaining exceptional query pe...

Job description About this Team: IT Data Platform (ITDP) is the powerhouse data platform driving Target'stech efficiencies, seamlessly integrating operational and analytical needs. Itfuels every facet of Target Tech, from boosting developer productivity andenhancing system intelligence to ensuring top-notch security andcompliance. Target Tech builds the technology that makes Target the easiest, safest andmost joyful place to shop and work. From digital to supply chain tocybersecurity, develop innovations that power the future of retail whilerelying on best-in-class data science algorithms that drive value. TargetTech is at the forefront of the industry, revolutionizing technology efficiencyw...

As a Senior Staff Software Engineer in Data Lake House Engineering, you will play a crucial role in designing and implementing the Data Lake house platform, supporting both Data Engineering and Data Lake house applications. Your responsibilities will include overseeing Data Engineering pipeline productionalization, end-to-end data pipelines, model development, deployment, monitoring, refresh, etc. Additionally, you will be involved in driving technology development and architecture to ensure the platforms, systems, tools, models, and services meet the technical standards for security, quality, reliability, usability, scalability, performance, efficiency, and operability to meet the evolving ...

We are looking for an experienced and dedicated Senior Manager of Business Intelligence & Data Engineering to lead a team of engineers. In this role, you will oversee various aspects of the Business Intelligence (BI) ecosystem, including designing and maintaining data pipelines, enabling advanced analytics, and providing actionable insights through BI tools and data visualization. Your responsibilities will include leading the design and development of scalable data architectures on AWS, managing Data Lakes, implementing data modeling and productization, collaborating with business stakeholders to create actionable insights, ensuring thorough documentation of data pipelines and systems, prom...

The candidate should have proven expertise in building scalable platforms that are customer facing and have expertise in evangelizing the platform with customers and with internal stakeholders Expert level knowledge of Cloud Computing including aspects of VPC Network Design, Shared Responsibility Matrix, Cloud databases, No SQL Databases, Data Pipelines on the cloud, VM and VM orchestration, Serverless frameworks. This should be across all 3 major cloud providers (AWS, Azure, GCP), preferably at least in 2 of the 3 Public Clouds Expert level knowledge in Data Ingestion paradigms & use of different types of databases like OLTP, OLAP for specific purposes Hands-on experience with Apache Spark,...

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 46 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 7+ years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.