124 Apache Iceberg Jobs - Page 2

Job Summary We are seeking an experienced Data Lake Expert to design, implement, and optimize enterprise-grade data lake and data warehouse systems using modern open-source technologies such as Apache Iceberg, Apache Hudi, Delta Lake, and Apache Spark . The ideal candidate will have deep expertise in data ingestion, transformation, storage optimization, and governance across structured and unstructured data sources. This role involves architecting and managing scalable data platforms that enable real-time analytics, AI/ML pipelines, and regulatory compliance . Key Responsibilities 1. Data Lake Architecture & Design Architect and implement scalable data lakehouse solutions using Apache Iceber...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Apache Impala Developer at BairesDev We are seeking an Apache Impala Developer with expertise in SQL query engines, columnar storage optimizatio...

We are looking for a skilled Data Engineer with a strong background in PostgreSQL, SQL, and Python to join our dynamic team in Kochi. The ideal candidate will be responsible for building and optimising data pipelines, ensuring seamless data flow across systems, and supporting our analytics and product teams with reliable data solutions. Job Title: Data Engineer Company: Pumex Infotech Pvt. Ltd. Experience: 5 to 10 Years Educational qualification: B Tech/ M Tech/ BCA/MCA (Computer Science) or related field. Location : InfoPark, Kochi Notice Period: Immediate Joiners Responsibilities: Design, develop, and maintain robust ETL/ELT data pipelines using tools such as Airflow or dbt . Optimize Post...

As an experienced Data Engineer, your role will involve designing ingestion pipelines, optimizing query performance, and ensuring data quality, governance, and cost efficiency at scale. You will be working with large workloads of 50-300 TB. **Key Responsibilities:** - Migration Strategy & Execution - Design and implement data ingestion pipelines to extract data from Oracle into GCS/Iceberg. - Migrate and modernize existing Oracle schemas, partitions, and materialized views into Iceberg tables. - Define CDC (Change Data Capture) strategies using custom ETL. - Data Lakehouse Architecture - Configure and optimize Trino clusters (coordinator/worker, Helm charts, autoscaling). - Design partitioni...

Role & responsibilities Key Qualifications Experience with open Data Lakehouse using Apache Iceberg Experience with Data Lakehouse architecture with Apache Iceberg and Trino Experience with building/managing Lakehouse for large scale data platforms storage, computing aspects, data ingestion/governance and data management and performance. Experience with distributed computing technology(s) like Spark. Experience with data platforms, data lakes and ingestion systems that work at scale. Skilled at troubleshooting complex issues in large scale distributed systems.

Introduction Joining the IBM Technology Expert Labs teams means you'll have a career delivering world-class services for our clients. As the ultimate expert in IBM products, you'll bring together all the necessary technology and services to help customers solve their most challenging problems. Working in IBM Technology Expert Labs means accelerating the time to value confidently and ensuring speed and insight while our clients focus on what they do bestrunning and growing their business. Excellent onboarding and industry-leading learning culture will set you up for a positive impact, while advancing your career. Our culture is collaborative and experiential. As part of a team, you will be su...

Big Data DeveLoper Responsibilities Design and implement data pipelines for migration from HDFS/Hive to cloud object storage (e.g., S3, Ceph). Optimize Spark (and optionally Flink) jobs for performance and scalability in a Kubernetes environment. Ensure data consistency, schema evolution, and governance with Apache Iceberg or equivalent table formats. Support migration strategy definition by providing technical input and identifying risks. Mentor junior developers and review their code / design decisions. Collaborate with platform engineers, cloud architects, and product stakeholders to align technical implementation with project goals. Troubleshoot complex distributed system issues in data ...

Big Data DeveLoper Responsibilities Design and implement data pipelines for migration from HDFS/Hive to cloud object storage (e.g., S3, Ceph). Optimize Spark (and optionally Flink) jobs for performance and scalability in a Kubernetes environment. Ensure data consistency, schema evolution, and governance with Apache Iceberg or equivalent table formats. Support migration strategy definition by providing technical input and identifying risks. Mentor junior developers and review their code / design decisions. Collaborate with platform engineers, cloud architects, and product stakeholders to align technical implementation with project goals. Troubleshoot complex distributed system issues in data ...

Skills : Snowflake + DBT + Iceberg + Apacheflink Exp: 5 to 10 years NP: Immediate to 15 Days Location: Hyderabad About the Role We are seeking a highly skilled Data Engineer to design, build, and optimize modern data pipelines and data models that power our analytics and data products. The ideal candidate will have strong experience with Snowflake and DBT , and familiarity or hands-on experience with stream processing using Apache Flink and data lakehouse technologies such as Apache Iceberg . You will work closely with data analysts, data scientists, and platform engineers to ensure efficient, scalable, and reliable data workflows across the organization. Key Responsibilities Design, develop...

As a Lead Big Data Developer at CGI, you will play a crucial role in designing, building, and optimizing large-scale data pipelines and processing systems. Here is a summary of your role, key responsibilities, and qualifications required: Role Overview: - Lead the design and development of scalable, efficient, and reliable data pipelines using PySpark, Python, and SQL. - Collaborate with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions. - Optimize data workflows for performance, scalability, and cost efficiency in big data environments. - Implement data ingestion, transformation, and aggregation processes from mu...

We're looking for a Data Platform Engineer with deep systems engineering expertise and a passion for building reusable infrastructure. You'll help architect and deliver the internal frameworks that power data ingestion, transformation, observability, and self-serve analytics across the company. Your work will be central to enabling scalable, multi-tenant ingestion pipelines (including CDC via Debezium), serving layers built on Trino, and robust tooling that lets teams move fast without compromising on data quality or platform health. What Makes This Opportunity Unique Build platform tooling for multi-region, multi-tenant data infrastructure using modern open-source and cloud-native technolog...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Apache Impala Developer at BairesDev We are seeking an Apache Impala Developer with expertise in SQL query engines, columnar storage optimizatio...

As a Technical Lead- Data Engineer at Sigmoid, you will play a crucial role in building a highly-scalable and extensible big data platform for collecting, storing, modeling, and analyzing massive data sets from various channels. Your primary responsibilities will include designing and optimizing scalable data pipelines, managing data lakehouse platforms, writing complex SQL queries, integrating structured and unstructured data sources, ensuring data governance and quality, and performance tuning of Spark jobs and SQL queries. You will collaborate closely with cross-functional teams to implement efficient data solutions in a cloud-based environment. Key Responsibilities: - Design, build, and ...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Apache Iceberg Developer at BairesDev We are seeking an Apache Iceberg Developer to work on distributed data table formats and metadata manageme...

Skill:-Apache Iceberg Exp- 3 -12 Yrs Job Description: Must Have Skills: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling Nice To Have Skills: Technical/Functional Skills Key Qualifications • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino • Experience with building/managing Lakehouse for large scale data platforms storage, computing aspects, data ingestion/governance and data management and performance. • Experience with distributed computing technology(s) like Spark. • Experience with data platforms, data lakes and ingestion systems that work at scale. • Skille...

Required Skills: Data Engineer, AWS, Spark, Apache Airflow, Kafka, Snowflake, DBT, Flink, CI / CD Pipelines, Apache Iceberg, Glue

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 4-6 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Job Description We are Omnissa! The world is evolving fast, and organizations everywherefrom corporations to schoolsare under immense pressure to provide flexible, work-from-anywhere solutions. They need IT infrastructure that empowers employees and customers to access applications from any device, on any cloud, all while maintaining top-tier security. That's where Omnissa comes in. The Omnissa Platform is the first AI-driven digital work platform that enables smart, seamless and secure work experiences from anywhere. It uniquely integrates multiple industry-leading solutions including Unified Endpoint Management, Virtual Apps and Desktops, Digital Employee Experience, and Security & Complia...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Staff Apache Impala Engineer at BairesDev We are seeking a Staff Apache Impala Engineer with deep expertise in SQL query engines, columnar stora...

Job Description Key Responsibilities . Implement frontend (React + TypeScript) and backend (TypeScript on Node.js Java/Spring Boot where needed) features with CI/CD discipline. . Rapidly provision and operate AWS components per sprint needs following leastprivilege and security best practices. . Ensure code quality: MR reviews, automated tests (unit/integration/E2E), green pipelines, release readiness. . Produce concise technical documentation and reference guides for enablement across teams. . Collaborate with architects and product stakeholders refine requirements support cutovers. MustHave Skills: . AWS (strong, handson): quickly spin up services (IAM, VPC/SecGroups, KMS, CloudWatch, Secr...

Role Overview: As the Team Lead of Data Engineering, you will be responsible for driving the design, development, and optimization of data engineering pipelines and services with a strong software engineering mindset. Your role will also involve fostering growth and collaboration within the team, mentoring junior colleagues, and refining processes to maintain excellence. If you are eager to have a lasting impact both technically and as a team leader, we are excited to meet you. Key Responsibilities: - Utilize very strong software engineering principles to write maintainable, scalable code, primarily in Python (experience with Rust or C# is a plus). - Perform data processing and manipulation ...

Role Overview: As a Technical Lead Engineer - Data at our company, you will be responsible for architecting, implementing, and scaling our end-to-end data platform built on AWS S3, Glue, Lake Formation, and DMS. Your role will involve leading a small team of engineers and collaborating cross-functionally with stakeholders from various departments to ensure reliable, timely, and secure data access across the business. Additionally, you will champion best practices in data design, governance, and observability, leveraging GenAI tools to enhance engineering productivity and accelerate time to insight. Key Responsibilities: - Own the design and scalability of the data lake architecture for both ...

As a Software Data Engineer at FTSE Russell, you will be part of a team dedicated to building scalable data systems with real-world impact. Your role will involve designing and developing testable data pipelines using Python and Apache Spark. You will orchestrate data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3. Applying modern software engineering practices will be a key part of your responsibilities, including version control, CI/CD, modular design, and automated testing. Additionally, you will contribute to the development of a lakehouse architecture using Apache Iceberg. Collaboration with business teams to translate requirements into data-driven solutions, buildin...

As an experienced Data Engineer at Regnology, you will play a crucial role in designing ingestion pipelines, optimizing query performance, and ensuring data quality, governance, and cost efficiency at scale. Your responsibilities will include: - **Migration Strategy & Execution**: - Design and implement data ingestion pipelines to extract data from Oracle into GCS/Iceberg. - Migrate and modernize existing Oracle schemas, partitions, and materialized views into Iceberg tables. - Define CDC (Change Data Capture) strategies using custom ETL. - **Data Lakehouse Architecture**: - Configure and optimize Trino clusters (coordinator/worker, Helm charts, autoscaling). - Design partitioning, compactio...

As a Senior Data Engineer at our company, you will play a crucial role in leading the development of a scalable data ingestion framework and ensuring high data quality and validation. Your responsibilities will include: - Architecting, developing, and maintaining end-to-end data ingestion framework for extracting, transforming, and loading data from diverse sources - Utilizing AWS services such as Glue, Lambda, EMR, ECS, EC2, and Step Functions to build scalable, resilient automated data pipelines - Developing and implementing automated data quality checks, validation routines, and error-handling mechanisms - Establishing comprehensive monitoring, logging, and alerting systems for data quali...