124 Apache Iceberg Jobs - Page 4

We are looking for a skilled Data Engineer with 8+ years of experience in big data technologies, particularly Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment. Key Responsibilities: Data Pipeline Development: Design, build, and optimize scalable data pipelines using Apache Spark. Implement and manage large-scale data processing solutions across data lakehouses. Data Lakehouse Management: Work with mode...

As a Senior Lead Engineer specializing in Python and Spark in AWS at LSEG, you will be responsible for designing, building, and maintaining robust, scalable, and efficient ETL pipelines. Your primary focus will be on ensuring alignment with data lakehouse architecture on AWS and optimizing workflows using services such as Glue, Lambda, and S3. Collaborating with cross-functional teams, you will gather requirements, provide technical insights, and deliver high-quality data solutions. Your role will involve driving the migration of existing data processing workflows to the lakehouse architecture, leveraging Iceberg capabilities, and enforcing best practices for coding standards and system arch...

Role Overview: As a Senior Data Warehouse Administrator at FloQast, you will play a crucial role in bolstering the expanding data infrastructure team. Your expertise in architecting, managing, and optimizing large-scale data warehousing environments is essential to ensure the efficiency and reliability of the data platform. You will be responsible for designing, implementing, and maintaining scalable and secure data warehouse environments, optimizing data warehouse performance, managing data ingestion pipelines, enforcing data governance and compliance policies, and mentoring junior data warehouse administrators. Key Responsibilities: - Design, implement, and maintain scalable and secure dat...

Required Skills: Data Engineer, AWS, Spark, Apache Airflow, Kafka, Snowflake, DBT, Flink, CI / CD Pipelines, Apache Iceberg, Glue

Role Overview: You will be responsible for leading and managing large-scale data integration initiatives across multiple cloud platforms as a Senior Data Integration Engineer. Your expertise in designing and implementing modern data pipelines, integrating heterogeneous data sources, and enabling scalable data storage solutions will be crucial. This role will involve working with Azure, AWS, GCP, cloud storage systems, and Apache Iceberg. Key Responsibilities: - Design, build, and manage data integration solutions across hybrid and multi-cloud environments. - Develop and optimize data pipelines to ensure scalability, performance, and reliability. - Implement and manage cloud storage solutions...

You are being sought after to take on the role of a Senior Data Engineer, where your primary responsibility will be leading the development of a scalable data ingestion framework. In this capacity, you will play a crucial role in ensuring high data quality and validation, as well as designing and implementing robust APIs for seamless data integration. Your expertise in building and managing big data pipelines using modern AWS-based technologies will be put to good use, making sure that quality and efficiency are at the forefront of data processing systems. Key Responsibilities: - **Data Ingestion Framework**: - Design & Development: You will architect, develop, and maintain an end-to-end dat...

We are looking for a highly motivated and enthusiastic Intermediate Software Developer to join our growing engineering team. This role is suitable for an individual with 3-5 years of experience who is eager to learn and excel within a fast-paced environment. You will have the opportunity to work on exciting projects involving large-scale data processing, analytics, and software development, utilizing technologies such as Java, Apache Spark, Python, and Apache Iceberg. This position provides a unique chance to gain hands-on experience with cutting-edge data lake technologies and play a vital role in enhancing critical data infrastructure. As an Intermediate Software Developer, your responsibi...

About Us At Cloudflare, we are on a mission to help build a better Internet. Today the company runs one of the worlds largest networks that powers millions of websites and other Internet properties for customers ranging from individual bloggers to SMBs to Fortune 500 companies. Cloudflare protects and accelerates any Internet application online without adding hardware, installing software, or changing a line of code. Internet properties powered by Cloudflare all have web traffic routed through its intelligent global network, which gets smarter with every request. As a result, they see significant improvement in performance and a decrease in spam and other attacks. Cloudflare was named to Ent...

Expedia Group brands power global travel for everyone, everywhere. We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success. Why Join Us To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and know that when one of us wins, we all win. We provide a full benefits package, including exciting travel perks, generous time-off, parental leave, a flexible work model (with some pretty cool offices), and career development ...

As a skilled engineer, you will play a crucial role in designing robust and scalable data platforms and microservices. Your primary responsibility will be ensuring that architectural and coding best practices are adhered to in order to deliver secure, reliable, and maintainable systems. You will actively engage in debugging complex system issues and work collaboratively with cross-functional teams to define project scope, goals, and deliverables. In this role, you will be tasked with managing priorities, effectively allocating resources, and ensuring the timely delivery of projects. Additionally, you will have the opportunity to lead, mentor, and cultivate a team of 10-15 engineers, fosterin...

As a Data Platform Engineering Manager at Traceable AI, you will be responsible for designing robust and scalable data platforms and microservices. It will be your duty to ensure that architectural and coding best practices are followed to deliver secure, reliable, and maintainable systems. You will actively participate in debugging complex system issues and work with cross-functional teams to define project scope, goals, and deliverables. Managing priorities, allocating resources effectively, and ensuring timely project delivery will be crucial aspects of your role. In this position, you will lead, mentor, and grow a team of 10-15 engineers, fostering a collaborative and high-performance cu...

Are you ready to take your engineering career to the next level As a Mid Lead Engineer, you will have the opportunity to contribute to building state-of-the-art data platforms in AWS by leveraging Python and Spark. Join a dynamic team that is focused on driving innovation and scalability for data solutions in a supportive and hybrid work environment. In this role, you will design, implement, and optimize ETL workflows using Python and Spark, playing a key part in building a robust data lakehouse architecture on AWS. Success in this position requires technical expertise, strong problem-solving skills, and the ability to collaborate effectively within an agile team. Your responsibilities will ...

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

Job Description Join our dynamic Digital Marketing Data Engineering team at Fanatics, where you&aposll play a critical role in shaping the big data ecosystem that powers our eCommerce and Digital Marketing platforms. As a full-time Staff Data Engineer, youll design, build, and optimize scalable data pipelines and architectures, ensuring seamless data flow and effective collection across cross-functional teams. You will also leverage your backend engineering skills to support API integrations and real-time data exchanges. What We&aposre Looking For BTech/MTech/BS/MS in Computer Science or a related field, or equivalent practical experience. 9+ years of software engineering experience, with a ...

About the Role We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data platforms that empower data-driven decision-making. This role requires deep technical expertise in modern data engineering frameworks, architectural patterns, and cloud-native solutions on AWS. You will be a key contributor to our data strategy, ensuring data quality, governance, and reliability while mentoring other engineers in the team. Key Responsibilities Design, develop, and own robust, scalable, and maintainable data pipelines (batch & real-time). Architect and implement Data Lake, Data Warehouse, and Lakehouse solutions using modern frameworks and architectural patterns. E...

About Role We are looking for an experienced Backend Developer with strong expertise in Python, FastAPI, Socket.IO, and Kubernetes to join our team in Bangalore. The ideal candidate has a proven track record of building and scaling production-grade systems handling 10k+ users, with deep knowledge of microservices, asynchronous programming, and distributed architectures. Responsibilities Design, develop, and maintain backend services and APIs using FastAPI and Python. Build and optimize real-time applications leveraging WebSockets (Socket.IO) and asyncio. Architect scalable microservices and ensure high availability in Kubernetes (K8s) environments. Implement message queues (Celery, RabbitMQ,...

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 7+ years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 46 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

The Platform Data Engineer will be responsible for designing and implementing robust data platform architectures, integrating diverse data technologies, and ensuring scalability, reliability, performance, and security across the platform. The role involves setting up and managing infrastructure for data pipelines, storage, and processing, developing internal tools to enhance platform usability, implementing monitoring and observability, collaborating with software engineering teams for seamless integration, and driving capacity planning and cost optimization initiatives.

When 5% of Indian households shop with us, it's important to build data-backed, resilient systems to manage millions of orders every day. We've done this - with zero downtime! ???? Sounds impossible Well, that's the kind of Engineering muscle that has helped Meesho become the e-commerce giant that it is today. We value speed over perfection, and see failures as opportunities to become better. We've taken steps to inculcate a strong Founder's Mindset across our engineering teams, making us grow and move fast. We place special emphasis on the continuous growth of each team member - and we do this with regular 1-1s and open communication. Tech Culture We have a unique tech culture where enginee...

As a Software Development Engineer at our company, you will have the opportunity to work in either Hyderabad, Telangana, India or Bengaluru, Karnataka, India. You should hold a Bachelor's degree in Computer Science, a related technical field, or have equivalent practical experience. With at least 8 years of experience in software development using general-purpose programming languages, you will be responsible for designing and building solutions for data migrations and developing software for Data Warehouse migration. Your expertise in programming languages such as Java, C/C++, Python, or Go will be crucial for this role. Additionally, experience in systems engineering with designing and bui...

We are looking for a highly skilled Scala Data Engineer to design, build, and optimize large-scale data platforms. The ideal candidate will have deep expertise in Scala, Spark, and SQL, with proven experience delivering scalable and high-performance data solutions in cloud-native environments. Key Responsibilities Design, build, and optimize scalable data pipelines using Apache Spark and Scala. Develop real-time streaming pipelines leveraging Kafka/Event Hubs. Own the design and architecture of data systems with a strong focus on performance, scalability, and reliability. Collaborate with cross-functional teams to deliver high-quality data products. Mentor junior engineers and enforce best p...

ROLE PROFILE: Are you ready to take your engineering career to the next level Join us as a Mid Lead Engineer and contribute to building state-of-the-art data platforms in AWS , leveraging Python and Spark . Be part of a dynamic team, driving innovation and scalability for data solutions in a supportive and hybrid work environment. ROLE SUMMARY: This role is ideal for an experienced data engineer looking to step into a leadership position while remaining hands-on with cutting-edge technologies. You will design, implement and optimize ETL workflows using Python and Spark, contributing to our robust data lakehouse architecture on AWS. Success in this role requires technical expertise, strong pr...