86 Apache Iceberg Jobs - Page 2

Job Description Become a substential part of the Data Platform team, which provides engineering services across the Media Measurement department. Design, develop, test and deploy big data applications and pipelines. Ensure that your and the teams work follow best practices for ensuring validity, efficiency and maintainability. Collaborate in a distributed, cross-functional agile team of Data Scientists and Cloud Engineers to reached shared goals. Take ownership of technical implementations and architecture together with your team and in alignment with communities of practices. Communicate with data producers & consumers to align on data exchange formats and interfaces. Provide suggestions to...

As a Senior Data Engineer specializing in Data Quality, Ingestion & API Development, your role will involve leading the development of a scalable data ingestion framework with a strong focus on ensuring high data quality and validation. You will also be responsible for designing and implementing robust APIs for seamless data integration. Your expertise in building and managing big data pipelines using modern AWS-based technologies will be crucial for driving quality and efficiency in data processing systems. - **Data Ingestion Framework:** - **Design & Development:** Architect, develop, and maintain an end-to-end data ingestion framework that efficiently extracts, transforms, and loads data ...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Senior Data Lake Engineer at BairesDev We are seeking a Senior Data Lake Engineer with expertise in data lake architecture, schema-on-read, data...

About Us MatchMove is a leading embedded finance platform that empowers businesses to embed financial services into their applications. We provide innovative solutions across payments, banking-as-a-service, and spend/send management, enabling our clients to drive growth and enhance customer experiences. Are You The One As a Technical Lead Engineer - Data, you will architect, implement, and scale our end-to-end data platform built on AWS S3, Glue, Lake Formation, and DMS. You will lead a small team of engineers while working cross-functionally with stakeholders from fraud, finance, product, and engineering to enable reliable, timely, and secure data access across the business. You will champi...

Job Title: Data Engineer Job Category: Full-Time Work Location: Hyderabad Qualifications: BE/B.Tech./MTech. /MCA/Degree in Computer Science Experience: 4+ years Key Responsibilities: Build and maintain the ETL pipeline using NiFi and Debezium. Implement data transformations and processing workflows. Configure Iceberg tables and MinIO storage. Optimize query performance with Trino. Required Skills: Proficiency in Apache NiFi, Debezium, and Kafka for data ingestion. Experience with Apache Iceberg and MinIO for data lakehouse setup. Strong SQL skills for data transformation and querying. Familiarity with Trino or Presto for distributed querying. Basic scripting skills (e.g., Python, Bash) for a...

Company Description It all started in sunny San Diego, California in 2004 when a visionary engineer, Fred Luddy, saw the potential to transform how we work. Fast forward to today ServiceNow stands as a global market leader, bringing innovative AI-enhanced technology to over 8,100 customers, including 85% of the Fortune 500. Our intelligent cloud-based platform seamlessly connects people, systems, and processes to empower organizations to find smarter, faster, and better ways to work. But this is just the beginning of our journey. Join us as we pursue our purpose to make the world work better for everyone. Job Description What you get to do in this role: Build high-quality, clean, scalable an...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Principal Data Lake Engineer at BairesDev We are seeking a Principal Data Lake Engineer with deep expertise in data lake architecture, schema-on...

Job Description: Manager Technology This position is a hands-on Engineering Leader for new Network Surveillance Management technology that supports AT&T's leading-edge networks and services, such as network virtualization and 5G, which require extensive surveillance and cross-domain correlation across multiple services. Evaluate and introduce strategic technology to enable Service Assurance architectural goals and transformation goals. This position requires technical leadership, innovation, and deep subject matter expertise in Network Service Assurance. Will be responsible for leading software development, data modeling, testing, deployment, and support of Network Service Assurance applicat...

As a Data Engineer at Omnissa, you will play a crucial role in delivering as an individual contributor. You should have a minimum of 2 to 8 years of experience in Data Engineering on AWS. Your responsibilities will include: - Strong Proficiency in SQL, Python, Apache Spark, AWS Data Stack (S3, EMR, Glue Catalog, Athena, Lambda, Airflow), Starburst / Trino. - Solid Understanding of Lakehouse Architecture - Apache Iceberg / Delta Lake. - Experience with CI/CD practices using GitHub Actions. - Ability to collaborate in cross-functional teams and communicate technical concepts clearly. - Mindset to design solutions for scale, proactively involve in additional areas and domains. - Experience with...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Data Lake Engineer at BairesDev We are seeking a Data Lake Engineer with expertise in data lake architecture, schema-on-read, data governance, a...

About The Team/Role As a Senior Staff SWE of Data Lake House Engineering, this technical leader will be responsible for the design and implementation of the Data Lake house platform (supporting both Data Engineering and Data Lake house applications), Data Engineering pipeline productionalization (E2E data pipelines, model development, deployment, monitoring, refresh, etc), Data Lake house, and related technology, systems, and tools. This technologist will drive the technology development and architecture in these areas to ensure the platforms, systems, tools, models, and services generated from this team are technically sound with strong security, quality, reliability, usability, scalability...

Job Description Alimentation Couche-Tard Inc., (ACT) is a global Fortune 200 company. A leader in the convenience store and fuel space with over 17,000 stores in 31 countries, serving more than 6 million customers each day It is an exciting time to be a part of the growing Data Engineering team at Circle K. We are driving a well-supported cloud-first strategy to unlock the power of data across the company and help teams to discover, value and act on insights from data across the globe. With our strong data pipeline, this position will play a key role partnering with our Technical Development stakeholders to enable analytics for long term success. About The Role We are looking for a Data Engi...

We are looking for a skilled Data Engineer with 8+ years of experience in big data technologies, particularly Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment. Key Responsibilities: Data Pipeline Development: Design, build, and optimize scalable data pipelines using Apache Spark. Implement and manage large-scale data processing solutions across data lakehouses. Data Lakehouse Management: Work with mode...

As a Senior Lead Engineer specializing in Python and Spark in AWS at LSEG, you will be responsible for designing, building, and maintaining robust, scalable, and efficient ETL pipelines. Your primary focus will be on ensuring alignment with data lakehouse architecture on AWS and optimizing workflows using services such as Glue, Lambda, and S3. Collaborating with cross-functional teams, you will gather requirements, provide technical insights, and deliver high-quality data solutions. Your role will involve driving the migration of existing data processing workflows to the lakehouse architecture, leveraging Iceberg capabilities, and enforcing best practices for coding standards and system arch...

Role Overview: As a Senior Data Warehouse Administrator at FloQast, you will play a crucial role in bolstering the expanding data infrastructure team. Your expertise in architecting, managing, and optimizing large-scale data warehousing environments is essential to ensure the efficiency and reliability of the data platform. You will be responsible for designing, implementing, and maintaining scalable and secure data warehouse environments, optimizing data warehouse performance, managing data ingestion pipelines, enforcing data governance and compliance policies, and mentoring junior data warehouse administrators. Key Responsibilities: - Design, implement, and maintain scalable and secure dat...

Required Skills: Data Engineer, AWS, Spark, Apache Airflow, Kafka, Snowflake, DBT, Flink, CI / CD Pipelines, Apache Iceberg, Glue

Role Overview: You will be responsible for leading and managing large-scale data integration initiatives across multiple cloud platforms as a Senior Data Integration Engineer. Your expertise in designing and implementing modern data pipelines, integrating heterogeneous data sources, and enabling scalable data storage solutions will be crucial. This role will involve working with Azure, AWS, GCP, cloud storage systems, and Apache Iceberg. Key Responsibilities: - Design, build, and manage data integration solutions across hybrid and multi-cloud environments. - Develop and optimize data pipelines to ensure scalability, performance, and reliability. - Implement and manage cloud storage solutions...

You are being sought after to take on the role of a Senior Data Engineer, where your primary responsibility will be leading the development of a scalable data ingestion framework. In this capacity, you will play a crucial role in ensuring high data quality and validation, as well as designing and implementing robust APIs for seamless data integration. Your expertise in building and managing big data pipelines using modern AWS-based technologies will be put to good use, making sure that quality and efficiency are at the forefront of data processing systems. Key Responsibilities: - **Data Ingestion Framework**: - Design & Development: You will architect, develop, and maintain an end-to-end dat...

We are looking for a highly motivated and enthusiastic Intermediate Software Developer to join our growing engineering team. This role is suitable for an individual with 3-5 years of experience who is eager to learn and excel within a fast-paced environment. You will have the opportunity to work on exciting projects involving large-scale data processing, analytics, and software development, utilizing technologies such as Java, Apache Spark, Python, and Apache Iceberg. This position provides a unique chance to gain hands-on experience with cutting-edge data lake technologies and play a vital role in enhancing critical data infrastructure. As an Intermediate Software Developer, your responsibi...

About Us At Cloudflare, we are on a mission to help build a better Internet. Today the company runs one of the worlds largest networks that powers millions of websites and other Internet properties for customers ranging from individual bloggers to SMBs to Fortune 500 companies. Cloudflare protects and accelerates any Internet application online without adding hardware, installing software, or changing a line of code. Internet properties powered by Cloudflare all have web traffic routed through its intelligent global network, which gets smarter with every request. As a result, they see significant improvement in performance and a decrease in spam and other attacks. Cloudflare was named to Ent...

Expedia Group brands power global travel for everyone, everywhere. We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success. Why Join Us To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and know that when one of us wins, we all win. We provide a full benefits package, including exciting travel perks, generous time-off, parental leave, a flexible work model (with some pretty cool offices), and career development ...

As a skilled engineer, you will play a crucial role in designing robust and scalable data platforms and microservices. Your primary responsibility will be ensuring that architectural and coding best practices are adhered to in order to deliver secure, reliable, and maintainable systems. You will actively engage in debugging complex system issues and work collaboratively with cross-functional teams to define project scope, goals, and deliverables. In this role, you will be tasked with managing priorities, effectively allocating resources, and ensuring the timely delivery of projects. Additionally, you will have the opportunity to lead, mentor, and cultivate a team of 10-15 engineers, fosterin...

As a Data Platform Engineering Manager at Traceable AI, you will be responsible for designing robust and scalable data platforms and microservices. It will be your duty to ensure that architectural and coding best practices are followed to deliver secure, reliable, and maintainable systems. You will actively participate in debugging complex system issues and work with cross-functional teams to define project scope, goals, and deliverables. Managing priorities, allocating resources effectively, and ensuring timely project delivery will be crucial aspects of your role. In this position, you will lead, mentor, and grow a team of 10-15 engineers, fostering a collaborative and high-performance cu...

Are you ready to take your engineering career to the next level As a Mid Lead Engineer, you will have the opportunity to contribute to building state-of-the-art data platforms in AWS by leveraging Python and Spark. Join a dynamic team that is focused on driving innovation and scalability for data solutions in a supportive and hybrid work environment. In this role, you will design, implement, and optimize ETL workflows using Python and Spark, playing a key part in building a robust data lakehouse architecture on AWS. Success in this position requires technical expertise, strong problem-solving skills, and the ability to collaborate effectively within an agile team. Your responsibilities will ...

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino