12 Apache Hudi Jobs

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Principal Data Lake Engineer at BairesDev We are seeking a Principal Data Lake Engineer with deep expertise in data lake architecture, schema-on...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Data Lake Engineer at BairesDev We are seeking a Data Lake Engineer with expertise in data lake architecture, schema-on-read, data governance, a...

Description Introduction : We are on a mission to rid the world of bad customer service by mobilizing the way help is delivered. Today's consumers want an always-available customer service experience that leaves them feeling valued and respected. Helpshift helps B2B brands deliver this modern customer service experience through a mobile-first approach. We have changed how conversations take place, moving the conversation away from slow, outdated email and desktop experience to an in-app chat experience that allows users to interact with brands in their time. Through our market leading chatbots and automations, we help brands deliver instant and rapid resolutions. Because agents play a key ro...

About The Team/Role As a Senior Staff SWE of Data Lake House Engineering, this technical leader will be responsible for the design and implementation of the Data Lake house platform (supporting both Data Engineering and Data Lake house applications), Data Engineering pipeline productionalization (E2E data pipelines, model development, deployment, monitoring, refresh, etc), Data Lake house, and related technology, systems, and tools. This technologist will drive the technology development and architecture in these areas to ensure the platforms, systems, tools, models, and services generated from this team are technically sound with strong security, quality, reliability, usability, scalability...

Skills Required: Familiarity with data processing engines such as Apache Spark, Flink, or other big data tools. Design, develop, and implement robust data lake architectures on cloud platforms (AWS/Azure). Implement streaming and batch data pipelines using Apache Hudi, Apache Hive, and cloud-native services like AWS Glue, Azure Data Lake, etc. Architect and optimize ingestion, compaction, partitioning, and indexing strategies in Apache Hudi. Develop scalable data transformation and ETL frameworks using Python, Spark, and Flink. Work closely with DataOps/DevOps to build CI/CD pipelines and monitoring tools for data lake platforms. Ensure data governance, schema evolution handling, lineage tra...

Roles and Responsibilities : Design, build, and optimize scalable ETL pipelines using Apache Airflow or similar frameworks to process and transform large datasets efficiently. Utilize Spark (PySpark), Kafka, Flink, or similar tools to enable distributed data processing and real-time streaming solutions. Deploy, manage, and optimize data infrastructure on cloud platforms such as AWS, GCP, or Azure, ensuring security, scalability, and cost-effectiveness. Design and implement robust data models, ensuring data consistency, integrity, and performance across warehouses and lakes. Enhance query performance through indexing, partitioning, and tuning techniques for large-scale datasets. Manage cloud-...

AWS Storage & Compute Services S3: Strong understanding of bucket policies, lifecycle rules & data organization Glue: ETL jobs, crawlers & data cataloging Athena: Querying data directly from S3 using SQL EMR: Big data processing with Spark/Hadoop Lakehouse Technologies Apache Hudi: Hands-on knowledge of transactional data lakes & schema evolution Data Cataloging & Governance AWS Glue Data Catalog Integration with Lake Formation Security & Compliance IAM roles & policies Encryption strategies Data masking & tokenization Locations : Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune,Remote

As a Senior Staff Software Engineer in Data Lake House Engineering, you will play a crucial role in designing and implementing the Data Lake house platform, supporting both Data Engineering and Data Lake house applications. Your responsibilities will include overseeing Data Engineering pipeline productionalization, end-to-end data pipelines, model development, deployment, monitoring, refresh, etc. Additionally, you will be involved in driving technology development and architecture to ensure the platforms, systems, tools, models, and services meet the technical standards for security, quality, reliability, usability, scalability, performance, efficiency, and operability to meet the evolving ...

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines with a strong focus on time series forecasting and upsert-ready architectures. This role requires end-to-end ownership of the data lifecycle, from ingestion to partitioning, versioning, and BI delivery. The ideal candidate must be highly proficient in AWS data services, PySpark, versioned storage formats like Apache Hudi/Iceberg, and must understand the nuances of data quality and observability in large-scale analytics systems. Role & responsibilities Design and implement data lake zoning (Raw Clean Modeled) using Amazon S3, AWS Glue, and Athena. Ingest structured and uns...

Job Title: Senior Data Engineer (4-6 Years Experience) Location: Kotak Life HO Department: Data Science & Analytics Employment Type: Full-Time About the Role: We are seeking a highly skilled Data Engineer with 4-6 years of hands-on experience in designing and developing scalable, reliable, and efficient data solutions. The ideal candidate will have a strong background in cloud platforms (AWS or Azure), experience in building both batch and streaming data pipelines, and familiarity with modern data architectures including event-driven and medallion architectures. Key Responsibilities: .Design, build, and maintain scalable data pipelines (batch and streaming) to process structured and unstruct...

Your day at NTT DATA We are seeking an experienced Data Engineer to join our team in delivering cutting-edge Generative AI (GenAI) solutions to clients. The successful candidate will be responsible for designing, developing, and deploying data pipelines and architectures that support the training, fine-tuning, and deployment of LLMs for various industries. This role requires strong technical expertise in data engineering, problem-solving skills, and the ability to work effectively with clients and internal teams. What youll be doing Key Responsibilities: Design, develop, and manage data pipelines and architectures to support GenAI model training, fine-tuning, and deployment Data Ingestion an...

Data Engineer Experience: 3 - 5 Years Exp Salary : Upto INR 35 Lacs per annum Preferred Notice Period : Within 30 Days Shift : 10:30AM to 7:30PM IST Opportunity Type: Remote Placement Type: Permanent (*Note: This is a requirement for one of Uplers' Clients) Must have skills required : Apache Airflow, Spark, AWS, Kafka, SQL Good to have skills : Apache Hudi, Flink, Iceberg, Azure, GCP Nomupay (One of Uplers' Clients) is Looking for: Data Engineer who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you. Role Overview De...