21920 Hadoop Jobs - Page 20

This role is for one of the Weekday's clients Salary range: Rs 500000 - Rs 800000 (ie INR 5-8 LPA) Min Experience: 3 years Location: Bengaluru JobType: full-time Requirements Data Engineering: Design, build, and maintain robust ETL/ELT pipelines for data integration and transformation. Develop and manage data lakes, data warehouses, and data marts to support analytics and reporting needs. Build efficient and scalable data models ensuring structured, high-quality datasets. Support both real-time and batch data processing (experience with Spark, Kafka, or Hadoop is a plus). Ensure data quality, governance, and security while optimizing performance across systems. Collaborate with technology an...

Role & responsibilities Job Title: Developer Skills Required: Digital: BigData and Hadoop Ecosystems, Digital: PySpark Experience Range in Required Skills: 4 to 6 Years Job Description: Big Data (PySpark, HIVE) 1. Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing. 2. Work on Hadoop, Hive SQLs, Spark, and Bigdata Eco System Tools. 3. Experience in working with teams in a complex organization involving multiple reporting lines.? 4. The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking terminologies. 5. The candidate should have strong DevOps and Agile Developme...

About Us: Soul AI is a pioneering company founded by IIT Bombay and IIM Ahmedabad alumni, with a strong founding team from IITs, NITs, and BITS. We specialize in delivering high-quality human-curated data, AI-first scaled operations services, and more. Based in SF and Hyderabad, we are a young, fast-moving team on a mission to build AI for Good, driving innovation and positive societal impact. About the Role We are looking for a Data Engineer to join us for critical projects. You will be working with big techs to help them in building cutting-edge AI foundational models meant to perform exceptionally well on engineering and infrastructure tasks. In this role, you will be part of a highly ski...

Senior Software Engineer (Full Stack) (AI) Role Overview As a Senior Software Engineer (Full Stack) (AI), you will play a key role in designing, developing, and deploying generative AI, machine learning models, and AI systems. You will work closely with data scientists, software engineers, and business stakeholders to transform data-driven insights into real-world applications. Responsibilities Design, develop, and optimize machine learning models for production environments. Build scalable AI systems, including data pipelines, model training, and deployment frameworks. Work with deep learning frameworks (TensorFlow, PyTorch) and traditional ML libraries (Scikit-learn, XGBoost). Collaborate ...

4 to 10 years of hands-on data engineering experience. •Strong expertise in Apache Spark (batch + streaming) and Hive. •Proficiency in Python, Scala, or Java. •Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred). •Experience working with Kafka, Solace, and object stores (S3, MinIO). •Exposure to Docker/Kubernetes for deployment. •Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi). Interested can share your resume to uthirakumar.k@tekskills.in

68 years of hands-on data engineering experience. •Strong expertise in Apache Spark (batch + streaming) and Hive. •Proficiency in Python, Scala, or Java. •Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred). •Experience working with Kafka, Solace, and object stores (S3, MinIO). •Exposure to Docker/Kubernetes for deployment. •Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi). 4 to 5.11 years of hands-on data engineering experience. •Strong expertise in Apache Spark (batch + streaming) and Hive. •Proficiency in Python, Scala, or Java. •Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frame...

Job Title: Data Engineer Job Location: Chennai, Tamil Nadu Job Type: Full-time Educational Qualifications: B. E or B. Tech (any discipline) / MCA or M.Sc. (CS) If you are interested, please send your CV and usual related documentation to the following email address: Kavitha.Hariharan@dsp.team Job Description: We are searching for our office in Chennai, India a Data Engineer with a university degree in computer science or mathematics and a proven working experience in requirement analysis and building and maintaining data systems (SQL or NO-SQL) of minimum 5 years. Experience gained during the development of the Master or PhD projects, and the-sis will be positively taken in account. Key Resp...

Location: Offshore(India) / Onshore(USA) Experience: 8+ years Employment Type: Full-time Overview We are seeking an experienced Data Modeler with a strong background in healthcare data environments. The ideal candidate will have deep expertise in IBM UDMH, data modeling techniques, and healthcare data standards, while working closely with business and technical stakeholders to ensure data structures align with organizational needs. Required Skills & Experience 8+ years of experience as a Data Modeler or Data Architect. Strong hands-on experience with IBM UDMH in a healthcare environment. Expertise in ER modeling tools and methodologies. Proficiency in SQL with solid understanding of relation...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, ...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, ...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, ...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, ...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, ...

Preferable - Tamilnadu Candidates Exp: 6+yrs NP: Imm Rounds: 2 Rounds (Virtual) Mandate Skills: Scala, Spark, Databricks Job Description The Role Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficient ETL processes from various sources and storage systems. Leading the implementation of algorithms and prototypes to transform raw data into useful information. Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations. Creating innovative data validation methods and data analysis tools. Ensuring compliance with data g...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing Collaborating closely with diverse teams, you'll play an important role in deciding the mos...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing Collaborating closely with diverse teams, you'll play an important role in deciding the mos...

Job Title: Junior Software Engineer (Java / Big Data) Location: Pune, India (Onsite) Experience: 1–3 Years (Immediate Joiners Preferred) About the Role We are seeking a passionate and driven Junior Software Engineer to join our dynamic team in Pune. The ideal candidate should have hands-on experience with Java and Big Data technologies , excellent problem-solving skills, and a strong desire to learn and grow in a fast-paced technology environment. Key Responsibilities Develop, test, and maintain scalable software applications using Java and Big Data frameworks . Work closely with senior engineers to design and implement data processing pipelines and backend systems. Participate in code revie...

We are Hiring - Data Engineer Hyderabad ! Job Title: Data Engineer Location: Hyderabad Experience: 5+Years Notice : Immediate Joiners Job Summary: We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects. The ideal candidate should possess strong expertise in SQL, Python, data modeling, data warehousing, and ETL pipeline development. Experience with big data tools like Hadoop and Spark, along with AWS services such as Redshift, S3, Glue, EMR, and Lambda, is essential. This role provides an excellent opportunity to work on large-scale data solutions, enabling data-driven decision-making and operational excellenc...

Roles & Responsibilities Key Responsibilities Analyze existing Hadoop, Pig, and Spark scripts from Dataproc and refactor them into Databricks-native PySpark. Implement data ingestion and transformation pipelines using Delta Lake best practices. Apply conversion rules and templates for automated code migration and testing. Conduct data validation between legacy and migrated environments (schema, count, and data-level checks). Collaborate on developing AI-driven tools for code conversion, dependency extraction, and error remediation. Ensure best practices for code versioning, error handling, and performance optimization. Participate in UAT, troubleshooting, and post-migration validation activi...

Roles & Responsibilities Key Responsibilities Lead design and execution of Dataproc ? Databricks PySpark migration roadmap. Define modernization strategy, including data ingestion, transformation, orchestration, and governance. Architect scalable Delta Lake and Unity Catalog–based solutions. Manage and guide teams on code conversion, dependency mapping, and data validation. Collaborate with platform, infra, and DevOps teams to optimize compute costs and performance. Own the automation & GenAI acceleration layer, integrating code parsers, lineage tools, and validation utilities. Conduct performance benchmarking, cost optimization, and platform tuning (Photon, Auto-scaling, Delta Caching). Men...

Job Description: Xiaomi India is seeking a highly skilled Data Engineer to join our dynamic team. As a Data Engineer, you will be responsible for designing, implementing, and maintaining our data infrastructure to support our rapidly growing business needs. The ideal candidate will have expertise in Apache Iceberg, Apache Hive, Apache Hadoop, SparkSQL, YARN, HDFS, MySQL, Data Modeling, Data Warehousing, Spark Architecture, and SQL Query Optimization. Experience with Apache Flink, PySpark, Automated Data Quality testing & Data Migration is considered a plus. Also, it's mandatory to know any one cloud stack (AWS or Azure) for Data Engineering to Create Data Jobs and Workflows and Scheduler it ...

You are seeking a Spark, Big Data - ETL Tech Lead for Commercial Cards Global Data Repository development team at Citigroup. You will be interacting with various teams within Citigroup and need to have exceptional communication skills across technology and business areas. As a technical lead, you will design and implement large-scale data processing pipelines using Apache Spark on the BigData Hadoop Platform. Your responsibilities will include developing and optimizing Spark applications, providing technical leadership for global software solutions, integrating data from various sources, and managing development scope, budgets, and timelines. You will also mentor and guide junior developers ...

Role Overview: As a Production Specialist at Wipro Limited, your role is to support process delivery by ensuring daily performance of the Production Specialists, resolving technical escalations, and developing technical capability within the team. You will be responsible for overseeing and supporting the process, reviewing daily transactions on performance parameters, and supporting the team in improving performance parameters by providing technical support and process guidance. Additionally, you will handle technical escalations through effective diagnosis and troubleshooting of client queries, manage and resolve technical roadblocks/escalations, and provide product support and resolution t...

As a Big Data Developer at Unison Consulting Pte Ltd, you will play a crucial role in designing, developing, and maintaining big data solutions to enhance data-driven decision making and analytics capabilities for our clients. You will leverage cutting-edge technologies and platforms to work with large volumes of structured and unstructured data. Responsibilities: - Design, develop, and maintain big data solutions utilizing technologies like Hadoop, HDFS, Spark, and Hive. - Ingest, process, and analyze substantial amounts of structured and unstructured data from various sources. - Develop and optimize ETL processes to facilitate efficient data extraction, transformation, and loading. - Imple...

Job Requirements Phenom Intro : Our purpose is to help a billion people find the right work! Phenom is an AI-Powered talent experience platform that is redefining the HR tech space. We have grown into a global organization with offices in 6 countries and over 1,500 employees. As an HR tech unicorn organization, innovation and creativity is within our DNA. Come help us make every talent moment Phenomenal! What You’ll Do Design, develop, and implement machine learning algorithms and models to improve our products and services Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions Conduct research and stay updated with the latest ...