15749 Hadoop Jobs - Page 10

6+ years of experience in Data engineering projects using COSMOS DB- Azure Databricks (Min 3-5 projects) Strong expertise in building data engineering solutions using Azure Databricks, Cosmos DB Strong T-SQL programming skills or with any other flavor of SQL Experience working with high volume data, large objects, complex data transformations Experience working in DevOps environments integrated with GIT for version control and CI/CD pipeline. Good understanding of data modelling for data warehouse and data marts Strong verbal and written communication skills Ability to learn, contribute and grow in a fast phased environment Nice to have: Expertise in Microsoft Azure is mandatory including co...

Experience : 12- 18 Yrs of experience as a Data Architect in complex, large-scale data environments. SSIS, Databricks Expertise : Strong hands-on experience working with SSIS, Databricks, including using Apache Spark for data processing and optimization. ETL Tools : Proficient with various ETL tools and frameworks such as Informatica, Talend, or SSIS or DataBricks Big Data Technologies: In-depth knowledge of big data processing frameworks like Spark, Flink, Hadoop, Kafka, etc. NoSQL/MongoDB: Expertise in working with NoSQL databases, especially MongoDB, for large-scale data storage, retrieval, and optimization. Gen AI/AIML Models : Familiarity with data consumption. Programming Skills: Profi...

Responsibilities Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis Collaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions Ensure data quality and integrity through effective validation, monitoring, and troubleshooting techniques Optimize data processing workflows for maximum performance and efficiency Stay up-to-date with evolving Big Data technologies and methodologies to enhance existing systems Implement best practices for dat...

RiskInsight Consulting Pvt Ltd is looking for a Data Analyst with expertise in SQL and HDFS (Hadoop Distributed File System) to join our dynamic team. The selected candidate will play a crucial role in analyzing large datasets and providing actionable insights to drive business decisions. If you are a detail-oriented individual with a passion for data, we want to hear from you! Design, implement, and maintain data pipelines and workflows using HDFS for data management Conduct data extraction, transformation, and loading (ETL) processes using SQL and Hive Perform rigorous data analysis to identify trends, inconsistencies, and insights for stakeholders Collaborate with business units to unders...

We are looking for a skilled Data Engineer with 7 to 10 years of experience to join our team in Bangalore, Hyderabad, Chennai, and Pune. The ideal candidate will have expertise in data engineering, Azure cloud, Snowflake, and Snowflake ELT methodologies. Roles and Responsibility Design and develop scalable data pipelines using Azure cloud and Snowflake. Implement ETL processes on Azure and manage data management tasks. Develop automation scripts using Python, Scala, or other languages. Collaborate with cross-functional teams to ensure seamless data integration. Ensure data integrity and accuracy during migration efforts. Optimize data processing workflows for improved efficiency. Job Require...

We are looking for a skilled Hadoop Admin with 7 to 12 years of experience, located in Hyderabad, Bangalore, Chennai, and Bhubaneshwar. The ideal candidate will have hands-on experience in building EKS clusters, Spark on EKS, AWS (EMR, S3, RDS), Python as programming skills, Shell scripting, and any CI/CD. Roles and Responsibility Manage and maintain large-scale Hadoop clusters for high availability and performance. Design and implement scalable data processing pipelines using Spark and Airflow instances. Integrate Ranger and Hive Metastore services in EKS for secure data storage and access. Collaborate with cross-functional teams to identify and resolve technical issues. Develop and impleme...

We are looking for a skilled professional with 5-10 years of experience to join our team as a GCP expert. The ideal candidate will have a strong background in BigQuery, GCP, and coding languages like Java, Python, and Hadoop. This position is available in Bangalore, Hyderabad, Chennai, Pune, Noida, and Gurgaon. Roles and Responsibility Design and develop scalable and efficient data pipelines using BigQuery and GCP. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data architectures using Cloud SQL and Apache Airflow. Ensure high-quality data processing and analysis by implementing robust testing and validation framework...

We are looking for a skilled professional with 5-10 years of experience to join our team in Bangalore, Hyderabad, Chennai, Pune, Noida, and Gurgaon. The ideal candidate will have expertise in GCP Hadoop ecosystem and be able to handle batch data processing on the Hadoop ecosystem. Roles and Responsibility Design and develop scalable data pipelines using GCP Hadoop ecosystem. Process large datasets using Scala Spark, Hive, and other relevant tools. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain technical documentation for data processing systems. Troubleshoot and resolve issues related to data processing and system performance. Op...

We are looking for a skilled Data Engineer with expertise in Integration and Data Processing to join our team. The ideal candidate will have hands-on experience with Spark and Python as programming skills, along with advanced SQL programming skills. This position is available in Bengaluru, Hyderabad, Pune, or Gurgaon. Roles and Responsibility Design and develop data integration solutions using various tools and technologies. Process and analyze large datasets to extract insights and trends. Develop and maintain complex databases and data warehouses. Collaborate with cross-functional teams to identify and prioritize project requirements. Ensure data quality and integrity by implementing data ...

We are looking for a skilled Hadoop professional with 7-14 years of experience to join our team in Pune and Hyderabad. The ideal candidate will have expertise in Hadoop, Spark, Kafka, Data Visualization, scripting languages in Python and R, SQL and NoSQL, Agile methodology, ETL / ELT data pipelines. Roles and Responsibility Design and develop scalable Hadoop applications using Spark and Kafka. Create data visualizations using tools like Splunk, Qlik, and Grafana. Develop and maintain ETL/ELT data pipelines using Agile methodology. Collaborate with cross-functional teams to identify and prioritize project requirements. Troubleshoot and resolve technical issues related to Hadoop applications. ...

We are looking for skilled Hadoop Developers with 5-10 years of experience to join our team in Bangalore, Kolkata, Hyderabad, and Pune. The ideal candidate should have strong proficiency in Hadoop, Scala, Spark, and SQL. Roles and Responsibility Design, develop, and implement scalable data processing systems using Hadoop. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines using Spark and Scala. Troubleshoot and resolve complex technical issues related to Hadoop. Participate in code reviews and ensure high-quality code standards. Stay updated with the latest trends and technologies in Hadoop development. Job...

Join our data team and play a crucial role in powering the next generation of property technology. You'll design and build robust data pipelines that transform how estate agents and property professionals make decisions, using both Microsoft's leading-edge data stack and our innovative in-house C# solutions. Working closely with our Data Analysts, you'll ensure clean, structured data flows seamlessly into Power BI dashboards that drive business growth for thousands of property professionals worldwide. Essential Requirements Create highly scalable Data Dashboard solution that can collect, store,process & analyse sets of data from a wide variety of sources Be responsible for setting up Data In...

We are looking for a skilled GCP Data Architect with 11-16 years of experience to join our team in Bangalore, Pune, Chennai, Hyderabad, and Noida. The ideal candidate will have expertise in designing databases, data models, and data flow on the Google Cloud platform. Roles and Responsibility Design and implement scalable data architectures on the Google Cloud platform. Develop and maintain large-scale databases using Cloud SQL and BigQuery. Create data models and data flows to support business intelligence and analytics. Collaborate with cross-functional teams to identify and prioritize project requirements. Ensure data quality and integrity through data validation and testing procedures. Im...

We are looking for a skilled professional with 7-11 years of experience to join our team as an Azure Snowflake specialist in Chennai, Bangalore, and Hyderabad. Roles and Responsibility Design and implement data models using Snowflake on Azure. Develop and maintain ETL processes for data migration and integration. Collaborate with cross-functional teams to ensure seamless data flow and analysis. Troubleshoot and resolve technical issues related to Azure Snowflake. Optimize database performance and ensure data quality and integrity. Implement cloud-based computing environments using Azure. Job Requirements Strong knowledge of SQL and ETL processes. Experience with data modeling and cloud compu...

We are looking for a skilled Data Engineer with 4 to 7 years of experience to join our team in Bangalore. The ideal candidate will have expertise in Python, SQL, AWS, Spark, and Big Data processing frameworks. Roles and Responsibility Design and develop scalable data pipelines using Python, Scala, or Java. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems using Hadoop and Big Data processing frameworks. Ensure data quality and integrity by implementing robust testing and validation procedures. Optimize data storage and retrieval processes for improved performance and efficiency. Participate in code reviews t...

Responsibilities Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis Collaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions Ensure data quality and integrity through effective validation, monitoring, and troubleshooting techniques Optimize data processing workflows for maximum performance and efficiency Stay up-to-date with evolving Big Data technologies and methodologies to enhance existing systems Implement best practices for dat...

Position: Azure Data QA (DT40SI RM 3582) Hybrid Working Azure Data QA: Must Have Azure Data Bricks, Azure Data Factory, Spark SQL knowledge Years 4-5 years of testing experience in Azure Data Bricks Strong experience in SQL along with performing Azure Data bricks Quality Assurance. Understand complex data system by working closely with engineering and product teams Develop scalable and maintainable applications to extract, transform, and load data in various formats to SQL Server, Hadoop Data Lake or other data storage locations. Job Category: Digital_Cloud_Web Technologies Job Type: Full Time Job Location: Bangalore Pune Experience: 4+ years Notice period: Immediate

Job Description Position Title: Head – Data Lake, AI & RPA Department: Enterprise IT Location: Mumbai Reports To: CIO, Enterprise IT Role Overview Lead the strategic development, implementation, and management of the Data Lake, Artificial Intelligence (AI), and Robotic Process Automation (RPA) capabilities. Ensure robust, scalable data architecture and intelligent automation solutions to drive data-driven decision-making and operational efficiency across the organization. The role requires balancing strategic vision with execution excellence, ensuring strong governance, ROI measurement, vendor management, and business alignment. Key Responsibilities Strategic Leadership: Define and execute t...

We are looking for a skilled professional with expertise in Spark and Scala to join our team. The ideal candidate will have 4 to 10 years of experience. Roles and Responsibility Design, develop, and implement scalable data processing pipelines using Spark and Scala. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems using Big Data Technologies such as Hadoop, Hive, and HBase. Troubleshoot and optimize performance issues in existing data systems. Participate in code reviews and ensure adherence to coding standards. Stay updated with industry trends and emerging technologies in the field of Spark and Scala. Job...

We are looking for a skilled Data Engineer with 4 to 10 years of experience to join our team in Bangalore. The ideal candidate will have expertise in SQL, ETL, data modeling, and programming languages such as Python, C++, C#, Scala, etc. Roles and Responsibility Design, implement, and maintain large-scale data pipelines using ETL tools like Presto, Spark, and MySQL. Develop and optimize data models to support business intelligence and data science applications. Create data visualizations using Power BI and Tableau to provide insights to stakeholders. Collaborate with cross-functional teams to identify and prioritize project requirements. Ensure data quality and integrity by implementing robu...

At Index Exchange, we’re reinventing how digital advertising works—at scale. As a global advertising supply-side platform, we empower the world’s leading media owners and marketers to thrive in a programmatic, privacy-first ecosystem. We’re a proud industry pioneer with over 20 years of experience accelerating the ad technology evolution. Our proprietary tech is trusted by some of the world’s largest brands and media owners and plays a crucial role in keeping the internet open, accessible, and largely free. We process more than 550 billion real-time auctions every day (in comparison, Google processes 8.5 billion searches per day) with ultra-low latency. Our platform is vertically integrated ...

We are looking for a skilled Java Architect with 12 to 18 years of experience, located in Bangalore. The ideal candidate should have expertise in coding, design, and architecture using Java with Springboot/Spring, Kafka, Cloud Native, Apache Pinot, Apache Hadoop Ecosystem, Apache Spark Java, Apache Hive, Hive Query, REST/RESTCONF, Database: Redis/Pinot/Hadoop, and debugging skills. Roles and Responsibility Design and develop scalable and efficient software systems using Java with Springboot/Spring. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain high-quality code that meets industry standards and best practices. Troubleshoot and d...

We are looking for a skilled Data Engineer with 5-9 years of experience to join our team in Kolkata, Vizag, and Bhubaneswar. The ideal candidate will have expertise in Databricks, Azure Data Factory, and Python. Roles and Responsibility Design and develop data pipelines using Databricks and Azure Data Factory. Implement structured and unstructured data solutions. Develop data migration strategies and data warehousing solutions. Process and analyze large datasets using Hadoop, Java, Spark, Scala, and Hive. Work with NoSQL data stores and ensure data quality and integrity. Collaborate with cross-functional teams to identify and prioritize project requirements. Job Requirements Strong skills in...

Role & responsibilities A DevOps Engineer (DOE) is responsible for deploying and managing Through automation, security maintenance, and troubleshooting, the DOE works with relevant stakeholders to create maintenance-free environments. As a DOE, you will be joining a team of enthusiastic engineers set out to deliver secure infrastructure that directly impacts core-business outcomes. You will develop code for automated deployment and configuration, and manage deployments by maintaining configurations and upgrading software. Environments vary from on-site Hadoop/Spark Clusters to Amazon AWS deployments (EC2, S3, EMR, VPC, Lambda) and Google Cloud deployments (CloudStorage, Dataproc, VPC). In th...

About the Role We are seeking a highly skilled Lead Data Engineer to define and drive the data migration strategy from legacy RDBMS platforms to PostgreSQL for a mission-critical billing and invoicing system. This role requires a hands-on technical leader who can design the migration architecture, oversee implementation, and ensure seamless transition without disruption to core business functions. The ideal candidate will have deep expertise in data migration frameworks, large-scale distributed processing (Spark preferred), and experience with both open-source and AWS ecosystem tools. They will work closely with business stakeholders and customers to manage expectations, deliver with precisi...