582 Hdfs Jobs - Page 17

As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your role will involve building data ingestion and transformation frameworks for structured and unstructured data sources. Collaborating with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions will be a key aspect of your responsibilities. Additionally, you will work with large volumes of data to ensure quality, integrity, and consistency, optimizing data workflows for performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. Implementin...

As a Software Engineer II at JPMorgan Chase within the Employee Platforms team, you will be part of an agile team responsible for designing and delivering trusted market-leading technology products in a secure, stable, and scalable manner to support the firm's business objectives. Your role involves executing creative software solutions, developing high-quality production code, and identifying opportunities to automate remediation of recurring issues to enhance operational stability. You will lead evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing...

Key Skills: Hadoop, Cloudera, HDFS, YARN, Spark, Delta Lake, Linux, Docker, Kubernetes, Jenkins, REST API, Prometheus, Grafana, Splunk, PySpark, Python, Terraform, Ansible, GCP, DevOps, CI/CD, SRE, Agile, Infrastructure Automation Roles & Responsibilities: Lead and support technology teams in designing, developing, and managing data engineering and CI/CD pipelines, and infrastructure. Act as an Infrastructure/DevOps SME in designing and implementing solutions for risk analytics systems transformation, both tactical and strategic, aligned with regulatory and business initiatives. Collaborate with other technology teams, IT support teams, and architects to drive improvements in product deliver...

5 years of experience in Bigdata, Hive Experience in Python(Flask / FastAPI), Pyspark Experience in MongoDB, Postgres Extensive knowledge in SQL Writing. Good Understanding of Performance Tuning, Security aspects. Very good Analytical skills. Very good knowledge and hands-on experience in Bigdata, Hive. Very good knowledge and hands-on experience in Python(Flask / FastAPI), Pyspar Good understanding of Functional application development Unit Testing: Should be efficient in writing Unit Test cases [positive and negative scenarios] and executing them Experience in working in Agile environment

Principal responsibilities Primary Skills - HDP, CDP, Linux, Python, Ansible and Kubernetes Proficient in Shell scripting, YAML Good Technical Design, Problem Solving and debugging skills. Understanding of how to use GitHub, Jenkins, Ansible etc. Understanding of CI/CD concept. Familiar with Test automation tools JUnit, Selenium, Cucumber Hands on development solutions using industry leading Cloud technologies. Working knowledge of Git Ops and DevSecOps Good To Have Skills: Linux, Kubernetes, Ansible, Spring Web Services, Micro Services Agile proficient and knowledgeable/experience in other agile methodologies. Ideally certified. Strong communication networking skills Ability to work autonom...

Responsibilities of the Candidate : - Be responsible for the design and development of big data solutions. Partner with domain experts, product managers, analysts, and data scientists to develop Big Data pipelines in Hadoop - Be responsible for moving all legacy workloads to a cloud platform - Work with data scientists to build Client pipelines using heterogeneous sources and provide engineering services for data PySpark science applications - Ensure automation through CI/CD across platforms both in cloud and on-premises - Define needs around maintainability, testability, performance, security, quality, and usability for the data platform - Drive implementation, consistent patterns, reusable...

We are seeking a PySpark Developer with IT experience . The ideal candidate will possess strong PySpark knowledge and hands-on experience in SQL, HDFS, Hive, Spark, PySpark, and Python . You will be instrumental in developing and optimizing data pipelines, working with large datasets, and implementing data processing solutions, particularly within the Azure Databricks environment. Key Responsibilities PySpark Development : Design, develop, and maintain robust and scalable data processing solutions using PySpark and Python . Data Lake & Warehousing : Work with large datasets stored in HDFS and Hive , applying concepts of Partitions and Bucketing for optimized data storage and retrieval. SQL &...

No of years experience 5+ Detailed job description - Skill Set: Bigdata Testing - Hadoop, HDFS, Hive, Kafka, Spark, SQL UNIX Mandatory Skills Bigdata Testing - Hadoop, HDFS, Hive, Kafka, Spark, SQL UNIX Good to Have Skills Bigdata Testing - Hadoop, HDFS, Hive, Kafka, Spark, SQL UNI

As an Associate Product Support Engineer focused on Hadoop distributed systems, you will be responsible for providing technical support to enterprise clients. Your main tasks will involve troubleshooting and resolving issues within Hadoop environments, ensuring the stability and reliability of our customers" infrastructures. Working closely with experienced engineers, you will collaborate with customers to understand their problems and deliver effective solutions with empathy and professionalism. Your key responsibilities will include addressing customer issues related to Hadoop clusters, core components (HDFS, YARN, MapReduce, Hive, etc.), and performing basic administrative tasks such as i...

About Company Founded in 2011, ReNew, is one of the largest renewable energy companies globally, with a leadership position in India. Listed on Nasdaq under the ticker RNW, ReNew develops, builds, owns, and operates utility-scale wind energy projects, utility-scale solar energy projects, utility-scale firm power projects, and distributed solar energy projects. In addition to being a major independent power producer in India, ReNew is evolving to become an end-to-end decarbonization partner providing solutions in a just and inclusive manner in the areas of clean energy, green hydrogen, value-added energy offerings through digitalisation, storage, and carbon markets that increasingly are integ...

Experience with ETL processes and data warehousing Proficient in SQL and Python/Java/Scala Team Lead Experience

Responsibilities: Develop and maintain data processing workflows using Apache Spark and Scala Implement batch and streaming data pipelines Optimize Spark jobs for better performance and scalability Collaborate with data engineers and analysts to deliver data solutions Debug and resolve issues in production big data environments Integrate with data storage systems like HDFS, Kafka, and NoSQL databases Write clean, maintainable code with best practices in mind Required Skills: Strong programming skills in Scala and hands-on experience with Apache Spark Knowledge of Spark Core, Spark SQL, Spark Streaming, and MLlib Experience with Hadoop ecosystem components (HDFS, Hive, Kafka) Familiarity with...

So, what’s the role all about? We are looking for a highly driven and technically skilled Software Engineer to lead the integration of various Content Management Systems with AWS Knowledge Hub, enabling advanced Retrieval-Augmented Generation (RAG) search across heterogeneous customer data—without requiring data duplication. This role will also be responsible for expanding the scope of Knowledge Hub to support non-traditional knowledge items and enhance customer self-service capabilities. You will work at the intersection of AI, search infrastructure, and developer experience to make enterprise knowledge instantly accessible, actionable, and AI-ready. How will you make an impact? Integrate C...

The Applications Development Intermediate Programmer Analyst position at our organization entails actively participating in the establishment and execution of new or modified application systems and programs in collaboration with the Technology team. Your primary goal in this role is to contribute to applications systems analysis and programming activities. Your responsibilities will include utilizing your knowledge of applications development procedures and concepts, as well as basic understanding of other technical areas, to identify and define necessary system enhancements. This will involve using script tools, analyzing code, and interpreting code. You will also be expected to consult wi...

Hi, Wishes from GSN!!! Pleasure connecting with you!!! We been into Corporate Search Services for Identifying & Bringing in Stellar Talented Professionals for our reputed IT / Non-IT clients in India. We have been successfully providing results to various potential needs of our clients for the last 20 years. At present, GSN is hiring DATA ENGINEERING - Solution Architect for one of our leading MNC client. PFB the details for your better understanding : 1. WORK LOCATION : PUNE 2. Job Role: DATA ENGINEERING - Solution Architect 3. EXPERIENCE : 13+ yrs 4. CTC Range: Rs. 35 LPA to Rs. 50 LPA 5. Work Type : WFO Hybrid ****** Looking for SHORT JOINERS ****** Job Description : Who are we looking fo...

Mandatory Skill Set Big Data developer Interested can share your updated resume to karthigaa.chinnasamy@aspiresys.com Thanks & Regards Karthigaa Chinnasamy | HR - Talent Acquisition Mobile: +91- 9092938886 Website: www.aspiresys.com | Blog: http://blog.aspiresys.com

Position Summary MetLife established a Global capability center (MGCC) in India to scale and mature Data & Analytics, technology capabilities in a cost-effective manner and make MetLife future ready. The center is integral to Global Technology and Operations with a with a focus to protect & build MetLife IP, promote reusability and drive experimentation and innovation. The Data & Analytics team in India mirrors the Global D&A team with an objective to drive business value through trusted data, scaled capabilities, and actionable insights Role Value Proposition MetLife Global Capability Center (MGCC) is looking for a Senior Cloud data engineer who has the responsibility of building ETL/ELT, d...

As a seasoned Testing Sr. Manager in Treasury and FP&A Technology, you will play a crucial role in defining, planning, and executing the testing automation strategy for the Global Funds Transfer Pricing Application. Your expertise in automation tools, agile methodologies, and quality engineering best practices will be instrumental in transforming and enhancing the current testing automation landscape. Your responsibilities will include continuously monitoring automation coverage, enhancing the existing automation framework, designing scalable automation frameworks for UI, API, and data validation testing on Big Data/Hadoop platform, collaborating with various teams to integrate automation in...

As a Software Engineer II at JPMorgan Chase within the Employee Platforms team, you will have the opportunity to enhance your software engineering career while working with a team of agile professionals. Your main responsibility will be to design and deliver cutting-edge technology products in a secure, stable, and scalable manner. You will play a crucial role in developing technology solutions across different technical areas to support the firm's business objectives. Your key responsibilities will include executing innovative software solutions, developing high-quality production code, and identifying opportunities to enhance operational stability. You will lead evaluation sessions with ex...

NTT DATA is looking for a Data Ingest Engineer to join the team in Pune, Mahrshtra (IN-MH), India (IN). As a Data Ingest Engineer, you will be part of the Ingestion team of the DRIFT data ecosystem, focusing on ingesting data in a timely, complete, and comprehensive manner using the latest technology available to Citi. Your role will involve leveraging new and creative methods for repeatable data ingestion from various sources while ensuring the highest quality data is provided to downstream partners. Responsibilities include partnering with management teams to integrate functions effectively, identifying necessary system enhancements for new products and process improvements, and resolving ...

As a Talend ETL Lead, you will be responsible for leading the design and development of scalable ETL pipelines using Talend, integrating with big data platforms, and mentoring junior developers. This is a high-impact, client-facing role requiring hands-on leadership and solution ownership. Lead the end-to-end development of ETL pipelines using Talend Data Fabric. Collaborate with data architects and business stakeholders to understand requirements. Build and optimize data ingestion, transformation, and loading processes. Ensure high performance, scalability, and reliability of data solutions. Mentor and guide junior developers in the team. Troubleshoot and resolve ETL-related issues quickly....

Looking for: Hadoop Developer Experience: 3 to 6 years Requirement 1: Primary Skills: Apache Hadoop and Apache Spark Required Skills Must be strong in Hadoop and Spark Architecture Hands-on knowledge on how HDFS/Hive/Impala/Spark works Strong in logical reasoning capabilities Should have strong hands-on experience on Hive/Impala/Spark query performance tuning concepts Good UNIX Shell, Python/Scala scripting knowledge Should have working knowledge of Github, DevOps, CICD/ Enterprise code management tools Familiarity with Java, Spark Strong collaboration and communication skills Must possess strong team-player skills and should have excellent written and verbal communication skills Ability to ...

As a Site Reliability Engineering (SRE) Technical Leader on the Network Assurance Data Platform (NADP) team at ThousandEyes, you will be responsible for ensuring the reliability, scalability, and security of cloud and big data platforms. Your role will involve representing the NADP SRE team, working in a dynamic environment, and providing technical leadership in defining and executing the team's technical roadmap. Collaborating with cross-functional teams, including software development, product management, customers, and security teams, is essential. Your contributions will directly impact the success of machine learning (ML) and AI initiatives by ensuring a robust and efficient platform in...

As an experienced professional with 3-5 years of experience, you will be responsible for working with a range of technical skills including Azure Data Factory, Talend/SSIS, MSSQL, Azure, and MySQL. Your primary focus will be on Azure Data Factory, where you will utilize your expertise to handle complex data analysis tasks effectively. In this role, you will demonstrate advanced knowledge in Azure SQL DB & Synapse Analytics, Power BI, SSIS, SSRS, T-SQL, and Logic Apps. It is essential that you possess a solid understanding of Azure Data Lake and Azure Services such as Analysis Service, SQL Databases, Azure DevOps, and CI/CD processes. Furthermore, your responsibilities will include mastering ...

You will be joining Lifesight as a Data Engineer in our Bengaluru office, playing a pivotal role in the Data and Business Intelligence organization. Your primary focus will be on leading deep data engineering projects and contributing to the growth of our data platform team. This is an exciting opportunity to shape our technical strategy and foster a strong data engineering team culture in India. As a Data Engineer at Lifesight, you will be responsible for designing and constructing data platforms and services, managing data infrastructure in cloud environments, and enabling strategic business decisions across Lifesight products. Your role will involve building highly scalable, fault-toleran...