582 Hdfs Jobs - Page 14

As a software developer, you will be working in a constantly evolving environment driven by technological advances and the strategic direction of the organization you are employed by. Your primary responsibilities will include creating, maintaining, auditing, and enhancing systems to meet specific needs, often based on recommendations from systems analysts or architects. You will be tasked with testing both hardware and software systems to identify and resolve system faults. Additionally, you will be involved in writing diagnostic programs and designing and developing code for operating systems and software to ensure optimal efficiency. In situations where necessary, you will also provide re...

As a Hadoop Admin, you will be responsible for managing and supporting Hadoop clusters and various components such as HDFS, HBase, Hive, Sentry, Hue, Yarn, Sqoop, Spark, Oozie, ZooKeeper, Flume, and Solr. With a minimum of 4 years of experience in Hadoop administration, you will play a crucial role in installing, configuring, maintaining, troubleshooting, and monitoring these clusters to ensure their efficient functioning in production support projects. Your primary duties will include integrating analytical tools like Datameer, Paxata, DataRobot, H2O, MRS, Python, R-Studio, SAS, and Dataiku-Bluedata with Hadoop, along with conducting job level troubleshooting for components such as Yarn, Im...

The Applications Development Sr Programmer Analyst will work in Information management team of Services Technologies, on projects focusing Big Data and public cloud adoption. It is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised appli...

As a Hadoop Developer with 6+ years of experience and a notice period of immediate to 15 days, you will be responsible for hands-on experience in Hadoop, working with HDFS (Hadoop Distributed File System), and utilizing Python and Spark. Your role will involve designing and developing scalable and robust Hadoop-based data pipelines. This position is based in Hyderabad with a general shift schedule. You should possess key skills in Hadoop, Python, HDFS, and Spark to excel in this role. A Bachelor's degree in a related field is required for this full-time, permanent position in the IT/Computers - Software industry. As a valued team member, you will contribute to the growth and success of the o...

You have a great opportunity to join as a Data Software Engineer with 5-12 years of experience in Big Data & Data related technology. We are looking for candidates with an expert level understanding of distributed computing principles and hands-on experience in Apache Spark along with proficiency in Python. You should also have experience with technologies like Hadoop, Map Reduce, HDFS, Sqoop, Apache Storm, Spark-Streaming, Kafka, Hive, Impala, and integration of data from various sources such as RDBMS, ERP, and Files. Additionally, knowledge of NoSQL databases, ETL techniques, SQL queries, joins, stored procedures, relational schemas, and performance tuning of Spark Jobs is required. Moreov...

About the job: At Citi, we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with a laser focus on evolving the ways of doing things. As one of the world's most global banks, we're changing how the world does business. Shape your Career with Citi We're currently looking for a high-caliber professional to join our team as AVP- Data Engineer based in Pune, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy ...

As a Data Software Engineer at KG Invicta Services Pvt Ltd, you will leverage your 5-12 years of experience in Big Data & Data-related technologies to drive impactful solutions. Your expertise in distributed computing principles and Apache Spark, coupled with hands-on programming skills in Python, will be instrumental in designing and implementing efficient Big Data solutions. You will demonstrate proficiency in a variety of tools and technologies including Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, Spark-Streaming, Kafka, RabbitMQ, Hive, Impala, and NoSQL databases such as HBase, Cassandra, and MongoDB. Your ability to integrate data from diverse sources like RDBMS, ERP, and files, a...

As a Data Ops Capability Deployment Analyst at our organization, you will be a seasoned professional contributing to the development of new solutions, frameworks, and techniques while improving processes and workflow for the Enterprise Data function. Your role will involve integrating subject matter and industry expertise within a defined area, requiring an in-depth understanding of how different areas collectively integrate within the sub-function to contribute to the overall business objectives. Your primary responsibility will be to perform data analytics and analysis across various asset classes, as well as to establish data science and tooling capabilities within the team. You will coll...

At least 8+ years of experience and strong knowledge in Scala programming language. Able to write clean, maintainable and efficient Scala code following best practices. Good knowledge on the fundamental Data Structures and their usage At least 8+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies. Having expertise in Spark Core, Spark SQL and Spark Streaming. Experience with Hadoop, HDFS, Hive and other BigData technologies. Familiarity with Data warehousing and ETL concepts and techniques Having expertise in Database concepts and SQL/NoSQL operations. UNIX shell scripting will be an added advantage i...

As a Sr. Data Engineer at Lifesight in Bangalore, you will be responsible for building highly scalable, fault-tolerant distributed data processing systems that handle massive amounts of data ingested daily. You will work on processing petabyte-sized data warehouses and Elasticsearch clusters, optimizing data pipelines for quality and resilience, and refining diverse datasets into simplified models to encourage self-service. Your role will involve owning data mapping, business logic, transformations, and ensuring data quality through low-level systems debugging and performance optimization on large production clusters. Additionally, you will participate in architecture discussions, influence ...

You are a highly skilled and detail-oriented ETL QA - Technical Lead with a solid background in Big Data Testing, the Hadoop ecosystem, and SQL validation. Your primary responsibility will be leading end-to-end testing efforts for data/ETL pipelines across various big data platforms. You will be working closely with cross-functional teams in an Agile environment to ensure the quality and integrity of large-scale data solutions. Your key responsibilities include designing and implementing test strategies for validating large datasets, transformations, and integrations. You will be hands-on testing Hadoop-based data platforms such as HDFS, Hive, and Spark. Additionally, you will develop comple...

ETL QA - Technical Lead Experience: 7 to 11 Years Job Locations: Hyderabad (1 position) | Gurgaon (1 position) Job Summary: We are looking for a highly skilled and detail-oriented ETL QA - Technical Lead with strong experience in Big Data Testing , Hadoop ecosystem , and SQL validation . The ideal candidate should have hands-on experience in test planning, execution, and automation in a data warehouse/ETL environment. You&aposll work closely with cross-functional teams in an Agile environment to ensure the quality and integrity of large-scale data solutions. Key Responsibilities: Lead end-to-end testing efforts for data/ETL pipelines across big data platforms Design and implement test strate...

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solutions. Utilize a deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution Your Impact: Data Ingestion, Integration and Transformation Data Storage and Computation Frameworks, Performance Optimizations Analytics & Visualizations Infrastructure & Cloud Computing Data Management Platforms Build functionality for data ingestion from multiple...

The Senior Associate People Senior Associate L2 in Data Engineering, you will translate client requirements into technical design, and implement components for a data engineering solutions. Utilize a deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution Your Impact: Data Ingestion, Integration and Transformation Data Storage and Computation Frameworks, Performance Optimizations Analytics & Visualizations Infrastructure & Cloud Computing Data Management Platforms Build functionality for data ingestion from multip...

Minimum 2 years of experience in Big Data technologies Hands-on experience with the Hadoop stack HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end-to-end data pipelines. Working knowledge of real-time data pipelines is added advantage. Strong experience in at least the programming language Java, Scala, and Python. Java preferable Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery, etc. Well-versed and working knowledge with data platform-related services on AWS Bachelor's degree and year of work experience of 4 to 6 years or ...

The Senior Associate People Senior Associate L2 in Data Engineering, you will translate client requirements into technical design, and implement components for a data engineering solutions. Utilize a deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution Your Impact: Data Ingestion, Integration and Transformation Data Storage and Computation Frameworks, Performance Optimizations Analytics & Visualizations Infrastructure & Cloud Computing Data Management Platforms Build functionality for data ingestion from multip...

As a Data Software Engineer, you will be responsible for utilizing your 5-12 years of experience in Big Data & Data-related technologies to contribute to the success of projects in Chennai and Coimbatore in a Hybrid work mode. You should possess an expert level understanding of distributed computing principles and a strong knowledge of Apache Spark, with hands-on programming skills in Python. Your role will involve working with technologies such as Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, and Spark-Streaming to build stream-processing systems. You should have a good grasp of Big Data querying tools like Hive and Impala, as well as experience in integrating data from various sources ...

At Iron Mountain, we believe that work, when done well, can have a positive impact on our customers, employees, and the planet. That's why we are looking for smart and committed individuals to join our team. Whether you are starting your career or seeking a change, we invite you to explore how you can enhance the impact of your work at Iron Mountain. We offer expert and sustainable solutions in records and information management, digital transformation services, data centers, asset lifecycle management, and fine art storage, handling, and logistics. Collaborating with over 225,000 customers worldwide, we aim to preserve valuable artifacts, optimize inventory, and safeguard data privacy throu...

As a Big Data Engineer at KGIS, you will be an integral part of the team dedicated to building cutting-edge digital and analytics solutions for global enterprises. With a focus on designing, developing, and optimizing large-scale data processing systems, you will lead the way in creating scalable data pipelines, driving performance tuning, and spearheading cloud-native big data initiatives. Your responsibilities will include designing and developing robust Big Data solutions using Apache Spark, building both batch and real-time data pipelines utilizing technologies like Spark, Spark Streaming, Kafka, and RabbitMQ, implementing ETL processes for data ingestion and transformation, and optimizi...

The ideal candidate for the position of AL/ML Architect should possess a B.SC, B.E/B.Tech./M.E./M.S./M.Tech degree with a strong academic record. With at least 10 years of overall experience, including 4-5 years in an Architect role, the candidate should also have 4+ years of hands-on experience applying statistical/ML algorithms and techniques to real-world data sets. Additionally, 4+ years of experience as a developer working on scalable and secure applications is required, along with 2+ years of experience independently designing core product modules or complex components. Proficiency in GenAI is a must-have for this role. Key Responsibilities: The responsibilities of this role include de...

As a Technical Support Engineer specializing in Big Data applications and systems within the Hadoop ecosystem, you will play a crucial role in ensuring the smooth operation, performance, and security of our data infrastructure. Your responsibilities will include providing technical support and troubleshooting for Big Data applications, monitoring system performance, collaborating with engineering teams to deploy and configure Hadoop clusters, and assisting in maintenance and upgrades of Hadoop environments. You will also be responsible for developing and maintaining documentation, implementing data backup and recovery procedures, participating in on-call rotations, and staying up to date wit...

You will be responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Your focus will be on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing. Your responsibilities will include developing Spark applications using Scala or Python (Pyspark) for data transformation, aggregation, and analysis. You will also need to develop and maintain Kafka-based data pipelines, which involves designing Kafka Streams, setting up Kafka Clusters, and ensuring efficient data flow. Additionally, you will create and optimize Spark applications using Scala and PySp...

Position : AL/ML Architect Location : Mumbai/Bangalore Qualifications: The successful candidate should have either of the below B.SC, B.E/B.Tech./M.E./M.S./M.Tech with good academic record Experience: 10+ years overall with 4-5 years as Architect 4+ years of experience applying statistical/ML algorithms and techniques to real-world data sets. 4+ years as a developer with experience in related scalable and secure applications. 2+ years experience independently designing core product modules or complex components Hands on Experience on GenAI Key Responsibilities Involves in Designing and Overseeing the overall architecture of ML systems, including data pipelines, model selection, deployment st...

Job Description About Oracle Analytics & Big Data Service: Oracle Analytics is a complete platform that supports every role within analytics, offering cloud-native services or on-premises solutions without compromising security or governance. Our platform delivers a unified system for managing everything from data collection to decision-making, with seamless integration of AI and machine learning to help businesses accelerate productivity and uncover critical insights. Oracle Big Data Service, a part of Oracle Analytics, is a fully managed, automated cloud service designed to help enterprises create scalable Hadoop-based data lakes. The service work scope encompasses not just good integratio...

You should have 5-12 years of experience in Big Data & Data related technologies. Your expertise should include a deep understanding of distributed computing principles and strong knowledge of Apache Spark. Proficiency in Python programming is required, along with experience using technologies such as Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, and Spark-Streaming for building stream-processing systems. You should have a good understanding of Big Data querying tools like Hive and Impala, as well as experience in integrating data from various sources such as RDBMS, ERP, and Files. Knowledge of SQL queries, joins, stored procedures, and relational schemas is essential. Experience with No...