582 Hdfs Jobs - Page 13

Organization: At CommBank, we never lose sight of the role we play in other people's financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things. Job Title: Staff. Data Engineer Location: Bangalore - Manyata Tech Park Business & Team: RBS (HB Domain) ---Search & Find Impact & contribution: You will play a pivotal role in driving business outcomes through effective data management an...

Data Software Engineer Job Description: 1. 5-12 Years of in Big Data & Data related technology experience 2. Expert level understanding of distributed computing principles 3. Expert level knowledge and experience in Apache Spark 4. Hands on programming with Python 5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop 6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming 7. Experience with messaging systems, such as Kafka or RabbitMQ 8. Good understanding of Big Data querying tools, such as Hive, and Impala 9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files 10. Good under...

Times Internet is the largest digital products company and the digital venture of Times of India, India's largest media and entertainment group. TIL websites are among the fastest growing Web / Mobile based networks worldwide. Since its inception in 1999, Times Internet has led the Internet revolution in India and has emerged as India's foremost web entity, running diverse portals and niche websites. The Database Professional will participate in the design, implementation, automation, optimization, and ongoing operational administration related tasks backend system running on MySQL/MariaDB/ MongoDB/RedisDB/Hadoop database infrastructure. Candidates should be ready to take the instant support...

Job Description: About Us At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities, and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work for our teammates around the world. We're devoted to being a diverse and inclusive workplace for everyone. We hire individuals with a broad range of backgrounds and experiences and invest heavily in our teammates and their families by offering competitive benefits to support their physical, emotional, and financial well-being. Bank o...

As a Python Senior Developer, you should have a minimum of 5 years of experience in Python programming. You must possess extensive experience in handling complex projects and demonstrate strong leadership skills to lead projects independently. Your effective communication and teamwork skills will be crucial in collaborating with team members. Your role will require good analytical skills and proficiency in web-based development technologies. It is essential to have expertise in Agile development methodology and strong problem-solving abilities. You should also possess a deep knowledge of Design patterns, Security, Performance tuning, and App monitoring. Experience in API Design, security pat...

As the Head of Technology - New Initiatives at Matrimony.com, you will be at the forefront of driving innovation in our WedTech domain through a groundbreaking project. Your role will be pivotal in orchestrating the convergence of cutting-edge technology with matrimonial services, leading a minimum of one product tech team towards technological supremacy. Your key responsibilities will include: - Developing and implementing a strategic tech roadmap that aligns with industry advancements and business benchmarks. - Overseeing the product engineering lifecycle of our flagship product, ensuring scalability and efficiency from prototype to production readiness. - Building and nurturing a high-cal...

You are an experienced PySpark ETL Lead responsible for driving data integration efforts in analytics and data warehousing projects. Your role includes developing and managing PySpark scripts, creating ETL workflows, and ensuring efficient data processing and integration across systems. You should be strong in PySpark and Python for ETL and data processing, with experience in ETL pipeline design, data integration, and data warehousing. Proficiency in SQL and working with large datasets is required, along with familiarity with workflow schedulers such as Airflow and Cron. Hands-on experience with Big Data tools like Hive, HDFS, and Spark SQL is essential. It would be beneficial if you have ex...

As a Senior Big Data Developer specializing in PySpark, with a robust 57 years of experience, you will be responsible for designing, developing, and optimizing scalable data pipelines. Collaboration with cross-functional teams is essential to deliver reliable, high-performance data solutions in the Apache Big Data ecosystem. Your primary responsibilities will encompass building and enhancing large-scale data processing pipelines using PySpark, working with tools like Apache Hadoop, Hive, HDFS, Spark, and more to manage big data workloads efficiently. Additionally, you will be engaged in ETL development, data ingestion, transformation, integration from diverse sources, and collaborating with ...

6+ years of experience in information technology, Minimum of 3-5 years of experience in managing and administering Hadoop/Cloudera environments. Cloudera CDP (Cloudera Data Platform), Cloudera Manager, and related tools. Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark, Impala, etc.). Linux system administration with experience with scripting languages (Python, Bash, etc.) and configuration management tools (Ansible, Puppet, etc.) Tools like Kerberos, Ranger, Sentry), Docker, Kubernetes, Jenkins Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certification. Cluster Management, Optimization, Best practice implementation, collaboration and support.

We are seeking an experienced PySpark Developer / Data Engineer to design, develop, and optimize big data processing pipelines using Apache Spark and Python (PySpark). The ideal candidate should have expertise in distributed computing, ETL workflows, data lake architectures, and cloud-based big data solutions. Key Responsibilities: Develop and optimize ETL/ELT data pipelines using PySpark on distributed computing platforms (Hadoop, Databricks, EMR, HDInsight). Work with structured and unstructured data to perform data transformation, cleansing, and aggregation. Implement data lake and data warehouse solutions on AWS (S3, Glue, Redshift), Azure (ADLS, Synapse), or GCP (BigQuery, Dataflow). Op...

JOB DESCRIPTION Develop, test, and deploy data processing applications using Apache Spark and Scala. Optimize and tune Spark applications for better performance on large-scale data sets. Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions. Collaborate with data scientists, business analysts, and other developers to understand data requirements and deliver solutions. Design and implement high-performance data processing and analytics solutions. Ensure data integrity, accuracy, and security across all processing tasks. Troubleshoot and resolve performance issues in Spark, Cloudera, and related technologies. Implement ve...

Experience in the Big Data technologies (Hadoop, Spark, Nifi, Impala) 5+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines. High proficiency in Scala/Java and Spark for applied large-scale data processing. Expertise with big data technologies, including Spark, Data Lake, and Hive. Solid understanding of batch and streaming data processing techniques. Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion. Expert-level ability to write complex, optimized SQL queries across extensive data volumes. Experience ...

As a Data Governance Architect at Tiger Analytics, you will play a crucial role in designing, architecting, deploying, and maintaining big data-based data governance solutions. Your responsibilities will include technical management throughout the project life cycle, collaboration with various teams, exploring new technologies, and leading a team of data governance engineers. You are expected to have a minimum of 10 years of technical experience, with at least 5 years in the Hadoop ecosystem and 3 years in Data Governance Solutions. Hands-on experience with Data Governance Solutions is essential, along with a good understanding of data catalog, business glossary, metadata, data quality, data...

Wipro Limited is a prominent technology services and consulting company committed to developing innovative solutions to meet the intricate digital transformation requirements of its clients. With a comprehensive range of capabilities in consulting, design, engineering, and operations, Wipro assists clients in achieving their ambitious goals and establishing sustainable, future-ready businesses. The company, with over 230,000 employees and business partners spread across 65 countries, is dedicated to supporting customers, colleagues, and communities in navigating through a constantly evolving world. For more information, please visit www.wipro.com. **Job Title: Hadoop Admin** **Primary Skills...

Job description Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

Role & responsibilities At least 5+ years of experience and strong knowledge in Scala programming language. Able to write clean, maintainable and efficient Scala code following best practices. Good knowledge on the fundamental Data Structures and their usage At least 5+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies. Having expertise in Spark Core, Spark SQL and Spark Streaming. Experience with Hadoop, HDFS, Hive and other BigData technologies. Familiarity with Data warehousing and ETL concepts and techniques Having expertise in Database concepts and SQL/NoSQL operations. UNIX shell scripting will...

As a Director of Engineering at Citi's Banking Technology division in India, you will play a pivotal role in overseeing the strategic Big Data and Reporting initiatives while leading and ensuring the growth and operational excellence of the engineering site in India. You will be responsible for driving the delivery of large-scale big data platforms and sophisticated reporting solutions for the banking sector by leveraging extensive datasets for insights, risk management, fraud detection, and regulatory compliance. Your role will involve defining and executing the technical roadmap for big data and reporting, aligning it with overall business objectives and industry best practices. Collaborat...

Job Requirements Job Requirements Role/ Job Title: Data Engineer Function/ Department: Data & Analytics Job Purpose The Data Engineer is responsible for developing day to day data engineering projects and aiding in building the business data collection systems and processing pipelines. Responsible for efficient code for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department. He/She works with Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, Python, and Spark. The Data Engineer strives to continuously develop most efficient code for data pipe...

We are seeking an experienced Software Engineer with deep expertise in Scala programming and Big Data technologies to design, develop, and maintain large-scale distributed data processing systems. The ideal candidate will be a hands-on developer with a strong understanding of data pipelines, Spark ecosystem, and related technologies, capable of delivering clean, efficient, and scalable code in an Agile environment. Key Responsibilities Develop and maintain scalable, efficient, and robust data processing pipelines using Scala and Apache Spark (Spark Core, Spark SQL, Spark Streaming). Write clean, maintainable, and well-documented Scala code following industry best practices and coding standar...

USEReady helps businesses to be self-reliant on data. Growing over 3000% since inception, USEReady achieved #113 rank in Inc 500 2015 and honoured top 100 companies in North America 2015 by Red Herring. USEReady is built on strong entrepreneurial spirit with unprecedented opportunities for career growth. At USEReady, we believe in achieving career growth while improving our individual competencies. If you desire to be part of a team that believes in mutual success and inspiration, you are welcome to apply. USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners ...

7+ years of data warehousing/engineering, software solutions design and development experience. • Experience in designing and architecting distributed data systems • Code, test, and document new or modified data systems to create robust and scalable applications for data analytics. • Work with other Bigdata developers to make sure that all data solutions are consistent. • Partner with business community to understand requirements, determine training needs and deliver user training sessions • Perform technology and product research to better define requirements, resolve important issues and improve the overall capability of the analytics technology stack. • Evaluate and provides feedback on f...

We are looking for lead or principal software engineers to join our Data Cloud team. Our Data Cloud team is responsible for the Zeta Identity Graph platform, which captures billions of behavioural, demographic, environmental, and transactional signals, for people-based marketing. As part of this team, the data engineer will be designing and growing our existing data infrastructure to democratize data access, enable complex data analyses, and automate optimization workflows for business and marketing operations. Job Description: Essential Responsibilities: As a Lead or Principal Data Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and ba...

As a Data Platform Engineer - Tech Lead at Deutsche Bank in Pune, India, you will be part of the DB Technology global team of tech specialists. Your role involves leading a group of engineers working on cutting-edge technologies in Hadoop, Big Data, GCP, Terraform, Big Query, Data Proc, and data management to develop robust data pipelines, ensure data quality, and implement efficient data management solutions. Your leadership will drive innovation, maintain high standards in data infrastructure, and mentor team members to support data-driven initiatives. You will collaborate with data engineers, analysts, cross-functional teams, and stakeholders to ensure the data platform meets the organiza...

You should have a Bachelors Degree in Computer Science, Computer Engineering or related technical field, while a Masters Degree or other advanced degree is preferred. With 4-6+ years of total experience, you should possess at least 2+ years of relevant experience in Big Data platforms. Your skill set should include strong analytical, problem solving, and communication/articulation skills. Furthermore, you are expected to have 3+ years of experience with big data and the Hadoop ecosystem, including Spark, HDFS, Hive, Sqoop, Hudi, Parquet, Apache Nifi, and Kafka. Proficiency in Scala/Spark is required, and knowledge of Python is considered a plus. Hands-on experience with Oracle and MS-SQL dat...

The Applications Development Sr Programmer Analyst will work in the Information Management team of Services Technologies, focusing on Big Data and public cloud adoption projects. This intermediate-level position involves participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The primary objective is to contribute to applications systems analysis and programming activities. Responsibilities: - Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems an...