896 Hdfs Jobs - Page 9

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Apache Hadoop Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs,...

Organization: At CommBank, we never lose sight of the role we play in other people's financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things Job Title: Senior Data Engineer Location: Bangalore-Manyata Tech Park (Hybrid) Business & Team : Retail Technology Impact & contribution Contribute to internal online discussion around engineering, delivery and technology e.g. blog posts, k...

Roles and Responsibilities : Manage Hadoop clusters on Cloudera platform, ensuring high availability and performance. Design, implement, and maintain ETL processes using Hive, Impala, Spark, and Python scripts. Troubleshoot issues related to HDFS, HBase, and other Big Data technologies in the stack. Configure and manage data storage policies, replication, and fault tolerance. Implement and manage security policies, including Kerberos for authentication and Ranger or Sentry for authorization. Conduct regular security audits and compliance checks. Deploy Cloudera clusters from scratch, including hardware and software configuration. Configure clusters to meet specific business requirements and ...

Job Title: Cloudera Hadoop Administrator Location: Hyderabad Experience: 4+ years Employment Type: Full Time Job Summary We are looking for a proactive and technically strong Hadoop Administrator who has hands-on experience working with Red Hat Linux systems, basic automation through Bash scripting, and foundational exposure to Hadoop and Cloudera platforms. The ideal candidate has been involved in real-time system recovery, automation, and training initiatives and is eager to expand their skills in Linux system administration and Big Data platforms. Key Responsibilities Administered and managed Red Hat Enterprise Linux (RHEL 7/8) servers to ensure system stability, performance, and security...

Loc: Manyata Tech Park Exp: 2-5 yrs, AI/ML Engineer Skills Required: ? Exp. with Data engineering & Data/ML pipelines for Analytics & AI/ML Models ? Working with large sets of structured & unstructured data from disparate sources ? Foundational acquaintance of Data Science & AI/ML models ? SQL, Python, Spark, PySpark ? Big Data systems - Hadoop Ecosystem (HDFS, Hive, MapReduce) ? Analytics database like Druid, Data visualization/exploration tools like Superset ? Understanding of cloud platforms (AWS, Azure, or GCP) ? GIT Secondary Skills (desired/preferred): ? Apache Airflow, Apache Oozie, Nifi ? GCP cloud exp, Big Query, GCP Big Data Ecosystem ? Trino/Presto ? Familiar with forecasting algo...

Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience ¢ Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms) ¢ Proficient in developing automated frameworks for unit & integration testing. ¢ Proficient with Kubernetes, Docker, Helm and related container technologies. ¢ Proficient in deploying and managing Spark workloads on Kubernetes clusters. ¢

Skills : Spark, Scala, Unix, Shell Scripting, GCP - BigQuery, BigTable, DataProc, AirflowManual testing All ETL jobs are written in Scala, preferably Scala with Spark will help supporting and migrating to Lumi also helps.(No python for this Roles and Responsibilities Skills : Spark, Scala, Unix, Shell Scripting, GCP - BigQuery, BigTable, DataProc, AirflowManual testing All ETL jobs are written in Scala, preferably Scala with Spark will help supporting and migrating to Lumi also helps.(No python for this

Embark on an exciting journey into the realm of data engineering and architecture with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. As a Data Architect you will be at the heart of the organization and support our clients to take control of their data and get value out of it by defining a reference architecture for our customers. This means that you will work closely with business leaders and information management teams to define and implement a roadmap on data architecture, data management, business intelligence or analytics solutions.. Minimum Qualifications: Translate business requirements into data requests, reports and dashboards. Stro...

At EY, we're all in to shape your future with confidence. We'll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY-Advisory - Data and Analytics Senior Data Engineer(Scala) EY's Advisory Services is a unique, industry-focused business unit that provides a broad range of integrated services that leverage deep industry experience with strong functional and technical capabilities and product knowledge. EY's financial services practice provides integrated advisory services to financial institutions and other capital markets participants, including commercial banks, retail banks...

Company Description LinkedIn is the world's largest professional network, built to create economic opportunity for every member of the global workforce. Our products help people make powerful connections, discover exciting opportunities, build necessary skills, and gain valuable insights every day. We're also committed to providing transformational opportunities for our own employees by investing in their growth. We aspire to create a culture that's built on trust, care, inclusion, and fun where everyone can succeed. Join us to transform the way the world works. Job Description At LinkedIn, every change to our products and services is rigorously tested through experiments to assess their imp...

Cloudera Administrator Location: Remote (UAE Time Zone) Experience: 5+ Years Availability: Immediate joiners or candidates who can start within 30 days preferred Role Overview We are seeking an experienced Cloudera Administrator to manage and maintain our Cloudera Data Platform (CDP) environment. The ideal candidate will have hands-on experience in installation, configuration, performance optimization, and operational support of Cloudera components in a production-grade, enterprise environment. Key Responsibilities Install, configure, and maintain Cloudera Data Platform (CDP) components including Hive, NIFI, Impala, Oozie, and CML . Manage cluster upgrades, patches, and scaling with minimal ...

Cloudera Administrator Location: Remote (UAE Time Zone) Experience: 5+ Years Availability: Immediate joiners or candidates who can start within 30 days preferred Role Overview We are seeking an experienced Cloudera Administrator to manage and maintain our Cloudera Data Platform (CDP) environment. The ideal candidate will have hands-on experience in installation, configuration, performance optimization, and operational support of Cloudera components in a production-grade, enterprise environment. Key Responsibilities Install, configure, and maintain Cloudera Data Platform (CDP) components including Hive, NIFI, Impala, Oozie, and CML . Manage cluster upgrades, patches, and scaling with minimal ...

Role Overview: You will be part of a high performing team implementing a strategic future state technology landscape for all DWS Corporate Functions globally. As an ETL developer, you will be responsible for developing, testing, and maintaining robust and scalable ETL processes to support data integration initiatives. This role requires a strong understanding of database, Unix, and ETL concepts, excellent SQL skills, and experience with ETL tools and databases. Key Responsibilities: - Create good quality software using standard coding practices - Hands-on code development - Thorough testing of developed ETL solutions/pipelines - Code review of other team members - Take end-to-end accountabil...

As a Senior Data Engineer (Scala) at EY Advisory - Data and Analytics, you will have the opportunity to leverage your expertise in the Financial Services domain and Big Data ecosystem to design, build, and maintain scalable data pipelines and solutions. Your role will involve working with Spark, Scala, SQL, and Cloud technologies like GCP, Azure, or AWS to manage and process large datasets efficiently. Additionally, you will be expected to participate in all aspects of the Big Data solution delivery life cycle, from analysis to production deployment and support. **Key Responsibilities:** - Strong knowledge in Spark with a good understanding of the Spark framework and performance tuning - Pro...

We are seeking an experienced Hadoop Developer to design, develop, and maintain big data solutions using the Hadoop ecosystem. The ideal candidate will have strong expertise in distributed computing , data processing frameworks , and performance optimization , ensuring scalable and reliable data workflows. Key Responsibilities: Design and implement data processing pipelines using Hadoop ecosystem tools (HDFS, MapReduce, Hive, Pig). Develop and optimize ETL workflows for large-scale data ingestion and transformation. Work with Spark , Hive , and HBase for advanced analytics and storage. Collaborate with data engineers , data scientists , and business teams to deliver big data solutions. Ensur...

Role Overview: As an experienced Oracle Big Data Service (BDS) Specialist at Kenexai, your main responsibility will be to design, implement, and maintain the Oracle BDS infrastructure. Your expertise in Oracle BDS cluster setup, configuration, administration, and performance optimization will be crucial in managing Hadoop ecosystem components within the Oracle Cloud Infrastructure (OCI). Key Responsibilities: - Design and deploy Oracle BDS clusters on OCI - Configure BDS cluster topologies including HA and non-HA setups - Set up and configure cluster nodes (master, utility, worker, edge nodes) - Implement cluster security with Kerberos authentication and Ranger policies - Configure network s...

Role Overview: Cisco ThousandEyes is a Digital Assurance platform that enables organizations to ensure flawless digital experiences across all networks, including those not owned by the organization. As a Site Reliability Engineering (SRE) Technical Leader on the Network Assurance Data Platform (NADP) team, you will be responsible for ensuring the reliability, scalability, and security of cloud and big data platforms. Your role will involve collaborating with cross-functional teams to design, influence, build, and maintain SaaS systems at a multi-region scale. Key Responsibilities: - Design, build, and optimize cloud and data infrastructure to ensure high availability, reliability, and scala...

Ascendeum is looking for veterans with extensive hands-on experience in the field of data engineering to build cutting-edge solutions for large-scale data extraction, processing, storage, and retrieval. About Us: We provide AdTech strategy consulting to leading Internet websites and apps globally hosting over 200 million monthly worldwide audiences. Since 2015, our team of consultants and engineers have been consistently delivering intelligent solutions that enable enterprise-level websites and apps to maximize their digital advertising returns. Job Responsibilities: Understand long-term and short-term business requirements to precisely match them with the capabilities of different distribut...

Job Description Key Responsibilities : Design, develop, and maintain high-performance, scalable applications using Java and Big Data technologies Build and manage data pipelines to process structured and unstructured data from multiple sources Develop and maintain microservices, RESTful APIs, and other distributed systems Work with modern Big Data tools and technologies such as : Apache Spark HDFS Ceph Storage Solr / Elasticsearch Apache Kafka Delta Lake Write clean, efficient, and well-documented code Collaborate with cross-functional teams including Data Engineers, QA, and DevOps Participate in code reviews and contribute to continuous improvement of development processes Mentor and suppor...

Experience At least 5 years of experience in AWS based projects. Technical skills Proficiency in Python and PySpark for data engineering tasks. Big Data Strong knowledge of Big Data technologies and data warehousing concepts. AWS services Experience with AWS Data Engineering stack, including S3, RDS, Athena, Glue, Lambda, and Step Functions. SQL Strong SQL skills for data manipulation and querying. CI CD Experience with CI CD tools like Terraform and Git Actions. Soft skills Good communication skills and ability to work in a multicultural team. Design and implement data pipelines Develop ETL jobs to ingest and move data within the AWS environment using tools like AWS Glue Data storage and pr...

At EY, we're all in to shape your future with confidence. We'll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY - GDS Consulting - AI & DATA - Snowflake & Databrick with Big Data or Pyspark- Manager EY's GDS Consulting AI & DATA is a unique, industry-focused business unit that provides a broad range of integrated services that leverage deep industry experience with strong functional and technical capabilities and product knowledge. EY's financial services practice provides integrated advisory services to financial institutions and other capital markets participants, incl...

At eBay, we're more than a global ecommerce leader we're changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We're committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts. Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work every day. We're in this together, sustaining the future of our customers, our company, and our planet. Join a team of passionate thinkers, innovators, and dreamers and help us connect people and build communities to create economic opportunity for all. At eBa...

The Company PayPal has been revolutionizing commerce globally for more than 25 years. Creating innovative experiences that make moving money, selling, and shopping simple, personalized, and secure, PayPal empowers consumers and businesses in approximately 200 markets to join and thrive in the global economy. We operate a global, two-sided network at scale that connects hundreds of millions of merchants and consumers. We help merchants and consumers connect, transact, and complete payments, whether they are online or in person. PayPal is more than a connection to third-party payment networks. We provide proprietary payment solutions accepted by merchants that enable the completion of payments...

Role The Senior Software Engineer will be responsible for development solutions with high level of innovation, high quality and faster time to market. This position interacts with product managers, engineering leaders, architects and software developers and business operations on the definition and delivery of highly scalable and secure solutions. The role includes: Hands-on developer who writes high quality, secure code that is modular, functional and testable. Create or introduce, test, and deploy new technology to optimize the service Contribute to all parts of the software's development including design, development, documentation, and testing. Have strong ownership Communicate, collabor...

Educational Requirements Bachelor of Engineering,BTech,BCA,MTech,MCA Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to provide best fit architectural solutions for one or more projects. You would also provide technology consultation and assist in defining scope and sizing of work You would implement solutions, create technology differentiation and leverage partner technologies. Additionally, you would participate in competency development with the objective of ensuring the best-fit and high quality technical solutions. You would be a key contributor in creating thought leadership within th...