575 Hdfs Jobs - Page 9

We are looking for lead or principal software engineers to join our Data Cloud team. Our Data Cloud team is responsible for the Zeta Identity Graph platform, which captures billions of behavioural, demographic, environmental, and transactional signals, for people-based marketing. As part of this team, the data engineer will be designing and growing our existing data infrastructure to democratize data access, enable complex data analyses, and automate optimization workflows for business and marketing operations. Job Description: Essential Responsibilities: As a Lead or Principal Data Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and ba...

Role & responsibilities • Hands-on experience - Hadoop, System administration with sound knowledge In Unix based Operating System internals. • Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. • Linux experience (RedHat, CentOS, Ubuntu). • Experience in setting up and supporting Hadoop environment (Cloud and On-premises). • Ability to setup, configure and implement security for Hadoop clusters using Kerberos. • Ability to implement data-at-rest encryption (required), data-in-transit encryption(optional). • Ability to setup and troubleshoot Data Replication peers and policies. • Experience in setting up services like YARN, HDFS, Zookeeper, Hive, Spark, HBase etc. • ...

Role Responsibilities: Build scalable data pipelines and applications from scratch on AWS Design and manage data ingestion processes and workflows Collaborate with functional and data science teams to deliver data solutions Optimize data storage and organization using HDFS, S3, and Delta Lake Job Requirements: Strong programming experience in Python or Java Hands-on experience with AWS services and deployment Proficiency in HDFS, S3, SQL, and data ingestion tools Exposure to Delta Lake or Databricks is an added advantage

Role Responsibilities: Build scalable data pipelines and applications from scratch on AWS Design and manage data ingestion processes and workflows Collaborate with functional and data science teams to deliver data solutions Optimize data storage and organization using HDFS, S3, and Delta Lake Job Requirements: Strong programming experience in Python or Java Hands-on experience with AWS services and deployment Proficiency in HDFS, S3, SQL, and data ingestion tools Exposure to Delta Lake or Databricks is an added advantage

Role Responsibilities: Build scalable data pipelines and applications from scratch on AWS Design and manage data ingestion processes and workflows Collaborate with functional and data science teams to deliver data solutions Optimize data storage and organization using HDFS, S3, and Delta Lake Job Requirements: Strong programming experience in Python or Java Hands-on experience with AWS services and deployment Proficiency in HDFS, S3, SQL, and data ingestion tools Exposure to Delta Lake or Databricks is an added advantage

As a PySpark Developer at our company, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines for processing structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and design efficient data models and transformations - Optimizing and tuning PySpark jobs for performance, scalability, and reliability - Implementin...

As a Data Scientist at mPokket, you will be an integral part of a rapidly growing fintech startup in the lending domain specializing in offering microloans to college students and young professionals through a mobile app, with over 11 million loans disbursed. Backed by a US-based private equity firm, the profitable organization is poised to scale its business 10x-20x in the upcoming years. If you are an ambitious individual with a passion for working with raw data to derive meaningful insights, this is an opportunity for you to join on this incredible journey. Your primary responsibility involves collaborating with the data science team to plan and execute projects aimed at building advanced...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your key responsibilities will include: - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity, and consistency. - Optimizing data workflows for enhanced performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. - Implementing dat...

Role Overview: As a Data Scientist at mPokket, you will play a crucial role in deriving valuable insights from raw data to drive business decisions and improve products. You will collaborate with the data science team, plan projects, and develop analytics models. Your problem-solving skills and expertise in statistical analysis will be key in aligning data products with business objectives. Key Responsibilities: - Oversee the data scientists team and guide colleagues on new techniques - Work closely with data and software engineers to deploy scalable technologies - Lead the conception, planning, and prioritization of data projects - Build analytic systems and predictive models - Explore new ...

SUMMARY Summary We are seeking experienced Palantir Foundry Professionals to join our team in Bangalore. The ideal candidate will have a strong background in Palantir Foundry, with a minimum of 5 years of relevant experience. This role requires a deep understanding of data integration, application development, and cloud-based deployment. The successful candidate will be responsible for troubleshooting and optimizing application performance, and will work closely with our team to deliver high-quality solutions. Responsibilities Develop and implement data integration solutions using Palantir Foundry Design and deploy cloud-based applications using Agile methodologies Troubleshoot and optimize ...

Work Location: Pan India Experience: 12+yrs Responsibility: Provide top-quality solution design and execution Provide support in defining the scope and sizing of work Align the organizations big data solutions with their Client initiatives as requested Engage with clients to understand strategic requirements Responsible for translating business requirements into technology solutions Work with domain experts to put together a delivery plan with and stay on track Utilize Big Data technologies to design, develop, and evolve scalable and fault-tolerant distributed components Organize all meetings with customers and ensure prompt resolution of gaps and roadblocks Stay current on latest technology...

Greetings from Garmin! We are hiring Senior Database Administrator (Big Data Admin). Please go through the below JD. About Company: Garmin Ltd. is an American multinational technology company founded in 1989 by Gary Burrell and Min Kao in Lenexa, Kansas, United States, with headquarters in Olathe, Kansas. Since 2010, the company is incorporated in Schaffhausen, Switzerland. The company specializes in GPS technology for automotive, aviation, marine, outdoor, and sport activities. At Garmin, we like to work hard and play hard. It comes easy when you work on cool products with hard-working individuals who share the same passion. We make products that are engineered on the inside for life on the...

As a Data Ops Capability Deployment Analyst at Citi, your role will involve applying your in-depth disciplinary knowledge to contribute to the development of new solutions and frameworks within the Enterprise Data function. You will be responsible for performing data analytics and data analysis across different asset classes, as well as building data science capabilities within the team. You will work closely with the wider Enterprise Data team to deliver business priorities and collaborate with the front to back leads. Key Responsibilities: - Hands-on experience with a data engineering background and a thorough understanding of Distributed Data platforms and Cloud services. - Sound understa...

Job Summary Candidate should have 5+ years of Banking domain project experience in Hadoop technologies. Required skill sets: Technical: Very good knowledge in Big Data Hadoop architecture, HDFS, Hive, Spark, Scala, Yarn, Zookeeper, Unix shell scripts, Control-M, DevOps. Functional: Work experience in Banking domain projects Candidate should have work experience agile projects and experience to handle multiple stakeholders and work with business users to effectively gather requirements and preparing FSD, TSD, etc. Qualifications Education-BE/ME in Computer Science Certifications-Any hadoop related Languages-English Skills And Experience Hadoop technologies: Big Data Hadoop architecture, HDFS,...

Job Description Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD Responsibilities Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD Qualifications B.Tech

Hiring for Product Based organization Summary: The Engineer will be responsible for designing, deploying, and automating AWS Infrastructure based on customer requirements to deliver analytical solutions. This role will work closely with Managers, Clients, and Managed Services teams. Role : Senior Platform Engineer Location : Bengaluru/Pune Total Experience: 5+ years Relevant Experience: 3+ years Responsibilities of the Role: Develop automation-first solutions for cloud data services provisioning, integrating IAM, networking, security policies as code, and observability from the ground up. Collaborate with developers and internal stakeholders to deliver automated, cloud-native, and AI-ready s...

Our client is a global commerce leader where you can influence how the world buys, sells, and gives. Youll be part of a work culture thats been genuinely committed to diversity and inclusion since its founding over twenty five years ago. Here, you can be yourself, do your best work along with a team of professionals, and have a meaningful impact on people across the globe. We seek people with drive, ideas, and a passion for helping small businesses succeed to help. About the team: We are the AI Platform Team, providing highly available, scalable, and automated machine learning and big data infrastructure for researchers and data scientists globally. We are looking for a motivated, self-relia...

To ensure efficient and effective functioning of Coating plant in shifts and proper maintenance of machines in the plant Should have Knowledge of PLC, AC / DC Drvies (GE Fanuc / siemens / schneider), Induction Unit (Electrotherm / Inductotherm), HT & LT Braker Maintenance (Schneider / L&T), AC & DC Motor Maintenance, Temperature Controller, Switchgear, Starters, SAP Knowledge Industrial Safety - Proper Housekeeping of departnental Electrical systems & spares as per OHSAS & EMS standard. Maximizing the machine uptime - Attend breakdowns in time. Analyze the breakdown & take corrective action for repetitive/non repetitive breakdowns. Documentations as per SOP (maintenance). In-house repairing ...

Role Overview: As a Strong Data Architect, Lead Data Engineer, Engineering Manager / Director in our company, you will play a crucial role in designing and implementing data engineering systems from the early stages. Your expertise in large data technologies and hands-on tech development will be instrumental in creating scalable and maintainable data architectures. Key Responsibilities: - Possess 10+ years of experience in data engineering roles, with at least 2+ years in a leadership position - Demonstrate 7+ years of hands-on tech development experience with Java (highly preferred) or Python, Node.JS, GoLang - Showcase recent 4+ years of experience with high-growth product startups, specif...

Practice Overview : Skill/Operating Group Technology Consulting Level Consultant Location Gurgaon/Mumbai/ Bangalore Travel Percentage Expected Travel could be anywhere between 0-100% Why Technology Consulting The Technology Consulting business within Capability Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. As technology rapidly evolves, it's more important than ever to have an innovation advisor who can create a new vision or put one into placeto solve the client's toughest business problems. Specialize in management or technology consulting to transform the world's leading organiza...

Our client is a global commerce leader where you can influence how the world buys, sells, and gives. Youll be part of a work culture thats been genuinely committed to diversity and inclusion since its founding over twenty five years ago. Here, you can be yourself, do your best work along with a team of professionals, and have a meaningful impact on people across the globe. We seek people with drive, ideas, and a passion for helping small businesses succeed to help About the team: We are the AI Platform Team! We are looking for a highly motivated, self-reliant, experienced SRE and customer support engineer who is passionate about driving major transformation within the AI organization. This r...

Strong verbal and written communication skills Excellent analytical and problem-solving skills Hands-on and good understanding of DevOps tools and automation frameworks Demonstrated hands-on experience with DevOps techniques, building continuous integration solutions using Jenkins, Git, Maven, Nexus Experience with containerization orchestration using Docker, Docker Compose, Kubernetes Experience with configuration management tools such as Ansible, Chef, or Puppet Proficiency in at least one scripting language: Batch/PowerShell Scripting, Shell Scripting, NSIS Scripting Experience working in UNIX, Linux, and/or Windows environments Hands-on experience with cloud technology (AWS/Azure/GCP) Ex...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...