575 Hdfs Jobs

Scala Developer Experience : 7-12 Years Location : Mumbai Desired Competencies (Technical/Behavioral Competency) Must-Have Minimum 5+ years of experience in development of Spark Scala 2. Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as with Hadoop Bigdata components like HDFS, Spark, Hive Parquet File format, YARN, MapReduce, Sqoop Good Experience in writing and optimizing Spark Jobs, Spark SQL etc. Should have worked on both batch and streaming data processing. 4. Experience in writing and optimizing complex Hive and SQL queries to process huge data. good with UDFs, tables, joins, Views etc 5. Experience in debugging the Spark cod...

As an experienced professional with over 3 years of experience in Big Data & data related technologies, you possess expert level understanding of distributed computing principles. Your expertise lies in Apache Spark and you have hands-on programming skills in Python. You are proficient in Hadoop v2, Map Reduce, HDFS, and Sqoop. Additionally, you have experience in building stream-processing systems using technologies like Apache Storm or Spark-Streaming. Key Responsibilities: - Hands-on experience in Apache Spark - Proficient in programming with Python - Strong knowledge of Hadoop v2, Map Reduce, HDFS, and Sqoop - Building stream-processing systems using Apache Storm or Spark-Streaming - Wor...

As a Data Ops Capability Deployment Analyst at Citi, you will be a seasoned professional contributing to the development of new solutions and techniques for the Enterprise Data function. Your primary role will involve performing data analytics and analysis across various asset classes, as well as building data science capabilities within the team. You will collaborate closely with the Enterprise Data team to deliver on business priorities. Key Responsibilities: - Hands-on experience with data engineering and a strong understanding of Distributed Data platforms and Cloud services. - Sound knowledge of data architecture and integration with enterprise applications. - Research and evaluate new ...

As a Data Analytics Intermediate Analyst at our company, you will be a developing professional dealing with most problems independently, integrating specialty area knowledge with industry standards, and contributing to the team's objectives. You will apply your analytical thinking skills along with data analysis tools and methodologies to make accurate judgments and recommendations based on factual information. Your attention to detail will be crucial as you analyze information and communicate it effectively to others. Your communication and diplomacy skills will be essential to exchange complex/sensitive information and provide quality service to ensure team effectiveness. **Responsibilitie...

We are looking for lead or principal software engineers to join our Data Cloud team. Our Data Cloud team is responsible for the Zeta Identity Graph platform, which captures billions of behavioural, demographic, environmental, and transactional signals, for people-based marketing. As part of this team, the data engineer will be designing and growing our existing data infrastructure to democratize data access, enable complex data analyses, and automate optimization workflows for business and marketing operations. Job Description: Essential Responsibilities: As a Lead or Principal Data Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and ba...

Practice Overview : Skill/Operating Group Technology Consulting Level Consultant Location Gurgaon/Mumbai/ Bangalore Travel Percentage Expected Travel could be anywhere between 0-100% Why Technology Consulting The Technology Consulting business within Capability Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. As technology rapidly evolves, it's more important than ever to have an innovation advisor who can create a new vision or put one into placeto solve the client's toughest business problems. Specialize in management or technology consulting to transform the world's leading organiza...

Position Summary Role Value Proposition: Work and collaborate with a nimble, autonomous, cross-functional team of makers, breakers, doers, and disruptors who love to solve real problems and meet real customer needs. You will be using cutting-edge technologies and frameworks to analyze data, to help create the pipeline, and collaborate with the data science team to enable the innovative work in machine learning and AI. Eagerness to learn new technologies on the fly and ship to production Knowledge in data science is a plus More than just a job we hire people who love what they do! Job Responsibilities Building and Implementing data ingestion and curation process developed using Big data tools...

Designation: Senior Engineer Data Science and Engineering Location: 22nd Floor Tower C Building No - 5 DLF Epitome Gurgaon Haryana. Tbo.com(www.tbo.com) TBO is a global platform that aims to simplify all buying and selling travel needs of travel partners across the world. The proprietary technology platform aims to simplify the demands of the complex world of global travel by seamlessly connecting the highly distributed travel buyers and travel suppliers at scale. The TBO journey began in 2006 with a simple goal to address the evolving needs of travel buyers and suppliers, and what started off as a single product air ticketing company, has today become the leading B2A (Business to Agents) tr...

Skills- Databricks+ Pyspark Experience: 4 to 13 years Location: AIA-Pune We are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets. Key Responsibilities: Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on Databricks. Data Architecture & Engineering: Work on distributed computing solutions, optimize Spark jobs, and build efficient data models. Performance & Cost Optimization: Fine-tune Spark configurations, optimize Databricks clusters, and reduce compute/storage costs. Collaboration: Work closely with Data Scientists, Analysts, and DevOps tea...

Skills- Pyspark Experience: 6 to 9 years Location: AIA-Pune Responsibilities Develop and maintain scalable data pipelines using Python and PySpark. Collaborate with data engineers and data scientists to understand and fulfil data processing needs. Optimize and troubleshoot existing PySpark applications for performance improvements. Write clean, efficient, and well-documented code following best practices. Participate in design and code reviews. Develop and implement ETL processes to extract, transform, and load data. Ensure data integrity and quality throughout the data lifecycle. Stay current with the latest industry trends and technologies in big data and cloud computing.

Skills-GCP Data Engineer Experience: 6 to 12 years Location: AIA-Gurgaon Hands-on experience with GCP services, specifically BigQuery, Cloud Storage, and Composer for data pipeline orchestration Proficiency in Databricks platform with PySpark for building and optimizing large-scale ETL/ELT processes Expertise in writing and tuning complex SQL queries for data transformation, aggregation, and reporting on large datasets Experience integrating data from multiple sources such as APIs, cloud storage, and databases into a central data warehouse Familiarity with workflow orchestration tools like Apache Airflow or Cloud Composer for scheduling, monitoring, and managing data jobs Knowledge of versio...

We're looking for a Data Analyst with strong experience in Palantir Foundry and Python to design scalable data solutions, automate workflows, and enable data-driven decision-making. The role involves working closely with business stakeholders to translate requirements into actionable insights through Foundry-based analytics and GenAI integrations. Key Responsibilities: Design, develop, and manage data pipelines, reports, and dashboards in Palantir Foundry. Build and optimize Python transforms for data manipulation and automation. Integrate GenAI models using Foundry AIP and develop effective prompt logic aligned with business goals. Create and maintain reusable SQL scripts, ensuring data qua...

Role Overview: At Improzo, we are looking for a highly skilled Data and Reporting Developer (Improzo Level - Associate) to join our dynamic team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using big data technologies. This exciting opportunity is suitable for an individual with a strong technical background and a passion for working with large datasets to deliver high-quality solutions. Key Responsibilities: - Design, develop, and maintain scalable data pipelines and big data applications. - Work with distributed processing frameworks (e.g., Apache Hadoop, Apache Spark) to process and analyze large datasets....

Senior Data Engineer - Palantir Foundry We are looking for a Senior Data Engineer with strong hands-on experience in Palantir Foundry to design, build, and optimize scalable data pipelines and analytics solutions. Required Skills & Qualifications: 5+ years of experience in data engineering, analytics, or software development. Proven expertise in Palantir Foundry (pipelines, ontology, fusion, transformations). Strong programming skills in Python and SQL . Experience with AWS , data lakes, and cloud data warehouses. Familiarity with ETL processes, CI/CD pipelines, and Git. Strong problem-solving and debugging skills. Excellent communication and teamwork abilities. Required Skills Python,AWS,Pa...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Apache Hadoop Developer at BairesDev We are seeking an Apache Hadoop Developer with expertise in big data ecosystem, HDFS architecture, MapReduc...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...

Role & responsibilities Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integration testing and a fully automated build and deployment process as part of DevOps tooling. Must have the ability to understand and develop the logical flow of applications on technical code level. Strong interpersonal skills and ability to work in a team and in global environments. Should be p...