603 Hdfs Jobs - Page 19

As a PySpark Data Engineer, you must have a minimum of 2 years of experience in PySpark. Strong programming skills in Python, PySpark, and Scala are preferred. It is essential to have experience in designing and implementing CI/CD, Build Management, and Development strategies. Additionally, familiarity with SQL and SQL Analytical functions is required, along with participation in key business, architectural, and technical decisions. There is an opportunity for training in AWS cloud technology. In the role of a Python Developer, a minimum of 2 years of experience in Python/PySpark is necessary. Strong programming skills in Python, PySpark, and Scala are preferred. Experience in designing and ...

Job Posting: Support Analyst Big Data & Application Support (Chennai) Location: Chennai, India (Prefers only chennai based candidates ) Experience: 2 to 5 years Employment Type: Full-Time | Hybrid Model Department: Digital Technology Services IT Digital Function: DaaS (Data as a Service), AI & RPA Support * Note : Only candidates with above criteria will be contacted for further process ! Role Overview We are looking for a Support Analyst to join our dynamic DTS IT Digital team in Chennai. In this role, you will support and maintain data platforms, AI/RPA systems, and big data ecosystems. You'll play a key part in production support, rapid incident recovery, and platform improvements, workin...

Requirement : Immediate or Max 15 days Job Description : Big Data Developer (Hadoop/Spark/Kafka) - This role is ideal for an experienced Big Data developer who is confident in taking complete ownership of the software development life cycle - from requirement gathering to final deployment. - The candidate will be responsible for engaging with stakeholders to understand the use cases, translating them into functional and technical specifications (FSD & TSD), and implementing scalable, efficient big data solutions. - A key part of this role involves working across multiple projects, coordinating with QA/support engineers for test case preparation, and ensuring deliverables meet high-quality st...

Data Engineering / Big Data part time Work from Home (Any where in world) Warm Greetings from Excel Online Classes, We are a team of industry professionals running an institute that provides comprehensive online IT training, technical support, and development services. We are currently seeking Data Engineering / Big Data Experts who are passionate about technology and can collaborate with us in their free time. If you're enthusiastic, committed, and ready to share your expertise, we would love to work with you! Were hiring for the following services: Online Training Online Development Online Technical Support Conducting Online Interviews Corporate Training Proof of Concept (POC) Projects Res...

Description: Hiring Data Engineer with AWS or GCP Cloud Requirements: Role Summary: The Data Engineer will be responsible for designing, implementing, and maintaining the data infrastructure and pipelines necessary for AI/ML model training and deployment. They will work closely with data scientists and engineers to ensure data is clean, accessible, and efficiently processed Required Experience: • 6-8 years of experience in data engineering, ideally in financial services. • Strong proficiency in SQL, Python, and big data technologies (e.g., Hadoop, Spark). • Experience with cloud platforms (e.g., AWS, Azure, GCP) and data warehousing solutions. • Familiarity with ETL processes and tools. • Kn...

Job Summary We are seeking a highly motivated Senior Data Engineer with expertise in designing, building, and securing data systems. The ideal candidate will have a strong background in data engineering, security compliance, and distributed systems, with a focus on ensuring adherence to industry standards and regulatory requirements. Location: Bangalore Experience: 4 to 13 Years Must Have: Informatica BDM, Oozie Scheduling, Hive, HDFS Key Responsibilities Design, implement, and maintain secure data systems, including wrapper solutions for components with minimal security controls, ensuring compliance with bank standards. Identify security design gaps in existing and proposed architectures an...

Streaming data Technical skills requirements :- Experience- 5+ Years Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven syste...

Key Responsibilities Training & Mentoring Design, structure, and deliver hands-on technical training programs tailored to real-world client projects. Develop and maintain training materials, assignments, and project-based learning paths for: .NET technologies (C#, ASP.NET MVC, Razor Pages, Blazor, REST APIs) ReactJS (SPA fundamentals and advanced features) Python with Machine Learning foundations Data infrastructure tools like HDFS, Kafka, RabbitMQ, MQTT Embedded C and DLMS protocols Mentor and guide new hires and junior developers through technical problem-solving, code reviews, and best practices. Conduct code walkthroughs, mock evaluations, and project reviews. Evaluate trainee performanc...

We're Hiring | Big Data Engineer (6-8 Yrs) | Bangalore | Hybrid (3 days/week office) Location: Bangalore (Hybrid 3 days/week in office) Experience: 6 to 8 years Joiners: Immediate to Max 2 Weeks Notice Period ONLY Must-Have Skills: Strong hands-on experience in Hadoop ecosystem , HDFS , Hive , Spark with Scala Familiarity with Oozie , ScalaTest Expertise in performance tuning and debugging using Spark UI and YARN logs Good understanding of CI/CD processes , Unit Testing , GitHub , Maven , and Nexus To Apply: Send your resume to vijay.s@xebia.com with the following details: Full Name: Total Experience: Current CTC: Expected CTC: Current Location: Preferred Location: Notice Period / Last Worki...

Role Overview: Synechron is looking for an experienced Scala Spark Developer to join our advanced analytics and big data engineering team in Pune. The ideal candidate should have a strong background in data processing using Scala and Spark in distributed environments, with the ability to handle large-scale data pipelines for complex business needs. Key Responsibilities: Design and develop scalable big data processing solutions using Scala and Apache Spark. Optimize data workflows and Spark jobs for performance and cost-efficiency. Work with large datasets across structured and unstructured sources. Collaborate with data engineers, analysts, and stakeholders to build end-to-end solutions. Mai...

Mandatory Skills: PySpark Big Data Technologies Role Overview: Synechron is hiring a skilled PySpark Developer for its advanced data engineering team in Pune. The ideal candidate will have strong experience in building scalable data pipelines and solutions using PySpark, with a solid understanding of Big Data ecosystems. Key Responsibilities: Design, build, and maintain high-performance batch and streaming data pipelines using PySpark. Work with large-scale data processing frameworks and big data tools. Optimize and troubleshoot PySpark jobs for efficient performance. Collaborate with data scientists, analysts, and architects to translate business needs into technical solutions. Ensure best ...

Role & responsibilities Key Skills: Bigdata Frameworks like: Hadoop, Sprak, Pyspark, Hive - Mandatory

Description: To work on Data Analytical to triage and investigate data quality and data pipeline exceptions and reporting issues. Requirements: This role will support Data Operations and Reporting related projects but also will be helping with other projects as well if needed. In this role, you will leverage your strong analytical skills to triage and investigate data quality and data pipeline exceptions and reporting issues. The ideal candidate should be able to work independently and actively engage other functional teams as needed. This role requires researching transactions and events using large amounts of data. Technical Experience/Qualifications: • At least 5 years of experience in so...

Key Responsibilities Develop & Deliver: Build applications/features/components as per design specifications, ensuring high-quality code adhering to coding standards and project timelines. Testing & Debugging: Write, review, and execute unit test cases; debug code; validate results with users; and support defect analysis and mitigation. Technical Decision Making: Select optimal technical solutions including reuse or creation of components to enhance efficiency, cost-effectiveness, and quality. Documentation & Configuration: Create and review design documents, templates, checklists, and configuration management plans; ensure team compliance. Domain Expertise: Understand customer business domai...

Job Summary We are seeking a highly motivated Senior Data Engineer with expertise in designing, building, and securing data systems. The ideal candidate will have a strong background in data engineering, security compliance, and distributed systems, with a focus on ensuring adherence to industry standards and regulatory requirements. Key Responsibilities Design, implement, and maintain secure data systems, including wrapper solutions for components with minimal security controls, ensuring compliance with bank standards. Identify security design gaps in existing and proposed architectures and recommend enhancements to strengthen system resilience. Develop and enforce security controls for dat...

Responsibilities: Lead technical projects end-to-end right from design to release, owning the complete delivery. Handle a large Portfolio and account P&L or experience in handling cross market global rollouts, Provide Delivery Oversight including operational parameters. Guide the team to Design and build cloud solutions on Azure / AWS/ GCP, choosing the best-in-class services, tools, methodologies. Translating business requirements into technology specifications, architecture design, data modeling, build, release management, DevOps and stakeholder management including tech/IT teams Experience in one or more Agile and DevOps practices and tools such as Jenkins, JIRA, Confluence, Selenium, Son...

Major skillset: GCP , Pyspark , SQL , Python , Cloud Architecture , ETL , Automation 4+ years of Experience in Data Engineering, Management with strong focus on Spark for building production ready Data Pipelines. Experienced with analyzing large data sets from multiple data sources and build automated testing and validations. Knowledge of Hadoop eco-system and components like HDFS , Spark , Hive , Sqoop Strong Python experience Hands on SQL , HQL to write optimized queries Strong hands-on experience with GCP Big Query , Data Proc , Airflow DAG , Dataflow , GCS , Pub/sub , Secret Manager , Cloud Functions , Beams . Ability to work in fast passed collaborative environment work with various sta...

experience in big data development and data engineering. - Proficiency in Java and experience with Apache Spark. - Experience in API development and integration. - Strong understanding of data engineering principles and big data concepts. Required Candidate profile - Familiarity with big data tools such as Hadoop, HDFS, Hive, HBase, and Kafka. - Experience with SQL and NoSQL databases. - Strong communication and collaboration skills

Roles and Responsibilities Role- Bigdata Tech Lead Location-Bangalore,Hyderabad,Pune and Chennai Exp Range- 8 years to 15 years Mandatory skills- Pyspark and Bigdata Design, develop, and maintain large-scale data processing pipelines using Hadoop, PySpark, Hive, and HDfS. Ensure scalability, reliability, and performance of big data systems by monitoring logs, troubleshooting issues, and implementing improvements. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time. Develop expertise in multiple programming languages such as Java or Python to support various projects. Participate in code reviews to ensure adherence to coding standards and ...

Role & responsibilities Data Engineer with Python, Spark, PySpark & HDFS: 9 Positions Preferred candidate profile

ISA Non captive RTH-Y Note: 1.This position requires the candidate to work from the office starting from day one clinet office. 2.Ensure that you perform basic validation and gauge the interest level of the candidate before uploading their profile to our system. 3.Candidate Band will be count as per their relevant experience. We will not entertain lesser experience profile for higher band. 4. Candidate full BGV is required before onboarding the candidate. 5. If required will regularize the candidate after 6months. Hence 6 months NOC is required from the DOJ. Mode of Interview: Face to Face (Mandatory). **JOB DESCRIPTION** Total Years of Experience : 7-10 Years Relevant Years of Experience : ...

ISA NC, Candidate full BGV is required before on-boarding And need to provide NOC after 90days Post DOJ. Education qualification: B.tech or BE. Candidate should appear for the client interview at Mumbai client office. RTH-Y Note: 1. This position requires the candidate to work from the office starting from day one. 2. Ensure that you perform basic validation and gauge the interest level of the candidate before uploading their profile to our system. 3. Candidate Band will be count as per their relevant experience. We will not entertain lesser experience profile for higher band. Mode of Interview: Face to Face (Mandatory). Mandatory Skills : Strong hands-on with Cloudera Manager, Ambari, HDFS,...

We are seeking an experienced PySpark Developer / Data Engineer to design, develop, and optimize big data processing pipelines using Apache Spark and Python (PySpark). The ideal candidate should have expertise in distributed computing, ETL workflows, data lake architectures, and cloud-based big data solutions. Key Responsibilities: Develop and optimize ETL/ELT data pipelines using PySpark on distributed computing platforms (Hadoop, Databricks, EMR, HDInsight). Work with structured and unstructured data to perform data transformation, cleansing, and aggregation. Implement data lake and data warehouse solutions on AWS (S3, Glue, Redshift), Azure (ADLS, Synapse), or GCP (BigQuery, Dataflow). Op...

Job Title: Senior Engineer | Java and Big Data Company Name: Impetus Technologies Job Description: Impetus Technologies is seeking a skilled Senior Engineer with expertise in Java and Big Data technologies. As a Senior Engineer, you will be responsible for designing, developing, and deploying scalable data processing applications using Java and Big Data frameworks. Your role will involve collaborating with cross-functional teams to gather requirements, developing high-quality code, and optimizing data processing workflows. You will also mentor junior engineers and contribute to architectural decisions to enhance the performance and scalability of our systems. Key Responsibilities: - Design, ...

Responsibilities: * Design data solutions using Java, Python & Apache Spark. * Collaborate with cross-functional teams on Azure cloud projects. * Ensure data security through Redis caching and HDFS storage.