895 Hdfs Jobs - Page 6

Skill: Azure Databricks Experience: 4 to 9 years Location: AIA Chennai Job Summary We are seeking a Sr. Developer with 6 to 10 years of experience to join our team. The ideal candidate will have expertise in Databricks SQL Delta Lake Workflows and PySpark. This role involves working in a hybrid model during day shifts focusing on developing and optimizing data solutions. Experience in Research and Development Manufacturing or Supply Chain is a plus. Responsibilities Develop and optimize data solutions using Databricks SQL to enhance data processing capabilities. Implement and manage Databricks Delta Lake to ensure efficient data storage and retrieval. Design and automate workflows using Data...

Skill: Snowflake+ DBT Experience: 6 to 12 years Location: AIA Chennai Job Summary We are seeking a highly skilled Sr. Developer with 5 to 9 years of experience in Python and Snowflake with DBT technologies. The ideal candidate will work in a hybrid model focusing on developing and optimizing data solutions using Snowflake. This role requires expertise in Snowflake Tasks Streams SQL and Snowpipe ensuring efficient data management and processing. Responsibilities Develop and maintain scalable data solutions using Python and Snowflake technologies to support business objectives. Collaborate with cross-functional teams to design and implement data architecture that enhances data accessibility an...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

Must-Have 5+ years of hands on experience managing a multi-node Hadoop cluster (preferably Cloudera distribution) Python or Shell scripting) 5+ years hands on experience administering HDFS, YARN, Hive, HBase, Impala services for stability and scalability Strong understanding of core Hadoop concepts and design principles Strong experience integrating Hadoop environments with Active Directory / Kerberos services and 3rd party tools Strong understating of Linux Operating System concepts, memory, CPU, storage and networks Experience in at least one scripting language (Shell, Python, Perl or other) Familiarity with configuration management and deployment tools Ability to proactively identify, tro...

Minimum of 5 Years of Mongo DBA Architect, Strong Internals, Aggregation framework, Indexing and experience with MongoDB Atlas Minimum of 5+ Years of Hadoop DBA Architect, experience with Hadoop distributions (Cloudera, Hortonworks, or Apache) and Strong knowledge of Hadoop ecosystem components: HDFS, YARN, Hive, HBase, Spark, Oozie, Zookeeper, etc Minimum of 2 years experience with open source or cloud-based databases Recent experience with Apache Hadoop versions 3 -4 and 3 -3Proficiency in Hadoop ecosystem tools like Pig, Hive, HBase, and OozieExperience integrating MongoDB with big data ecosystems (Kafka, Spark, etc ) is a plus Recent experience with MariaDB, PostgreSQL, Snowflake, SQL Se...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Principal Apache Hadoop Engineer at BairesDev We are seeking a Principal Apache Hadoop Engineer with deep expertise in big data ecosystem, HDFS ...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

As a Delivery Consultant, you will work closely with IBM clients and partners to design, deliver, and optimize IBM Technology solutions that align with your clients' goals. In this role, you will apply your technical expertise to ensure world-class delivery while leveraging your consultative skills such as problem-solving issue- / hypothesis-based methodologies, communication, and service orientation skills. As a member of IBM Technology Expert Labs, a team that is client focused, courageous, pragmatic, and technical, you'll collaborate with clients to optimize and trailblaze new solutions that address real business challenges. If you are passionate about success with both your career and so...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform * Responsibilities: * Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS * Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform * Experience in developi...

What you will be doing: Service Management: Administer, configure, and maintain Hadoop ecosystem services within both Cloudera Data Platform (CDP) and Cloudera Distributed Hadoop (CDH), including HDFS, YARN, Hive, Spark, Ranger, Sentry, Kerberos security, KMS, KTS, Zookeeper, and more. Job Scheduling and Workflow Management : Utilize Oozie, Zena, Airflow to schedule and orchestrate complex data jobs and workflows, meeting business requirements efficiently. This includes day-end SLA jobs to ensure timely data processing. Security and Access Control: Manage security measures such as Kerberos authentication, safeguarding Hadoop clusters and data in both CDP and CDH environments. Implement and m...

Position Overview Job Title: Senior Data Engineer, AVP Location: Pune, India Role Description Engineer who would be working with existing team, responsibilities would be to understand requirements, analyze and refine stories, participating in designing solutions, implement them, test them and take it till production Use TDD, write clean code and refactor constantly. Make sure we are building the thing right. Write code and write it well. Be proud to call yourself a programmer. Use BDD techniques, collaborating closely with users, analysts, developers and other testers. Make sure we are building the right thing. Be ready to work on a range of technologies and components, including services an...

We are looking for a skilled Big Data Scala Developer with 6-15 years of experience to join our team on a contract-to-hire basis in Chennai and Pune. Roles and Responsibility Design, develop, and implement large-scale data processing systems using Scala and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain high-quality, efficient, and scalable code. Troubleshoot and resolve complex technical issues. Participate in code reviews and contribute to improving overall code quality. Work with Cloudera on open issues that would result in cluster configuration changes. Job Requirements Strong experience in programming languages such a...

Project Role :Custom Software Engineer Project Role Description :Design, build and configure applications to meet business process and application requirements. Must have skills :Apache Hadoop Good to have skills :Python (Programming Language) Minimum 3 year(s) of experience is required Educational Qualification :15 years full time education Summary:As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications are opt...

We are looking for a skilled Big Data Scala Developer with 6-15 years of experience to join our team on a contract-to-hire basis in Chennai and Pune. Roles and Responsibility Design, develop, and implement large-scale data processing systems using Scala and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain high-quality, efficient, and scalable code. Troubleshoot and resolve complex technical issues. Participate in code reviews and contribute to improving overall code quality. Work with Cloudera on open issues that would result in cluster configuration changes. Job Requirements Strong experience in programming languages such a...

We are looking for a skilled Big Data Scala Developer with 6-15 years of experience to join our team on a contract-to-hire basis in Chennai and Pune. Roles and Responsibility Design, develop, and implement large-scale data processing systems using Scala and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain high-quality, efficient, and scalable code. Troubleshoot and resolve complex technical issues. Participate in code reviews and contribute to improving overall code quality. Work with Cloudera on open issues that would result in cluster configuration changes. Job Requirements Strong experience in programming languages such a...

You will join a team of highly skilled hadoop engineers who are responsible for delivering Acceldata's support services in vendor-agnostic environments. As a Site Reliability Engineer , you will actively learn from experienced team members, contributing to improving the availability, scalability, performance, and reliability of our products and our customers data lake environments. You will be expected to listen actively to customer concerns and demonstrate empathy in providing solutions. This is an exciting opportunity to grow your technical, business, and interpersonal skills in a dynamic and collaborative environment. What makes you the right fit for this position 5+ years of experience i...

About Position: We're looking for an experienced skilled Support Data Engineer with 57 years of experience in Azure , Databricks platform and relational database related task. He/She should be able to take ownership of incidents and resolve incidents as per the SLA. We are looking for someone who has technical proficiency in Data engineering support. Role: Support Data Engineer Location: Pune Experience: 5+ Years Job Type: Full Time Employment What You'll Do: Databricks: Proficient in building scalable data pipelines and notebooks using pyspark /python and SQL's Azure Data Factory (ADF): Experience in orchestrating data workflows and integrating diverse data sources. Good hands on the Pyspar...

Maintain and decommission legacy HDP/Cloudera clusters during transition. Manage HDFS/YARN/Hive/HBase Plan data export/import Migrate Ranger/Atlas policies Advise on transition architecture Optimize clusters Hadoop, Hive, HDFS, YARN, Ranger, Atlas, Kerberos, Linux Scripting

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Python (Programming Language) Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project m...

Job Summary: The Associate Data Engineer works on different projects of data engineering to support the use cases, data ingestion pipeline and identify potential process or data quality issues. For the SaaS Integration team, the resource will contribute to the design, development, and maintenance of API integrations across enterprise SaaS platforms including Reltio, OneTrust, and Adobe CDP. This will include developing and maintaining RESTful APIs to integrate internal systems with Reltio (MDM), OneTrust (Privacy & Compliance), and Adobe CDP (Customer Data Platform), participating in technical design reviews and contributing to solution architecture discussions and collaborating with product...

Job Overview: As a Data Science Developer, you will collaborate with data engineers, analysts, and business stakeholders to develop machine learning models, predictive analytics, and data pipelines. You will work on end-to-end data science solutions, from data collection and preprocessing to model deployment and visualization. Key Responsibilities: Design, develop, and deploy machine learning and statistical models. Collect, clean, and preprocess large datasets from multiple sources. Develop and maintain data pipelines for efficient data processing. Collaborate with cross-functional teams to define business requirements and translate them into data-driven solutions. Conduct exploratory data ...

Job Summary The Big Data Engineer is responsible for designing, developing, and maintaining large-scale data processing systems. The role includes building efficient data pipelines, working with distributed technologies, and ensuring data availability, reliability, and performance for analytics and business use cases. Key Responsibilities 1. Data Pipeline Development Build scalable ETL/ELT pipelines for ingesting and transforming large datasets. Develop batch and real-time data processing solutions using Apache Spark, Kafka, Hive, Flink , or similar tools. Optimize data workflows for performance and cost. 2. Big Data Ecosystem Ownership Work with Hadoop ecosystem components (HDFS, Hive, HBas...