895 Hdfs Jobs - Page 13

Role Overview: As a Hadoop Admin at Wipro Limited, you will be responsible for the administration and management of Hadoop and HDFS, with a focus on building and managing data pipelines. Your role will involve strong expertise in Unix/Linux server setup, maintenance, and troubleshooting skills, along with scripting languages for automation tasks. Additionally, familiarity with virtualization technologies and ML pipelines in NVIDIA GPU clusters will be beneficial. Key Responsibilities: - Instrumental in understanding product/software requirements and designing solutions by studying information needs, systems flow, and work processes - Perform coding and ensure optimal software/module developm...

Experience: 6-9 years Location: Chennai, Bengaluru Notice Period: Immediate to 30 Days Job Description Technical Skills: Strong hands-on experience with Apache Spark, PySpark, and Scala. Proficiency in SQL and data manipulation techniques. Experience with Hadoop ecosystem tools (HDFS, Hive, YARN, etc.). Knowledge of cloud platforms (AWS, Azure, or GCP) and managed Spark services (e.g., Databricks, EMR, Glue). Familiarity with data lakehouse concepts, Delta Lake, or Iceberg is a plus. Experience with CI/CD, Git, and workflow orchestration tools (Airflow, Oozie, etc.). Understanding of data modeling, partitioning, and schema evolution strategies. Exposure to containerization (Docker, Kubernete...

We are Hiring Big Data Engineer/Data Engineer with SCALA & Spark Location: Nagpur NP: 60 days & Less Experience: 5-9 Years Skills: Preferred Skillset: Spark, Scala, Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.), SQL, Jenkins, and Unix Commands . Experience in Big data technologies , real time data processing platform( Spark Streaming - Kafka ) experience in Cloudera would be an advantage . Consistently demonstrates clear and concise written and verbal communication. Ability to multi-task and weekend support for production releases. Hands-on experience on Unix Command. Strong foundation in computer science fundamentals: Data structure, Algorithms, and coding. Experienced in P...

Job profiles - Big Data Developer (Senior Manager) Experience - 7-10 yrs Location - Pune Job Description We are seeking a highly skilled Lead Big Data Developer to architect, drive, and optimize large-scale data engineering solutions in a modern, modular, and highly automated environment. The ideal candidate will have extensive experience in building and leading end-to-end big data pipelines, with deep expertise in distributed systems, data transformation frameworks (such as DBT), and advanced data engineering best practices. Key Responsibilities Technical Leadership: Lead the design, development, and deployment of big data solutions using Java, Apache Spark, and modern orchestration tools. ...

Educational Requirements Bachelor of Engineering,BTech,BCA,MTech,MCA Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to provide best fit architectural solutions for one or more projects. You would also provide technology consultation and assist in defining scope and sizing of work You would implement solutions, create technology differentiation and leverage partner technologies. Additionally, you would participate in competency development with the objective of ensuring the best-fit and high quality technical solutions. You would be a key contributor in creating thought leadership within th...

Role & responsibilities Experience working with NoSQL in at least one of the data stores - HBase, Cassandra, MongoDB Experience in HDFS, Hive, Impala Must have strong technical skills in Neo4j. Experience is schedulers like Airflow, Nifi etc. Experienced in Hadoop clustering and Auto scaling. Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data acquisition, transformation, and analysis. Define and develop client specific best practices around data management within a Hadoop environment on Azure cloud Monitoring health, tuning, and growth of such on-premises and cloud databases Experience in managing security and maintenance o...

As a Python Senior Developer, you will be responsible for the following: - Must have 5+ years of experience in Python programming. - Extensive experience in handling complex projects. - Good leadership skills and ability to lead projects independently. - Good communication and teamwork skills. - Good analytical skills and proficiency in Web-based development technologies. - Expertise in Agile development methodology. - Strong problem-solving skills. - Strong knowledge in Design patterns, Security, Performance tuning, and App monitoring. - Experience in API Design, security patterns, Microservices architecture. - Must have experience in building scalable, low latency web applications using Fl...

As the Head of Technology - New Initiatives at Matrimony.com, you will play a pivotal role in driving the convergence of cutting-edge technology and matrimonial services to spearhead the next groundbreaking project. You will lead a minimum of 1 product tech team towards achieving tech supremacy. Your responsibilities will include: - Strategic Tech Road mapping: - Architect and implement a technological vision aligned with industry advancements. - Ensure seamless alignment of technology strategies with business benchmarks. - Product Engineering: - Pilot the development and deployment of the flagship min1 product, focusing on scalability and efficiency. - Oversee the product's lifecycle from p...

As a Senior Big Data Developer with expertise in PySpark and the Apache Big Data ecosystem, your role will involve designing, developing, and optimizing scalable data pipelines. You will collaborate with cross-functional teams to deliver reliable, high-performance data solutions. Your responsibilities will include: - Building and optimizing large-scale data processing pipelines using PySpark - Working with Apache Hadoop, Hive, HDFS, Spark, and other tools for managing big data workloads - Handling ETL development, data ingestion, transformation, and integration from various sources - Collaborating with Data Scientists, Analysts, and business stakeholders to deliver data-driven solutions - En...

Role Overview: As a PySpark ETL Lead, you will be responsible for driving the data integration efforts in analytics and data warehousing projects. Your role will involve developing and managing PySpark scripts, creating ETL workflows, and ensuring efficient data processing and integration across systems. Key Responsibilities: - Lead ETL development using PySpark - Design and schedule data workflows - Optimize data processing performance - Collaborate with cross-functional teams Qualifications Required: - Strong in PySpark and Python for ETL and data processing - Experience with ETL pipeline design, data integration, and data warehousing - Proficient in SQL and working with large datasets - F...

As a MySql Database Admin at Times Internet, your role will involve participating in the design, implementation, automation, optimization, and ongoing operational administration tasks for backend systems running on MySQL/MariaDB/MongoDB/RedisDB/Hadoop database infrastructure. You should be prepared to handle instant support and operational challenges related to infrastructure platforms running database services. Key Responsibilities: - Hands-on experience with MySQL/MariaDB RDBMS and related tools such as Percona Xtrabackup and Percona-tool-kit. - Proficiency in MongoDB NoSQL and Cache (RedisDB) datastores. - Experience working on both Private and Public IT Infrastructure clouds like AWS. - ...

Expert-level Python programming, software design & data structures. Exp. deploying ML/LLM models into production. Hands-on with MLOps/GenAIOps stacks (MLflow, Kubeflow, SageMaker, LangChain, AutoGen, LangGraph, CrewAI) & their real-world application. Required Candidate profile Sr.. Python Developer lead 8–10 yrs of exp. who can not only build, deploy scalable Python and ML/GenAI solutions lead & mentor a technical team. end-to-end project delivery, ensure high engineering

Senior Data Specialist Primary Skills MapReduce, HDFS, Spark - Pyspark, ETL Fundamentals, SQL, SQL (Basic + Advanced), Spark - Scala, Python, Data Warehousing, Hive, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures, Oozie Job requirements Bachelor's or Master's degree in Computer Science, Information Systems, or a related field. Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other relevant technologies. Good Hands on Exp with java Minimum of 6+ years of experience as a Data Engineer, with proven expertise in Python, Hive, and Spark. Advance SQL experience and working experience with Big data Hadoop ...

About ProcDNA ProcDNA is a global consulting firm. We fuse design thinking with cutting-edge technology to create game-changing Commercial Analytics and Technology solutions for our clients. We're a passionate team of 275+ across 6 offices, all growing and learning together since our launch during the pandemic. Here, you won't be stuck in a cubicle - you'll be out in the open water, shaping the future with brilliant minds. At ProcDNA, innovation isn't just encouraged; it's ingrained in our DNA. Ready to join our epic growth journey? What we are looking for Youll be driving the adoption of the latest technologies in our solutions, bringing in thought leadership to guide clients on complex dat...

• Big Data/Hadoop Experience particularly in ingesting data and implementing Data ingestion pipelines, SQOOP, HADOOP, HDFS, HIVE, IMPALA, Java, Scala, Spark • Scala is Mandatory

Job Title: Hadoop Administrator Location: Chennai, India Experience: 4+yrs of experience in IT, with At least 3+ years of experience with cloud and system administration. At least 3 years of experience with and strong understanding of big data technologies in Hadoop ecosystem Hive, HDFS, Map/Reduce, Flume, Pig, Cloudera, HBase Sqoop, Spark etc. Company: Smartavya Analytica Private limited is a niche Data and AI company. Based in Pune, we are pioneers in data-driven innovation, transforming enterprise data into strategic insights. Established in 2017, our team has experience in handling large datasets up to 20 PBs in a single implementation, delivering many successful data and AI projects acr...

Role & responsibilities Work as a project manager to Lead the design and evolution of our large-scale data lakehouse architecture, ensuring it is scalable, reliable, and cost-effective. Provide technical leadership and mentorship to data engineers and analysts. Collaborate closely with Software Engineers, and business stakeholders to understand requirements and deliver effective data solutions. Experience in data modelling, data mapping, data profiling and meta data management. Architect and tune high-performance data processing pipelines. Identify and resolve complex performance issues in distributed computing environments involving Spark execution, Hive query optimization, and Iceberg meta...

Hi Candidate, Opening for Data Scientist Location :Airoli (Navi Mumbai) Experience : 3+ yrs Key Responsibilities: Develop and maintain data pipelines using Hadoop ecosystem. Build and maintain Spark SQL applications for big data processing and analysis. Use Tableau to visualize data, create dashboards, and extract insights from Hadoop data. Optimize data processing performance, including query optimization and data storage strategies. Collaborate with other engineers and stakeholders to understand data requirements and deliver solutions. Troubleshoot and resolve issues related to data processing and visualization. Stay up to date with the latest technologies and best practices in the field. ...

Job Description: Job Description About Us At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work for our teammates around the world. We're devoted to being a diverse and inclusive workplace for everyone. We hire individuals with a broad range of backgrounds and experiences and invest heavily in our teammates and their families by offering competitive benefits to support their physical, emotional, and financial wel...

AIA-Pune Job Summary Data Engineer 4 to 9 Years Responsibilities Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation stored procedures etc. Experience writing data Pipelines using Python and PySpark. Relational data model / data schema design experience Understanding of DevOps and CI/CD in the context of Data Engineering Familiarity with working in a scrum environment and supporting scrum ceremonies Certifications Required Azure Pyspark Python

Scala Developer Experience : 7-12 Years Location : Mumbai Desired Competencies (Technical/Behavioral Competency) Must-Have Minimum 5+ years of experience in development of Spark Scala 2. Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as with Hadoop Bigdata components like HDFS, Spark, Hive Parquet File format, YARN, MapReduce, Sqoop Good Experience in writing and optimizing Spark Jobs, Spark SQL etc. Should have worked on both batch and streaming data processing. 4. Experience in writing and optimizing complex Hive and SQL queries to process huge data. good with UDFs, tables, joins, Views etc 5. Experience in debugging the Spark cod...

As an experienced professional with over 3 years of experience in Big Data & data related technologies, you possess expert level understanding of distributed computing principles. Your expertise lies in Apache Spark and you have hands-on programming skills in Python. You are proficient in Hadoop v2, Map Reduce, HDFS, and Sqoop. Additionally, you have experience in building stream-processing systems using technologies like Apache Storm or Spark-Streaming. Key Responsibilities: - Hands-on experience in Apache Spark - Proficient in programming with Python - Strong knowledge of Hadoop v2, Map Reduce, HDFS, and Sqoop - Building stream-processing systems using Apache Storm or Spark-Streaming - Wor...

As a Data Ops Capability Deployment Analyst at Citi, you will be a seasoned professional contributing to the development of new solutions and techniques for the Enterprise Data function. Your primary role will involve performing data analytics and analysis across various asset classes, as well as building data science capabilities within the team. You will collaborate closely with the Enterprise Data team to deliver on business priorities. Key Responsibilities: - Hands-on experience with data engineering and a strong understanding of Distributed Data platforms and Cloud services. - Sound knowledge of data architecture and integration with enterprise applications. - Research and evaluate new ...

As a Data Analytics Intermediate Analyst at our company, you will be a developing professional dealing with most problems independently, integrating specialty area knowledge with industry standards, and contributing to the team's objectives. You will apply your analytical thinking skills along with data analysis tools and methodologies to make accurate judgments and recommendations based on factual information. Your attention to detail will be crucial as you analyze information and communicate it effectively to others. Your communication and diplomacy skills will be essential to exchange complex/sensitive information and provide quality service to ensure team effectiveness. **Responsibilitie...

We are looking for lead or principal software engineers to join our Data Cloud team. Our Data Cloud team is responsible for the Zeta Identity Graph platform, which captures billions of behavioural, demographic, environmental, and transactional signals, for people-based marketing. As part of this team, the data engineer will be designing and growing our existing data infrastructure to democratize data access, enable complex data analyses, and automate optimization workflows for business and marketing operations. Job Description: Essential Responsibilities: As a Lead or Principal Data Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and ba...