895 Hdfs Jobs - Page 4

As a MySql Database Admin at Times Internet, you will play a crucial role in the design, implementation, automation, optimization, and ongoing operational administration of the backend system running on MySQL/MariaDB/MongoDB/RedisDB/Hadoop database infrastructure. You will be expected to take on instant support and operational challenges of infrastructure platforms running database services. Key Responsibilities: - Hands-on experience on MySQL/MariaDB RDBMS and related tools like Percona Xtrabackup and Percona-tool-kit - Hands-on experience on MongoDB NoSQL and Cache (RedisDB) datastores - Hands-on experience working on Private as well Public IT Infrastructure clouds like AWS - Hands-on expe...

Role Overview: You will be responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective will be to lead applications systems analysis and programming activities. Additionally, you will lead a data engineering team, from sourcing to closing, and drive the strategic vision for the team and product. Key Responsibilities: - Partner with multiple management teams to ensure appropriate integration of functions and identify necessary system enhancements - Manage a data-focused product and ML platform - Design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scal...

Req ID: 345781 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Web-Java Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Job Duties: Key Responsibilities: Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the ap...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data processing pipelines using Apache Spark. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. Troubleshoot complex issues related to HDFS, Hive, Pyspark, and other big data technologies. Ensure scalability, performance, and reliability of the system by implementing efficient algorithms and data structures. Participate in code reviews and contribute to the development of best practices for software engineering. Desired Candidate Profile 5-10 years of experience in working with Apache Spark (Scala/Python) on Hadoop ecosystem (HDFS). Strong understanding of Big D...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Staff Apache Ozone Engineer at BairesDev We are seeking a Staff Apache Ozone Engineer with deep expertise in distributed storage systems, object...

As a Technical Project Manager, you will be leading a team to build a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels. Responsibilities : Align Sigmoid with key Client initiatives Interface daily with customers across leading Fortune 500 companies to understand strategic requirements Connect with CIO, VP and Director level clients on a regular basis. Travel to client locations Ability to understand business requirements and tie them to technology solutions Build a delivery plan with domain experts and stay on track Design, develop and evolve highly scalable and fault-tolera...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform * Responsibilities: * Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS * Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform * Experience in d...

Databricks, PySpark, Azure Data Factory, Python

Qualifications: The successful candidate should have either of the below B.SC, B.E/B.Tech./M.E./M.S./M.Tech with good academic record Experience: 10+ years overall with 4-5 years as Architect 4+ years of experience applying statistical/ML algorithms and techniques to real-world data sets. 4+ years as a developer with experience in related scalable and secure applications. 2+ years experience independently designing core product modules or complex components Hands on Experience on GenAI Key Responsibilities Involves in Designing and Overseeing the overall architecture of ML systems, including data pipelines, model selection, deployment strategies, and ensuring scalability and performance. Req...

About the Role The Data Platform team is part of the Platform org which is based in Bangalore and is responsible for building Uber's Data Lake for the rest of the company. Team owns end to end charters for multiple batch data areas around storage (HDFS and Cloud Storage), Analytics (Hive, Spark), Pipeline Orchestration and Authoring, Data Quality, Metadata and Lineage, Jupyter Notebooks, Data LifeCycle Management, and Data Engineering. Team drives multiple global efforts around cloud migration and disaster recovery. We have an eclectic team with a diverse set of backgrounds from companies such as Facebook, Google, Cloudera, Amazon, Microsoft, LinkedIn, Twitter, other startups and college gra...

? Strong programming skills in Python, PySpark, and SQL. ? Solid experience building and optimizing ETL or ELT workflows using technologies such as Spark, Snowflake, Airflow, and/or Azure Data Factory. ? Strong ability to craft and optimize complex SQL queries, stored procedures, and data transformations. ? Experience developing and maintaining scalable, high-performance data models. ? Hands-on experience with Snowflake, including the use of Snowpark for in-platform data processing. ? Experience integrating external data sources and APIs into data workflows. ? Good experience in Azure cloud infrastructure ? Familiarity with data governance and data quality frameworks (e.g., Microsoft Purview...

About the Team! DE team at HYD collaborates with other engineering/product/analyst teams across tech sites to collectively accomplish OKRs to take Uber forward. We constantly enrich our data layer to effectively deal with the next generation of products which are a result of Uber's big bold bets. We design and build data pipelines to schedule & orchestrate a variety of tasks such as extract, cleanse, transform, enrich & load data as per the business needs. We serve the data insights at depth to various teams at Uber-like business analysts, data analysts, data science and business stakeholders to take strategic decisions, train DS models, perform health checks etc in order to grow and support...

SQL, GCP BQ, ETL pipelines using Pythin/Airflow,Experience on Spark/Hive/HDFS,Data modeling for Data conversion Resources Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Common Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow,Experience on Spark/Hive/HDFS,Data modeling for Data conversion Resources Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skillsData Engineer -Knows HR Knowledge , all other requirement from Functional Area given by UBER

Skill: Azure Databricks Experience: 6 to 12 years Location: AIA Kolkata 1. Job Title : Developer 2. Job Summary : We are seeking an experienced Developer with 7 to 10 years of expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. The role involves working in a hybrid model with a focus on developing and optimizing data solutions. The ideal candidate will contribute to our data-driven initiatives enhancing our capabilities and driving impactful outcomes. 3. Experience : 7 - 10 years 4. Required Skills : Technical Skills: Databricks SQL Databricks Delta Lake Databricks Workflows PySpark Domain Skills: 5. Nice to have skills : Domain Skills: 6. Technology : Cloud M...

Key skills: Informatica BDM/DEI, Hadoop/HDFS Experience level: 5+ years Evaluation criteria: Mandatory Requirements Proven development experience with Informatica DEI tools (PowerCenter or BDM) Strong understanding of Informatica BDM architecture and components Hands-on experience with Hadoop/HDFS ecosystem: o Hive SQL o File system types and structures Preferred Skills (Good to Have) Knowledge on Teradata/Cloudera Agility in responding to tasks and challenges Effective communication skills Proactive and solution-oriented approach

Senior Data Engineer with expertise in AWS, SQL, Pyspark, Databricks, Snowflake and Python to design, build, and optimize data pipelines and platforms. To perform Bug fixing and work on enhancements. This role requires strong technical leadership and the ability to guide a team of engineers toward delivering scalable, secure, and high-performance data solutions.

Roles and responsibilities As a Data Engineer, you will Design and build solutions for near real-time stream processing as well as batch processing on the Big Data platform. Set up and run Hadoop development frameworks. Collaborate with a team of business domain experts, data scientists, and application developers to identify relevant data for analysis and develop the Big Data solution. Explore and learn new technologies for creative business problem-solving. Roles and Responsibilities What we are looking for: Ability to develop and manage scalable Hadoop cluster environments. Ability to design solutions for Big Data applications. Experience in Big Data technologies like HDFS, Hadoop, Hive e...

As a skilled PySpark Developer with 4-5 or 2-3 years of experience, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate closely with data engineers, data scientists, and stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines to handle large volumes of structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and create efficient data models and transformations - Optimizing and tuning PySpark jobs for improved per...

Role Overview: You will be a seasoned professional in the role of Data Ops Capability Deployment - Analyst. Your main responsibility will be to perform data analytics and data analysis across different asset classes while building data science/Tooling capabilities within the team. You will work closely with the wider Enterprise Data team to deliver business priorities. Additionally, you will be a part of the B & I Data Capabilities team within the Enterprise Data, managing the Data quality/Metrics/Controls program and implementing improved data governance and data management practices throughout the region. Key Responsibilities: - Hands-on experience with a data engineering background and a ...

As a skilled Senior Engineer at Impetus Technologies, you will utilize your expertise in Java and Big Data technologies to design, develop, and deploy scalable data processing applications. Your responsibilities will include collaborating with cross-functional teams, developing high-quality code, and optimizing data processing workflows. Additionally, you will mentor junior engineers and contribute to architectural decisions to enhance system performance and scalability. Key Responsibilities: - Design, develop, and maintain high-performance applications using Java and Big Data technologies. - Implement data ingestion and processing workflows with frameworks like Hadoop and Spark. - Collabora...

As a candidate for the position, here is a summary of the job description: Role Overview: - You should have expertise in Neo4j, including its core concepts, Cypher query language, and best practices. - Responsible for designing and implementing graph database solutions, creating and maintaining graph schemas, models, and architectures. - Your role will involve migrating data from relational or other databases into Neo4j and optimizing Cypher queries for performance to ensure efficient data retrieval and manipulation. - Essential to have familiarity with graph theory, graph data modeling, and other graph database technologies. - Developing and optimizing Cypher queries, as well as integrating...

At ClearTrail, you will be part of a team dedicated to developing solutions that empower those focused on ensuring the safety of individuals, locations, and communities. For over 23 years, ClearTrail has been a trusted partner of law enforcement and federal agencies worldwide, committed to safeguarding nations and enhancing lives. We are leading the way in the future of intelligence gathering through the creation of innovative artificial intelligence and machine learning-based lawful interception and communication analytics solutions aimed at addressing the world's most complex challenges. As a Big Data Java Developer at ClearTrail, your responsibilities will include: - Designing and develop...

As an Associate Product Support Engineer focused on Hadoop distributed systems, your role will involve providing technical support to enterprise clients. You will be responsible for troubleshooting and resolving issues within Hadoop environments to ensure the stability and reliability of customers" infrastructures. Working closely with experienced engineers, you will collaborate with customers to understand their problems and deliver effective solutions with empathy and professionalism. Key Responsibilities: - Address customer issues related to Hadoop clusters and core components such as HDFS, YARN, MapReduce, Hive, etc. - Perform basic administrative tasks including installation, configurat...

Skill: Spark Scala Experience : 4 to 9 Years Location : AIA – Pune Minimum 2 years of experience in application development. Strong hands on experience in Apache Spark, Scala, and Spark SQL for distributed data processing. Hands-on experience with Cloudera Hadoop (CDH) components such as HDFS, Hive, Impala, HBase, Kafka, and Sqoop. Familiarity with other Big Data technologies, including Apache Kafka, Flume, Oozie, and Nifi. Experience building and optimizing ETL pipelines using Spark and working with structured and unstructured data. Experience with SQL and NoSQL databases such as HBase, Hive, and PostgreSQL. Knowledge of data warehousing concepts, dimensional modeling, and data lakes. Abili...

We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to - Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment Understand and translate business needs into data engineering solutions Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams Partner with cross-...