896 Hdfs Jobs - Page 18

Job Description YOUR IMPACT Are you passionate about leveraging cutting-edge AI/ML techniques, including Large Language Models, to solve complex, mission-critical problems in a dynamic environment Do you want to contribute to safeguarding a leading global financial institution OUR IMPACT We are Compliance Engineering, a global team of engineers and scientists dedicated to preventing, detecting, and mitigating regulatory and reputational risks across Goldman Sachs. We build and operate a suite of platforms and applications that protect the firm and its clients. We Offer Access to vast amounts of structured and unstructured data to fuel your AI/ML models, including textual data suitable for LL...

Join us as a Senior Data Engineer at Barclays where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionise our digital offerings, ensuring unapparelled customer experiences. As a part of team of developers, you will deliver technology stack, using strong analytical and problem solving skills to understand the business requirements and deliver quality solutions. To be successful as a Senior Data Engineer you should have experience with: AWS hands on with S3, EC2, Lambda, RDS, Glue, Athena, data pipelines, step function and knowledge of Analytical services. Experience of distributed computing architect...

Job description: 5+ years of strong experience on Bigdata testing, HDFS, Hive 3+ years of strong experience on ETL testing, 3+ years of strong experience on SQL 2+ years of experience on health care domain Please share if any one interested sayini.prasanthi@spsoftglobal.com

Role Overview: As a Big Data Admin with RDBMS skills, your role will involve supporting a broad variety of solutions including HDFS, Kubernetes, Kafka, Elastic search, and SQL Server or Postgres or MySQL. You will be responsible for analyzing existing database systems, recommending improvements, establishing database security structures, and ensuring Sox compliance. Your strong verbal, written, and interpersonal communication skills will be essential in effectively implementing new technologies and optimizing database objects. Additionally, you will provide 24x7 on-call support and technical input to solution development plans. Key Responsibilities: - Analyze existing database systems and re...

You are applying for the position of Data Ingest Engineer at NTT DATA in Pune, Maharashtra, India. As a part of the Ingestion team of the DRIFT data ecosystem, your primary focus will be on ingesting data efficiently using the latest technology available at Citi. You will be responsible for ensuring that data is ingested in a timely, complete, and comprehensive manner while continuously seeking innovative solutions to improve data quality for downstream partners. **Responsibilities:** - Partner with multiple management teams for seamless integration of functions to meet objectives and identify system enhancements for deploying new products and process improvements. - Solve high impact proble...

As an experienced Big Data professional with over 7 years of experience, you will be required to have a strong expertise in Spark and Scala. Your mandatory skills should include proficiency in Big Data, particularly in Spark and Scala. Additionally, you should possess a strong knowledge of HDFS, Hive, and Impala. Familiarity with Unix, Oracle, and Autosys is also essential for this role. Experience with Agile Methodology and Banking Expertise would be a plus. Key Responsibilities: - Proficiency in Big Data, focusing on Spark and Scala - Strong knowledge of HDFS, Hive, and Impala - Familiarity with Unix, Oracle, and Autosys - Experience with Agile Methodology and Banking Expertise - Excellent...

You will be responsible for supporting data and analytics solutions across projects as an Azure Data Engineer with 3 to 8 years of experience. Collaborating with various projects, you will create data solutions for a variety of business use cases. Your role will involve driving innovation within Data Engineering to shape the future of data and analytics technology. As a critical team member, you will design and develop highly complex data projects, identifying gaps in the data stack and guiding advancements for the team. You will provide technical expertise in all project phases, including analysis, design, development, and implementation of cutting-edge solutions. Your input will continuous...

As a skilled and experienced Big Data Engineer, you will be responsible for building scalable data pipelines, managing data lakes, and optimizing large-scale data processing frameworks. Your role will involve: - Designing, developing, and maintaining scalable and efficient big data pipelines using PySpark and Spark. - Working with large datasets using Hadoop, Hive, and HDFS. - Developing data ingestion frameworks using Python and Spark to integrate structured and unstructured data from multiple sources. - Implementing data transformation, data validation, and data quality frameworks. - Optimizing performance of big data processing applications. - Working on cloud platforms (AWS) and managing...

Position Summary Role Value Proposition: Work and collaborate with a nimble, autonomous, cross-functional team of makers, breakers, doers, and disruptors who love to solve real problems and meet real customer needs. You will be using cutting-edge technologies and frameworks to analyze data, to help create the pipeline, and collaborate with the data science team to enable the innovative work in machine learning and AI. Eagerness to learn new technologies on the fly and ship to production Knowledge in data science is a plus More than just a job we hire people who love what they do! Job Responsibilities 1. Building and Implementing data ingestion and curation process developed using Big data to...

As a Senior Developer at TransUnion, you will play a crucial role in designing, developing, and implementing cutting-edge solutions using a variety of technologies such as Java, Big Data frameworks, AI/ML concepts, and cloud platforms. Your contributions will be vital to the architecture, development, and deployment of data-driven applications. Key Responsibilities: - Design, develop, and maintain high-performance applications utilizing Java and related technologies. - Work with Big Data technologies like Hadoop, HDFS, Spark, and Scala to process and analyze large datasets. - Develop data pipelines for ingestion, transformation, and storage. - Write, optimize, and debug SQL queries across pl...

Role Overview: As a Bigdata - SSE - TechnoFunctional, your main responsibility is to understand the software requirements and develop them into working source code. You will be expected to deliver high-quality code within specified timelines and collaborate with team members to complete deliverables. Additionally, you will be responsible for recording management for production incidents/errors, participating in production incidents resolution, managing application/infrastructure production configuration/processes/releases, and contributing to all stages of the software development lifecycle. It is important to write well-designed, testable code, envision system features and functionality, an...

Responsibilities Job Summary The ETL Design and Development will be responsible for CCIB DCDA BI Reporting projects on-premises & on-cloud with good technical knowledge on big data and Tools. Good exposure in problem solving & stakeholder management skills. This role will report into the Lead for ETL Reporting and product development team in CCIB DCDA 8 to 12 years Experience of working in data analytical ETL Design & Development projects Expertise in tools like Informatica, Talend, SSIS, DataStage, and custom Python-based ETL frameworks Must know Programming skills like ETL using Hadoop, Python, SQL, Shell scripting, PySpark, Scoop, Scala SQL Expertise in writing complex queries, performanc...

Position Overview Job Title: Senior Engineer, VP Location: Pune, India Role Description Engineer is responsible for managing or performing work across multiple areas of the bank's overall IT Platform/Infrastructure including analysis, development, and administration. It may also involve taking functional oversight of engineering delivery for specific departments. Work includes: Planning and developing entire engineering solutions to accomplish business goals Building reliability and resiliency into solutions with appropriate testing and reviewing throughout the delivery lifecycle Ensuring maintainability and reusability of engineering solutions Ensuring solutions are well architected and can...

About Delhivery: Delhivery is India's leading fulfillment platform for digital commerce. With a vast logistics network spanning 18,000+ pin codes and over 2,500 cities, Delhivery provides a comprehensive suite of services including express parcel transportation, freight solutions, reverse logistics, cross-border commerce, warehousing, and cutting-edge technology services. Since 2011, we've fulfilled over 550 million transactions and empowered 10,000+ businesses, from startups to large enterprises. Vision : To become the operating system for commerce in India by combining world-class infrastructure, robust logistics operations, and technology excellence . About the Role: Data Engineer / Sr. D...

About Delhivery: Delhivery is India's leading fulfillment platform for digital commerce. With a vast logistics network spanning 18,000+ pin codes and over 2,500 cities, Delhivery provides a comprehensive suite of services including express parcel transportation, freight solutions, reverse logistics, cross-border commerce, warehousing, and cutting-edge technology services. Since 2011, we've fulfilled over 550 million transactions and empowered 10,000+ businesses, from startups to large enterprises. Vision : To become the operating system for commerce in India by combining world-class infrastructure, robust logistics operations, and technology excellence . About the Role: Data Engineer / Sr. D...

Data Engineer About VOIS VOIS (Vodafone Intelligent Solutions) is a strategic arm of Vodafone Group Plc, creating value and enhancing quality and efficiency across 28 countries, and operating from 7 locations: Albania, Egypt, Hungary, India, Romania, Spain and the UK. Over 29,000 highly skilled individuals are dedicated to being Vodafone Group's partner of choice for talent, technology, and transformation. We deliver the best services across IT, Business Intelligence Services, Customer Operations, Business Operations, HR, Finance, Supply Chain, HR Operations, and many more. Established in 2006, VOIS has evolved into a global, multi-functional organization, a Centre of Excellence for Intellig...

Data Engineer2 ommon Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow,Experience on Spark/Hive/HDFS,Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Common Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow, Experience on Spark/Hive/HDFS, Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Data Engineer - Knows HR Knowledge , all other requirement from Functional Area given by UBER

Arista Networks is seeking high-caliber self-motivated test engineers for its enterprise and cloud-managed Wi-Fi products. This role will involve testing and validating complex cloud infrastructure and services that power Wi-Fi networks, ensuring they perform at scale and meet the highest standards of reliability, security, and efficiency. You will play a key role in the design, development, and execution of comprehensive test plans for cloud-based Wi-Fi solutions, working closely with cross-functional teams to deliver industry-leading products. Responsibilities: Lead the testing efforts on the cloud infrastructure that supports Wi-Fi networks, ensuring scalability, performance, and resilien...

Arista Networks is seeking high-caliber self-motivated test engineers for its enterprise and cloud-managed Wi-Fi products. This role will involve testing and validating complex cloud infrastructure and services that power Wi-Fi networks, ensuring they perform at scale and meet the highest standards of reliability, security, and efficiency. You will play a key role in the design, development, and execution of comprehensive test plans for cloud-based Wi-Fi solutions, working closely with cross-functional teams to deliver industry-leading products. Responsibilities: Lead the testing efforts on the cloud infrastructure that supports Wi-Fi networks, ensuring scalability, performance, and resilien...

Role Overview: As a developer, you will need to demonstrate creativity in crafting applications by choosing suitable technical options, enhancing application development, maintenance, and performance. Your responsibilities will include implementing design patterns, utilizing proven solutions, considering others" development tasks, and supporting the Project Manager in the project's daily operations. Key Responsibilities: - Interpret application feature and component designs to ensure development aligns with specifications - Code, debug, test, document, and communicate the progress of product component and feature development - Validate outcomes with user representatives and integrate the ove...

As a Big Data Developer, you will be responsible for utilizing your strong experience with big data technologies such as Hadoop, Unix, HDFS, Hive, Impala, etc. You should be proficient in using Spark/Scala and have experience with data Import/Export using tools like Sqoop. Additionally, you will work with automation tools like Airflow, Jenkins, or similar ones. Your excellent knowledge of SQL Server and database structures will be crucial in writing and optimizing T-SQL queries and stored procedures. Your key responsibilities will include: - Demonstrating ability to write and optimize T-SQL queries and stored procedures - Working with Jira/Confluence/GitLab - Handling multiple activities wit...

As a Data Engineer, your primary responsibility will be to design and build production data pipelines within a hybrid big data architecture. You will use languages such as Scala, Python, and Talend to create efficient pipelines from data ingestion to consumption. Additionally, you will contribute to the entire data life-cycle, focusing on coding, debugging, and testing processes. Troubleshooting and optimizing ETL pipelines will also be a key aspect of your role, along with thorough documentation of each process. - Design and build production data pipelines using Scala, Python, and Talend within a hybrid big data architecture. - Gather and address technical and design requirements. - Refacto...

As a DevOps Administrator, you will be responsible for working with the latest cloud-native and distributed data platforms. Your role will involve administering Hadoop, ensuring the performance, reliability, and optimization of big data clusters, and recommending resources for deploying and optimizing big data technologies. If you are passionate about big data and leveraging cutting-edge technology to deliver concrete business outcomes, we are looking forward to your application. **Key Responsibilities:** - Install, manage, and configure big data clusters - Monitor the performance of distributed systems and middleware applications - Configure and optimize the Hadoop environment - Troubleshoo...

As a Technical Support Engineer for Big Data applications and systems in a Hadoop ecosystem, your responsibilities will include: - Providing technical support and troubleshooting for Big Data applications and systems built on the Hadoop ecosystem. - Monitoring system performance, analyzing logs, and identifying potential issues before they impact services. - Collaborating with engineering teams to deploy and configure Hadoop clusters and related components. - Assisting in maintenance and upgrades of Hadoop environments to ensure optimum performance and security. - Developing and maintaining documentation for processes, procedures, and system configurations. - Implementing data backup and rec...

As a Lead Data Engineer in our organization, you will play a crucial role in the data engineering team focused on on-premise solutions. Your responsibilities will include: - Designing, developing, and maintaining robust, scalable, and high-performance data pipelines and integration solutions. - Managing and optimizing data storage in Hadoop Distributed File System (HDFS). - Designing and implementing data workflows using Apache NiFi for data ingestion, transformation, and distribution. - Collaborating with cross-functional teams to understand data requirements and deliver efficient solutions. - Ensuring data quality, governance, and security standards are met within the on-premise infrastruc...