21920 Hadoop Jobs - Page 19

JD GCP Data Engineer Responsibilities: Design, develop, and maintain data pipelines using GCP services such as Dataflow, BigQuery, and Pub/Sub. Implement and manage data warehouse solutions using BigQuery, ensuring data is stored securely and efficiently. Use Pub/Sub for real time data ingestion and streaming analytics. Provision and manage GCP infrastructure using Terraform, ensuring best practices in IaC are followed. Optimize data storage and retrieval processes to enhance performance and reduce costs. Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability of data services. Ensure data quality and integrity through robust testing and validation processes...

Azure Databricks: 1. Mounting Files 2. PAT Tokens 3. Cluster Auto scaling 4. Cluster Types 5. Nesting notebooks 6. Reading file using default and custom schema 7. Types Secrets and scopes in databricks 8. Delta Tables 9. Basic file loading scenario with its steps 10. Optimization 11. User defined functions 12. Transformations ADF: 1. Integration runtimes familiar with and where they are used 2. Components in creating pipelines 3. Scenario based question for creating a basic pipeline for loading data to Azure SQL table 4. Basic steps to improve the loading activity of a huge amount of file , ex: 1TB file size

Description: We are looking for a skilled Data Engineer/Developer with expertise in Snowflake and Spark as primary skills. The ideal candidate should also have a strong understanding of SQL and Hadoop. Primary Skills: Snowflake Spark Secondary Skills:SQL Hadoop Other Requirement:Strong communication skills Additional Details Global Grade : C Remote work possibility : Yes Global Role Family : 60236 (P) Software Engineering Local Role Name : 6504 Developer / Software Engineer Local Skills : 57676 Snowflake Languages Required: : English

DESCRIPTION AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Generative AI Innovation Center. You’ll join a diverse team of technical experts in dozens of countries who help customers achieve more with the AWS cloud. Are you a Data Analytics specialist? Do you have Data Warehousing, Hadoop/D...

ROLES & RESPONSIBILITIES Key Responsibilities Analyze existing Hadoop, Pig, and Spark scripts from Dataproc and refactor them into Databricks-native PySpark. Implement data ingestion and transformation pipelines using Delta Lake best practices. Apply conversion rules and templates for automated code migration and testing. Conduct data validation between legacy and migrated environments (schema, count, and data-level checks). Collaborate on developing AI-driven tools for code conversion, dependency extraction, and error remediation. Ensure best practices for code versioning, error handling, and performance optimization. Participate in UAT, troubleshooting, and post-migration validation activi...

ROLES & RESPONSIBILITIES Key Responsibilities Lead design and execution of Dataproc ? Databricks PySpark migration roadmap. Define modernization strategy , including data ingestion, transformation, orchestration, and governance. Architect scalable Delta Lake and Unity Catalog –based solutions. Manage and guide teams on code conversion, dependency mapping, and data validation. Collaborate with platform, infra, and DevOps teams to optimize compute costs and performance. Own the automation & GenAI acceleration layer , integrating code parsers, lineage tools, and validation utilities. Conduct performance benchmarking, cost optimization, and platform tuning (Photon, Auto-scaling, Delta Caching). ...

DESCRIPTION AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Ge...

DESCRIPTION Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you ...

DESCRIPTION Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you ...

DESCRIPTION AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Generative AI Innovation Center. You’ll join a diverse team of technical experts in dozens of countries who help customers achieve more with the AWS cloud. Are you a Data Analytics specialist? Do you have Data Warehousing, Hadoop/D...

DESCRIPTION AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Ge...

Job Title: Trainer – Data Engineer Location: Hyderabad Type: Freelancer Job Summary: We are looking for a passionate and knowledgeable Data Engineering Trainer to conduct interactive and practical training sessions for learners and professionals. The ideal candidate should have hands-on experience in data engineering tools, cloud platforms, and coding best practices, along with the ability to simplify complex concepts for diverse audiences. Key Responsibilities: Deliver engaging sessions on Data Engineering fundamentals and advanced topics . Teach practical implementation of: Data pipelines, ETL processes, and data warehousing. SQL and NoSQL databases. Python for Data Engineering. Big Data t...

Job Title: Data Engineer Job Location: Chennai, Tamil Nadu Job Type: Full-time Educational Qualifications: B. E or B. Tech (any discipline) / MCA or M.Sc. (CS) If you are interested, please send your CV and usual related documentation to the following email address: Kavitha.Hariharan@dsp.team Job Description: We are searching for our office in Chennai, India a Data Engineer with a university degree in computer science or mathematics and a proven working experience in requirement analysis and building and maintaining data systems (SQL or NO-SQL) of minimum 5 years. Experience gained during the development of the Master or PhD projects, and the-sis will be positively taken in account. Key Resp...

SynapseIndia is a software development company with over 24 years of experience, featuring development offices in India and the USA. We serve clients worldwide, delivering innovative solutions tailored to their needs. Our Noida SEZ office is conveniently located just a 10-minute walk from the nearest metro station. Why work with us? Partnerships with Industry Leaders: We are a Google and Microsoft partner, staffed by certified professionals. Global Presence: As a multinational corporation, we have clients and employees across the globe. Structured Environment: We follow CMMI Level-5 compliant processes to ensure quality and efficiency. Timely Salaries: We have consistently paid salaries on t...

SENIOR SPECIALIST Noida 7-9 Years INDIA Job Family Development Job Description (Posting). Job Title: GenAI EngineerAbout the Role:A GenAI Engineer (Generative AI Engineer) is a specialized professional at the forefront of artificial intelligence, focused on developing and implementing systems that can autonomously generate new and original content. This content can take many forms, including text, images, audio, video, and code.This role combines expertise in machine learning, data science, and software engineering to build the next generation of creative and intelligent AI applicationsJD:At least 5 years of experience designing and implementing artificial intelligence systems. Strong experi...

Job Title: Head of Data Engineering Department: Data Engineering Reporting To: Chief Technology Officer (CTO) Role Overview Spiro is seeking an exceptional Head of Data Engineering to lead, design, and scale our next-generation data infrastructure. You will oversee the development of robust, scalable data pipelines, architect our big data platforms, and ensure the integrity and quality of the data fueling our AI, analytics, and digital products. As a hands-on leader, you will guide a team of talented engineers while collaborating closely with analytics, software development, and product teams to drive business outcomes through data. This is a critical leadership role shaping the future of da...

Summary Position Summary Artificial Intelligence & Engineering AI & Engineering leverages cutting-edge engineering capabilities to help build, deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These solutions insights are powered by engineering for business advantage, helping transforming mission-critical operations. Join our AI & Engineering team to help in transforming technology platforms, driving innovation, and helping help make a significant impact on our clients' success achievements. You’ll work alongside talented professionals reimagining and re-engineering operations and processes that are could be critical...

The Data Analyst is responsible for analyzing, managing, and visualizing data across the organization’s Identity and Access Management (IAM) ecosystem. The role focuses on generating actionable insights from identity data to enhance user security, streamline access management, and ensure compliance with organizational and regulatory standards. Roles and Responsibilities Lead the design and implementation of solutions within the team, ensuring compliance with design standards and alignment with business objectives. Analyze existing business models, data flows, and system architectures to develop functional specifications and test plans. Create high-level and detailed process models, sequence ...

Location: Gurugram, Haryana Experience: 2–4 Years Department: Data Science / Artificial Intelligence / Analytics About the Role: We are seeking an innovative and analytical Data Scientist / AI & Machine Learning Specialist to join our fast-growing technology and analytics team in Gurugram. In this role, you will be responsible for designing intelligent algorithms, analyzing large volumes of data, and developing predictive models to solve complex business problems. The ideal candidate is passionate about using AI, data science, and automation to deliver insights that drive strategic decision-making and business performance. You’ll work closely with cross-functional teams—product, marketing, e...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you’ll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express Join Team Amex and let’s lead the way together. Business Overview: Credit and Fraud Risk (CFR) team helps drive profitable business growth by reducing the risk of f...

About Company Kenexai delivers smart, data-driven solutions that empower businesses across industries. Our mission is to combine deep, domain-specific expertise with cutting-edge technology to drive meaningful impact. With a trusted team, consistent quality, and a growing global presence, we remain committed to delivering excellence while staying true to our core values: innovation, integrity, and client success. Be part of a team that's not just building solutions, but shaping the future with intelligence. Key Responsibilities Oracle BDS Cluster Setup & Configuration We are seeking an experienced Oracle Big Data Service (BDS) Specialist to design, implement, and maintain our Oracle BDS infr...

Job Title: Bigdata + Pyspark Location: Hyderabad Experience Level: 5-8 Years Notice Period: Immediate to 30 Days I nterview Mode : L1 Virtual / L2 F2F (Mandatory) Work Mode : Hybrid Job Summary We are seeking a Specialist with 5 to 8 years of experience in the Big Data Hadoop Ecosystem to design and implement scalable data solutions. Detailed Job Description Work extensively with the Big Data Hadoop Ecosystem to process and analyze large datasets. Utilize Hadoop, Spark, and SparkSQL technologies to build efficient and high-performance data pipelines. Leverage Python for data manipulation, automation, and integration tasks. Collaborate with cross-functional teams to gather data requirements a...

Description Are you excited about the digital media revolution and passionate about designing and delivering advanced analytics that directly influence the product decisions of Amazon's digital businesses. Do you see yourself as a champion of innovating on behalf of the customer by turning data insights into action? The Amazon Digital Acceleration (DA) org is looking for an analytical and technically skilled data engineer to join our team. In this role, you will play a critical part in developing foundational analytical datasets spanning orders, subscriptions, discovery, promotions, pricing and royalties. Our mission is to enable digital clients to easily innovate with data on behalf of cust...

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title And Summary Managing Consultant - Analytics Advisors & Consulting Services Services within Mastercard is responsible for acquiring, engaging, and retaining customers by managing fraud an...

JD - Data Scientist About Credit Saison India Established in 2019, Credit Saison India (CS India) is one of the country’s fastest growing Non-Bank Financial Company (NBFC) lenders, with verticals in wholesale, direct lending and tech-enabled partnerships with Non-Bank Financial Companies (NBFCs) and fintechs. Its tech-enabled model coupled with underwriting capability facilitates lending at scale, meeting India’s huge gap for credit, especially with underserved and under penetrated segments of the population. Credit Saison India is committed to growing as a lender and evolving its offerings in India for the long-term for MSMEs, households, individuals and more. Credit Saison India is registe...