23077 Pyspark Jobs - Page 9

About The Role: Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of different components. Candidate will also involve in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimized performance of overall ecosystem. The ideal candidate is an experienced data wrangler who will support our software developers, database architects and data analysts...

Greetings from HCLTECH! We are hiring for GCP Data Engineer role for the location #Chennai #Hyderabad Role: GCP Data Engineer Experience: 4 Years to 13 Years Location: Chennai, Hyderabad Interview Mode: F2F Date: 22- Nov-2025 Job Description: Mandatory Skill: GCP BigQuery Cloud Composer or Airflow (Anyone is fine) Dataflow or Dataproc or Datafusion (Anyone is fine) Python or Pyspark (Anyone is fine) DBT SQL Nice to have: GKE, Spanner, Harness, GEN AI, looker Interested candidates, please share the details using the link GCP DE – Fill out form Note: Only candidates with GCP experience are eligible to apply

Role Summary: Support the Unified Matching Platform project to enable business use cases utilizing ML and AI-based product matching. Duties & Responsibilities: Improve existing ML models for text and image processing Perform GPT prompt engineering to enhance performance and accuracy Deploy and optimize ML and AI modules and pipelines Collaborate with other Business and Technology units to develop innovative solutions that meet business needs with respect to functionality, performance, scalability, reliability. Requirements Years of Experience: 3–5 years: Python and SQL programming, Machine Learning model development and deployment 2+ years: Generative AI, Prompt Engineering Basic Qualificati...

Company Description Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments. We invest in people and new advanced technologies to unlock the power of data. A...

Our vision is to transform how the world uses information to enrich life for all . Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of information into intelligence, inspiring the world to learn, communicate and advance faster than ever. Responsibilities include, but not limited to: Design, develop, and deploy machine learning models for predictive, prescriptive, and generative tasks using advanced statistical, ML, Deep Learning and analytical techniques. Architect and lead the end-to-end development of ML & Agentic workflows from concept to production. Build and operationalize agentic workflows capable of autonomous decision-m...

About the Role We are seeking an experienced Data Engineer Lead to design, develop, and maintain scalable data solutions on Azure and Databricks as part of our enterprise data modernization initiatives. The ideal candidate will have a strong background in data pipeline development, data integration frameworks , and cloud-based data engineering , with deep expertise in tools such as Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica . This role will lead the design and delivery of high-performance, governed data architectures for large-scale enterprise clients, driving data reliability, compliance, and analytics readiness. Key Responsibilities Design and implement sca...

We’re looking for a Data Engineer (2–3 years) with strong hands-on experience in Azure Data Factory, Azure Storage, SQL and ETL/ELT pipelines. The ideal candidate can independently build & maintain ADF pipelines, manage data flows, and ensure smooth data movement across Azure services, who have worked closely in connecting to multiple data sources & transforming data & storing without any manual intervention What you’ll do: Build/modify ADF pipelines, datasets, linked services Work with Copy Activity, Mapping Data Flows, triggers, Integration Runtime Ingest and transform data using Azure SQL / Synapse Implement incremental loads, data quality checks, and ETL best practices Handle Blob/ADLS s...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using DataBricks, Spark, SQL, and Python. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Develop scalable APIs using Django or Flask for data ingestion and processing. Ensure high-quality code by writing unit tests and integrating with CI/CD pipelines. Troubleshoot issues related to data pipeline failures and optimize performance.

As a Data Intern, you will play a crucial role in supporting our data team with various tasks related to data processing, analysis, and reporting. This position is ideal for a recent graduate or entry-level professional with a strong foundation in Python, PySpark, and SQL, and a keen interest in developing practical skills in a real-world data environment. Key Responsibilities: 1. Data Processing and Analysis: · Contribute to the development and implementation of data processing pipelines using PySpark. · Assist in cleaning, transforming, and analyzing large datasets to extract meaningful insights. 2. Python Programming: · Collaborate with the data team to write Python code for data manipula...

Extensive implementation experience in data analytics space or a senior developer role in one of the modern technology stack Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Ability to build API based scalable solutions and debug & troubleshoot software or design issues Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Thorough understanding of prompt engineering; implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to quickly conduct experiments and analyze the features and capabilities of newer...

Extensive implementation experience in data analytics space or a senior developer role in one of the modern technology stack Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Ability to build API based scalable solutions and debug & troubleshoot software or design issues Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Thorough understanding of prompt engineering; implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to quickly conduct experiments and analyze the features and capabilities of newer...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. The AIM (Analytics, Investment & Marketing Enablement) team – a part of GCS Marketing– is the analytical engine that enables Global Commercial business portfolio o...

ROLES & RESPONSIBILITIES Key Responsibilities: Claims Data Analytics & Modeling Analyze medical, pharmacy, and hospital claims data (e.g., CPT, HCPCS, ICD-10, DRG, NPI, etc.). Identify fraud, waste, and abuse (FWA) patterns in claims data. Develop predictive models for risk stratification, readmission prediction, and cost forecasting. Work on value-based care models, risk adjustment, and population health analytics. Perform longitudinal analysis of patient journeys across different providers and payers. Data Engineering & Management Work with large-scale healthcare databases Work with SQL, Python, R, PySpark, and cloud-based big data solutions (AWS, Azure, GCP). Required Skills & Qualificati...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Associate Managing Consultant, Advisors & Consulting Services, Performance Analytics Associate Managing Consultant – Performance Analytics Advisors & Consulting Services Serv...

Description - External At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? This Position is in GSG Advanced Analytics team as part of GSG MIS COE, is looking for full time c...

About Us Diligent is the AI leader in governance, risk and compliance (GRC) SaaS solutions, helping more than 1 million users and 700,000 board members to clarify risk and elevate governance. The Diligent One Platform gives practitioners, the C-Suite and the board a consolidated view of their entire GRC practice so they can more effectively manage risk, build greater resilience and make better decisions, faster. At Diligent, we're building the future with people who think boldly and move fast. Whether you're designing systems that leverage large language models or part of a team reimaging workflows with AI, you'll help us unlock entirely new ways of working and thinking. Curiosity is in our ...

, India Job Family Group: Information Technology (IT) Worker Type: Regular Posting Start Date: November 18, 2025 Business unit: Experience Level: Experienced Professionals Job Description: What’s the role We’re scaling decision intelligence for Non‑Fuel Retail (NFR) across priority market clusters (CEE, DACH, UK, Benefrux). The Team Lead will turn ambiguous business problems into structured analytical work, raise the bar on visualization and data storytelling, and coach a team of 7–8 to deliver consistently high‑quality insights and BI products that land with business impact. What you’ll be doing Partner with business stakeholders to decompose complex problems into analytical hypotheses and ...

Location: Chennai (Onsite) Experience: 2–4 years Budget: 20 to 25 LPA Job Description: · To work in capacity of AWS Databricks, Python, Pyspark Hands ON developer · Work with stakeholders for regular updates, requirement understanding and design discussions. Hands-on experience on designing and developing scripts for custom ETL processes and automation in AWS, data bricks, Python, Pyspark etc. · Should be skilled in creating architecture and design documentation and knowledge of data engineering tools & technologies · Expertise in Cloud related Big Data integration and infrastructure Tec stack using Databricks & Apache Spark framework · Well-versed with scripting languages – Python · Well ve...

Primary Skills ETL testing & data pipeline frameworks Strong SQL Coding Data Bricks Planning, Test Engineering, Execution, & Test Cases design Automation & Functional testing Test automation frameworks, CI/CD Azure Ecosystem-Databricks, ADF, Synapse

Description : We are seeking a talented Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes. Responsibilities : - Data Pipeline Development : Build and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes. - Data Infrastructure : Design, implement, and manage data infrastructure components, including data warehouses, data lakes, and data marts. - Data Quality : Ensu...

Posted On: 17 Nov 2025 Location: Noida, UP, India Company: Iris Software Why Join Iris? Are you ready to do the best work of your career at one of India’s Top 25 Best Workplaces in IT industry ? Do you want to grow in an award-winning culture that truly values your talent and ambitions ? Join Iris Software — one of the fastest-growing IT services companies — where you own and shape your success story . About Us At Iris Software, our vision is to be our client’s most trusted technology partner, and the first choice for the industry’s top professionals to realize their full potential. With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technol...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role Build and optimize data pipelines and design scalable architectures on Azure Cloud Platform. Collaborate with data scientists, analysts, and developers to deliver high-quality data solutions. Ensure data quality, governance, and secu...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...