22907 Pyspark Jobs - Page 2

Role: Databricks Technical Lead/ Data Architect Experience: 8-14 years Location: Bangalore, Chennai, Delhi, Pune, Kolkata About Tredence: Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies across industries are engaging with us and deploying their prediction and optimization solutions at scale. Head quartered in the San Francisco Bay Area, we serve clients in the US, Canada, Europe, and South East Asia. We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity an...

Cloud Data Architect Role: T0 The Ideal Professional For This Cloud Architect Role Will Have a passion for design, technology, analysis, collaboration, agility, and planning, along with a drive for continuous improvement and innovation. Exhibit expertise in managing high-volume data projects that leverage Cloud Platforms, Data Warehouse reporting and BI Tools, and the development of relational databases. Research, identify, and internally market enabling data management technologies based on business and end-user requirements. Seek ways to apply new technology to business processes with a focus on modernizing the approach to data management. Consult with technical subject matter experts and ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Azure Data Services, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring...

We are seeking a highly skilled and experienced technical engineers proficient in Azure Databricks, Azure Data Factory, and Python/Spark. The ideal candidate will have a strong technical background and a passion for delivering exceptional customer support. Key Responsibilities: • Technical Expertise: Utilize deep knowledge of Data Scale, Azure Data Factory, and Python/Spark to troubleshoot and resolve complex technical issues. • Customer Support: Ensure high levels of customer satisfaction by providing timely and effective support to clients. • Process Improvement: Identify and implement process improvements to enhance the efficiency and effectiveness of the support team. • Collaboration: Wo...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Data Services, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to ...

About The Role: Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of different components. Candidate will also involve in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimized performance of overall ecosystem. The ideal candidate is an experienced data wrangler who will support our software developers, database architects and data analysts...

Greetings from HCLTECH! We are hiring for GCP Data Engineer role for the location #Chennai #Hyderabad Role: GCP Data Engineer Experience: 4 Years to 13 Years Location: Chennai, Hyderabad Interview Mode: F2F Date: 22- Nov-2025 Job Description: Mandatory Skill: GCP BigQuery Cloud Composer or Airflow (Anyone is fine) Dataflow or Dataproc or Datafusion (Anyone is fine) Python or Pyspark (Anyone is fine) DBT SQL Nice to have: GKE, Spanner, Harness, GEN AI, looker Interested candidates, please share the details using the link GCP DE – Fill out form Note: Only candidates with GCP experience are eligible to apply

Role Summary: Support the Unified Matching Platform project to enable business use cases utilizing ML and AI-based product matching. Duties & Responsibilities: Improve existing ML models for text and image processing Perform GPT prompt engineering to enhance performance and accuracy Deploy and optimize ML and AI modules and pipelines Collaborate with other Business and Technology units to develop innovative solutions that meet business needs with respect to functionality, performance, scalability, reliability. Requirements Years of Experience: 3–5 years: Python and SQL programming, Machine Learning model development and deployment 2+ years: Generative AI, Prompt Engineering Basic Qualificati...

Company Description Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments. We invest in people and new advanced technologies to unlock the power of data. A...

Our vision is to transform how the world uses information to enrich life for all . Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of information into intelligence, inspiring the world to learn, communicate and advance faster than ever. Responsibilities include, but not limited to: Design, develop, and deploy machine learning models for predictive, prescriptive, and generative tasks using advanced statistical, ML, Deep Learning and analytical techniques. Architect and lead the end-to-end development of ML & Agentic workflows from concept to production. Build and operationalize agentic workflows capable of autonomous decision-m...

About the Role We are seeking an experienced Data Engineer Lead to design, develop, and maintain scalable data solutions on Azure and Databricks as part of our enterprise data modernization initiatives. The ideal candidate will have a strong background in data pipeline development, data integration frameworks , and cloud-based data engineering , with deep expertise in tools such as Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica . This role will lead the design and delivery of high-performance, governed data architectures for large-scale enterprise clients, driving data reliability, compliance, and analytics readiness. Key Responsibilities Design and implement sca...

We’re looking for a Data Engineer (2–3 years) with strong hands-on experience in Azure Data Factory, Azure Storage, SQL and ETL/ELT pipelines. The ideal candidate can independently build & maintain ADF pipelines, manage data flows, and ensure smooth data movement across Azure services, who have worked closely in connecting to multiple data sources & transforming data & storing without any manual intervention What you’ll do: Build/modify ADF pipelines, datasets, linked services Work with Copy Activity, Mapping Data Flows, triggers, Integration Runtime Ingest and transform data using Azure SQL / Synapse Implement incremental loads, data quality checks, and ETL best practices Handle Blob/ADLS s...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using DataBricks, Spark, SQL, and Python. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Develop scalable APIs using Django or Flask for data ingestion and processing. Ensure high-quality code by writing unit tests and integrating with CI/CD pipelines. Troubleshoot issues related to data pipeline failures and optimize performance.

As a Data Intern, you will play a crucial role in supporting our data team with various tasks related to data processing, analysis, and reporting. This position is ideal for a recent graduate or entry-level professional with a strong foundation in Python, PySpark, and SQL, and a keen interest in developing practical skills in a real-world data environment. Key Responsibilities: 1. Data Processing and Analysis: · Contribute to the development and implementation of data processing pipelines using PySpark. · Assist in cleaning, transforming, and analyzing large datasets to extract meaningful insights. 2. Python Programming: · Collaborate with the data team to write Python code for data manipula...

Extensive implementation experience in data analytics space or a senior developer role in one of the modern technology stack Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Ability to build API based scalable solutions and debug & troubleshoot software or design issues Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Thorough understanding of prompt engineering; implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to quickly conduct experiments and analyze the features and capabilities of newer...

Extensive implementation experience in data analytics space or a senior developer role in one of the modern technology stack Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Ability to build API based scalable solutions and debug & troubleshoot software or design issues Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Thorough understanding of prompt engineering; implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to quickly conduct experiments and analyze the features and capabilities of newer...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. The AIM (Analytics, Investment & Marketing Enablement) team – a part of GCS Marketing– is the analytical engine that enables Global Commercial business portfolio o...

ROLES & RESPONSIBILITIES Key Responsibilities: Claims Data Analytics & Modeling Analyze medical, pharmacy, and hospital claims data (e.g., CPT, HCPCS, ICD-10, DRG, NPI, etc.). Identify fraud, waste, and abuse (FWA) patterns in claims data. Develop predictive models for risk stratification, readmission prediction, and cost forecasting. Work on value-based care models, risk adjustment, and population health analytics. Perform longitudinal analysis of patient journeys across different providers and payers. Data Engineering & Management Work with large-scale healthcare databases Work with SQL, Python, R, PySpark, and cloud-based big data solutions (AWS, Azure, GCP). Required Skills & Qualificati...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Associate Managing Consultant, Advisors & Consulting Services, Performance Analytics Associate Managing Consultant – Performance Analytics Advisors & Consulting Services Serv...

Description - External At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? This Position is in GSG Advanced Analytics team as part of GSG MIS COE, is looking for full time c...

About Us Diligent is the AI leader in governance, risk and compliance (GRC) SaaS solutions, helping more than 1 million users and 700,000 board members to clarify risk and elevate governance. The Diligent One Platform gives practitioners, the C-Suite and the board a consolidated view of their entire GRC practice so they can more effectively manage risk, build greater resilience and make better decisions, faster. At Diligent, we're building the future with people who think boldly and move fast. Whether you're designing systems that leverage large language models or part of a team reimaging workflows with AI, you'll help us unlock entirely new ways of working and thinking. Curiosity is in our ...

, India Job Family Group: Information Technology (IT) Worker Type: Regular Posting Start Date: November 18, 2025 Business unit: Experience Level: Experienced Professionals Job Description: What’s the role We’re scaling decision intelligence for Non‑Fuel Retail (NFR) across priority market clusters (CEE, DACH, UK, Benefrux). The Team Lead will turn ambiguous business problems into structured analytical work, raise the bar on visualization and data storytelling, and coach a team of 7–8 to deliver consistently high‑quality insights and BI products that land with business impact. What you’ll be doing Partner with business stakeholders to decompose complex problems into analytical hypotheses and ...

Location: Chennai (Onsite) Experience: 2–4 years Budget: 20 to 25 LPA Job Description: · To work in capacity of AWS Databricks, Python, Pyspark Hands ON developer · Work with stakeholders for regular updates, requirement understanding and design discussions. Hands-on experience on designing and developing scripts for custom ETL processes and automation in AWS, data bricks, Python, Pyspark etc. · Should be skilled in creating architecture and design documentation and knowledge of data engineering tools & technologies · Expertise in Cloud related Big Data integration and infrastructure Tec stack using Databricks & Apache Spark framework · Well-versed with scripting languages – Python · Well ve...