22907 Pyspark Jobs - Page 3

Primary Skills ETL testing & data pipeline frameworks Strong SQL Coding Data Bricks Planning, Test Engineering, Execution, & Test Cases design Automation & Functional testing Test automation frameworks, CI/CD Azure Ecosystem-Databricks, ADF, Synapse

Description : We are seeking a talented Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes. Responsibilities : - Data Pipeline Development : Build and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes. - Data Infrastructure : Design, implement, and manage data infrastructure components, including data warehouses, data lakes, and data marts. - Data Quality : Ensu...

Posted On: 17 Nov 2025 Location: Noida, UP, India Company: Iris Software Why Join Iris? Are you ready to do the best work of your career at one of India’s Top 25 Best Workplaces in IT industry ? Do you want to grow in an award-winning culture that truly values your talent and ambitions ? Join Iris Software — one of the fastest-growing IT services companies — where you own and shape your success story . About Us At Iris Software, our vision is to be our client’s most trusted technology partner, and the first choice for the industry’s top professionals to realize their full potential. With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technol...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role Build and optimize data pipelines and design scalable architectures on Azure Cloud Platform. Collaborate with data scientists, analysts, and developers to deliver high-quality data solutions. Ensure data quality, governance, and secu...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Data Services, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand proj...

Primary skills : Gemini Agent space, AI multi cloud experience, Experience in running AI services, vibe coding(Cursor), Vertex Secondary skills : Terraform, Multi cloud experience (AWS) Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Hands on exposure to using GCP services for storage, serverless-logic, search, transcription, and chat Thorough understanding of Cursor coding(Vibe coding); implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to qui...

Contract Duration : 4 Months +Exendible based on Performance Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implem...

Contract Duration : 4 Months +Exendible based on Performance Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implem...

Contract Duration : 4 Months +Exendible based on Performance Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implem...

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

Key Responsibilities: Design, build, and maintain scalable ETL pipelines to extract, transform, and load (ETL) data from various sources, including ERP systems, CRM platforms, and databases. Develop and optimize data models to support BI tools like Looker, Tableau, QlikView/Qlik Sense, or similar. Work closely with BI developers and business analysts to understand data requirements and translate them into effective technical solutions. Create and maintain data warehouses or data lakes to support advanced analytics and reporting. Ensure data accuracy, consistency, and reliability across all data sources and reports. Implement data governance best practices, including data security, privacy, a...

Greetings from TCS Recruitment Team! SKILL: Databricks. Years of experience :5 to 14 Location: PAN INDIA Job Description: Hands on experience as Lead with executing and delivering 1 or more full life cycle SAP S/4HANA MDG implementation project. Deep understanding on key SAP MDG concepts Data Modelling, UI Modelling, Process Modelling, Governance Process, Mass Processing, DRF, DIF, BRF+ and Data Quality. Configure, Design, Build, Test and Deploy the core SAP MDG solution and ensure it works seamlessly. Experience in implementing SAP S/4HANA. Expertise in Implementation of SAP MDG Solution for masters like Material, BP, etc. Experience in Configuring rule-based Workflows - serial, parallel an...

We are seeking a detail-oriented and analytical Finance Analyst to join our team. The ideal candidate will be responsible for analyzing financial data, preparing reports, supporting budgeting activities, and providing insights to support business decisions. Key Accountabilities: Develop automated reporting processes and data pipelines (e.g., using Microsoft Fabric or similar tools) to deliver timely and accurate outputs for key stakeholders across all business functions Design, build, and maintain Power BI dashboards and data models, ensuring accuracy, clarity, performance, and alignment with stakeholder needs. Extract, query, and validate raw data from multiple sources (databases, APIs, Fab...

Role: Senior Databricks Engineer / Databricks Technical Lead/ Data Architect Experience: 8-15 years Location: Bangalore, Chennai, Delhi, Pune About Tredence : Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies across industries are engaging with us and deploying their prediction and optimization solutions at scale. Head quartered in the San Francisco Bay Area, we serve clients in the US, Canada, Europe, and South East Asia. We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the...

Condé Nast is a global media company producing the highest quality content with a footprint of more than 1 billion consumers in 32 territories through print, digital, video and social platforms. The company’s portfolio includes many of the world’s most respected and influential media properties including Vogue, Vanity Fair, Glamour, Self, GQ, The New Yorker, Condé Nast Traveler/Traveller, Allure, AD, Bon Appétit and Wired, among others. Job Description Location: Chennai, TN About The Role : The Data Solutions Engineering team has a wide range of responsibilities and plays a critical role in shaping how Condé Nast enables its business using data. The team is responsible for building data pipe...

Job Description A.P. Moller - Maersk A.P. Moller – Maersk is the global leader in container shipping services. The business operates in 130 countries and employs 80,000 staff. An integrated container logistics company, Maersk aims to connect and simplify its customers’ supply chains. Today, we have more than 180 nationalities represented in our workforce across 131 Countries and this mean, we have elevated level of responsibility to continue to build inclusive workforce that is truly representative of our customers and their customers and our vendor partners too. We are responsible for moving 20 % of global trade & is on a mission to become the Global Integrator of Container Logistics. To ac...

Minimum Years of Experience 3-5 years in Python and SQL programming, ETL 2+ years in Spark, data visualization Duties and Responsibilities: Data processing: data querying, analysis and validation Build, optimize and maintain data flows within data warehouses, storage and cloud environments. Data Modeling: Design scalable data pipeline schemas and architecture. Deploy and monitor data pipelines in production. Requirements Basic Qualification: Bachelor’s degree in computer science, Information Systems, Engineering, or a related field Proficient in Python, SQL, and Spark/PySpark. Design, develop, and maintain efficient and scalable data pipelines for both batch and real-time processing. Build a...

We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. You will be responsible for designing, developing, and optimizing scalable data pipelines and big data solutions to support analytics and business intelligence initiatives.

About Position: We are looking for Azure Architects who have experience in Azure data bricks, Data factory, to join our development team. Role: Azure Architect Location: Pune/ Bangalore Experience: 10+ years Job Type: Full Time Employment What You'll Do: Design and implement complex, scalable data pipelines for ingestion, processing, and transformation using Azure Databricks technologies. Worked as lead, mentoring, leading and owning delivery from onsite and offshore team members. Develop and manage data storage solutions including Azure SQL Database, Data Lake, and Blob Storage. working as technical architect and delivering complex, critical, secure, available and scalable projects. Leverag...

About the Role We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms. Key Responsibilities Design, develop, and optimize scalable data pipelines using Databricks (PySpark/Spark SQL) . Implement and tune data workloads using Azure Data Factory , Azure Synapse Analytics , and Microsoft Fabric . Apply best practices in Delta Lake , including performance tuning, partitioning, indexing, and caching. Troubleshoot complex ingestion, transformation, orchestrati...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. The AIM (Analytics, Investment & Marketing Enablement) team – a part of GCS Marketing– is the analytical engine that enables Global Commercial business portfolio o...

Primary skills : Gemini Agent space, AI multi cloud experience, Experience in running AI services, vibe coding(Cursor), Vertex Secondary skills : Terraform, Multi cloud experience (AWS) Excellent programming skills and proficiency in at least one of the major programming scripting languages used in Gen AI orchestration such as Python or PySpark or Java Hands on exposure to integrating with Gemini Pro 1.x using API endpoints. Hands on exposure to using GCP services for storage, serverless-logic, search, transcription, and chat Thorough understanding of Cursor coding(Vibe coding); implementation exposure to LLM agents like LangChain & vector databases Pinecone or Chroma or FAISS Ability to qui...

Requirements Experience At least 5 years of experience in AWS based projects. Technical skills Proficiency in Python and PySpark for data engineering tasks. Big Data Strong knowledge of Big Data technologies and data warehousing concepts. AWS services Experience with AWS Data Engineering stack, including S3, RDS, Athena, Glue, Lambda, and Step Functions. SQL Strong SQL skills for data manipulation and querying. CI CD Experience with CI CD tools like Terraform and Git Actions. Soft skills Good communication skills and ability to work in a multicultural team.