28509 Pyspark Jobs - Page 6

TCS is inviting applications!!! Role: AWS Data Engineer EXP: 6 - 8 YEARS LOCATION: Chennai/ Pune / Bangalore/ Hyderabad / Kolkata/ Delhi/ Kochi/ Indore Job Description Professional experience in data integration and management 3+ years of hands-on development experience in AWS Glue, Lambda functions, AWS Athena, Redshift, Pyspark is a Must Hands on working experience in AWS GLUE ETL tool and having good knowledge on any of the ETL tools (Informatica cloud, Talend etc.) Must have working experience with SQL, and databases (e.g.: Postgres, Redshift) implementing Data Models. Expertise with Python Language and Apache Spark. Have working experience in projects involving these. Strong hands-on ex...

Desired Competencies Must-Have Hands-on experience in Pyspark, Notebook • Experience in Azure Data Factory, Synapse Analytics Experience in converting SQL Stored Procedure into Pyspark Experience in Medallion Architecture • Experience in SQL Aggregation functions Good-to-Have Certification in DP-600: Implementing Analytics Solutions Using Microsoft Fabric and DP-203T00: Data Engineering on Microsoft Azure Power BI and Power platform Experience in Agile methodology using JIRA Knowledge over Investment functional domain, LOMA certification Responsibility of / Expectations from the Role Collaborate with technology, architecture, product management and design teams to drive outcomes with focus o...

TCS is inviting applications!!! Role: Azure Data Engineer EXP: 5 - 8 YEARS LOCATION: Pune / Bangalore/ Bhubaneshwar/ Kolkata **Immediate - 60 days Notice period** Job Description Strong knowledge of Extraction Transformation and Loading (ETL) processes using frameworks like Azure Data Factory or Synapse or Databricks ; establishing cloud connectivity between different systems like ADLS, ADF, Synapse, Databricks etc. Candidates must possess hands on Power BI skills. Candidates must have good understanding of Informatica. Design and develop ETL processes based on functional and non-functional requirements in python / pyspark within Azure platform. A minimum of 5 years' experience with large SQ...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Manager Job Description & Summary A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology. We make impactful decisions by mixing mind and machine to leverage data, un...

Job Description: • Lead the end-to-end migration of data pipelines and workloads from Teradata to Databricks. • Analyze existing Teradata workloads to identify optimization opportunities and migration strategies. • Design and implement scalable, efficient, and cost-effective data pipelines using PySpark/Scala on Databricks Skills Required: • Strong experience with Databricks, Apache Spark, and Delta Lake. • Strong problem-solving and communication skills. • Proficiency in PySpark, SQL, and optionally Scala. • Strong understanding of Spark, Databricks performance tuning techniques • Understanding of Teradata utilities, BTEQ scripts, stored procedures • Experience with cloud platforms (Azure, ...

TCS has been a great pioneer in feeding the fire of Techies like you. We are a global leader in the technology arena and there's nothing that can stop us from growing together. TCS Hiring for AWS Data Engineer Experience Range: - 5 years & Above Location: Chennai Must-Have Strong hands-on experience in Python programming and PySpark Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop ecosystem. Experience in writing and optimizing SQL for data manipulations. Good Exposure to scheduling tools. Airflow is preferable. Must – Have Data Warehouse Experience with AWS Redshift or Hive Experience in implementing security measures for data ...

About Client: Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate innovation. It partners with major firms in banking, healthcare, telecom, and media. Our Client is known for combining deep industry expertise with agile development practices, enabling scalable and cost-effective digital transformation. The company operates in over 50 locations across more than 25 countries, has delivery centers in Asia, Europe, an...

Job Title: Data Scientist Work Mode: Remote Experience: 5–7 Years Domain: Transportation We are seeking an experienced Senior Data Scientist with strong domain expertise in Transportation and Logistics to lead advanced analytics and optimization initiatives. The role focuses on building scalable, production-ready data science solutions that deliver measurable business impact. Key Responsibilities: Design, develop, and deploy predictive and prescriptive models for transportation use cases such as route optimization, demand forecasting, fleet predictive maintenance, logistics planning, and anomaly detection. Build scalable data pipelines, ML workflows, and ETL processes using PySpark within th...

This role is within enterprise data office and product solution team; focused on ensuring accurate, timely, and fit for purpose data for business, risk management and regulatory reporting requirements. Engage with Markets, Risk, Finance, Tech, Client org, and Data Engineering Teams to gather requirements, understand application processing, identify gaps and design systematic solution for business needs. Average day is highly collaborative, focusing on reaching out to application teams and users to understand Markets products processing in Regulatory Reporting data Flow. Document product data flows along with transformation /mapping/enrichments/logic within enterprise systems. Key Responsibil...

This role is within enterprise data office and product solution team; focused on ensuring accurate, timely, and fit for purpose data for business, risk management and regulatory reporting requirements. Engage with Markets, Risk, Finance, Tech, Client org, and Data Engineering Teams to gather requirements, understand application processing, identify gaps and design systematic solution for business needs. Average day is highly collaborative, focusing on reaching out to application teams and users to understand Markets products processing in Regulatory Reporting data Flow. Document product data flows along with transformation /mapping/enrichments/logic within enterprise systems. Key Responsibil...

This role is within enterprise data office and product solution team; focused on ensuring accurate, timely, and fit for purpose data for business, risk management and regulatory reporting requirements. Engage with Markets, Risk, Finance, Tech, Client org, and Data Engineering Teams to gather requirements, understand application processing, identify gaps and design systematic solution for business needs. Average day is highly collaborative, focusing on reaching out to application teams and users to understand Markets products processing in Regulatory Reporting data Flow. Document product data flows along with transformation /mapping/enrichments/logic within enterprise systems. Key Responsibil...

This role is within enterprise data office and product solution team; focused on ensuring accurate, timely, and fit for purpose data for business, risk management and regulatory reporting requirements. Engage with Markets, Risk, Finance, Tech, Client org, and Data Engineering Teams to gather requirements, understand application processing, identify gaps and design systematic solution for business needs. Average day is highly collaborative, focusing on reaching out to application teams and users to understand Markets products processing in Regulatory Reporting data Flow. Document product data flows along with transformation /mapping/enrichments/logic within enterprise systems. Key Responsibil...

This role is within enterprise data office and product solution team; focused on ensuring accurate, timely, and fit for purpose data for business, risk management and regulatory reporting requirements. Engage with Markets, Risk, Finance, Tech, Client org, and Data Engineering Teams to gather requirements, understand application processing, identify gaps and design systematic solution for business needs. Average day is highly collaborative, focusing on reaching out to application teams and users to understand Markets products processing in Regulatory Reporting data Flow. Document product data flows along with transformation /mapping/enrichments/logic within enterprise systems. Key Responsibil...

Description We are seeking a highly skilled Data Engineer with expertise in AWS, DataBricks, Snowflake, Python, and PySpark to join our dynamic team. In this role, you will be responsible for building and optimizing our data pipelines, ensuring high performance and reliability as we scale our data infrastructure. You will work closely with data scientists, analysts, and other stakeholders to create robust data solutions that support business intelligence and data analysis. As a Data Engineer, you will harness the power of cloud technologies to streamline our data processes, and your proficiency in both Python and PySpark will be vital for developing and maintaining ETL workflows. Your contri...

Job Description: We are a rapidly expanding AI startup that is transforming digital advertising with groundbreaking AI technology and innovative solutions. We are seeking a seasoned and visionary Principal Data Engineer & Tech Lead Manager to join our team and play a pivotal role in building and leading our data infrastructure that powers the future of advertising. As a Principal Data Engineer & Tech Lead Manager, you will be responsible for architecting and leading the development of large-scale data platforms that process petabytes of consumer behavioral data. You will take ownership of the entire data ecosystem, from ingestion to transformation to serving, ensuring that our machine learni...

Job Title: AWS Data Engineer Location: Chennai Experience: 5 10 Years Interview Mode: Face-to-Face (Walk-in) Walk-in Date: 10th Jan 2026 Time: 10:30 AM – 3:00 PM Mandatory Skills AWS (S3, Glue, EMR, Redshift, Lambda) Python SQL (Complex Queries) PySpark Databricks Data Modeling & Data Warehousing ETL / ELT Pipelines Job Summary Looking for an experienced AWS Data Engineer with strong hands-on expertise in Python, PySpark, Databricks, SQL, and AWS to build and maintain scalable data pipelines. Contact gopi.c@acesoftlabs.com 9701923036

At Citi we’re not just building technology, we’re building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world’s most global banks we’re changing how the world does business. Shape your Career with Citi We’re currently looking for a high caliber professional to join our team as Officer, Big Data Engineer -Hybrid based in Chennai, India. Being part of our team means that we’ll provide you with the resources to meet your unique needs, empower you to make health...

As a Lead Software Engineer / Associate Architect, your role will involve designing and implementing scalable and robust data engineering solutions. You will be responsible for: - Having an in-depth understanding of Big data concepts and distributed systems - Demonstrating proficiency in Python and writing code in PySpark - Possessing a good understanding of SQL concepts and being proficient in writing complex SQL queries - Ensuring optimization and performance improvement for Spark based workloads - Working on at least one Cloud platform (preferred) - Having conceptual knowledge and fitment for Data Structures & Algorithms - Demonstrating in-depth knowledge of various Data Processing Patter...

As a Scala Developer at the technology services organisation in the IT & Cloud Software sector, you will be responsible for designing and implementing production-grade backend microservices using Scala and functional programming patterns. Your key responsibilities will include: - Developing and optimizing batch and streaming data pipelines leveraging Apache Spark and Apache Kafka for real-time processing and analytics. - Building cloud-native applications and deploying services on AWS, collaborating on infrastructure-as-code, security, and cost optimization. - Writing automated tests, maintaining CI/CD pipelines, performing code reviews, and enforcing engineering best practices for performan...

As a Senior Data Engineer in the EY GDS Data and Analytics (D&A) MS Fabric team, you will have the exciting opportunity to work with cutting-edge technology and be part of a growing team that is shaping the future of data solutions within the Microsoft Fabric platform. Your main responsibilities will include: - Designing, developing, and managing data solutions using Microsoft Fabric tools such as Lakehouse, Data Engineering, Pipelines, Spark, Notebooks, and KQL Database. - Implementing ETL/ELT processes to ensure efficient data integration and transformation. - Creating and maintaining data models to support business intelligence and analytics initiatives. - Utilizing Azure data storage ser...

Role Overview: You will be responsible for in-patient journey analysis and working with patient-level data to develop a robust solution for the client's teams as a Patient Analytics expert at Axtria. Your role will involve effectively managing the client/onshore stakeholders, defining algorithms, running analysis, and delivering high-quality analytics solutions/reports to the client. Additionally, you will contribute towards Axtria tools and capabilities, participate in Hackathons, and build organization capabilities by engaging in solution design and process automation. Key Responsibilities: - Effectively manage client/onshore stakeholders to ensure successful business delivery - Work close...

Role Overview: As a skilled Data Engineer with experience in Google Cloud Platform (GCP), PySpark, SQL, and ETL processes, you will be responsible for building, optimizing, and maintaining scalable data pipelines and workflows. Your expertise in technologies such as Apache Airflow, PySpark, and cloud-native tools will be crucial in ensuring efficient data processing. Key Responsibilities: - Design, develop, and maintain efficient and scalable data pipelines using PySpark and SQL. - Build and manage workflows/orchestration using Apache Airflow. - Utilize GCP services like BigQuery, Cloud Storage, Dataflow, and Composer. - Implement and optimize ETL processes to guarantee data quality, consist...

As a Developer at our company, you will play a crucial role in enhancing data solutions and optimizing workflows to impact both the company and society positively. Leveraging your expertise in Azure Data Lake Store, Azure DevOps, Python, Databricks SQL, Databricks Workflows, and PySpark, you will contribute to innovative projects that drive our company's success. **Key Responsibilities:** - Develop and implement data solutions using Azure Data Lake Store to enhance data storage and retrieval processes. - Collaborate with cross-functional teams to integrate Azure DevOps for streamlined development and deployment workflows. - Utilize Python to design and execute efficient data processing scrip...

Role Overview: You will be responsible for developing and maintaining scalable data processing solutions using Databricks SQL and PySpark to enhance data analysis capabilities. Collaborate with cross-functional teams to design and implement Databricks Workflows, analyze Medicare and Medicaid Claims data, ensure data integrity, and optimize existing data pipelines. Provide technical expertise, monitor workflows, document technical specifications, and stay updated with the latest advancements in Databricks and PySpark technologies. Additionally, you will collaborate with stakeholders to understand business requirements, participate in code reviews, and contribute to the development of best pra...

Must-Have Skills • Python Automation (minimum 3+ years in Python scripting) • Strong OOP concepts & automation frameworks • PyTest / PySpark (preferred) • Automotive domain – Mandatory • UI & API Automation experience