15947 Pyspark Jobs - Page 15

Role: Data Analyst Industry: Fintech, AI Location: Bangalore (Hybrid) Type: Full-time Experience: 2-5 years About the compan yThe company is an early-stage, venture-backed, AI-first finance technology company redefining how high-volume businesses run their finance operations. Instead of stitching together legacy, manual tools, it is building an intelligent finance operating system that centralizes data, automates reconciliation, streamlines payouts and collections, and delivers real-time result s Role Overvi ewOwn complex SQL: write, review, and optimize queries across large, sometimes messy datasets (billions of rows) for analysis and production us e.Build reliable data pipelines and analys...

We are hiring a Python Fullstack Developer with 8–15 yrs of experience in Python, Azure, and JavaScript. The role involves building scalable cloud apps, APIs, and data workflows with a strong focus on Azure services and integration. Required Candidate profile Experienced Python Fullstack Developer skilled in Python, Azure, and JavaScript with strong expertise in APIs, SQL, and microservices. Hands-on Databricks and cloud-native development is a plus.

Hiring for Azure DevOps with data Factory & PySpark or Python Location: Pan India Experience: 4+ Years Notice Period: Immediate to 30 days preferred Key Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Data Factory. Develop and optimize Python or PySpark scripts for data transformation and processing. Build and manage CI/CD pipelines using Azure DevOps for data and analytics projects. Monitor, troubleshoot, and enhance ETL workflows and pipeline performance in Azure. Collaborate with data architects, analysts, and developers to deliver robust data solutions. Automate deployment, testing, and monitoring through modern DevOps practices. Ensure adherence to da...

Hiring for Azure DevOps with data Factory & PySpark or Python Location: Pan India Experience: 4+ Years Notice Period: Immediate to 30 days preferred Key Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Data Factory. Develop and optimize Python or PySpark scripts for data transformation and processing. Build and manage CI/CD pipelines using Azure DevOps for data and analytics projects. Monitor, troubleshoot, and enhance ETL workflows and pipeline performance in Azure. Collaborate with data architects, analysts, and developers to deliver robust data solutions. Automate deployment, testing, and monitoring through modern DevOps practices. Ensure adherence to da...

Hiring for Azure DevOps with data Factory & PySpark or Python Location: Pan India Experience: 4+ Years Notice Period: Immediate to 30 days preferred Key Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Data Factory. Develop and optimize Python or PySpark scripts for data transformation and processing. Build and manage CI/CD pipelines using Azure DevOps for data and analytics projects. Monitor, troubleshoot, and enhance ETL workflows and pipeline performance in Azure. Collaborate with data architects, analysts, and developers to deliver robust data solutions. Automate deployment, testing, and monitoring through modern DevOps practices. Ensure adherence to da...

Job Title: Software Engineer -Data Engineer Position: Software Engineer Experience: 4-6 years (Less YOE will be Rejected) Category: Software Development/ Engineering Shift Timings: 1:00 pm to 10:00 pm Main location: Hyderabad Work Type: Work from office Notice Period: 0-30 Days Skill: Python, Pyspark, Data Bricks Employment Type: Full Time • Bachelor's in Computer Science, Computer Engineering or related field Required qualifications to be successful in this role Must have Skills: • 3+ yrs. Development experience with Spark (PySpark), Python and SQL. • Extensive knowledge building data pipelines • Hands on experience with Databricks Devlopment • Strong experience with • Strong experience dev...

Job Title: Software Engineer -Data Engineer Position: Software Engineer Experience: 4-9 years Category: Software Development/ Engineering Shift Timings: 1:00 pm to 10:00 pm Main location: Hyderabad Work Type: Work from office Notice Period: 0-30 Days Skill: Python, Pyspark, Data Bricks Employment Type: Full Time • Bachelor's in Computer Science, Computer Engineering or related field Required qualifications to be successful in this role Must have Skills: • 3+ yrs. Development experience with Spark (PySpark), Python and SQL. • Extensive knowledge building data pipelines • Hands on experience with Databricks Devlopment • Strong experience with • Strong experience developing on Linux OS. • Exper...

We are looking for a skilled Lead Data Engineer with 10-15 years of experience to lead our team in designing, implementing, and optimizing scalable data solutions using Azure data services. The ideal candidate will have a strong background in data engineering best practices, data modeling, and ETL processes. Roles and Responsibility Lead and mentor a team of Azure Data Engineers, providing technical guidance and support. Design, implement, and optimize scalable data solutions using Azure data services such as Azure Databricks, Azure Data Factory, Azure SQL Database, and Azure Synapse Analytics. Ensure data engineering best practices and data governance are followed. Collaborate with cross-fu...

Job type Contract to HIRE Primary Skills 6-8yrs of Experience in working as bigdata developer/supporting environemnts Strong knowledge in Unix/BigData Scripting Strong understanding of BigData (CDP/Hive) Environment Hands-on with GitHub and CI-CD implementations. Attitude to learn / understand ever task doing with reason Ability to work independently on specialized assignments within the context of project deliverable Take ownership of providing solutions and tools that iteratively increase engineering efficiencies. Excellent communication skills & team player Good to have hadoop, Control-M Tooling knowledge. Good to have Automation experience, knowledge of any Monitoring Tools. Role You wil...

Greetings!!! This is regarding job opportunity with CGI for Data Engineer. Exp: 3-8 Years Job Location: Hyderabad Job Type: Permanent Work Mode: Work from Office Mandatory Skills: Data engineer, Python, Pyspark, Data bricks, SQL Interview Process: 3 Rounds of Interview (all virtual) If interested, request you to share your updated resume to madhura.r@talent21.in with the below details: Total Exp: Relevant Exp in Data Engineer: Relevant Exp in Python: Relevant Exp in Pyspark: Relevant Exp in Databricks: Current CTC: Expected CTC: Any offers in hand (Y/N) If Yes, offered CTC please: Serving Notice Period: (Y/N) If Yes, LWD please: Notice Period: Current Location: UAN and PF History (its mandat...

Must Have skills: Python, Pyspark, Databricks 5 days work from office (ODC) Data Engineer Software Engineer (Data) What are we looking for? Role models . Leading by example and setting direction on engineering practices and habits are core to our mission. Problem solvers . We pursue some of the most challenging cross-cutting problems/issues and treat them as opportunities to align and shape our culture. Change agents . We acknowledge that we might not always understand a particular domain; however, we will ask probing questions and many times will leverage data to drive decision making or challenge the status quo. Agile in mindset . We provide incremental value and seek customer feedback to ...

Primary Skill Python, Spark, Unix, Hive, Hadoop - Big Data Technologies Good to Have -- Hive, Big data Components Exp 5- 12 Yrs Senior Developer – 8+ years of experience Strong proficiency in programming with hands-on experience with relevant frameworks/libraries. Experience with PySpark and Pandas/Polars for data manipulation. Solid understanding of Test-driven Development and best coding practices. Experience with API architecture, microservices and Kubernetes. Familiarity with CI/CD pipelines and Azure DevOps. Excellent cross-team collaboration and communication skills. Experience with orchestration tools like Airflow and reporting tools such as Superset .

We are seeking an experienced Senior Generative AI Engineer with a strong background in Databricks and data lake architectures.

Job Location: Pune Position Summary As a data engineer, you will be responsible for delivering data intelligence solutions to our customers all around the globe, based on an innovative product, which provides insights into the performance of their material handling systems. You will be working on implementing and deploying the product as well as designing solutions to fit it to our customer needs. You will work together with an energetic and multidisciplinary team to build end-to-end data ingestion pipelines and implement and deploy dashboards. Roles And Responsibilities You will design and implement data & dashboarding solutions to maximize customer value. You will deploy and automate the d...

Location: Sector 63, Noida (WFO) Timings: Mon - Fri; 10:30 AM to 7:30 PM About the Role We are seeking a skilled Data Engineer with hands-on experience in building and maintaining scalable data pipelines and analytics solutions. The ideal candidate will be highly proficient in PySpark and Data Build Tool (DBT), with strong experience in managing large datasets, data warehousing, and modern data stack technologies. Key Responsibilities Design, build, and maintain ETL/ELT pipelines for efficient data ingestion, transformation, and storage. Collaborate with data analysts, data scientists, and stakeholders to understand data requirements and deliver actionable solutions. Implement and maintain d...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Associate Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and d...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Associate Job Description & Summary At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In business intelligence at PwC, you will focus on leveraging data and analytics to provide strategic insights and drive informed decision-making for clients. You will develop and implement innovative solutions to optimise business performance and enhance competitive advantage. Why PWC At PwC, you will be par...

At least 6 years of hands on exp with Python, PySpark and building testcases. Data Engineers working on Databricks only shouldn't be brought forward as this role will require writing Python scripts with OOPs concepts.

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Associate Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and d...

Skill:Data Engineer Notice Period:Immediate. Employment Type: Contract Job Description : Experience in Cloud platform, e.g., AWS, GCP, Azure, etc. Experience in distributed technology tools, viz. SQL, Spark, Python, PySpark, Scala Performance Turing Optimize SQL, PySpark for performance Airflow workflow scheduling tool for creating data pipelines GitHub source control tool & experience with creating/ configuring Jenkins pipeline Experience in EMR/ EC2, Databricks etc. DWH tools incl. SQL database, Presto, and Snowflake Streaming, Serverless Architecture .

Experience: 5+ Years Skill: ITSS- Senior Azure Developer / Data Engineer Notice Period:Immediate. Employment Type: Contract Working Mode: Hybrid Job Description Loc - Bangalore , Hyd, Chennai and Noida Experience: Mid-Level Primary Skillsets: Azure (ADF/ADLS/Key vaults) Secondary Skillsets: Databricks Good to have Skillsets: Ability to communicate well Experience in cloud applications, especially in Azure (basically things covered in Primary skillsets listed below) Work in agile framework Experience in ETL, SQL, PySpark Run with a task without waiting for direction all the time Experience with git repository and release pipelines. Any certifications on Azure Any certification on Databricks w...

Role: Senior Engineer Job Location: Hyderabad Mode: Hybrid Experience: 7-9y rs Notice Immediate Joiners Preferred Job Responsibilities - Design & build backend components of MLOps platform on AWS. - Collaborate with geographically distributed cross-functional teams. - Participate in on-call rotation with the rest of the team to handle production incidents. Nice to have skills: Experience with Apache Kafka and developing Kafka client applications in Python . Experience with MLOps platforms such as AWS Sagemaker, Kubeflow or MLflow. Experience with big data processing frameworks, preferably Apache Spark. Experience with container platforms such as AWS ECS or AWS EKS. Experience with DevOps & I...

At least 8 + years of experience in any of the ETL tools Prophecy, Datastage 11.5/11.7, Pentaho.. etc . At least 3 years of experience in Pyspark with GCP (Airflow, Dataproc, Big query) capable of configuring data pipelines . Strong Experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc . Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools ) Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineering efficiencies . Design should help embed standard pr...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Associate Job Description & Summary At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In business intelligence at PwC, you will focus on leveraging data and analytics to provide strategic insights and drive informed decision-making for clients. You will develop and implement innovative solutions to optimise business performance and enhance competitive advantage. *Why PWC At PwC, you will be part of a...

Notice Period: Immediate Joiners Job Description: Must have skills Azure DataBricks, Python And Pyspark, Spark. Please find the JD In the mail chain Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on experience in Azure Databricks , Data Factory, Data Lake store/Blob storage, SQL DB Experience in creating Big data Pipelines with Azure components Hands on programing with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Experience with messaging systems, such as Kafka or RabbitMQ Good understanding of...