23220 Pyspark Jobs - Page 48

About mPrompto At mPrompto, we’re redefining how businesses engage with their customers by building an AI-powered Conversion Co-Pilot. Our mission is to help brands optimize customer journeys, increase conversions, and deliver personalized experiences to their customers on their web properties. We’re a fast-growing B2B tech startup with strong founders backed by engineering and AI/ML expertise — and we’re looking for a passionate Full-Stack Engineer who is excited to build innovative AI-driven products at scale. About the Role As a Full-Stack Engineer , you will play a pivotal role in building the core mPrompto platform from backend services and data pipelines to frontend interfaces and AI-i...

Looking for 5+ Years of experience notice period- immediate to 60 days Skill required - GCP - Data Side, Storage Classes, Dataflow, Big query, Pyspark / Python, Airflow

Designation: Assistant Manager - Data Science. Level: L3 Experience: 5 to 10 years. Location: Chennai, Tamil Nadu , India. Job Description: We are looking for a skilled Data Scientist and Data Engineer to join our team, which is a mix of Data Science and Data Engineering professionals. The ideal candidate will have hands-on experience with large-scale data processing, ETL/ELT pipeline development, and cloud-based data architectures, along with a strong foundation in analytics, statistical modeling, and machine learning. This role is ideal for someone who thrives in a fast-paced, data-rich environment, and is passionate about leveraging modern data science and engineering tools to enable data...

Job Description Job Title: Pyspark Lead Developer Req Id: 111187 City: Pune State/Province: Maharashtra Posting Start Date: 11/13/25 Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thriv...

Location: Bangalore - Karnataka, India - EOIZ Industrial Area Job Family: Artificial Intelligence & Machine Learning Worker Type Reference: Regular - Permanent Pay Rate Type: Salary Career Level: T3(B) Job ID: R-49648-2025 Description & Requirements Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related field.5+ yrs. experience in data engineering roles.3+ years of experience in DWH/ETL Domain using AWS tech stack such as Redshift, S3, Glue, Lambda and IAM.Proficiency in PySpark/Python/SQL and designing data models and data standards.Deep understanding of ETL/ELT processes, data warehousing concepts, and big data technologies.Experience building & managing data...

A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to lead the engagement effort of providing high-quality and value-adding consulting solutions to customers at different stages- from problem definition to diagnosis to solution design, development and deployment. You will review the proposals prepared by consultants, provide guidance, and analyze the solutions defined for the client business problems to identify any potential risks and issues. You will identify change Management requirements and propose a structured approach to client for managing the change using multiple communication mechanisms. You will also coach and create a vision for ...

Job Title: Data Developer (Python/PySpark/Big Data Platforms) We are looking for a skilled Data Developer to design, build, and maintain efficient and scalable ETL/ELT data pipelines. The ideal candidate will have hands-on experience with Python or PySpark, data integration, and modern big data platforms. You will work closely with data engineers, analysts, and stakeholders to ensure reliable and high-quality data delivery across systems. Key Responsibilities Design, develop, and maintain efficient and scalable ETL/ELT data pipelines using Python or PySpark. Collaborate with data engineers, analysts, and business stakeholders to understand requirements and deliver technical data solutions. P...

Description Strong proficiency in Python for data processing and ETL Advanced SQL skills, including query optimization, indexing, joins, and analytical functions Hands-on experience with ClickHouse, MongoDB, Redis, and ElasticSearch Proficiency in Apache Spark/PySpark and working with data lakes Experience with ETL and data ingestion tools such as Apache NiFi Familiarity with messaging and streaming platforms like Kafka, RabbitMQ, and ActiveMQ Experience with workflow orchestration frameworks such as Apache Airflow Exposure to cloud platforms (AWS, GCP, or Azure) and their data services (S3, Redshift, BigQuery, Dataproc, etc.) Understanding of data warehousing, data modeling, and performance...

About The Role We are looking for a Machine Learning Scientist to join our Payments Machine Learning team and make meaningful contributions to our mission to streamline and optimize Uber's global payment experiences. In this role, you will be able to use your strong quantitative skills in the fields of machine learning, statistics, economics, operations research to improve the Uber Payments experience. Example projects we work on include optimizing transaction routing and automated retry strategies to improve payment authorization success rates and rider/eater conversion. We are looking for experienced candidates with a passion for solving new and difficult problems with data. In this role, ...

Role & responsibilities Developing Modern Data Warehouse solutions using Databricks and Azure Stack Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs Drive technical discussion with client architect and team members Orchestrate the data pipelines in scheduler via Airflow Skills and Qualifications: Bachelor's and/or masters degree in computer science or equivalent experience. Good underst...

Hiring for Data Enginee r with - Experience - 7+ Years Notice Period - Immediate Joiner- 20 days Mode of Work - Work from Office Technical Skills (Required) Azure Data Factory (ADF): design, develop and orchestrate scalable data pipelines Azure Databricks: build and manage Spark clusters, notebooks, and jobs Microsoft Fabric (OneLake): ingest, organize and transform data within the Fabric ecosystem PySpark & SparkSQL: author complex transformations and SQL queries for largescale data processing Delta Tables: create, optimize and maintain Delta Lake tables (VACUUM, ZORDER, OPTIMIZE) JSON Processing: parse, validate and transform semistructured JSON datasets API Integration: consume and build ...

Role: Application Architect - Data Governance Duration:3-6 Months Contract to Hire Location: Pune, Maharashtra 411014, India Technical Experience: 3 years of experience in owning and managing a technical platform including but not limited to: environment management, user access management, uptime management, platform enhancements, platform updates, SaaS platform performance, etc. 1 to 2 years of experience in managing a small team of contractors building and maintaining system to system integrations, metadata/data extractions, and data quality rules. Advanced knowledge in writing SQL and Python with a demonstrated ability to coach others. 2 years of experience managing large data quality rul...

As an ETL Developer at our company, you will be responsible for designing, developing, and implementing performant ETL pipelines using Python API (pySpark) of Apache Spark on AWS EMR. Your role will involve the following key responsibilities: - Programming with Python with a strong proficiency in Python - Utilizing functional programming concepts and Data Modelling - Hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Understanding Sparks Data frame and API - Configuring EMR clusters on AWS - Dealing with AWS S3 object storage from Spark - Troubleshooting Spark jobs and monitoring them using Spark UI Qualifications required for this role include: - 5+ years of exp...

Your opportunity to make a real impact and shape the future of financial services is waiting for you. Let's push the boundaries of what's possible together. As a Senior Director of Software Engineering at JPMorgan Chase within the Consumer and Community Banking division, you will be responsible for leading various technical domains, overseeing the activities of multiple departments, and fostering cross-functional collaboration. Your technical expertise will be utilized across different teams to promote the adoption and implementation of advanced technical methods, helping the firm stay ahead of industry trends, best practices, and technological advancements. - Leads multiple technology and p...

As an Azure Databricks Architect at UST, you will be responsible for designing and implementing cloud-native solutions on Microsoft Azure. Your role will involve leading and implementing Azure infrastructure projects, architecting cloud-native solutions using Azure, Databricks, PySpark, Python, and AKS, and collaborating with different teams to ensure alignment with business and technical requirements. Additionally, you will be migrating application services to the cloud, building Proof of Concepts (PoCs), providing architectural leadership, and ensuring security principles are integrated into solution designs. Your key responsibilities will include: - Leading and implementing Azure infrastr...

Embark on a transformative journey as an AI Incubator Data Scientist at Barclays, where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unapparelled customer experiences. Seeking a talented Data Scientist to join our Data Science team in particular our AI incubator where you'll spearhead the evolution of our digital landscape, driving innovation dedicated to driving commercial value from data across the Corporate Bank. In this role, you will have the opportunity to shape and lead projects with other members of the team - leveraging machine learning techniques an...

Role: Data Engineer- Scala Years of Experience: 5 to 10 yrs Location: Bangalore, Hyderabad, Chennai, Pune, Gurugram Date of Interview: 15th Nov, 2025 Mode of Interview: Virtual Role & responsibilities Must-Have Good work experience & Proven experience working on Python (Data Engineering), Scala. Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL Experience working on Data Engineering projects Good Understanding of SQL Good understanding of Unix and HDFS commands Communication Experience working on Data Analytics and Pyspark Exposure to Tableau & SAS Required Technical Skill Set: Knowledge of Data Lakes environment, Scala, Data pipelines, SQL, AWS

About the role We’re looking for a Senior Python Data Engineer (4+ years) with deep hands-on experience in Azure Databricks, PySpark, Spark SQL, and Power BI. You’ll design, build, and optimize reliable data platforms and analytics solutions—partnering with product, analytics, and business teams in a fast-paced, collaborative environment. What you’ll do • Design and implement scalable ETL/ELT pipelines on Azure Databricks using PySpark/Spark SQL. • Model data for semantic layers powering BI and analytics use cases. • Build and maintain Power BI datasets, reports, and dashboards (DAX, data modeling, Power Query). • Write, refactor, and optimize Python notebooks for transformation, feature eng...

1. Design the end-to-end Test Strategy and Test plans for project deliveries and get signoffs from stakeholders 2. Able to drive enterprise-wide Automation Frameworks suited for Enterprise data platforms with data being fed by different business entities 3. Having good hands-on experience with Microsoft Azure along with Data Platform Testing. Should have azure Data Bricks, ADF, ADLS and Medallion Architectures based product testing experience 4. Should have hands-on knowledge of Python based testing frameworks i.e. PySpark/Py-Test and Python Data Quality libraries 5. Must have experience with Automaton framework design in Azure, CICD integration in pipelines and maintaining the overall Test ...

Total years of experience: Relevant years of experience: CTC (Including Variable): ECTC: Notice Period: Reason for change: Current location: Willingness to relocate (If other than Hyderabad ): Yes Availability for a teams call (two slots): Brief write up on how you could be a great fit for the role : Required Skills • 5- years of IT experience • Advanced skill in configuring and supporting software applications • Good knowledge in writing complex SQL queries • Experience with object-oriented language (python is a Plus) • Strong understanding of job orchestration tools like Jenkins and/or Airflow, code repository tools like Git and/or Bitbucket • Strong software development and debug/troubles...

We are seeking a Junior Data Engineer who is enthusiastic about working with data, quick to learn new technologies, and ready to take on challenges in a collaborative environment. If youre a team player with a positive attitude and a drive to grow, we encourage you to apply! Roles and Responsibilities: Assist in building and maintaining data ingestion, processing, and analytical pipelines using AWS services (ETL/ELT preferred) Support the design of data models for performance and reliability Work with big data, relational databases, NoSQL, and data warehouse solutions under guidance Collaborate with senior engineers on developing, monitoring, and maintaining data pipelines Handle moderate da...

Hello, Greetings from GlobalData!!! Urgent job openings for Team Lead - Python :- if job requirement is matching to your profile please share your updated resume @ mail id (m.salim@globaldata.com). Mention Subject Line :- Applying for Team Lead - Python @ GlobalData(Hyd) Share your details in the mail :- Full Name : Mobile # : Qualification : Company Name : Designation : Total Work Experience Years : Current CTC : Expected CTC : Notice Period : Current Location : If out-station willing to relocate to Hyd? : Office Address : 3rd Floor, Jyoti Pinnacle Building, Opp to Prestige IVY League Appt, Kondapur Road, Hyderabad, Telangana-500081. About the Role :- Roles and Responsibilities :- Design an...

- Design and develop scalable data engineering solutions. - Build and maintain ETL (Extract, Transform, Load) pipelines. - Implement data warehousing solutions. - Utilize Databricks, PySpark, and AWS services. Required Candidate profile - Contribute to the adoption of DevOps and Agile practices. Core Competencies: - SQL - Databricks - PySpark - Python - Pandas and NumPy AWS Cloud: S3, Lambda, Glue, CodeCommit, and Redshift.

Azure Data Engineer Skills:Azure Data Factory, Data Bricks, Data Warehousing, Data Modelling, SQL, ETL, Python, Pyspark Location-Pune/ Bangalore Exp-5+ Years Pkg-15 LPA MAX Send CV: riya.imaginators@gmail.com

Role: Data Engineering Remote Looking for 5-10 years of Exp • Design, develop & maintain ETL/ELT pipelines • Ingest & transform data from APIs, DBs, files, streams • Build real-time & batch processing solutions • Data validation, quality & cleansing • Translate business needs into data models • Ensure data security, access control & compliance • Maintain documentation & best practices Snowflake, AWS/GCP/Azure, Databricks (Lakehouse + Spark), BigQuery (Google Cloud), Azure Synapse / AWS Redshift, Kafka, Azure Data Factory, AWS Glue, Apache Spark, Apache Hadoop, Airflow, dbt, PySpark, SQL, Real-time & Batch Pipelines, Data Quality & Governance, Informatica, Talend, Collibra.