23220 Pyspark Jobs - Page 47

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. Function Description: Launched in 2012, Amex Offers is a digital advertising platform that connects Merchants and brands with the tens of millions of American Expr...

Requisition ID: 7554 Location: Thane (W), MH, IN, 400604 Business Unit / Group Function: Information Technology Work Arrangement: Onsite Level of Experience: Intermediate Job Highlights Design, build, and operate secure, transparent data pipelines—moving and transforming data from SAP BW/ERP, OSIsoft PI, Dynamics CRM, web APIs, SFTP, and file shares to drive analytics and AI initiatives. Develop scalable batch and streaming processes using PySpark, working in environments like Azure Synapse. Build, test, and deploy CI/CD workflows for data pipelines (using Git, Azure DevOps, DataLake, ADF, Synapse Pipelines). Maintain documentation, unit tests, and code reviews for robustness and maintainabi...

Job Title: Senior Data Engineer Experience: 6+ Years Employment Type: Contract Location: Remote Overview We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines. The ideal candidate will have strong experience with Azure Data Factory (ADF), Azure Synapse, PySpark, and SQL, along with solid data orchestration and integration skills. DP-600 certification is mandatory. Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines. Build and optimize data workflows using ADF, Synapse, and PySpark. Write and tune SQL queries for transformation and performance. Integrate and orchestrate d...

Educational Requirements MCA,MSc,Bachelor of Engineering,BBA,BCom Service Line Data & Analytics Unit Responsibilities Roles & Responsibilities: Understand the requirements from the business and translate it into an appropriate technical requirement. Responsible for successful delivery of MLOps solutions and services in client consulting environments; Define key business problems to be solved; formulate high level solution approaches and identify data to solve those problems, develop, analyze/draw conclusions and present to client. Assist clients with operationalization metrics to track performance of ML Models Help team with ML Pipelines from creation to execution Guide team to debug on issu...

Role Overview: You will be responsible for designing and deploying predictive models, building ML pipelines, optimizing model performance, implementing Delta Lake, enabling CI/CD for ML pipelines, and troubleshooting issues in Spark Jobs and Databricks Environment. As a Senior Programmer Analyst in AI-ML Engineering, you are expected to have strong technical expertise and experience in Python, PySpark, MLflow, and predictive modeling techniques. Key Responsibilities: - Design and deploy predictive models (e.g., forecasting, churn analysis, fraud detection) using Python/SQL, Spark MLlib, and Databricks ML. - Build end-to-end ML pipelines (data ingestion, feature engineering, model training, d...

As a Data Engineer at our company, you will play a crucial role in designing, building, and maintaining end-to-end data platforms to support product-performance analytics. Your responsibilities will include: - Engineering robust ETL/ELT pipelines and data models for processing structured and unstructured product and customer data. - Managing data lake migrations and ensuring One-Lake availability for various stakeholders. - Implementing CI/CD for data deployments, ensuring pipeline observability, and maintaining data quality. - Creating dashboards and visual insights for Product, Marketing, and Engineering teams. - Collaborating with cross-functional teams, mentoring junior engineers, and es...

As a Senior Data Engineer in the EY GDS Data and Analytics (D&A) team, you will have the opportunity to leverage your strong technology and data understanding in the big data engineering space to design, develop, and manage data solutions using Microsoft Fabric. Your role will be crucial in implementing ETL/ELT processes, creating and maintaining data models, and collaborating with cross-functional teams to deliver data solutions that meet business needs. Additionally, you will play a key role in optimizing code for data processing and analysis, implementing DevOps practices, and monitoring data pipelines for reliability and performance. **Key Responsibilities:** - Design, develop, and manag...

In this role at EY, you will be responsible for providing expert level data solutions using software to process, store, and serve data to others. You will test data quality, optimize data availability, and ensure that data pipelines are scalable, repeatable, and secure. Your role will involve utilizing deep analytical skills on internal and external data, as well as leading, instructing, and mentoring newer Data Engineering crew. Key Responsibilities: - Write ETL (Extract / Transform / Load) processes, design database systems, and develop tools for real-time and offline analytic processing. - Troubleshoot software and processes for data consistency and integrity, integrating complex and larg...

We are hiring for Azure databricks Engineer for EY GDS. Exp Range: 4-7yrs Location: Kolkata/Trivandrum/Coimbatore/Kochi We have virtual interviews planned for 22nd Nov'2025, If interested kindly apply for the role via below link & keep yourself available for the same https://careers.ey.com/job-invite/1650765/ Your key responsibilities Architecting big data solutions in a cloud environment using Azure Cloud services ETL design, development, and deployment to Cloud Service Interact with Onshore, understand their business goals, contribute to the delivery of the workstreams Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data ac...

About the Role We are seeking a Lead GCP Data Engineer to design, build, and optimize scalable data platforms and pipelines on Google Cloud Platform (GCP) . Youll lead a team of data engineers, drive data architecture decisions, and ensure the reliability and performance of our data infrastructure supporting analytics, AI/ML, and business intelligence initiatives. Key Responsibilities Lead the design and implementation of end-to-end data pipelines using GCP services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, etc.). Architect and optimize data lake and data warehouse solutions for scalability and performance. Define and implement data ingestion, transformation, and integration st...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Data Eng Mandatory Skills Required : ADB, ADF, Deltalake/Datalake and Pyspark, Data Modelling and Optimization, Unity Catalog Experience Required: 6+ years of relevant experience Location : Bengaluru/Trivandrum Notice Period : Immediate joiners to 30 days Key Responsibilities Design, develop, and implement end-to-end data pipelines across various sources and destinations. Lead and support data migration initiatives from legacy to modern data platforms. Develop robust ETL/ELT workflows using Azure Data Factory. Write clean, efficient, and scalable code in Python or PySpark for data transformation and automation. Utilize SQL for complex querying, data manipulation, and performance tuning. Coll...

Cognizant Hiring For Big data Engineer!!! Years of Experience: 8-12 Years Skills: Big Data Engineer with primary skills - Pyspark, Azure Synapse, ADF, Bigdata, SQL Job Location: PAN India Interview Mode: Video discussion

TECHNICAL SKILLS : Python, SQL, Data Warehouse concept, Data Modeling GCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions) Docker, Kubernetes, Github Microservices and Serverless Architecture Designing scalable, secure, and cost-optimized cloud solutions Infrastructure as Code (IaC) using Terraform Cloud security principles and identity & access management (IAM) Python, SQL, Data Warehouse concept, Data Modeling GCP services (Big Query, Cloud Run, Pub/Sub, Cloud Storage, Spanner, Cloud Composer, Dataflow, Cloud Functions) Docker, Kubernetes, Github Microservices and Serverless Architecture Designing scalable, secure, and cost-optimiz...

hackajob is collaborating with UnitedHealth Group to connect them with exceptional tech professionals for this role. Data Analyst Requisition Number: 2299804 Job Category: Business & Data Analytics Primary Location: Hyderabad, Telangana, IN Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities ...

Big Data Engineer Location: Pune/Nagpur (WFO) Experience Required: 8+ Years **Long term onsite to Abu Dhabi** Job Summary : We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability. Key Responsibilities: • Design, develop, and maintain data pipelines using PySpark and related Big Data technologies. • Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem. • Write efficient HiveQL and SQL queries to handle large-scale datasets. • Perform performance tuning and optimization of...

4+ years of experience in data analytics role using Pyspark, Python & SQL Proficient in Excel and PPT and can work hands-on

Coupa makes margins multiply through its community-generated AI and industry-leading total spend management platform for businesses large and small. Coupa AI is informed by trillions of dollars of direct and indirect spend data across a global network of 10M+ buyers and suppliers. We empower you with the ability to predict, prescribe, and automate smarter, more profitable business decisions to improve operating margins. Why join Coupa? 🔹 Pioneering Technology: At Coupa, we're at the forefront of innovation, leveraging the latest technology to empower our customers with greater efficiency and visibility in their spend. 🔹 Collaborative Culture: We value collaboration and teamwork, and our cult...

Coupa makes margins multiply through its community-generated AI and industry-leading total spend management platform for businesses large and small. Coupa AI is informed by trillions of dollars of direct and indirect spend data across a global network of 10M+ buyers and suppliers. We empower you with the ability to predict, prescribe, and automate smarter, more profitable business decisions to improve operating margins. Why join Coupa? 🔹 Pioneering Technology: At Coupa, we're at the forefront of innovation, leveraging the latest technology to empower our customers with greater efficiency and visibility in their spend. 🔹 Collaborative Culture: We value collaboration and teamwork, and our cult...

About Business Unit: Powerful predictors of the future - our Analytics team seamlessly blends data science and business intelligence to provide unparalleled foresight for our platform offerings. They turn insights into actionable results for our clients by collaborating with senior team members and staying ahead of market trends, powered by our data assets. By using new-age analytic tools and emerging technologies, the team cultivates a dynamic learning environment for curious minds to thrive. Who we are looking for? At Epsilon, our people use real human insights to find solutions that empower some of the biggest brands in the world. Our team is now growing, and we are on the lookout for ded...

EY GDSData and Analytics - D and A – Azure Data Engineer We’re looking for candidates with strong technology and data understanding in big data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team. Your key responsibilities Develop & deploy azure databricks in a cloud environment using Azure Cloud services ETL design, development, and deployment to Cloud Service Interact with Onshore, understand their business goals, contribute to the delivery of the workstreams Design and optimize model codes for faster execution Skills and attributes for success 3 to 8 years of Experience in...

Hi Hope you are doing good, This is Rahul from Aveto Consulting, we have a vacancy for the Data Engineer – Remote, Let me know your Interest. Job title: Data Engineer – Digital & Analytics Platform (GA4 + Cloud + BI Enablement) Location: Remote Duration -6months Contract Only immediate joiner (Within 20days) Years of experience - 6 to 8years Overview Data Engineer who combines engineering precision with analytical curiosity. The Data Engineer will design and automate pipelines that integrate digital behavior data (GA4 / clickstream) with transactional and operational data into a unified analytics layer. The role requires strong understanding of BigQuery, Databricks , and ELT automation frame...

JOB DESCRIPTION SOLUTION ARCHITECT Experience Level: Senior Level Experience Required: 10+ Years About Us: Celebal Technologies is a leading Solution Service company that provide Services the field of Data Science, Big Data, Enterprise Cloud & Automation. We are at the forefront of leveraging cutting-edge technologies to drive innovation and enhance our business processes. As part of our commitment to staying ahead in the industry, we are seeking a talented and experienced Data & AI Engineer with strong Azure cloud competencies to join our dynamic team. Overview: As a Data Architect you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tool...

Hi, Exp: 7-12 Years Must have skills ML , AWS , Python , MI Ops Secondary skills Pyspark , Air flow , Jenkins , cloud formation, Deep learning If interested please share resume at deepika.eaga@quesscorp.com

Job Title: Senior Data Analyst / Data Analytics Specialist Location: EXL Services Experience Level: 4+ Years Industry: Banking / Financial Services Department: Data Management Job Overview We are seeking a highly skilled and experienced Data Analyst/Data Analytics Specialist to join our company. This role is ideal for someone with a strong background in Python, PySpark, SQL, Tableau, and advanced data analytics, including experience with classification and regression algorithms. The ideal candidate will have a solid understanding of the banking domain mainly Home Equity, be an excellent communicator, and possess a passion for transforming data into actionable insights. Key Responsibilities D...