23086 Pyspark Jobs - Page 27

Embark on an exciting journey into the realm of data engineering and architecture with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. As a Data Architect you will be at the heart of the organization and support our clients to take control of their data and get value out of it by defining a reference architecture for our customers. This means that you will work closely with business leaders and information management teams to define and implement a roadmap on data architecture, data management, business intelligence or analytics solutions.. Minimum Qualifications: Translate business requirements into data requests, reports and dashboards. Stro...

A.P. Moller - Maersk A.P. Moller – Maersk is the global leader in container shipping services. The business operates in 130 countries and employs 80,000 staff. An integrated container logistics company, Maersk aims to connect and simplify its customers’ supply chains. Today, we have more than 180 nationalities represented in our workforce across 131 Countries and this mean, we have elevated level of responsibility to continue to build inclusive workforce that is truly representative of our customers and their customers and our vendor partners too. We are responsible for moving 20 % of global trade & is on a mission to become the Global Integrator of Container Logistics. To achieve this, we a...

Role & responsibilities Expansion of Data Products Support pipeline buildout, implementation, and RFC deployments for new business use cases • Assignment group/project repo segregation / automated CICD deployments for better organization and support • Enhance existing pipelines for performance and reliability Platform Integrations Falcon Shield (security & threat intelligence) • Collibra (data governance & cataloging) • XSOAR (security orchestration) • BIG ID (data privacy & compliance) • SPLUNK (SIEM & monitoring) Evidence Collection & Compliance Enable traceability of pipelines for audit readiness • Collaborate with ISS to collect and publish control implementation evidence • Support onboa...

Role & responsibilities Create and maintain optimal data pipeline architecture for efficient and reliable data processing. Assemble large, complex data sets that meet functional / non-functional business requirements. Database management, ETL process, Data modelling, ensuring data quality and integrity. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Data Bricks JD Key responsibilities Data pipeline development: Design, develop, and deploy scalable ETL (Extract, Transform, Load) pipelines using Databricks, Spark, and other cloud-native tools. Platform optimization: Optimize data processing workflows for performance, scalability, and cost efficiency within the Databricks environment. Data modeling and architecture: Contribute to the design and implementation of data models, schemas, and architectures for data lakes and data lakehouses. Data quality and governance: Implement data quality checks, validation routines, and ensure data security and compliance standards are met. Collaboration: Work closely with data scientists, analysts, and ot...

Job Summary The role of a Senior Specialist on the FCSO TM & Fraud -Tuning team is that of an individual contributor and involves working with three facets – 1. Working with large volumes of confidential and regulated data associated with customer’s financial transactions 2. Development of scripts and routines using new age programming languages and working of the Oracle / Hive environments to work with large volumes of structured and unstructured data 3. Apply analytical / statistical / quantitative models on data to identify anomalies and patterns which help in providing insight on the AML behaviour associated with customers and the transactions conducted by them. The role is a global role...

Roles and Responsibilities Design, develop, test, and maintain large-scale data pipelines using PySpark and SQL. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. Develop scalable data architectures that meet business needs and ensure data quality through testing and monitoring. Participate in code reviews to improve coding standards and best practices. Troubleshoot issues related to data processing, storage, and retrieval. Desired Candidate Profile 3-6 years of experience as a Data Engineer with expertise in PySpark and SQL PREFERABLY FROM BFSI Industry. Bachelor's degree (B.Tech/B.E.) from a reputed institution in Any Specialization. Strong ...

Requisition ID: 7554 Location: Thane (W), MH, IN, 400604 Business Unit / Group Function: Information Technology Work Arrangement: Onsite Level of Experience: Intermediate Job Highlights Design, build, and operate secure, transparent data pipelines—moving and transforming data from SAP BW/ERP, OSIsoft PI, Dynamics CRM, web APIs, SFTP, and file shares to drive analytics and AI initiatives. Develop scalable batch and streaming processes using PySpark, working in environments like Azure Synapse. Build, test, and deploy CI/CD workflows for data pipelines (using Git, Azure DevOps, DataLake, ADF, Synapse Pipelines). Maintain documentation, unit tests, and code reviews for robustness and maintainabi...

Roles and Responsibilities Design, develop, and maintain large-scale data pipelines using AWS services such as S3, Lambda, Step Functions, etc. Develop ETL processes using PySpark and Python to extract insights from structured and unstructured data sources. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Ensure high-quality code by writing unit tests, integrating with CI/CD pipelines, and following best practices for scalability and performance. Troubleshoot issues related to data processing workflows and provide timely resolutions.

Role Overview: Join Amgen's mission to serve patients by becoming an Associate Data Engineer. You will play a crucial role in designing, building, and maintaining data lake solutions for scientific data that drive business decisions for Research. Your work will contribute to the development of innovative medicines to help patients live longer, fuller, and happier lives. As part of the Amgen team, you will have the opportunity to make a lasting impact on patients' lives while advancing your career. Key Responsibilities: - Design, develop, and implement data pipelines, ETL/ELT processes, and data integration solutions - Contribute to data pipeline projects from inception to deployment, managin...

Job Title: Lead Data Engineer Location: Mumbai / Pune Experience: 8+ yrs Job Summary: We are seeking a technically strong and delivery-focused Lead Engineer to support and enhance enterprise-grade data and application products under the Durables model. The ideal candidate will act as the primary technical interface for the client, ensuring high system availability, performance, and continuous improvement. This role requires a hands-on technologist with strong team management experience, cloud (AWS) expertise, and excellent communication skills to handle client interactions and drive technical decisions. Key Responsibilities: Support & Enhancement Leadership Act as the primary technical lead ...

Role: Senior Data Engineer Location: Mumbai & Pune Experience: 3yrs to 8yrs Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, NumPy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines. Responsibilities: • Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy. • Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications. • Communicate results and business impacts of insight initiatives to key stakeholders to collaborat...

Job Description Lead Data Engineer - 8+ years Senior Data Engineer - 3+ years Skills: Scala, Python, DataBricks, Apache PySpark EMPLOYMENT TYPE: Full-Time, Permanent LOCATION: Remote (Pan India) SHIFT TIMINGS: 2.00 pm-11:00 pm IST Budget- As per company standards REPORTING: This position will report to our CEO or any other Lead as assigned by Management. The Lead/Senior Data Engineer will be responsible for building and extending our data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys working with big data and building systems from the ground up. You wil...

**Job Description** As an Associate Managing Consultant in Performance Analytics at Mastercard, your role involves managing deliverable development and workstreams on projects across various industries and problem statements. You will contribute to and/or develop analytics strategies and programs for large, regional, and global clients by utilizing data and technology solutions to unlock client value. Building and managing a strong working relationship with client managers, you will act as a trusted and reliable partner. Your responsibilities also include creating predictive models using segmentation and regression techniques to drive profits and reviewing analytics end-products to ensure ac...

Join us as a Software Engineer, Tableau And PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll build a wide network of stakeholders of varying levels of seniority It’s a chance to hone your existing technical skills and advance your career We're offering this role at associate vice president What you'll do In your new role, you’ll engineer and maintain innovative, customer centric, high performance, secure and robust solutions. You’ll be working within a feature team and using your extensive experience to engineer software, scripts and tools that are often complex, as well as liaising with other engineers, architects a...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : PySpark Good to have skills : NA Minimum 5 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks that drive innovation, efficiency, and business transformation. As a Cust...

Job description About Client Hiring for One of the Most Prestigious Multinational Corporations! Job Description Job Title :Data Engineer Mandatory skills : Python Pyspark SQL Qualification : Any Graduate or Above Relevant Experience :8+ yrs Location : Hyderabad CTC Range : 35 -40 lpa Notice period : Immediate joiners only Mode of Interview : Virtual Joel. IT Staff.Black and White Business solutions PVT Ltd Bangalore, Karnataka, INDIA. 8067432472 I joel.manivasan@blackwhite.in I www.blackwhite.in

As a Product Data Analyst- Data Engineering at ZF, you will have the opportunity to shape the future of mobility by driving and managing analytics initiatives from concept to deployment. You will apply structured software development principles to ensure robust, scalable, and maintainable solutions. Your role will involve designing, building, and optimizing scalable data models and pipelines in various big data environments. Additionally, you will develop high-performance data models tailored for analytical workloads and collaborate closely with technical teams and business stakeholders to deliver actionable solutions. Your responsibilities will also include creating user-centric dashboards ...

Job description Hiring for Azure Databrick Engineer Mandatory Skills: Databrick, Data lake, Azure Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and d...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : O9 Solutions Good to have skills : NA Minimum 7.5 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education , Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing i...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Analytics Services, Microsoft SQL Server, Microsoft Azure Databricks, PySpark, Azure Data Factory, Databricks REST API Good to have skills : NA Minimum 2 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: Job Description: Azure Data Engineer – ADF, Databricks, ADLS Role Overview We are seeking a skilled and detail-o...

Job description Hiring for Hadoop Developer Mandatory Skills: Hadoop, bigdata, spark Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed funct...

Data Scientist - Machine Learning - 3+ Years - Gurgaon/Bangalore A high-impact Data Science role where you will work closely with global clients in the Sports, Media & Entertainment domainbuilding predictive models, designing end-to-end ML solutions, and influencing strategic decision-making through advanced analytics. This is an excellent opportunity for professionals who want to work on cutting-edge machine learning, optimization, and statistical modeling projects. Location: Gurgaon / Bangalore Your Future Employer: You will be working with a prestigious organization known for its commitment to diversity, equality and inclusion. They offer a dynamic work environment, opportunities for care...

Job description Hiring for Hadoop Developer Mandatory Skills: Hadoop, bigdata, spark Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed funct...

We are seeking an experienced Databricks Developer with 5 to 7 years of expertise in big data engineering and analytics platforms. The ideal candidate will have deep experience in Apache Spark Databricks PySpark and cloud platforms such as Azure or AWS and a strong background in data pipeline development performance optimization and data governance. Key Responsibilities Design develop and maintain scalable data pipelines using Apache Spark on Databricks Write efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrate data from various structured and unstructured sources into a unified platform Implement Delta Lake and manage data versioning updat...