1540 Apache Spark Jobs - Page 3

What you will do In this vital role you will design, build and maintain data lake solutions for scientific data that drive business decisions for Research. You will build scalable and high-performance data engineering solutions for large scientific datasets and collaborate with Research stakeholders. The ideal candidate possesses experience in the pharmaceutical or biotech industry, demonstrates strong technical skills, has experience with big data technologies, and understands data architecture and ETL processes Design, develop, and implement data pipelines, ETL/ELT processes, and data integration solutions Contribute to data pipeline projects from inception to deployment, manage scope, tim...

Role Description: We are looking for highly motivated expert Principal Data Engineer who can own the design & development of complex data pipelines, solutions and frameworks with domain expertise in R&D domain. The ideal candidate will be responsible to design, develop, and optimize data pipelines, data integration frameworks, and metadata-driven architectures that enable seamless data access and analytics. This role prefers deep expertise in big data processing, distributed computing, data modeling, and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management. Roles & Responsibilities: Architect and maintain robust, scalable data pipel...

You have an exciting opportunity to join a staffing and HR services firm focused on placing talent into Enterprise IT, Analytics, and Big Data engineering projects across various sectors. As an on-site Big Data Engineer in India, your role will involve designing, building, and optimizing data pipelines and platform components for critical data applications. **Role Overview:** As a Big Data Engineer, you will be responsible for: - Designing and implementing end-to-end batch and streaming data pipelines using Spark and Hadoop ecosystem components. - Developing ETL jobs and data transformation logic in Python/SQL to ensure scalability and maintainability. - Integrating and processing real-time ...

Design and maintain robust bigdata solutions on Databricks. Partner with analytics teams to deliver clean, timely data and insights. Required Candidate profile Build scalable data pipelines, optimize Spark jobs, design Delta Lake architectures, integrate with cloud storage, implement CI/CD &automation,distributed computing, containerization Docker/Kubernetes

We are seeking a Senior Technical Pre-Sales Engineer with strong hands-on background in enterprise systems, data platforms, and AI/ML solutioning for Manufacturing, Logistics, and Supply Chain clients.This role requires deep technical articulation , not just sales support. The candidate must be able to whiteboard architectures , challenge client assumptions, define scalable IT service delivery models , and translate operational pain points into data-driven, AI-enabled transformation solutions . Core Technical Responsibilities 1. Deep Technical Discovery & Domain Mapping Conduct architecture-level discovery sessions with enterprise and mid-market clients. Analyze existing systems such as: ERP...

Your Opportunity As an architect, you will get the opportunity to grow in the field of high-performance computing, complex system design and low-level optimizations for better cost of ownership. Roles and Responsibility As a Software Architect, you will be responsible for designing and implementing High performance computing software solutions for our organization. You will work closely with cross-functional teams, including software engineers, product managers, and business stakeholders, to understand requirements and translate them into architectural/software designs that meet business needs. You will be coding and developing quick prototypes to establish your design with real code and dat...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Spring Boot Good to have skills : Java Enterprise Edition, Python (Programming Language), Apache Spark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project specifications, developing application features, and ensuring that the applicatio...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : Python (Programming Language), MySQL, Google BigQuery Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing innovative solutions, and ensuring that applications are optimized for perfo...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Python (Programming Language), Apache Spark Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that application development aligns with business objectives, overseeing proje...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : Apache Spark Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications are optimize...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : Apache Spark Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications are optimize...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : Health Insurance Operations Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks that drive innovation, efficiency, and business transformation.Roles & Responsibilities:Lead AI-driven solution design and delivery by applying GenAI and Agentic AI to...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly within the existing infrastructure. You will also e...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Apache Spark Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, addressing any challenges that arise, and providing guidance to team members to foster a productive work environment. Yo...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing innovative Java-based solutions, and ensuring that applications are optimized for performance and usability. You will engage in ...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing innovative Java-based solutions, and ensuring that applications are optimized for performance and usability. You will engage in ...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : MySQL, Apache Spark, Google BigQuery Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks that drive innovation, efficiency, and business transformation.Roles & Responsibilities:Lead AI-driven solution design and delivery by applyi...

Your role and responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform...

Key Responsibilities: Designing and developing robust PySpark applications for large-scale data processing Building and optimizing data ingestion, transformation, and storage processes Implementing efficient algorithms and data structures for distributed computing Collaborating with cross-functional teams to integrate data-driven solutions into business processes Troubleshooting performance bottlenecks and ensuring high availability and reliability of data pipelines Writing and optimizing SQL queries for data extraction and manipulation Required Skills and Qualifications : Bachelors/Masters degree in Computer Science, Engineering, Proven experience (3-10 years) in Python development with a f...

About The Role Project Role : Custom Software Engineer Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : SAP FI CO Finance Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that project goals are met, facilitating discussions to address challenges, and guiding your team through the development process. You ...

ABOUT THE ROLE Role Description: The role is responsible for designing, building, maintaining , analyzing, and interpreting data to provide actionable insights that drive business decisions. This role involves working with large datasets, developing reports, supporting and executing data governance initiatives and visualizing data to ensure data is accessible, reliable, and efficiently managed. The ideal candidate has strong technical skills, experience with big data technologies, and a deep understanding of data architecture and ETL processes Roles & Responsibilities: Design, develop, and maintain data solutions for data generation, collection, and processing Be a key team member that assis...

About The Role Project Role : Custom Software Engineer Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : SAP FI CO Finance Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure that project goals are met, facilitating discussions to address challenges, and guiding your team through the development process. You ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...