908 Apache Spark Jobs - Page 9

As an AI Architect specializing in Agentic AI and Generative AI, your role will involve designing, developing, and deploying cutting-edge autonomous AI systems. You will be responsible for building LLM-powered agents with memory, tool usage, planning, and reasoning capabilities to create intelligent, goal-oriented systems. Your technical leadership will oversee AI initiatives from research and architecture design to deployment in cloud environments. Key Responsibilities: - Architect and construct LLM-based agents for autonomous task execution, memory management, tool usage, and multi-step reasoning. - Develop modular, goal-oriented agentic systems utilizing tools like LangChain, Auto-GPT, Cr...

Role Overview: As a Senior Data Scientist, you will play a crucial role in leading data-driven projects and shaping the data science strategy of the company. Your responsibilities will include developing advanced algorithms, managing large datasets, and mentoring junior team members. Key Responsibilities: - Design and implement complex statistical models and machine learning algorithms. - Lead data science projects from inception to deployment, ensuring high-quality deliverables. - Provide technical mentorship and guidance to junior data scientists. - Collaborate with business stakeholders to define project objectives and offer actionable insights. - Explore and implement new technologies an...

Role Overview: As a Research Scientist at Adobe, you will play a crucial role in the Media and Data Science Research Laboratory within the Digital Experience business unit. Your primary responsibility will involve designing, implementing, and optimizing machine learning algorithms to address real-world problems. Your focus will be on understanding user behavior, enhancing marketing performance through content generation, and developing scalable synthetic data generation techniques. Additionally, you will collaborate with various teams to prototype and transition research concepts into production. Key Responsibilities: - Design, implement, and optimize ML algorithms for understanding user beh...

Your opportunity to make a real impact and shape the future of financial services is waiting for you. Let's push the boundaries of what's possible together. As a Senior Director of Software Engineering at JPMorgan Chase within the Consumer and Community Banking division, you will be responsible for leading various technical domains, overseeing the activities of multiple departments, and fostering cross-functional collaboration. Your technical expertise will be utilized across different teams to promote the adoption and implementation of advanced technical methods, helping the firm stay ahead of industry trends, best practices, and technological advancements. **Role Overview:** In this role, ...

As a Generative AI Developer, your role will involve designing, developing, and implementing generative models using cutting-edge techniques in AI and ML, including GANs, VAEs, and Transformer-based architectures. You will work on reinforcement learning and generative adversarial networks, while also being familiar with data engineering and big data technologies. Your strong programming skills in Python, TensorFlow, PyTorch, Keras, Apache Spark, and Hugging Face transformers will be essential for this role. Additionally, you should be comfortable working with large scale datasets and distributed computing frameworks. Your innovative thinking and passion for exploring new ideas will be crucia...

As an Applications Development Group Manager at Citigroup, you will be responsible for managing a team to establish and implement new or revised application systems and programs in coordination with the Technology Team. Your key responsibilities include: - Managing multiple teams to achieve established goals and conducting personnel duties such as performance evaluations, hiring, and disciplinary actions - Providing strategic influence, exercising control over resources and budget management, and monitoring end results - Utilizing in-depth knowledge to resolve issues and contribute to defining standards - Integrating applications development knowledge with the overall technology function to ...

Java BE with Apache spark & Azure Experience: 8 to 12 Years Location: Bengaluru, Chennai, Hyderabad, Pune, Kolkata Joining Timeline: October & November 2025 Job Summary: We are seeking a highly skilled and experienced Java Backend Developer with strong expertise in Spring Boot , Apache Spark , and Microsoft Azure . The ideal candidate will be responsible for designing, developing, and maintaining scalable backend systems and services that power enterprise-grade applications. Key Responsibilities: Design and implement robust backend solutions using Java and Spring Boot. Develop and optimize data processing pipelines using Apache Spark. Integrate and deploy applications on Microsoft Azure clou...

Name of Position: Databricks Lead No. of Positions : 1 Company Name: Whiteblue Cloud Services Location: Chennai, Tamil Nadu Job Description Role Overview: We are looking for an experienced Databricks Lead to design, develop, and optimize large-scale data solutions leveraging Azure Databricks, Azure Data Factory, and PySpark . The ideal candidate will have strong leadership skills, hands-on technical expertise, and a passion for driving cloud-based data modernization initiatives. Key Responsibilities Lead end-to-end data engineering and modernization projects using Azure Databricks and related technologies. Design, develop, and maintain scalable data pipelines and workflows using Azure Data F...

Description The Data Engineering Lead is a senior-level position within our Data Engineering team, responsible for spearheading the design, implementation, and optimization of advanced data architectures using Databricks. This role involves leading a team of data engineers, collaborating with business and data analysts, and business stakeholders, and delivering scalable, secure data solutions tailored to the needs of commercial analytics in the pharmaceutical industry. Responsibilities 1. Architecture Design Develop and maintain scalable data architectures using Databricks, including data lakes, data warehouses, and real-time processing systems. Create detailed blueprints for data processes ...

Role: Data Engineering Lead Location: Bangalore (Work from Office) Experience: 12+ Years Notice Period: Immediate to 30 Days Job Description: Design, develop, and optimize scalable data architectures using Databricks and Apache Spark . Build and maintain ETL/ELT pipelines for large-scale data processing. Ensure data governance , security , and compliance across the Databricks environment. Collaborate with analytics, marketing, and business teams to deliver data-driven solutions. Integrate Databricks with AWS/Azure/GCP , and CRM tools like Salesforce or Veeva . Lead and mentor a team of data engineers, ensuring delivery excellence. Implement Medallion Architecture and Delta Lake for efficient...

In this role, you will be developing and implementing quantitative software applications to process and analyze real-time financial market data in a high-performance computing environment. Your responsibilities will include maintaining and optimizing existing software applications, as well as recommending and implementing improvements. You will design and develop solutions using Python and PySpark in a spark/Hadoop environment. Additionally, you will build, optimize, and troubleshoot ETL workflows to extract, transform, and load data from diverse sources into big data platforms. Collaborating with data engineers, data scientists, and business stakeholders to gather requirements and deliver d...

As a Data Engineer, your role involves building products in the data analytics space with a focus on shipping high-quality data products and contributing to the overall company and product development. You should possess the right attitude, aptitude, skills, empathy, compassion, and a hunger for learning. Here are the key responsibilities and qualifications required for this position: **Key Responsibilities:** - Understand business requirements and implement scalable solutions. - Translate complex technical and functional requirements into detailed designs. - Develop a highly scalable, reliable, and high-performance data processing pipeline for extracting, transforming, and loading data from...

Role Overview: As an Applications Development Group Manager at Citigroup, you will be responsible for managing a team to establish and implement new or revised application systems and programs in coordination with the Technology Team. Your role involves managing multiple teams to achieve established goals, conducting personnel duties such as performance evaluations, hiring, and disciplinary actions, and providing strategic influence to exercise control over resources and budget management while monitoring end results. You will utilize your in-depth knowledge to resolve issues, contribute to defining standards, and integrate applications development knowledge with the overall technology funct...

As a Data Science/AI/ML professional at Cybage in Pune, you will have the opportunity to work with a leading company specialized in outsourced product engineering services. Cybage collaborates with top independent software vendors and utilizes a data-driven system called Decision Mines for Digital Excellence to provide innovative solutions leveraging modern technologies. Key Responsibilities: - Design, implement, and optimize ML pipelines with tools like TensorFlow, PyTorch, MLflow, Scikit-learn, or ONNX. - Operationalize models using CI/CD for ML practices (MLOps) on platforms like Kubeflow, Azure ML, or SageMaker. - Collaborate with data scientists to convert prototypes into production-lev...

You are an accomplished and dynamic Data Engineering Lead responsible for driving the design, development, and delivery of cutting-edge data solutions. Your role involves overseeing the design and development of robust data pipelines and systems, managing and mentoring a team of 5 to 7 engineers, and playing a critical role in architecting innovative solutions tailored to client needs. You will lead by example, fostering a culture of accountability, ownership, and continuous improvement while delivering impactful, scalable data solutions in a fast-paced consulting environment. Key Responsibilities: - Architecture & Delivery: - Lead the end-to-end architecture of data pipelines leveraging Azu...

As an AI Architect specializing in Agentic AI and Generative AI, your primary responsibility will be to design, develop, and deploy cutting-edge autonomous AI systems. You will focus on building LLM-powered agents with memory, tool usage, planning, and reasoning capabilities to create intelligent, goal-oriented systems. Your role as a technical leader will involve overseeing AI initiatives from research and architecture design to deployment in cloud environments. Key Responsibilities: - Architect and construct LLM-based agents capable of autonomous task execution, memory management, tool usage, and multi-step reasoning. - Develop modular, goal-oriented agentic systems utilizing tools like La...

As a member of the Managed Services team at PwC, you will focus on providing outsourced solutions and supporting clients across various functions. Your role will involve streamlining operations, reducing costs, and enhancing efficiency by managing key processes and functions on behalf of organizations. You will leverage your skills in project management, technology, and process optimization to deliver high-quality services to clients. Specifically, in managed service management and strategy, your responsibilities will include transitioning and running services, managing delivery teams, programmes, commercials, performance, and delivery risk. Your work will revolve around continuous improveme...

As a Data Engineer II, your role involves managing the deprecation of migrated workflows and ensuring the seamless migration of workflows into new systems. Your expertise in building and maintaining scalable data pipelines, both on-premises and on the cloud, will be crucial. You should have a deep understanding of input and output data sources, upstream downstream dependencies, and data quality assurance. Proficiency in tools like Git, Apache Airflow, Apache Spark, SQL, data migration, and data validation is essential for this role. Your key responsibilities will include: - Workflow Deprecation: - Evaluate current workflows" dependencies and consumption for deprecation. - Identify, mark, and...

As a Senior Data Engineer specializing in Data Quality, Ingestion & API Development, your role will involve leading the development of a scalable data ingestion framework with a strong focus on ensuring high data quality and validation. You will also be responsible for designing and implementing robust APIs for seamless data integration. Your expertise in building and managing big data pipelines using modern AWS-based technologies will be crucial for driving quality and efficiency in data processing systems. - **Data Ingestion Framework:** - **Design & Development:** Architect, develop, and maintain an end-to-end data ingestion framework that efficiently extracts, transforms, and loads data ...

As a senior member of the team responsible for developing large-scale, highly available, and fault-tolerant next-generation SaaS solutions for Health Care Analytics, your role will involve the following key responsibilities: - Design and implement scalable, secure, and maintainable enterprise-level solutions. - Establish coding standards, best practices, and architectural guidelines. - Integrate business process management solutions with existing systems. - Ensure system architectures align with business requirements and technology roadmap. Your core technical skills should include: - Advanced Java development with extensive experience in Java 21 features. - Deep understanding of Spring Fram...

About the role: Data Management roles are skills and expertise in the secure storage and management of data in compliance with guidelines, standards and best practice They enable efficient and high quality data analysis by designing and implementing approaches to clean and transform data and store data in the most appropriate way Purpose of Position: We are seeking a highly skilled and experienced Data Engineering Lead to spearhead our data architecture strategy, oversee robust data engineering initiatives, and manage data lake operations The ideal candidate will be a hands-on leader with deep expertise in building and optimizing data platforms that support advanced analytics and business in...

Role Overview: At PwC, your focus in managed services will involve providing outsourced solutions and supporting clients across various functions. You will assist organisations in streamlining operations, reducing costs, and enhancing efficiency by managing key processes and functions on their behalf. Your expertise in project management, technology, and process optimization will enable you to deliver high-quality services to clients. As part of the managed service management and strategy team at PwC, you will be responsible for transitioning and running services, managing delivery teams, programmes, commercials, performance, and delivery risk. Your role will revolve around continuous improv...

Role Overview: As a Spark (Scala) Developer at Viraaj HR Solutions, your primary responsibility is to design, develop, and implement data processing applications using Apache Spark and Scala. You will work closely with cross-functional teams to gather requirements, define project specifications, and build scalable ETL pipelines for efficient data processing. Your role also involves optimizing Spark jobs for performance, utilizing SQL for data querying and transformation, and developing data models to support analytics initiatives. Additionally, you will be engaged in data analysis, integration of big data tools and frameworks, code reviews, troubleshooting application issues, and ensuring hi...

You will be joining GlobalLogic to work on a significant software project for a world-class company specializing in M2M / IoT 4G/5G modules for industries like automotive, healthcare, and logistics. Your role will involve contributing to the development of end-user modules" firmware, implementing new features, ensuring compatibility with the latest telecommunication and industry standards, and analyzing customer requirements to provide estimations. - Design end-to-end AI/ML architecture and solutions in alignment with business objectives - Lead the development and deployment of ML models using frameworks like TensorFlow, PyTorch, or Scikit-learn - Architect scalable data pipelines for traini...

Role Overview: Do you aspire to drive the next level of application architecture in the Compliance IT space Are you passionate about applying Agile methodologies at scale and developing a cutting-edge Data Ingestion, Aggregation & Calculation Platform on Cloud Would you like to contribute to shaping the future of Compliance technology alongside a dynamic cross-functional team If so, we have an exciting opportunity for you as a seasoned full stack developer. In this role, you will be responsible for implementing and delivering high-quality software solutions for our state-of-the-art Data Ingestion, Aggregation & Calculation platform. Key Responsibilities: - Implement and deliver high-quality ...