1597 Spark Jobs - Page 9

About The Role Project Role : Software Development Engineer Project Role Description : Analyze, design, code and test multiple components of application code across one or more clients. Perform maintenance, enhancements and/or development work. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Software Development Engineer, you will engage in a dynamic work environment where you will analyze, design, code, and test various components of application code across multiple clients. Your typical day will involve collaborating with team members to ensure the successful implementatio...

JD : Technical Skills: Languages: Scala, Java • Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms) Proficient in developing automated frameworks for unit & integration testing. Experience with Docker and Helm and related container technologies. Proficient in deploying and managing Spark workloads on Kubernetes clusters. Experience in evaluation and implementation of Data Validation & Data Qua...

About The Role Job Title – Ind & Func AI Decision Science Analyst - S&C GN Management Level :11 - Analyst Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can autonomous...

About The Role Job Title – Ind & Func AI Decision Science Analyst - S&C GN Management Level :11 - Analyst Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can autonomous...

About The Role Job Title – Ind & Func AI Decision Science Analyst - S&C GN Management Level :11 - Analyst Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can autonomous...

About The Role Job Title – Ind & Func AI Decision Science Manager - S&C GN Management Level :7 - Manager Location :Open Must have skills:Generative AI, Agentic AI, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration, Machine Learning, Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Manager, you will lead cross-functional teams in the design, development, and deployment of advanced AI agents and multi-modal AI so...

About The Role Job Title – Ind & Func AI Decision Science Manager - S&C GN Management Level :7 - Manager Location :Open Must have skills:Generative AI, Agentic AI, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration, Machine Learning, Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Manager, you will lead cross-functional teams in the design, development, and deployment of advanced AI agents and multi-modal AI so...

About The Role Job Title – Ind & Func AI Decision Science Analyst - S&C GN Management Level :11 - Analyst Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can autonomous...

About The Role Job Title – Ind & Func AI Decision Science Manager - S&C GN Management Level :7 - Manager Location :Open Must have skills:Generative AI, Agentic AI, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration, Machine Learning, Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Manager, you will lead cross-functional teams in the design, development, and deployment of advanced AI agents and multi-modal AI so...

About The Role Job Title – Ind & Func AI Decision Science Consultant - S&C GN Management Level :9 - Consultant Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills :AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can auto...

About The Role Job Title – Ind & Func AI Decision Science Consultant - S&C GN Management Level :9 - Consultant Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills :AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can auto...

About The Role Job Title – Ind & Func AI Decision Science Consultant - S&C GN Management Level :9 - Consultant Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills :AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can auto...

About The Role Job Title – Ind & Func AI Decision Science Consultant - S&C GN Management Level :9 - Consultant Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills :AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can auto...

About The Role Job Title – Ind & Func AI Decision Science Consultant - S&C GN Management Level :9 - Consultant Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills :AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can auto...

About The Role Job Title – Ind & Func AI Decision Science Analyst - S&C GN Management Level :11 - Analyst Location :Gurgaon Must have skills :Generative AI, Agentic AI Systems, Model Context Protocol (MCP), Large Language Models (LLMs), Multi-Agent Systems, Python, SQL, AI Agent Orchestration Good to have skills:AI Refinery, LangChain, LlamaIndex, AutoGen, Crew AI, Spark, Cloud Platforms (AWS, Azure, GCP), NLP, Computer Vision, Vector Databases, RAG (Retrieval-Augmented Generation), Function Calling, Tool Integration Job Summary :As an AI Decision Science Analyst, you will lead the design, development, and deployment of sophisticated AI agents and multi-modal AI solutions that can autonomous...

Should have strong experience in ETL testing with 6 to 9 years Should be strong in SQL query writing Experience with data warehousing and ETL processes Proficiency in Hadoop, Spark, and Hive Knowledge of SQL and NoSQL databases Experience with data warehousing and ETL processes This role involves working closely with data engineers, data analysts, and other stakeholders to ensure the accuracy, efficiency, and reliability of big data systems and applications

The Team: We are looking for a highly motivated, enthusiastic, and skilled engineering leader for S&P Global Energy. We strive to deliver solutions that are sector-specific, data-rich, and hyper-targeted for evolving business needs. The Impact: S&P Global Energy is seeking a Lead Software Developer to provide technical leadership for all stages of software design, development, and maintenance for data processing applications. This person would be part of a development team that manages and supports the internal applications that support Horizon business datasets. This role requires a candidate to handle data processing and develop applications for big data. We have teams composed of individu...

Looking for a skilled AI Data Engineer to join our team in Hyderabad. The ideal candidate will have 2-7 years of experience and a strong background in artificial intelligence, data engineering, and machine learning. Roles and Responsibility Design and develop scalable data pipelines using AI and ML techniques. Collaborate with cross-functional teams to integrate data from various sources. Develop and maintain large-scale data systems and architectures. Ensure data quality and integrity through data validation and testing procedures. Implement data governance policies and standards. Optimize data processing workflows for improved performance and efficiency. Job Requirements Strong knowledge o...

Job Purpose To lead the Search AI capability by developing state-of-the-art search systems and machine learning models, enabling scalable, accurate, and intelligent search experiences across BFL businesses. The role contributes directly to enhancing customer experience, driving efficiency, and building AI-based horizontal capabilities for search, ranking, and personalization. Duties and Responsibilities Lead a team of search engineers to design, build, and optimize horizontal search systems. Develop and prototype algorithms to improve indexing, query relevance, and search performance. Build and manage taxonomy, thesaurus, ontology, and metadata models to improve search accuracy. Partner with...

Bigdata->Pyspark , Bigdata->Python , Bigdata->Scala , Bigdata->Spark , Opensource->Apache Kafka 5+ years of hands on programming experience in Scala backend development and Frameworks such as Akka/Apache Pekko, PlayIn depth understanding of functional programming and Implementation Such as: Try, Option, Future Monad, CollectionsGood knowledge on OOPS Concepts: Classes, Traits and Objects (Singleton and Companion), Case Classes Good Understanding of Scala Language Features: Type System, Implicit/Givens Knowledge of Scala 3 SyntaxExperience on MVC Architecture, Knowledge of Frameworks such as: Akka/Apache Pekko, Play 2. 9, Play 3. 0 Ability to work with clients to identify business challenges ...

Job Requirements 4- 6yrs of experience with strong proficiency in Python and PySpark. Experience working on Spark SQL, RDD, and DataFrame APIs. Good understanding of Hadoop ecosystem (Hive, HDFS, YARN). Knowledge of data formats like Parquet, Avro, JSON, etc. Experience with SQL and writing efficient queries. Familiarity with job orchestration tools (Airflow, Oozie, or similar). Version control systems like Git. Exposure to cloud platforms (AWS/GCP/Azure) is a plus. Key responsibilities: 1. Design, develop, and maintain robust ETL/ELT pipelines using PySpark and other big data technologies. Optimize Spark jobs for performance and scalability. 2. Implement data transformations, aggregations, ...

At Jacobs, we're challenging today to reinvent tomorrow by solving the world's most critical problems for thriving cities, resilient environments, mission-critical outcomes, operational advancement, scientific discovery and cutting-edge manufacturing, turning abstract ideas into realities that transform the world for good. Your impact Proficiency in using cloud services such as AWS, Azure, or Google Cloud for data storage and architecture. Ability to design and manage SQL and NoSQL databases, create data models, and work with database management systems (DBMS). Experience working with Microsoft Fabric, including Lakehouses, Data Pipelines, Notebooks, and semantic models, with a strong unders...

Project and Development Services – Corporate Solutions (country/region) What this job involves: Directing projects every step of the way You’ll develop big ideas that will spark the effective management and successful execution of all phases of a project—from initiating to final handover to the client. You’ll need to carefully identify and take note of our clients’ needs, and figure out what exactly needs to be done in a way that meets JLL’s commercial objectives. This involves defining the scope of the work and expected outcome, at the same time detailing all the necessary objectives to get there. While you do all of these, you’ll need to keep tabs on company resources and allocate them eff...

?Experience: 4-6 yrs ?Qualifications: Bachelors/masters degree in computer science, MIS, Information Technology, or related field of study. ?Strong understanding of data engineering concepts including ETL/ELT processes, data warehousing, and data modeling ?Proficiency in SQL for complex data validation and querying ?Experience with scripting languages such as Python or Shell scripting for automation ?Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Glue) ?Knowledge of data quality frameworks and tools (e.g., Great Expectations, Deequ, custom validation scripts) ?Familiarity with cloud platforms (AWS, GCP) and their data services ?Under...

Job Description Job Title - S&C Global Network - AI - Healthcare Analytics - Consultant Management Level: 9-Team Lead/Consultant Location: Bangalore/Gurgaon Must-have skills: GenAI, RAG, Agentic AI, Python, SQL, AI/ML, Visualization (Power BI or Tableau) Good to have skills: R, Spark, data bricks, Ability to leverage design thinking, business process optimization, and stakeholder management skills. Job Summary: This role involves driving strategic initiatives, managing business transformations, and leveraging industry expertise to create value-driven solutions. Roles & Responsibilities: Deliver strategic advice to clients by providing data-driven recommendations that enhance business outcome...