530 Apache Spark Jobs - Page 8

Expertise and experience of Apache Spark, PySpark, Python based pipeline jobs Solid Data Lake/Data Warehouse principles, techniques and technologies - Star Schema, SQL (AWS EMR, Apache Iceberg, parquet) Strong database skills, NoSQL databases as well as relational databases in use often with large data volumes Strong data modelling concepts and principles experience of building data architectures Cloud preferably: AWS experience working with of APIs (designing with OpenAPI is desirable) CI/CD pipelines (Git-lab desirable) Kubernetes and Docker Experience developing microservices-based architectures, including distributed messaging patterns is a plus

At Crimson Enago, our main focus is on developing AI-powered tools and services that can significantly enhance the productivity of researchers and professionals. We understand that the journey of every researcher or professional involves stages of knowledge discovery, knowledge acquisition, knowledge creation, and knowledge dissemination, all of which can be mentally demanding and interconnected. To address this challenge, we have introduced our flagship products, Trinka and RAx, which are designed to simplify and expedite these four stages seamlessly. Trinka is an AI-powered English grammar checker and language enhancement writing assistant tailored for academic and technical writing. Creat...

The ideal candidate for the Data QA position should have 6 to 10 years of experience in ETL+ DWH Testing and Cloud Testing with expertise in AWS (S3 & Glue) and Azure, along with proficiency in SQL Server, including writing SQL queries. Additionally, familiarity with Snowflake, ADF, and API Testing is essential. Key technical skills required for this role include a strong understanding of Boomi or similar integration tools like MuleSoft, Informatica, Talend, or Workato, experience with Data Bricks, in-depth knowledge of ETL Testing, proficiency in writing SQL Queries, and good technical documentation skills. The candidate should also possess strong communication skills to effectively interac...

A career in the Advisory Acceleration Centre at PwC is an opportunity to leverage PwC's global delivery capabilities and provide premium, cost-effective, and high-quality services to support client engagements. As an Azure Senior Developer at PwC - AC, you will collaborate with the Offshore Manager and Onsite Business Analyst to grasp the project requirements and take charge of the complete implementation of Cloud data engineering solutions. Your role will involve utilizing your expertise in Azure cloud services such as Storage services (Blob, ADLS, PaaS SQL), Azure Data Factory, and Azure Synapse. You should excel in planning, organization, and have the ability to lead as a cloud developer ...

As a part of the data and analytics engineering team at PwC, your focus will be on utilizing advanced technologies and techniques to create robust data solutions for clients. Your role will involve transforming raw data into actionable insights, enabling informed decision-making, and contributing to business growth. Specifically in data engineering at PwC, you will be responsible for designing and constructing data infrastructure and systems that facilitate efficient data processing and analysis. This will include the development and implementation of data pipelines, data integration, and data transformation solutions. At PwC - AC, we are seeking an Azure Manager specializing in Data & AI, w...

As a Senior Data Architect, you will play a crucial role in designing and implementing scalable, secure, and high-performance Big Data architectures using Databricks, Apache Spark, and cloud-native services. Your expertise will be essential in leading the end-to-end data architecture lifecycle, from requirements gathering to deployment and optimization. Additionally, you will design repeatable and reusable data ingestion pipelines for various ERP source systems like SAP, Salesforce, HR, Factory, and Marketing systems. Collaboration with cross-functional teams to integrate SAP data sources into modern data platforms will be a key aspect of your role, along with driving cloud cost optimization...

You are looking for a Principal Data Architect (Contract) role in Jodhpur with AnandRathi IT Pvt Ltd Services. AnandRathi is a prominent financial services firm in India, offering a diverse range of services in wealth management, capital markets, brokerage, investment banking, and financial planning. As part of their digital transformation journey, they are focused on developing a state-of-the-art data ecosystem to enhance insights, scalability, and innovation across all sectors. As a Principal Data Architect, you will play a crucial role in shaping and advancing the enterprise data architecture. Your responsibilities will include defining the data architecture strategy aligned with business...

The Applications Development Technology Lead Analyst position is a senior-level role that involves establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective will be to lead applications systems analysis and programming activities. You will partner with multiple management teams to ensure the integration of functions to meet goals and identify necessary system enhancements for deploying new products and process improvements. Additionally, you will resolve high-impact problems and projects through an in-depth evaluation of complex business processes and industry standards. Your responsibilities will include provi...

The Applications Development Senior Programmer Analyst position involves participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your role will contribute to applications systems analysis and programming activities. Responsibilities include conducting tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establishing and implementing new or revised applications systems and programs to meet specific business needs or user areas. You will monitor and control all phases of the development process, provide user and oper...

At Crimson Enago, we are dedicated to developing AI-powered tools and services that enhance the productivity of researchers and professionals. We understand that the stages of knowledge discovery, acquisition, creation, and dissemination can be cognitively demanding and interconnected. In response to this, our flagship products Trinka and RAx have been designed to streamline these stages and make them efficient and straightforward. Trinka is an AI-powered English grammar checker and language enhancement writing assistant tailored for academic and technical writing purposes. Developed by experts in linguistics, science, and language, Trinka is capable of identifying and correcting a wide arra...

We are seeking a seasoned Senior Developer & Tech Lead who is enthusiastic about writing clean and efficient code, constructing scalable systems, promoting engineering excellence, and supervising a team of skilled developers in a fast-paced, Agile environment. This position is well-suited for developers with extensive hands-on experience in Java and Apache Spark, coupled with a solid understanding of object-oriented design principles. Your responsibilities will include conducting detailed impact analysis for code changes, designing and implementing scalable, high-performance code using Java and Bigdata/Apache Spark, and ensuring the code is of high quality, maintainable, modular, and adheres...

Greetings from Themesoft! We are currently seeking a Senior Associate in GenAI & Data Science to join our CES Innovation team in either Chennai or Pune. In this role, you will have the opportunity to work on cutting-edge GenAI, Natural Language Processing (NLP), and Large Language Models (LLM) solutions to address real-world Environmental, Social, and Governance (ESG) challenges. Key Responsibilities: - Build multi-agentic RAG pipelines using LLMs - Develop NLP solutions utilizing high-quality domain training data - Apply Machine Learning (ML), Optical Character Recognition (OCR), and machine translation techniques to process complex document types - Collaborate closely with stakeholders to ...

As a Data Scientist with 3-8 years of experience, you will collaborate with interdisciplinary scientists, IT, and engineering professionals to solve critical problems and support business decision-making. Strong communication and organizational skills are essential, paired with a problem-solving attitude. Key expertise required for this role includes at least 3 years in analytics, particularly with big data solutions. You should have Subject Matter Expertise (SME) in Statistics, Analytics, Data Science, Machine Learning, Deep Learning, Big Data Platforms (SQL, Hadoop, Apache Spark, etc.), Programming Languages (Python and R), and Techniques such as Statistical models, ML algorithms (e.g., Ra...

A career in our Advisory Acceleration Centre is the natural extension of PwC's leading-class global delivery capabilities. We provide premium, cost-effective, high-quality services that support process,

As a Senior Associate in the Data, Analytics & Specialist Managed Service Tower at PwC, with 6 to 10 years of experience, you will be part of a team of problem solvers addressing complex business issues from strategy to execution. Your responsibilities at this management level include utilizing feedback and reflection for personal development, being flexible in stretch opportunities, and demonstrating critical thinking skills to solve unstructured problems. You will also be involved in ticket quality reviews, project status reporting, and ensuring adherence to SLAs and incident, change, and problem management processes. Seeking diverse opportunities, communicating effectively, upholding ethi...

As a Senior Data Engineer with 9 to 15 years of experience, you will be joining our team in Bangalore, Pune, or Hyderabad in a hybrid work environment. Your role will involve collaborating closely with client teams, particularly ML engineers and product owners, to design, optimize, and implement advanced data and AI solutions. Your responsibilities will include providing technical leadership to clients in the data domain, optimizing and expanding data pipelines using Apache Spark, working with client ML engineers to establish and execute MLOps and LLMOps processes, supporting Generative AI workflows, and utilizing Databricks for advanced analytics and large-scale data processing. Hands-on ex...

A career in our Advisory Acceleration Centre is the natural extension of PwC's leading-class global delivery capabilities. We provide premium, cost-effective, high-quality services that support process,

You have 5+ years of experience in designing and building data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. You possess hands-on expertise with streaming and messaging systems like Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub. Your experience includes creating producers, consumers, and topics, and integrating them into downstream processing. You have a deep understanding of relational databases and Change Data Capture (CDC). You are proficient in SQL Server, Oracle, or other RDBMSs, and have experience capturing change events using tools like Debezium or native CDC tools, transforming them for downstream consumption. Your proficiency extends to ...

As a Senior Associate in GenAI & Data Science at Themesoft, you will be an integral part of our CES Innovation team, focusing on cutting-edge GenAI, NLP, and LLM solutions to address real-world ESG challenges. With a primary presence in either Chennai or Pune, you will utilize your 5+ years of experience to build multi-agentic RAG pipelines using LLMs, develop NLP solutions with high-quality domain training data, and apply ML, OCR, and machine translation techniques to various complex document types. Collaboration with stakeholders to effectively tackle ESG data challenges will also be a key aspect of your role. Your profile should include: - A minimum of 4 years of experience in ML, Python,...

You are a resilient, forward-thinking Python Developer with expertise in AI and Machine Learning deployment, interested in joining our growing technology team as a Python with AI/ML Developer. Your role will involve building, deploying, and maintaining scalable ML models and backend systems to support AI-driven products. To excel in this position, you must have a passion for innovation, attention to detail, and the ability to solve complex problems with effective technical solutions. Your responsibilities will include developing, testing, and deploying machine learning models using Python and AI frameworks such as TensorFlow, PyTorch, and scikit-learn. You will collaborate with data scientis...

We help the world run better. At SAP, you are encouraged to bring out your best. Our organizational culture is centered around collaboration and a shared dedication to improving the world's functionality. We focus daily on laying the groundwork for the future and cultivating a workplace that celebrates diversity, values adaptability, and is dedicated to our purpose-driven and forward-thinking projects. We provide a highly collaborative and supportive team environment that emphasizes continuous learning and growth, acknowledges your unique contributions, and offers a range of benefits for you to select from. In this role, you will: - Collaborate with a team of skilled and motivated developers...

We are looking for a skilled and enthusiastic Applied AI/ML Engineer to be a part of our team. As an Applied AI/ML Engineer, you will be responsible for leading the entire process of foundational model development, focusing on cutting-edge generative AI techniques. Your main objective will be to implement efficient learning methods for data and compute, specifically addressing challenges relevant to the Indian scenario. Your tasks will involve optimizing model training and inference pipelines, deploying production-ready models, ensuring scalability through distributed systems, and fine-tuning models for domain adaptation. Collaboration with various teams will be essential as you work towards...

You are the leading global provider of managed services, cybersecurity, and business transformation for mid-market financial services organizations across the globe. From your unmatched range of services, you provide stability, security, and improved business performance, freeing your clients from technology concerns and enabling them to focus on running their businesses. More than 1,000 customers worldwide with over $3 trillion of assets under management put their trust in you. You believe that success is driven by passion and purpose. Your passion for technology is only surpassed by your commitment to empowering your employees around the world. You have an exciting Opportunity for a Cloud ...

We help the world run better. At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and future-focused work. We offer a highly collaborative, caring team environment with a strong focus on learning and development, recognition for your individual contributions, and a variety of benefit options for you to choose from. You will work with a team of highly technical and motivated developers. Collaborate with stakeholders to understand busine...

At Crimson Enago, we are dedicated to developing AI-powered tools and services that enhance the productivity of researchers and professionals. We understand that the stages of knowledge discovery, acquisition, creation, and dissemination can be cognitively demanding and interconnected. This is why our flagship products, Trinka and RAx, have been designed to streamline and accelerate these processes. Trinka, available at www.trinka.ai, is an AI-powered English grammar checker and language enhancement writing assistant specifically tailored for academic and technical writing. Developed by linguists, scientists, and language enthusiasts, Trinka is capable of identifying and correcting numerous ...