530 Apache Spark Jobs - Page 14

At Crimson Enago, we are dedicated to developing AI-powered tools and services that enhance the productivity of researchers and professionals. Through our flagship products Trinka and RAx, we aim to streamline the stages of knowledge discovery, acquisition, creation, and dissemination. Trinka is an AI-driven English grammar checker and writing assistant tailored for academic and technical writing. Crafted by linguists, scientists, and language enthusiasts, Trinka identifies and rectifies numerous intricate writing errors, including contextual spelling mistakes, advanced grammar issues, and vocabulary enhancements in real-time. Moreover, Trinka offers writing suggestions to ensure professiona...

As a talented Big Data Engineer, you will be responsible for developing and managing our company's Big Data solutions. Your role will involve designing and implementing Big Data tools and frameworks, implementing ELT processes, collaborating with development teams, building cloud platforms, and maintaining the production system. To excel in this position, you should possess in-depth knowledge of Hadoop technologies, exceptional project management skills, and advanced problem-solving abilities. A successful Big Data Engineer comprehends the company's needs and establishes scalable data solutions to meet current and future requirements effectively. Your responsibilities will include meeting wi...

The Digital Success Engineering team is looking for an experienced Marketing Cloud Engineer (SMTS) to join the Customer Engagement Engineering team. In this role, you will be responsible for providing product support to customers and business users by leveraging your strong Salesforce Marketing Cloud platform knowledge, technical expertise, and exceptional business-facing skills. You will work closely with various teams to understand, troubleshoot, and coordinate operational issues within the Customer Engagement Ecosystem. Your responsibilities will include conducting technical requirements gathering, designing and implementing robust solutions for Salesforce Marketing Cloud projects, and en...

Key Skills: Solution Architecture, Google Cloud, Python, Java, Big Data, Apache Spark, Agile Delivery, Scrum, Kanban, CI/CD, DevOps, Cloud Solutions, Software Development, Testing, Production Support. Roles & Responsibilities: Own and manage end-to-end technical deliveries of products within your agile team. Lead technical deliveries for the agile team and the product. Manage activities related to design and development (CTB) as well as production processing support (RTB). Provide support across the full delivery lifecycle, including software development, testing, and operational support, adapting to demand. Create robust technical designs and development strategies for new components to mee...

As an AI Ops Expert, you will be responsible for the delivery of projects with defined quality standards within set timelines and budget constraints. Your role will involve managing the AI model lifecycle, versioning, and monitoring in production environments. You will be tasked with building resilient MLOps pipelines and ensuring adherence to governance standards. Additionally, you will design, implement, and oversee AIops solutions to automate and optimize AI/ML workflows. Collaboration with data scientists, engineers, and stakeholders will be essential to ensure seamless integration of AI/ML models into production systems. Monitoring and maintaining the health and performance of AI/ML sys...

At Citi, we are dedicated to building the future of banking through our cutting-edge technology and global presence. As a part of our team, you will have access to resources that cater to your unique needs, support your well-being, and empower you to plan for your future. We offer programs and services for physical and mental wellness, financial planning support, and continuous learning and development opportunities to enhance your skills and knowledge as you progress in your career. As an Officer, Java Full Stack Developer-C11, based in Pune/Chennai, India, you will play a crucial role in our development team by coding, testing, documenting, and releasing stories. Your responsibilities will...

This position is for Ultimo Software solutions Pvt Ltd (Ultimosoft.com). You will be working as a Java/Scala Developer with the following responsibilities and requirements: - Advanced proficiency in one or more programming languages such as Java and Scala, along with database skills. - Hands-on experience as a Scala/Spark developer. - Self-rated Scala proficiency should be a minimum of 8 out of 10. - Proficiency in automation and continuous delivery methods, along with a deep understanding of the Software Development Life Cycle. - Strong knowledge of agile methodologies like CI/CD, Application Resiliency, and Security. - Demonstrated expertise in software applications and technical processes...

The role is seeking a highly experienced Big Data Engineer with a strong background in Python, Java, or Scala. You will be responsible for designing, building, and maintaining scalable data pipelines and data lake architectures. The ideal candidate should have a proven ability to manage and deliver on complex data engineering projects. This position offers an excellent opportunity to work on scalable data solutions in a collaborative environment. Key responsibilities include developing robust data engineering solutions, working with various technologies such as Apache Spark, Hadoop, Hive, HBase, Kafka, Airflow, and Oozie. Collaboration with cross-functional teams is essential to ensure high ...

As an experienced Big Data Engineer with 6 to 8 years of experience, you will be responsible for designing, developing, and maintaining robust and scalable data processing applications using Apache Spark with Scala. Your key responsibilities include implementing and managing complex data workflows, conducting performance optimization and memory tuning for Spark jobs, and working proficiently with Apache Hive for data warehousing and HDFS for distributed storage within the Hadoop ecosystem. You will utilize your strong hands-on experience with Apache Spark using Scala, Hive, and HDFS, along with proficiency in Oozie workflows, ScalaTest, and Spark performance tuning. Your deep understanding o...

Job Description: As the Digital Transformation Lead at Godrej Agrovet Limited (GAVL) in Mumbai, you will play a crucial role in driving innovation and productivity in the agri-business sector. GAVL is dedicated to enhancing the livelihood of Indian farmers by developing sustainable solutions that enhance crop and livestock yields. With leading market positions in Animal Feed, Crop Protection, Oil Palm, Dairy, Poultry, and Processed Foods, GAVL is committed to making a positive impact on the agricultural industry. With an impressive annual sales figure of 6000 Crore INR in FY 18-19, GAVL has a widespread presence across India, offering high-quality feed and nutrition products for cattle, poul...

As a Data Platform Engineer Lead at Barclays, your role is crucial in building and maintaining systems that collect, store, process, and analyze data, including data pipelines, data warehouses, and data lakes. Your responsibility includes ensuring the accuracy, accessibility, and security of all data. To excel in this role, you should have hands-on coding experience in Java or Python and a strong understanding of AWS development, encompassing various services such as Lambda, Glue, Step Functions, IAM roles, and more. Proficiency in building efficient data pipelines using Apache Spark and AWS services is essential. You are expected to possess strong technical acumen, troubleshoot complex syst...

Greetings From Finix !!! Experience required in Data Engineering or related roles. Strong expertise in Databricks and apache spark. Experience in EDA, SQL, GCP(Bigquery). Strong knowledge required in Python. we are looking for immediate joiners.

You should be proficient in programming languages such as Python and R. You must have a strong understanding of machine learning algorithms and frameworks like TensorFlow, PyTorch, and scikit-learn. Experience with conversational AI platforms, especially Oracle Digital Assistant, is preferred. Additionally, you should have familiarity with data processing tools and platforms such as Apache Spark and Hadoop. It is essential to be familiar with Oracle Cloud Infrastructure (OCI) and its services for deploying ODA & AI/ML models. Experience with natural language processing (NLP) techniques and libraries like NLTK and SpaCy is required. Understanding deep learning architectures for NLP, such as t...

As a skilled Senior Engineer at Impetus Technologies, you will utilize your expertise in Java and Big Data technologies to design, develop, and deploy scalable data processing applications. Your responsibilities will include collaborating with cross-functional teams, developing high-quality code, and optimizing data processing workflows. Additionally, you will mentor junior engineers and contribute to architectural decisions to enhance system performance and scalability. Key Responsibilities: - Design, develop, and maintain high-performance applications using Java and Big Data technologies. - Implement data ingestion and processing workflows with frameworks like Hadoop and Spark. - Collabora...

As a Senior Azure Data Engineer, you will be responsible for designing, building, and optimizing scalable data pipelines and solutions using Databricks and modern Azure data engineering tools. Your expertise in Databricks and Azure services will be crucial in delivering high-quality, secure, and efficient data platforms. Your key skills and expertise should include a strong hands-on experience with Databricks, proficiency in Azure Data Factory (ADF) for orchestrating ETL workflows, excellent programming skills in Python with advanced PySpark skills, solid understanding of Apache Spark internals and tuning, and expertise in SQL for writing complex queries and optimizing joins. You should also...

As a Software Engineer - Backend (Python) with 7+ years of experience, you will be responsible for designing and building the backend components of the GenAI Platform in Hyderabad. Your role will involve collaborating with geographically distributed cross-functional teams and participating in an on-call rotation to handle production incidents. The GenAI Platform offers safe, compliant, and cost-efficient access to LLMs, including Opensource & Commercial ones, while adhering to Experian standards and policies. You will work on building reusable tools, frameworks, and coding patterns for fine-tuning LLMs or developing RAG-based applications. To succeed in this role, you must possess the follow...

You will be responsible for building the most personalized and intelligent news experiences for India's next 750 million digital users. As Our Principal Data Engineer, your main tasks will include designing and maintaining data infrastructure to power personalization systems and analytics platforms. This involves ensuring seamless data flow from source to consumption, architecting scalable data pipelines to process massive volumes of user interaction and content data, and developing robust ETL processes for large-scale transformations and analytical processing. You will also be involved in creating and maintaining data lakes/warehouses that consolidate data from multiple sources, optimized f...

As a Java/Scala Developer at our company located in Hyderabad, you will play a crucial role in developing and maintaining software applications. Your responsibilities will include implementing functional programming methodologies, engaging in test-driven development, and building microservices. Collaboration with cross-functional teams will be essential to ensure the delivery of high-quality software solutions. To excel in this role, you should possess advanced skills in one or more programming languages such as Java and Scala, along with database proficiency. Prior hands-on experience as a Scala/Spark developer is required. You should be able to self-rate your Scala expertise as a minimum o...

Sciative is on a mission to create the future of dynamic pricing powered by artificial intelligence and big data. Our Software-as-a-Service products are used globally across various industries - retail, ecommerce, travel, and entertainment. We are a fast growth-oriented startup, with an employee strength of 60+ employees based in a plush office at Navi Mumbai. With our amazing result-oriented product portfolio, we want to become the most customer-oriented company on the planet. To get there, we need exceptionally talented, bright, and driven people. If you'd like to help us build the place to find and buy anything online, this is your chance to make history. We are looking for a dynamic, org...

The Content and Data Analytics team is a part of DataOps within Global Operations at Elsevier, focusing on providing data analysis services primarily using Databricks. The team primarily serves product owners and data scientists of Elsevier's Research Data Platform, contributing to the delivery of leading data analytics products for scientific research, including Scopus and SciVal. As a Senior Data Analyst at Elsevier, you are expected to have a solid understanding of best practices and the ability to execute projects and initiatives independently. You should be capable of creating advanced-level insights and recommendations, as well as leading analytics efforts with high complexity autonomo...

Job Description: As a Software and Data Science Engineer, you will be a crucial part of our team, bringing your strong engineering background and analytical skills to the table. Your main responsibility will be to develop and maintain our platform, utilizing large-scale data to extract valuable insights and drive business growth. Your role will involve working on both front-end and back-end components, requiring collaboration with various technical and non-technical stakeholders. In this role, we highly value your analytical mindset and problem-solving skills in utilizing data structures, storage systems, cloud infrastructure, front-end frameworks, and other technical tools to address comple...

As an experienced data engineer specializing in dashboard story development and data engineering pipelines, you will play a crucial role in analyzing log data to extract actionable insights for product enhancements and feature optimization. With 5+ years of hands-on experience, you will collaborate with cross-functional teams to gather business requirements, translate them into technical specifications, and design interactive dashboards using tools like Tableau, Power BI, or ThoughtSpot AI. You will be responsible for managing large volumes of application log data using Google Big Query, ensuring data integrity, consistency, and accessibility for analytical purposes. Your expertise in identi...

Must have : - Strong on programming languages like Python, Java - One cloud hands-on experience (GCP preferred) - Experience working with Dockers - Environments managing (e.g venv, pip, poetry, etc.) - Experience with orchestrators like Vertex AI pipelines, Airflow, etc. - Understanding of full ML Cycle end-to-end - Data engineering, Feature Engineering techniques - Experience with ML modelling and evaluation metrics - Experience with Tensorflow, Pytorch or another framework - Experience with Models monitoring - Advance SQL knowledge - Aware of Streaming concepts like Windowing , Late arrival , Triggers etc. Good to have : - Hyperparameter tuning experience. - Proficient in either Apache Spa...

As a Mid Databricks Engineer, you will play a pivotal role in designing, implementing, and optimizing data processing pipelines and analytics solutions on the Databricks platform. You will collaborate closely with cross-functional teams to understand business requirements, architect scalable solutions, and ensure the reliability and performance of our data infrastructure. This role requires deep expertise in Databricks, strong programming skills, and a passion for solving complex engineering challenges. What you'll do : - Design and develop data processing pipelines and analytics solutions using Databricks. - Architect scalable and efficient data models and storage solutions on the Databrick...

You will be joining our data engineering team as an experienced Python + Databricks Developer. Your role will involve designing, developing, and maintaining scalable data pipelines using Databricks and Apache Spark. You will write efficient Python code for data transformation, cleansing, and analytics. Collaboration with data scientists, analysts, and engineers to understand data needs and deliver high-performance solutions is a key part of this role. Additionally, you will optimize and tune data pipelines for performance and cost efficiency and implement data validation, quality checks, and monitoring. Working with cloud platforms, preferably Azure or AWS, to manage data workflows will also...