530 Apache Spark Jobs - Page 5

You should possess a qualification of M.Tech / M.E / MS / M.Sc / Ph.D in Computer Science or a related discipline (Applied Mathematics, Statistics, Electrical and/or Computer Engineering) with a focus on Artificial Intelligence / Machine Learning. A CGPA of 7.5/10 or above in UG as well as PG level (in case of percentage, 70% or above) is required. You should have a minimum of 2 years of experience. As a team member, you will be responsible for applied research, development, and validation of advanced machine learning and AI algorithms for solving complex real-world problems at scale. Collaborating closely with the product team, you will analyze business problems and product vision to innova...

A career at HARMAN Automotive offers you the opportunity to be part of a global, multi-disciplinary team dedicated to leveraging the innovative power of technology to transform the future. At HARMAN Automotive, you will have the chance to fast-track your career by engineering audio systems and integrated technology platforms that enhance the driving experience. Through a combination of ingenuity, thorough research, and a collaborative spirit, you will contribute to advancing in-vehicle infotainment, safety, efficiency, and enjoyment. As a Principal Engineer at HARMAN, your role will involve contributing to the design and development of new software, including prototyping, writing functional ...

As a Software Engineer at micro1, you will play a crucial role in developing robust and scalable backend and data engineering solutions. Your expertise in data engineering, backend development, and cloud infrastructure will be utilized to design, build, and maintain cutting-edge systems. You will have the opportunity to work with industry-leading technologies and be at the forefront of AI innovation. Your responsibilities will include working with distributed processing systems such as Apache Spark to create data pipelines, deploying applications on AWS cloud platforms using services like EKS, S3, EC2, and Lambda, containerizing and orchestrating services with Docker and Kubernetes (EKS), an...

You will be part of the Cloud Platform Engineering team at Motorola Solutions, supporting the development and operations of critical cloud infrastructure, reliability, security, and business operational services for the company's public and hybrid cloud-based Software as a Service (SaaS) solutions. Your role will involve the following responsibilities: - Supporting the cloud monthly budget and forecast process by managing consumption, chargeback, and savings in coordination with accounting, finance, and technical owners. - Assisting stakeholders in understanding their cloud spend based on their business roadmap and budgeted forecast. - Monitoring cloud consumption at the account level to ide...

We are seeking a skilled Senior Developer & Tech Lead who is dedicated to producing clean and effective code, constructing scalable systems, fostering engineering excellence, and overseeing a team of proficient developers in a dynamic Agile setting. This position is well-suited for individuals with extensive practical experience in Java and Apache Spark, coupled with a solid understanding of object-oriented design principles. Your responsibilities will include conducting thorough impact analyses for code modifications while considering dependencies across various application components, designing and implementing scalable, high-performance code utilizing Java and Bigdata/Apache Spark, crafti...

You are a highly skilled and experienced Data Architect with expertise in cloud-based solutions. You will be responsible for designing, implementing, and optimizing data architecture to meet the organization's current and future needs. Your role will involve data modeling, transformation, governance, and hands-on experience with modern cloud platforms and tools such as Snowflake, Spark, Data Lakes, and Data Warehouses. Collaboration with cross-functional teams and stakeholders is crucial, and you will establish and enforce standards and guidelines across data platforms to ensure consistency, scalability, and best practices. You will be accountable for architecting and implementing scalable, ...

As a Spark Developer, you will play a crucial role in designing, developing, and optimizing large-scale data processing pipelines using Apache Spark. Your responsibilities will include designing and implementing scalable, high-performance data pipelines, optimizing Spark applications for efficient processing of large datasets, collaborating with cross-functional teams to understand data requirements, integrating Spark jobs with various data sources, monitoring and troubleshooting Spark jobs, writing clean and well-documented code, implementing best practices for data processing, security, and storage, and staying up to date with the latest trends in big data technologies. To excel in this ro...

As a Data Product Manager at our company based in Mumbai, you will play a crucial role in driving the development of data-driven products that align with our business objectives. Your responsibilities will involve collaborating with data engineers, analysts, and stakeholders to build and scale data-powered solutions. By defining and executing the data product roadmap, you will ensure that our data strategy meets both company goals and customer needs. Additionally, you will work on developing product features that enhance data accessibility, analytics, and AI/ML adoption. It will be your responsibility to integrate data-driven insights into products and decision-making processes while ensurin...

As a Back End Developer at Infilytics, you will be responsible for software and web development by utilizing Object-Oriented Programming (OOP) principles. Your primary role will involve designing, developing, and maintaining backend services using Java and Spring Boot. You will be instrumental in building and integrating RESTful APIs, working with databases through Hibernate, JDBC, and writing optimized SQL queries. Additionally, you will be tasked with processing and analyzing large datasets using Apache Spark. Collaboration is key in this role, as you will work closely with front-end developers and other team members to ensure the delivery of high-quality software. It will be essential for...

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

JD: • Good experience in Apache Iceberg, Apache Spark, Trino • Proficiency in SQL and data modeling • Experience with open Data Lakehouse using Apache Iceberg • Experience with Data Lakehouse architecture with Apache Iceberg and Trino

As a Fullstack Developer / Tech Lead at our leading insurance technology company, you will be responsible for working in Indore & Bengaluru with 6-8 years of experience. Your technical skills will include Node.js, Nest.js, React.js, Next.js & SQL/Postresql. Joining us as a Backend Developer (SDE 3) in Bengaluru, Indore, Delhi with 5+ years of experience, you will be proficient in Node.js, Nest.js, and SQL/Postgresql. In the role of Data Engineer in Bengaluru & Indore, with 3-6 years of experience, your technical skills should include SQL/Posgresql with a preference for posgresql, along with proficiency in Python, Node.js, and knowledge of Big data technologies such as Apache Spark and Hadoop...

As the Lead Developer for the next-generation data processing engine at Datapelagos, you will play a pivotal role in overseeing the architecture, design, and implementation of our cutting-edge technology that leverages accelerated computing capabilities. Your primary responsibilities will include collaborating with cross-functional teams to enhance the core architecture of Datapelagos" parallel and distributed execution engine, which is based on accelerated computing. You will lead the execution engine team in designing, implementing, and deploying an enterprise-grade acceleration engine for data processing, ensuring high reliability and performance. In this role, you will be responsible for...

The Data Engineer / Machine Learning Engineer with GenAI Expertise position in Hyderabad, Telangana, IN is a full-time role that requires a highly skilled and motivated individual to join the team. The ideal candidate should possess expertise in Python, Apache Spark, Snowflake, Machine Learning (ML), and Generative AI (GenAI) technologies. This role involves designing and implementing data solutions, developing machine learning models, and utilizing generative AI to address complex business challenges. Key Responsibilities: Data Engineering and Development - Design, build, and maintain scalable data pipelines using Apache Spark and Snowflake. - Develop ETL workflows to ingest, transform, and...

As a Backend Engineer, you will be responsible for architecting and constructing intricate, scalable, and high-performance backend systems. Your role will involve overseeing the complete development and deployment process of features and enhancements. You will be tasked with designing and refining extensive distributed data processing using technologies such as Apache Spark and Nifi. Collaboration with product managers, architects, and various stakeholders to ensure alignment on technical objectives will be a key aspect of your responsibilities. Additionally, you will lead code reviews to guarantee adherence to best practices and maintain high code quality standards. A crucial part of your r...

You will be responsible for designing, developing, and implementing big data solutions using Databricks and Apache Spark. Collaborate with cross-functional teams to gather requirements and design big data solutions. Develop and maintain Databricks notebooks, jobs, and workflows. Troubleshoot and resolve big data processing issues. Optimize Databricks solutions for performance, scalability, and cost-effectiveness. Collaborate with other teams to ensure seamless integration of big data solutions. Develop and maintain technical documentation. Requirements: - 6-10 years of experience in big data engineering and development. - Strong experience with Databricks and Apache Spark. - Experience with ...

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. As a Software Engineer III at JPMorgan Chase within the Risk Technology, you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for carrying out critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives. You will execute software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches to build solutions or brea...

Your opportunity to make a real impact and shape the future of financial services is waiting for you. Let's push the boundaries of what's possible together. As a Senior Director of Software Engineering at JPMorgan Chase within the Consumer and Community Banking division, you will be responsible for leading various technical domains, overseeing the activities of multiple departments, and fostering cross-functional collaboration. Your technical expertise will be utilized across different teams to promote the adoption and implementation of advanced technical methods, helping the firm stay ahead of industry trends, best practices, and technological advancements. Leads multiple technology and pro...

You are a Spark, Big Data - ETL Tech Lead for Commercial Cards Global Data Repository development team at Citigroup. In this role, you will collaborate with the Development Project Manager, development, testing, and production support teams, and various departments within Citigroup to ensure the success of TTS platforms. Your exceptional communication skills across technology and business realms will be crucial as you play a visible role as a technical lead in building scalable, enterprise-level global applications. Your responsibilities will include leading the design and implementation of large-scale data processing pipelines using Apache Spark on the BigData Hadoop Platform. You will deve...

Key Skills: Spark, Azure Databricks, Apache Spark, Python, Delta Lake, Kafka, Snowflake, Qlik, CI/CD, Azure Roles & Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Databricks and Spark. Implement data processing solutions with Delta Lakes and manage multiple file formats. Collaborate with cross-functional teams to ensure data quality and integrity. Utilize CI/CD practices for efficient deployment and management of data solutions. Work with Snowflake and Qlik for data visualization and reporting. Leverage Kafka or similar streaming technologies for real-time data processing. Ensure compliance with data governance and security standards in Azure. Experience ...

Data Software Engineer Job Description: 1. 5-12 Years of in Big Data & Data related technology experience 2. Expert level understanding of distributed computing principles 3. Expert level knowledge and experience in Apache Spark 4. Hands on programming with Python 5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop 6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming 7. Experience with messaging systems, such as Kafka or RabbitMQ 8. Good understanding of Big Data querying tools, such as Hive, and Impala 9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files 10. Good under...

Role & responsibilities: The Data Engineer, Specialist is responsible for building and maintaining scalable data pipelines to ensure seamless integration and efficient data flow across various platforms. This role involves designing, developing, and optimizing ETL (Extract, Transform, Load) processes, managing databases and leveraging big data technologies to support analytics and business intelligence initiatives. Preferred candidate profile: Design, develop and maintain scalable ETL (Extract, Transform, Load) processes to efficiently extract data from various structured and unstructured sources, ensuring accuracy, consistency and performance optimization. Architect and manage database syst...

Role & responsibilities: The Manager, Data Engineering is responsible for leading a team of data engineers to design, develop, and optimize scalable data pipelines and infrastructure. This role focuses on enabling high-performance data workflows, ensuring seamless data integration and driving automation to enhance data accessibility. Preferred candidate profile : Hire, mentor and manage a team of data engineers, fostering a culture of innovation, collaboration and continuous learning in data engineering best practices. Design, develop and maintain scalable ETL (Extract, Transform, Load) pipelines to efficiently process and integrate structured and unstructured data from multiple sources. Bui...

As a qualified candidate for this role, you have a minimum of 5 years of experience in Apache Spark along with proficiency in Scala/Java/Kotlin. It is essential that you possess expertise in Scala language with spark, Hadoop, Hive, and SQL. Your educational background includes a 4-year degree in Quantitative disciplines or equivalent experience, or an MS in Computer Science, Applied Mathematics, Statistics, Physics, or relevant industry experience. With at least 6 years of experience in end-to-end application development, data exploration, data pipelining, API design, and model latency optimization, you are well-versed in deploying Machine Learning algorithms into production environments. Yo...