530 Apache Spark Jobs - Page 13

We are looking for a highly skilled and motivated Python, AWS, Big Data Engineer to join our data engineering team. The ideal candidate should have hands-on experience with the Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. Your responsibilities will include designing, developing, and optimizing scalable data pipelines and big data solutions to support analytics and business intelligence initiatives. Virtusa is a company that values teamwork, quality of life, and professional and personal development. We are proud to have a team of 27,000 people globally who care about your growth and seek to provide you with exciting projects, opportunities, ...

*APPLICABLE ONLY FOR IMMEDIATE JOINERS* Job Title: Big Data Developer Location: Pune Work Type: Full-Time Experience Required: 3+ Years Position Overview: We are seeking a talented Software Development Engineer- Big Data with strong expertise in big data technologies. The ideal candidate will have hands-on experience with Apache Spark, Hadoop, AWS (Amazon Web Services), Kafka, NoSQL databases (e.g., MongoDB, Cassandra), Hive, Docker, Kubernetes, and proficiency in SQL and one or more of the coding languages such as Java, Scala, Python. This role requires a deep understanding of software development principles, distributed systems, and container orchestration. Responsibilities: Design, develo...

Position Overview We are seeking a skilled Data Engineer with 2-4 years of experience to design, build, and maintain scalable data pipelines and infrastructure. You will work with modern data technologies to enable data-driven decision making across the organisation. Key Responsibilities Design and implement ETL/ELT pipelines using Apache Spark and orchestration tools (Airflow/Dagster). Build and optimize data models on Snowflake and cloud platforms. Collaborate with analytics teams to deliver reliable data for reporting and ML initiatives. Monitor pipeline performance, troubleshoot data quality issues, and implement testing frameworks. Contribute to data architecture decisions and work with...

As a Senior Data Engineer + AI, you will play a crucial role in designing and optimizing distributed data pipelines using PySpark, Apache Spark, and Databricks to cater to both analytics and AI workloads. Your expertise in PySpark, Apache Spark, and Databricks for batch and streaming data pipelines will be instrumental in contributing to high-impact programs with clients. Your strong SQL skills for data analysis, transformation, and modeling will enable you to drive data-driven decision-making and facilitate rapid insight generation. Your responsibilities will involve supporting RAG pipelines, embedding generation, and data pre-processing for LLM applications, as well as creating and maintai...

As a Python Back End Developer, you will be responsible for various day-to-day tasks related to back-end web development, software development, and object-oriented programming (OOP). This contract role offers a hybrid work model, with the primary location being Hyderabad and some flexibility for work-from-home. You should possess proficiency in Back-End Web Development and Software Development, along with a strong understanding of Object-Oriented Programming (OOP). Basic skills in Front-End Development, solid programming skills, and experience with Cloud platforms such as GCP/AWS are essential for this role. Additionally, excellent problem-solving and analytical skills are required, enabling...

As a Lead Developer at NTT DATA Business Solutions, you will play a crucial role in leading and mentoring team members to foster a collaborative and high-performing environment. Your responsibilities will include designing complex solutions that consider both business and technical aspects, acting as an AI advocate and ambassador, and proposing scalable, cloud-native solutions in collaboration with architects. You will be developing and maintaining backend services primarily using Java and Python, as well as contributing to frontend development using Angular. In addition to providing L3 support and ensuring system reliability, you will collaborate with cross-functional teams to align on tech...

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. As a Software Engineer III at JPMorgan Chase within the Asset & Wealth Management, you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for carrying out critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives. Executes software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches to build solutions or br...

As the heart of our business is driven by technology, we attribute our success to our global and diverse culture. At Quantiphi, we cherish our people and take pride in fostering a culture that thrives on transparency, diversity, integrity, learning, and growth. If the idea of working in an environment that not only encourages you to innovate and excel professionally but also supports your personal growth interests you, then a career with Quantiphi is the right fit for you! Responsibilities: - Creating and maintaining LookML code to define data models, dimensions, measures, and relationships within Looker. - Developing reusable LookML components to ensure consistency and efficiency in report ...

As a Lead Data Engineer, you will be responsible for leading cloud modernization initiatives, developing scalable data pipelines, and enabling real-time data processing for enterprise-level systems. Your expertise in Google Cloud Platform (GCP) and BigQuery will be crucial in driving the transformation of legacy infrastructure into a robust, cloud-native data ecosystem. Your key responsibilities will include analyzing legacy on-premises and hybrid cloud data warehouse environments, leading the migration of large-scale datasets to Google BigQuery, and designing data migration strategies to ensure data quality, integrity, and performance. You will also be responsible for integrating data from ...

At Improzo, we are dedicated to improving life by empowering our customers through quality-led commercial analytical solutions. Our team of experts in commercial data, technology, and operations collaborates to shape the future and work with leading Life Sciences clients. We prioritize customer success and outcomes, embrace agility and innovation, foster respect and collaboration, and are laser-focused on quality-led execution. As a Data and Reporting Developer (Improzo Level - Associate) at Improzo, you will play a crucial role in designing, developing, and maintaining large-scale data processing systems using big data technologies. You will collaborate with data architects and stakeholders...

We are searching for a skilled and adaptable Data Engineer with proficiency in PySpark, Apache Spark, and Databricks, combined with knowledge in analytics, data modeling, and Generative AI/Agentic AI solutions. This position suits individuals who excel at the convergence of data engineering, AI systems, and business insights, contributing to impactful programs with clients. Your responsibilities will include designing, constructing, and enhancing distributed data pipelines utilizing PySpark, Apache Spark, and Databricks to cater to both analytics and AI workloads. You will also be tasked with supporting RAG pipelines, embedding generation, and data pre-processing for LLM applications. Additi...

As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining robust data pipelines on the Databricks platform. Your primary responsibility will be to ensure the smooth flow of data by collaborating with data scientists, analysts, and stakeholders to guarantee data availability and integrity for analytics and reporting purposes. Your key responsibilities will include designing and developing data pipelines using Databricks and Apache Spark to create scalable and efficient solutions. You will be responsible for integrating data from various sources such as databases, APIs, and external datasets, focusing on maintaining data quality and consistency throughout the...

About Position: We are looking for Spark/Scala developer with hands on experience on ETL, Azure Databricks. Role: Spark Scala Developer Location: All Persistent Locations Experience: 8+ Years Job Type: Full Time Employment What You'll Do: Design and Development: Design and develop scalable data processing applications using Apache Spark and Scala. Data Processing: Develop Spark jobs to process large datasets, including data ingestion, transformation, and aggregation. Data Pipeline Development: Build and maintain data pipelines using Spark, Scala, and other related technologies. Data Quality: Ensure data quality and integrity by implementing data validation, data cleansing, and data normaliza...

Job Title: AWS Databricks Developer Experience Required: 6 to 12 Years Location: PAN India (Onsite/Hybrid as per project requirement) Employment Type: Permanent Open Positions: 5 Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines using Databricks (PySpark/Scala) Implement scalable data workflows on AWS (S3, Glue, Lambda, EMR, Redshift) Collaborate with cross-functional teams to build end-to-end data solutions Optimize pipeline performance and ensure data quality and governance Troubleshoot issues across cloud environments and ensure smooth data operations Use CI/CD tools for deployment and manage code via Git Other Details: Interview Drive: This Saturday (July 27th, 2025) ...

As a Scala Developer at our company, you will play a crucial role in designing, building, and enhancing our clients" online platform to ensure optimal performance and reliability. Your responsibilities will include researching, proposing, and implementing cutting-edge technology solutions while adhering to industry best practices and standards. You will be accountable for the resilience and availability of various products and will collaborate closely with a diverse team to achieve collective goals. To excel in this role, we are seeking a highly skilled Scala Developer with over 7 years of experience in crafting scalable and high-performance backend systems. Your expertise in functional prog...

We are seeking an experienced and forward-thinking Lead Data Engineer to spearhead the development of scalable, secure, and high-performance data solutions. You must possess in-depth technical knowledge in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory. Your expertise should also include a solid understanding of data governance, metadata management, and regulatory compliance within the insurance and financial services sectors. Proficiency in developing Python applications, Spark-based workflows, utilizing Delta Lake, and orchestrating jobs with Databricks Workflows or Azure Data Factory is essential. Furthermore, you should be able t...

Job Title: Software Engineering Manager Location: Bangalore Industry: E-commerce / Internet Experience: 5+ years (including 2+ years in a managerial role) Employment Type: Full-time Job Description: We are looking for an experienced and passionate Software Engineering Manager to lead a high-performing backend engineering team working on a cloud-based, customer-facing eCommerce platform. This role offers an opportunity to work on a high-scale product, driving performance, reliability, and innovation. Key Responsibilities: Lead and mentor a team of software engineers, promoting continuous growth and development. Design and architect scalable, cloud-native backend solutions for real-time eComme...

Share resume to : sowmya.v@acesoftlabs.com Hiring Now: Azure Data Engineer (36 Years) – Immediate Opportunity! Location : Bangalore Experience : 3–6 Years Key Responsibilities: Design and build scalable data solutions using Azure Data Factory, Azure Databricks, and Azure Synapse . Develop ETL/ELT pipelines for structured and unstructured data from various sources and formats. Write efficient SQL queries, stored procedures, functions , and dynamic SQL for data transformation and analysis. Implement data lake architecture and optimize data ingestion using Apache Spark with Scala or Python . Handle and maintain Delta Tables within the Databricks environment. Collaborate with DevOps team to supp...

Job Description/Preferred Qualifications We are seeking a highly skilled and motivated MLOps Site Reliability Engineer (SRE) to join our team. In this role, you will be responsible for ensuring the reliability, scalability, and performance of our machine learning infrastructure. You will work closely with data scientists, machine learning engineers, and software developers to build and maintain robust and efficient systems that support our machine learning workflows. This position offers an exciting opportunity to work on cutting-edge technologies and make a significant impact on our organization's success. Responsibilities : Design, implement, and maintain scalable and reliable machine lear...

Job Description/Preferred Qualifications We are seeking a highly skilled and motivated MLOps Site Reliability Engineer (SRE) to join our team. In this role, you will be responsible for ensuring the reliability, scalability, and performance of our machine learning infrastructure. You will work closely with data scientists, machine learning engineers, and software developers to build and maintain robust and efficient systems that support our machine learning workflows. This position offers an exciting opportunity to work on cutting-edge technologies and make a significant impact on our organization's success. Responsibilities : Design, implement, and maintain scalable and reliable machine lear...

As a Lead Data Engineer based in Vijayawada, Andhra Pradesh, you will be responsible for leveraging your extensive experience in data engineering and data architecture to design and develop end-to-end data solutions, data pipelines, and ETL processes. With a Bachelor's or Master's degree in Computer Science, Information Systems, or a related field, along with over 10 years of relevant experience, you will play a crucial role in ensuring the success of data projects. You will demonstrate your strong knowledge in data technologies such as Snowflake, Databricks, Apache Spark, Hadoop, Dbt, Fivetran, and Azure Data Factory. Your expertise in Python and SQL will be essential in tackling complex da...

As a Data Engineer II in our team, you will be responsible for managing the deprecation of migrated workflows and ensuring the seamless migration of workflows into new systems. Your expertise in building and maintaining scalable data pipelines, both on-premises and on the cloud, will be crucial. You should have a deep understanding of input and output data sources, upstream downstream dependencies, and data quality assurance. Proficiency in tools like Git, Apache Airflow, Apache Spark, SQL, data migration, and data validation is essential for this role. Your key responsibilities will include: Workflow Deprecation: - Evaluate current workflows" dependencies and consumption for deprecation. - ...

The Applications Development Programmer Analyst position is an intermediate level role where you will participate in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. Your responsibilities will include utilizing your knowledge of applications development procedures and concepts to identify and define necessary system enhancements. You will be expected to identify and analyze issues, make recommendations, and implement solutions based on your understanding of business processes, system processes, and industry sta...

As an SDE-3 Backend at Crimson Enago, you will lead a team of web developers and play a major role in end-to-end project development and delivery. Your responsibilities will include setting engineering examples, hiring and training the team, and coding 100% of the time. You will collaborate with the Engineering Manager, Principal Engineer, SDE-3 leads, and Technical Project Manager to ensure successful project outcomes. The ideal candidate for this position is an SDE-3 Backend with over 5 years of enterprise backend web experience, particularly in the NodeJS-AWS stack. You should possess excellent research skills to address complex business problems, experience in unit and integration testin...

As a PySpark Data Engineer, you will play a crucial role in developing robust data processing and transformation solutions within our data platform. Your responsibilities will include designing, implementing, and maintaining PySpark-based applications to handle complex data processing tasks, ensuring data quality, and integrating with diverse data sources. To excel in this role, you should possess strong PySpark development skills, experience with big data technologies, and the ability to thrive in a fast-paced, data-driven environment. Your primary responsibilities will involve designing, developing, and testing PySpark-based applications to process, transform, and analyze large-scale datas...