876 Apache Spark Jobs - Page 5

As a Senior Developer with 7 to 10 years of experience in Spark Optimization, Python, and Apache Spark, specializing in Asset and Wealth Management, your role will involve developing and optimizing data solutions to enhance asset management processes. This hybrid position does not require travel and operates during day shifts. - Develop and optimize data processing solutions using Apache Spark to improve efficiency and performance in asset management workflows. - Collaborate with cross-functional teams to design and implement scalable data architectures that support business objectives. - Analyze complex data sets to identify trends and insights that drive strategic decision-making in wealth...

Role Overview: You will serve as the Senior Analytics Engineer at Lennox, taking on the responsibility of designing, developing, and supporting the Azure Databricks platform and Power BI Dashboards & reports. Your key focus will be on independently gathering requirements, designing and building models and reports. You will also be required to interact with various levels of business audiences to clearly define the necessary data to support strategic and tactical decision making. In collaboration with the Data Engineering team, Business Users, and other stakeholders, you will design and deploy end-to-end Analytics solutions to meet business needs. Your involvement in the project will span acr...

As a Scala Developer at the technology services organisation in the IT & Cloud Software sector, your role will involve owning backend services and streaming solutions that deliver low-latency, high-throughput business capabilities for enterprise customers. **Key Responsibilities:** - Design and implement production-grade backend microservices using Scala and functional programming patterns to deliver reliable APIs and business logic. - Develop and optimize batch and streaming data pipelines leveraging Apache Spark and Apache Kafka for real-time processing and analytics. - Build cloud-native applications and deploy services on AWS, collaborating on infrastructure-as-code, security, and cost o...

As a Software Data Engineer at FTSE Russell, you will be part of a team dedicated to building scalable data systems with real-world impact. Your role will involve designing and developing testable data pipelines using Python and Apache Spark. You will orchestrate data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3. Applying modern software engineering practices will be a key part of your responsibilities, including version control, CI/CD, modular design, and automated testing. Additionally, you will contribute to the development of a lakehouse architecture using Apache Iceberg. Collaboration with business teams to translate requirements into data-driven solutions, buildin...

As a Data Engineer at Accenture, you will be responsible for designing, developing, optimizing, and maintaining data pipelines that adhere to ETL principles and business goals. Your role will involve solving complex data problems to deliver insights that help the business achieve its goals. You will source data from various touchpoints, format and organize them into an analyzable format, create data products for analytics team members, and call AI services like vision and translation to generate outcomes for further pipeline steps. Additionally, fostering a culture of sharing, re-use, design, and operational efficiency of data and analytical solutions will be part of your responsibilities. Y...

As a Senior Data Engineer at our company, you will be responsible for designing, building, and optimizing scalable data pipelines on AWS. Your key responsibilities will include: - Implementing data ingestion, transformation, and integration solutions using Spark, Glue, and SQL - Managing and optimizing cloud storage and compute environments - Ensuring robust and automated deployments with Terraform and Jenkins - Collaborating with cross-functional teams to deliver high-quality data products To excel in this role, you must have the following technical skills: - Strong programming skills in Python and/or Scala - Hands-on experience with Apache Spark for big data processing on AWS cloud - Profi...

As a Team Lead and Lead Developer in Backend and Database at Automios Technologies, your primary responsibility will be to architect and oversee the development of backend systems in Node.js. Your strong grasp of MVC patterns, design patterns, and data structures will be crucial in ensuring the efficiency and quality of our products. Additionally, your expertise in database design and structures, including data modeling and proficiency with SQL and NoSQL databases, will play a key role in maintaining and optimizing our data pipelines. Key Responsibilities: - Architect and oversee development of backend systems in Node.js - Implement software architecture design with a strong understanding of...

You will be joining a growing services provider in the Cloud & Data Analytics sector, focusing on building scalable data pipelines, real-time analytics, and cloud-native applications for enterprise customers. As a Junior Cloud & Data Engineer (Trainee) based in India, you will work on developing, testing, and maintaining data ingestion and transformation pipelines using Python and orchestration tools. Additionally, you will implement and support cloud-native services and containerized applications for analytics workloads. Collaborating with senior engineers, you will design ETL/ELT processes, schema models, and batch/streaming solutions. Your responsibilities will also include writing and ma...

Role Overview: As a Technical Manager with expertise in Oracle, PL/SQL, and data workflows optimization on the Databricks platform, you will play a crucial role in designing, developing, and optimizing data solutions. Your deep knowledge in Apache Spark, PySpark, and Python will be essential in supporting scalable data engineering and analytics solutions. You will spend 80% of your time on hands-on development, design, and architecture, with the remaining 20% focused on guiding the team on technology and removing impediments. Capital Markets Projects experience is preferred. Key Responsibilities: - Design and implement scalable data Lakehouse solutions using Databricks and Delta Lake. - Deve...

As a Data Engineer with 5+ years of experience, your role will involve working with data engineering, data analysis, and data architecture. You will be expected to demonstrate strong expertise in SQL and Python. Additionally, you should be familiar with tools such as Databricks, Snowflake, and Power BI. Experience with distributed data systems like Hadoop and Spark, as well as Cloud platforms such as Snowflake, BigQuery, and Redshift, will be crucial for this role. Key Responsibilities: - Develop and maintain ETL/ELT pipelines using technologies like Apache Spark, Airflow, and dbt. - Utilize APIs and data services for real-time and batch processing. - Ensure data governance, security, and co...

As a Data Architect specializing in Databricks and Snowflake, your role involves designing and optimizing cloud-native data architectures to support scalable data engineering, advanced analytics, and AI/ML solutions in alignment with business requirements. Your key responsibilities include: - Designing and implementing Lakehouse architectures utilizing Databricks, Delta Lake, and Apache Spark. - Leading the development of data pipelines, ETL/ELT processes, and data integration strategies. - Collaborating with both business and technical teams to establish data architecture standards, governance, and security models. - Optimizing the performance and cost-efficiency of Databricks clusters and ...

Role Overview: As a technical architect and hands-on lead for Data Engineering practices within the Smart Supply Chain initiative at Amgen, your primary responsibility is to design, build, maintain, analyze, and interpret data to provide actionable insights for driving business decisions. You will work with large datasets, develop reports, support data governance initiatives, and visualize data to ensure accessibility, reliability, and efficient management. Your role will also involve architecting, building, and optimizing enterprise-grade data pipelines using Databricks and AWS-native services. Key Responsibilities: - Design, develop, and maintain data solutions for data generation, collect...

As a Senior Data Engineer at our company, you will play a crucial role in leading the development of a scalable data ingestion framework and ensuring high data quality and validation. Your responsibilities will include: - Architecting, developing, and maintaining end-to-end data ingestion framework for extracting, transforming, and loading data from diverse sources - Utilizing AWS services such as Glue, Lambda, EMR, ECS, EC2, and Step Functions to build scalable, resilient automated data pipelines - Developing and implementing automated data quality checks, validation routines, and error-handling mechanisms - Establishing comprehensive monitoring, logging, and alerting systems for data quali...

In this role at PwC, you will be part of the data and analytics engineering team, focusing on utilizing advanced technologies to develop robust data solutions for clients. Your main responsibility will involve transforming raw data into actionable insights, facilitating informed decision-making and driving business growth. Specifically in data engineering at PwC, you will concentrate on designing and constructing data infrastructure and systems to enable efficient data processing and analysis. Your duties will also include developing and implementing data pipelines, data integration, and data transformation solutions. Key Responsibilities: - Interact with Offshore Manager/Onsite Business Ana...

As a software developer at our company, you will be responsible for the following: - Designing, coding, testing, documenting, and implementing application release projects as part of the development team. - Utilizing your experience in programming/debugging for business applications. - Demonstrating working knowledge of industry practices and standards. - Proactively notifying the Development Project Manager of risks, bottlenecks, problems, issues, and concerns. - Ensuring compliance with Citi's System Development Lifecycle and Information Security requirements. - Collaborating closely with Quality Control teams to deliver quality software within agreed project schedules. - Working with Busi...

You will play a crucial role in leading the design efforts and formulating implementation strategies for AI solutions within the CPG integrated Marketing department. Your responsibilities will involve: - Leading the development and deployment of AI-enhanced BI solutions across various cloud platforms. - Building and optimizing data pipelines, models, and dashboards to derive real-time insights. - Collaborating with architects and stakeholders to ensure alignment with business objectives. - Implementing MLOps practices to facilitate continuous integration and delivery of ML models. - Ensuring adherence to data governance, privacy, and security standards. - Mentoring junior developers to foste...

Role & responsibilities Design, build, and optimize end-to-end data pipelines and ETL/ELT workflows using Databricks, Spark, and DBT . Architect and implement data lakehouse and big data solutions across cloud platforms. Develop and deploy AI/ML models with strong focus on MLOps, automation, and CI/CD pipelines . Implement Infrastructure as Code (IaC) using Terraform, ARM Templates, or CloudFormation for data workloads. Work with Fabric, Palantir, Synapse, Snowflake, and Hadoop ecosystems for large-scale data integration. Collaborate with cross-functional teams to enable data governance, lineage, and data quality frameworks . Manage and optimize cloud-based data solutions on Azure, AWS, and ...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Apache Spark, Delta Lake, ETL processes. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time. Develop complex SQL queries to optimize database performance and troubleshoot issues. Ensure scalability, reliability, and security of all systems by implementing best practices in coding standards. Participate in code reviews to improve overall quality of the system. Desired Candidate Profile 4-7 years of experience as a Databricks Engineer or similar role. Strong expertise in Apache Spark (Pyspark), Python programming language. Proficiency in...

Looking for Software Engineer with 3–6 yrs exp in Scala/Python, Apache Spark, SQL, and data pipelines. Must have strong backend development, data processing, performance tuning skills. Experience with Tableau/Zeppelin is a plus.University VTU Only

Software Engineer (3–6 yrs), Scala Programming, Apache Spark, SQL, Backend, Functional Programming, Performance Tuning. Exp in Tableau/Zeppelin is preferred. C2H via TE Infotech (Nutanix), Convertible 2 Permanent, BLR. Apply: ssankala@toppersedge.com

You will work as a senior member of a team responsible for developing large-scale, highly available, and fault-tolerant next-generation SaaS solutions that are purpose-built for Health Care Analytics in accordance with established processes. **Key Responsibilities:** - Design and implement scalable, secure, and maintainable enterprise-level solutions. - Establish coding standards, best practices, and architectural guidelines. - Integrate business process management solutions with existing systems. - Ensure system architectures align with business requirements and technology roadmap. - Lead and mentor development teams in technical implementation. - Conduct code reviews and provide constructi...

As a Data Engineer Offshore at PwC, you will be part of the managed services team focusing on outsourced solutions and supporting clients across various functions. Your role will involve streamlining operations, reducing costs, and improving efficiency for organizations by managing key processes and functions on their behalf. Utilizing your skills in project management, technology, and process optimization, you will deliver high-quality services to clients. Your main responsibilities will include transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. Continuous improvement and optimization of managed services processes, tools, and ...

Should have experience in building scalable enterprise data integration solutions preferred on the Microsoft SQL Server Azure Data Warehouse Database platforms

Should have experience in building scalable enterprise data integration solutions preferred on the Microsoft SQL Server Azure Data Warehouse Database platforms

Should have experience in building scalable enterprise data integration solutions preferred on the Microsoft SQL Server Azure Data Warehouse Database platforms