530 Apache Spark Jobs - Page 7

As a Software Engineer - Distributed Computing & Database Internals Specialist at Zettabolt in Noida, you will be responsible for engineering high-performance databases and distributed systems that drive large-scale applications. Your work will involve delving into the intricacies of execution engines, query optimizations, and database internals, while also considering the interactions of these components at scale. In this role, you will have the opportunity to work on both a worm's-eye view, where you will optimize query execution, indexing, storage engines, and concurrency mechanisms at the source code level of distributed computing frameworks like Apache Spark, and a bird's-eye view, wher...

As a Data Architect with over 6 years of experience, you will be responsible for designing and implementing modern data lakehouse architectures on cloud platforms such as AWS, Azure, or GCP. Your primary focus will be on defining data modeling, schema evolution, partitioning, and governance strategies to ensure high-performance and secure data access. In this role, you will own the technical roadmap for scalable data platform solutions, ensuring they are aligned with enterprise needs and future growth. You will also provide architectural guidance and conduct code/design reviews across data engineering teams to maintain high standards of quality. Your responsibilities will include building an...

The healthcare industry presents a significant opportunity for software development, and Health Catalyst stands out as a leading company in this sector. At Health Catalyst, we are dedicated to addressing national healthcare challenges and improving the lives of millions of individuals, including your loved ones. As a rapidly expanding organization, we value individuals who are intelligent, hardworking, and humble. Our product teams are focused on developing innovative tools to support our mission of enhancing healthcare performance, reducing costs, and improving quality. We are currently looking for a Principal Snowflake Data Engineer and Data Engineering Lead with a minimum of 8-10 years of...

You should have 5-8 years of experience in data analytics and be located in Bengaluru. As a Senior Associate Data Analytics Lead, your responsibilities will include implementing data pipelines, hands-on development experience in Spark and Data Bricks pipeline optimization, working with AWS services, ElasticSearch, SQL knowledge, REST API knowledge, and leading individual projects using AGILE methodologies. You will also lead a team of developers, collaborate with stakeholders, and work on defining the product roadmap. It is essential to be up to date in the data engineering space, deliver high-efficiency software systems, and design innovative solutions to complex problems. Your must-have sk...

As a Full Stack AI Developer at GlobalLogic, you will play a crucial role in developing cutting-edge AI solutions that drive innovation and impact for our clients. You will be responsible for leveraging your expertise in programming languages such as Python, Java, and Scala to build robust AI applications. Your proficiency in data processing libraries like Pandas, NumPy, and Scikit-learn will enable you to manipulate and analyze large datasets effectively. In this role, you must have hands-on experience in distributed computing platforms such as Apache Spark (PySpark, Scala) and Torch. Your ability to develop APIs using Fast API and Spring Boot, along with a solid understanding of O&M princi...

You are a PySpark Developer with 3-8 years of experience and have expertise in Apache Spark, Big Data Hadoop Ecosystem, SparkSQL, and Python. Your professional background includes working with Bigdata technologies like PySpark, HIVE, Hadoop, and PLSQL. You also possess a good understanding of AWS, Snowflake, CICD, and system design. It would be advantageous if you have prior experience in Fund transfer AML technologies. Strong communication skills, both written and verbal, are essential for this role. As a self-starter, you should have quick learning abilities and the capability to multitask effectively, even under stringent deadlines. You must be adept at understanding and working on variou...

The role of Scala Spark Developer requires 3-8 years of experience and can be based in Chennai or Pune with any notice period. As a Scala Spark Developer, you will need to have expertise in Apache Spark, Big Data Hadoop Ecosystem, SparkSQL, and Scala. You should possess a minimum of 5 years of Scala+Spark experience, demonstrating hands-on experience in designing and developing big data platforms. Your responsibilities will include working on data engineering projects, utilizing modern data processing technology stacks such as Spark, HBase, Hive, and other Hadoop ecosystem technologies. You will be expected to have a deep understanding of streaming data architectures for real-time and low-la...

We are looking for a Technical Manager (Java & Big Data) to join our team in Bangalore under a hybrid work model. As a seasoned Java professional, you will be responsible for leading talented engineering teams and spearheading key technical initiatives to develop innovative telecom solutions using modern Java and Big Data technologies. With over 8 years of software development experience, including at least 3 years in a people management role, you will lead and mentor a high-performing development team. Your responsibilities will include driving the design, development, and deployment of Java-based applications, overseeing Big Data projects, ensuring seamless integration with cloud platforms...

As a Data Engineer at Tata Electronics Private Limited (TEPL), you will play a crucial role in architecting and implementing scalable offline data pipelines for manufacturing systems, ensuring the smooth functioning of various components such as AMHS, MES, SCADA, PLCs, vision systems, and sensor data. Your expertise in designing and optimizing ETL/ELT workflows using tools like Python, Spark, SQL, and Airflow will be instrumental in transforming raw data into actionable insights. You will lead database design and performance tuning activities across SQL and NoSQL systems, optimizing schema design, queries, and indexing strategies to enhance the efficiency of manufacturing data processing. Yo...

About the Roe: Grade Leve (for interna use): 11 The Team: Our team is responsibe for the design, architecture, and deveopment of our cient facing appications using a variety of toos that are reguary updated as new technoogies emerge. You wi have the opportunity every day to work with peope from a wide variety of backgrounds and wi be abe to deveop a cose team dynamic with coworkers from around the gobe. The Impact: The work you do wi be used every singe day, its the essentia code you write that provides the data and anaytics required for crucia, daiy decisions in the capita and commodities markets. Whats in it for you: Buid a career with a goba company. Work on code that fues the goba financ...

We are seeking an experienced Java Technical Lead with 8-12 years of experience for an immediate joiner position in Noida. The role involves leading a technical team in the design, development, and implementation of scalable Java-based enterprise solutions using Agile methodologies. Responsibilities include defining architecture, capturing requirements, leveraging Spring Boot and Spring Cloud frameworks, performing code reviews, implementing PoCs, troubleshooting bottlenecks, and ensuring system scalability and performance. The candidate should possess strong expertise in distributed architecture, multithreading, RDBMS, and NoSQL databases, with added knowledge in Apache Spark, Data Science,...

The Applications Development Senior Programmer Analyst position is an intermediate level role that involves collaborating with the Technology team to establish and implement new or updated application systems and programs. Your main objective will be to contribute to applications systems analysis and programming activities. Responsibilities include conducting tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and model development. You will also be responsible for monitoring and controlling all phases of the development process, providing user and operational support, analyzing complex problems, and recommending security mea...

As a Software Engineer at Mahindra Teqo in Bangalore, you will be a part of the Software Department within the company. Mahindra Teqo, a tech-enabled Renewable Energy Asset Management offering from the Mahindra Group, is dedicated to providing a complete suite of products and services for streamlined asset management in the global renewable industry to maximize returns from Renewable Energy (RE) assets. Your role as a Data Engineer will involve working on enterprise products based on IoT and Big Data engineering. You will collaborate with Data Engineers, Integration Specialists, and UI developers to design, develop, and implement complex, scalable, and extensible functionalities. Additionall...

As a Software Development Engineer at our company, you will have the opportunity to work in either Hyderabad, Telangana, India or Bengaluru, Karnataka, India. You should hold a Bachelor's degree in Computer Science, a related technical field, or have equivalent practical experience. With at least 8 years of experience in software development using general-purpose programming languages, you will be responsible for designing and building solutions for data migrations and developing software for Data Warehouse migration. Your expertise in programming languages such as Java, C/C++, Python, or Go will be crucial for this role. Additionally, experience in systems engineering with designing and bui...

As a Data Scientist at our company, you will be responsible for collaborating with interdisciplinary scientists, IT, and engineering professionals to solve critical problems and support business decision-making. Your strong communication and organizational skills, along with a problem-solving attitude, will be essential in this role. To excel in this position, you should have at least 3 years of experience in analytics, particularly with big data solutions. Your expertise should encompass Statistics, Analytics, Data Science, Machine Learning, and Deep Learning. Proficiency in big data platforms such as SQL, Hadoop, Apache Spark, and programming languages like Python and R is crucial. You sho...

You are ArCent, a 17+ year young recruitment firm with a global reach, dedicated to providing top-level, dependable talent to your clients, which primarily include leading MNCs and CMM-level companies. Currently, you are looking to hire a Databricks Architect for one of your Indian clients, a 30-year-old digital engineering company with a global presence and a workforce of over 3,500 professionals. This organization specializes in IT services, software products, platforms, and infrastructure, with expertise in Big Data, AI/ML, and cloud solutions for large enterprises. Headquartered in the U.S., the company has major development centers across Noida, Gurugram, Pune, Indore, Bengaluru & Hyder...

You are a PySpark Developer with over 7 years of experience and expertise in Reltio MDM. As a part of the Data Engineering team, your primary responsibility will be to design, build, and optimize scalable data pipelines, ensuring seamless integration with Reltio's cloud-native MDM platform. This is an immediate requirement, and we are looking for candidates who can join us promptly. Your key responsibilities will include designing, developing, and maintaining scalable data pipelines using PySpark in distributed computing environments like AWS EMR and Databricks. You will also be responsible for integrating and synchronizing data between enterprise systems and the Reltio MDM platform, impleme...

We are looking for a highly skilled Scala Data Engineer to design, build, and optimize large-scale data platforms. The ideal candidate will have deep expertise in Scala, Spark, and SQL, with proven experience delivering scalable and high-performance data solutions in cloud-native environments. Key Responsibilities Design, build, and optimize scalable data pipelines using Apache Spark and Scala. Develop real-time streaming pipelines leveraging Kafka/Event Hubs. Own the design and architecture of data systems with a strong focus on performance, scalability, and reliability. Collaborate with cross-functional teams to deliver high-quality data products. Mentor junior engineers and enforce best p...

8+ years of relevant IT experience in the BI/DW domain with hands-on experience on the Azure modern data platform . Strong knowledge and experience with the Python programming language Experience using version control systems like Git, bitbucket . Experience in data analysis and transformation using PySpark on Azure Databricks or Apache Spark . Good knowledge of Distributed Processing using Databricks or Apache Spark. Experience in creating data structures optimized for storage and various query patterns. Experience of integrating Unity Data Catalog with the Databricks and registering data assets etc. Understanding of CI/CD pipelines. Knowledge of Agile/Scrum methodologies.

About this role: Wells Fargo is Seeking a Lead Data Engineer. In this role, you will: Lead complex initiatives with broad impact and act as key participant in large scale software planning for the Technology area Design, develop, and run tooling to discover problems in data and applications and report the issues to engineering and product leadership Review and analyze complex software enhancement initiatives for business, operational or technical improvements that require in depth evaluation of multiple factors including intangibles or unprecedented factors Make decisions in complex and multi-faceted data engineering situations requiring understanding of software package options and programm...

Hiring: Data Engineer SME Location: Hinjewadi, Pune (Hybrid) Contract: 12 Months Experience: 12 to 17 Years About the Role We are looking for an experienced Subject Matter Expert (SME) Cloud & Data Engineering to lead the design, implementation, and integration of cloud-based data solutions. The ideal candidate will have deep expertise in AWS, data engineering, and application integration , along with proven leadership capabilities. Key Responsibilities Lead the design and implementation of cloud-based solutions for large-scale data engineering projects. Orchestrate the integration of diverse applications and data sources across the cloud. Collaborate with cross-functional teams to ensure se...

Role Description: As a Senior Cloud Data Platform (AWS) Specialist at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining secu...

We are seeking a dynamic and experienced Tech Lead with a strong foundation in Java and Apache Spark to join our team. In this role, you will lead the development and deployment of scalable cloud-based data solutions, leveraging your expertise in AWS and big data technologies. Key Responsibilities: Lead the design, development, and deployment of scalable and reliable data processing solutions on AWS using Java and Spark. Architect and implement big data processing pipelines using Apache Spark on AWS EMR. Develop and deploy Serverless applications using AWS Lambda, integrating with other AWS services. Utilize Amazon EKS for container orchestration and microservices management. Design and impl...

Role & responsibilities Expected to be an SME. - Collaborate and manage the team to perform. - Responsible for team decisions. - Engage with multiple teams and contribute on key decisions. - Provide solutions to problems for their immediate team and across multiple teams. - Facilitate knowledge sharing sessions to enhance team capabilities. - Monitor and evaluate team performance to ensure alignment with project goals. Professional & Technical Skills: - Must To Have Skills: Proficiency in Scala. - Good To Have Skills: Experience with Java, Apache Spark, Microsoft Azure Data Services. - Strong understanding of data modeling and database design principles. - Experience with cloud-based data so...

Job Description: We are looking for a highly skilled Senior Data Engineer to join Catalog Management team. The ideal candidate will design, build, and maintain scalable data pipelines and infrastructure on Google Cloud Platform (GCP) , ensuring data availability and reliability for analytics and business needs. Key Responsibilities: Design and develop scalable data pipelines using Apache Spark (Java) , Apache Beam, or Kubeflow. Manage and configure GCP services : BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage. Develop and maintain RESTful APIs using Spring Boot. Implement test automation , CI/CD pipelines, and monitoring solutions. Collaborate with cross-functional teams to deliver opt...