530 Apache Spark Jobs - Page 9

You are an experienced Data Engineer proficient in Databricks, PySpark, and Python, seeking to join a healthcare data-focused team. Your responsibilities will include designing and maintaining scalable data pipelines and solutions for complex healthcare datasets, utilizing Azure Cloud and advanced distributed data processing frameworks. Strong client-facing communication skills are crucial as you will engage directly with US-based stakeholders. Your key responsibilities will involve designing, developing, and maintaining large-scale data processing systems using Databricks and PySpark. You will build and optimize robust, scalable data pipelines for data ingestion, cleaning, transformation, a...

You will need to have a total of 8 to 12 years of experience in the IT/Software/BFSI/Banking/Fintech industry. Your work arrangement will involve 5 days of working from the office in Noida. Your main responsibilities will include meeting with technology managers and the design team to understand the company's goals, leading a technical team, capturing requirements, designing technical solutions using the Spring Framework, examining and defining current architecture systems, designing scalable architecture systems for Java-based applications, reviewing code, troubleshooting design flaws, and ensuring the quality and performance of the technical solution. You should have advanced knowledge of ...

As a Spark and Scala Developer at Infosys, you will play a crucial role in facilitating digital transformation for our clients within a global delivery model. Your responsibilities will include conducting independent research on technologies, providing recommendations for suitable solutions, and contributing to technology-specific best practices and standards. It will be essential for you to effectively interact with key stakeholders and utilize your technical expertise across various stages of the Software Development Life Cycle. As part of our learning culture, teamwork and collaboration are highly encouraged, excellence is acknowledged, and diversity is both respected and valued. Required...

The Applications Development Intermediate Programmer Analyst position is a role at an intermediate level where you will be responsible for participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. Your responsibilities will include utilizing your knowledge of applications development procedures and concepts, as well as basic knowledge of other technical areas to identify and define necessary system enhancements. This will involve using script tools, analyzing/interpreting code, consulting with users, cl...

The Applications Development Senior Programmer Analyst position is an intermediate level role where you will be responsible for participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. You will be responsible for conducting tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establishing and implementing new or revised applications systems and programs to meet specific business needs or user areas. You will also m...

About the job: At Citi, we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with a laser focus on evolving the ways of doing things. As one of the world's most global banks, we're changing how the world does business. Shape your Career with Citi We're currently looking for a high-caliber professional to join our team as AVP- Data Engineer based in Pune, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy ...

As an experienced data analytics professional with 1 to 2 years of experience, you will be responsible for developing and implementing data analytics methodologies. Your role will require good interpersonal skills along with excellent communication abilities. Your technical skills must include proficiency in Python, machine learning, deep learning, data wrangling, and integration with Big Data tools such as Hadoop, Scoop, Impala, Hive, Pig, and Spark R. You should also have a solid understanding of statistics, data mining, algorithms, time series analysis, forecasting, SQL queries, and Tableau data visualization. Having a good grasp of technologies like Hadoop, HBase, Hive, Pig, Mapreduce, P...

As a Software Developer at our company, you will collaborate with client developers & off-shore developers to deliver project tasks efficiently. Your responsibilities will include analyzing system domain and client requirements, creating and maintaining a robust system to support business operations with a focus on usability, and ensuring the completion of necessary milestones within the constraints of planned projects. You should possess very strong knowledge in Object-Oriented Programming (OOP) and Core Java, along with hands-on experience in JavaEE and Springboot. Experience in technologies such as Spring, Spring Security, Hibernate, JPA, and Restful services is required. Additionally, fa...

As a Lead Data Engineer at Rearc, you will play a crucial role in establishing and maintaining technical excellence within our data engineering team. Your extensive experience in data architecture, ETL processes, and data modeling will be key in optimizing data workflows for efficiency, scalability, and reliability. Collaborating closely with cross-functional teams, you will design and implement robust data solutions that align with business objectives and adhere to best practices in data management. Building strong partnerships with technical teams and stakeholders is essential as you drive data-driven initiatives and ensure their successful implementation. With over 10 years of experience ...

You will be responsible for capturing user requirements and translating them into business and digitally enabled solutions across various industries. Your key responsibilities will include designing, developing, optimizing, and maintaining data pipelines that adhere to ETL principles and business goals. You will solve complex data problems to deliver insights that help the business achieve its objectives. Your tasks will involve sourcing structured and unstructured data from different touchpoints, formatting and organizing them into an analyzable format. Additionally, you will create data products for analytics team members to enhance productivity and utilize AI services like vision and tran...

You are seeking a skilled Azure Databricks Architect with 12 to 17 years of experience in Python, SQL. As an Azure Databricks Architect at our company, you will be responsible for data architecture, data engineering, and analytics. You should have at least 5 years of hands-on experience with Azure Databricks, Apache Spark, and Delta Lake. Your proficiency in Azure Data Lake, Azure Synapse, Azure Data Factory, and Azure SQL is essential. Expertise in Python, Scala, and SQL for data processing, along with a deep understanding of data modeling, ETL/ELT processes, and distributed computing, are key requirements. Experience with CI/CD pipelines and DevOps practices in data engineering is also exp...

As a Sr. Data Engineer at Lifesight in Bangalore, you will be responsible for building highly scalable, fault-tolerant distributed data processing systems that handle massive amounts of data ingested daily. You will work on processing petabyte-sized data warehouses and Elasticsearch clusters, optimizing data pipelines for quality and resilience, and refining diverse datasets into simplified models to encourage self-service. Your role will involve owning data mapping, business logic, transformations, and ensuring data quality through low-level systems debugging and performance optimization on large production clusters. Additionally, you will participate in architecture discussions, influence ...

The Applications Development Technology Lead Analyst position is a senior-level role that involves establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective will be to lead applications systems analysis and programming activities. As the Applications Development Technology Lead Analyst, your responsibilities will include partnering with multiple management teams to ensure appropriate integration of functions to meet goals, identifying and defining necessary system enhancements for deploying new products and process improvements, and resolving a variety of high-impact problems/projects through in-depth evaluatio...

As a member of the Dun & Bradstreet team, you will play a crucial role in unlocking the power of data through analytics to create a better tomorrow. Our global community of over 6,000 team members is dedicated to accelerating creativity, innovation, and growth in order to help clients turn uncertainty into confidence, risk into opportunity, and potential into prosperity. We welcome bold and diverse thinkers who are passionate about making a positive impact. You will be responsible for designing and developing data pipelines within our Big Data ecosystem using technologies such as Apache Spark and Apache Airflow. Your role will involve architecting, building, and deploying scalable and effici...

The Applications Development Intermediate Programmer Analyst position is an intermediate level role where you will be responsible for contributing to the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to assist in applications systems analysis and programming activities. You will utilize your knowledge of applications development procedures and concepts, along with basic knowledge of technical areas, to identify and define necessary system enhancements. This includes using script tools, analyzing code, and consulting with users, clients, and other technology groups to recommend programm...

You will be responsible for designing and building backend components of our MLOps platform in Python on AWS. This includes collaborating with geographically distributed cross-functional teams and participating in an on-call rotation with the rest of the team to handle production incidents. To be successful in this role, you should have at least 3+ years of professional backend development experience with Python. Additionally, you should have experience with web development frameworks such as Flask or FastAPI, as well as working with WSGI & ASGI web servers like Gunicorn and Uvicorn. Experience with concurrent programming designs such as AsyncIO, containers (Docker), AWS ECS or AWS EKS, unit...

As a Software Engineer - Backend (Python) with over 7 years of experience, you will be based in Hyderabad and play a crucial role in developing the backend components of the GenAI Platform. Your responsibilities will include designing and constructing backend features for the platform on AWS, collaborating with cross-functional teams spread across different locations, and participating in an on-call rotation for managing production incidents. To excel in this role, you must possess the following skills: - A minimum of 7 years of professional experience in backend web development using Python. - Proficiency in AI, RAG, DevOps, and Infrastructure as Code (IaC) tools like Terraform and Jenkins....

The ideal candidate should have 3-5 years of experience in implementing scalable and sustainable data engineering solutions using tools such as Databricks, Snowflake, Teradata, Apache Spark, and Python. Your responsibilities will include creating, maintaining, and optimizing data pipelines as workloads transition from development to production for specific use cases. You will be responsible for end-to-end development, including coding, testing, debugging, and deployment. Automation is key, and you will drive the use of modern tools and techniques to automate repetitive data preparation and integration tasks to enhance productivity. You will be mapping data between source systems, data wareho...

As a Data Software Engineer, you will be responsible for utilizing your 5-12 years of experience in Big Data & Data-related technologies to contribute to the success of projects in Chennai and Coimbatore in a Hybrid work mode. You should possess an expert level understanding of distributed computing principles and a strong knowledge of Apache Spark, with hands-on programming skills in Python. Your role will involve working with technologies such as Hadoop v2, Map Reduce, HDFS, Sqoop, Apache Storm, and Spark-Streaming to build stream-processing systems. You should have a good grasp of Big Data querying tools like Hive and Impala, as well as experience in integrating data from various sources ...

The Retail Specialized Data Scientist will play a pivotal role in utilizing advanced analytics, machine learning, and statistical modeling techniques to help the retail business make data-driven decisions. You will work closely with teams across marketing, product management, supply chain, and customer insights to drive business strategies and innovations. The ideal candidate should have experience in retail analytics and the ability to translate data into actionable insights. Key Responsibilities: - Leverage Retail Knowledge: Utilize your deep understanding of the retail industry (merchandising, customer behavior, product lifecycle) to design AI solutions that address critical retail busine...

This is a data engineer position where you will be responsible for designing, developing, implementing, and maintaining data flow channels and data processing systems to support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable, and secure manner in coordination with the Data & Analytics team. Your main objective will be to define optimal solutions for data collection, processing, and warehousing, particularly within the banking & finance domain. You must have expertise in Spark Java development for big data processing, Python, and Apache Spark. You will be involved in designing, coding, and testing data systems and integrating the...

The Tech Consultant - Data & Cloud role involves supporting a leading international client with expertise in data engineering, cloud platforms, and big data technologies. As a skilled professional, you will contribute to large-scale data initiatives, implement cloud-based solutions, and collaborate with stakeholders to drive data-driven innovation. You will design scalable data architectures, optimize ETL processes, and leverage cloud technologies to deliver impactful business solutions. Key Responsibilities Data Engineering & ETL: Develop and optimize data pipelines using Apache Spark, Airflow, Sqoop, and Databricks for seamless data transformation and integration. Cloud & Infrastructure Ma...

As a Big Data Engineer at KGIS, you will be an integral part of the team dedicated to building cutting-edge digital and analytics solutions for global enterprises. With a focus on designing, developing, and optimizing large-scale data processing systems, you will lead the way in creating scalable data pipelines, driving performance tuning, and spearheading cloud-native big data initiatives. Your responsibilities will include designing and developing robust Big Data solutions using Apache Spark, building both batch and real-time data pipelines utilizing technologies like Spark, Spark Streaming, Kafka, and RabbitMQ, implementing ETL processes for data ingestion and transformation, and optimizi...

You will be responsible for designing and developing data solutions using Elasticsearch/OpenSearch, integrating with various data sources and systems. Your role will involve architecting, implementing, and optimizing data solutions, along with applying your expertise in machine learning to develop models, algorithms, and pipelines for data analysis, prediction, and anomaly detection within Elasticsearch/OpenSearch environments. Additionally, you will design and implement data ingestion pipelines to collect, cleanse, and transform data from diverse sources, ensuring data quality and integrity. As part of your responsibilities, you will manage and administer Elasticsearch/OpenSearch clusters, ...

Capgemini Invent is the digital innovation, consulting, and transformation brand of the Capgemini Group. As a global business line, Capgemini Invent combines expertise in strategy, technology, data science, and creative design to assist CxOs in envisioning and constructing what's next for their businesses. In this role, you will be responsible for developing and maintaining scalable data pipelines using AWS services. Your tasks will include optimizing data storage and retrieval processes, ensuring data security and compliance with industry standards, and handling large volumes of data while maintaining accuracy, security, and accessibility. Additionally, you will be involved in developing da...