49893 Spark Jobs - Page 4

Role Overview: You will be responsible for building scalable data pipelines (ETL/ELT) for diverse data sources and cleaning, validating, and transforming data for business consumption. Additionally, you will develop data model objects (views, tables) for downstream usage, optimize storage and query performance in AWS environments, collaborate with business & technical stakeholders, and support code reviews, documentation, and incident resolution. Key Responsibilities: - Build scalable data pipelines (ETL/ELT) for diverse data sources - Clean, validate, and transform data for business consumption - Develop data model objects (views, tables) for downstream usage - Optimize storage and query pe...

As a Senior Technical Architect at our company, you will be responsible for leading the architecture, design, and implementation of scalable data platforms and AI-powered applications for our global clients. Your expertise in Databricks, Apache Spark, and modern data engineering architectures will be pivotal in guiding technical teams and collaborating with stakeholders to address complex business challenges. Your role will involve mentoring data engineers and solution architects, ensuring best practices in security, governance, and performance, and staying updated with emerging trends in data engineering and AI systems. **Key Responsibilities:** - Lead architecture, design, and deployment o...

As a Senior Lead Software Engineer - Java, Cloud, Big Data, you will be an integral part of an agile team that focuses on enhancing, building, and delivering trusted market-leading technology products in a secure, stable, and scalable manner. Your deep technical expertise and problem-solving skills will be crucial in addressing a wide range of challenges across various technologies and applications. Key Responsibilities: - Provide technical guidance and direction to support the business and technical teams, contractors, and vendors. - Develop secure and high-quality production code, review and debug code written by others. - Drive decisions that impact product design, application functionali...

Role Overview: As a Senior Data Scientist specializing in AI/ML with a PhD, you will be an integral part of our Hyderabad-based team. Your primary responsibility will involve tackling complex challenges in artificial intelligence, developing advanced models, and creating impactful algorithms that enhance our products and services. Key Responsibilities: - Utilize your expertise in AI/ML to solve cutting-edge problems and contribute to the development of innovative solutions. - Build and implement algorithms, leveraging your strong knowledge of statistical modeling and deep learning frameworks such as TensorFlow, PyTorch, and Scikit-learn. - Demonstrate proficiency in programming languages lik...

Role Overview: As a member of the Data Science and Machine Learning (AI/ML) team at Agoda, you will have the opportunity to work on challenging machine learning and big data platforms. You will be responsible for designing, coding, and implementing models and algorithms to enhance customer experience, business outcomes, and infrastructure readiness. Your role will involve mining and analyzing a large amount of data to drive improvements and innovation. Collaboration with developers and various business owners will be essential for delivering high-quality results on a daily basis. Additionally, you will have the chance to research and explore new ideas that can make a difference in the field....

Role Overview: As an enthusiastic AI/ML Developer with 3-5 years of experience, you will be responsible for designing, developing, and deploying AI/ML solutions. Your passion for AI, expertise in machine learning, deep learning, and MLOps, along with your eagerness to work on cutting-edge projects will be key to your success in this role. Key Responsibilities: - Utilize programming skills in Python (TensorFlow, PyTorch, Scikit-learn, Pandas) to implement AI/ML solutions. - Apply knowledge of various machine learning techniques including Supervised, Unsupervised, Deep Learning, NLP, and Computer Vision. - Deploy models using technologies such as Flask, FastAPI, AWS SageMaker, Google Vertex AI...

As a Team Lead at Carelon Global Solutions India, you will be responsible for all design requirements for leader communications. You will report to the Manager of Communications and must have a good understanding of the design process, liaising with multiple stakeholders to comprehend the brief and deliver all in-house design requirements. Additionally, you will coordinate with external agencies to ensure brand guidelines are adhered to. **Key Responsibilities:** - Good experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR, Glue, Lambda, Athena, and Redshift. - Proficient in Bigdata technologies including Spark, Scala, Hive, and Hadoop. - Experi...

Role Overview: Agoda, an online travel booking platform, is looking for ambitious and agile data scientists to join their Data Science and Machine Learning (AI/ML) team based in Bangkok. As part of the team, you will have the opportunity to work on challenging machine learning and big data platforms, processing a massive amount of events daily and making predictions. Key Responsibilities: - Design, code, experiment, and implement models and algorithms to enhance customer experience, supply side value, business outcomes, and infrastructure readiness - Mine vast amounts of customer data and daily user-generated events to discover actionable insights for driving improvements and innovation - Co...

• Proficiency with Python (Pandas, NumPy), SQL, and Java. • Experience with LLMs, Lang Chain, and Generative AI technologies. • Familiarity with ML frameworks (TensorFlow, PyTorch) and data engineering tools (Spark, Kafka). • Microservices, CI CD, ML Required Candidate profile • Machine Learning, NLP, Recommender system, personalization, Segmentation, microservice architecture & API development • Data engineering concepts like data lakes, columnar formats, ETL & BI tools

Role Overview: As a Senior Solution Architect - Gen AI at CitiusTech, you will be a key member of an Agile team responsible for designing and building healthcare applications, implementing new features, and ensuring adherence to the best coding development standards. Your role will involve working closely with the development team to design, implement, and execute comprehensive test plans, including automation testing and API testing. Your expertise in software quality assurance and ophthalmology will be crucial in maintaining the quality, reliability, and performance of the application. Key Responsibilities: - Articulate, advise, and lead workshops with customers, demonstrating proficiency ...

As a highly skilled engineer with experience in fintech, you have the opportunity to join Agoda, an online travel booking platform that is part of Booking Holdings. In this role, you will be responsible for various tasks related to platforms and data pipelines in the fintech domain. Your key responsibilities will include: - Thinking and owning the full life cycle of products, from business requirements to proper monitoring - Designing, developing, and maintaining platforms and data pipelines across fintech - Improving scalability, stability, and efficiency of existing systems - Writing great code, mentoring team members, and collaborating with other teams - Assisting in hiring extraordinary ...

As a Data Engineering Lead at Metyis, you will play a crucial role in driving impactful change for our clients by independently leading and managing data engineering projects. Your responsibilities will include engineering complete technical solutions to address business challenges in digital marketing, eCommerce, Business Intelligence, and self-service analytics. You will collect functional and non-functional requirements, taking into account technical environments, business constraints, and enterprise organizations. Additionally, you will support clients in executing their Big Data strategies by designing and building operational data platforms, such as ETL pipelines, data anonymization pi...

As a Healthcare Data Architect, your role will involve designing and managing robust healthcare data systems that meet regulatory requirements, enable interoperability, and support advanced analytics. You will leverage your expertise in healthcare IT, data modelling, and cloud-based solutions to architect scalable, secure systems that serve EHRs, population health, claims, and real-time analytics use cases. **Key Responsibilities:** - Architect and implement scalable healthcare data platforms - Ensure compliance with HIPAA, GDPR, and other healthcare regulations - Design data models for EHR, claims, and clinical data - Optimize ETL pipelines and manage data flow integrity - Lead data warehou...

Role Overview: As a Data Architect with AEP Competency at Adobe, you will be responsible for working with experienced data modelers, SQL, ETL, and development background to define new data schemas and data ingestion for Adobe Experience Platform customers. You will have the opportunity to interface directly with enterprise customers, gather requirements, design solutions, and collaborate with internal teams to deliver exceptional technical specifications. Key Responsibilities: - Interface with Adobe customers to gather requirements, design solutions & make recommendations - Lead customer project conference calls or interface with a Project Manager - Deliver Technical Specifications documents...

As a seasoned Azure cloud data engineer at UBS, your role will involve developing curated data pipelines to load, transform, and egress data on the cloud. You will also be responsible for helping to build state-of-the-art solutions supporting M&S surveillance models and alert generation using technologies like Scala, Spark, Databricks, Azure Data Lake, and Airflow. Your key responsibilities include: - Developing curated data pipelines for data loading, transformation, and egress on the cloud. - Building solutions for M&S surveillance models and alert generation using Scala, Spark, Databricks, Azure Data Lake, and Airflow. - Demonstrating superior analytical and problem-solving skills. - Appl...

Role Overview: As a part of our team, you will be responsible for the full end-to-end testing of the Merchant Acquirer lifecycle, starting from merchant onboarding to transaction processing, billing & funding, and finally invoice generation and reporting. You will be based in our vibrant office location at the Indore hub, contributing to the dynamic and agile environment of our APAC operations. Key Responsibilities: - Demonstrate a proven track record of end-to-end systems integration testing across multiple team and application boundaries, including 3rd parties and vendors. - Utilize your expertise as a seasoned Core Java developer with exposure to multi-threading, garbage collection, desig...

As a Data Scientist at Agoda, you will have the opportunity to work on challenging machine learning and big data platforms processing 600B events daily and making 5B predictions. You will be part of the Data Science and Machine Learning team, where you will tackle real-world challenges such as dynamic pricing, customer intent prediction, search result ranking, content classification, personalized recommendations, and more. You will work with a large ML infrastructure with GPUs, CPU cores, and memory to innovate the user experience. **Key Responsibilities:** - Design, code, experiment, and implement models and algorithms to enhance customer experience and business outcomes - Mine big data to ...

As a Data Engineer at McDonald's in Hyderabad, you will play a crucial role in building and maintaining reliable Menu data products to support menu and marketing Analytics. Your responsibilities will include: - Developing and implementing new technology solutions to enhance data reliability and observability. - Leading data engineering initiatives supporting Product Mix Analytics for timely and accurate delivery of marketing and menu-related products. - Collaborating with the Product owner to define business rules that ensure the quality of Menu datasets. - Driving best practices for pipeline development, data governance, data security, and quality across marketing and menu-related datasets....

Role : Senior Cloud Data Engineer Location : Chennai Experience: 6 to 8 Years What awaits you/ Job Profile You will design, develop, and optimize large-scale data pipelines. You will take ownership of critical components of the data architecture, ensuring performance, security, and compliance. Design and implement scalable data pipelines for batch and real-time processing. Optimize data storage and computing resources to improve cost and performance. Ensure data security and compliance with industry regulations. Collaborate with data scientists, analysts, and application teams to align data storage strategies. Lead technical discussions with stakeholders to deliver the best possible solution...

Responsibilities A day in the life of an InfoscionAs part of the Infosys delivery team, your primary role would be to provide best fit architectural solutions for one or more projects. You would also provide technology consultation and assist in defining scope and sizing of work You would implement solutions, create technology differentiation and leverage partner technologies. Additionally, you would participate in competency development with the objective of ensuring the best-fit and high quality technical solutions. You would be a key contributor in creating thought leadership within the area of technology specialization and in compliance with guidelines, policies and norms of Infosys.If y...

Job description Hiring for Spark Developer Mandatory Skills: Spark, Python, SQL Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and detailed functional...

Job description Hiring for Azure Databrick Engineer Mandatory Skills: Databrick, Data lake, Azure Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and d...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data processing pipelines using Apache Spark. Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions. Troubleshoot complex issues related to HDFS, Hive, Pyspark, and other big data technologies. Ensure scalability, performance, and reliability of the system by implementing efficient algorithms and data structures. Participate in code reviews and contribute to the development of best practices for software engineering. Desired Candidate Profile 5-10 years of experience in working with Apache Spark (Scala/Python) on Hadoop ecosystem (HDFS). Strong understanding of Big D...

Job Summary We are seeking a skilled and innovative Data Processing Engineer to join our team. As a Data Processing Engineer, you will be responsible for the design, development and validation of software for Data Processing Engineering across both cloud and on-premises environments.. You will be responsible for contributing to the design, and development and testing of code. The software applications you build will be used by our internal product teams partners and customers. We are looking for a hands-on lead engineer who is familiar with Python, Spark, Scala and Java. Any cloud experience is an advantage. You should be passionate about learning, be creative and have the ability to work wi...

Roles and Responsibility Design, develop, and deploy scalable applications using Scala and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data processing pipelines using AWS services. Troubleshoot and resolve complex technical issues efficiently. Participate in code reviews and contribute to improving overall code quality. Stay updated with industry trends and emerging technologies to enhance skills and knowledge. Job Requirements Strong proficiency in Scala, Spark, and AWS. Experience with big data processing and analytics. Excellent problem-solving skills and attention to detail. Ability to work collaborative...