49893 Spark Jobs - Page 3

As an experienced Data Engineer with over 10+ years of experience, you will be responsible for designing and implementing scalable, reliable, and efficient data pipelines and architectures across Google products and services. Your key responsibilities will include: - Developing and maintaining data models, schemas, and ontologies to support a variety of data sources and use cases. - Evaluating and recommending new data technologies and tools to improve infrastructure and capabilities. - Collaborating with product managers, engineers, and researchers to define data requirements and deliver robust technical solutions. - Building and optimizing batch and real-time data pipelines using Google Cl...

Role Overview: You are a seasoned Data/Information Management Senior Analyst with in-depth disciplinary knowledge and expertise. You will contribute to the development of new techniques, improve processes, and workflows within the area or function. Your role involves integrating subject matter expertise and industry knowledge within a defined area. You must have a deep understanding of how different areas integrate within the sub-function, coordinate effectively, and contribute to the function's objectives and overall business goals. Your analytical skills will be crucial in filtering, prioritizing, and validating complex material from multiple sources. Excellent communication, diplomacy, an...

As a Data Engineer, you will be responsible for developing, maintaining, and optimizing robust data ingestion and ETL/ELT pipelines using Python on the Databricks platform. Your key responsibilities will include: - Working collaboratively with the architecture team to align implementation with long-term data strategies. - Leveraging Delta Lake for data storage optimization and performance tuning. - Utilizing Prometheus for monitoring the health and performance of data pipelines and services. - Ensuring quality, reliability, and scalability of the data platform across multiple domains and use cases. - Driving automation and reusability in data processing pipelines to reduce manual overhead. -...

As an Associate Data Scientist at Data Axle Inc., you will be joining the Data Science Client Services team to contribute to the identification of high-quality target audiences that drive profitable marketing returns for clients. Your role will involve designing, building, and deploying impactful predictive marketing solutions across various verticals and clients. You will have the opportunity to work with a talented team on a range of projects. Your responsibilities will include: - Ownership of designing, implementing, and deploying machine learning algorithms within a modern Python-based cloud architecture. - Designing or enhancing ML workflows for data ingestion, model design, inference, ...

Role Overview: You will be a key member of the team responsible for developing scalable, data-driven systems in the digital marketing domain. Working in close collaboration with cross-functional teams, you will contribute towards solving large-scale Machine Learning challenges and optimizing models for enhanced performance and scalability. Your role will involve deploying deep learning models in high-performance computing environments and integrating ML frameworks into cloud and distributed systems. Key Responsibilities: - Collaborate with Data Science, Ops, and Engineering teams to address large-scale ML challenges. - Develop and manage robust ML pipelines for ETL, training, and deployment ...

As a Backend Software Engineer at Wayfair, you will be joining the Fulfillment Optimization team responsible for enhancing and scaling customer-facing platforms to optimize Wayfair profitability and customer satisfaction. Your role will involve maintaining an accurate representation of the dynamic supply chain, predicting how products fit into boxes, and surfacing information for customers and suppliers in real-time. Your responsibilities will include: - Building and launching data pipelines and products focused on SMART Org. - Collaborating with teams to create new product features and machine learning models using data insights. - Establishing cross-functional relationships to understand d...

You will be joining Bridgenext, a global digital consultancy that focuses on helping clients innovate and achieve their digital goals by developing digital products, experiences, and solutions tailored to real people's needs. **Key Responsibilities:** - Work on internal and customer-based projects for Bridgenext - Ensure high quality code and provide optimal solutions to meet client needs - Anticipate future client needs based on market understanding **Qualifications Required:** - 3-4 years of overall experience - Strong programming skills in Python/Java - 3+ years of experience with distributed systems and frameworks, particularly Spark - Proficiency in batch data processing; experience wit...

As a Software Engineer III at JPMorgan Chase within the Corporate Technology, you will be a seasoned member of an agile team responsible for designing and delivering trusted market-leading technology products in a secure, stable, and scalable manner. Your role involves executing software solutions, design, development, and technical troubleshooting with the ability to think creatively to build solutions and address technical challenges. You will create secure and high-quality production code, maintain algorithms, produce architecture and design artifacts for complex applications, and ensure design constraints are met by software code development. Additionally, you will analyze and synthesize...

As a Machine Learning Engineer, you will work closely with cross-functional teams to define ML problems and objectives. You will be responsible for researching, designing, and implementing various machine learning algorithms and models such as supervised, unsupervised, deep learning, and reinforcement learning. Your tasks will also include analyzing and preprocessing large-scale datasets for training and evaluation, as well as training, testing, and optimizing ML models to ensure accuracy, scalability, and performance. Deploying ML models in production using cloud platforms and/or MLOps best practices will be a key part of your role. You will monitor and evaluate model performance over time ...

As a Staff Software Engineer in Visa's Visa Risk Manager team based in Austin, your role will primarily focus on creating new capabilities for the Risk Platform. You will be responsible for troubleshooting, problem-solving, designing, building, and maintaining solutions for clients. Your deep knowledge of computer science fundamentals, mobile & web technologies, multi-tiered architecture, and experience in tools development will be essential for ensuring client success. Additionally, your ability to think logically, apply programming techniques, and communicate effectively with external teams & stakeholders will be crucial. You will also be expected to mentor and coach junior software engine...

As a Senior Technology Engineer specializing in Big Data and DevOps, your role involves a variety of key responsibilities: - DevOps and CI/CD: You will be responsible for designing, implementing, and managing CI/CD pipelines using tools like Jenkins and GitOps to automate and streamline the software development lifecycle. - Containerization and Orchestration: Your tasks will include deploying and managing containerized applications using Kubernetes and OpenShift to ensure high availability and scalability. - Infrastructure Management: You will develop and maintain infrastructure as code (IaC) using tools such as Terraform or Ansible. - Big Data Solutions: Architecting and implementing big da...

Role Overview: As a Machine Learning Engineer at Worldline, you will be part of a diverse and inclusive environment dedicated to fostering innovation. Your role involves developing machine learning models, designing new models tailored to specific business cases, deploying scalable models leveraging big data technologies, optimizing ML infrastructure, and staying updated with industry trends and best practices in ML and big data. Key Responsibilities: - Develop machine learning models using classical algorithms and Gen AI technologies. - Design and implement new machine learning models for various business cases. - Deploy scalable machine learning models utilizing big data technologies. - Op...

You are the perfect fit for the role of Back End Developer - Python at NTT DATA if you are an exceptional, innovative, and passionate individual who is eager to grow with a forward-thinking organization. As a Back End Developer - Python, your role will involve the following: - Key Responsibilities: - Utilize your skills in Python, kubernetes, Docker, DevOps, pandas, spark, ansible, Aks, log analytics, azure cloud, and CICD pipeline to develop efficient back end solutions. - Work collaboratively with the team in Bangalore, Karnataka (IN-KA), India (IN) to ensure seamless integration of back end systems. - Qualifications Required: - Proficiency in Python and related technologies. - Experience ...

As a GCP Data Architecture at TekWissen, your role involves designing and deploying Data Centric Architecture in GCP for the Materials Management platform. This platform facilitates the exchange of data between various applications in Product Development, Manufacturing, Finance, Purchasing, N-Tier Supply Chain, and Supplier Collaboration as part of the MMP transformation initiative. **Key Responsibilities:** - Design and deploy Data Centric Architecture in GCP for the Materials Management platform. - Enable seamless data exchange between modern and legacy applications in different areas of the client's operations. - Contribute to the larger Industrial Systems IT Transformation effort. **Qual...

As a highly experienced Data Tech Lead with over 10 years of expertise in data engineering, data architecture, and leadership, you will be responsible for leading and mentoring a team of data engineers in designing and implementing scalable, high-performance data solutions. Your role will involve architecting and developing robust data pipelines (ETL/ELT) and workflows using Microsoft Fabric and other data engineering tools. You will oversee data modelling, data warehousing, and data integration strategies to support analytics and business intelligence. It will be essential for you to optimize SQL-based solutions for performance, scalability, and security. Additionally, you will define and e...

Role Overview: As a Machine Learning Engineer at our company in Coimbatore, Tamil Nadu, India, your main objectives will include developing, validating, and implementing machine learning models and algorithms. You will be responsible for analyzing large and complex datasets to derive actionable insights and collaborating with cross-functional teams to integrate models into our products and services. It will be crucial for you to stay updated with the latest techniques in statistical modeling and machine learning. Key Responsibilities: - Develop, validate, and implement machine learning models and algorithms. - Analyze large and complex datasets to derive actionable insights. - Collaborate wi...

As a Machine Learning Engineer at our company, your role will involve designing, developing, deploying, and maintaining scalable ML solutions in a cloud-native environment. You will work with data scientists, engineers, and product teams to translate business requirements into ML solutions. Key responsibilities include: - Designing and implementing machine learning models and pipelines using AWS SageMaker and related services. - Developing and maintaining robust data pipelines for training and inference workflows. - Collaborating with teams to implement MLOps best practices such as CI/CD for ML, model versioning, monitoring, and retraining strategies. - Optimizing model performance and ensur...

As a Lead Technical Architect specializing in Strategy & Optimization for Data Lake & Analytics, your role involves managing project delivery, scope, timelines, budget, resource allocation, and risk mitigation. You will be responsible for developing and maintaining robust data ingestion pipelines, providing architectural inputs during incident escalations, defining and enforcing data governance standards, and optimizing data formats for cost-efficiency. Additionally, you will lead and mentor a team of data engineers, guide schema designs, and coordinate cross-functional stakeholders for aligned execution. Key Responsibilities: - Manage Project Delivery, scope, timelines, budget, resource all...

Job Description: As a Spark Expert, you will be required to demonstrate expert proficiency in Spark and possess the ability to design and implement efficient data processing workflows. Your experience with Spark SQL and DataFrames, along with a good exposure to Big Data architectures and understanding of the Big Data ecosystem, will be crucial. Additionally, your expertise in framework building on Hadoop, strong DB knowledge with SQL tuning experience, and familiarity with Python APIs and exposure to Kafka will be valuable assets. Key Responsibilities: - Demonstrate expert proficiency in Spark - Design and implement efficient data processing workflows - Utilize Spark SQL and DataFrames effec...

Role Overview: As a Cloud Architect with over 18 years of experience, you will be responsible for leading and managing a portfolio of Data & AI projects and programs across multiple accounts. Your primary focus will be on developing and implementing data strategies that align with business objectives, overseeing the design, development, and deployment of advanced data analytics solutions, and providing strategic guidance on data and analytics best practices. Key Responsibilities: - Experience with any cloud platforms such as AWS, Azure, Google Cloud, and Big Data technologies like Hadoop and Spark. - Implementation experience in various Data Management projects including Data platform Modern...

Role Overview: As a Data Engineering Lead at Ascentt, you will be responsible for overseeing and guiding the organization's data engineering function. You will combine your deep technical expertise in big data and data engineering with strong leadership skills to drive innovation, best practices, and excellence across the data engineering teams. Your role will involve establishing and evolving the strategic direction for the data engineering practice, defining and implementing best practices, leading the design and architecture of scalable data solutions, overseeing multiple data engineering teams and projects, mentoring and developing talent, and collaborating with other practice leads and ...

You will be responsible for designing and implementing scalable, reliable, and efficient data pipelines and architectures across Google products and services. Your role will involve developing and maintaining data models, schemas, and ontologies to support various data sources and use cases. You will evaluate and recommend new data technologies and tools to enhance infrastructure and capabilities. Collaboration with product managers, engineers, and researchers to define data requirements and deliver technical solutions will be a key aspect of your responsibilities. Building and optimizing batch and real-time data pipelines using Google Cloud Platform (GCP) services like Dataflow, Dataproc, P...

As a Data & Applied Scientist at Office Product Group (OPG) within Microsoft, you will play a crucial role in leveraging big data to drive engineering and product decisions for one of the most popular productivity software products globally. You will be part of a team dedicated to delighting customers through data-informed engineering practices. If you have a passion for delivering business value through data insights and analytics, this is the perfect opportunity for you. **Responsibilities:** - **Dashboard Development and Maintenance:** - Design, build, and maintain interactive dashboards and reports in PowerBI to visualize key business metrics and insights. - Collaborate closely with stak...

As a Software Engineer III at JPMorgan Chase within the Corporate Technology, you will be part of an agile team responsible for designing and delivering trusted market-leading technology products in a secure, stable, and scalable manner. Your role involves executing software solutions, technical troubleshooting, and creating secure production code that aligns with design constraints. Additionally, you will contribute to the continuous improvement of software applications and systems through data analysis and insights. **Key Responsibilities:** - Execute software solutions and troubleshoot technical issues with innovative problem-solving approaches - Develop and maintain high-quality producti...

As an AI/ML Model Engineer, you will play a crucial role in designing, developing, deploying, and maintaining scalable machine learning models and AI-powered applications. Your responsibilities will involve collaborating with data scientists, data engineers, and product teams to create impactful AI/ML solutions that address complex business problems. Key Responsibilities: - Design, develop, and implement various machine learning models (e.g., supervised, unsupervised, reinforcement learning, deep learning) to tackle challenging business issues. - Conduct extensive experimentation to evaluate model performance, refining them iteratively for optimal accuracy and efficiency. - Keep abreast of t...