50049 Spark Jobs - Page 19

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Required technical and professional expertise Experience with Apache Spark (PySpark)...

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporat...

Key Responsibilities Design, build, and maintain end-to-end GCP data pipelines (batch and streaming). Ensure data platform uptime and performance in alignment with defined SLAs/SLOs. Develop and optimize ETL/ELT workflows using Cloud Composer (Airflow), Dataflow (Apache Beam), and BigQuery. Manage and enhance data lakes and warehouses using BigQuery and Cloud Storage. Implement streaming data solutions using Pub/Sub, Dataflow, or Kafka. Build data APIs and microservices for data consumption using Cloud Run, Cloud Functions, or App Engine. Define and enforce data quality, governance, and lineage using Data Catalog and Cloud Data Quality tools. Collaborate with DevOps to build CI/CD pipelines,...

Role Description Role Description:As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also...

About Agoda At Agoda, we bridge the world through travel. Our story began in 2005, when two lifelong friends and entrepreneurs, driven by their passion for travel, launched Agoda to make it easier for everyone to explore the world. Today, we are part of Booking Holdings [NASDAQ: BKNG], with a diverse team of over 7,000 people from 90 countries, working together in offices around the globe. Every day, we connect people to destinations and experiences, with our great deals across our millions of hotels and holiday properties, flights, and experiences worldwide. No two days are the same at Agoda. Data and technology are at the heart of our culture, fueling our curiosity and innovation. If you’r...

Role & Responsibilities We are looking for a Senior MLOps Engineer with 8+ years of experience building and managing production-grade ML platforms and pipelines. The ideal candidate will have strong expertise across AWS, Airflow/MWAA, Apache Spark, Kubernetes (EKS), and automation of ML lifecycle workflows. You will work closely with data science, data engineering, and platform teams to operationalize and scale ML models in production. Key Responsibilities Design and manage cloud-native ML platforms supporting training, inference, and model lifecycle automation. Build ML/ETL pipelines using Apache Airflow / AWS MWAA and distributed data workflows using Apache Spark (EMR/Glue). Containerize a...

Transform and process source data into data products and enable egression to other platforms Develop efficient Data Lake frameworks using Scala, PySpark, or Python, with robust data quality and security controls Create reusable libraries and config driven pipelines for enterprise wide ingestion and integration Ensure compliance with data governance and security standards using tools like AWS DataZone, Lake Formation, AWS Glue and MWAA Communicate effectively with internal stakeholders and contribute to team knowledge sharing Experience with CI/CD tools such as GitHub, GitHub Actions, ArgoCD, and automated deployment pipelines Experience with observability tools like Cloudwatch, Prometheus, o...

We are looking for an experienced Healthcare SME CCDA & FHIR with 8-15 years of experience in healthcare data integration, clinical data repositories, and interoperability standards The ideal candidate will have strong expertise in CCDA parsing, FHIR resource mapping, validation, ingestion processes, and healthcare data pipelines You will work closely with data engineering, product, and clinical informatics teams to design scalable data ingestion pipelines, ensure data quality and compliance, and enable seamless exchange of clinical data across systems Hands-on skills in FHIR servers (HAPI, IRIS), SQL, Python, cloud platforms (AWS/GCP/Azure), and healthcare regulations (HIPAA, PHI) are essen...

Job Summary: - Drive telecom data science projects from conception through deployment while leading a multidisciplinary team. Deliver high-quality data-driven solutions focused on telecom network analytics, customer churn, real-time monitoring, and service optimization. Key Responsibilities: - - Oversee development, validation, and deployment of data science models solving telecom-specific challenges such as churn, QoS, SINR, and Video on Demand analytics. - Lead data engineering efforts to build scalable data pipelines from telecom wireline/wireless network sources and real-time data streams. - Manage team priorities, resources, and timelines ensuring alignment with telecom business objecti...

We are hiring a Senior DevSecOps / Security Engineer with 8+ years of experience securing AWS cloud, on-prem infrastructure, DevOps platforms, MLOps environments, CI/CD pipelines, container orchestration, and data/ML platforms. This role is responsible for creating and maintaining a unified security posture across all systems used by DevOps and MLOps teams — including AWS, Kubernetes, EMR, MWAA, Spark, Docker, GitOps, observability tools, and network infrastructure. Key Responsibilities Cloud Security (AWS)- Secure all AWS resources consumed by DevOps/MLOps/Data Science: EC2, EKS, ECS, EMR, MWAA, S3, RDS, Redshift, Lambda, CloudFront, Glue, Athena, Kinesis, Transit Gateway, VPC Peering. Impl...

Company Description At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it’s consumed. Together, we are proudly rooted in our deep legacy as we stand at the forefront of the media revolution. When you join Nielsen, you will join a dynamic team committed to excellence, perseverance, and the ambition to make an impact together. We champion you, because when you succeed, we do too. We enable your best to power our future. Job Description As a Software ...

Company Description Sandisk understands how people and businesses consume data and we relentlessly innovate to deliver solutions that enable today’s needs and tomorrow’s next big ideas. With a rich history of groundbreaking innovations in Flash and advanced memory technologies, our solutions have become the beating heart of the digital world we’re living in and that we have the power to shape. Sandisk meets people and businesses at the intersection of their aspirations and the moment, enabling them to keep moving and pushing possibility forward. We do this through the balance of our powerhouse manufacturing capabilities and our industry-leading portfolio of products that are recognized globa...

HOW MIGHT YOU DEFY IMAGINATION? At Amgen, every challenge is an opportunity. And every opportunity brings the potential to change lives. Our shared mission—to serve patients—has driven our status as one of the world’s preeminent biotechnology companies, reaching over 10 million patients worldwide. Leverage your high-level expertise to lead and forge partnerships that advance the biotechnical and biopharmaceutical science that can turn the tide of serious, life-interrupting illnesses. Live What You Will Do Let’s do this. Let’s change the world. In this vital role the Director, Data Science – Marketing Investment Optimization (MIO) within the Customer Data & Analytics team is accountable for c...

We are seeking an experienced Data Architect with expertise in Snowflake, dbt, Apache Airflow, and AWS to design, implement, and optimize scalable data solutions. The ideal candidate will play a critical role in defining data architecture, governance, and best practices while collaborating with cross-functional teams to drive data-driven decision-making. Key Responsibilities Data Architecture & Strategy: Design and implement scalable, high-performance cloud-based data architectures on AWS. Define data modelling standards for structured and semi-structured data in Snowflake. Establish data governance, security, and compliance best practices. Data Warehousing & ETL/ELT Pipelines: Develop, main...

Job Summary: - Drive telecom data science projects from conception through deployment while leading a multidisciplinary team. Deliver high-quality data-driven solutions focused on telecom network analytics, customer churn, real-time monitoring, and service optimization. Key Responsibilities: - - Oversee development, validation, and deployment of data science models solving telecom-specific challenges such as churn, QoS, SINR, and Video on Demand analytics. - Lead data engineering efforts to build scalable data pipelines from telecom wireline/wireless network sources and real-time data streams. - Manage team priorities, resources, and timelines ensuring alignment with telecom business objecti...

About the Team! DE team at HYD collaborates with other engineering/product/analyst teams across tech sites to collectively accomplish OKRs to take Uber forward. We constantly enrich our data layer to effectively deal with the next generation of products which are a result of Uber's big bold bets. We design and build data pipelines to schedule & orchestrate a variety of tasks such as extract, cleanse, transform, enrich & load data as per the business needs. We serve the data insights at depth to various teams at Uber-like business analysts, data analysts, data science and business stakeholders to take strategic decisions, train DS models, perform health checks etc in order to grow and support...

About the Role As an ML Engineer, you should be able to apply Machine Learning to Solve complex business problems and optimize customer experience, and work closely with stakeholders to formulate machine learning problems within the business context. Build end-end ML systems for designing, training, testing, optimization, deploying, and serving machine learning models. Write ML system design docs and production-level codes to train your ML models into working pipelines and serve models for large-scale online traffic. About the team The Data Intelligence Platform team is responsible for designing core foundational data sets critical to understanding customers' needs and helping business teams...

Job Title : Technical Architect & Lead Developer - AI & Azure Web CoE About the role: Own and lead headcount and technology implementation in their area in line with approved budget and headcount. The job holder is expected to understand the existing landscape while conceptualize the design and architecture of new technology solutions in GBO in alignment with both GBO & D&T partners, thus ushering in impactful saving across project bottom-line through improvements opportunities in areas of productivity, accuracy, timeliness / efficiency, simplification, standardization, waste elimination, waste reduction, compliance alignment through deterministic automation, stochastic auautomationanging ac...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Job Title: Manager – Data Engineer - Snowflake As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. Roles and Responsibilities: Snowflake Development: Buil...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY-Advisory - Data and Analytics – Senior – Data Engineering (Snowflake) As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. Roles and Responsibilities: S...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY-Advisory - Data and Analytics – Senior – Data Engineering (Snowflake) As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. Roles and Responsibilities: S...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Job Title: Manager – Data Engineer - Snowflake As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. Roles and Responsibilities: Snowflake Development: Buil...

The Principal Data Engineer will be a senior individual contributor responsible for designing, building, and optimizing advanced data solutions that power enterprise-wide analytics, AI/ML, and data-driven decision-making. Reporting directly to the Senior Director of Data Platforms, this role will serve as a technical expert and thought partner in scaling modern data architectures, enabling AI orchestration, and delivering robust, secure, and compliant data products. This position is highly hands-on, requiring expertise in data engineering, Databricks, Python, AI/ML platforms, and orchestration frameworks. The Principal Engineer will work across functional teams to design and implement high-p...

Job Purpose The Senior Data Engineer will be responsible for designing, building, and maintaining scalable and efficient data pipelines and architectures for the Enterprise Data Platform. This role will focus on enabling high-quality, reliable, and timely data access for analytics, reporting, and business decision-making. Working closely with business analysts, data scientists, and architects, the Senior Data Engineer will ensure data solutions meet business needs and adhere to best practices and governance standards Duties and Responsibilities Design and implement robust, scalable, and high-performance data pipelines and ETL/ELT processes. Develop, optimize, and maintain data architectures ...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Job Title: Manager – Data Engineer - Snowflake As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. Roles and Responsibilities: Snowflake Development: Buil...