3201 Data Pipelines Jobs - Page 50

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform * Responsibilities: * Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS * Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform * Experience in developi...

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks The candidate will implement CI/CD best practices for DBT, manage automated deployments, troubleshoot pipeline issues, and collaborate cross-functionally ...

Your Impact * Run data pipelines in monthly cadence, owning the quality of the output * Clean data and maintain pipelines, and make changes to safeguard data quality and reduce execution risks * Work closely with data science team to develop new features and enable access for ad hoc research * Optimise running of slow or resource heavy pipelines, minimising unnecessary computation and reducing technical debt * Aid in transition from pandas pipelines to snowflake environment * Own and understand the datasets, their context and logic, and share knowledge with the data science team * Perform exploratory research on datasets, uncovering new signals for the team to incorporate in models * Develop...

As a Senior AI Engineer, you will be responsible for designing, building, and deploying AI/ML models for various use cases including recommendation engines, NLP, computer vision, forecasting, and predictive analytics. You will lead model experimentation, validation, and optimization efforts to ensure high performance and accuracy. Additionally, you will develop and maintain MLOps pipelines for model versioning, deployment, and monitoring. Collaboration with data scientists, data engineers, and software developers to integrate models into production systems is a key aspect of your role. It is essential to stay updated with the latest AI research and industry trends and translate them into act...

As a Hardware Engineer at Qualcomm India Private Limited, you will play a crucial role in planning, designing, optimizing, verifying, and testing electronic systems. Your responsibilities will include working on various systems such as circuits, mechanical systems, Digital/Analog/RF/optical systems, equipment and packaging, test systems, FPGA, and/or DSP systems. You will be instrumental in launching cutting-edge products by collaborating with cross-functional teams to develop solutions and meet performance requirements. Key Responsibilities: - Design and implement end-to-end data analytics workflows tailored to semiconductor engineering use cases. - Develop, train, and optimize ML models us...

Role Overview: You will be responsible for building scalable, fault-tolerant cloud-native services on Microsoft Azure to ensure high performance and reliability. Your role will involve developing secure, well-documented public APIs and SDKs for consumption by internal and external developers. Collaboration with cross-functional teams to deliver end-to-end solutions across data pipelines, orchestration, and service APIs will be a significant aspect of your work. Additionally, embedding robust security controls to protect sensitive data and ensure secure access to AI services will be a key focus. You will also contribute to design reviews, code reviews, and architectural discussions to uphold ...

Role Overview: As a Retail Specialized Data Scientist at Level 9 in the SnC GN Data & AI team, you will be responsible for utilizing advanced analytics, machine learning, and statistical modeling techniques to support data-driven decision-making in the retail sector. Your role will involve collaborating with various teams such as marketing, product management, supply chain, and customer insights to drive business strategies and innovations. The ideal candidate should possess experience in retail analytics and demonstrate the ability to transform data into actionable insights. Key Responsibilities: - Leverage Retail Knowledge: Utilize your in-depth understanding of the retail industry, includ...

Req ID: 344131 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Power BI-GCP Integration to join our team in Bangalore, Karn?taka (IN-KA), India (IN). We are seeking a skilled and proactive Developer with expertise in Power BI and Google Cloud Platform (GCP) to join our Integrations team. The ideal candidate will be responsible for designing, developing, and maintaining integration solutions that support business intelligence and cloud-based applications. Key Responsibilities: Perform detailed design of application an...

About This Role About the role The? Lead Data Quality and Reliability Engineering ?will architect and build the infrastructure, processes, technology and collaboration model that embeds?rigorous data quality and reliability practices?to ensure that data across all systems from ingestion through consumption is?accurate, consistent, and trustworthy. They will spearhead the development of our next-generation? data ecosystem ?with a quality-first mindset, helping? shape a platform that is not only scalable and innovative, but also delivers trusted data for critical financial and analytical use cases . As a leader, this position demands a balance of? hands-on execution, strategic direction, and t...

Job title: Web Scraping Engineer Location: Gurgaon About the Role We are looking for a Web Scraping Engineer with proven experience in building, maintaining, and scaling data extraction systems - especially from e-commerce platforms . The ideal candidate will design and implement robust scrapers to collect, clean, and normalize product data (pricing, availability, reviews, images, etc.) that power our analytics and competitive pricing systems. What you'll do : Develop and maintain scalable scraping pipelines to extract structured data from complex, dynamic e-commerce sites . Design and manage distributed scraping infrastructure (e.g., rotating proxies, request throttling). Handle dynamic ren...

We are looking for a skilled professional with 6-17 years of experience to join our team as an Agentic AI expert. The ideal candidate will have a strong background in Artificial Intelligence and experience working with agent frameworks. Roles and Responsibility Design and develop innovative AI solutions using various agent frameworks. Collaborate with cross-functional teams to integrate AI models into existing systems. Develop and implement automated testing scripts to ensure high-quality AI solutions. Troubleshoot and resolve complex technical issues related to AI model integration. Work closely with data engineers to design and implement data pipelines for AI model training. Develop and ma...

We are looking for a skilled Big Data Developer with 6-8 years of experience to join our team in Hyderabad/Pune. The ideal candidate will have strong knowledge of Unix/BigData Scripting and a solid understanding of BigData (CDP/Hive) Environment. Roles and Responsibility Design, develop, and implement big data solutions using Hadoop/CDP, Hive. Collaborate with the Lead Hadoop Data Engineer and Product Owner to achieve project goals. Support existing applications and design/build new Data Pipelines. Participate in service management and issue resolution activities. Ensure efficient use of Hive, Unix Scripting, and Control-M to reduce lead time to delivery. Provide solutions and tools to enhan...

We are looking for a skilled Snowflake Developer with 5-20 years of experience to join our team in Chennai. The ideal candidate will have strong expertise in Snowflake, ETL, and data ingestion pipelines. Roles and Responsibility Design, develop, and implement data pipelines using Snowflake. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain complex SQL queries and PL/SQL scripts. Work closely with stakeholders to understand business needs and provide technical solutions. Ensure high-quality deliverables and meet project deadlines. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements ...

Looking to onboard a skilled professional with 8-16 years of experience in the field, located anywhere in India. Roles and Responsibility Design and develop scalable data pipelines using Google Data Products such as Cloud Data Fusion, BigQuery, Dataflow, Dataproc, AI Building Blocks, Looker, Dataprep, etc. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data engineering solutions using E2E Data Engineering and Lifecycle management techniques. Ensure data quality and metadata management through effective MDM tools. Implement data visualization literacy for business insights. Participate in prototyping, usability testing...

Primary skills: Hana, Hadoop, Redis, Kafka, InfluxDB, Postgres, Cassandra, and MySQL Terraform and Ansible Python Primary Responsibilities The Cloud Database Engineer provides 24x7 operational support and designs and creates automation for the Hana, Hadoop, Redis, Kafka, InfluxDB, Postgres, Cassandra, and MySQL instances supporting the SAP Procurement and Business Network cloud platform in public cloud, SAP Business Technology Platform, and SAP Managed Data Centers Data Dynamo: Design, deploy, and maintain highly available, scalable, and secure cloud databases to support real-time data processing and fuel data-driven insights. Support Superhero: Provide 24x7 support, monitoring, and proactiv...

Responsibilities: We are looking for a highly skilled Databricks & Azure Senior Data Engineer with at least 6 to 10 years of experience to join our vibrant Data & Analytics team. The ideal candidate should have strong background in Azure data services, including Databricks, Synapse Analytics, Microsoft Fabric, PySpark and SQL. He/She will be responsible for designing, building, and maintaining scalable data solutions that meet business requirements and align with industry best practices. What is expected out of you? 6+ years of experience as a Databricks & Data Engineer, building Azure-based data solutions using Data Factory, Synapse, Databricks. Expertise in Python, SQL (SparkSQL, TSQL, PLS...

What you will do Build and maintain backend services, APIs, and integrations Own modules end-to-end — development, testing, documentation Improve code quality, performance, and automation workflows Work with cross-functional teams to understand requirements and turn them into simple, working solutions Debug production issues and close them quickly Review junior developers’ code and help them when needed Required education Bachelor's Degree Required technical and professional expertise Solid experience in Python programming (minimum 6 years). Expertise with at least one web frameworkFlask, Django, or FastAPI. Experience with databasesboth SQL (PostgreSQL, MySQL) and NoSQL (MongoDB, Redis). Co...

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by ? a collaborative community of colleagues around the world, and where you ’ ll be able to reimagine what ’ s possible. Join us and help the world ’ s leading organizations ? unlock the value of technology and build a more sustainable, more inclusive world. Your Role Design and maintain data pipelines for ingestion, transformation, and storage. Ensure data quality, integrity, and security across systems. Build and deploy machine learning models and integrate them into production environments. Optimize model performance and monitor reliab...

Work with broader team to build, analyze and improve the AI solutions. You will also work with our software developers in consuming different enterprise applications Required education Bachelor's Degree Required technical and professional expertise Resource should have 5-7 years of experience. Sound knowledge of Python and should know how to use the ML related services. Proficient in Python with focus on Data Analytics Packages. Strategy Analyse large, complex data sets and provide actionable insights to inform business decisions. Strategy Design and implementing data models that help in identifying patterns and trends. Collaboration Work with data engineers to optimize and maintain data pip...

Project Role :Data Engineer Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills :Python (Programming Language) Good to have skills :NA Minimum 5 year(s) of experience is required Educational Qualification :15 years full time education Summary:As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementin...

Project Role :Data Engineer Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills :Apache Spark Good to have skills :Python (Programming Language), Google BigQuery, PySpark Minimum 12 year(s) of experience is required Educational Qualification :15 years full time education Summary:As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

Project Role :Data Engineer Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills :Tableau Good to have skills :NA Minimum 5 year(s) of experience is required Educational Qualification :15 years full time education Summary:As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to eff...

Project Role :Data Engineer Project Role Description :Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills :Apache Airflow Good to have skills :NA Minimum 5 year(s) of experience is required Educational Qualification :15 years full time education Summary:As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes...

About the Job At Sanofi, were committed to providing the next-gen healthcare that patients and customers need. Its about harnessing data insights and leveraging AI responsibly to search deeper and solve sooner than ever before. Join our R&D Data & AI Products and Platforms Team as Data Engineering Manager and you can help make it happen. What you will be doing: Sanofi has recently embarked into a vast and ambitious digital transformation program. A cornerstone of this roadmap is the acceleration of its data transformation and of the adoption of artificial intelligence (AI) and machine learning (ML) solutions, to accelerate R&D, manufacturing and commercial performance and bring better drugs ...

Project Role :Custom Software Engineer Project Role Description :Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills :Unqork Enterprise Application Platform Good to have skills :NA Minimum 12 year(s) of experience is required Educational Qualification :15 years full time education About The Role We are looking for a Unqork Developer to join our team and help build, maintain, and scale enterprise-grade applications using the Unqork platform. This person will work closely with business stakeholders, ...