1569 Data Pipeline Jobs - Page 12

Job Summary Join our dynamic team as a Sr AI Engineer where you will leverage cutting-edge technologies to drive innovation and deliver impactful solutions. With a focus on AWS Machine Learning GenAI and other advanced tools you will play a crucial role in transforming data into actionable insights. This hybrid role offers the opportunity to work with a talented team in a day shift environment without the need for travel. Responsibilities Develop and implement machine learning models using AWS Machine Learning and Amazon Sagemaker to solve complex business problems and enhance decision-making processes. Utilize Neuro AI and Open AI Service to create innovative solutions that improve customer...

SUMMARY Position : Data Engineer (Senior/Junior) Experience : 4 Years 10 Years Job location : Hyderabad/Gurugram Working Mode : Hybrid(4 Days WFO) Shifts : 12.30 PM 9.30 PM Technical Skills (Required): Azure Data Factory (ADF): design, develop and orchestrate scalable data pipelines Azure Databricks: build and manage Spark clusters, notebooks, and jobs Microsoft Fabric (OneLake): ingest, organize and transform data within the Fabric ecosystem PySpark & SparkSQL: author complex transformations and SQL queries for large scale data processing Delta Tables: create, optimize and maintain Delta Lake tables (VACUUM, ZORDER, OPTIMIZE) JSON Processing: parse, validate and transform semi structured JS...

Designing & optimizing big data pipelines .Handle high-velocity AIS data Streaming data, geospatial analysis. Predictive modelling to enhance decision-making for fleet optimization, port congestion assessment & trade flow analysis. Annual bonus Health insurance

We are looking for an experienced Technical Delivery Manager with Cloud Data and Analytics to lead our data and analytics initiatives. The ideal candidate will oversee the design, development, and implementation of data strategies, ensuring our data pipelines and infrastructure are optimized for performance and aligned with business objectives. You will collaborate with cross-functional teams to deliver actionable insights and drive data-driven decision-making across the organization. You will lead a team of data engineers working on various projects and you will be expected to ensure the design and development quality of the work produced by the data engineers is of high quality and perform...

Position Title: Data Manager- Information Technology Company Name: JU Agri Sciences Pvt. Ltd. Department: Information Technology Experience: 8+ Years Location: Chennai Position Overview The Data Manager - Information Technology will be responsible for driving data, analytics, and digital transformation initiatives across the organization. This is a hands-on, individual contributor role requiring deep technical expertise in data analysis, business intelligence, and modern data technologies. The incumbent will independently design, develop, and manage BI systems, dashboards, and analytical models that support strategic and operational decision-making. Key Responsibilities 1. Data Strategy & Ma...

We are looking for an experienced Data Engineer with deep expertise in Google Cloud Platform (GCP) to design, build, and manage end-to-end data systems that drive analytics, business intelligence, and AI initiatives. In this role, youll be responsible for building high-performance data pipelines that handle data ingestion, transformation, and integration (ETL/ELT) across multiple systems. Beyond ETL, you will design the broader data infrastructure, including data lakes, workflow orchestration, API-driven integrations, and CI/CD automation to ensure scalability, reliability, and seamless data flow across the enterprise. This is a hands-on engineering position that blends data architecture, so...

Company Overview: BOLD is revolutionising the career landscape by empowering individuals to discover and develop their professional paths. Our recent acquisitions of Monster and CareerBuilder will expand our global reach and impact. With a range of brands like FlexJobs, Zety, and Sonara and more, we provide job seekers with the tools to create impressive resumes and connect with employers. BOLD fosters a dynamic, inclusive culture that prioritises innovation and growth. So don't wait and join us in shaping the future of work and redefining successful careers. Required Skills: Experience working as an ETL developer in a Data Engineering, Data Warehousing or Business Intelligence team. Underst...

Designation: AWS Data Engineer Experience: 6+Years Skill Tech stack : AWS Data Engineer, Python, Pyspark, SQL, Data Pipeline, AWS, AWS Glue, lambda Notice Period Immediate to 2 week Joiner only Location: Pune, Hyderabad and Bangalore Job Description: 6+ years of experience in data engineering, specifically in cloud environments like AWS. Dont Not Share Data Science Profiles. Proficiency in Python and PySpark for data processing and transformation tasks. Solid experience with AWS Glue for ETL jobs and managing data workflows. Hands-on experience with AWS Data Pipeline (DPL) for workflow orchestration. Strong experience with AWS services such as S3, Lambda, Redshift, RDS, and EC2. Technical Sk...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions. Outcomes: Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development mainten...

SignDesk pioneered document digitisation. Now, as Melento, we advance with a Collaborative Intelligence Platform combining trust, automation, and agentic AI to create intelligent, seamless workflows where humans and AI co-create without silos.

Required Skills & Qualifications Proven experience as a Python Developer. • Proficiency in Python and its core libraries. • Solid understanding of object-oriented programming (OOP). • Experience with a web framework like Django, Flask, or FastAPI. • Experience with task queues (e.g., Celery/RQ/ ActiveMQ/RabbitMQ or similar queues). • Experience with cache databases (e.g., Redis/Memcached/AeroSpike) • Familiarity with relational databases (e.g., PostgreSQL, MySQL) and ORMs (e.g., SQLAlchemy). • Experience with NoSQL databases (e.g., Aerospike/MongoDB). • Experience with data streaming and processing frameworks (e.g., Apache Kafka/Apache Flink). • Experience with version control systems, prefe...

Company- TOP MNC Location- Chennai, Delhi NCR, Pune, Bangalore Notice Period- 0- 60 Days Pyspark, SQL, ETL, AWS,Glue, Unix, DW concept Role & responsibilities Good Understanding of Cloud, Hadoop and DW concepts. Exposure to data governance or lineage tools such as Immuta and Alation is added advantage. Hands on experience in Pyspark and strong knowledge on Dataframes, RDD and SparkSQL Experience in using Orchestration tools such as Apache Airflow or Snowflake Tasks is added advantage. Hands on experience in Pyspark performance optimization techniques. Knowledge on Ab-initio ETL tool is a plus Hands on Experience in developing, testing, and maintaining applications on AWS Cloud. Strong hold o...

Key Accountabilities / Responsibilities: Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to End pipeline design, and code quality. Review and optimize PySpark , SQL , and Databricks code for performance, scalability, and maintainability. Offer engineering support and mentorship to data engineering teams within delivery squads, guiding them in building robust, reusable, and secure data solutions. Collaborate with architects to define data ingestion, transformation, and storage strategies leveraging Azure services such as Azure Data Factory , Azure Databricks, Azure Data Lakes Drive automati...

dunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First. Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of the world’s most competitive markets, with a deluge of multi-dimensional data – dunnhumby today enables businesses all over the world, across industries, to be Customer First. dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola,...

JOB DESCRIPTION FOR Sr. Data Engineer Market Facing Title: Sr. Data Engineer What do you drive for us? As a Sr. Data Engineer at Randstad, you will play a pivotal role in designing, implementing, and maintaining our data infrastructure on Google Cloud Platform. You will work closely with our business intelligence, engineering, and business teams to ensure that our data is collected, stored, processed, and analyzed effectively, enabling us to make informed decisions and drive impactful outcomes. What a typical day at work would look like? Collaborate with stakeholders: Engage in discussions with business users, BI analysts, data engineers to understand their data needs and translate them into...

Job Title: Data Engineer Job Type: Contract (Full-time) Location: Hybrid Delhi, Pune Job Summary: Join our customer’s team as a Data Engineer and play a key role in building scalable data solutions. In this hybrid role, you will work on designing data pipelines, ensuring data quality, and collaborating with cross-functional teams using Agile methodologies. Key Responsibilities: Design and maintain scalable data pipelines and architectures Cleanse, evaluate, and integrate data from multiple sources Implement data validation , data completeness , and quality checks Automate ETL processes using Python and SQL Manage repositories and documentation using Bitbucket , Jira , and Confluence Conduct ...

The Data Engineering team is responsible for building and maintaining the data applications and platform that enables the ISS business to operate. This role is appropriate for a senior data engineer capable of taking ownership and a delivering a subsection of the wider data platform or data applications. As a Senior Data Engineer, you will directly contribute to our key organizational objectives of Accelerated Innovation, Cost Optimization, Risk Mitigation, and Business Enablement Key Responsibilities • Implement and maintain data pipelines for ingestion, transformation, and delivery • Ensure data quality through validation and monitoring processes • Collaborate with senior engineers to desi...

Role & responsibilities to join KONE IT Enterprise Data & Analytics team in Pune, India. The team is driving forward Data Foundation, new way of working with data using latest cloud technology. Data Foundation is key enabler in our digital transformation creating ability to develop new scalable analytics, AI and digital use cases by leveraging data across the whole organization. In our approach, data products play a vital role in business value generation. Driving optimized data engineering practices is crucial to ensure reusability and maintainability of the data products. Principal Data Engineer leads the design, development, and industrialization of data pipelines to provide industrial gr...

Hello Everyone !! Greetings from Acro Technologies !! We are hiring for "Data Integration Engineer ", for one of our reputed Client, based out in Bangalore,Pune, Noida, Gurugram. Job Role:- Data Integration Engineer Exp Range:- 5+yrs Job Purpose/Summary:- The Advanced Intelligence Work Group is seeking a highly skilled API & Data Integration Engineer to design, build, and maintain scalable backend integrations that enable seamless data flow across applications, databases, AI copilots, and third-party platforms. The ideal candidate will have strong expertise in REST API development, cloud-native architecture, ETL/ELT workflows , and automation using no-code platforms. This role plays a key pa...

Required Technical Skill Set Snowflake, DBT, Python Data Engineering Design and implement data pipelines and ETL processes leveraging Snowflake, DBT, and Python. Develop and maintain DBT models (staging, intermediate, and marts layers). Optimize SQL queries and Snowflake performance (clustering, caching, warehouse sizing). Implement CI/CD pipelines for DBT and Snowflake deployments. Troubleshoot data issues, pipeline failures, and performance bottlenecks.

We are looking a Senior Data Engineer with extensive experience in designing and building scalable data pipelines, integrating AI/LLM models, and developing automation solutions using Python. The ideal candidate will have strong hands-on expertise in ETL workflows, deep proficiency with cloud platforms, and a proven track record of implementing robust data governance and security practices. Role & responsibilities Bachelors or Masters degree in Computer Science, Engineering, or a related field. Extensive full-lifecycle data engineering experience within enterprise environments. Strong expertise in ETL development, API integrations, data processing workflows, and implementation of GenAI-power...

Job : Design, develop, troubleshoot and debug software programs for databases, applications, tools, networks etc. Job description: Design, develop, troubleshoot and debug software programs for databases, applications, tools, networks etc. We are seeking a hands-on Senior Member of Technical Staff with proven experience in SQL and NoSQL databases, programming languages like Java, Python etc. In this role, you will design and maintain complex data pipelines, enable scalable AI solutions, and collaborate with multidisciplinary teams to advance our enterprise analytics and AI capabilities. Responsibilities: Minimum Qualifications: Bachelors or Masters degree in Computer Science, Data Engineering...

As an Enterprise Architect (AI / GenAI / Cloud) at HCLTech, your role will involve designing scalable architecture that integrates AI capabilities into existing ecosystems. You will be responsible for building AI / ML models on cloud platforms, managing data pipelines, and selecting and deploying models effectively. Additionally, you will need to have a deep understanding of GenAI concepts such as Generative Models, NLP, Image Generation, and Reinforcement Learning. Your expertise in Deep Learning, including Neural networks, transformers, RNNs, and CNNs, will be crucial for this role. Qualifications Required: - Strong AI system Architectural Skills - Proficiency in AI / ML model building on ...

Data Engineer Data Quality Build automated data quality checks using Databricks (Delta Lake) and PySpark . Perform profiling, anomaly detection, and reconciliation across systems. Investigate data issues using SQL, Python , drive root cause fixes. Collaborate on governance, metadata, and data catalog integration. Automate validations via Airflow , ADF , or Databricks Workflows . 510 years experience in data engineering or quality; strong SQL, Python, PySpark skills. NOTE: Those who have applied before or interviewed by us in the last 6 months are not eligible to apply. Only BENGALURU based candidates willing to join within 30 days upon offer should apply.

Han Digital Solution (P) is looking for Data Engineer with Python, Spark, HFDS, & MongoDB to join our dynamic team and embark on a rewarding career journey We are looking for a skilled Data Engineer with expertise in Python, Spark, HFDS, and MongoDB to join our team at Han Digital Solution (P) The ideal candidate will have a strong background in data engineering and excellent problem-solving skills As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems Roles and Responsibility Design and develop scalable data pipelines using Python, Spark, and other relevant technologies Collaborate with cross-functional teams to identify business requ...