867 Data Pipeline Jobs - Page 4

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/ AzureDatabricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Hands-on experience with creating ...

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure and AWS Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/Azure Databricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Azure Cloud, AWS Cloud, Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Ha...

DMI Innovations a Encora company is seeking a full-time Lead Data Engineer who has expertise with Logistic domain to support our manufacturing clients large scale digital transformation. The Lead Data Engineer is responsible for ensuring the day-to-day leadership and guidance of the local, India-based, data team. This role will be the primary interface with the management team of the client and will work cross functionally with various IT functions to streamline project delivery. Duties and Responsibilities: Work with a distributed team with primary focus on the management of resources, team assignments, and fostering career growth Input into the hiring of team members for the locally based ...

Job Title: Principal Data Engineer, Logistics Employment Type: Full Time Experience: 12+ Years About the Role: We are looking for a Principal Data Engineer to lead the design and delivery of scalable data solutions using Azure Data Factory and Azure Databricks. This is a consulting-focused role that requires strong technical expertise, stakeholder engagement, and architectural thinking. You will work closely with business, functional, and technical teams to define data strategies, design robust pipelines, and ensure smooth delivery in an Agile environment. Responsibilities Collaborate with business and technology stakeholders to gather and understand data needs Translate functional requireme...

Job Title: GCP Data Engineer Location: Pune, India Experience: 3 to 7 Years Job Type: Full-Time Job Summary: We are looking for a highly skilled GCP Data Engineer with 3 to 7 years of experience to join our data engineering team in Pune . The ideal candidate should have strong experience working with Google Cloud Platform (GCP) , including Dataproc , Cloud Composer (Apache Airflow) , and must be proficient in Python , SQL , and Apache Spark . The role involves designing, building, and optimizing data pipelines and workflows to support enterprise-grade analytics and data science initiatives. Key Responsibilities: Design and implement scalable and efficient data pipelines on GCP , leveraging D...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Designing and developing Data Pipeline Ensure data integrity and quality Design, develop, and maintain the data models Providing clean, reliable, and timely data Define high-quality data for Data Science and Analytics use-cases Leading a team. Required Candidate profile Proficient in writing complex SQL queries, Strong in Python, Analytical Mindset, strong on building data lake, data pipeline & and architecture. Hands on Data Modeling Data Streaming Leading team exp

About the Role: We are hiring a Senior Data Engineer with strong expertise in Palantir Foundry or Gotham to lead the design, development, and optimization of data engineering solutions across large-scale enterprise environments. The ideal candidate will have deep experience building secure, high-performance data pipelines using Python, ETL/ELT practices, distributed computing frameworks, and cloud platforms ( AWS ,Azure ,GCP)while working closely with DevOps, analytics, and business teams to enable data-driven decisions. Key Responsibilities: Architect, build, and optimize large-scale ETL/ELT pipelines using Palantir Foundry/Gotham . Integrate diverse data sources including SQL , NoSQL , API...

We are seeking a hands-on and forward-thinking Principal or Lead Engineer with deep expertise in Java-based backend development and a strong grasp of Generative AI technologies . You will lead the design, development, and deployment of Gen AI-based solutions, working across data, ML engineering, and software engineering teams to integrate AI capabilities into our core platforms and client-facing products. Responsibilities Lead end-to-end architecture and implementation of Generative AI solutions integrated with Java-based applications Evaluate, fine-tune, and deploy foundational and LLM models (e.g., GPT, LLaMA, Claude, Gemini) for use cases such as code generation, summarization, intelligen...

Role & responsibilities Role Purpose The Data Engineer will be responsible for developing and maintaining ADA (Analytical Data Architecture) solutions that support the Personal Bank Reporting teams. This role requires a strong technical foundation in data engineering, with a focus on building scalable, efficient data pipelines and enabling high-quality reporting and analytics. The successful candidate will: Apply critical thinking to solve complex data challenges, identify root causes, and propose innovative solutions that improve data quality and reporting outcomes. Design, build, and optimise ETL processes and ADA data pipelines using SQL , Python , and the Databricks platform , ensuring r...

Urgent Requirement Senior Azure Data Engineer - Microsoft Fabric Specialist Location: Noida Sector 62 (Onsite, Long-Term Project Full Time) Need Immediate Joiners Only Experience Required - 10 Years Key Responsibilities: Lead the design, development, and implementation of Fabric Pipelines, Data Flows , and ETL processes . Build and maintain Notebooks using PySpark / SQL for large-scale data processing. Perform SQL Data Modeling, Tuning, and Optimization for performance-efficient data solutions. Design and implement ETL workflows to move data from Fabric to on-premise systems . Develop and manage ADF / Databricks pipelines for data integration and transformation. Apply Medallion Architecture ...

Primary Responsibilities: Perform all phases of software engineering including; requirements analysis, application design, code development, and testing • Design and implement performance and operational enhancements • Design & Develop reusable components, patterns, and frameworks, encouraging innovation that will enable team velocity • Review code and provide feedback relative to best practices and performance improvement • Conduct design and code reviews to ensure that developed code meets software engineering best practices and guidelines, unit testing, security, and scalability and maintainability guidelines • Troubleshoot production support issues post-deployment and design solutions as...

Tech Stack - AWS Big Data Stack Expertise in ETL, SQL, Python and AWS tools like Redshift,S3, Glue, Data pipeline, Lambda is a must. Good to have knowledge on Glue Workflows, Step Functions, Quick sight, Athena, Terraform and Dockers. Responsibilities -Assists in the analysis, design and development of a roadmap, design pattern, and implementation based upon a current vs. future state from a architecture viewpoint. Participates in the data related technical and business discussions relative to future serverless architecture. Responsible for working with our Enterprise customers and migrate data into Cloud. Set up scalable ETL process to move data into Cloud warehouse. Deep understanding in D...

**Job Title:** Lead Data Engineer - Data **Company:** Happiest Minds **Location:** Bangalore **Job Type:** Full-Time **About Happiest Minds:** At Happiest Minds, we are driven by the belief that technology should be a source of happiness for individuals, businesses, and society. Our team is committed to delivering innovative solutions that empower our clients and drive digital transformation. We are currently seeking a skilled Lead Data Engineer to join our dynamic team. **Job Summary:** We are looking for an experienced Lead Data Engineer with a proven track record of developing and optimizing data pipelines and ETL processes. The ideal candidate will have 8 to 11 years of experience in dat...

**Job Title: Lead Data Engineer - Data** **Company Name: Happiest Minds** **Location: Bangalore **Job Type: Full-Time** **Experience Required: 8 to 11 years** --- ### **Job Description:** Happiest Minds is seeking a dynamic and experienced Lead Data Engineer to join our innovative Data team. The ideal candidate will possess strong expertise in data engineering, particularly with Azure Databricks, Python, SQL, and ETL processes. You will lead data initiatives from conception through to execution and work closely with stakeholders to ensure data solutions are aligned with business objectives. ### **Key Responsibilities:** - **Lead and Design:** Oversee the design and implementation of robust d...

**Job Title:** Lead Data Engineer - Data **Company:** Happiest Minds **Location:** Bangalore **Employment Type:** Full-Time **About Happiest Minds:** Happiest Minds is a market leader in providing digital transformation and IT services. We are committed to making our clients happier by leveraging technology and innovative solutions. We believe in creating an environment that promotes growth and collaboration, and we are looking for driven individuals to join our team. **Job Summary:** We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic data team. The ideal candidate will have 8 to 11 years of experience in data engineering with expertise in cloud-based solu...

**Job Title: Lead Data Engineer - Data** **Company: Happiest Minds** **Location:** Bangalore **Experience:** 8 to 11 years **About Us:** Happiest Minds is a leading Mindful Technology Solutions Company. We are dedicated to making our clients happy by delivering innovative technology solutions. Our team is passionate about fostering a work environment that encourages creativity, collaboration, and growth. **Job Overview:** We are seeking a skilled Lead Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in data engineering, particularly in designing and implementing data solutions that leverage Azure Databricks. The Lead Data Engineer will be responsible...

**Job Title:** Lead Data Engineer - Data **Company Name:** Happiest Minds **Location:** Bangalore **Job Type:** [Full-time/Part-time/Contract] **Experience Required:** 8 to 11 years --- ### Job Description Happiest Minds is seeking a skilled and highly motivated **Lead Data Engineer** to join our dynamic team. The ideal candidate will have a strong background in data engineering with a passion for leveraging data to drive insights and inform business decisions. ### Key Responsibilities: - Lead and manage data engineering projects, ensuring timely and high-quality delivery. - Design, develop, and maintain scalable data pipelines and ETL processes using **Azure Databricks**. - Collaborate with...

**Job Title:** Lead Data Engineer - Data **Company:** Happiest Minds **Location:** Bangalore **Employment Type:** Full-Time **About Happiest Minds:** Happiest Minds is a digital transformation and IT services company that focuses on leveraging cutting-edge technologies to drive business innovation and efficiency. We pride ourselves on creating a culture of collaboration and continuous improvement. We are looking for talented individuals who are passionate about technology and aspire to make a difference in the digital landscape. **Job Description:** We are seeking an experienced Lead Data Engineer with a strong background in data engineering and analytics to join our dynamic team. The ideal ...

**Job Title: Lead Data Engineer - Data** **Company: Happiest Minds** **Location: Bangalore **Job Type: Full-Time** **Experience: 8-11 Years** **About Happiest Minds:** At Happiest Minds, we take pride in creating innovative solutions that drive countless businesses towards their digital transformation goals. We focus on delivering quality, integrity, and customer satisfaction while promoting a culture of continuous learning and collaboration. Join our team of passionate professionals and help us make a difference! **Job Description:** We are seeking a highly skilled and motivated Lead Data Engineer to join our growing team. In this role, you will be responsible for designing, building, and m...

**Job Title: Lead Data Engineer - Data** **Company: Happiest Minds** **Location: Bangalore **Experience Required: 8 to 11 years** --- **Job Description:** Happiest Minds is seeking a dynamic and experienced Lead Data Engineer to join our rapidly growing team. In this role, you will be responsible for leading the design, development, and implementation of data solutions that empower our business to make better, data-driven decisions. The ideal candidate will possess a strong background in data engineering, with extensive experience in Azure Databricks, Python, SQL, and ETL processes. **Key Responsibilities:** - Lead the design and development of scalable and robust data pipelines to support i...

**Job Title:** Data Engineer - Data **Company Name:** Happiest Minds **Location:** Bangalore **Job Type:** Full-time **Experience Required:** 3 to 5 years **About Happiest Minds:** Happiest Minds Technologies is a next-generation digital transformation and technology services company, delivering delightful experiences to its clients through a blend of technology and domain expertise. Join us to create value and make businesses thrive in the ever-evolving digital landscape. **Job Summary:** We are seeking a talented and motivated Data Engineer to join our dynamic data team. The ideal candidate will have a strong foundation in data engineering, specifically in Python, MySQL, SQL, and AWS. You ...

Data Engineering & Pipeline Development Design implement and maintain ETL processes using ADF and ADB. Create and manage views in ADB and SQL for efficient data access. Optimize SQL queries for large datasets and high performance. Conduct end-to-end testing and impact analysis on data pipelines. Optimization & Performance Tuning Identify and resolve bottlenecks in data processing. Optimize SQL queries and Delta Tables for fast data processing. Data Sharing & Integration Implement Delta Share, SQL Endpoints, and other data sharing methods. Use Delta Tables for efficient data sharing and processing. API Integration & Development Integrate external systems through Databricks Notebooks and build...

**Job Title: Data Engineer - Data** **Company: Happiest Minds** **Location: Bangalore **Job Type: Full-time** **About Happiest Minds:** Happiest Minds is a next-generation digital transformation and IT services company focused on driving customer success through innovative solutions. We pride ourselves on our talented teams who foster a culture of continuous learning and excellence. **Job Description:** We are seeking a skilled and motivated Data Engineer to join our dynamic team at Happiest Minds. The ideal candidate will have a strong foundation in data engineering, with hands-on experience in AWS, Python, SQL, and Snowflake. You will play a critical role in designing, building, and mainta...

**Job Title: Data Engineer - Data** **Company Name: Happiest Minds** **Location:** Bangalore **Job Type:** Full-Time **About Happiest Minds:** Happiest Minds is a leading Mindful IT company that empowers enterprises to achieve their digital transformation goals through innovative technology solutions. We pride ourselves on our commitment to delivering exceptional quality and creating a culture of happiness among our employees. Join us to be a part of this journey! **Position Overview:** We are seeking a skilled Data Engineer with 3 to 5 years of experience to join our dynamic data team. The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines a...