1175 Data Pipeline Jobs - Page 16

Exp- 8-15 years 3+ years of hands on experience in Azure Data Engineering development project 2+ years of experience in Databricks development 2+ years of experience in PySpark. 2+ years of experience in SQL Proficient in SQL 2+ years of experience in leading at least a 5+ member team in data engineering delivery Must be fluent in communication Role & responsibilities Preferred candidate profile

Hi all, We are hiring for the role GCP DATA ENGINEER Experience: 5+ Years Location: Pune Notice Period: Immediate - 15 days Budget: 18 LPA only Skills: Ideally you will have a background in software engineering along with experience in working in a complex cloud environment ( Google Cloud Platform) Ideally you will have experience working with Python and SQL. You have experience building data pipelines and ETL frameworks both batch and real time using Python and any of the GCP capabilities such Apache Beam, Data Flow, Data Fusion You have Experience with using Terraform to build Infrastructure-as-a-code Experience working with Big Data Technologies (Spark, Cloud SQL, Big Query ) preferably i...

Duties and Responsibilities: Demonstrate skills in abstract development frameworks. Demonstrate accountability to assigned work & timelines. Demonstrated leadership skills. Demonstrated technical competency for the role. Demonstrated timely delivery on assigned tasks and effective communication on project status and escalation when needed. Education and Experience: 12+ Years or equivalent degree + experience Required Skills/Certifications: Estimating and Planning Work breakdown for the team. Data pipelines Data Transformations Code/Unit Test PR & Code Reviews Performance & Load Testing Functional & Regression Testing Tier 2 support/consulting Production Support Functional ask to Technical de...

Looking For a Better Opportunity? Join Us And Make Things Happen .. DMI is seeking a full-time Data Engineer - Operations to support our manufacturing clients large scale into digital transformation. The Data Engineer - Operations will work within a three to four person team ensuring the day-to-day operations of the data platforms run smoothly. This role will support leadership and will work cross functionally with various IT functions to streamline operations focusing primarily on common ITIL processes (incident, request, problem, and change). Note: We want someone who has hands on experience into data production support or willing to work into support role with a large scale manufacturing ...

Education and Experience: 5+ years of experience in an operational role dealing with data management, data integration or similar activities and/or responsibilities 3+ years of experience in building, mentoring, and leading a team Bachelors Degree and/or relevant field or equivalent experience Required Skills/Certifications: Strong analytical and problem-solving skills - Ability to determine data patterns and perform root cause analysis to resolve production reporting bugs Familiarity with the data, analytics and BI space and knowledge of Power BI Experience working with Microsoft data integration platforms such as Azure Data Factory, SSIS, Synapse Strong experience with SQL (Python experien...

Duties and Responsibilities: Demonstrate skills in abstract development frameworks. Demonstrate accountability to assigned work & timelines. Demonstrated leadership skills. Demonstrated technical competency for the role. Demonstrated timely delivery on assigned tasks and effective communication on project status and escalation when needed. Required Skills/Certifications: Data pipelines Data Transformations Code/Unit Test Performance & Load Testing Functional & Regression Testing Tier 2 support/consulting Production Support Functional ask to Technical design. Data Engineering Agile Methodologies CI/CD Metadata driven frameworks. Technical documentation Azure Synapse Spark Python Complex SQL P...

The Manager, Data Operations is responsible for ensuring the day-to-day operations of the data platforms run smoothly. This role will be the primary interface with the management team and will work cross functionally with various IT functions to streamline operations focusing primarily on common ITIL processes (incident, request, problem, and change). Duties and Responsibilities: Foster a collaborative team environment through mentorship and the development of team members Align team objectives with organizational goals Assist in building the team by identifying top talent and onboarding new team members Monitor and evaluate team performance, provide constructive feedback, and recognize achi...

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/Azure Databricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Azure Cloud, Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Hands-on experience w...

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/ AzureDatabricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Hands-on experience with creating ...

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure and AWS Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/Azure Databricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Azure Cloud, AWS Cloud, Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Ha...

DMI Innovations a Encora company is seeking a full-time Lead Data Engineer who has expertise with Logistic domain to support our manufacturing clients large scale digital transformation. The Lead Data Engineer is responsible for ensuring the day-to-day leadership and guidance of the local, India-based, data team. This role will be the primary interface with the management team of the client and will work cross functionally with various IT functions to streamline project delivery. Duties and Responsibilities: Work with a distributed team with primary focus on the management of resources, team assignments, and fostering career growth Input into the hiring of team members for the locally based ...

Job Title: Principal Data Engineer, Logistics Employment Type: Full Time Experience: 12+ Years About the Role: We are looking for a Principal Data Engineer to lead the design and delivery of scalable data solutions using Azure Data Factory and Azure Databricks. This is a consulting-focused role that requires strong technical expertise, stakeholder engagement, and architectural thinking. You will work closely with business, functional, and technical teams to define data strategies, design robust pipelines, and ensure smooth delivery in an Agile environment. Responsibilities Collaborate with business and technology stakeholders to gather and understand data needs Translate functional requireme...

Job Title: GCP Data Engineer Location: Pune, India Experience: 3 to 7 Years Job Type: Full-Time Job Summary: We are looking for a highly skilled GCP Data Engineer with 3 to 7 years of experience to join our data engineering team in Pune . The ideal candidate should have strong experience working with Google Cloud Platform (GCP) , including Dataproc , Cloud Composer (Apache Airflow) , and must be proficient in Python , SQL , and Apache Spark . The role involves designing, building, and optimizing data pipelines and workflows to support enterprise-grade analytics and data science initiatives. Key Responsibilities: Design and implement scalable and efficient data pipelines on GCP , leveraging D...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Designing and developing Data Pipeline Ensure data integrity and quality Design, develop, and maintain the data models Providing clean, reliable, and timely data Define high-quality data for Data Science and Analytics use-cases Leading a team. Required Candidate profile Proficient in writing complex SQL queries, Strong in Python, Analytical Mindset, strong on building data lake, data pipeline & and architecture. Hands on Data Modeling Data Streaming Leading team exp

About the Role: We are hiring a Senior Data Engineer with strong expertise in Palantir Foundry or Gotham to lead the design, development, and optimization of data engineering solutions across large-scale enterprise environments. The ideal candidate will have deep experience building secure, high-performance data pipelines using Python, ETL/ELT practices, distributed computing frameworks, and cloud platforms ( AWS ,Azure ,GCP)while working closely with DevOps, analytics, and business teams to enable data-driven decisions. Key Responsibilities: Architect, build, and optimize large-scale ETL/ELT pipelines using Palantir Foundry/Gotham . Integrate diverse data sources including SQL , NoSQL , API...

We are seeking a hands-on and forward-thinking Principal or Lead Engineer with deep expertise in Java-based backend development and a strong grasp of Generative AI technologies . You will lead the design, development, and deployment of Gen AI-based solutions, working across data, ML engineering, and software engineering teams to integrate AI capabilities into our core platforms and client-facing products. Responsibilities Lead end-to-end architecture and implementation of Generative AI solutions integrated with Java-based applications Evaluate, fine-tune, and deploy foundational and LLM models (e.g., GPT, LLaMA, Claude, Gemini) for use cases such as code generation, summarization, intelligen...

Role & responsibilities Role Purpose The Data Engineer will be responsible for developing and maintaining ADA (Analytical Data Architecture) solutions that support the Personal Bank Reporting teams. This role requires a strong technical foundation in data engineering, with a focus on building scalable, efficient data pipelines and enabling high-quality reporting and analytics. The successful candidate will: Apply critical thinking to solve complex data challenges, identify root causes, and propose innovative solutions that improve data quality and reporting outcomes. Design, build, and optimise ETL processes and ADA data pipelines using SQL , Python , and the Databricks platform , ensuring r...

Urgent Requirement Senior Azure Data Engineer - Microsoft Fabric Specialist Location: Noida Sector 62 (Onsite, Long-Term Project Full Time) Need Immediate Joiners Only Experience Required - 10 Years Key Responsibilities: Lead the design, development, and implementation of Fabric Pipelines, Data Flows , and ETL processes . Build and maintain Notebooks using PySpark / SQL for large-scale data processing. Perform SQL Data Modeling, Tuning, and Optimization for performance-efficient data solutions. Design and implement ETL workflows to move data from Fabric to on-premise systems . Develop and manage ADF / Databricks pipelines for data integration and transformation. Apply Medallion Architecture ...

Primary Responsibilities: Perform all phases of software engineering including; requirements analysis, application design, code development, and testing • Design and implement performance and operational enhancements • Design & Develop reusable components, patterns, and frameworks, encouraging innovation that will enable team velocity • Review code and provide feedback relative to best practices and performance improvement • Conduct design and code reviews to ensure that developed code meets software engineering best practices and guidelines, unit testing, security, and scalability and maintainability guidelines • Troubleshoot production support issues post-deployment and design solutions as...

Tech Stack - AWS Big Data Stack Expertise in ETL, SQL, Python and AWS tools like Redshift,S3, Glue, Data pipeline, Lambda is a must. Good to have knowledge on Glue Workflows, Step Functions, Quick sight, Athena, Terraform and Dockers. Responsibilities -Assists in the analysis, design and development of a roadmap, design pattern, and implementation based upon a current vs. future state from a architecture viewpoint. Participates in the data related technical and business discussions relative to future serverless architecture. Responsible for working with our Enterprise customers and migrate data into Cloud. Set up scalable ETL process to move data into Cloud warehouse. Deep understanding in D...

**Job Title:** Lead Data Engineer - Data **Company:** Happiest Minds **Location:** Bangalore **Job Type:** Full-Time **About Happiest Minds:** At Happiest Minds, we are driven by the belief that technology should be a source of happiness for individuals, businesses, and society. Our team is committed to delivering innovative solutions that empower our clients and drive digital transformation. We are currently seeking a skilled Lead Data Engineer to join our dynamic team. **Job Summary:** We are looking for an experienced Lead Data Engineer with a proven track record of developing and optimizing data pipelines and ETL processes. The ideal candidate will have 8 to 11 years of experience in dat...

**Job Title: Lead Data Engineer - Data** **Company Name: Happiest Minds** **Location: Bangalore **Job Type: Full-Time** **Experience Required: 8 to 11 years** --- ### **Job Description:** Happiest Minds is seeking a dynamic and experienced Lead Data Engineer to join our innovative Data team. The ideal candidate will possess strong expertise in data engineering, particularly with Azure Databricks, Python, SQL, and ETL processes. You will lead data initiatives from conception through to execution and work closely with stakeholders to ensure data solutions are aligned with business objectives. ### **Key Responsibilities:** - **Lead and Design:** Oversee the design and implementation of robust d...

**Job Title:** Lead Data Engineer - Data **Company:** Happiest Minds **Location:** Bangalore **Employment Type:** Full-Time **About Happiest Minds:** Happiest Minds is a market leader in providing digital transformation and IT services. We are committed to making our clients happier by leveraging technology and innovative solutions. We believe in creating an environment that promotes growth and collaboration, and we are looking for driven individuals to join our team. **Job Summary:** We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic data team. The ideal candidate will have 8 to 11 years of experience in data engineering with expertise in cloud-based solu...

**Job Title: Lead Data Engineer - Data** **Company: Happiest Minds** **Location:** Bangalore **Experience:** 8 to 11 years **About Us:** Happiest Minds is a leading Mindful Technology Solutions Company. We are dedicated to making our clients happy by delivering innovative technology solutions. Our team is passionate about fostering a work environment that encourages creativity, collaboration, and growth. **Job Overview:** We are seeking a skilled Lead Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in data engineering, particularly in designing and implementing data solutions that leverage Azure Databricks. The Lead Data Engineer will be responsible...