1175 Data Pipeline Jobs - Page 11

Role: Senior Data Engineer Experience: 5-10 Yrs Location: Chennai/Pune Notice Period: Immediate to 15 days only Interview: L1 - Virtual / L2 Virtual Mode: Hybrid – 4 Days WFO in a week Job Description: We are building a scalable data ingestion and streaming platform that ingests change data capture (CDC) events from diverse source systems (databases and applications), processes them in real time and lands curated data into our analytics lake. The platform uses Confluent connectors (Debezium/Oracle CDC) to emit Parquet files into cloud storage and leverages Databricks Auto Loader to incrementally ingest, deduplicate and write this data into Delta Lake Bronze table. To ensure broad applicabili...

Job Title: GCP Data Engineer Location: Pune, India Experience: 3 to 7 Years Job Type: Full-Time Job Summary: We are looking for a highly skilled GCP Data Engineer with 3 to 7 years of experience to join our data engineering team in Pune . The ideal candidate should have strong experience working with Google Cloud Platform (GCP) , including Dataproc , Cloud Composer (Apache Airflow) , and must be proficient in Python , SQL , and Apache Spark . The role involves designing, building, and optimizing data pipelines and workflows to support enterprise-grade analytics and data science initiatives. Key Responsibilities: Design and implement scalable and efficient data pipelines on GCP , leveraging D...

KiE Square is seeking a highly skilled Senior Full Stack Data Engineer with a strong background in analytics consulting. The ideal candidate must have hands-on experience designing and implementing robust data architectures, building scalable data pipelines, and enabling analytics and machine learning solutions for business impact. This role requires a blend of technical expertise, solution design capability, and business understanding to support analytical consulting projects across diverse industries. You will work closely with data scientists, BI developers, and client teams to ensure high-quality, production-ready data environments and insights delivery. Key Responsibilities Design, deve...

We are looking for a skilled and experienced Databricks Engineer to join our team at our Hyderabad (Madhapur) office. The ideal candidate will have a strong background in data engineering, hands-on experience with Databricks, and a solid understanding of cloud platforms (AWS, Azure, or GCP). This is a great opportunity to work on cutting-edge data projects in a collaborative and fast-paced environment. Key Responsibilities : - Design and implement scalable and efficient data pipelines, ETL/ELT processes, and data integration solutions using Databricks. - Work with large-scale structured and unstructured datasets using Python, PySpark, SQL, and NoSQL technologies. - Collaborate with cross-fun...

We are seeking dynamic candidate for the role of Data Engineer,who is proficient in Data Engineering,Fraud & Financial Crime Risk Management, Real-time Data Pipelines, Fraud Tools ,Python, SQL, Data Processing Frameworks and Machine Learning

we are hiring python ,Azure services (ADF,Data Factory,Databrikes),Rest APi Integration.Apache,kafka ,Data Pipeline

Key Responsibilities: • Design, develop, and maintain robust ETL/ELT ingestion pipelines using SQL and Python. • Build and optimize data workflows in Databricks for large-scale data processing. • Implement and manage data solutions on Azure, including but not limited to Azure Data Lake, and Azure Data Factory. • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. • Ensure data quality, integrity, and security across all data platforms. • Monitor and troubleshoot data pipelines and performance issues. • Document data architecture, processes, and best practices. Required Qualifications: • Bachelors degree in Computer Science, Engineering, or r...

Job Summary: • Experience in data engineering tools like Spark, Hadoop, Azure Data pipelines, Azure Data factory, Azure Data Lake Storage, Databricks, Delta Lake, Python, SQL Server, and experience with other cloud providers –AWS, GCP. (Experience preferably with insurance domain) • Hands on experience in CI/CD tools like Azure Devops utilizing services such as Azure Repos, Azure Boards, and Azure Test Plans • Strong knowledge and experience implementing solutions on Databases like Hadoop, SQLServer, Oracle, Snowflake • Demonstrated experience in building, tuning data pipelines on Spark, Cloud native tools like EMR, Azure Data Factory, SSIS • Understand the need for both batch and stream ing...

**Job Title:** Senior Data Engineer - Data **Company Name:** Happiest Minds **Location:** Noida **Job Type:** Full-Time **Experience Required:** 5-8 Years **About Happiest Minds:** Happiest Minds is a digital transformation and IT services company that delivers innovative solutions to clients globally. We are committed to creating happier experiences through cutting-edge technology and data-driven insights. We're looking for passionate professionals to join our team and contribute to our mission of transformation. **Job Overview:** We are seeking a skilled and motivated Senior Data Engineer to join our dynamic team at Happiest Minds. The ideal candidate will have a strong background in data ...

**Job Title:** Data Engineer - Data **Company Name:** Happiest Minds **Location:** Noida **Job Type:** Full-time **About Happiest Minds:** Happiest Minds is a leading digital transformation and IT services company, empowering businesses through innovative solutions and advanced technology. We pride ourselves on creating a vibrant and inclusive work culture that fosters collaboration, creativity, and growth. **Job Description:** We are seeking a talented and experienced Data Engineer to join our dynamic team. The ideal candidate will have a minimum of 3 years and a maximum of 5 years of hands-on experience in data engineering, with a strong focus on leveraging cutting-edge technologies. If yo...

We, Blobstation, are one of the fastest growing companies in the software development industry We work at a fast pace on awesome projects with our partners and clients We are always looking for energetic, smart and talented people to join our team We are looking for a Data Engineer who will act as a mentor to colleagues and promotes best practices in his/her field of expertise, contributes to design and architecture decisions, and makes technical recommendations The person has hands-on expertise with data pipeline development, data modelling and schemas design across a wide range of technologies Experience:3 to 5 Years Workplace Type:On-site Responsibilities Design and implement data pipelin...

Role : Data Ops engg Location : Hyderabad Skillset : Data pipeline orchestration Monitoring & logging (Prometheus, Grafana, Splunk, ELK) Incident management & troubleshooting Data quality frameworks

We are seeking experienced Data Engineers for a C2H role in Hyderabad / Pune The ideal candidates should have 6- 8 years of relevant experience in designing, developing, and maintaining scalable data pipelines, working with large datasets, and integrating data from multiple sources Responsibilities include building ETL processes, ensuring data quality, supporting analytics and reporting, and collaborating with cross-functional teams to deliver efficient data solutions Strong problem-solving and technical skills are required

Duration : 6 Months Timings : General IST Notice Period : within 15 days or immediate joiner About The Role : As a Data Engineer for the Data Science team, you will play a pivotal role in enriching and maintaining the organization's central repository of datasets. This repository serves as the backbone for advanced data analytics and machine learning applications, enabling actionable insights from financial and market data. You will work closely with cross-functional teams to design and implement robust ETL pipelines that automate data updates and ensure accessibility across the organization. This is a critical role requiring technical expertise in building scalable data pipelines, ensuring ...

Roles and Responsibilities : - Create and maintain efficient and scalable data models as per business needs - Create and maintain optimal data pipelines against multiple data sources lie SQL, BigData on Azure / AWS cloud; - Assemble and process large, complex data sets to meet both functional and non-functional business requirements; - Analyze and improve existing data models, pipelines, related infrastructure and processes - Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics; - Monitor and improve data quality and data governance policies - Collaborate with stakehold...

Role Proficiency: This role requires proficiency in data pipeline development including coding testing and implementing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc along with coding skills in Python PySpark and SQL. Works independently according to work allocation. Outcomes: Operate with minimal guidance to develop error-free code test applications and document the development process. Understand application features and component designs to develop them in accordance with user stories and requirements. Code debug test document and communicate the stages of product...

Role Proficiency: This role requires proficiency in data pipeline development including coding testing and implementing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc along with coding skills in Python PySpark and SQL. Works independently according to work allocation. Outcomes: Operate with minimal guidance to develop error-free code test applications and document the development process. Understand application features and component designs to develop them in accordance with user stories and requirements. Code debug test document and communicate the stages of product...

Role Proficiency: This role requires proficiency in data pipeline development including coding testing and implementing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc along with coding skills in Python PySpark and SQL. Works independently according to work allocation. Outcomes: Operate with minimal guidance to develop error-free code test applications and document the development process. Understand application features and component designs to develop them in accordance with user stories and requirements. Code debug test document and communicate the stages of product...

Role : Senior Databricks Engineer / Databricks Technical Lead/ Data Architect. Experience : 8-15 years. Primary Roles And Responsibilities : - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. - Ability to provide solutions that are forward-thinking in data engineering and analytics space. - Collaborate with DW/BI leads to understand new ETL pipeline development requirements. - Triage issues to find gaps in existing pipelines and fix the issues. - Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs. - Help joiner team members to resolve issues and technical challenges. - Drive technical discussion with...

Experience : 8-15 years. Primary Roles And Responsibilities : - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. - Ability to provide solutions that are forward-thinking in data engineering and analytics space. - Collaborate with DW/BI leads to understand new ETL pipeline development requirements. - Triage issues to find gaps in existing pipelines and fix the issues. - Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs. - Help joiner team members to resolve issues and technical challenges. - Drive technical discussion with client architect and team members. - Orchestrate the data pipelines in schedul...

Job Title: Senior Data Engineer, Microsoft Fabric + SSIS Specialist Location: Noida, Sector 62 ( Onsite/Hybrid, Full-Time, Long-Term Proje ct) Experience Required: 7 + Years Availability: Immediate Joiners Only Role Overview: We are looking for a highly skilled Senior Data Engineer 7+ years of experience as a Microsoft Fabric Specialist to lead the design, development, and optimization of modern data solutions. The role involves working extensively with Azure Fabric, ADF, Databricks, SSIS, Power BI, and PySpark to build scalable, high-performance data pipelines and analytics systems. Key Responsibilities: Lead the design, development, and implementation of Fabric Pipelines, SSIS Packages, Da...

**Job Title: Lead Data Engineer - Data** **Company: Happiest Minds** **Location:** Bangalore **Job Type:** Full-Time **Experience Required:** 5 - 8 years **About Happiest Minds:** Happiest Minds is a next-generation digital transformation and technology services company that enables businesses to achieve their vision by delivering innovative solutions with a human touch. Our mission is to leverage the latest technologies to help our clients stay ahead in the digital era. **Job Description:** We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team at Happiest Minds. The ideal candidate will have a strong background in data engineering with expertise in data...

About The Role : We are seeking a highly skilled and experienced Senior PySpark/Python Developer to play a critical role in building a robust and reliable system for managing and disseminating customer notifications regarding PG&E's Planned Power Outages (PPOs). This is an exciting opportunity to tackle complex data challenges within a dynamic environment and contribute directly to improving customer communication and experience. As a key member of the team, you will be responsible for developing and implementing data processing pipelines that can ingest, transform, and synthesize information from various backend systems related to PPOs. Your primary goal will be to "eat ambiguity and excret...

Quantitative Developer : A Quantitative Developer, often called a "Quant Dev," is a highly specialized professional who works at the intersection of finance, mathematics, and computer science. This role involves designing, implementing, and optimizing algorithms and software solutions for complex financial problems, such as trading strategies, risk management, and pricing models. A successful Quant Dev combines deep programming expertise with a strong grasp of quantitative finance principles. Key Responsibilities : - Develop, implement, and maintain complex quantitative models and algorithms for high-frequency trading, risk management, or portfolio optimization. - Write high-performance, low...

Welcome to Warner Bros. Discovery the stuff dreams are made of. Who We Are When we say, the stuff dreams are made of, we’re not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD’s vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life, the creators bringing them to your living rooms and the dreamers creating what’s next From brilliant creatives, to technology trailblazers, across the globe, WBD offers career defining opportunities, thoughtfully curated benefits, and the tools to explore and grow into your best selves. Here you are supported, here you are celebrated, here you c...