867 Data Pipeline Jobs - Page 3

Sr Data Architect with Banking & Financial Services (BFSI) expertise explicitly should have integrated domain-specific skills into both the technical and strategic areas: Role Overview A Senior Data Engineer & Architect specializing in Azure Data Lake who also brings deep banking and financial services (BFSI) experience. This professional design and governs secure, large-scale data platforms while aligning with the unique compliance, risk, and business intelligence needs of financial institutions. Core Technical Skills Azure Data Lake & Storage > Azure Data Lake Storage Gen2 (ADLS Gen2): Hierarchical namespaces, partitioning strategies, performance tuning. > Delta Lake / Lakehouse: ACID tran...

Job Description Job Title:*Lead Data Engineer Data Company:*Happiest Minds Experience:*8 to 11 years Job Description:** Happiest Minds is seeking a talented and experienced Lead Data Engineer to join our dynamic team The successful candidate will be responsible for designing and implementing robust data pipelines and engineering solutions that empower our analytics and business intelligence initiatives As a key member of our data engineering team, you will work closely with data scientists, analysts, and other stakeholders to turn raw data into actionable insights, Key Responsibilities:** Design, develop, and maintain scalable data pipelines using Azure services and Azure Databricks, Collabo...

Job Description Job Title: Software/Data Engineer AWS Role Overview We are seeking a skilled Data Engineer to develop and maintain a high-performance streaming data platform for a leading healthcare CRM product You will work across technologies like Python, SQL, NodeJS, Kafka, and cloud-native services, contributing to scalable, efficient, and robust data solutions, Key Responsibilities Design, develop, and maintain scalable data pipelines and streaming systems, Work with Python, SQL, Scala, NodeJS, and various data stores (relational, NoSQL, Kafka), Analyze healthcare and marketing datasets using SQL, Refactor and optimize existing codebases for performance improvements, Collaborate closely...

Job Description Job Title:*Lead Data Engineer Data Company:*Happiest Minds Location:*Bangalore Job Type:*Full-time Job Description:** Happiest Minds is seeking a highly skilled and experienced Lead Data Engineer to join our innovative team As a Lead Data Engineer, you will play a vital role in designing, developing, and managing data infrastructure and solutions that drive our data strategy You will lead a team of data engineers, ensuring the delivery of high-quality data pipelines and architecture to support our analytics and business initiatives, Key Responsibilities:** Design and build scalable data pipelines and ETL processes to ingest, process, and store large volumes of structured and ...

Job Description Job Title:*Data Engineer Data Company Name:*Happiest Minds Location:*Bangalore Job Type:*Full-time Experience Required:*3 to 5 years About Happiest Minds Happiest Minds Technologies is a next-generation digital transformation and technology services company, delivering delightful experiences to its clients through a blend of technology and domain expertise Join us to create value and make businesses thrive in the ever-evolving digital landscape, Job Summary We are seeking a talented and motivated Data Engineer to join our dynamic data team The ideal candidate will have a strong foundation in data engineering, specifically in Python, MySQL, SQL, and AWS You will be responsible...

About us: We are building a modern, scalable, end-to-end automated on-premises data platform designed to handle complex data workflows, including data ingestion, ETL processes, physics-based calculations and machine learning predictions. Our platform integrates with multiple data sources, edge devices, and storage systems. We are using core Python as programming language, Docker as deployment technology and Dagster as orchestrator. We are a small cross-functional team sharing a wide range of tasks from database operations to data science. We are looking for a data platform developer with expert Python and Docker knowledge who would help to develop, maintain, and optimize our platform. Key Re...

Role: Senior Member of Technical Staff Experience: 4 to 9 years Location: Manyata Tech Park, Bangalore Work mode: Hybrid Skills: Design Data Analytics, Data Engineering, Data Warehouse, Java, SQL, Spark & Spark-SQL, Autonomous Data Warehouse, Object Store, Data Flow/DIS (or similar tech stack), exposure to cloud Short description : Looking for a Software Developer (Data Engineer) who are willing to perform Data Engineering tasks with skills sets like SQL, Spark, Big Data Development, Java & Cloud experience. Job description: Design, develop, troubleshoot and debug software programs for databases, applications etc. Strong knowledge of SQL is required. You should be having programming experien...

Role & responsibilities We are seeking a highly motivated and technically strong Data Engineering Manager to lead our data engineering team and shape the data foundation of our organization. You will be responsible for architecting, building, and scaling robust data pipelines, storage systems, and real-time infrastructure to support analytics, product intelligence, and AI/ML use cases. This is a hands-on leadership role requiring both technical expertise and people management skills to guide a growing team and ensure delivery of high-impact data solutions. Key Responsibilities Lead and mentor a team of data engineers to design, develop, and maintain high-performance data pipelines and platfo...

What you will do: ACV’s Machine Learning (ML) team is looking to grow its MLOps team. Multiple ACV operations and product teams rely on the ML team’s solutions. Current deployments drive opportunities in the marketplace, in operations, and sales, to name a few. As ACV has experienced hyper growth over the past few years, the volume, variety, and velocity of these deployments has grown considerably. Thus, the training, deployment, and monitoring needs of the ML team has grown as we’ve gained traction. MLOps is a critical function to help ourselves continue to deliver value to our partners and our customers. Successful candidates will demonstrate excellent skill and maturity, be self-motivated...

Proven experience as a Data Solutions Architect or in a similar role. Strong expertise in Azure cloud services (Data Factory, Synapse, Databricks, Cosmos DB, etc.). Data modeling, data integration data governance, security, and frameworks. Health insurance Annual bonus

Data Engineer with a focus on GCP. Minimum 1 year hands-on experience with Python, SQL, DataFlow, DataStream, Cloud Functions, Cloud Run, Pub/Sub, BigQuery, Cloud Storage & data ingestion, CDC techniques, and real-time/batch pipeline orchestration .

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Are you ready to embark on an exhilarating journey as a Data Consultant? Join Kyndryl and become a driving force behind the transformative power of data! We're seeking an exceptionally talented individual to accelerate the competitive performance of our customers worldwide, establishing us as their unrivaled business and technology consulting p...

Design, build, and maintain scalable data pipelines for finance-related datasets. Partner with finance stakeholders (product & business teams) to understand requirements and translate them into robust technical solutions. Ensure data accuracy, consistency, and timeliness across reporting layers and downstream systems. Optimize SQL queries and Spark jobs for large-scale datasets to improve performance and cost efficiency. Implement scheduling, orchestration, and monitoring of workflows using Airflow (or similar tools). Collaborate with cross-functional teams (Finance, Product, Analytics, Engineering) to deliver business-critical solutions. Adhere to best practices for coding, testing, and dep...

Develop and deploy machine learning models on Azure cloud platform using Python programming language and Azure ML services. Design and implement scalable and efficient data pipelines for model training and inference. Collaborate with data scientists and business stakeholders to understand requirements and translate them into technical solutions. Implement best practices for ML development, including version control, testing, and documentation. Develop and maintain APIs on Azure cloud platform for model deployment and integration with applications. Monitor and optimize model performance, ensuring scalability, reliability, and security. Participate in code reviews, troubleshooting, and optimiz...

Description: We are seeking a Data Engineer to build and maintain scalable data pipelines that support our data platform. This role will focus on developing ETL processes, integrating data sources, and ensuring data consistency and quality for financial analytics. Requirements: 4+ years of experience in data engineering or similar roles. Strong experience with Snowflake, Kafka, and Debezium. Proficiency in SQL, Python, and ETL frameworks. Experience with data warehousing, data modeling, and pipeline optimization. Strong problem-solving skills and attention to detail. Experience in the financial services or fintech industry is highly desirable. Job Responsibilities: Design, implement, and mai...

We are looking for an experienced Data Engineer/BI Developer with strong hands-on expertise in Microsoft Fabric technologies, including OneLake, Lakehouse, Data Lake, Warehouse, and Real-Time Analytics, along with proven skills in Power BI, Azure Synapse Analytics, and Azure Data Factory (ADF). The ideal candidate should also possess working knowledge of DevOps practices for data engineering and deployment automation. Key Responsibilities: Design and implement scalable data solutions using Microsoft Fabric components: OneLake, Data Lake, Lakehouse, Warehouse, and Real-Time Analytics Build and manage end-to-end data pipelines integrating structured and unstructured data from multiple sources....

You are a highly motivated Product Manager with a strong interest in investment technologies, quantitative research, and project management to support the Quant research and development functions. As an Associate Product Manager on the Quantitative Product team, you will partner with quants, technology, and portfolio managers to deliver advanced portfolio optimization and trade-validation tools. Your role involves owning project plans, user stories, and release tracks end-to-end, ensuring the investment-tech roadmap advances on time and with high quality. The ideal candidate will have a strong project management mindset, experience working with different stakeholders, and a solid understandi...

Exp- 8-15 years 3+ years of hands on experience in Azure Data Engineering development project 2+ years of experience in Databricks development 2+ years of experience in PySpark. 2+ years of experience in SQL Proficient in SQL 2+ years of experience in leading at least a 5+ member team in data engineering delivery Must be fluent in communication Role & responsibilities Preferred candidate profile

Hi all, We are hiring for the role GCP DATA ENGINEER Experience: 5+ Years Location: Pune Notice Period: Immediate - 15 days Budget: 18 LPA only Skills: Ideally you will have a background in software engineering along with experience in working in a complex cloud environment ( Google Cloud Platform) Ideally you will have experience working with Python and SQL. You have experience building data pipelines and ETL frameworks both batch and real time using Python and any of the GCP capabilities such Apache Beam, Data Flow, Data Fusion You have Experience with using Terraform to build Infrastructure-as-a-code Experience working with Big Data Technologies (Spark, Cloud SQL, Big Query ) preferably i...

Duties and Responsibilities: Demonstrate skills in abstract development frameworks. Demonstrate accountability to assigned work & timelines. Demonstrated leadership skills. Demonstrated technical competency for the role. Demonstrated timely delivery on assigned tasks and effective communication on project status and escalation when needed. Education and Experience: 12+ Years or equivalent degree + experience Required Skills/Certifications: Estimating and Planning Work breakdown for the team. Data pipelines Data Transformations Code/Unit Test PR & Code Reviews Performance & Load Testing Functional & Regression Testing Tier 2 support/consulting Production Support Functional ask to Technical de...

Looking For a Better Opportunity? Join Us And Make Things Happen .. DMI is seeking a full-time Data Engineer - Operations to support our manufacturing clients large scale into digital transformation. The Data Engineer - Operations will work within a three to four person team ensuring the day-to-day operations of the data platforms run smoothly. This role will support leadership and will work cross functionally with various IT functions to streamline operations focusing primarily on common ITIL processes (incident, request, problem, and change). Note: We want someone who has hands on experience into data production support or willing to work into support role with a large scale manufacturing ...

Education and Experience: 5+ years of experience in an operational role dealing with data management, data integration or similar activities and/or responsibilities 3+ years of experience in building, mentoring, and leading a team Bachelors Degree and/or relevant field or equivalent experience Required Skills/Certifications: Strong analytical and problem-solving skills - Ability to determine data patterns and perform root cause analysis to resolve production reporting bugs Familiarity with the data, analytics and BI space and knowledge of Power BI Experience working with Microsoft data integration platforms such as Azure Data Factory, SSIS, Synapse Strong experience with SQL (Python experien...

Duties and Responsibilities: Demonstrate skills in abstract development frameworks. Demonstrate accountability to assigned work & timelines. Demonstrated leadership skills. Demonstrated technical competency for the role. Demonstrated timely delivery on assigned tasks and effective communication on project status and escalation when needed. Required Skills/Certifications: Data pipelines Data Transformations Code/Unit Test Performance & Load Testing Functional & Regression Testing Tier 2 support/consulting Production Support Functional ask to Technical design. Data Engineering Agile Methodologies CI/CD Metadata driven frameworks. Technical documentation Azure Synapse Spark Python Complex SQL P...

The Manager, Data Operations is responsible for ensuring the day-to-day operations of the data platforms run smoothly. This role will be the primary interface with the management team and will work cross functionally with various IT functions to streamline operations focusing primarily on common ITIL processes (incident, request, problem, and change). Duties and Responsibilities: Foster a collaborative team environment through mentorship and the development of team members Align team objectives with organizational goals Assist in building the team by identifying top talent and onboarding new team members Monitor and evaluate team performance, provide constructive feedback, and recognize achi...

Minimum Requirements: 5+ years of experience overall in IT Current - 2+ years of experience on Azure Cloud as Data Engineer Current - 1.5+ years of hands-on experience on Databricks/Azure Databricks Proficient in Python/PySpark Proficient in SQL/TSQL Proficient in Data Warehousing Concepts (ETL/ELT, Data Vault Modelling, Dimensional Modelling, SCD, CDC) Primary Skills: Azure Cloud, Databricks, Azure Data Factory, Azure Synapse Analytics, SQL/TSQL, PySpark, Python Technical Expertise: ETL/ELT Pipelines: Design, implementation, and optimization. Azure ADF and Synapse: Proficiency in creating pipelines, developing notebooks, and performing data transformations. Databricks: Hands-on experience w...