1578 Data Pipeline Jobs - Page 30

Experience: 7+ Years Location: Bangalore Key Responsibilities Design, develop, and maintain scalable ETL pipelines using AWS Glue and PySpark . Lead the data engineering team , ensuring best practices in coding, testing, and deployment. Implement workflow orchestration solutions for complex data pipelines. Optimize data pipelines and queries for advanced SQL performance. Collaborate with data scientists, analysts, and stakeholders to deliver reliable data solutions. Ensure data quality, governance, and security across systems. Drive continuous improvements in ETL processes and architecture. Requirements 7+ years of hands-on experience in Data Engineering . Strong expertise in AWS Glue , ETL ...

Requirements and responsibilities We are seeking a Senior ETL Data Engineer to join our data engineering team supporting compliance and risk initiatives. This is a mission-critical role focused on building scalable, high-performance data pipelines that deliver trusted, high-quality data to business and regulatory stakeholders. What youll do Design, build, and maintain robust, scalable ETL processes for data acquisition, transformation, and delivery; Collaborate with Data Analysts, Architects, and Compliance stakeholders to translate requirements into efficient ETL solutions; Ensure data quality, integrity, lineage, and traceability across large-scale datasets; Optimize ETL workflows for perf...

Key Responsibilities: • Architect, develop, and scale advanced Conversational AI Systems powered by LLMs , RAG , and Agentic AI workflows • Implement Retrieval-Augmented Generation (RAG) pipelines using tools like LangChain , LangGraph , LlamaIndex , and custom retrieval stacks • Engineer agentic behaviors with tools like AutoGen , CrewAI , LangGraph , or custom frameworks for multi-step reasoning and tool usage • Apply prompt engineering , fine-tuning , and LoRA/QLoRA techniques to customize LLMs (e.g., OpenAI GPT-4/5, Mistral, Claude, LLaMA) • Design pipelines to minimize hallucination and toxic output , ensuring factual accuracy, safety, and alignment • Integrate structured (e.g., MongoDB...

Job Description JD Dir/Sr Manager Product Lead Role for Bodhi The Role: As the Dir/Sr Manager Product, you will be a key driver in developing our Agentic AI product suite Collaborating closely with cross-functional teams, you will ensure Director/Senior Manager, Product Agentic AI Are you passionate about AI, data, and building cutting-edge products that drive real impactWere looking for a Director/Senior Manager of Product to lead the development of our Agentic AI product suite This is a unique opportunity to work at the intersection of AI, data, and enterprise solutions, shaping products that enable exponential business outcomes In this role, you will collaborate closely with cross-functio...

This role will be based in Bangalore, India. At LinkedIn, we trust each other to do our best work where it works best for us and our teams. This role offers a hybrid work option, meaning you can both work from home and commute to a LinkedIn office, depending on whats best for you and when it is important for your team to be together. Responsibilities: - Deliver impact by driving innovation while building and shipping software at scale - Provide architectural guidance and mentorship to up-level the engineering organization - Actively improve the level of craftsmanship at LinkedIn by developing best practices and defining best strategies - Design products/services/tools and code that can be us...

This role will be based in Bangalore, India. At LinkedIn, we trust each other to do our best work where it works best for us and our teams. This role offers a hybrid work option, meaning you can both work from home and commute to a LinkedIn office, depending on whats best for you and when it is important for your team to be together. Responsibilities: - Deliver impact by driving innovation while building and shipping software at scale - Provide architectural guidance and mentorship to up-level the engineering organization - Actively improve the level of craftsmanship at LinkedIn by developing best practices and defining best strategies - Design products/services/tools and code that can be us...

This role will be based in Bangalore, India. At LinkedIn, we trust each other to do our best work where it works best for us and our teams. This role offers a hybrid work option, meaning you can both work from home and commute to a LinkedIn office, depending on whats best for you and when it is important for your team to be together. Responsibilities: - Deliver impact by driving innovation while building and shipping software at scale - Provide architectural guidance and mentorship to up-level the engineering organization - Actively improve the level of craftsmanship at LinkedIn by developing best practices and defining best strategies - Design products/services/tools and code that can be us...

This role will be based in Bangalore, India. At LinkedIn, we trust each other to do our best work where it works best for us and our teams. This role offers a hybrid work option, meaning you can both work from home and commute to a LinkedIn office, depending on whats best for you and when it is important for your team to be together. Responsibilities: - Deliver impact by driving innovation while building and shipping software at scale - Provide architectural guidance and mentorship to up-level the engineering organization - Actively improve the level of craftsmanship at LinkedIn by developing best practices and defining best strategies - Design products/services/tools and code that can be us...

Lead Data Engineer Data Management Job description Company Overview Accordion is a global private equity-focused financial consulting firm specializing in driving value creation through services rooted in Data & Analytics and powered by technology. Accordion works at the intersection of Private Equity sponsors and portfolio companies' management teams across every stage of the investment lifecycle. We provide hands-on, execution-oriented support, driving value through the office of the CFO by building data and analytics capabilities and identifying and implementing strategic work, rooted in data and analytics. Accordion is headquartered in New York City with 10 offices worldwide. Join us and...

Design, develop, optimize Azure SQL databases for analytics applications Build maintain ETL/ELT pipelines using Azure Data Factory Azure Databricks for big data processing Leverage Azure data services Synapse Analytics, Cosmos DB, Data Lake Storage Required Candidate profile Implement maintain data warehousing solutions, including data modeling dimensional modeling Ensure data quality, security governance across cloud data platforms Collaborate with cross-functional teams

Project Overview: Offshore Bangalore team is currently engaged in developing real-time ingestion pipelines , pulling timeseries and event frame data from 34+ diverse data sources . This complex data ecosystem requires deep validation strategies, robust automation, and tight collaboration with the Tredence QA team member to ensure high-quality delivery. Key Responsibilities: Data Validation & Quality Strategy: Lead validation of real-time series and event frame data from multiple data sources. Collaborate with Tredence QA to define and implement a robust data QA strategy tailored for real-time ingestion pipelines. Ensure accuracy, completeness, and timeliness of data across the pipeline. Appl...

Job description 12+ years' of relevant experience, with 3+ years with Snowflake. Snowflake Solution Architect: You are an expert with hands-on solution design and development experience You will be an influencer and thought leader with in-depth technical expertise, credibility, and field experience to establish yourself as a subject-matter expert in a company that is leading innovation within the integration services industry You have a passion for continuous learning You enjoy mentoring and developing the skills of those around you. Job Responsibilities: Lead and Architect migration of data analytics environment to Snowflake with performance and reliability. Assess and understand the ETL jo...

Job Title: Founding Backend Engineer Location: Remote Type: Full-time | Founding Team | Schedule: Evening Shift Regular Shift Why Join Now? This is a rare opportunity to: Join as a founding engineer and shape the architecture, product, and engineering culture from day one. Work on meaningful, high-impact problems that affect millions of people. Help build a modern backend from scratch with clarity, scale, and resilience in mind. Be part of a mission-driven team operating at the intersection of technology, finance, and public policy. What You'll Do As the Founding Backend Engineer, you will: Own and build core backend services and APIs design, implement, and maintain secure, scalable endpoint...

We are looking for a motivated and enthusiastic Trainee Data Engineer to join our Engineering team. This is an excellent opportunity for recent graduates to start their career in data engineering, work with modern technologies, and learn from experienced professionals. The candidate should be eager to learn, curious about data, and willing to contribute to building scalable and reliable data systems. Responsibilities: Understand and align with the values and vision of the organization. Adhere to all company policies and procedures. Support in developing and maintaining data pipelines under supervision. Assist in handling data ingestion, processing, and storage tasks. Learn and contribute to ...

Dear All!!!! Job Title:- Senior Database Developer Mandatory Skills:- 1) Database Develope r , Azure SQL Managed Instance , Stored Procedures, Triggers, Scheduling, Data Migration , Etl Developer, SQL Server, Data Pipeline Must have Microsoft Certified: Azure Data Fundamentals(DP 900) or Azure Database Administrator Associate (DP-300) or equivalent. ****************All Mandatory Skills should be in the projects.******************* Job type:- Contract Work Mode:- Hybrid Interview Mode:- L1 Virtual, --L2 F2F Interview Shift timings:-General N.p:- Immediate to 15 Days Experience:- 6 Years Budget:-19 LPA Location:- Bangalore If Any one are interested Drop me your Cv hemanandana.a@pacific.co.in i...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Responsibilities: Work closely with product managers to understand customer needs and translate those into technical requirements. • Build and maintain scalable data management and serving infrastructure on cloud, managing terabytes of data and integrating cost. • Design a comprehensive target data architecture vision working closely with product, design and development and NatCo teams to plan transition towards the target vision • Responsible for implementation of architecture concepts in product development and services • Consult and engage with engineering teams to solve problems at their root cause • Create deliverables that provide exceptional clarity and connections to business objecti...

Location - Hyderabad and Gurgaon only. EXP - 6-8 YRS NP - Immediate - 15 Days JD- Job Title: Senior Data Analyst We are looking for a highly skilled and detail-oriented Senior Data Analyst with a strong analytical mindset. This crucial role requires advanced SQL to transform raw data into actionable insights that support strategic decision-making. A deep understanding of business operations is essential. Key Responsibilities: Data Analysis & SQL Proficiency: Collect, clean, and analyze datasets for trends; translate business needs into data solutions; conduct root cause analysis; optimize complex SQL queries; Set up data pipelines and ETL. Cross-functional Collaboration: Collaborate with pro...

About ProcDNA ProcDNA is a global consulting firm. We fuse design thinking with cutting-edge technology to create game-changing Commercial Analytics and Technology solutions for our clients. We're a passionate team of 300+ across 6 offices, all growing and learning together since our launch during the pandemic. Here, you won't be stuck in a cubicle - you'll be out in the open water, shaping the future with brilliant minds. At ProcDNA, innovation isn't just encouraged; it's ingrained in our DNA. Ready to join our epic growth journey? What we are looking for Youll be driving the adoption of the latest technologies in our solutions, bringing in thought leadership to guide clients on complex dat...

World Wide Technology Holding Co, LLC (WWT) is a privately held global technology integrator and supply chain solutions provider. Through our culture of innovation, we inspire, build and deliver business results, from idea to outcome. Based in St. Louis, WWT works closely with industry leaders such as Cisco, VMware, NetApp, Dell EMC, HPE and several others, but also include emerging tech players. WWT employs more than 10,000 people and operates in more than 2 million square feet of state-of-the-art warehousing, distribution and integration space strategically located throughout the world. WWT ranks Glassdoors Best Places to Work and has been on Fortune’s 100 Best Companies to Work For list 1...

Exp required 7+ yrs with data governance informatica tool Key Responsibilities: Data Governance Framework Development : Develop, implement, and maintain data governance frameworks, policies, and standards to ensure high-quality, consistent, and secure data across the organization. Collaborate with business units and stakeholders to define and enforce data governance policies, ensuring alignment with business goals and regulatory requirements. Data Quality Management : Define and enforce data quality standards, monitoring key data quality metrics. Identify, analyze, and resolve data quality issues across various data sources and platforms. Work with cross-functional teams to implement data qu...

Role & responsibilities Required skills and technologies - Extensive Expertise in Azure SQL Managed Instance (SQL MI). - Advanced proficiency in SQL programming and DBMS. - Writing Stored Procedures, Functions,triggers,Indexing. - Experience with database migration tools like Azure Data Factory (ADF). - Strong understanding of ETL processes and data pipelines. - Proven experience in database architecture, design, and ER modeling. - Skilled in performance tuning, query optimization, and batch processing. - Knowledge of data security, backup, recovery, and compliance standards. - Familiarity with database monitoring tools and analysis. - Strong scripting skills (PowerShell, T-SQL, or others). ...

Role : Backend Engineer Location : Hyderabad, India (Hybrid Knowledge City) Experience : 7 - 10 years Employment : Full - Time with Crunchyroll Job Description: We are looking for a highly skilled Backend Engineer with strong expertise in Python and cloud platforms. The role involves building scalable backend services, integrating with distributed systems, and supporting data-driven workflows. Key Responsibilities: Design and develop backend services and APIs using Python. Build and optimize applications for performance, scalability, and reliability. Work with cloud platforms (GCP/AWS) to deploy and manage services. Collaborate with cross-functional teams to deliver high-quality, production-...

Job Title : Senior Data Engineer Job Summary A Sr Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support data collection, transformation, and analysis. They work closely with data scientists, analysts, and business stakeholders to ensure data is accessible, reliable, and optimized for performance. Key responsibilities include developing ETL processes, managing data warehouses and lakes, ensuring data quality, and implementing best practices for data governance and security. Overall having experience between 6 to 10 years of experience in IT and minimum 3+ years experience in cloud. Hands-on in Architecture, Design, Delivery...

7+ Years of expertise required in below skillsets: Data Engineering, Python, SQL, PySpark, Cloud (Any), Airflow, DevOps & Orchestration, Agile, Scrum, Data Pipeline Development, Stakeholder Management, Banking/ Telecom Domain Experience.