1578 Data Pipeline Jobs - Page 21

Seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and warehouses. Strong in SQL, Python, ETL, and cloud platforms (AWS/Azure/GCP). Experience with Spark or Airflow preferred. Required Candidate profile Candidate should have strong skills in SQL, Python, ETL, and cloud data platforms. Must handle big data tools, pipelines, and ensure data quality and performance optimization.

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Role & responsibilities Develop and maintain SQL-based analytical models, queries, and dashboards to support marketing performance measurement. Build and automate data pipelines to unify data from advertising platforms, CRM, web analytics, and internal systems. Define, track, and report on marketing KPIs such as CAC, cross-sell rates, conversion rates, funnel drop-offs, and campaign ROI. Partner with marketing and growth teams to analyze funnel performance, identify optimization opportunities, and improve conversion efficiency. Conduct ad-hoc deep-dive analyses to uncover insights into customer acquisition, retention, and engagement trends. Provide clear, actionable insights and recommendati...

Permanent Role with Genpact Location: Genpact, Hyderabad Uppal Office Hybrid Shift Timing- 12:00 PM to 10:00 PM IST Advanced Python Developer / Python Programming Expert (ETL | Cloud | Gen AI) Job Summary: We are seeking an Advanced Python Developer to design and develop scalable, data-driven solutions and contribute to innovative projects. This role is ideal for individuals with a deep understanding of Python programming, ETL processes, and modern data architectures across cloud platforms. Key Responsibilities: Develop and optimize scalable code using Advanced Python Programming . Build and maintain ETL pipelines for extracting, transforming, and loading data. Collaborate on architecture an...

AIA-Pune Job Summary Data Engineer 4 to 9 Years Responsibilities Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation stored procedures etc. Experience writing data Pipelines using Python and PySpark. Relational data model / data schema design experience Understanding of DevOps and CI/CD in the context of Data Engineering Familiarity with working in a scrum environment and supporting scrum ceremonies Certifications Required Azure Pyspark Python

Mission Were looking for an experienced Scrum Master with 57 years of industry experience to lead agile delivery across globally distributed teams focused on business intelligence and analytics. This role is central to enabling high-impact digital solutions that integrate data, drive insight, and support strategic decision-making. Youll work closely with business, IT, and BI teams to deliver scalable data pipelines, intuitive dashboards, and KPI-driven reporting tools. The ideal candidate brings a product mindset, agile fluency, and a passion for unlocking business value through data. Responsibilities Facilitate agile ceremonies and ensure team adherence to Scrum principles Coach teams on ag...

Guarantee that Databricks best practices are applied throughout all projects Your role will involve designing and building robust data pipeline development and deployment of innovative big data and AI applications advanced data engineering techniques Required Candidate profile Expert-level proficiency in Spark Scala, Python, and PySpark. Knowledge of data architecture, including Spark Streaming, Spark SQL Data Bricks expertise Spark Scala, Python and PySpark etc. Perks and benefits Please reply ,share resume at info@whitepepper.in

Role and Responsibilities Design, develop, and maintain ETL pipelines across Zoho applications and SaaS platforms. Implement Deluge scripts to automate workflows and data transformations within Zoho. Configure and manage Zoho Analytics for data modeling, cleansing, dashboards, and BI reporting. Integrate Zoho with external SaaS tools and cloud platforms using Zoho Flow, REST APIs, and third-party connectors (Zapier, Make, etc.). Perform data migration, validation, and quality checks to ensure accuracy and consistency across systems. Collaborate with product, finance, sales, and operations teams to deliver data-driven insights. Ensure data governance, security, and compliance within SaaS and ...

Role Description This is a full-time hybrid role for an Apache Nifi Developer based in Gurugram with some work-from-home options. The Apache Nifi Developer will be responsible for designing, developing, and maintaining data workflows and pipelines. The role includes programming, implementing backend web development solutions, using object-oriented programming (OOP) principles, and collaborating with team members to enhance software solutions. Qualifications Knowledge of Apache Nifi and experience in programming Skills in Back-End Web Development and Software Development Data Pipeline Strong understanding of APACHE NIFI Background in Computer Science Excellent problem-solving and analytical s...

Perform end-to-end ETL testing for data extraction, transformation, loading; write complex SQL queries; prepare and maintain test cases and defect logs; validate data accuracy &integritycoordinate with teams for issue resolution, quality assurance. Required Candidate profile Hands-on experience in ETL Testing and Data Warehouse Testing. Strong knowledge of SQL / PL-SQL. Experience with Informatica Familiarity with JIRA / HP ALM. Contact- 8072363518

Skills- AWS Data Engineer Experience: 9 to 13 years Location: AIA-Pune We are seeking skilled and dynamic Cloud Data Engineers specializing in AWS, Databricks. The ideal candidate will have a strong background in data engineering, with a focus on data ingestion, transformation, and warehousing. They should also possess excellent knowledge of PySpark or Spark, and a proven ability to optimize performance in Spark job executions. Key Responsibilities: - Design, build, and maintain scalable data pipelines for a variety of cloud platforms including AWS. - Implement data ingestion and transformation processes to facilitate efficient data warehousing. - Utilize cloud services to enhance data proce...

Experience : 2+ years Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Office (Mumbai) Placement Type : Full time Permanent Position (*Note: This is a requirement for one of Uplers' client- Shaadi.com) Required skills: Kafka and Redshift Your Superpowers (a.k.a. What You'll Do) Build data pipelines that could carry a whale Stream data in real time like a Netflix show on 5G Clean, organize, and serve data so clean you could eat off it Make insights magically appear (okay, not magically, but with great logic) Write ETL code that makes even your past self say, Whoa Partner with teams across the org and become their data superhero Tools in Your Utility Belt Python (or you talk to snakes...

Experience : 2+ years Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Office (Mumbai) Placement Type : Full time Permanent Position (*Note: This is a requirement for one of Uplers' client- Shaadi.com) Required skills: Kafka and Redshift Your Superpowers (a.k.a. What You'll Do) Build data pipelines that could carry a whale Stream data in real time like a Netflix show on 5G Clean, organize, and serve data so clean you could eat off it Make insights magically appear (okay, not magically, but with great logic) Write ETL code that makes even your past self say, Whoa Partner with teams across the org and become their data superhero Tools in Your Utility Belt Python (or you talk to snakes...

Experience : 2+ years Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Office (Mumbai) Placement Type : Full time Permanent Position (*Note: This is a requirement for one of Uplers' client- Shaadi.com) Required skills: Kafka and Redshift Your Superpowers (a.k.a. What You'll Do) Build data pipelines that could carry a whale Stream data in real time like a Netflix show on 5G Clean, organize, and serve data so clean you could eat off it Make insights magically appear (okay, not magically, but with great logic) Write ETL code that makes even your past self say, Whoa Partner with teams across the org and become their data superhero Tools in Your Utility Belt Python (or you talk to snakes...

Skills- AWS +Pyspark Experience: 4 to 9 years Location: AIA-Pune As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and decision-making. Key Responsibilities : - Data Pipeline Development: Design, implement, and maintain scalable data pipelines using AWS services such as S3, Glue, Lambda, EMR, and Kinesis. - Data Processing and Transformation: Utilize PySpark, Spark, and SQL to perform complex data transformations and aggregations on large datasets within the AWS ecosy...

Skills- Databricks+ Pyspark Experience: 4 to 13 years Location: AIA-Pune We are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets. Key Responsibilities: Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on Databricks. Data Architecture & Engineering: Work on distributed computing solutions, optimize Spark jobs, and build efficient data models. Performance & Cost Optimization: Fine-tune Spark configurations, optimize Databricks clusters, and reduce compute/storage costs. Collaboration: Work closely with Data Scientists, Analysts, and DevOps tea...

Skills- Pyspark Experience: 6 to 9 years Location: AIA-Pune Responsibilities Develop and maintain scalable data pipelines using Python and PySpark. Collaborate with data engineers and data scientists to understand and fulfil data processing needs. Optimize and troubleshoot existing PySpark applications for performance improvements. Write clean, efficient, and well-documented code following best practices. Participate in design and code reviews. Develop and implement ETL processes to extract, transform, and load data. Ensure data integrity and quality throughout the data lifecycle. Stay current with the latest industry trends and technologies in big data and cloud computing.

Skills- Azure Databricks Experience: 9 to 13 years Location: AIA-Pune Responsibilities Design, develop, and deploy data processing solutions using Azure Databricks. Collaborate with data engineers, architects, and business analysts to gather and analyze requirements. Develop, optimize, and manage complex data pipelines and workflows. Ensure data quality, integrity, and security across various data sources. Troubleshoot and resolve performance issues related to Databricks solutions. Document technical specifications and maintain comprehensive project documentation. Provide technical guidance and mentorship to junior developers. Qualifications Minimum of 5 years of experience in data engineeri...

We are seeking a skilled Data Engineer with strong SQL and ETL/ELT experience to join a dynamic team supporting one of the largest insurance providers in the United States. This role involves designing and building data pipelines, transforming raw data into actionable insights, and collaborating with cross-functional teams to deliver high-quality data solutions. Key Responsibilities Design, build, test, and implement scalable data products and pipelines. Write and optimize complex SQL queries for data transformation, validation, and extraction. Collaborate with data analysts, scientists, and business teams to understand data needs. Maintain and enhance existing ETL workflows for performance ...

We are hiring a Data Engineer with strong experience in AWS cloud services, Python, and data pipeline development. The ideal candidate will work on designing scalable data solutions, building ETL pipelines, and collaborating with cross-functional teams to deliver high-quality data products. Key Responsibilities: Design and develop data pipelines using AWS Glue, Lambda, Airflow, and S3. Build dimensional data models and dashboards using Power BI. Implement CI/CD pipelines for automated deployments. Work with APIs, JSON/XML data formats, and data governance practices. Collaborate with data scientists and business teams to deliver insights.

Prescience Decision Solutions - A Movate Company Role: Data Engineer (Operations) Location: Bangalore(Hybrid) Job Description: We are seeking a skilled Ops Data Engineer to maintain robust data infrastructure and pipelines that support our operational analytics and business intelligence needs. Candidates will bridge the gap between data engineering and operations, ensuring reliable, scalable, and efficient data systems that enable data-driven decision making across the organization. Apply : Immediate Joiners/ Serving notice period only - preferred Key Responsibilities: Maintain ETL/ELT pipelines using modern data engineering tools and frameworks Requires On-call support data pipeline health,...

Dear Candidate , Please find the details below: Skills - Data Lead (MS Fabric - Mandatory Skill) Exp - 8 to 12 years Location Pune-Viman Nagar (WFO) Please find the job description below: Design and build data pipelines, lakehouse architectures (Bronze–Silver–Gold) , and semantic models in Microsoft Fabric. Develop data ingestion and transformation workflows using Data Factory, Dataflows Gen2, SQL, and PySpark. Integrate Fabric with Power BI for analytics and reporting, ensuring data quality, governance, and performance optimization. Implement DevOps (CI/CD, Git) practices for Fabric assets and stay updated with Microsoft Fabric roadmap (Cortex AI, Copilot, Real-Time Intelligence). Kindly Sh...

Skills-GCP Data Engineer Experience: 6 to 12 years Location: AIA-Gurgaon Hands-on experience with GCP services, specifically BigQuery, Cloud Storage, and Composer for data pipeline orchestration Proficiency in Databricks platform with PySpark for building and optimizing large-scale ETL/ELT processes Expertise in writing and tuning complex SQL queries for data transformation, aggregation, and reporting on large datasets Experience integrating data from multiple sources such as APIs, cloud storage, and databases into a central data warehouse Familiarity with workflow orchestration tools like Apache Airflow or Cloud Composer for scheduling, monitoring, and managing data jobs Knowledge of versio...

What will you drive? As a BI team player, you will be responsible for overseeing the planning, execution, and delivery of complex and challenging technical projects involved in Business Intelligence, within our organization. Following are your key responsibilities for the role: Data-Driven Decision Making: At CARS24, we thrive on insights derived from data. Your role would involve harnessing data-backed insights to inform decisions across all functions. Whether its refining strategies, optimizing processes, or implementing innovative solutions, your analytical prowess will be instrumental. Core of Business Efforts: The BI team holds a pivotal position, bridging data with business context to ...

Are you ready to design, build, and scale next-gen data solutions? Do you thrive on turning data into decisions and guiding teams toward success? What Were Looking For: 10+ years in Datawarehousing & ETL Hands-on with Azure, Databricks, ADF, Synapse, PySpark/Python Strong in STTM (Source-to-Target Mapping), design & documentation mentor at heart who can guide junior talent Good to Have: Certifications in Azure/Data Engineering Banking exposure in Risk & Regulatory / Commercial / Credit Cards / Retail What You'll Achieve With Us: Architect & optimize cloud-first data ecosystems Solve real-world enterprise-scale data challenges Be the trusted expert in a high-impact role Grow into leadership a...