1166 Data Pipeline Jobs - Page 6

Responsibilities Design, develop, and maintain ETL/ELT pipelines to collect, process, and store large volumes of structured and unstructured data. Build and optimize data models to support analytics, business intelligence, and machine learning.

Role & responsibilities I came across your profile for Databricks Architect role with us and would like to check with you in reference to the below Full Time/Permanent job Opportunity. Please check the details below and let me know if you would be interested in applying your Candidature for the same . Position :: Databricks Architect Location :: Noida Role :: Full-Time Company :: Stefanini Group (https://stefanini.com/en/) Role Overview: A Data Architect designs, implements, and manages scalable data solutions on the IICS and Databricks platform. This role involves collaborating with business stakeholders to develop data pipelines, optimize performance, and ensure best practices in data hand...

As a Senior Data Engineer in the Engineering team: Join us in: Building and optimizing data architectures to handle large-scale data from various sources, including cloud and on-premise systems. Architecting scalable ETL and data workflows to support advanced analytics and machine learning models. Collaborating with cross-functional teams to deliver end-to-end data solutions that drive business insights. Owning and leading projects that focus on data platform modernization, ensuring high performance, security, and scalability. Working on complex data engineering challenges that push the boundaries of cloud data infrastructure. You'll have: A Bachelors or Master’s degree in Computer Science, ...

Role Overview: As a Project Manager/Senior Project Manager at GormalOne LLP, you will play a crucial role in driving data-centric product initiatives for the DairyTech ecosystem. Your focus will be on building scalable data platforms, enabling advanced analytics, and delivering actionable insights through intuitive dashboards and visualization tools to empower decision-making across supply chain, operations, sales, and customer engagement. Key Responsibilities: - Define and own the product strategy for the Data & Analytics platform to support business growth in the DairyTech domain. - Identify data-driven opportunities in collaboration with business stakeholders, ensuring alignment with orga...

Job Purpose and Impact The Professional, Data Engineering job designs, builds and maintains moderately complex data systems that enable data analysis and reporting With limited supervision, this job collaborates to ensure that large sets of data are efficiently processed and made accessible for decision making, Key Accountabilities DATA & ANALYTICAL SOLUTIONS: Develops moderately complex data products and solutions using advanced data engineering and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust, DATA PIPELINES: Maintains and supports the development of streaming and batch data pipelines that facilitate the seamless ingestion of data fr...

Job Purpose and Impact The Senior Professional, Data Engineering job designs, builds and maintains complex data systems that enable data analysis and reporting With minimal supervision, this job ensures that large sets of data are efficiently processed and made accessible for trading decision making, Key Accountabilities DATA INFRASTRUCTURE: Prepares data infrastructure to support the efficient storage and retrieval of data, DATA FORMATS: Examines and resolves appropriate data formats to improve data usability and accessibility across the organization, DATA & ANALYTICAL SOLUTIONS: Develops complex data products and solutions using advanced engineering and cloud based technologies, ensuring t...

What We Expect Data Engineering & Pipeline Management Design, manage, monitor, and maintain data pipelines and ETL processes using PySpark, Azure DataBricks, SQL Server Studio and Azure Data Factory, Ensure data flows are reliable, scalable, and optimized for performance, Validate data processes through automated testing and monitoring, Continuous Integration, Deployment & Reliability Perform deployments using CI/CD pipelines in Azure DevOps, aligned with team practices, Ensure continuous availability, reliability, and scalability of connected systems, Support performance, uptime, disaster recovery, and capacity growth, Operational Support & Collaboration Serve as Level 3 support for digital...

Join General Mills Digital & Technology team shaping the future of data engineering for one of the worlds most loved food companies! We’re looking for experienced Data Engineers (5–8 yrs) skilled in SQL, Python, and Cloud Data Warehousing (GCP preferred) to drive enterprise-scale digital transformation . Location: Pune (Kalyani Nagar) Walk-In Drive: 1st November, Friday Step 1: Complete a 60-min online technical test before Monday to qualify for the interview! What You’ll Work On: Design and build scalable data pipelines & warehouse solutions on Cloud (GCP). Integrate business data for actionable insights across global systems. Collaborate with global teams using Agile methods. You Bring: St...

Job Title: Senior Data Engineer & Power BI Developer AWS Glue, Redshift, Python, PySpark, Power BI Location: Hyderabad (On-site) Experience: 4+ Years Job Type: Full-Time About the Role We are seeking a Senior Data Engineer & Power BI Developer who possesses strong experience in both cloud-based data engineering and business intelligence (BI) development . The ideal candidate will be adept at designing and managing ETL pipelines using AWS technologies and transforming data into actionable insights through Power BI dashboards and reports . This role demands a professional who can work across data integration, modeling, visualization, and performance optimization , collaborating closely with te...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Job Title: Data Engineer Shift: UK Shift (5 days working) Shift Timing: 12:30 PM to 9:30 PM | 1:30 Pm to 10:30 PM Job Location: Ahmedabad (Work from Office) Key Responsibilities: Design, build and maintain scalable data pipelines and transformation processes using Microsoft Fabric components including Data Factory, OneLake, Dataflows and Notebooks. Develop and manage data models and analytics solutions. Develop ETL processes and integrate data from diverse sources (on-cloud and on-premise) into centralised and governed environments using Fabrics capabilities. Ensure data quality, integrity, consistency, security, governance and compliance with industry standards (Uniclass & SFG20) across all...

Capgemini is hiring a Data Engineer to design, build, and maintain data architecture, databases, and ETL processes for a leading U.S. insurance client. The role involves working with SQL/NoSQL databases, optimizing ETL pipelines, and collaborating with analysts and stakeholders to deliver reliable data solutions. Candidates must have strong experience in Informatica , Snowflake , and SQL , along with good communication and analytical skills. Key Skills: AWS Glue Pyspark AWS Lambda Informatica Snowflake SQL

Russell Investments is actively hiring for Data Platform Engineer for Mumbai, Goregaon (E) location. Years of Experience 2+ years of experience required, preferably within the financial services domain (investment bank, asset management, etc.). Role & responsibilities Strong proficiency in Snowflake and related cloud-based platforms like Fivetran , dBT , etc. Hands-on experience with SQL , Python , or other scripting languages. Expertise in data modelling , data pipeline design , ETL tools , and MDM platforms preferably in the financial services/ asset management domain. Architect and administer Snowflake data warehouses , including performance tuning, resource management, and security confi...

(Contractual Role) We are seeking a highly skilled Sr. Associate/ Manager - IICS Developer to join our team in Kochi. The ideal candidate will have experience in Informatica IICS, ETL/ELT, Data Pipelines, Data Integration, SQL, AWS, and Azure. Responsibilities Design, develop, and implement data integration solutions using Informatica IICS. Develop and maintain ETL/ELT processes to extract, transform, and load data from various sources. Create and manage data pipelines to integrate data from multiple sources and systems. Collaborate with cross-functional teams to design and implement data integration solutions. Troubleshoot and resolve issues related to data integration and ETL/ELT processes...

Skilled Data Engineer with 0-1 years of relevant experience in SQL, PySpark, ETL, Data Lakes and Azure Tech Stack. Responsibilities 0-1 years of experience in building data Pipelines with Python/PySpark. 0-1 years of experience in Azure ETL stack ( eg. Blog Storage, Data Lake, Data Factory, Synapse). 0-1 years of experience with SQL. Qualifications Excellent verbal and written communication skills. UG: B.Sc in Any Specialization, BCA in Any Specialization, B.Tech/B.E. in Any Specialization. A good internet connection is a must. Azure certifications will be a bonus

SUMMARY Wissen Technology is Hiring for Senior Data Engineer About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindset ensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia. Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this? Through a combination of deep domain knowledge, cutting-edge...

Were Hiring: Data Engineer | Dataiku + SQL Expert Location: Hyderabad | Full-Time | Hybrid Experience: 5+ years We’re looking for a skilled Data Engineer with strong expertise in Dataiku and SQL to design, build, and optimize data pipelines that power analytics and business insights. Key Responsibilities: Develop and maintain robust data pipelines and workflows using Dataiku . Write and optimize complex SQL queries for data extraction, transformation, and analysis. Collaborate with data scientists and business teams to operationalize models and reporting solutions. Ensure data quality, integrity, and performance across systems. Support integration of multiple data sources and automation of E...

Hiring Senior Data Engineer Required Experience: 5-8 years Job Summary: The Senior Data Engineer will lead the design and development of our data pipelines and architecture. This role is central to building and maintaining a robust data infrastructure that enables analytics and business intelligence within the financial domain. Key Responsibilities: Lead the design, construction, and maintenance of scalable and reliable data pipelines. Build and optimize ETL/ELT processes to move and transform large datasets. Architect and manage data warehouses and data lakes. Ensure data quality and integrity across all data systems. Collaborate with data scientists and analysts to understand their data ne...

Role Overview: We are looking for a skilled Data Engineer with hands-on experience in Dataiku and SQL , preferably within the Pharma or Life Sciences domain . The ideal candidate will design, build, and optimize data pipelines to support analytics, reporting, and advanced data science initiatives that drive business insights and regulatory compliance. Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines using Dataiku for large-scale data processing. Integrate data from various sources (clinical, commercial, manufacturing, R&D systems) into unified data models. Write and optimize complex SQL queries for data transformation, validation, and analytics use cases. Collaborate wit...

We are hiring for one of our client company based in bangalore - Position : Senior Apache Spark Expert Experience : 6+ Years Client location: Bangalore Mode of work : Work from Office Mode of Interview : Virtual Qualification: Bachelors degree in computer science, Engineering, or a related field (preferred) Immediate joiners only Job Description Strong knowledge of Spark architecture, able to set up, manage and admin Spark cluster with Kubernetes, Strong on Scala, build efficient ETL and data pipelines. Experience in building end-to-end flows, orchestrate and optimize. Strong experience database RDBMS, SQL, Snowflake.

Job Summary We are seeking a highly skilled Sr. Developer with 8 to 10 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Azure Data Factory SQL Microsoft Business Intelligence and a background in the Payer domain is advantageous. This role involves developing and optimizing data solutions to enhance business processes and drive efficiency. The position operates during day shifts with no travel required. Responsibilities Develop and implement data solutions using Azure Data Factory SQL Microsoft Business Intelligence to streamline business operations and improve data management. Collaborate with cross-functional teams to gather requirement...

Skill: Pyspark Experience: 6 to 9 years Location: AIA Kochi Responsibilities Develop and maintain scalable data pipelines using Python and PySpark. Collaborate with data engineers and data scientists to understand and fulfill data processing needs. Optimize and troubleshoot existing PySpark applications for performance improvements. Write clean, efficient, and well-documented code following best practices. Participate in design and code reviews. Develop and implement ETL processes to extract, transform, and load data. Ensure data integrity and quality throughout the data lifecycle. Stay current with the latest industry trends and technologies in big data and cloud computing.

Nielsen Sports is looking for Machine Learning Engr (MTS 4) to join our dynamic team and embark on a rewarding career journey. A Machine Learning Engineer is responsible for designing, building, and maintaining systems that use artificial intelligence (AI) and machine learning (ML) algorithms to solve complex problems. 1. Developing and implementing ML algorithms : The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling. 2. Building data pipelines : The Machine Learning Engineer is responsible for building data pipelines that collect, store, and preprocess data u...

Whats On India Media Pvt. Ltd is looking for Machine Learning Engr (MTS 4) to join our dynamic team and embark on a rewarding career journey. A Machine Learning Engineer is responsible for designing, building, and maintaining systems that use artificial intelligence (AI) and machine learning (ML) algorithms to solve complex problems. 1. Developing and implementing ML algorithms : The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling. 2. Building data pipelines : The Machine Learning Engineer is responsible for building data pipelines that collect, store, and pr...

Send resumes to manju@nam-it.com Role : Data Engineer with AWS and Snowflake are must Location : Banashankari, Bengaluru ( All 5 Days Onsite) Duration : FTE Need Immediate Joiners or 2 weeks notice candidates. Skills : Strong hands-on UNIX/Shell scripting experience not just basic commands but comfort working directly on servers, SSHing into clusters, handling file operations, and executing CLI-based workflows. Deep expertise in AWS data services (S3, Glue, Lambda, EC2) integrated with Snowflake or Databricks for data lake development. Solid ETL/ELT design background with Python, PySpark, and SQL for transformation and pipeline orchestration. Familiarity with version control (GitHub) and aut...