1175 Data Pipeline Jobs - Page 25

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Lead Data Engineer Data Management Job description Company Overview Accordion works at the intersection of sponsors and management teams throughout every stage of the investment lifecycle, providing hands-on, execution-focused support to elevate data and analytics capabilities. So, what does it mean to work at Accordion? It means joining 1,000+ analytics, data science, finance & technology experts in a high-growth, agile, and entrepreneurial environment while transforming how portfolio companies drive value. It also means making your mark on Accordions futureby embracing a culture rooted in collaboration and a firm-wide commitment to building something great, together. Headquartered in New Y...

We are looking for a motivated and enthusiastic Trainee Data Engineer to join our Engineering team. This is an excellent opportunity for recent graduates to start their career in data engineering, work with modern technologies, and learn from experienced professionals. The candidate should be eager to learn, curious about data, and willing to contribute to building scalable and reliable data systems. Responsibilities: Understand and align with the values and vision of the organization. Adhere to all company policies and procedures. Support in developing and maintaining data pipelines under supervision. Assist in handling data ingestion, processing, and storage tasks. Learn and contribute to ...

About the job : Role : Microsoft Fabric Data Engineer Experience : 6+ years as Azure Data Engineer including at least 1 E2E Implementation in Microsoft Fabric. Responsibilities : - Lead the design and implementation of Microsoft Fabric-centric data platforms and data warehouses. - Develop and optimize ETL/ELT processes within the Microsoft Azure ecosystem, effectively utilizing relevant Fabric solutions. - Ensure data integrity, quality, and governance throughout Microsoft Fabric environment. - Collaborate with stakeholders to translate business needs into actionable data solutions. - Troubleshoot and optimize existing Fabric implementations for enhanced performance. Skills : - Solid foundat...

Greetings from Encora Innovation Labs!(Encora)! Encora is looking for DataOps Engineer with 5-8 years experience in Data Engineer, Spark, AWS services, Python, AWS Glue, Redshift, Lambda and Airflow. Pease find the below detailed job description and the company profile for your better understanding. Position: DataOps Engineer Experience: 5-8 years Job Location: Chennai/Bangalore/Pune/Hyderabad/Noida Position Type: Full time Qualification: Any graduate Technical Skills: AWS services AWS Glue, Redshift, Lambda, IAM, IaC Terraform, CloudFormation, GitHub. Programming - Python, SQL and Spark Data Engineering - Data pipelines, Airflow orchestration Job Summary Responsibilities and Duties • Monito...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Role & responsibilities Job Description Summary The purpose of this role is to oversee the development of our database marketing solutions, using database technologies such as Microsoft SQL Server/Azure, Amazon Redshift, Google BigQuery. The role will be involved in design, specifications, troubleshooting and issue resolution. The ability to communicate to both technical and non-technical audiences is key. Business Title Associate Technical Architect Years of Experience 10+ Years Must have skills 1. Database (one or more of MS SQL Server, Oracle, Cloud SQL, Cloud Spanner, etc.) 2. Data Warehouse (one or more of Big Query, SnowFlake, etc.) 3. ETL tool (two or more of Cloud Data Fusion, Datafl...

About the Role: We are looking for a highly experienced and hands-on Senior Data Analyst (AdTech) to lead our analytics team. This role is ideal for someone with a strong background in log-level data handling , cross-platform data engineering , and a solid command of modern BI tools . You'll play a key role in building scalable data pipelines, leading analytics strategy, and mentoring a team of analysts. Note: Shift Timings: 5PM - 2AM IST Key Responsibilities: Lead and mentor a team of data analysts, ensuring quality delivery and technical upskilling. Design, develop, and maintain scalable ETL/ELT pipelines using GCP tools (BigQuery, Dataflow, Cloud Composer, Cloud Functions, Pub/Sub). Inges...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

UK-based company needed Responsibilities: * Design data solutions using Azure Data Architecture principles. * Collaborate with cross-functional teams on data integration projects. * Ensure data quality through warehousing best practices. Office cab/shuttle Food allowance Provident fund Annual bonus Health insurance

Responsibilities : - Develop and maintain data pipelines using GCP. - Write and optimize queries in BigQuery. - Utilize Python for data processing tasks. - Manage and maintain SQL Server databases. Must-Have Skills : - Experience with Google Cloud Platform (GCP). - Proficiency in BigQuery query writing. - Strong Python programming skills. - Expertise in SQL Server. Good to Have : - Knowledge of MLOps practices. - Experience with Vertex AI. - Background in data science. - Familiarity with any data visualization tool

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Role & responsibilities Data Engineering & Analytics: Strong background in building scalable data pipelines and analytics platforms. Databricks (AWS preferred): Mandatory hands-on expertise in Databricks, including cluster management, notebooks, job orchestration, and optimization. AWS Cloud Services: Proficiency in AWS ecosystem (S3, Glue, EMR, Lambda, Redshift, IAM, CloudWatch). Programming: Expertise in PySpark and Python for ETL, transformations, and analytics. GenAI & LLMs: Experience with Large Language Models (LLM), fine-tuning, and enterprise integration. CI/CD & DevOps Knowledge: Familiarity with Git-based workflows, deployment pipelines, and automation. Preferred candidate profile ...

We are looking for skilled Data Engineer to join our team. This role involves robust ETL/ELT pipelines, developing dashboards using Power BI. Work in AWS & Conduct data profiling, validation & QC. Follow best practices in data architecture. Health insurance Provident fund

About Loadshare: LoadShare is one of India’s most innovative and resilient logistics platforms, transforming how goods move across the country. Founded in 2017, we’ve grown to become a pan-India, multi-category last-mile network with a unique presence in both Tier 1 metros and deep Tier 2/3 towns. What sets us apart: • Scale with depth: We handle ~400K deliveries/day across food, e-commerce (B2C), B2B retail, and digital commerce, powered by a 20,000+ rider network. • Strong financials, stronger ambition: With 430 Cr in FY25 revenue and support from global investors (Tiger Global, Matrix, BII, Stellaris, BeeNext), we have a long runway and a path to profitability • Dual-engine model: We not ...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Primary Roles And Responsibilities : - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. - Ability to provide solutions that are forward-thinking in data engineering and analytics space. - Collaborate with DW/BI leads to understand new ETL pipeline development requirements. - Triage issues to find gaps in existing pipelines and fix the issues. - Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs. - Help joiner team members to resolve issues and technical challenges. - Drive technical discussion with client architect and team members. - Orchestrate the data pipelines in scheduler via Airflow. Skills An...

Build data pipelines using Spark, PySpark, Python. Strong in Data Lake/Warehouse, SQL, EMR, Iceberg, Parquet. Expertise in NoSQL/Relational DBs, data modeling, AWS, APIs, CI/CD (GitLab), Docker, Kubernetes, microservices, and messaging systems.

Hi Everyone, We are hirnig for Immediate to serving candidates only. Share your resume in the given mail id rachel.sabastian@innovation.com

Job Description: Azure Databricks Engineer Position: Azure Databricks Engineer Location: [Specify Location / Remote / Hybrid] Experience Required: 4 10 years (depending on role level) Role Overview We are seeking an experienced Azure Databricks Engineer to design, develop, and optimize data pipelines and analytics solutions on Azure. The ideal candidate will have strong expertise in building scalable big data solutions using Databricks, Spark, and Azure cloud services, with a focus on delivering reliable, secure, and high-performing data platforms. Key Responsibilities Design, build, and optimize scalable ETL/ELT pipelines on Azure Databricks . Work with structured, semi-structured, and unst...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Snowflake Developer DBT Python to design scalable data pipelines, App design optimize SQL queries, and build models using DBT. Must have strong Python skills, experience with Snowflake architecture, and a passion for data engineering best practices

SUMMARY This position is responsible for the design, development, and management of data workflows and pipelines in our Integrated Data and Analytics environment. The role combines technical leadership, team leadership, and hands-on development. RESPONSIBILITIES / TASKS Collect, move, store, and prepare data for analytics enablement. Participate in project planning sessions, working closely with business analysts and team members to analyze requirements and provide design recommendations for complex systems. Design and develop new data pipelines for ingesting high-volume data sets. Develop pipelines using disciplined processes, adhering to industry standards and best practices, with full pro...

Duration : 6 Months Timings : General IST Notice Period : within 15 days or immediate joiner About The Role : As a Data Engineer for the Data Science team, you will play a pivotal role in enriching and maintaining the organization's central repository of datasets. This repository serves as the backbone for advanced data analytics and machine learning applications, enabling actionable insights from financial and market data. You will work closely with cross-functional teams to design and implement robust ETL pipelines that automate data updates and ensure accessibility across the organization. This is a critical role requiring technical expertise in building scalable data pipelines, ensuring ...

Job Title: BI Engineer (Data Pipelines, DBT & Visualization) Experience Required: 35 Years Location: [Specify Location or Remote] Employment Type: Full-time Job Summary We are looking for a highly skilled BI Engineer with strong experience in building data pipelines, working with DBT, and creating interactive dashboards using Power BI or Symphony. The ideal candidate will be responsible for building data infrastructure and converting raw data into meaningful business insights. Key Responsibilities Design, build, and maintain ETL/ELT data pipelines. Develop and manage models using DBT (Data Build Tool). Create dashboards and reports using Power BI or Symphony. Work closely with data analysts,...