874 Data Pipeline Jobs - Page 31

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Location : Bangalore, Chennai, Delhi, Pune. Primary Roles And Responsibilities : - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. - Ability to provide solutions that are forward-thinking in data engineering and analytics space. - Collaborate with DW/BI leads to understand new ETL pipeline development requirements. - Triage issues to find gaps in existing pipelines and fix the issues. - Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs. - Help joiner team members to resolve issues and technical challenges. - Drive technical discussion with client architect and team members. - Orchestrate the data p...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

Dear Candidate, We are hiring a Cloud Architect to design and oversee scalable, secure, and cost-efficient cloud solutions. Great for architects who bridge technical vision with business needs. Key Responsibilities: Design cloud-native solutions using AWS, Azure, or GCP Lead cloud migration and transformation projects Define cloud governance, cost control, and security strategies Collaborate with DevOps and engineering teams for implementation Required Skills & Qualifications: Deep expertise in cloud architecture and multi-cloud environments Experience with containers, serverless, and microservices Proficiency in Terraform, CloudFormation, or equivalent Bonus: Cloud certification (AWS/Azure/...

Experience in end-to-end data pipeline development/troubleshooting using Snowflake and Matillion Cloud. 5+ years of experience in DWH, 2-4 years of experience in implementing DWH on Snowflake using Matillion. Design, develop, and maintain ETL processes using Matillion to extract, transform, and load data into Snowflake and Develop and Debug ETL programs primarily using Matillion Cloud. Collaborate with data architects and business stakeholders to understand data requirements and translate them into technical solutions. We seek a skilled technical professional to lead the end-to-end system and architecture design for our application and infrastructure. Data validation & end to end testing of ...

Immediate openings for the position of Senior Data Engineer / Technical Lead for one of the reputed company Mynd Integrated Solutions located in Gurgaon Sector 68 Key Skills: 68 years of experience in data engineering, analytics, or related fields . Hands-on expertise in SQL, Python , and modern ETL frameworks (Airflow, dbt, etc.). Proven experience with cloud data platforms like Snowflake, Redshift, or BigQuery. Strong understanding of data modeling, warehousing, and performance optimization . Familiarity with data governance , compliance frameworks (e.g., ISO 27701, GDPR). Experience in delivering dashboards via Power BI, Tableau, or Looker . Excellent communication and stakeholder managem...

Role: Celonis Data Engineer Skills: Celonis, Celonis EMS, Data Engineer, SQL, PQL, ETL, OCPM Notice Period: 30-45 Days Role & responsibilities : Hands-on experience with Celonis EMS (Execution Management System). Strong SQL skills for data extraction, transformation, and modeling. Proficiency in PQL (Process Query Language) for custom process analytics. Experience integrating Celonis with SAP, Oracle, Salesforce, or other ERP/CRM systems. Knowledge of ETL, data pipelines, and APIs (REST/SOAP). Process Mining & Analytical Skills: Understanding of business process modeling and process optimization techniques. At least one OCPM project experience Ability to analyze event logs and identify bottl...

We're seeking a Data Engineer to help build and maintain scalable data pipelines for Intelligine, our AI content platform. Requires 1–2 years' experience, strong Python/SQL skills, and a passion for clean, efficient code and problem-solving.

Job Title: ML Engineer Location: Bengaluru, KA (Hybrid) Duration: 12 Months (Possible extension) Job Type: Contract Role Job Description: Role Overview: Client is seeking a Machine Learning Engineer to develop, deploy, and optimize ML models for industrial and business applications The ideal candidate will have expertise in machine learning, deep learning, MLOps, and cloud-based AI solutions, working closely with data scientists, software engineers, and domain experts to enhance decision-making and automation within the organization, Required Skills & Experience: Technical Expertise: 3-7 years of experience in machine learning, deep learning, and AI model development, Proficiency in Python, ...

About Responsive Responsive, formerly RFPIO, is the market leader in an emerging new category of SaaS solutions called Strategic Response Management Responsive customers including Google, Microsoft, Blackrock, T Rowe Price, Adobe, Amazon, Visa and Zoom are using Responsive to manage business critical responses to RFPs, RFIs, RFQs, security questionnaires, due diligence questionnaires and other requests for information Responsive has nearly 2,000 customers of all sizes and has been voted ?best in class? by G2 for 13 quarters straight It also has more than 35% of the cloud SaaS leaders as customers, as well as more than 15 of the Fortune 100 Customers have used Responsive to close more than $3...

US Shift- 5 working days. Remote Work. (US Airline Group) Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Strong focus on AWS and PySpark. Knowledge of AWS services, including but not limited to S3, Redshift, Athena, EMR, and Glue. Proficiency in PySpark and related Big Data technologies for ETL processing. Strong SQL skills for data manipulation and querying. Familiarity with data warehousing concepts and dimensional modeling. Experience with data governance, data quality, and data security practices. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills to work effectively with cross-functional...

Data Engineer with Python development experience Experience: 3+ Years Mode: Hybrid (2-3 days/week) Location: Hyderabad Key Responsibilities Develop, test, and deploy data processing pipelines using AWS Serverless technologies such as AWS Lambda, Step Functions, DynamoDB, and S3. Implement ETL processes to transform and process structured and unstructured data eiciently. Collaborate with business analysts and other developers to understand requirements and deliver solutions that meet business needs. Write clean, maintainable, and well-documented code following best practices. Monitor and optimize the performance and cost of serverless applications. Ensure high availability and reliability of ...

DE Architect Objective 1 : Develop and Implement Metadata-Driven Framework for Medallion Architecture Strong in data modeling and pipeline design Experience with metadata driven frameworks and governance practices Strong analytical skills to identify and reduce redundancies Knowledge of Snowflake and Medallion Architecture Objective 2 : Optimize Data Pipeline Performance and Reliability Expertise in data pipeline optimization and performance tuning Experience with Indexing and efficient orchestration techniques Ability to identify and implement cost-saving measures Knowledge of monitoring tools and processes Objective 3 : Enhance data modelling and Reusability Strong Communication and traini...

Role Overview: We are looking BI + data-engineer kind of skills. The main work is to design & build complex Tableau dashboards, set up robust automated data pipelines and backend tables and do this in an optimized manner. There is also LLM integration into the dashboard, which will require some initiative and exploration. Automations + LLM integration would require some Python experience Job Description: Requirements 3+5 Years of experience in a product analytics, data analyst, or analytics product role Hands on experience in advanced SQL, python Advanced Tableau experience (Min 2yrs hands on experience in Tableau mandatory) Experience in LLM integration into the dashboard • Proven ability t...

Dear Candidate, Looking for a Cloud Data Engineer to build cloud-based data pipelines and analytics platforms. Key Responsibilities: Develop ETL workflows using cloud data services. Manage data storage, lakes, and warehouses. Ensure data quality and pipeline reliability. Required Skills & Qualifications: Experience with BigQuery, Redshift, or Azure Synapse. Proficiency in SQL, Python, or Spark. Familiarity with data lake architecture and batch/streaming. Soft Skills: Strong troubleshooting and problem-solving skills. Ability to work independently and in a team. Excellent communication and documentation skills. Note: If interested, please share your updated resume and preferred time for a dis...

Azure DE Primary Responsibilities - Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage. Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure Create data models for analytics purposes Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations Use Azure Data Factory and Databricks to assemble large, complex data sets Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data. Ensure data security and compliance Collaborate with data engineers, and other stakeho...

The role will require a unique blend of strong DataOps technical and design skills to translate business decisions into data requirements. This individual will build a deep understanding of the infrastructure data we use in order to work across the ID&A team and key stakeholders the appropriate data to tell a data story. This includes implementing and maintaining a data architecture that follows data management best practices for ensuring data ingestion, transformation, storage and analytics are handled according to their specific purpose using the appropriate tools: ingestion captures raw data without applying business logic, transformation processes data discretely for auditability, and an...

The IT Architect will design and implement data pipelines, cloud infrastructure, and AI-driven solutions that support business intelligence and analytics. They will collaborate with Data Engineers, Data Scientists, and Cloud teams to ensure seamless integration of technology solutions. Roles and Responsibilities Data Architecture & Engineering : Design and optimize data pipelines, ETL processes, and data lakes for structured and unstructured data. Cloud Infrastructure : Architect cloud-based solutions using platforms like AWS, Azure, or Google Cloud for scalability and security. Machine Learning & AI Integration : Work with Data Scientists to deploy ML models and AI-driven analytics. Big Dat...

About The Role : We are seeking a highly skilled and experienced Senior PySpark/Python Developer to play a critical role in building a robust and reliable system for managing and disseminating customer notifications regarding PG&E's Planned Power Outages (PPOs). This is an exciting opportunity to tackle complex data challenges within a dynamic environment and contribute directly to improving customer communication and experience. As a key member of the team, you will be responsible for developing and implementing data processing pipelines that can ingest, transform, and synthesize information from various backend systems related to PPOs. Your primary goal will be to "eat ambiguity and excret...

Job Summary NetApp is a cloud-led, data-centric software company that helps organizations put data to work in applications that elevate their business. We help organizations unlock the best of cloud technology. As a member of Solutions Integration Engineering you work cross-functionally to define and create engineered solutions /products which would accelerate the field adoption. We work closely with ISV’s and with the startup ecosystem in the Virtualization, Cloud, and AI/ML domains to build solutions that matter for the customers You will work closely with product owner and product lead on the company's current and future strategies related to said domains. Job Requirements • Lead to deliv...

Position - Pyspark Developer Experience - 5+ yrs Location - Bangalore Notice Period - Immediate - 30 days Roles & Responsibilities: 5+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform. PySpark: Advanced proficiency in PySpark, including working with RDDs, Data Frames, and optimization techniques. Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase. Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala). Big Data Technologies: Familiarity with Hadoop, Kafka, and other d...

Role : Principal ML Ops Architect Responsibilities : 1. Strategic Leadership :a. Define and drive the overall ML Ops strategy and roadmap for the organization, aligning it with business objectives and technical capabilities.b. Oversee the design, development, and implementation of ML Ops platforms, frameworks, and processes.c. Foster a culture of innovation and continuous improvement within the ML Ops team. 2. Technical Architecture :a. Design and implement scalable, reliable, and efficient ML Ops architectures.b. Select and integrate appropriate tools, technologies, and frameworks to support the ML lifecycle.c. Ensure compliance with industry best practices and standards for ML Ops. 3. Team...

Role : Principal ML Ops Architect Responsibilities : 1. Strategic Leadership :a. Define and drive the overall ML Ops strategy and roadmap for the organization, aligning it with business objectives and technical capabilities.b. Oversee the design, development, and implementation of ML Ops platforms, frameworks, and processes.c. Foster a culture of innovation and continuous improvement within the ML Ops team. 2. Technical Architecture :a. Design and implement scalable, reliable, and efficient ML Ops architectures.b. Select and integrate appropriate tools, technologies, and frameworks to support the ML lifecycle.c. Ensure compliance with industry best practices and standards for ML Ops. 3. Team...

Job Title: Senior Data Engineer Overview: We are seeking a highly skilled and experienced Senior Data Engineer to join our data team. This role is pivotal in designing, building, and maintaining robust data infrastructure to support analytics, reporting, and machine learning initiatives. The ideal candidate will have strong technical expertise in data engineering tools and best practices, and a passion for transforming raw data into actionable insights. Responsibilities: Design, develop, and optimize scalable data pipelines to process large volumes of structured and unstructured data. Build and maintain efficient ETL (Extract, Transform, Load) workflows and automate data integration from var...

Role : Principal ML Ops Architect Responsibilities : 1. Strategic Leadership : a. Define and drive the overall ML Ops strategy and roadmap for the organization, aligning it with business objectives and technical capabilities. b. Oversee the design, development, and implementation of ML Ops platforms, frameworks, and processes. c. Foster a culture of innovation and continuous improvement within the ML Ops team. 2. Technical Architecture : a. Design and implement scalable, reliable, and efficient ML Ops architectures. b. Select and integrate appropriate tools, technologies, and frameworks to support the ML lifecycle. c. Ensure compliance with industry best practices and standards for ML Ops. 3...