867 Data Pipeline Jobs - Page 8

We are looking for a skilled and experienced Data Engineer with experience in building scalable data solutions on the Microsoft Azure ecosystem . The ideal candidate must have strong hands-on experience with Azure Databricks , along with Spark , Python and SQL expertise. Familiarity with Data Lake , Data Warehouse concepts, and end-to-end data pipelines is essential. Key Responsibilities: Lead the design end-to-end data engineering solutions across cloud platforms. Develop and oversee robust data pipelines and ETL workflows using Python and Apache Spark. Implement scalable Delta Lake solutions for structured and semi-structured data. Orchestrate complex workflows using Databricks Workflows o...

Responsibilities: * Design, develop & maintain data pipelines using Snowflake, DBT,SQL, ETL tools. * Collaborate with cross-functional teams on project requirements & deliverables. Office cab/shuttle

Responsibilities: * Collaborate with cross-functional teams on project delivery. * Design, implement & optimize Azure solutions using certified expertise. * Ensure compliance, security & scalability standards met. Food allowance Health insurance Provident fund Annual bonus

2+ Yrs of relevant experience Strong Python programming skills Experience with Generative AI / LLMs Hands-on with LangChain / Hugging Face / OpenAI APIs Knowledge of ML workflows & data pipelines Opportunity to work on cutting-edge AI solutions

2+ Yrs of relevant experience Strong Python programming skills Experience with Generative AI / LLMs Hands-on with LangChain / Hugging Face / OpenAI APIs Knowledge of ML workflows & data pipelines Opportunity to work on cutting-edge AI solutions

About this role: Aurigo is leading the charge in transforming enterprise SaaS for infrastructure, and AI is at the core of this revolution. With Aurigo Lumina, our AI copilot, we are redefining how infrastructure owners interact with Masterworks, driving smarter, faster, and more efficient decision-making. We have set up AI Labs to build AI-powered software and transform Aurigo into an AI-first company. We have AI initiatives across the products and platforms. As Platform and Applied AI Manager based in Bengaluru, you will lead a world-class team to build cutting-edge products/solutions, production grade AI/Gen AI platforms, build AI agents while addressing critical challenges such as AI gov...

As a Data Engineer at our organization, you will play a crucial role in designing, implementing, and maintaining data pipelines to support our data infrastructure and analytics initiatives. Collaborating closely with data scientists, analysts, and stakeholders, you will ensure a seamless flow of data across various systems and platforms. Your responsibilities will include proficiency in Python programming for data manipulation and pipeline development, a strong understanding of SQL and relational databases for querying and manipulation, as well as experience with version control systems like Git for code management and collaboration. Additionally, working knowledge of at least one cloud plat...

As a DevOps Engineer at Cisco Cloud Security Engineering, you will be an integral part of the dynamic software development team. Your role will involve automating and optimizing the software delivery process, managing the organization's cloud infrastructure, troubleshooting system issues, collaborating on deployment strategies, and ensuring a seamless transition from development to production. You will also have the opportunity to learn and adapt to new technologies in the DevOps landscape. Key responsibilities include supporting the development and operations teams, monitoring system health and security, troubleshooting across various domains, participating in deployment strategies, and cre...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and a commitment to creating value for clients. We serve leading enterprises worldwide, leveraging our expertise in digital operations, data, technology, and AI. We are seeking a Lead Consultant- Databricks Developer to solve cutting-edge problems and meet functional and non-functional requirements. As a Databricks Developer, you will work closely with architects and lead engineers to design solutions and stay abreast of industry trends and standards. Responsibilities: - Stay updated on new technologies for potential application in service offerings. - Coll...

About us: Vertiv brings together hardware, software, analytics, and ongoing services to ensure its customers" vital applications run continuously, perform optimally, and scale with business needs. Vertiv solves the most important challenges facing today's data centers, communication networks, and commercial and industrial facilities with a portfolio of power, cooling, and IT infrastructure solutions and services that extends from the cloud to the edge of the network. Job Summary: As an AI Developer at Vertiv, you will be responsible for designing, developing, and implementing AI models and algorithms to address complex business challenges. You will work with large datasets, collaborate with ...

Required Skills & Experience 6-9 years of experience in Consulting, Data, and Analytics , with a strong background in Databricks solutions . Hands-on experience in designing and implementing big data solutions , including data pipelines for large and complex datasets. Expertise in Python, PySpark , and orchestration frameworks such as Airflow, Oozie, Luigi , etc. Strong understanding of Databricks workspace and its components (workspace, compute, clusters, jobs, Unity Catalog, UC permissions). Ability to configure, schedule, and manage Databricks clusters and jobs , including performance optimization and monitoring. Experience with data modeling , SQL queries, joins, stored procedures, relat...

We are seeking a motivated Big Data Engineer with 2-4 years of hands-on experience to join our dynamic data engineering team You will design, develop, and optimize scalable data pipelines, enabling advanced analytics and data-driven decision making If you are passionate about big data technologies and thrive in a collaborative environment, we'd love to meet you Key Responsibilities: Design, build, and maintain scalable and reliable data pipelines using big data technologies Bachelors degree in Computer Science, Information Technology, or related field 2-4 years of experience as a Big Data Engineer or in a similar data engineering role Hands-on experience with big data frameworks such as Hado...

Location : Remote Duration : Full-time Experience : 8+ years About The Role : We are seeking a highly skilled and experienced Senior PySpark/Python Developer to play a critical role in building a robust and reliable system for managing and disseminating customer notifications regarding PG&E's Planned Power Outages (PPOs). This is an exciting opportunity to tackle complex data challenges within a dynamic environment and contribute directly to improving customer communication and experience. As a key member of the team, you will be responsible for developing and implementing data processing pipelines that can ingest, transform, and synthesize information from various backend systems related to...

Duration : 6 Months Timings : General IST Notice Period : within 15 days or immediate joiner About The Role : As a Data Engineer for the Data Science team, you will play a pivotal role in enriching and maintaining the organization's central repository of datasets. This repository serves as the backbone for advanced data analytics and machine learning applications, enabling actionable insights from financial and market data. You will work closely with cross-functional teams to design and implement robust ETL pipelines that automate data updates and ensure accessibility across the organization. This is a critical role requiring technical expertise in building scalable data pipelines, ensuring ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

Job Purpose This role is central to managing the airlines operational data, building scalable data pipelines, enabling organization-wide reporting, and driving automation using the Microsoft ecosystem. Key Accountabilities Functional Activities Leading a team of experienced data engineers and analyst Design and maintain end-to-end data pipelines using SQL, Microsoft Fabric, and Python to support operations and planning analytics. Develop and manage centralized databases and reusable data models for seamless integration across Power BI, Power Apps, and other analytics platforms. Build and optimize Power BI dashboards and data models, implementing Row-Level Security (RLS), performance tuning, ...

We have urgent requirement for Data Engineer position. Must be able to join in 15 to 20 days . Must relocate to bangalore if shortlisted. Required skills- Must have experience in projects related to banking domain & loans and NPA or regulatory reporting. Strong experience in Oracle SQL / PL-SQL. Proficient in PySpark for handling big data processing. Deep understanding of data warehouse concepts (star/snowflake schema, facts & dimensions, SCDs etc.)

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Snowflake as the primary database engine. Collaborate with cross-functional teams to gather requirements and design solutions that meet business needs. Develop complex SQL queries to extract insights from large datasets stored in Snowflake tables. Troubleshoot issues related to data quality, performance tuning, and security compliance. Participate in code reviews to ensure adherence to coding standards and best practices. Desired Candidate Profile 3-7 years of experience working with Snowflake as a Data Engineer or similar role. Strong understanding of SQL programming language with ability...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for int...

Greetings from Global Healthcare Billing Partners Private Limited..! Job Title: Python Developer - Desktop Application Location: Velachery, Chennai Experience: 1 to 3 Years About the Role: We are seeking a highly skilled Python Desktop Developer with strong expertise in Desktop Application Development . The ideal candidate should have hands on experience in building scalable applications, writing efficient code, and collaborating with cross functional teams to deliver innovative solutions for the healthcare domain. Key Responsibilities: Good Knowledge & expertise Python: Strong expertise in Core Python with clean, efficient coding practices. Hands-on experience in Desktop application framewo...

If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for you! We at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. Resourcefulness is a necessary skill in this role. If you truly love gaining new technical knowledge and can add more awesomeness to the team, you are eligible! Experience : 5 to 8 years. Location : Vadodara & Pune...