1586 Data Pipeline Jobs - Page 37

As a Data Engineer at our organization, you will play a crucial role in designing, implementing, and maintaining data pipelines to support our data infrastructure and analytics initiatives. Collaborating closely with data scientists, analysts, and stakeholders, you will ensure a seamless flow of data across various systems and platforms. Your responsibilities will include proficiency in Python programming for data manipulation and pipeline development, a strong understanding of SQL and relational databases for querying and manipulation, as well as experience with version control systems like Git for code management and collaboration. Additionally, working knowledge of at least one cloud plat...

As a DevOps Engineer at Cisco Cloud Security Engineering, you will be an integral part of the dynamic software development team. Your role will involve automating and optimizing the software delivery process, managing the organization's cloud infrastructure, troubleshooting system issues, collaborating on deployment strategies, and ensuring a seamless transition from development to production. You will also have the opportunity to learn and adapt to new technologies in the DevOps landscape. Key responsibilities include supporting the development and operations teams, monitoring system health and security, troubleshooting across various domains, participating in deployment strategies, and cre...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and a commitment to creating value for clients. We serve leading enterprises worldwide, leveraging our expertise in digital operations, data, technology, and AI. We are seeking a Lead Consultant- Databricks Developer to solve cutting-edge problems and meet functional and non-functional requirements. As a Databricks Developer, you will work closely with architects and lead engineers to design solutions and stay abreast of industry trends and standards. Responsibilities: - Stay updated on new technologies for potential application in service offerings. - Coll...

About us: Vertiv brings together hardware, software, analytics, and ongoing services to ensure its customers" vital applications run continuously, perform optimally, and scale with business needs. Vertiv solves the most important challenges facing today's data centers, communication networks, and commercial and industrial facilities with a portfolio of power, cooling, and IT infrastructure solutions and services that extends from the cloud to the edge of the network. Job Summary: As an AI Developer at Vertiv, you will be responsible for designing, developing, and implementing AI models and algorithms to address complex business challenges. You will work with large datasets, collaborate with ...

Required Skills & Experience 6-9 years of experience in Consulting, Data, and Analytics , with a strong background in Databricks solutions . Hands-on experience in designing and implementing big data solutions , including data pipelines for large and complex datasets. Expertise in Python, PySpark , and orchestration frameworks such as Airflow, Oozie, Luigi , etc. Strong understanding of Databricks workspace and its components (workspace, compute, clusters, jobs, Unity Catalog, UC permissions). Ability to configure, schedule, and manage Databricks clusters and jobs , including performance optimization and monitoring. Experience with data modeling , SQL queries, joins, stored procedures, relat...

We are seeking a motivated Big Data Engineer with 2-4 years of hands-on experience to join our dynamic data engineering team You will design, develop, and optimize scalable data pipelines, enabling advanced analytics and data-driven decision making If you are passionate about big data technologies and thrive in a collaborative environment, we'd love to meet you Key Responsibilities: Design, build, and maintain scalable and reliable data pipelines using big data technologies Bachelors degree in Computer Science, Information Technology, or related field 2-4 years of experience as a Big Data Engineer or in a similar data engineering role Hands-on experience with big data frameworks such as Hado...

Location : Remote Duration : Full-time Experience : 8+ years About The Role : We are seeking a highly skilled and experienced Senior PySpark/Python Developer to play a critical role in building a robust and reliable system for managing and disseminating customer notifications regarding PG&E's Planned Power Outages (PPOs). This is an exciting opportunity to tackle complex data challenges within a dynamic environment and contribute directly to improving customer communication and experience. As a key member of the team, you will be responsible for developing and implementing data processing pipelines that can ingest, transform, and synthesize information from various backend systems related to...

Duration : 6 Months Timings : General IST Notice Period : within 15 days or immediate joiner About The Role : As a Data Engineer for the Data Science team, you will play a pivotal role in enriching and maintaining the organization's central repository of datasets. This repository serves as the backbone for advanced data analytics and machine learning applications, enabling actionable insights from financial and market data. You will work closely with cross-functional teams to design and implement robust ETL pipelines that automate data updates and ensure accessibility across the organization. This is a critical role requiring technical expertise in building scalable data pipelines, ensuring ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

About the Role : Join TA Digital (Credera), a global consulting firm delivering AI-driven customer experiences. As a Senior Salesforce CPQ Developer , you will design, develop, and implement advanced CPQ solutions tailored to client needs, driving efficiency across product configuration, pricing, and quoting processes. Key Responsibilities: Collaborate with product specialists and the Development Manager to understand complex requirements and translate them into Salesforce solutions Design and develop Salesforce CPQ solutions to streamline the sales process Customize Salesforce CPQ to meet business requirements, including product configuration, pricing, and quoting Collaborate with business ...

Job Purpose This role is central to managing the airlines operational data, building scalable data pipelines, enabling organization-wide reporting, and driving automation using the Microsoft ecosystem. Key Accountabilities Functional Activities Leading a team of experienced data engineers and analyst Design and maintain end-to-end data pipelines using SQL, Microsoft Fabric, and Python to support operations and planning analytics. Develop and manage centralized databases and reusable data models for seamless integration across Power BI, Power Apps, and other analytics platforms. Build and optimize Power BI dashboards and data models, implementing Row-Level Security (RLS), performance tuning, ...

We have urgent requirement for Data Engineer position. Must be able to join in 15 to 20 days . Must relocate to bangalore if shortlisted. Required skills- Must have experience in projects related to banking domain & loans and NPA or regulatory reporting. Strong experience in Oracle SQL / PL-SQL. Proficient in PySpark for handling big data processing. Deep understanding of data warehouse concepts (star/snowflake schema, facts & dimensions, SCDs etc.)

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Snowflake as the primary database engine. Collaborate with cross-functional teams to gather requirements and design solutions that meet business needs. Develop complex SQL queries to extract insights from large datasets stored in Snowflake tables. Troubleshoot issues related to data quality, performance tuning, and security compliance. Participate in code reviews to ensure adherence to coding standards and best practices. Desired Candidate Profile 3-7 years of experience working with Snowflake as a Data Engineer or similar role. Strong understanding of SQL programming language with ability...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for int...

Greetings from Global Healthcare Billing Partners Private Limited..! Job Title: Python Developer - Desktop Application Location: Velachery, Chennai Experience: 1 to 3 Years About the Role: We are seeking a highly skilled Python Desktop Developer with strong expertise in Desktop Application Development . The ideal candidate should have hands on experience in building scalable applications, writing efficient code, and collaborating with cross functional teams to deliver innovative solutions for the healthcare domain. Key Responsibilities: Good Knowledge & expertise Python: Strong expertise in Core Python with clean, efficient coding practices. Hands-on experience in Desktop application framewo...

If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for you! We at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. Resourcefulness is a necessary skill in this role. If you truly love gaining new technical knowledge and can add more awesomeness to the team, you are eligible! Experience : 5 to 8 years. Location : Vadodara & Pune...

Salary: 25 to 40 LPA Exp: 5 to 10 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Da...

Job Title - Snowflake Data Engineer Key Skills - Snowflake , Snowpipe , Data Warehousing , Advance SQL Job Description: Design, develop, and maintain robust ELT/ETL data pipelines to load structured and semi-structured data into Snowflake. Implement data ingestion workflows using tools like Azure Data Factory, Informatica, DBT, or custom Python/SQL scripts. Write and optimize complex SQL queries, stored procedures, views, and UDFs within Snowflake. Use Snowpipe for continuous data ingestion and manage tasks, streams, and file formats for near real-time processing Optimize query performance using techniques like clustering keys, result caching, materialized views, and pruning strategies. Moni...

The Data Engineer should have strong data infrastructure and data architecture skills, a proven track record of administration for data engineering teams, strong operational skills to drive efficiency and speed, strong project management skills, and a core value base for how analytics can proactively improve organizations. You will be part of the Data & Analytics team and will partner closely with a team of data scientists, business analysts & business intelligence leading a cloud-based strategy for our clients. You'll work on our Azure based data pipeline projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights us...

Only immediate joiner preferred. Experience: 8-12 years Responsibilities: Design, develop & maintain ETL pipelines using Python, PySpark & SQL on AWS/Azure/GCP. Collaborate with cross-functional teams for data transformation & quality assurance.

We are seeking Data Engineers (GCP) who can develop and maintain a high-performance streaming data platform for a leading healthcare CRM product. You will work across technologies like Python, PySpark, SQL, Continuous Integration/ Continuous Development, Kafka, GCP - BigQuery, Google cloud Compute, Looker, Data Modelling. Required Skill Set: Programming & Data Engineering: Python, PySpark, SQL, CICD Streaming & Orchestration: Kafka, GCP Composer/Airflow Cloud Tools: Google Cloud Platform (Dataproc, BigQuery, Looker) Data Architecture: Data Warehousing, Data Modeling Location : Manyata Tech Park, Bengaluru Roles and Responsibilities Responsibilities: Design, develop, and maintain scalable dat...

Key Responsibilities: Design, develop, and maintain scalable and secure ETL pipelines using PySpark , Python , and SQL . Build and orchestrate data workflows leveraging AWS services such as Glue , Lambda , Step Functions , S3 , Athena , and Redshift . Perform data profiling, cleansing, and transformation to ensure data quality and consistency. Optimize and tune data pipelines for performance and cost-efficiency. Work with data architects, analysts, and stakeholders to understand data requirements and translate them into technical solutions. Develop and maintain technical documentation for all ETL processes and data flows. Monitor data pipelines and troubleshoot any issues related to performa...