749 Data Pipelines Jobs - Page 4

As an experienced professional with 5+ years of overall experience and 3+ years of relevant experience, you will be responsible for the following key responsibilities: - Develop and maintain data pipelines using Azure Databricks. - Write efficient and scalable code in Python. - Work with large datasets and optimize performance. - Collaborate with data scientists and engineers. - Deploy and monitor data solutions in the Azure cloud. The company is currently offering a Contract to Hire Position for individuals with the required expertise. Additionally, you will have the opportunity to participate in a Walk-in Drive consisting of only 1 round of interview, with specific dates allocated for Chen...

Job Title: Senior SQL Developer with Snowflake Exposure Work Location: Hyderabad & Chennai (Hybrid) Company: ConglomerateIT (Deployed at Infosys) Experience: 7+ Years Joiners: Immediate Only!!! About the Opportunity ConglomerateIT is seeking highly skilled SQL Developers with exposure to Snowflake to join our team and work with our esteemed client Infosys . This role offers the best of both worlds the flexibility of hybrid work and the chance to contribute to enterprise-scale data projects . If you are an expert in SQL and eager to grow your career with modern cloud data platforms, this opportunity is for you! Key Roles & Responsibilities Design, develop, and optimize complex SQL queries, st...

Job Role Description Previous experience in a Data Engineer role Hands on Experience with Azure Data Factory (AWS cloud services) especially on Building Data Pipelines Experience with relational SQL and NoSQL databases. Experience with data pipeline and workflow management tools Extensive knowledge of data engineering tools, technologies, and approaches Hands-on experience with ETL tools. Informatica, Datastage etc. BI tools knowledge (PowerBI) Experience with big data tools Excellent analytical and problem-solving skills. Job Location : Remote Working, Can work from anywhere in India with good network connectivity (6 Days working) Experience Level : Expecting 3+ Years with good hands on exp...

Role Summary: The Solution Architect is responsible for designing robust, scalable, and high-performance AI and data-driven systems that align with enterprise goals. This role serves as a critical technical leaderbridging AI/ML, data engineering, ETL, cloud architecture, and application development. The ideal candidate will have deep experience across traditional and generative AI, including Retrieval-Augmented Generation (RAG) and agentic AI systems, along with strong fundamentals in data science, modern cloud platforms, and full-stack integration. Key Responsibilities: Design and own the end-to-end architecture of intelligent systems including data ingestion (ETL/ELT), transformation, stor...

Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL

As a Data Engineer III at JPMorgan Chase within the Consumer and Community Banking - Data Technology, you will be a seasoned member of an agile team, tasked with designing and delivering reliable data collection, storage, access, and analytics solutions that are secure, stable, and scalable. Your responsibilities will include developing, testing, and maintaining essential data pipelines and architectures across diverse technical areas, supporting various business functions to achieve the firm's objectives. - Executes software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches to build solutions or break down techn...

As a HITL Engineer at Phenom, you will play a crucial role in supporting the operation and enhancement of Human-in-the-Loop systems. Your strong analytical and troubleshooting skills will be essential in monitoring workflows, identifying anomalies, debugging integration/ML feedback loops, and collaborating with cross-functional teams. **Key Responsibilities:** - Monitor and analyze HITL pipelines for errors, bottlenecks, and anomalies. - Troubleshoot issues across data ingestion, model inference, human validation, and feedback loops. - Debug failures in integration flows, APIs, or job orchestration within the loop. - Work closely with data engineers, ML engineers, and HITL operators for time...

Role Overview: You will be responsible for conducting thorough data analysis using tools like Excel and SQL to uncover trends, patterns, and anomalies. Additionally, you will apply statistical methods for in-depth interpretation of complex datasets and design data pipelines to support strategic decision-making. Generating business intelligence reports and actionable insights to guide data-driven decision-making will also be part of your role. Key Responsibilities: - Conduct thorough data analysis using Excel and SQL - Apply statistical methods for in-depth interpretation of complex datasets - Design and develop data pipelines and models for strategic decision-making - Generate business intel...

Job Description: You will be joining LeanQubit, a Certified Integrator of Ignition, dedicated to crafting custom Ignition-based solutions for manufacturing, energy, and utility industries. Based in Mumbai, LeanQubit invites potential partners to connect with them. Role Overview: As a Java Team Lead for a hybrid/remote role in Mumbai, you will lead a team of developers, oversee Java-based applications" design and implementation, and ensure best practices are followed. Your responsibilities will include managing microservices architecture, applying design patterns, and driving the software development lifecycle. Key Responsibilities: - Lead the design, development, and deployment of complex so...

Total Experience : 1+ years Mode of Hire: Permanent Required Skills Set (Mandatory): JavaScript/Node.js/Python, Web Crawling (Puppeteer/Playwright), Strong knowledge of web technologies Desired Skills (Good if you have): HTML, Distributed Systems, Data pipelines, Proxy Management, Anti-bot Techniques Job Responsibilities Design scalable web crawling solutions using modern technologies like Puppeteer and Playwright. Develop optimal and cost-effective methods for efficiently crawling diverse websites at scale. Implement robust proxy management and sophisticated anti-bot techniques. Create comprehensive debugging and monitoring systems for crawler performance. Actively contribute to technical d...

Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL Interested candidates can share their resume with the following details to navaneetha@suzva.com Current CTC Expected CTC Location

Your IT Future, Delivered. Associate DevOps Engineer Open to candidates PAN India (Offices in Chennai, Mumbai, Hyderabad, Bengaluru & Indore). With a global team of 6000+ IT professionals, DHL IT Services connects people and keeps the global economy running by continuously innovating and creating sustainable digital solutions. We work beyond global borders and push boundaries across all dimensions of logistics. You can leave your mark shaping the technology backbone of the biggest logistics company of the world. Our offices in Cyberjaya, Prague, and Chennai have earned #GreatPlaceToWork certification, reflecting our commitment to exceptional employee experiences. Digitalization. Simply deliv...

We are looking out for Postgres Lead/ Manager Role. Experience - 10 Plus Work Location - Hyderabad/ Chennai Work Mode -05 days Office Notice Period - Maximum 30 days Responsibilities Strong knowledge of Postgres database systems, data modeling techniques, and SQL proficiency. Strong experience in integrations: Build and maintain integrations with internal and external data sources and APIs. Experienced in handling JDBC-based connections, API calls, and implementing microservices. Implement RESTful APIs and web services for data access and consumption. Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstruct...

The Job Design / Implementation of Data Pipelines for processing large volumes of data Ingesting batch and streaming data from various data sources. Writing complex SQL using any RDBMS (Oracle, PostgreSQL, SQL Server etc.) Data Modeling: Proficiency in creating both normalized and denormalized database schemas. Developing applications in Python. Developing ETL, OLAP based and Analytical Applications. Working experience in Azure / AWS services Working in Databricks, Snowflake or other cloud data platforms. Working in Agile / Scrum methodologies Good knowledge of cloud security concepts and implementation of different types of authentication methods working on Azure DevOps; Create and manage G...

As a DataOps Engineer, your primary responsibility will be designing and maintaining scalable ML model deployment infrastructure using Kubernetes and Docker. You will need to implement CI/CD pipelines for ML workflows, ensure adherence to security best practices, and set up monitoring tools for tracking system health, model performance, and data pipeline issues. Collaboration with cross-functional teams to optimize the end-to-end lifecycle of data products and identify performance bottlenecks and data reliability issues in the ML infrastructure will also be part of your role. Key Responsibilities: - Design and maintain scalable ML model deployment infrastructure using Kubernetes and Docker -...

As a VP - Reporting & Analytics at Barclays, you will spearhead the evolution of the Risk function by leading the Reporting Production Team for Operational Risk and Risk Oversight. You will provide high-quality MI and reporting to support colleagues across Operational Risk & Risk Oversight, enable senior management in risk management decision making, and ensure compliance with external and regulatory reporting commitments. You will work closely with the Reporting Change Team and receive local and functional leadership support from respective leads in India and globally. **Key Responsibilities:** - Provide leadership to the production reporting team - Set day-to-day direction for team members...

About us: At Articul8 AI, we relentlessly pursue excellence and create exceptional AI products that exceed customer expectations. We are a team of dedicated individuals who take pride in our work and strive for greatness in every aspect of our business. Job Description: Articul8 AI is seeking an exceptional Senior/Staff AI Researcher to join us in shaping the future of Generative Artificial Intelligence (GenAI). As a member of our Applied Research team, you will be responsible for conducting cutting-edge research to advance the capabilities of our AI systems. Your role will involve designing, implementing, and evaluating novel approaches to improve our GenAI models, working at the intersecti...

As a Technical Architect at Quadance Technologies Private Limited, your role will involve leading the design and development of scalable, enterprise-level applications and AI/ML solutions. You will be responsible for collaborating with stakeholders to translate requirements into maintainable architectures, implementing end-to-end solutions, providing technical leadership to development teams, and ensuring adherence to architectural standards. Additionally, you will participate in technical discussions with client teams, evaluate emerging technologies, and drive innovation in architecture and solutioning. Your key responsibilities will include: - Lead the architectural design and development ...

Role Overview: You will be working on end-to-end development of features for a major Technology company. The project consists of different teams focusing on various aspects of the search features development process. Specific roles and responsibilities for this profile include: Key Responsibilities: - Extracting data from external websites using Python and other internal applications and ingesting it into the Clients databases - Data Modelling and Schema Creation / Maintenance - Creating and maintaining data pipelines using internal applications, utilizing 5-10% Python - Performing data analysis, Data visualisation, creating SQL reports and dashboards using internal applications - Creating c...

Role Overview: As a Data Engineer at our company, you will play a crucial role in simplifying our organization by developing innovative data-driven solutions through data pipelines, modeling, and ETL design. Your main goal will be to ensure commercial success while prioritizing customer safety and data security. You will be responsible for driving customer value by solving complex business problems and implementing the most suitable tools to build data solutions. Additionally, you will engage with the data engineering community to deliver opportunities and build a scalable data architecture. Key Responsibilities: - Build advanced automation for data engineering pipelines by eliminating manua...

Key Responsibilities Design & implement data models, warehouses, and databases using Microsoft Fabric, Azure Synapse Analytics, and Azure Data Lake. Develop ETL pipelines with SQL Server Integration Services (SSIS), Azure Synapse Pipelines, and Azure Data Factory. Leverage Fabric Lakehouse for Power BI reporting, real-time analytics, and automation. Optimize data integration, governance, security, and performance. Collaborate with cross-functional teams to develop scalable data solutions. Implement Medallion Architecture for efficient data processing. Work in an Agile environment, using DevOps principles for automation & CI/CD. Required Skills Microsoft Fabric & Azure Lakehouse, OneLake, Dat...

As the Tech Lead - Quality Assurance in the Digital Business division at Sony Pictures Networks, your primary responsibility will be to establish and lead the quality engineering and automation framework for the SonyLIV platform's analytics product. This role necessitates a comprehensive understanding of data analytics testing, data pipelines, ETL processes, and data warehousing. **Key Responsibilities:** - Design and implement automation frameworks for front-end and back-end systems using industry-standard tools like Selenium, Appium, Pytest, Android Studio, Xcode, Eclipse, Jenkins, Jira, etc. - Establish quality processes and metrics for applications across platforms - Develop automation d...

Role Overview: At PwC, as a member of the data and analytics engineering team, your focus will be on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. You will play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. Your primary responsibility will be in data engineering, where you will concentrate on designing and building data infrastructure and systems to facilitate efficient data processing and analysis. Your role will involve developing and implementing data pipelines, data integration, and data transformation solutions. Key Responsibilities: - Oversee the...

Role Overview: As a Technical Lead specializing in System Integration, you will play a crucial role in integrating Benchling ELN into Pharma systems. Your expertise in ELN workflows, data interoperability, and system integration best practices will be instrumental in designing and implementing efficient integration solutions. Your responsibilities will include developing API-based integrations, middleware, and data pipelines to facilitate seamless data exchange between Benchling ELN and other lab systems such as LIMS, CDS, and SDMS. Key Responsibilities: - Develop API-based integrations, middleware, and data pipelines for seamless data exchange - Ensure compliance with regulatory standards (...

Role Overview: You have extensive experience in analytics and large-scale data processing across diverse data platforms and tools. Your responsibilities will include managing data storage and transformation across AWS S3, DynamoDB, Postgres, and Delta Tables with efficient schema design and partitioning. You will develop scalable analytics solutions using Athena and automate workflows with proper monitoring and error handling. Ensuring data quality, access control, and compliance through robust validation, logging, and governance practices will be a crucial part of your role. Additionally, you will design and maintain data pipelines using Python, Spark, Delta Lake framework, AWS Step functio...