1644 Data Pipelines Jobs - Page 50

We are looking for a highly skilled and experienced Data Engineer with over 5 years of experience to join our growing data team. The ideal candidate will be proficient in Databricks, Python, PySpark, and Azure, and have hands-on experience with Delta Live Tables. In this role, you will be responsible for developing, maintaining, and optimizing data pipelines and architectures to support advanced analytics and business intelligence initiatives. You will collaborate with cross-functional teams to build robust data infrastructure and enable data-driven decision-making. Key Responsibilities: .Design, develop, and manage scalable and efficient data pipelines using PySpark and Databricks .Build an...

We are seeking an experienced Azure Data Engineer with 36 years of experience for a 6-month remote contract. The candidate will be responsible for developing and supporting IT solutions using technologies like Azure Data Factory, Azure Databricks, Azure Synapse, Python, PySpark, Teradata, and Snowflake. The role involves designing ETL pipelines, developing Databricks notebooks, handling CI/CD pipelines via Azure DevOps, and working on data warehouse modeling and integration. Strong skills in SQL, data lake storage, and deployment/monitoring are required. Prior experience in Power BI and DP-203 certification is a plus. Location: Remote- Bengaluru,Hyderabad,Delhi / NCR,Chennai,Pune,Kolkata,Ahm...

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 7+ years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

Hiring an AWS Data Engineer for a 6-month hybrid contractual role based in Bellandur, Bengaluru. The ideal candidate will have 46 years of experience in data engineering, with strong expertise in AWS services (S3, EC2, RDS, Lambda, EKS), PostgreSQL, Redis, Apache Iceberg, and Graph/Vector Databases. Proficiency in Python or Golang is essential. Responsibilities include designing and optimizing data pipelines on AWS, managing structured and in-memory data, implementing advanced analytics with vector/graph databases, and collaborating with cross-functional teams. Prior experience with CI/CD and containerization (Docker/Kubernetes) is a plus.

The Platform Data Engineer will be responsible for designing and implementing robust data platform architectures, integrating diverse data technologies, and ensuring scalability, reliability, performance, and security across the platform. The role involves setting up and managing infrastructure for data pipelines, storage, and processing, developing internal tools to enhance platform usability, implementing monitoring and observability, collaborating with software engineering teams for seamless integration, and driving capacity planning and cost optimization initiatives.

JOB DESCRIPTION: Designation: Manager Bex Data and Insights Band : 3A Experience: 05 Years + Location: Baddi Qualification: B. Pharma / M. Pharma / B. Tech. / M. Tech. / MBA knowledge of Microsoft applications including Power Automate and Power BI Hands on experience of LEAN Tools. Green Belt Certified It's an Individual Contributor Role, where individual will be Responsible for AI and Advance analytics in Operations, The BEx Data and Insights Manager is responsible for/ helps delivering targeted business impact per initiative in collaboration with key stakeholders. Manage regular business and IT stakeholder interactions to understand needs, get feedback and update on project progress. Condu...

As a part of DataWeave's Product Engineering team, you will play a crucial role in developing and maintaining advanced sales optimization solutions for Digital Commerce businesses, consumer brands, delivery intermediaries, and marketplaces. Our machine learning-enabled SaaS platform caters to over 400 global brands and retailers, providing them with actionable insights through Digital Shelf Analytics and Dynamic Pricing solutions. Your responsibilities will include ensuring the quality of code developed by the team, overseeing the delivery of solutions into production, and mentoring developers while contributing to capacity planning and career development. Collaborating with technical projec...

You are an experienced Quality Engineer with 6 to 8 years of experience (B2 level) and possess a strong skillset in testing analytical systems. Joining the RE&ILS Operations value stream, you will play a crucial role in ensuring the quality and reliability of data solutions within the Operations Value Stream. Your responsibilities include developing and executing comprehensive test plans and test cases for data solutions such as data pipelines, ETL processes, and data warehouses. You will conduct data validation and verification to guarantee data accuracy, completeness, and consistency. It is essential for you to identify, document, and track defects and issues while collaborating with devel...

You will be joining Soothsayer Analytics, a global AI & Data Science consultancy based in Detroit, with a thriving delivery center in Hyderabad. The company specializes in designing and implementing custom Machine Learning solutions encompassing predictive analytics, optimization, NLP, and GenAI to assist leading enterprises in forecasting, automation, and gaining a competitive advantage. As a Senior AI Scientist, your role will involve leading the development of scalable AI/LLM solutions. You will be responsible for designing, fine-tuning, and deploying models such as LLMs and RAG architectures, while ensuring the robustness of data pipelines and MLOps practices. Your key responsibilities w...

You should have at least 5 years of IT experience with a good understanding of analytics tools for effective data analysis. You must be capable of leading teams and have prior experience working in production deployment and production support teams. Proficiency in Big Data tools such as Hadoop, Spark, Apache Beam, Kafka, etc., and object-oriented/object function scripting languages like Python, Java, C++, Scala, etc., is required. Experience with DW tools like BQ, Redshift, Synapse, or Snowflake, as well as ETL and Data Warehousing, is essential. You should have a solid understanding of relational and non-relational databases such as MySQL, MS SQL Server, Postgres, MongoDB, Cassandra, etc., ...

As a Data Engineer, you will play a crucial role in designing and developing business-critical backend systems using stream processors and high-quality data pipelines. Your responsibilities will include assembling large, complex data sets to meet both functional and non-functional business requirements. You will be tasked with building the necessary infrastructure for optimal extraction, transformation, and loading of data from various sources using SQL and Azure "big data" technologies. Additionally, you will be responsible for constructing a cloud-native, real-time stream processing and data lake platform that can scale into the Zettabytes and beyond. Developing a data pipeline to offer ac...

The primary purpose of this role is to lead and manage engineering teams, providing technical guidance, mentorship, and support to ensure the delivery of high-quality software solutions. As a leader, you will drive technical excellence, foster a culture of innovation, and collaborate with cross-functional teams to align technical decisions with business objectives. Your key responsibilities will include leading engineering teams effectively, fostering a collaborative and high-performance culture to achieve project goals and meet organizational objectives. You will oversee timelines, team allocation, risk management, and task prioritization to ensure the successful delivery of solutions withi...

The Senior GCP Data Engineer role at Deutsche Bank in Bangalore, India, offers you the opportunity to be part of an ambitious initiative focused on Sustainable Finance, ESG Risk Mitigation, and Corporate Sustainability. As Climate Change presents new challenges and opportunities, the bank is investing in developing a Sustainability Technology Platform, Sustainability data products, and sustainability applications to support its goals. To contribute to this initiative, we are seeking a highly motivated and experienced Senior GCP Data Engineer who is passionate about Climate Change and eager to leverage their Technology Skillset in Cloud/Hybrid Architecture. In this role, you will be responsib...

Join GlobalLogic and become a valuable part of the team working on a significant software project for a world-class company that provides M2M/IoT 4G/5G modules to industries such as automotive, healthcare, and logistics. Through your engagement, you will contribute to the development of end-user modules" firmware, implement new features, maintain compatibility with the latest telecommunication and industry standards, and perform analysis and estimations of customer requirements. As a Data Engineer at GlobalLogic, you will be responsible for designing, building, and maintaining scalable and efficient ETL pipelines that integrate Guidewire data with data warehouses, data lakes, and other enter...

As a Data Integration Specialist (Talend Developer), you will be responsible for designing, developing, and implementing data integration solutions using Talend and other ETL tools. With a minimum of 5 years of experience in data integration, you will play a key role in ensuring seamless integration of data from various sources, including databases, APIs, and flat files. Your expertise in ETL processes and proficiency in Java will be essential in creating and maintaining ETL workflows and processes to extract, transform, and load data into target systems. In this role, you will be involved in designing and managing data pipelines to support data ingestion, processing, and storage. You will c...

We are seeking a highly skilled and motivated Senior Product Manager specializing in AI Solutions to join our expanding AI team. The ideal candidate will possess a profound understanding of Artificial Intelligence, encompassing both conceptual knowledge and coding expertise, along with a demonstrated track record of successfully implementing AI solutions to address business challenges. Your role will be pivotal in identifying and prioritizing AI-driven use cases, collaborating closely with internal business teams, and leading IT service providers to craft and deliver AI-powered solutions that foster business growth, streamline costs, and enhance processes. In this position, you will be respo...

As a Data Engineer, you will provide support to the Global BI team for Isolation Valves in their migration to Microsoft Fabric. Your primary focus will be on data gathering, modeling, integration, and database design to facilitate efficient data management. Your responsibilities will include developing and optimizing scalable data models to meet analytics and reporting requirements and utilizing Microsoft Fabric and Azure technologies for high-performance data processing. In this role, you will collaborate with cross-functional teams, including data analysts, data scientists, and business collaborators, to understand their data needs and deliver effective solutions. You will leverage Fabric ...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role IT experience with a minimum of 5+ years of experience in creating data warehouses, data lakes, ETL/ELT, data pipelines on cloud. Has data pipeline implementation experience with any of these cloud providers - AWS, Azure, GCP. prefer...

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and build a more sustainable, more inclusive world. Your Role Design, implement, and administer Cribl Stream, Edge, and Search environments for enterprise-scale observability. Deploy, configure, maintain, and optimize Cribl solutions for data streaming, log routing, and telemetry processing. Design and manage data pipelines for logs, metrics, an...

You should have 5-8 years of experience in data analytics and be located in Bengaluru. As a Senior Associate Data Analytics Lead, your responsibilities will include implementing data pipelines, hands-on development experience in Spark and Data Bricks pipeline optimization, working with AWS services, ElasticSearch, SQL knowledge, REST API knowledge, and leading individual projects using AGILE methodologies. You will also lead a team of developers, collaborate with stakeholders, and work on defining the product roadmap. It is essential to be up to date in the data engineering space, deliver high-efficiency software systems, and design innovative solutions to complex problems. Your must-have sk...

As a Software Developer - Tech Tools at Graviton Research Capital LLP, you will be an integral part of the high-frequency trading firm's team focused on creating advanced tools, services, and visualizations for the Tech and Quants departments. Your primary responsibility will involve developing robust services that enhance trading operations, optimize efficiency, and offer real-time insights into our trading data. Your key responsibilities will include designing and managing scalable, low-latency data pipelines for both real-time and historical market data to support analysis and decision-making. You will also build dashboards and alerting systems for monitoring system health and detecting i...

Job description Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

As an Azure Data Engineer at Norconsulting, you will have the opportunity to work with a major Banking organization in India on a long-term freelance basis. Based in Chennai with a Hybrid work model, you will be compensated at a rate of 110 USD per day (approximately 2400 USD per month) for a full-time commitment (8 hours a day, Monday to Friday). To excel in this role, you should possess a Master's degree or equivalent experience along with a minimum of 5 years of experience in Data Science roles. Holding relevant certifications in data engineering, cloud computing, or Databricks would be beneficial. Your proven expertise should include designing and implementing data pipelines on the Datab...

Job description Some careers have more impact than others. If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be. HSBC is one of the largest banking and financial services organisations in the world, with operations in 62 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to join our team in the role of Performance Analytics Senior Associate. Principal responsibilities The candidate will deliver on the implementation of the framework t...

Job Description : ROLE PROFILE Job Title: Martech Solution Architecture - Analytics & AI Lead Location: INDIA or UK (London) Reports To: Head of Martech Solution Architecture & Enterprise Integration Department Marketing CDT Context/Scope: Diageo is the world's leading premium drinks company, a business built on the principles and foundations laid by the giants of our industry. Arthur Guinness, Alexander Walker, and many other dedicated people that followed in their footsteps, cared deeply about the people and businesses they fostered, and the communities in which they operated. Today we stand on the shoulders of these giants and act with the same entrepreneurial spirit. We are a spirited co...