1175 Data Pipeline Jobs - Page 12

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Role: Technical Project Manager Experience: 12 to 16 years Notice Period: Immediate to 30 days ONLY Job Description: Should have 12-15 years of hands-on project management experience and leading projects with in-depth knowledge of SDLC. Should be able to lead multiple projects simultaneously. Should be able to communicate across a diverse set of stakeholders (Business stakeholders, third parties, Suppliers). Excellent customer relationship management & collaboration skills. Ability to challenge Ideas and speak up to raise risks, issues. Experience working with solution architects and in setting up the Project / Product roadmaps involving all the stakeholders. Experience in working with proje...

ABOUT FLUTTER ENTERTAINMENT Flutter Entertainment is the worlds largest sports betting and iGaming operator with 13.9 million Average Monthly Players worldwide and an annual revenue of $14Bn in 2024. We have a portfolio of iconic brands, including Paddy Power, Betfair, FanDuel, PokerStars, Junglee Games and Sportsbet. Flutter Entertainment is listed on both the New York Stock Exchange (NYSE) and the London Stock Exchange (LSE). In 2024, we were recognized in TIMEs 100 Most Influential Companies under the 'Pioneers' categorya testament to our innovation and impact. Our ambition is to transform global gaming and betting to deliver long-term growth and a positive, sustainable future for our sec...

Primary Roles And Responsibilities : - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. - Ability to provide solutions that are forward-thinking in data engineering and analytics space. - Collaborate with DW/BI leads to understand new ETL pipeline development requirements. - Triage issues to find gaps in existing pipelines and fix the issues. - Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs. - Help joiner team members to resolve issues and technical challenges. - Drive technical discussion with client architect and team members. - Orchestrate the data pipelines in scheduler via Airflow. Skills An...

Role & Responsibilities : - Design, develop, and maintain robust AWS-based data pipelines using Python and PySpark. - Implement efficient ETL processes, ensuring data integrity and optimal performance across AWS services (S3, Glue, EMR, Redshift). - Collaborate with cross-functional teams to integrate data engineering solutions within broader business-critical applications. - Troubleshoot and optimize existing data workflows, ensuring high availability, scalability, and security of cloud solutions. - Exercise best practices in coding, version control, and documentation to maintain a high standard of engineering excellence. Skills & Qualifications : Must-Have : 4+ years of hands-on experience...

The Senior Data Engineer will be responsible for designing, building, and managing the data infrastructure and data pipeline processes for the bank. This role involves leading a team of data engineers, working closely with data scientists, analysts, and IT professionals to ensure data is accessible, reliable, and secure. The ideal candidate will have a strong background in data engineering, excellent leadership skills, and a thorough understanding of the banking industrys data requirements. Leadership and Team Management . Lead, mentor, and develop a team of data engineers. Establish best practices for data engineering and ensure team adherence. Coordinate with other IT teams, business units...

Ensure adherence to Security and Compliance policies and procedures Collaborate with various stakeholders and assist in architecting, designing and implement the optimal solutions with best practice and standards Expertise in Ataccama DQ tool. Create robust data quality pipeline using Ataccama DQ and ONE Desktop, ensuring it is ready for analytics and reporting. Be proficient in writing scripting or programming language to handle large volume of data processing. Adopt DevOps methodologies and tools for continuous integration and deployment (CI/CD) and automate to enhance data engineering processes. Experience to work on AWS data lakes to store and process both structured and unstructured dat...

Establish and enforce best practices in data pipeline development, code quality, testing, deployment, and monitoring Ensure data solutions comply with security, privacy, and governance policies Optimize the performance and costefficiency of data processing workflows Drive continuous improvement by evaluating and adopting new technologies and tools within the GCP ecosystem

Build and Maintain Data Architectures: Develop and maintain data pipelines that enable the transfer and processing of durable, complete, and consistent data 1. Design and Implement Data Warehouses and Data Lakes: Manage appropriate data volumes and velocity while adhering to required security measures 1. Develop Processing and Analysis Algorithms: Create algorithms fit for the intended data complexity and volumes 1. Collaborate with Data Scientists: Work closely with data scientists to build and deploy machine learning models 1. Some other highly valued skills may include below: Building and maintaining scalable, reliable data pipelines Ensuring data durability, completeness, and consistency...

Role & responsibilities We are looking for a Senior Data Engineer who is responsible for the design and development of scalable data pipelines and integrations to support continual increases in data volume and complexity. Work with analytics and business teams to understand their needs, create pipelines, improve data models that feed BI and visualization tools. Key Responsibilities: Data Pipeline Development : Design, develop, and maintain robust ETL pipelines to move data from source systems to our data storage platforms. This includes data ingestion, transformation, and data quality checks to ensure accuracy and completeness. Data Modeling : Create and maintain data models and schemas that...

Title: Data Scientist Location: Chennai Job Description: Be a hands on problem solver with consultative approach, who can apply Machine Learning & Deep Learning algorithms to solve business challenges Use the knowledge of wide variety of AI/ML techniques and algorithms to find what combinations of these techniques can best solve the problem Improve Model accuracy to deliver greater business impact Estimate business impact due to deployment of model Work with the domain/customer teams to understand business context , data dictionaries and apply relevant Deep Learning solution for the given business challenge Working with tools and scripts for sufficiently pre-processing the data & feature eng...

Role Overview: As a senior data pipeline engineer at Agoda, you will be working on distributed systems across multiple data centers, managing hundreds of billions of messages daily. Your primary responsibility will be to ensure data quality, integrity, and accuracy. You will have the opportunity to design new scalable systems to handle the increasing volume of data, including auditing and monitoring systems. Additionally, you will be managing projects with a small team, enhancing your ownership and leadership skills. Your role will involve solving complex problems related to managing and analyzing large amounts of data, such as schema registry, real-time data-ingestion, cross data center rep...

Job Description: Agoda is an online travel booking platform that connects travelers worldwide with a vast network of 4.7M hotels, holiday properties, flights, activities, and more. As part of Booking Holdings and with over 7,100 employees from 95+ nationalities in 27 markets, Agoda fosters a diverse and collaborative work environment. The Data department at Agoda plays a critical role in leveraging data to empower decision-making processes across the company. As a Senior Data Pipeline Engineer, you will work on distributed systems, ensuring data quality, integrity, and accuracy in processing hundreds of billions of messages daily. Key Responsibilities: - Build, administer, and scale data pip...

8+ years of experience in application maintenance and support, with a focus on data-driven systems. Experience of managing Cloud Data based applications (preferably Snowflake, Matillion or other ETL tools) especially from Support & Maintenance perspective. Experience with ITSM tools (e.g., ServiceNow, Jira) for tracking Service Requests and enhancements. Performing Root Cause analysis and taking steps to reduce recurring incidents. Collaborate with cross-functional teams including Data Engineers, Data Architects and business analysts to ensure seamless application operations. Manage application upgrades, patches, and configuration changes. Develop and maintain documentation for application a...

Collaborate with client facing teams to understand solution context and contribute in technical requirement gathering and analysis Design and implement technical features leveraging best practices for technology stack being used Work with technical architects on the team to validate design and implementation approach Write production-ready code that is easily testable, understood by other developers and accounts for edge cases and errors Ensure highest quality of deliverables by following architecture/design guidelines, coding best practices, periodic design/code reviews Write unit tests as well as higher level tests to handle expected edge cases and errors gracefully, as well as happy paths...

Design, build, and maintain scalable data pipelines to ingest and process data from multiple sources. Develop and optimize data models to support business intelligence and reporting. Partner with internal teams to define and deliver metrics and KPIs that support decision-making. Ensure data quality, governance, and security across systems. Evaluate and implement modern data tools and practices to enhance scalability and performance.

Location: Mumbai, India Experience: 5+ years Employment Type: Long-term Contract Role & Responsibilities: Design, develop, and maintain backend systems using Python. Build, optimize, and manage databases and data pipelines. Implement ETL workflows and cloud-based solutions on AWS. Monitor system performance, troubleshoot issues, and ensure reliability. Collaborate with Agile/Scrum teams to deliver high-quality software. Preferred Candidate Profile: 5+ years of backend development experience. Strong programming skills in Python and proficiency in SQL/database design. Hands-on experience with AWS cloud services. Knowledge of ETL frameworks, data pipelines, and workflow orchestration. Familiari...

need handson on snowflake , dbt , azure in creating models , pipelines . basic idea on devops

Join Datamatics Global Services Be a Microsoft Fabric Expert! Position: Senior MS Fabrics Developer Location: Bangalore (Bannerghatta) / Mumbai (Andheri East) Experience: 8+ Years Employment Type: Full-time About the Role: Are you passionate about building cutting-edge data platforms on Microsoft Fabric? Datamatics is looking for a Senior MS Fabrics Developer to lead the design, implementation, and optimization of Fabric-centric data solutions. Work with cloud technologies, advanced ETL/ELT processes, and data pipelines to empower business insights. Key Responsibilities: Lead design and implementation of Microsoft Fabric-centric data platforms and warehouses Develop and optimize ETL/ELT proc...

We are seeking an experienced and highly skilled Senior Data Engineer to drive data-driven decision-making and innovation. In this role, you will leverage your expertise in advanced analytics, machine learning, and big data technologies to solve complex business challenges. You will be responsible for designing predictive models, building scalable data pipelines, and uncovering actionable insights from structured and unstructured datasets. Collaborating with cross-functional teams, your work will empower strategic decision-making and foster a data-driven culture across the organization. Role & responsibilities 1 Position Overview: We are seeking an experienced and highly skilled Senior Data ...

So, what’s the role all about? Within Actimize, the AI and Analytics Team is developing the next generation advanced analytical cloud platform that will harness the power of data to provide maximum accuracy for our clients’ Financial Crime programs. As part of the PaaS/SaaS development group, you will be responsible for developing this platform for Actimize cloud-based solutions and to work with cutting edge cloud technologies. How will you make an impact? NICE Actimize is the largest and broadest provider of financial crime, risk and compliance solutions for regional and global financial institutions & has been consistently ranked as number one in the space At NICE Actimize, we recognize th...

*Hiring: Backend Engineer* Location: Hyderabad (Hybrid) Working hours - 9am to 6pm IST Interview - 1 round with Algoscape Systems and 1 round with Client Must Have: • Strong programming in *Python* • Hands-on with *GCP / AWS* Nice to Have: • Data pipelines (*Airflow, Netflix Conductor*) • *Apache Spark/Beam, Kafka* or distributed data frameworks Role Overview As a Backend Engineer, you will be responsible for designing, developing, and optimizing scalable backend services. You will work closely with cross-functional teams to implement efficient data pipelines, integrate with distributed frameworks, and ensure cloud-based solutions are robust and reliable. What Were Looking For A problem-solv...

Full Stack+PowerBI Shift: 2pm - 11pm IST Duration - 32 Weeks(Initial Contract with extension) Start Date - 06th Oct Job Profile: We are looking for an experienced Technical Lead to guide the development team and ensure delivery of high-quality solutions. This role requires a balance of technical expertise, leadership skills, and the ability to translate business requirements into scalable technology solutions. Key Responsibilities: Define architecture and system design; review code and enforce best practices. Lead and mentor a small engineering team; coordinate sprint planning and delivery. Ensure APIs/integrations, performance, reliability and security standards. Collaborate with stakeholde...

Roles & Responsibilities: Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure (on cloud - GCP) for efficient ETL processes from various sources and storage systems. Creating innovative data validation methods and data analysis tools. Ensuring compliance with data governance and security policies. Conducting complex data analysis and presenting results effectively. Continuously exploring opportunities to enhance data quality and reliability. Data Modeling: oCreate and maintain data models that support business requirements and data analytics. oEnsure data structures are optimized for p...

Job Description Kafka/Streaming Data Engineer Overview We are seeking a Kafka/Streaming Data Engineer to join our team and support a new high-volume data ingestion and fulfillment architecture. The role requires strong hands-on expertise in Kafka cluster management, monitoring, and performance tuning, along with the ability to troubleshoot issues, propose solutions, and contribute to automation efforts. Our current architecture based on a custom solution leveraging Ab Initio for CDC, Kafka for event streaming, and Apache Flink for downstream fulfillment into databases. The production environment is being built, and we need a Kafka expert to help fine-tune, monitor, and stabilize the ecosyste...