790 Etl Pipelines Jobs - Page 25

We're Hiring | Senior Data Engineer (Spark + Scala + Databricks) | 6-8 Yrs | Bangalore | Hybrid Location: Bangalore (Hybrid 3 days/week in office) Experience: 6 to 8 years Joiners: Immediate to Max 2 Weeks Notice Period ONLY Key Responsibilities: Design, build, and maintain scalable ETL pipelines using Apache Spark and Databricks Write production-grade code in Scala for large-scale data processing and transformation Implement observability, monitoring, and alerting to ensure system reliability and data quality Set up and manage CI/CD pipelines using tools like GitHub Actions or Azure DevOps Must-Have Skills: 6+ years of experience in Data Engineering and DevOps Deep expertise in Apache Spark...

Title: Sr Azure Data Engineer Experience: 5 to 8 Years Location: Hyderabad Working Hours: 12.00pm to 9.00pm Key Skills: SQL, Azure Data factory, Azure Data Lake, Azure Data bricks, Synapse, Data Fabric Architecture. We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, you'll create pipelines and help to design data warehousing systems. Overall, you'll strive for efficiency by aligning data systems with business goals. To succeed in this data engineering position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also includ...

Greetings! We have permanent opportunities for GCP Data Engineers in Chennai Location . Experience Required : 3 Years and above Location : Chennai (Elcot - Sholinganallur) Work Mode : Hybrid Skill Required: GCP Data Engineer, Advanced SQL, ETL Data pieplines, BigQuery, Dataflow, Bigtable, Data fusion, cloud spanner, python, java, javascript, If interested, kindly share the below details along with updated CV and to Narmadha.baskar @getronics.com Regards, Narmadha Getronics Recruitment team

Looking for a DevOps Senior Engineer in the Data Engineering team who can help us support next-generation Analytics applications over Oracle cloud. This posting is for DevOps Senior Engineer in the Oracle Analytics Warehouse product development organization. Fully handled Cloud service that provides customers a turn-key enterprise warehouse on the cloud for Fusion Applications. The service is being built on a sophisticated technology stack demonstrating a brand-new data integration platform and the industry's most sophisticated analytical business analytics platform. https://www.oracle.com/solutions/business-analytics/analytics-for-applications.html We are looking for senior engineer with ex...

Job Overview As a Senior Backend Engineer at Streamlyn, you will be responsible for designing, building, and scaling backend services and APIs that support our ad-serving infrastructure, analytics pipelines, and internal tools. You will work closely with cross-functional teams, mentor junior developers, and contribute to high-performance, scalable architecture in a fast-paced AdTech environment. Roles and Responsibilities Design and develop robust backend services and APIs using Spring Boot and Java. Build and maintain microservices that support scalable ad-serving and reporting infrastructure. Write unit, integration, and performance tests to ensure software quality and reliability. Work wi...

Capgemini Invent Capgemini Invent is the digital innovation, consulting and transformation brand of the Capgemini Group, a global business line that combines market leading expertise in strategy, technology, data science and creative design, to help CxOs envision and build whats next for their businesses. Your role Develop and maintain data pipelines tailored to Azure environments, ensuring security and compliance with client data standards. Collaborate with cross-functional teams to gather data requirements, translate them into technical specifications, and develop data models. Leverage Python libraries for data handling, enhancing processing efficiency and robustness. Ensure SQL workflows ...

Responsibilities :- Build and manage data infrastructure on AWS , including S3, Glue, Lambda, Open Search, Athena, and CloudWatch using IaaC tool like Terraform Design and implement scalable ETL pipelines with integrated validation and monitoring. Set up data quality frameworks using tools like Great Expectations , integrated with PostgreSQL or AWS Glue jobs. Implement automated validation checks at key points in the data flow: post-ingest, post-transform, and pre-load. Build centralized logging and alerting pipelines (e.g., using CloudWatch Logs, Fluent bit ,SNS, File bit ,Logstash , or third-party tools). Define CI/CD processes for deploying and testing data pipelines (e.g., using Jenkins,...

As a software engineer focused on Marketing and Customer Engagement at GoDaddy, you will have the opportunity to design, build, and maintain a platform that is a keystone to our customer experience, marketing, and business objectives. Everything we do starts with data. Ensure our team continues with a Shift Left” focus on security. This includes the design and development of systems that can contain sensitive customer information. You will partner closely and collaborate with other GoDaddy teams of Engineers, Marketing Professionals, QA and Operations teams. Leverage industry best practices and methodologies such as Agile, Scrum, testing automation and Continuous Integration and Deployment. ...

Must-Have Skills (Core Requirements) Look for resumes that mention hands-on experience with: Amazon S3 storing and organizing data AWS Glue – running ETL jobs (basic PySpark knowledge is a plus) Glue Catalog – maintaining metadata for datasets Amazon Athena – querying data using SQL Parquet or CSV – basic familiarity with data file formats AWS Lambda – for simple automation or triggers Basic IAM knowledge – setting up access permissions CloudWatch – monitoring jobs or logs Understanding of ETL/ELT pipelines Good-to-Have Skills (Preferred but not mandatory) These add value, but are not essential at this level: AWS Lake Formation – access control and permissions Apache Airflow or Step Function...

Contractual (Project-Based) Notice Period: Immediate - 15 Days Fill this form: https://forms.office.com/Pages/ResponsePage.aspx?id=hLjynUM4c0C8vhY4bzh6ZJ5WkWrYFoFOu2ZF3Vr0DXVUQlpCTURUVlJNS0c1VUlPNEI3UVlZUFZMMC4u Resume- shweta.soni@panthsoftech.com

We are seeking a highly skilled ETL/Data Engineer with expertise in Informatica DEI BDM to design and implement robust data pipelines handling medium to large-scale datasets. The role involves building efficient ETL frameworks that support batch .

, Required Candidate profile .

At Allstate, great things happen when our people work together to protect families and their belongings from lifes uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. The Business Intelligence Consultant II is responsible for leveraging data and analytics to answer complex questions and influence business strategy through communication of findings to Stake holders. "The ideal candidate will have a strong background in SQL, Data Modeling, R...

As a Data Engineer at IBM, you'll play a vital role in the development, design of application, provide regular support/guidance to project teams on complex coding, issue resolution and execution. Your primary responsibilities include: Lead the design and construction of new solutions using the latest technologies, always looking to add business value and meet user requirements. Strive for continuous improvements by testing the build solution and working under an agile framework. Discover and implement the latest technologies trends to maximize and build creative solutions Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Ex...

Experience:- 5+ yrs Location:- Remote Budget:- 15 LPA Contact:9916086641 anamika@makevisionsoutsourcing.in

Develop, optimize, and maintain scalable data pipelines using Python and PySpark. Design and implement data processing workflows leveraging GCP services such as: BigQuery Dataflow Cloud Functions Cloud Storage

Project description A DevOps Support Engineer will perform tasks related to data pipeline work and monitor and support related to job execution, data movement, and on-call support. In addition, deployed pipeline implementations will be tested for production validation. Responsibilities Provide production support for 1st tier, after hours and on call support. The candidate will eventually develop into more data engineering within the Network Operations team. The selected resource will learn the Telecommunications domain while also developing data learning skills. Skills Must have ETL pipeline, data engineering, data movement/monitoring Azure Databricks Watchtower Automation tools Testing Nice...

Project description We are looking for the star Python Developer who is not afraid of work and challenges! Gladly becoming a partner with famous financial institution, we are gathering a team of professionals with wide range of skills to successfully deliver business value to the client. Responsibilities Analyse existing SAS DI pipelines and SQL-based transformations. Translate and optimize SAS SQL logic into Python code using frameworks such as Pyspark. Develop and maintain scalable ETL pipelines using Python on AWS EMR. Implement data transformation, cleansing, and aggregation logic to support business requirements. Design modular and reusable code for distributed data processing tasks on ...

Experience:- 5+ yrs Location:- Remote Budget:- 15 LPA Contact:9916086641 anamika@makevisionsoutsourcing.in

Must have: Scala, Spark, Azure databricks, Kubernetes Note: Quantexa certification is a Must. Good to have: Python, Pyspark, Elastic, Restful APIs ROLE PURPOSE The purpose of the Data Engineer is to design, build and unit test data pipelines and jobs for Projects and Programmes on Azure Platform. This role is purposed for Quantexa Fraud platform programme, Quantexa certified engineer is a preferred. KEY ACCOUNTABILITIES Analyse business requirements and support and maintain Quantexa platform. Build and deploy new/changes to data mappings, sessions, and workflows in Azure Cloud Platform – key focus area would be Quantexa platform on Azure. Develop both batch (using Azure Databricks) and real ...

Data Engineer, Chennai, India. About the job: The Data Engineer is a cornerstone of Vendasta's R&D team, driving the efficient processing, organization, and delivery of clean, structured data in support of business intelligence and decision-making. By developing and maintaining scalable ELT pipelines, they ensure data reliability and scalability, adhering to Vendasta's commitment to delivering data solutions aligned with evolving business needs. Your Impact: Design, implement, and maintain scalable ELT pipelines within a Kimball Architecture data warehouse. Ensure robustness against failures and data entry errors, managing data conformation, de-duplication, survivorship, and coercion. Manage...

We are looking for a highly skilled Senior ETL Consultant with strong expertise in Informatica Intelligent Data Management Cloud (IDMC) components such as IICS, CDI, CDQ, IDQ, CAI, along with proven experience in Databricks.

Data Engineer1 Common Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow,Experience on Spark/Hive/HDFS,Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Common Skils - SQL, GCP BQ, ETL pipelines using Pythin/Airflow, Experience on Spark/Hive/HDFS, Data modeling for Data conversion Resources (4) Prior experience working on a conv/migration HR project is additional skill needed along with above mentioned skills Data Engineer - Knows HR Knowledge , all other requirement from Functional Area given by UBER Customer Name Customer Nameuber

Key Responsibilities: • Designing and building a scalable Datawarehouse using Azure Data Factory (ADF) and Azure Synapse Pipelines and SSIS. • Creating visually appealing BI dashboards using Power BI and other reporting tools to deliver data-driven insights. • Collaborating with cross-functional teams, communicating complex concepts, and ensuring data governance and quality standards. Basic Qualifications: • 9-12 years of strong Business Intelligence/Business Analytics experience or equivalency is preferred. • B. Tech/B.E. - Any Specialization, M.E/M.Tech - Any Specialization • Strong proficiency in SQL and experience with database technologies (e.g., SQL Server). • Solid understanding of da...

Overview: TresVista is looking to hire an Associate in its Data Intelligence Group team, who will be primarily responsible for managing clients as well as monitor/execute projects both for the clients as well as internal teams. The Associate may be directly managing a team of up to 3-4 Data Engineers & Analysts across multiple data engineering efforts for our clients with varied technologies. They would be joining the current team of 70+ members, which is a mix of Data Engineers, Data Visualization Experts, and Data Scientists. Roles and Responsibilities: Interacting with the client (internal or external) to understand their problems and work on solutions that address their needs Driving pro...