469 Etl Pipelines Jobs - Page 10

Job Title: AWS Databricks Developer Experience Required: 6 to 12 Years Location: PAN India (Onsite/Hybrid as per project requirement) Employment Type: Permanent Open Positions: 5 Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines using Databricks (PySpark/Scala) Implement scalable data workflows on AWS (S3, Glue, Lambda, EMR, Redshift) Collaborate with cross-functional teams to build end-to-end data solutions Optimize pipeline performance and ensure data quality and governance Troubleshoot issues across cloud environments and ensure smooth data operations Use CI/CD tools for deployment and manage code via Git Other Details: Interview Drive: This Saturday (July 27th, 2025) ...

Mandatory Skills: Azure Skills Like Azure Databricks Azure Synapse Azure Functions Azure Data Lake Azure Integration Services Azure API Management CI/CD and PowerShell Scripts ADF ETL Pipelines Unity Catalog Azure SQL Database SQL Coding Storage Account, Azure Data Explorer, Spark Structured Streaming, ADF, Netezza SQL queries, Data lineage, Spark jobs. Role & Responsibilities / Job Description: Overall 4 to 8 years of experience with a minimum 4+ years of relevant professional work experience. Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure. Work together with data analysts to understand the needs for data and create effective data ...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for leveraging your 12+ years of experience in Microsoft Azure Data Engineering to drive analytical projects. Your expertise will be crucial in designing, developing, and deploying high-volume ETL pipelines using Azure, Microsoft Fabric, and Databricks for complex models. Your hands-on experience with Azure Data Factory, Databricks, Azure Functions, Synapse Analytics, Data Lake, Delta Lake, and Azure SQL Database will be utilized for managing and processing large-scale data integrations. In this role, you will be expected to optimize Databricks clusters and manage workflows to ensure cost-effective and high-perf...

As a Data Modeler at TRST01, a premier Sustainable Tech company in Hyderabad, you will play a crucial role in designing, implementing, and optimizing data models to support carbon accounting, lifecycle analysis, and sustainability reporting. Your expertise in Carbon Footprint and Reduction Data (CFRD) will be instrumental in tracking carbon emissions, energy consumption, and sustainability metrics. Collaborating with cross-functional teams, including data scientists, software engineers, and sustainability experts, you will ensure the integrity, accuracy, and scalability of environmental data models. Your responsibilities will include developing robust data models, establishing reliable and s...

About the Role We are seeking a Senior Python Developer to lead the development of scalable, high-performance backend systems and data-driven applications. You will work on building robust services, APIs, and data workflows that power real-time and batch applications. This role is ideal for a seasoned developer who thrives on solving complex problems, building reliable software, and working across data and engineering teams. Youll play a key role in designing and developing Python-based systems that interface with big data frameworks, cloud platforms, and analytics tools. Key Responsibilities Design and develop robust Python-based backend services and microservices. Build RESTful APIs and in...

Job Title : Mid-Level Data Engineer Healthcare Domain Location: Remote Option Experience: 2 to 3 Years Employment Type: Full-Time About the Role We are seeking a motivated and detail-oriented Mid-Level Data Engineer with 2 to 3 years of experience in designing, developing, and optimizing data pipelines within the healthcare domain. The ideal candidate will have hands-on experience with Databricks , strong SQL skills, and a solid understanding of healthcare data standards (e.g., HL7, EDI X12 837/835, HCC, CPT/ICD codes). Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Databricks , PySpark , and Delta Lake for large-scale healthcare datasets. Collaborate wit...

Build and Optimize ELT/ETL Pipelines using BigQuery, GCS, Dataflow, PubSub and Orchestration services Composer/Airflow • Hands-On experience in building ETL/ELT Pipelines with developing software code in Python • Experience in working with data warehouses, data warehouse technical architectures, reporting/analytic tools • Develop and implement data quality and governance procedures to ensure the accuracy and reliability of data • Demonstrate extensive skills and success in the implementation of technology projects within a professional environment, with a particular focus on data engineering • Eager to learn and explore new services within GCP to enhance skills and contribution to Projects •...

Role & responsibilities Design, implement and operate ETL workflows (Airflow, AWS Glue). Administer PostgreSQL, MySQL or SQL Server: performance tuning, backups, restores. Manage schema migrations via Flyway/Liquibase and version control. Provision and maintain data infrastructure with Terraform & Ansible. Automate scaling, snapshots and cost controls for RDS clusters. Secure data environment with IAM, network policies and encryption. Participate in 247 on-call rotations; author runbooks and postmortems. Collaborate on data modeling, indexing strategies and query optimization. Document all data flows, tuning guides, and runbooks in Confluence. Preferred candidate profile B.Tech/BE in CS, Inf...

Key Responsibilities Design conformed star & snowflake schemas , implement SCD2 dimensions and fact tables. Lead Spark (PySpark/Scala) or AWSGlue ELT pipelines from RDSZeroETL/S3 into Redshift. Tune RA3 clusterssort/dist keys, WLM queues, Spectrum partitionsfor subsecond BI queries. Establish dataquality, lineage, and costgovernance dashboards using CloudWatch & Terraform/CDK. Collaborate with Product & Analytics to translate HR KPIs into selfservice data marts. Mentor junior engineers; drive documentation and coding standards. MustHave Skills AmazonRedshift (sort & dist keys, RA3, Spectrum) Spark on EMR/Glue (PySpark or Scala) Dimensional modelling (Kimball), star schema, SCD2 Advanced SQL ...

As a seasoned IT professional with over 15 years of progressive experience, you will be responsible for providing strategic and technical leadership in key areas such as Application & Data Modernization, AI, and BI technologies. Your role will involve defining and developing new service offerings that are aligned with emerging technologies and market demands. You will lead high-performing teams to deliver enterprise-grade software products using modern tech stacks such as Java, Spring, Python, Flutter, Angular, React, Vue, microservices, Kafka, Azure, AWS, and GCP. You will also be tasked with scaling and managing application development and data engineering practices across different geogra...

> Strong programming expertise in PySpark and Python. > Solid understanding of Spark internals, DAG optimization, partitioning, broadcast joins, etc. > Hands-on experience with one or more cloud platforms > Experience with API integrations Required Candidate profile The ideal candidate has strong expertise in PySpark optimization, API integration, and big data ingestion using AWS, GCP, or Azure. A solid foundation in SQL.

As a Senior Software Engineer Java at SAP, you will be a key member of the App2App Integration team within SAP Business Data Cloud. Your primary focus will be to accelerate the integration of SAP's application ecosystem with its unified data fabric, facilitating low-latency, secure, and scalable data exchange processes. Your responsibilities will include designing and developing core integration frameworks using Java, RESTful APIs, and messaging frameworks like Apache Kafka. You will also be tasked with building and maintaining scalable data processing and ETL pipelines that support real-time and batch data flows. Additionally, you will work on integrating data engineering workflows with too...

As a Data Engineer for our data-rich e-commerce platform catering to the life sciences sector, your primary responsibility will be to support infrastructure, develop data pipelines, and deploy pricing logic. You will play a crucial role in ensuring the usability and interface design of internal tools that facilitate experimentation, pricing configuration, and real-time monitoring. Your key responsibilities will include: - Building and maintaining ETL pipelines for pricing, shipping, and behavioral datasets - Collaborating with data scientists and product managers to facilitate model development and experimentation - Developing APIs or backend logic to implement dynamic pricing algorithms - C...

BambooBox is a SaaS-based software start-up with 4+ years of expertise, offering a platform for growth marketers. With its advanced AI and ML-driven platform, BambooBox helps companies deliver untapped marketing intelligence to business teams and increase the marketing contribution to the overall revenue. The company collaborates with clients like Airtel Business, Darwinbox, Acalvio, and other global organizations. BambooBox is backed by investors such as Peak XV (earlier Sequoia Surge), Emergent Ventures, and Arc180. Currently, BambooBox is in the process of scaling the organization and is in a high-growth phase. Located in Bangalore, BambooBox is seeking a talented developer to join its te...

You are a talented SAP HANA Developer sought after by RiskInsight Consulting Pvt Ltd, a company specializing in providing innovative risk management solutions to clients across finance and insurance sectors. Your role in this growing team involves designing and implementing sophisticated SAP HANA applications to enhance business value and efficiency. As a key member of the team, you will be responsible for developing and maintaining data models in SAP HANA, designing, coding, and testing applications, optimizing SQL queries for high performance, and collaborating with business analysts to translate requirements into technical specifications. Additionally, you will implement data integration ...

Company Overview Incedo is a US-based consulting, data science and technology services firm with over 3000 people helping clients from our six offices across US, Mexico and India. We help our clients achieve competitive advantage through end-to-end digital transformation. Our uniqueness lies in bringing together strong engineering, data science, and design capabilities coupled with deep domain understanding. We combine services and products to maximize business impact for our clients in telecom, Banking, Wealth Management, product engineering and life science & healthcare industries. Working at Incedo will provide you an opportunity to work with industry leading client organizations, deep te...

Role : Enterprise Architect (Data) Location : Bangalore Role overview We are seeking an experienced Enterprise Data Architect to join our team, focusing primarily on Data engineering, Data Analytics/Visualization and cloud engineering. The ideal candidate will play a crucial role in shaping our technology landscape, participating in project delivery, and contributing to presales activities during lean periods. 14 to 20 years of total experience in IT, with a minimum of 3 years in an Enterprise Data Architect capacity. Strong expertise in data and cloud technologies, with hands-on experience in data architecture, cloud migrations, and modern data platforms. Knowledge of design patterns and ar...

We are looking for a Senior Data Engineer with deep experience in SnapLogic, SQL, ETL pipelines, and data warehousing, along with hands-on experience with Databricks.in designing scalable data solutions and working across cloud and big data .

We are seeking experienced Data Analysts / Data Engineers with strong expertise in U.S. pharmaceutical commercial datasets to support critical Data Operations initiatives. This role will be focused on onboarding third-party data, ensuring data quality , and implementing outlier detection techniques . Familiarity with ML/A I approaches for anomaly detection is highly desirable. Key Responsibilities: Pharma Data Integration: Work extensively with U.S. pharmaceutical commercial datasets. Ingest and onboard third-party data sources such as IQVIA, Symphony Health, Komodo Health etc. Ensure alignment of data schemas, dictionary mapping, and metadata integrity. Data Quality & Governance: Design and...

Responsibilities: * Design, develop & maintain data pipelines using Python, AWS & SQL. * Collaborate with cross-functional teams on ETL projects. * Optimize performance & scalability of data systems.

Handling of petabytes of data and experience required in design and delivery of distributed system in cloud by using server less architecture & in Data lakes and data sourcing platforms and in python and pyspark immediate joiners are prefer.

As an Experienced Senior Data Engineer at Adobe, you will utilize Big Data and Google Cloud technologies to develop large-scale, on-cloud data processing pipelines and data warehouses. Your role will involve consulting with customers worldwide on their data engineering needs around Adobe's Customer Data Platform and supporting pre-sales discussions regarding complex and large-scale cloud data engineering solutions. You will design custom solutions on cloud by integrating Adobe's solutions in a scalable and performant manner. Additionally, you will deliver complex, large-scale, enterprise-grade on-cloud data engineering and integration solutions in a hands-on manner. To be successful in this ...

We are seeking an experienced Project Manager specializing in Data & Analytics to oversee and supervise data-driven projects spanning various business domains. The successful candidate will possess significant expertise in managing projects related to data engineering, business intelligence, and analytics, demonstrating a track record of collaborating effectively with technical and business stakeholders to deliver impactful solutions within specified timeframes and budgets. Key Responsibilities: - Lead the end-to-end project management of data and analytics initiatives. - Define project scope, objectives, deliverables, schedules, and resource allocations. - Collaborate closely with data engi...

As a Data Analyst with deep experience in the US Healthcare industry and skills around Machine Learning (ML), you will play a vital role in conducting comprehensive data analysis using statistical and exploratory methods to uncover patterns and insights that drive data-driven decision-making in the healthcare domain. Leveraging your knowledge of healthcare industry metrics such as HEDIS, CMS Star Ratings, risk adjustment models, and revenue cycle data, you will optimize analytics strategies to ensure efficient data quality, availability, and reliability for AI/ML-driven healthcare analytics solutions. Your responsibilities will include designing and maintaining data pipelines for the ingesti...

As a Senior Consultant - Data Engineer at AstraZeneca, you will be an integral part of the R&D IT team, a driving force behind technological advancement in areas such as Drug Research and Development. With our global presence spanning across multiple countries, including North America, the UK, Sweden, India, and Mexico, we are united by our passion for innovation. The Data & Analytics team, specializing in cutting-edge fields like Data Engineering, Generative AI, and Data Science, is at the heart of our mission. Join us to not only keep up with the latest technological trends but to set them. In this role, you will work collaboratively in a dynamic, globally distributed team to design and im...