974 Apache Airflow Jobs - Page 11

We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema evolution, and data versioning. - Collaborate with cross-functional teams to deliver high-quality data solutions. Req...

We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, GCP). - Build and manage data warehouse solutions, schema evolution, and data versioning. - Implement and manage workflow orchestration u...

We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema evolution, and data versioning. - Collaborate with cross-functional teams to deliver high-quality data solutions. Req...

Contract Duration : 4 Months +Exendible based on Performance Job Timings : India Evening Shift (till 11 : 30 PM IST) We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema ...

Contract Duration : 4 Months (Extendable based on Performance) Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, ...

Contract Duration : 4 Months (Extendable based on Performance) Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, ...

Contract Duration : 4 Months (Extendable based on Performance) Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, ...

Your Opportunity As an architect, you will get the opportunity to grow in the field of high-performance computing, complex system design and low-level optimizations for better cost of ownership. Roles and Responsibility As a Software Architect, you will be responsible for designing and implementing High performance computing software solutions for our organization. You will work closely with cross-functional teams, including software engineers, product managers, and business stakeholders, to understand requirements and translate them into architectural/software designs that meet business needs. You will be coding and developing quick prototypes to establish your design with real code and dat...

Python Lead Developer (Contract Position) Location : Chennai Hybrid (3 Days a Week in Office) Experience : 8-10 Years Employment Type: Contract Shift : UK Shift (if applicable) Job Summary We are looking for a highly skilled Python Lead Developer with strong expertise in Python, SQL, AWS, and Apache Airflow. The ideal candidate will lead a team of developers to build scalable data pipelines, backend systems, and cloud-based solutions, ensuring high performance, maintainability, and adherence to best practices. Key Responsibilities Lead and mentor a team of Python developers, ensuring delivery of high-quality software solutions. Design, develop, and optimize data pipelines and backend systems...

Job Title: Data Engineer Pharma/Life Sciences Domain Company: Indegene (Client Location) Experience: 5–10 Years Location: Bangalore (Onsite/Hybrid) Employment Type: Full-Time Industry: Healthcare / Pharma / Life Sciences Job Description We are looking for a highly skilled Data Engineer with strong experience in data engineering technologies and a solid understanding of the Healthcare/Pharma/Life Sciences domain . The ideal candidate will be responsible for designing, building, and optimizing end-to-end data pipelines that support analytics, insights, and data-driven decisions. Roles & Responsibilities Design, develop, and maintain scalable data pipelines, ETL workflows, and data integration ...

Lead Software Engineer Backend Were seeking a Lead Software Engineer to join one of our Data Layer teams. As the name implies, the Data Layer is at the core of all things data at Zeta. Our responsibilities include: Developing and maintaining the Zeta Identity Graph platform, which collects billions of behavioural, demographic, environmental, and transactional signals to power people-based marketing. Ingesting vast amounts of identity and event data from our customers and partners. Facilitating data transfers across systems. Ensuring the integrity and health of our datasets. And much more. As a member of this team, the data engineer will be responsible for designing and expanding our existing...

As a Data Engineer at our company, you will play a crucial role in designing, building, and managing robust and scalable data pipelines using Python, SQL, and AWS. Your key responsibilities will include: - Designing, building, and managing robust and scalable data pipelines (batch and real-time). - Developing ETL processes to acquire, transform, and integrate data from multiple sources. - Building and maintaining data warehouses, data lakes, and other storage solutions. - Optimizing data systems for performance, reliability, and scalability. - Collaborating with cross-functional teams to understand data requirements and deliver efficient solutions. - Ensuring data quality, consistency, and i...

As a hands-on technical lead and senior individual contributor in BI Engineering within Visa's Global Data organization, reporting to the Lead of Data Insights Studio, you will partner with internal analytics teams to deliver high-impact BI solutions across products and markets. Operating from the Global Shared Services hub in Bengaluru, you will leverage Visa's rich datasets to drive actionable insights that create business value. **Key Responsibilities:** - Partner with internal analytics and business teams to translate complex, ambiguous problems into clear, measurable outcomes and actionable data products. - Lead end-to-end BI solution delivery, including requirements refinement, data mo...

As a skilled Data Engineer, your role will involve designing, building, and maintaining efficient data pipelines and architectures to support analytics and business intelligence initiatives. Your hands-on experience in data integration, ETL development, and cloud-based data platforms will be crucial for this position. Key Responsibilities: - Design and implement data pipelines for the ingestion, transformation, and storage of large datasets. - Develop and maintain ETL processes to ensure data quality, consistency, and availability. - Collaborate with data analysts, scientists, and business teams to support data-driven initiatives. - Optimize data workflows for scalability and performance. - ...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : GCP Dataflow, Spring Boot, Apache Airflow Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications are optimized for performance and usabilit...

Role & responsibilities: Role and Responsibilities: Design, build, and maintain scalable and efficient data pipelines for ingesting and transforming data from multiple sources. Develop and manage cloud-based data lakes optimized for analytics and business intelligence usage . Work closely with data science and BI teams to understand data requirements and ensure seamless integration of datasets. Implement and maintain data quality checks, data profiling, and monitoring mechanisms . Establish and enforce data governance policies including data lineage, cataloging, and access control. Optimize data processing workloads for performance and cost-efficiency in cloud environments. Collaborate acros...

Role Overview: As a skilled and experienced Technology Stack Administrator, your primary responsibility will be to manage and optimize the on-premises infrastructure of the company. This includes deploying, configuring, monitoring, and troubleshooting various platforms such as RedHat OpenShift, Apache Spark, Apache Airflow, Delta Lake, Apache Kafka, and Debezium. Your expertise will ensure high availability, performance, and security of these platforms. Key Responsibilities: - Install, configure, and maintain RedHat OpenShift clusters in an on-premises environment. - Build and orchestrate data pipelines using Apache Airflow. - Implement and manage Delta Lake for ACID-compliant data lake arch...

As a Staff Data Engineer at Pattern, you will play a crucial role in designing and scaling the canonical data model to ensure the delivery of trusted, high-quality data. Your primary focus will be on transforming complex, raw data into easily understandable tables that power analytics, forecasting, and AI/ML applications. With your technical expertise, you will lead the Patterns Data Engineering team by guiding technical direction, mentoring engineers, and establishing a single source of truth that drives Patterns" global growth. **Roles and Responsibilities:** - Design and evolve canonical and medallion-layer data models (bronze/silver/gold) to enable scalable, governed data across the orga...

As a Data Engineer at our company, you will have the following responsibilities: - Review and understand business requirements to ensure development tasks are completed within the provided timeline and are thoroughly tested with minimal defects. - Collaborate with a software development team to implement best practices and optimize the performance of Data applications. - Engage with various teams across the company and interact with customers to define, design, and showcase new concepts and solutions. - Conduct research on new Big Data technologies, evaluating their maturity and alignment with business and technology strategies. - Work closely with other developers to meet client needs effic...

Job Description Python Lead Developer (Contract Position) Location : Chennai Hybrid (3 Days a Week in Office) Experience : 8 - 10 Years Employment Type : Contract Shift : UK Shift (if applicable) Job Summary We are looking for a highly skilled Python Lead Developer with strong expertise in Python, SQL, AWS, and Apache Airflow. The ideal candidate will lead a team of developers to build scalable data pipelines, backend systems, and cloud-based solutions, ensuring high performance, maintainability, and adherence to best practices. Key Responsibilities Lead and mentor a team of Python developers, ensuring delivery of high-quality software solutions. Design, develop, and optimize data pipelines ...

The CDP ETL & Database Engineer will specialize in architecting, designing, and implementing solutions that are sustainable and scalable. The ideal candidate will understand CRM methodologies, with an analytical mindset, and a background in relational modeling in a Hybrid architecture. The candidate will help drive the business towards specific technical initiatives and will work closely with the Solutions Management, Delivery, and Product Engineering teams. The candidate will join a team of developers across the US, India & Costa Rica. Responsibilities : ETL Development The CDP ETL & Database Engineer will be responsible for building pipelines to feed downstream data processes. They will be...

CDP ETL & Database Engineer The CDP ETL & Database Engineer will specialize in architecting, designing, and implementing solutions that are sustainable and scalable. The ideal candidate will understand CRM methodologies, with an analytical mindset, and a background in relational modeling in a Hybrid architecture. The candidate will help drive the business towards specific technical initiatives and will work closely with the Solutions Management, Delivery, and Product Engineering teams. The candidate will join a team of developers across the US, India & Costa Rica. Responsibilities : ETL Development The CDP ETL & Database Engineer will be responsible for building pipelines to feed downstream ...

Role: Tech Lead Description The Data Engineering Lead is a senior-level position within our Data Engineering team, responsible for spearheading the design, implementation, and optimization of advanced data architectures using Databricks. This role involves leading a team of data engineers, collaborating with business and data analysts, and business stakeholders, and delivering scalable, secure data solutions tailored to the needs of commercial analytics in the pharmaceutical industry. Responsibilities 1. Architecture Design o Develop and maintain scalable data architectures using Databricks, including data lakes, data warehouses, and real-time processing systems. o Create detailed blueprints...

We are seeking an accomplished Senior DevOps Engineer with 15-20 years of experience in cloud infrastructure management, automation, and CI/CD implementation The ideal candidate will architect, maintain, and optimize Azure cloud environments while ensuring scalability, performance, and reliability across enterprise applications Key responsibilities include managing Azure resources (VMs, AKS, API Management, Service Bus, Event Hub), administering Azure Kubernetes Service (AKS) and Apache Airflow, automating deployments via Python/Bash/PowerShell, and maintaining middleware systems like TIBCO The role demands strong expertise in CI/CD pipelines, Infrastructure as Code (Pulumi/ARM), and monitor...

Hiring Now: Big Data Engineer Bangalore || Coforge Ltd Note: This position requires 5 days mandatory - Work From Office. Send your updated CV to: Gaurav.2.Kumar@coforge.com For queries, reach out on WhatsApp: 9667427662 Job Location: Bangalore Experience Required: 6 Years Availability: Immediate Joiners Preferred Key Technologies & Skills Required: Big Data Tools: Hadoop, Hive Programming Languages: Python, SQL Distributed Processing Frameworks: Apache Spark, PySpark Workflow Orchestration: Apache Airflow Version Control: Bitbucket Repository. Job Description: Key Responsibilities: Design and implement robust data pipelines using Spark and PySpark. Develop and optimize Hive queries for large...