988 Apache Airflow Jobs - Page 32

We are looking for an accomplished and hands-on Senior AI Lead to drive the development and deployment of AI-powered solutions for supply chain optimization. This senior technical leadership role requires deep AI/ML expertise, client-facing engagement, and team management to deliver intelligent, scalable products on the AWS cloud platform. You will be responsible for leading the end-to-end lifecycle of AI solutions that address real-world supply chain challenges such as demand planning and forecasting, inventory optimization, purchasing, fulfillment, route planning, risk prediction, and anomaly detection. Your key responsibilities will include designing, building, and deploying AI/ML models ...

As a Senior Database Administrator (DBA) specializing in Cloud & Big Data, you will be responsible for designing, managing, and optimizing large-scale, high-performance databases across cloud and on-premises environments. With 9+ years of experience, you will demonstrate expertise in SQL and NoSQL databases, including Couchbase, PostgreSQL, MySQL, Oracle, MongoDB, and DynamoDB. Additionally, familiarity with Databricks, Apache Spark, Python, R, and cloud-based data processing solutions is required. Your role will involve database performance tuning, replication strategies, high availability solutions, data pipeline management, and security enforcement. Experience in telecommunications data m...

As a Data Architect with over 6 years of experience, you will be responsible for designing and implementing modern data lakehouse architectures on cloud platforms such as AWS, Azure, or GCP. Your primary focus will be on defining data modeling, schema evolution, partitioning, and governance strategies to ensure high-performance and secure data access. In this role, you will own the technical roadmap for scalable data platform solutions, ensuring they are aligned with enterprise needs and future growth. You will also provide architectural guidance and conduct code/design reviews across data engineering teams to maintain high standards of quality. Your responsibilities will include building an...

You should have over ten years of experience in software development, including recent years in a leadership position with increasing responsibilities. Your expertise should cover various areas such as C#, .Net, designing and developing cloud-native solutions on platforms like Azure, AWS, and Google Cloud Platform, using Infrastructure as Code tools like Terraform and Pulumi, employing configuration management tools like Ansible and Chef, working with containerization and orchestration technologies like Docker and Kubernetes, and integrating with Native and third-party Databricks solutions. Additionally, you should possess extensive experience specifically in Azure. It would be advantageous ...

The role involves Java and Python development with a focus on Apache Airflow for workflow automation. You will be responsible for Python and Java Development, with proficiency in both languages being crucial for building Airflow. Your tasks will include using Airflow for workflow automation, encompassing designing, implementing, and maintaining Directed Acyclic Graphs (DAGs). Additionally, you will be managing task dependencies and configuring scheduling and error handling. An understanding of database systems, data warehousing concepts, and data architecture is essential for building robust data pipelines. To be successful in this role, you should have 5 to 8 years of experience in Java and...

As a Java Full Stack Lead Developer at Dedalus, you will utilize your extensive experience in modern web development, backend technologies, and cloud infrastructure to make a profound impact in the healthcare sector. You will take ownership of complex projects, mentor junior developers, optimize existing systems, and contribute to new feature development. Your role will involve designing and implementing backend solutions using Java and Python, with additional experience in PHP and the Laravel Framework being a plus. You will also be responsible for developing and maintaining scalable front-end applications using Angular, as well as leading development efforts and deployments using CI/CD pip...

The role of Lead - Data Engineering in the Digital Business division at SonyLIV involves leading the data engineering strategy, architecting scalable data infrastructure, driving innovation in data processing, ensuring operational excellence, and fostering a high-performance team. This position plays a crucial role in enabling data-driven insights for OTT content and user engagement. The job location can be either in Bangalore or Mumbai with a Work From Office (WFO) setup. As a Lead - Data Engineering, you are expected to have a minimum of 10 years of experience in the field. Your responsibilities will include defining the technical vision for scalable data infrastructure by establishing a r...

As a Data Engineer at Tata Electronics Private Limited (TEPL), you will play a crucial role in architecting and implementing scalable offline data pipelines for manufacturing systems, ensuring the smooth functioning of various components such as AMHS, MES, SCADA, PLCs, vision systems, and sensor data. Your expertise in designing and optimizing ETL/ELT workflows using tools like Python, Spark, SQL, and Airflow will be instrumental in transforming raw data into actionable insights. You will lead database design and performance tuning activities across SQL and NoSQL systems, optimizing schema design, queries, and indexing strategies to enhance the efficiency of manufacturing data processing. Yo...

Join us at Barclays in the role of Strategic Adoption AI Engineer, where you will be tasked with enhancing existing processes, reporting, and controls to ensure flawless execution of BAU. Your responsibilities will include driving efficiencies, implementing process improvements, and standardizing processes across SBUs wherever feasible. At Barclays, we not only anticipate the future but also actively participate in creating it. To excel in this role, you should possess the following skills: - Proficient programming abilities in Python and hands-on experience with ML libraries such as scikit-learn, TensorFlow, and PyTorch. - Familiarity with automation tools like Jenkins, GitHub Actions, or G...

You are a meticulous and highly responsible Senior Data Specialist with 4-7 years of professional experience, capable of independently designing, managing, and optimizing data workflows and pipelines. Your expertise lies in databases such as PostgreSQL and MySQL, along with practical experience in modern ETL tools like Apache NiFi, Talend, or Apache Airflow. Your role demands structured work, a sense of ownership, and precise delivery in fast-paced environments. As a Senior Data Specialist, your duties and responsibilities include designing, building, and maintaining scalable ETL pipelines for both structured and unstructured data. You will work with PostgreSQL, MySQL, and other relational d...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Job description : We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment. Key Responsibilities : - Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond). - Model and structure data for performance, scalability, and usability. -...

Job description : We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment. Key Responsibilities : - Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond). - Model and structure data for performance, scalability, and usability. -...

Job description : We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment. Key Responsibilities : - Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond). - Model and structure data for performance, scalability, and usability. -...

Job description : We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment. Key Responsibilities : - Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond). - Model and structure data for performance, scalability, and usability. -...

Job description : We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment. Key Responsibilities : - Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond). - Model and structure data for performance, scalability, and usability. -...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

We are seeking an experienced Amazon Redshift Developer / Data Engineer to design, develop, and optimize cloud-based data warehousing solutions. The ideal candidate should have expertise in Amazon Redshift, ETL processes, SQL optimization, and cloud-based data lake architectures. This role involves working with large-scale datasets, performance tuning, and building scalable data pipelines. Key Responsibilities: Design, develop, and maintain data models, schemas, and stored procedures in Amazon Redshift. Optimize Redshift performance using distribution styles, sort keys, and compression techniques. Build and maintain ETL/ELT data pipelines using AWS Glue, AWS Lambda, Apache Airflow, and dbt. ...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...