682 Apache Airflow Jobs - Page 13

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

As a Digital Engineer at Medtronic's Diabetes business, you will be at the forefront of innovation in medical technology, working to improve healthcare access for people worldwide. Your role as a PySpark Data Engineer will involve designing, developing, and maintaining data pipelines using PySpark, collaborating with data scientists, analysts, and stakeholders to ensure efficient processing and analysis of large datasets. **Key Responsibilities:** - Design, develop, and maintain scalable and efficient ETL pipelines using PySpark. - Work with structured and unstructured data from various sources. - Optimize and tune PySpark applications for performance and scalability. - Collaborate with data...

Role Overview: As a Developer contracted by Luxoft for supporting customer initiatives, your main task will involve developing solutions based on client requirements within the Telecom/network work environment. You will be responsible for utilizing technologies such as Databricks and Azure, Apache Spark, Python, SQL, and Apache Airflow to create and manage Databricks clusters for ETL processes. Integration with ADLS, Blob Storage, and efficient data ingestion from various sources including on-premises databases, cloud storage, APIs, and streaming data will also be part of your role. Key Responsibilities: - Develop solutions based on client requirements in the Telecom/network work environment...

Role Overview: As a Senior Software Engineer-AI/ML Backend Developer at CGI, you will be joining a dynamic technology team and playing a crucial role in shaping the future of technology by integrating cutting-edge AI/ML techniques into scalable backend solutions. Your responsibilities will include developing, optimizing, and maintaining backend services for AI/ML applications, implementing and deploying machine learning models to production environments, collaborating with data scientists and frontend engineers for seamless integration, monitoring and improving performance, reliability, and scalability of existing AI/ML services, designing robust data pipelines and processing workflows, iden...

As a Product Developer at Citi, your role involves staying abreast of developments in the field and contributing to directional strategy by considering their application in your job and the business. You will be recognized as a technical authority within the business, requiring basic commercial awareness and developed communication and diplomacy skills to guide, influence, and convince others. Your work will have a significant impact on the area through complex deliverables, providing advice and counsel related to technology or business operations, ultimately affecting the overall performance and effectiveness of the sub-function/job family. In this role, you are expected to: - Develop repor...

As a Data Engineer, you will be responsible for designing and building efficient data pipelines using Azure Databricks (PySpark). You will implement business logic for data transformation and enrichment at scale, as well as manage and optimize Delta Lake storage solutions. Additionally, you will develop REST APIs using FastAPI to expose processed data and deploy them on Azure Functions for scalable and serverless data access. Key Responsibilities: - Design and build efficient data pipelines using Azure Databricks (PySpark) - Implement business logic for data transformation and enrichment at scale - Manage and optimize Delta Lake storage solutions - Develop REST APIs using FastAPI to expose p...

As a highly skilled and experienced Snowflake Architect, your role will involve designing, developing, and deploying enterprise-grade cloud data solutions. You should have a robust background in data architecture, cloud data platforms, and Snowflake implementation. Hands-on experience in end-to-end data pipeline and data warehouse design is essential for this position. **Key Responsibilities:** - Lead the architecture, design, and implementation of scalable Snowflake-based data warehousing solutions - Define data modeling standards, best practices, and governance frameworks - Design and optimize ETL/ELT pipelines using tools such as Snowpipe, Azure Data Factory, Informatica, or DBT - Collabo...

Required Skills: Data Engineer, AWS, Spark, Apache Airflow, Kafka, Snowflake, DBT, Flink, CI / CD Pipelines, Apache Iceberg, Glue

Role Overview: At EY, you'll have the opportunity to build a career tailored to your unique skills and attributes. With global scale, support, an inclusive culture, and advanced technology, you can become the best version of yourself. Your unique voice and perspective are essential in helping EY improve. Join us in creating an exceptional experience for yourself and contributing to a better working world for all. Key Responsibilities: - Investigate, research, review, evaluate, and elicit requirements. - Conduct risk assessments and provide mitigation recommendations. - Perform functional data or back-end test executions using SQL and automation scripts. - Facilitate and support User Acceptan...

What are we looking for? We are looking for a passionate and experienced Data Engineer to join our team and help build scalable, reliable, and efficient data pipelines on cloud platforms like Primarily on Google Cloud Platform (GCP) and secondary on Amazon Web Services (AWS) . You will work with cutting-edge technologies to process structured and unstructured data, enabling data-driven decision-making across the organization. What does the job entail? Design, develop, and maintain robust data pipelines and ETL/ELT workflows using PySpark , Python , and SQL . Build and manage data ingestion and transformation processes from various sources including Hive , Kafka , and cloud-native services. O...

Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar framew...

Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar framew...

Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and appli...

Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and appli...

Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and appli...

Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud envi...

Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar framew...

Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud envi...

Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solution...

Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, GCP). - Build and manage data warehouse s...

Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar framew...

We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema evolution, and data versioning. - Collaborate with cross-functional teams to deliver high-quality data solutions. Req...

We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema evolution, and data versioning. - Collaborate with cross-functional teams to deliver high-quality data solutions. Req...

We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow orchestration using Airflow or similar frameworks. - Design, implement, and manage data warehouse solutions, schema evolution, and data versioning. - Collaborate with cross-functional teams to deliver high-quality data solutions. Req...

As a Senior Manager, Integrated Test Lead in Data Product Engineering & Delivery at McDonald's Corporation, your role will involve leading comprehensive testing strategy and execution for complex data engineering pipelines and product delivery initiatives. Your primary responsibilities will include designing and implementing end-to-end testing frameworks, ensuring test coverage across data workflows, developing integrated testing strategies, and establishing testing standards and best practices. Key Responsibilities: - Design and implement end-to-end testing frameworks for data pipelines, ETL / ELT processes, and analytics platforms - Ensure test coverage across ETL / ELT, data transformatio...