682 Apache Airflow Jobs - Page 19

You are an experienced Python developer with a focus on the stock market domain, having a minimum of 4 years of relevant experience. As a Python developer, you will be responsible for various data engineering tasks using your expertise in Python programming. Your qualifications include a Bachelors or Masters degree in Computer Science, Data Engineering, or a related field. You should have at least 4 years of hands-on experience in Python programming specifically for data engineering tasks. Proficiency in web server technologies using frameworks like Flask, FastAPI, and Django is essential for this role. Additionally, you should have expertise in creating and managing workflows using Apache A...

As a DataOps Engineer at BlackLine, you will play a crucial role in ensuring the robustness, reliability, and scalability of our data pipelines and infrastructure. Your responsibilities will include developing and maintaining data pipelines, orchestrating workflows, containerizing applications, managing cloud infrastructure, scripting automation, implementing CI/CD pipelines, providing system administration support, monitoring data pipelines, collaborating with various stakeholders, and documenting operational procedures. To excel in this role, you are required to have 2-6 years of professional experience in DataOps, Data Engineering, or a related field. Proficiency in Python and SQL for dat...

You are invited to apply for the position of Data Engineer at our company based in Bengaluru. As a Data Engineer with 5 to 7 years of experience, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure. Collaborating closely with data scientists, analysts, and software engineers, you will play a key role in ensuring the reliable and efficient delivery of data across our data ecosystem. The ideal candidate should hold a Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field. You should have at least 3 to 5 years of experience in data engineering or a similar role. Proficiency in SQL and one programmin...

The position is based at Altimetrik with base locations in Bangalore, Chennai, Pune, Jaipur, Hyderabad, Gurugram. The ideal candidate should be able to join immediately or within 10 days. As a Machine Learning Engineer, your primary responsibility will be to design and implement ML solutions while architecting scalable and efficient systems. You should be proficient in Machine Learning Algorithms, Data Engineering, ETL/ELT processes, data cleaning, preprocessing, EDA, feature engineering, data splitting, and encoding. Additionally, you should have experience in MLOps, including model versioning, training, experimenting, deployment, and monitoring using tools such as Python, Pandas, TensorFlo...

As a GCP Data Engineer specialized in Data Migration & Transformation, you will be responsible for designing and constructing robust, scalable data pipelines and architectures on Google Cloud Platform (GCP), particularly focusing on BigQuery. Your primary tasks will involve migrating and transforming large-scale data systems and datasets to GCP while emphasizing performance, scalability, and reliability. It will be crucial for you to automate data lineage extraction and ensure data integrity across various systems and platforms. Collaborating closely with architects and stakeholders, you will play a key role in implementing GCP-native and 3rd-party tools for data ingestion, integration, and ...

As a Data Quality Developer joining our Enterprise Data Governance team, you will be responsible for implementing data quality rules within our SuperZoom data quality platform. Your primary tasks will include translating business data quality requirements into technical solutions, developing SQL-based data quality rules, and collaborating with various team members and stakeholders to ensure successful implementation of data quality rules. You will need to leverage your strong SQL expertise and understanding of data quality challenges to monitor and enhance data integrity. Working closely with the data governance team, data quality team, and business stakeholders, you will play a crucial role...

As a Senior Data Engineer with 4+ years of experience, you will be responsible for designing, developing, and optimizing data pipelines using Airflow, Python, and DBT. Your role will involve building and maintaining data models and data marts in Snowflake, while implementing best practices in data transformation, testing, and documentation. Collaborating with data analysts, scientists, and business stakeholders is crucial to understand data needs and ensure data quality, governance, and lineage across data pipelines. Monitoring and troubleshooting data pipeline issues proactively, optimizing query performance and data storage in Snowflake, as well as automating workflows and scheduling tasks...

You have more than 7 years of IT experience and have expertise in working with at least two structural databases such as SQL, Oracle, or Postgres, and one NoSQL database. You are capable of collaborating with the Presales team to propose optimal solutions and architectures. Additionally, you possess design experience with BQ, Redshift, and Synapse. Your responsibilities include managing the entire product life cycle, from proposal to delivery, and continuously evaluating architecture enhancements with the delivery team. You are well-versed in security protocols for in-transit data, as well as encryption and decryption of PII data. Moreover, you have a strong understanding of analytics tools ...

You will be joining eMeasurematics Pvt Ltd, a leading innovator in intelligent material handling and advanced automation solutions for large-scale manufacturing and warehousing facilities. Our commitment to pioneering autonomous systems and tailored automation solutions has enabled us to work with industry giants across the global steel, mega-manufacturing, and warehousing sectors. By harnessing cutting-edge data-driven models and robust engineering expertise, we empower businesses to enhance productivity and achieve operational excellence. As a forward-thinking leader in industrial automation, eMeasurematics is dedicated to revolutionizing the future of manufacturing through smart, sustaina...

As a Python Developer at MicroGenesis TechSoft in Bangalore, India, you will be responsible for designing, developing, and maintaining robust Python-based applications and data pipelines. Your role will involve building and orchestrating workflows using Apache Airflow, collaborating with cross-functional teams to gather requirements and deliver high-quality solutions, and optimizing existing systems for scalability, reliability, and performance. Additionally, you will participate in code reviews, mentor junior developers, troubleshoot and resolve issues in development, testing, and production environments, and ensure code quality, documentation, and adherence to best practices. To be success...

About Markovate: At Markovate, you don't just follow trends, we drive them. We transform businesses through innovative AI and digital solutions that turn vision into reality. Our team harnesses breakthrough technologies to craft bespoke strategies that align seamlessly with our clients" ambitions. From AI Consulting And Gen AI Development To Pioneering AI Agents And Agentic AI, We Empower Our Partners To Lead Their Industries With Forward-thinking Precision And Unmatched Overview. We are seeking a highly experienced and innovative Senior Data Engineer with a strong background in hybrid cloud data integration, pipeline orchestration, and AI-driven data modelling. Requirements: - 9+ years of e...

Join our Transformation Liquidity Engineering team in Gurugram and collaborate with a peer network of technologists to play a key role in implementing critical liquidity calculations, creating data visualisations, and delivering data to downstream systems. At Macquarie, we bring together diverse people and empower them to shape various possibilities. As a global financial services group operating in 34 markets with 55 years of unbroken profitability, you will be part of a friendly and supportive team where everyone contributes ideas and drives outcomes regardless of their role. In this role, you will leverage your deep understanding of big data technologies and enthusiasm for DevOps practice...

We are seeking a SQL Expert with approximately 5 years of practical experience in creating, enhancing, and refining intricate SQL queries, stored procedures, and database solutions. Your primary responsibility will involve supporting our data-driven applications to ensure efficient data processing, performance optimization, and robust database design. In this pivotal role, you will collaborate closely with product, engineering, and analytics teams to deliver high-quality, reliable, and scalable data solutions. Your duties will encompass designing and implementing complex SQL queries, stored procedures, views, and functions, optimizing query performance, developing ETL/ELT pipelines for data ...

As a Data Engineer, you will be responsible for designing and developing scalable data engineering solutions using Google Cloud Platform (GCP) and PySpark. Your main focus will be on optimizing Spark jobs for performance, scalability, and efficient resource utilization. You will also be involved in developing, maintaining, and enhancing ETL pipelines using BigQuery, Apache Airflow, and Cloud Composer. Collaborating with data scientists, analysts, and DevOps teams to translate business requirements into technical solutions will be a key aspect of your role. Ensuring data integrity and security by implementing data governance, compliance, and security best practices will be crucial. Monitoring...

About the Role We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data platforms that empower data-driven decision-making. This role requires deep technical expertise in modern data engineering frameworks, architectural patterns, and cloud-native solutions on AWS. You will be a key contributor to our data strategy, ensuring data quality, governance, and reliability while mentoring other engineers in the team. Key Responsibilities Design, develop, and own robust, scalable, and maintainable data pipelines (batch & real-time). Architect and implement Data Lake, Data Warehouse, and Lakehouse solutions using modern frameworks and architectural patterns. E...

Key Responsibilities: Identify & analyze MIM workflows, Synchronization, policy rules (Management Policy Rules). Synchronization Service (FIM/MIM) extensively worked on the Metaverse extension code Deep understanding of synchronization rules & Troubleshoot MIM synchronization issues and ensure data integrity across connected systems. Collaborate with security, infrastructure, and application teams to define and enforce identity governance policies. Develop technical documentation, including architecture diagrams, configuration guides, and operational procedures. Support migration from MIM to other platforms. Experience on migrating from MIM synchronization service to other modern identity pl...

We are seeking an experienced Amazon Redshift Developer / Data Engineer to design, develop, and optimize cloud-based data warehousing solutions. The ideal candidate should have expertise in Amazon Redshift, ETL processes, SQL optimization, and cloud-based data lake architectures. This role involves working with large-scale datasets, performance tuning, and building scalable data pipelines. Key Responsibilities: Design, develop, and maintain data models, schemas, and stored procedures in Amazon Redshift. Optimize Redshift performance using distribution styles, sort keys, and compression techniques. Build and maintain ETL/ELT data pipelines using AWS Glue, AWS Lambda, Apache Airflow, and dbt. ...

We are seeking a highly skilled Senior Data Engineer to join our dynamic team in Bangalore. You will design, develop, and maintain scalable data ingestion frameworks and ELT pipelines using tools such as DBT, Apache Airflow, and Prefect. The ideal candidate will have deep technical expertise in cloud platforms (especially AWS), data architecture, and orchestration tools. You will work with modern cloud data warehouses like Snowflake, Redshift, or Databricks and integrate pipelines with AWS services such as S3, Lambda, Step Functions, and Glue. A strong background in SQL, scripting, and CI/CD practices is essential. Experience with data systems in manufacturing is a plus.

About Inspire Brands: Inspire Brands is looking for a Data Architect, a self-driven senior technical professional to collaborate with our business, Product and Development teams and help qualify, identify, and build Data & Analytics solutions. As part of the Enterprise Data Team, this position is responsible for actively defining robust, efficient, scalable and innovation solutions that are custom fit to our needs. Job Summary: Play a key role working with product and business teams to understand their strategic, business, and technical needs. Define technical architecture roadmaps which help realize business values in both tactical and strategic ways. Define and continuously update architec...

We are looking for an accomplished and hands-on Senior AI Lead to drive the development and deployment of AI-powered solutions for supply chain optimization. This senior technical leadership role requires deep AI/ML expertise, client-facing engagement, and team management to deliver intelligent, scalable products on the AWS cloud platform. You will be responsible for leading the end-to-end lifecycle of AI solutions that address real-world supply chain challenges such as demand planning and forecasting, inventory optimization, purchasing, fulfillment, route planning, risk prediction, and anomaly detection. Your key responsibilities will include designing, building, and deploying AI/ML models ...

As a Senior Database Administrator (DBA) specializing in Cloud & Big Data, you will be responsible for designing, managing, and optimizing large-scale, high-performance databases across cloud and on-premises environments. With 9+ years of experience, you will demonstrate expertise in SQL and NoSQL databases, including Couchbase, PostgreSQL, MySQL, Oracle, MongoDB, and DynamoDB. Additionally, familiarity with Databricks, Apache Spark, Python, R, and cloud-based data processing solutions is required. Your role will involve database performance tuning, replication strategies, high availability solutions, data pipeline management, and security enforcement. Experience in telecommunications data m...

As a Data Architect with over 6 years of experience, you will be responsible for designing and implementing modern data lakehouse architectures on cloud platforms such as AWS, Azure, or GCP. Your primary focus will be on defining data modeling, schema evolution, partitioning, and governance strategies to ensure high-performance and secure data access. In this role, you will own the technical roadmap for scalable data platform solutions, ensuring they are aligned with enterprise needs and future growth. You will also provide architectural guidance and conduct code/design reviews across data engineering teams to maintain high standards of quality. Your responsibilities will include building an...

You should have over ten years of experience in software development, including recent years in a leadership position with increasing responsibilities. Your expertise should cover various areas such as C#, .Net, designing and developing cloud-native solutions on platforms like Azure, AWS, and Google Cloud Platform, using Infrastructure as Code tools like Terraform and Pulumi, employing configuration management tools like Ansible and Chef, working with containerization and orchestration technologies like Docker and Kubernetes, and integrating with Native and third-party Databricks solutions. Additionally, you should possess extensive experience specifically in Azure. It would be advantageous ...

The role involves Java and Python development with a focus on Apache Airflow for workflow automation. You will be responsible for Python and Java Development, with proficiency in both languages being crucial for building Airflow. Your tasks will include using Airflow for workflow automation, encompassing designing, implementing, and maintaining Directed Acyclic Graphs (DAGs). Additionally, you will be managing task dependencies and configuring scheduling and error handling. An understanding of database systems, data warehousing concepts, and data architecture is essential for building robust data pipelines. To be successful in this role, you should have 5 to 8 years of experience in Java and...

As a Java Full Stack Lead Developer at Dedalus, you will utilize your extensive experience in modern web development, backend technologies, and cloud infrastructure to make a profound impact in the healthcare sector. You will take ownership of complex projects, mentor junior developers, optimize existing systems, and contribute to new feature development. Your role will involve designing and implementing backend solutions using Java and Python, with additional experience in PHP and the Laravel Framework being a plus. You will also be responsible for developing and maintaining scalable front-end applications using Angular, as well as leading development efforts and deployments using CI/CD pip...