Senior Data Engineer

Job Description: As a Senior Data Engineer at Inferenz, you will play a crucial role in designing, implementing, and optimizing data pipelines and workflows using Apache Airflow. Your responsibilities will include developing incremental and full-load strategies with monitoring, retries, and logging, building scalable data models and transformations in dbt, and developing and maintaining data warehouses in Snowflake. You will ensure data quality, integrity, and reliability through validation frameworks and automated testing, and tune performance through clustering keys, warehouse scaling, materialized views, and query optimization. Additionally, you will monitor job performance, resolve data pipeline issues proactively, and build and maintain data quality frameworks. Key Responsibilities: - Design, implement, and optimize data pipelines and workflows using Apache Airflow - Develop incremental and full-load strategies with monitoring, retries, and logging - Build scalable data models and transformations in dbt, ensuring modularity, documentation, and test coverage - Develop and maintain data warehouses in Snowflake - Ensure data quality, integrity, and reliability through validation frameworks and automated testing - Tune performance through clustering keys, warehouse scaling, materialized views, and query optimization - Monitor job performance and resolve data pipeline issues proactively - Build and maintain data quality frameworks (null checks, type checks, threshold alerts) - Partner with data analysts, scientists, and business stakeholders to translate reporting and analytics requirements into technical specifications Qualifications: - Proficiency in Snowflake (data modeling, performance tuning, access control, external tables, streams & tasks) - Experience with Apache Airflow (DAG design, task dependencies, dynamic tasks, error handling) - Proficiency in dbt (Data Build Tool) including modular SQL development, jinja templating, testing, and documentation - Strong skills in SQL, Spark, and Python - Experience building data pipelines on cloud platforms like AWS, GCP, or Azure - Strong knowledge of data warehousing concepts and ELT best practices - Familiarity with version control systems like Git and CI/CD practices - Familiarity with infrastructure-as-code tools like Terraform for provisioning Snowflake or Airflow environments - Excellent problem-solving skills and the ability to work independently - Ability to work collaboratively in a team environment About Uplers: Uplers" goal is to make hiring reliable, simple, and fast. They aim to help all talents find and apply for relevant contractual onsite opportunities and progress in their careers. Uplers offer support for any grievances or challenges faced during the engagement. Numerous opportunities are available on the portal apart from this specific role, allowing candidates to apply based on assessments cleared. If you are prepared for a new challenge, a great work environment, and an opportunity to advance your career, don't hesitate to apply today. Uplers is eagerly waiting for you!,