Data Engineer - SQL & Python

The ideal candidate for the SE/ Senior Data Engineer position should possess deep experience in ETL/ELT processes, data warehousing principles, and real-time and batch data integrations. As a senior member of the team, you will play a crucial role in mentoring and guiding junior engineers, defining best practices, and contributing to the overall data strategy. Strong hands-on experience in SQL, Python, and ideally Airflow and Bash scripting is essential for this role. Key Responsibilities: - Architect and implement scalable data integration and data pipeline solutions using Azure cloud services. - Design, develop, and maintain ETL/ELT processes, including data extraction, transformation, loading, and quality checks using tools like SQL, Python, and Airflow. - Build and automate data workflows and orchestration pipelines, with knowledge of Airflow or equivalent tools being a plus. - Write and maintain Bash scripts for automating system tasks and managing data jobs. - Collaborate with business and technical stakeholders to understand data requirements and translate them into technical solutions. - Develop and manage data flows, data mappings, and data quality & validation rules across multiple tenants and systems. - Implement best practices for data modeling, metadata management, and data governance. - Configure, maintain, and monitor integration jobs to ensure high availability and performance. - Lead code reviews, mentor data engineers, and help shape engineering culture and standards. - Stay current with emerging technologies and recommend tools or processes to improve the team's effectiveness. Required Qualifications: - Bachelors or Masters degree in Computer Science, Information Systems, or related field. - 3+ years of experience in data engineering, with a strong focus on Azure-based solutions. - Proficiency in SQL and Python for data processing and pipeline development. - Experience in developing and orchestrating pipelines using Airflow (preferred) and writing automation scripts using Bash. - Proven experience in designing and implementing real-time and batch data integrations. - Hands-on experience with Azure Data Factory, Azure Data Lake, Azure Synapse, Databricks, or similar technologies. - Strong understanding of data warehousing principles, ETL/ELT methodologies, and data pipeline architecture. - Familiarity with data quality, metadata management, and data validation frameworks. - Strong problem-solving skills and the ability to communicate complex technical concepts clearly. Preferred Qualifications: - Experience with multi-tenant SaaS data solutions. - Familiarity with DevOps practices, CI/CD pipelines, and version control systems (e.g., Git). - Experience mentoring and coaching other engineers in technical and architectural decision-making.,