Lead Data Engineer

As a Technical Lead with over 8 years of experience in Data Engineering, Analytics, and Python development, including at least 3 years in a Technical Lead / Project Management role, you will play a crucial role in driving data engineering and analytics projects for our clients. Your client-facing skills will be essential in ensuring successful project delivery and effective communication between technical and business stakeholders. Your responsibilities will include designing and implementing secure, scalable data architectures on cloud platforms such as AWS, Azure, or GCP. You will lead the development of cloud-based data engineering solutions covering data ingestion, transformation, and storage while defining best practices for integrating diverse data sources securely. Overseeing security aspects of integrations and ensuring compliance with organizational and regulatory requirements will be part of your role. In addition, you will develop and manage robust ETL/ELT pipelines using Python, SQL, and modern orchestration tools, as well as integrate real-time streaming data using technologies like Apache Kafka, Spark Structured Streaming, or cloud-native services. Collaborating with data scientists to integrate AI models into production pipelines and cloud infrastructure will also be a key aspect of your responsibilities. Furthermore, you will work on advanced data analysis to generate actionable insights for business use cases, design intuitive Tableau dashboards and data visualizations, and define data quality checks and validation frameworks to ensure high-integrity data pipelines. Your expertise in REST API development, backend services, and integrating APIs securely will be crucial in developing and deploying data products and integrations. To excel in this role, you must have deep hands-on experience with cloud platforms, expertise in Python, SQL, Spark, Kafka, and streaming integration, proven ability with data warehousing solutions like BigQuery, Snowflake, and Redshift, and a strong understanding of integration security principles. Proficiency in data visualization with Tableau, REST API development, and AI/ML integration will also be essential. Preferred qualifications include prior experience managing enterprise-scale data engineering projects, familiarity with DevOps practices, and understanding of regulatory compliance requirements for data handling. Your ability to lead technical teams, ensure project delivery, and drive innovation in data engineering and analytics will be key to your success in this role.,