Data Engineer

Role Overview: You will be responsible for designing and implementing scalable and secure cloud architecture on cloud platforms like AWS, Azure, and GCP. Your expertise in building enterprise-grade data platforms using technologies such as Databricks, Snowflake, Spark, and Data Lake/Warehouse solutions will be crucial in supporting advanced analytics and AI/ML integrations. Additionally, you will conduct research and development for custom architecture PoCs, including AI/ML model deployment, security enhancements, and scalable data workflows to meet evolving business needs. Key Responsibilities: - Design and deploy data pipelines with technologies like AWS (S3, Lambda, Glue, Data Pipeline), Databricks, and Snowflake for real-time streaming, batch processing, and analytics workloads. - Provision cloud infrastructure using Terraform, CloudFormation, and Control Tower to ensure multi-AZ, high availability, and multi-tenant architectures. - Implement hybrid cloud architecture for cloud-to-cloud integrations, cross-cloud data migration, and multi-cloud orchestration involving AWS/GCP or Azure, and on-prem systems. - Design and implement CI/CD pipelines using CloudFormation, Jenkins, Git, Terraform to support automated deployment of applications and infrastructure. - Develop SaaS solutions with secure management provisioning, license management, and custom deployment architectures. - Utilize monitoring and logging tools like Prometheus, Grafana, Elasticsearch, Kibana to ensure performance optimization, fault detection, and quick bug resolution. - Design, develop, and optimize data pipelines and implement data warehousing solutions using platforms like AWS, Snowflake, Databricks. - Create scalable, distributed data processing systems using Apache Spark, Hadoop, or similar frameworks. - Develop and maintain logical and physical data models to support analytics, reporting, and ML use cases. - Collaborate with Data Scientists, Analysts, and Business Teams to support data-driven decision-making and provide guidance on data best practices, performance tuning, and data-driven solutions. - Mentor and train junior data engineers and contribute to team growth. Qualifications Required: - 8 years of experience as a Cloud Architect with proficiency in cloud platforms AWS, GCP, and Azure. - Hands-on experience with Data Platforms & Tools such as Databricks, Snowflake, Hadoop, Spark, Data Lake, and Data Warehouse. - Expertise in DevOps & Automation tools like Terraform, CloudFormation, Kubernetes, Docker, and CI/CD pipelines. - Proficiency in Monitoring & Logging tools including Prometheus, Grafana, ELK Stack. - Strong programming and scripting skills in Python, Java, Bash, and SQL. - Knowledge of Security & Compliance standards such as SOC2, HIPAA, audit logs, and security controls.,