Data Platform Engineer

Job Role -

Location

Experience

• Should have good knowledge on data warehouse and lakehouse concepts
• Working knowledge on Data technologies tools such as Azure Databricks, Azure ML, Azure Datafactory, Azure Datalake storage
• Service provisioning through terraform across the environment
• Deployment of application on AKS and REDHAT open shift clusters, managing the dockers and monitor the containerize applications
• Hands on experience on Azure Databricks and Azure AKS administration. Good working knowledge on PAAS offering such as ADLS Gen2, ADF, Function App, logic App, MS SQL, Postgre SQL (both IAAS and PAAS)
• Good understanding on ETL framework for batch and micro batches
• Should be able to create new custom role as per Banks IAM policy and segregate the RBAC for data analyst & data engineers
• Collaborate with development and operations teams to define and implement Azure infrastructure requirements for applications, including networking, security, and scalability.
• Configure and optimize Azure services, including virtual machines, virtual networks, load balancers, VPN gateways, ExpressRoute, Azure Firewall, and Azure DNS, to ensure optimal performance, security, and availability.
• Monitor and troubleshoot issues in Azure environment, utilizing Azure monitoring tools, log analytics, and other monitoring solutions to proactively identify and resolve issues.
• Implement and manage Azure security measures, including Azure AD authentication, Azure AD roles and permissions, network security groups (NSGs), firewall rules, and encryption, to protect Azure resources and data.
• Automate deployment and management of Azure resources using Infrastructure-as-Code (IaC) tools, such as Azure Resource Manager (ARM) templates, PowerShell, Azure CLI, and Azure DevOps.
• Collaborate with cross-functional teams, including development, operations, and security, to ensure smooth integration and operation of Azure services and solutions.
• Stay up-to-date with the latest Azure service features, updates, and best practices, and provide recommendations for improving Azure infrastructure and operations.
• Provide technical expertise and guidance to other team members and stakeholders on Azure platform and infrastructure topics.
• Design, implement, and manage end-to-end CI/CD pipelines using Azure DevOps, including source code management, build automation, release automation, testing, and deployment.
• Collaborate with development, operations, and quality assurance teams to define and implement DevOps practices and tools to improve software development and deployment processes.
• Configure and manage Azure DevOps services, such as Azure Repos, Azure Pipelines, Azure Artifacts, Azure Boards, and Azure Test Plans, to ensure optimal performance, security, and availability.
• Implement and automate infrastructure provisioning and deployment using Infrastructure-as-Code (IaC) tools, such as Azure Resource Manager (ARM) templates, Terraform, or Ansible, in conjunction with Azure DevOps pipelines

Key Skills-

• Experienced technology engineer with a minimum of 5+ years of data tools implementation and data platform handling experience with deep technology expertise
• Understanding of Lakehouse and delta lake architecture
• Having knowledge of ETL/ELT and data warehouse concepts
• Experience to SNOW to raise change as part of change management process
• Should handle technology obsolescence, baseline, platform related audit etc
• Raising TCO for cloud infra
• Responsible of new integration to the data platform
• Closure of platform vulnerabilities and responsible of the security as per Banks standard
• Understanding of cloud architecture and deployment models (IAAS,PAAS,SAAS)
• Experience with Azure DevOps services (e.g. pipelines, boards, repos , artifacts)
• Hands on experience on automation tools like terraform, ARM templates, bicep for infrastructure as code (Iac)
• Knowledge of languages like PowerShell, python, bash or shell for scripting and automation