Staff Cloud Data Platform Engineer

Job Description

Lead the design and implementation of enterprise-level scalable, secure, and high-performance cloud data storage solutions to meet business needs and data growth. Architect and oversee the development of complex Azure Data Factory workflows, Data Lake Storage, and secure cloud integration using Azure Private Link.Design and implement scalable, secure, and high-performance data storage solutions in the cloud, ensuring optimal accessibility and reliability. Develop zone-redundant and multi-region data storage and data warehouse (DW) solutions on Azure, along with the design and implementation of storage lifecycle policies. Integrate cloud platforms (Azure or AWS) with third-party SaaS data platforms like Snowflake, DataIku, Fivetran, DBT Cloud, etc, ensuring seamless data flow, secure networking, and Single Sign-On (SSO) integration. Securely integrate and manage data and applications between ADI s data centers and various cloud environments. Ensure compliance with data governance policies, including data encryption, access control, and other security measures. Monitor, analyze, and improve cloud data systems performance at scale, identifying optimization opportunities and implementing performance tuning practices across the infrastructure. Deploy Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and serverless solutions on Azure and AWS. Utilize Terraform, ARM, and CloudFormation Templates (CFT) to implement Infrastructure as Code (IaC) for deployments in Azure and AWS.Design and development of data pipelines, automation, and visualization tools to enable modern Data and analytics solutions. Partner with the data platform and infrastructure teams to architect and connect high quality, resilient data feeds from across the enterprise. Manage the Data platform from change management, availability and troubleshooting perspective. Ability to automate deployments and maintain high levels of uptime. Qualifications: 8+ years of experience in software engineering, including 5+ years working with data engineering technologies and cloud technologies. Degree in Computer Science, Electrical Engineering, Computer Engineering, or a related field. Strong expertise in Azure Data Factory, Data Lake Storage, and leveraging Azure Private Link for secure, enterprise-level cloud integrations. Exposure to Azure Data Fabric or Databricks Hands-on knowledge of Azure, AWS and SaaS based data platforms. Proven experience to deploy, manage and support cloud-based data platforms. Exceptional problem-solving skills, able to troubleshoot and optimize cloud infrastructure and data systems. In-depth understanding of data governance, security protocols, compliance frameworks, and implementing encryption and access control measures. Experience with automation and CI/CD pipelines in cloud environments. Experience in leading cross-functional teams, collaborating with business stakeholders, and translating technical requirements into scalable solutions. Experience with advanced monitoring and logging tools such as Azure Insights, AWS CloudWatch, and Datadog.