Data Engineer

The Landmark Group

Job Purpose

Azure Data Engineer

As a Data Engineer, you will take a pivotal role in designing, deploying, and optimizing data solutions leveraging Azure's powerful ecosystem. Your expertise will drive the development of efficient data pipelines using Azure Data Factory, ensuring data quality and management in Azure Data Lake. Additionally, you'll utilize your proficiency in complex stored procedures within Azure Databricks or Synapse. Collaborating cross-functionally, you'll contribute to data-driven initiatives, harnessing Azure Synapse Analytics to provide insightful intelligence that shapes informed decision-making."

• Responsible for the design, implementation, and optimization of data solutions as an expert Azure Data Engineer with 6-8 years of experience.
• Proficiency in programming languages like Python, SQL, or Scala for data manipulation, transformation, and analysis
• Expertise in data modelling, data warehousing, and data integration concepts.
• Sound knowledge of data governance practices, data quality management, and data security principles.
• Develop efficient ETL pipelines using Azure Data Factory and ensure data quality and storage via Azure Data Lake.
• Utilize your proficiency in complex stored procedures within Azure Databricks to enable advanced analytics on data volumes exceeding 50TB.
• Collaborate cross-functionally to contribute to data-driven initiatives, leveraging Azure Synapse Analytics to provide actionable insights.
• Apply your deep understanding of the retail domain to enhance data strategies and solutions for retail-specific challenges.
• Play a pivotal role in shaping our organization's data-driven journey, driving innovation through data analytics and insights.
• Optimize data storage, processing and retrieval mechanisms for performance, cost, and scalability using data storage services (such as Azure Data Lake Storage, Azure SQL Database, etc.), data processing services (such as Azure Data Bricks, Azure Synapse, etc.) and data visualization (PowerBI, Qlik, etc.) & integration services (Data API builder, logic apps, etc.)
• Monitor and troubleshoot data platform performance, identify and resolve issues, and provide recommendations for continuous improvement.
• Collaborate with DevOps teams to automate deployment, configuration, and monitoring processes using Azure DevOps, PowerShell, or other relevant tools.
• Stay up to date with the latest trends and advancements in cloud data services and provide recommendations on adopting new technologies or features to enhance the data platform.
• Document technical designs, procedures, and guidelines for data platform engineering and operations

Knowledge, Skills & Experience

Relevant Job Experience

• Bachelor's degree in computer science, Engineering, or a related field. Advanced degree preferred.
• Proven 6-8 years' experience in designing, implementing solutions on the Azure ecosystem including expertise in Azure services such as Azure Data Factory, Azure Databricks, Azure Machine Learning, Azure Data Lake Storage, and Azure SQL Database, Azure Synapse, PowerBI etc...
• Experience with big data technologies such as Apache Spark, Hadoop, or similar frameworks.
• Solid understanding of cloud computing concepts and experience with cloud infrastructure management and provisioning.
• Solid understanding of network security concepts and technologies (such as firewalls, VPNs, intrusion detection/prevention systems, etc.) and data security concepts and technologies (such as access controls, encryption, observability, privacy laws/regulations, etc.)
• Experience in a Retail setup is preferred
• Azure certifications such as Azure Data Engineer Associate or Azure Solutions Architect Expert is preferred

Required Leadership Skills

The position will require someone with the following:

• Strategic Planning
• Communication and Collaboration
• Problem Solving Skills