Azure Data Engineer

Job Title

Location

Skills Required: Digital : Databricks, Azure Data Factory

Experience Range in Required Skills: 10 & Above Years (Relevant should be 8 + yrs.)

• Deep Microsoft Azure product knowledge and understanding of product features.
• Proven track record in designing, developing and implementing data modelling, at a conceptual, logical and physical level.
• Experience with accountability for E2E Data Architecture and Lifecycle (including non-functional requirements and operations) management.
• Understands industry recognised data modelling patterns and standards
• Understands overall IT system design, in particular networking, authorization and authentication protocols, data security, disaster recovery.
• Knowledge of the latest technologies to process large amount of data (Apache Spark; Databricks is a plus)
• Experience in MDM, Metadata Management, Data Quality and Data Lineage tools.
• Expertise in cloud security and identity best practices
• A deep understanding of ETL and ELT tools and techniques
• Exposure to other cloud solutions (Google, AWS) a plus.
• Strong experience in architecting fit-for-purpose and fit-for-use complex solutions on the Microsoft Azure cloud stack [IaaS, PaaS and SaaS]
• Very strong hands-on experience in using Microsoft Azure services like Azure Data Factory, Data brick services (like processing streaming data using Spark clusters), usage of Blob containers, ESB, Event Grid, Azure SQL server, Cosmos DB, Azure functions, Analytics (like Power BI) is a mandatory requirement.
• Excellent understanding of architecture patterns and best practices with proven innovative and leading-edge thinking to build cloud-ready systems (e.g., Domain Driven Architecture, multi-tenancy, building for resilience, scalability, performance, and federation, microservices etc.)
• Hands-on work experience on application platforms like .Net, Python, Java etc. preferred
• Understanding of big data platforms concepts and NoSQL technologies, and ability to map such
• The candidate should have experience in modeling data for traditional structured data, as well as information design in Data Lake and NoSQL database engines.
• Fluent with modelling, development and project execution processes around star, snowflake, 3NF, and Data Vault approaches.
• Good experience in working with data using SQLs, Python and Scala in Microsoft based resources like Azure Synapse and Data Lake.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.