Data Engineering

As a Data Engineering Senior Associate at Microsoft Fabric, Azure (Databricks & ADF), PySpark, you will be responsible for designing and implementing scalable data solutions within the Microsoft Azure ecosystem. With 4-10 years of experience in the field, you will leverage your expertise in Microsoft Fabric, Azure Databricks, PySpark, Python, and SQL to build end-to-end data pipelines, ensuring efficient data processing and extraction. Your primary responsibilities will include gathering requirements, designing and implementing data pipelines using Microsoft Fabric & Databricks, performing ETL operations to extract data into Azure Data Lake Storage, implementing data security measures, monitoring pipeline performance, and troubleshooting data engineering issues. Additionally, you will work with a variety of data sources like Relational DB, API, File System, Realtime streams, and CDC. The ideal candidate will possess hands-on experience in Microsoft Fabric, Azure Databricks, and proficiency in PySpark for data processing. A strong command over Python and SQL is essential for writing complex queries, optimizing performance, and ensuring data integrity. Experience with Azure Data Lake Storage, Data Warehouse concepts, and familiarity with Databricks and Delta tables will be crucial for success in this role. Furthermore, you should have a good understanding of CI/CD practices in a data engineering context, along with excellent problem-solving and communication skills. Exposure to BI tools like Power BI, Tableau, or Looker will be beneficial. Experience with Azure DevOps, data security, and compliance in the cloud, as well as different databases like Synapse, SQL DB, and Snowflake, will be considered advantageous. If you are a data engineering professional looking to work on challenging projects within a dynamic environment, this role offers an exciting opportunity to showcase your skills and contribute to the development of cutting-edge data solutions.,