Azure Databricks design and architecture - Senior Associate

As an Azure Senior Developer at PwC - AC, your role involves collaborating with Offshore Manager/Onsite Business Analyst to comprehend requirements and taking charge of end-to-end execution of Cloud data engineering solutions. Your responsibilities include leveraging Azure cloud expertise, particularly in Storage services (Blob, ADLS, PaaS SQL), Azure Data factory, and Azure Synapse. You are expected to excel in planning and organization, functioning as a cloud developer/lead in an agile team, delivering automated cloud solutions. **Key Responsibilities:** - Proficient in Azure cloud computing, encompassing big data technologies. - Deep understanding of traditional and modern data architecture and processing concepts, covering relational databases, Data warehousing, big data, NoSQL, and business analytics. - Hands-on experience in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services. - Building stream-processing systems utilizing solutions like Storm or Spark-Streaming. - Designing and implementing scalable ETL/ELT pipelines using Databricks and Apache Spark. - Optimizing data workflows to ensure efficient data processing. - Familiarity with big data use-cases and Lambda/Kappa design patterns. - Demonstrated experience in implementing Big Data solutions using Microsoft Data Platform and Azure Data Services. - Exposure to Open-Source technologies such as Apache Spark, Hadoop, NoSQL, Kafka, Solr/Elastic Search. - Well-versed with quality processes and implementation. - Proficiency in Application DevOps tools like Git, CI/CD Frameworks, Jenkins, or Gitlab. - Excellent communication and presentation skills. **Qualifications Required:** - BE / B.Tech / MCA / M.Sc / M.E / M.Tech / MBA. - Good analytical & problem-solving skills. Please note that the additional details of the company were not present in the provided job description.,