Senior Data Architect

Role & responsibilities

• minimum 4 yrs. of relevant hands-on experience in any of the two or more Azure Services like: DataLake, ADB, ADF, ADLS, Functions, Azure Synapse, Azure Cosmos DB, Azure DB for PostgreSQL, AKS, Azure DevOps etc
• Experience into Pyspark; Mastery of DataFrame APIs and Spark SQL
  - Performance tuning: broadcast joins, caching strategies, shuffle reduction - Handling skewed data and optimizing wide transformations - Writing scalable, testable PySpark code with modular design
• Good knowledge into Data BRicks; Cluster sizing, autoscaling, and spot instance optimization
  - Job orchestration using Workflows with retries, alerts, and dependencies - Delta Lake: - OPTIMIZE with Z-Ordering - MERGE operations and schema evolution - Time travel and versioning - Use of Unity Catalog for data governance and access control - Monitoring via Databricks metrics and audit logs