Senior Data Engineer

Position                : Data Engineer (Senior/Junior)

Experience          : 4 Years 10 Years

Job location        : Hyderabad/Gurugram

Working Mode : Hybrid(4 Days WFO)

Shifts                     : 12.30 PM 9.30 PM

Technical Skills (Required):

• Azure Data Factory (ADF): design, develop and orchestrate scalable data pipelines
  
• Azure Databricks: build and manage Spark clusters, notebooks, and jobs
  
• Microsoft Fabric (OneLake): ingest, organize and transform data within the Fabric ecosystem
  
• PySpark & SparkSQL: author complex transformations and SQL queries for large scale data processing
  
• Delta Tables: create, optimize and maintain Delta Lake tables (VACUUM, ZORDER, OPTIMIZE)
  
• JSON Processing: parse, validate and transform semi structured JSON datasets
  
• API Integration: consume and build REST/OData services for custom data ingestion
  
• Medallion Architecture: implement bronze silver gold layers in data lakes for clean, reliable data
  
• Large Scale Performance Optimization: partitioning, caching, resource tuning to efficiently process high volumes of data
  
• Multidimensional Modeling: design star/snowflake schemas, fact and dimension tables for reporting use cases
  
• Azure Analysis Services & Cubes: understand tabular and OLAP cube structures to enable downstream BI
  
• DevOps Collaboration: partner with the DevOps team to define infrastructure, manage access/security, and automate deployments
  

Technical Skills (Good to Have):

• SAP Integration via Data Sphere: ingest and harmonize data from SAP ECC/S4HANA systems
  
• CI/CD & DevOps Tooling: Git, Azure DevOps Pipelines, Terraform or ARM templates
  
• Data Observability & Monitoring: leverage Azure Monitor, Log Analytics, and data pipeline metrics
  
• Performance Troubleshooting: query diagnostics, bottleneck analysis, and root cause identification
  
• Data Governance & Cataloging: apply metadata management, data lineage, and compliance best practices
  
• Confluence: for documentation