Senior Data Engineer

As a Senior Data Engineer at Milestone Technologies, you will play a crucial role in leading data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. Your responsibilities will include: - Building and managing high-volume data pipelines using Spark/Databricks. - Implementing ELT frameworks using Azure Data Factory/Synapse Pipelines. - Optimizing large-scale datasets in Delta/Iceberg formats. - Implementing robust data quality, monitoring, and governance layers. - Collaborating with Data Scientists, Analysts, and Business stakeholders. Your technical stack will involve: - Big Data: Apache Spark, Kafka, Hive, Airflow, Hudi/Iceberg. - Cloud: Azure (Synapse, ADF, ADLS Gen2), Databricks, AWS (Glue/S3). - Languages: Python, Scala, SQL. - Storage Formats: Delta Lake, Iceberg, Parquet, ORC. - CI/CD: Azure DevOps, Terraform (infra as code), Git. Milestone Technologies is committed to creating a workplace that champions diversity and inclusion, where all employees feel empowered to bring their full selves to work. The company provides equal employment opportunities for all applicants and employees. Join Milestone Technologies and be part of a global IT managed services firm that focuses on employee-first culture, innovative solutions, and continuous service improvement to deliver transformational benefits to clients.,