Data Engineer- Night Shift

WORK YOULL DO

As a Data Engineer within Deloitte’s Customer Strategy & Design (CS&D) practice, you will design, build, and optimize large-scale data ingestion, transformation, and orchestration systems that support analytics, AI, and digital transformation initiatives. You will play a critical role in building resilient data ecosystems, driving automation, and enabling actionable insights for global clients.

Data Pipeline Development & Architecture

• Design, build, and optimize ETL/ELT pipelines using Python/PySpark/SQL on distributed data platforms.
• Engineer streaming and batch workflows using Azure Data Factory/Databricks/Apache Spark/Snowflake.
• Implement ingestion frameworks for structured, semi-structured, and unstructured data (APIs/Kafka/SFTP/Delta Lake).
• Design scalable Lakehouse architectures integrating Delta Lake/Azure Synapse/AWS Glue.

Cloud & Big Data Platforms

• Deploy and maintain data pipelines on Azure/AWS/GCP using Data Factory/Glue/Dataflow/BigQuery/Redshift.
• Leverage Databricks/Spark SQL/Hadoop/Hive/Presto for distributed processing.
• Work with Snowflake/Azure Synapse/BigQuery for data warehousing and analytics.
• Implement partitioning, clustering, and indexing for optimal performance.

Data Modelling & Management

• Design robust data models (3NF/dimensional/data vault/star/snowflake).
• Implement metadata management, lineage, and cataloging with Apache Atlas/Purview/Collibra.
• Enforce data governance and compliance (GDPR/HIPAA/CCPA).
• Ensure data quality using Great Expectations/Deequ/Monte Carlo.

Orchestration, Automation & CI/CD

• Automate pipelines using Apache Airflow/dbt/Azure Synapse Pipelines.
• Implement CI/CD with Azure DevOps/GitHub Actions/Jenkins.
• Use Terraform/Ansible for Infrastructure-as-Code (IaC).
• Build observability dashboards using Prometheus/Grafana/Datadog.

Streaming & Real-Time Processing

• Design and deploy real-time pipelines using Apache Kafka/Kinesis/Azure Event Hubs.
• Implement streaming ETL with Spark Structured Streaming/Kafka Streams/Flink.
• Integrate near real-time data into analytics and ML workflows.

Collaboration & Documentation

• Collaborate with data scientists, architects, analysts, and product teams to translate business needs into engineering solutions.
• Document architecture, logic, and lineage for audit and reproducibility.
• Contribute to internal knowledge sharing and reusable frameworks.

If you are open to working night shifts, please let our Talent team know during the hiring process.

`

KEY AREAS OF EXPERTISE

• Languages & Frameworks: Python/PySpark/Scala/SQL/Java
• Big Data & Cloud: Hadoop/Spark/Databricks/Snowflake/Synapse/BigQuery/Redshift
• ETL & Orchestration: Azure Data Factory/Apache Airflow/dbt/NiFi/Informatica
• Data Modeling: Kimball/Inmon/Data Vault/Star/Snowflake
• Data Quality & Governance: Great Expectations/Apache Atlas/Collibra/Purview
• Visualization Tools: Power BI/Tableau/Looker/Fabric