Data Engineer

Azure Data Engineer

Data Engineers will be responsible for developing and maintaining the data pipelines that populate and process data within the existing Azure Data Lake They will need to ensure that data is properly ingested transformed and made accessible for analytics reporting and realtime insights

Key Skills
Azure Data Factory ADF Proficiency in building data pipelines using Azure Data Factory particularly for ETLELT processes from D365 FO and other systems to the data lake o Linked Services Connecting ADF with D365 FO other databases and external sources o Data Flows Implementing complex data transformations using ADFs data flow capabilities RealTime Data Integration Experience with realtime data ingestion and processing using tools like Azure Stream Analytics especially for pushing data into the data lake in nearrealtime Databricks or Synapse Analytics Knowledge of using Azure Databricks or Synapse for data processing transformation and analytics within the data lake environment This includes working with largescale distributed data processing Data Storage Optimization Skills in optimizing data storage within the data lake including choosing the right storage tiers eg hot cool archive and compressing large datasets Data Lake Delta Lake File Formats Familiarity with working with optimized file formats such as Parquet Avro or Delta Lake for efficient querying and data storage Power BI service for scheduled refreshes Azure Data Factory for orchestrating data refreshes Monitoring and Performance Tuning Experience with monitoring data pipelines troubleshooting performance bottlenecks and optimizing for costeffective usage of Azure resources

Kindly share your resume to Jyoti.dhabadi@ltimindtree.com/ Dhanasurya.Velusamy@ltimindtree.com