Data Engineer

0 years

0 Lacs

Posted:1 month ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Data Engineer to design, develop, and maintain data pipelines and ETL workflows for processing large-scale structured and unstructured data. The ideal candidate will have expertise in Azure Data Services (Azure Data Factory, Synapse, Databricks, SQL, SSIS, and Data Lake) along with big data processing, real-time analytics, and cloud data integration. Key Responsibilities Data Pipeline Development & ETL/ELT Design and build scalable data pipelines using Azure Data Factory, Synapse Pipelines ,Databricks, SSIS and ADF Connectors like Salesforce. Implement ETL/ELT workflows for structured and unstructured data processing. Optimize data ingestion, transformation, and storage strategies. Cloud Data Architecture & Integration Develop data integration solutions for ingesting data from multiple sources (APIs, databases, streaming data). Work with Azure Data Lake, Azure Blob Storage, and Delta Lake for data storage and processing. Database Management & Optimization Design and maintain cloud data bases (Azure Synapse, BigQuery, Cosmos DB). Optimize SQL queries and indexing strategies for performance. Implement data partitioning, compression, and caching for efficiency. Data Governance, Security & Compliance Ensure data quality, lineage, and governance with tools like Purview. Implement role-based access control (RBAC), encryption, and security policies. Ensure compliance with GDPR, HIPAA, and ISO 27001 regulations. Monitoring & Performance Tuning Use Azure Monitor, Log Analytics, and OpenTelemetry to track performance and troubleshoot data issues. Automate data pipeline testing and validation. Collaboration & Documentation Document data models, pipeline architectures, and data workflows. Technical Skills Cloud Data Services: Azure Data Factory, Azure Synapse, Databricks, SSIS, BigQuery. ETL & Data Pipelines: Apache Spark, Python, SQL. Big Data Processing: Delta Lake, Parquet, Hadoop, Event Hubs. Database Management: SQL Server, Cosmos DB. Security & Compliance: RBAC, Data Masking, Encryption, Purview. Scripting & Automation: Python, PowerShell, Terraform for IaC. Show more Show less

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You

Chennai, Tamil Nadu, India

Pune, Maharashtra, India

Pune, Maharashtra, India

Gurugram, Haryana, India

Noida, Uttar Pradesh, India

Indore, Madhya Pradesh, India

Hyderabad, Telangana, India