Data Engineer

Experience :6 - 8 Years

Key Responsibilities:

Design and Develop Data Solutions:

Build scalable data pipelines and workflows using Databricks and Azure services.

Leverage low-code/no-code tools (e.g., Power Apps, Azure Data Factory, Logic Apps) for rapid solution development.

Data Integration:

Integrate data from multiple sources (structured, semi-structured, unstructured) into Databricks.

Performance Optimization:

Optimize Spark jobs and workflows for cost and performance efficiency.

Collaboration:

Work closely with data analysts, business users, and citizen developers to enable self-service analytics.

Automation:

Implement automation for data ingestion, transformation, and deployment using low-code/no-code frameworks.

Governance & Security:

Ensure compliance with data governance, security, and privacy standards.

Required Skills & Qualifications:

Technical Expertise:

Hands-on experience with Databricks, Apache Spark, and Azure ecosystem (Data Lake, Synapse, ADF).

Proficiency in Python, SQL, and familiarity with Delta Lake.

Low-Code/No-Code Experience:

Experience with platforms like Power Apps, Power Automate, Azure Logic Apps, or similar.

Ability to design workflows and integrate APIs using low-code tools.

Data Engineering Fundamentals:

Strong understanding of ETL/ELT processes, data modeling, and performance tuning.

Soft Skills:

Excellent communication and collaboration skills.

Ability to work in agile environments and manage multiple priorities.

Preferred Qualifications:

Experience with CI/CD pipelines for Databricks.

Knowledge of MLflow and machine learning workflows.

Familiarity with data governance tools like Azure Purview