Data Engineer

Infiniti Research

2 - 5 years

20 - 25 Lacs

bengaluru

Posted:-1 days ago| Platform:

Apply

Skills Required

azure data factory pyspark unity catalog data bricks

Work Mode

Work from Office

Job Type

Full Time

Job Description

Experience:2-5 years

Location: Bengaluru

Role Overview:

We are seeking a Data Engineer to support the design, development, and implementation of our Data Observability & Monitoring solution. The engineer will build data pipelines to capture metrics from Azure Monitor, Databricks, Purview, and other APIs, and integrate them into Delta Lake for reporting via Power BI/React dashboards.

Responsibilities:

Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake).
Integrate data from multiple APIs (Azure Monitor, Azure Cost Management, Log Analytics, Purview).
Implement ETL/ELT processes for structured/unstructured data.
Ensure data quality, reliability, and observability throughout the pipeline.
Optimize pipeline performance (cost, CPU, memory usage).
Collaborate with frontend developers to expose APIs for dashboards.
Write documentation and maintain version control (Git/DevOps).

Required Skills:

Strong experience with
Azure Data Services
(Databricks, ADLS, Synapse optional).
Knowledge of
Delta Lake, PySpark, SQL
.
Experience with
API integration
(REST APIs, JSON, authentication).
Familiarity with
Azure Monitor, Cost Management, Log Analytics
.
CI/CD experience (Azure DevOps preferred).
Strong debugging and performance optimization skills.

Preferred:

Exposure to
data observability tools
(e.g., Monte Carlo, Datadog, OpenTelemetry).
Knowledge of
Unity Catalog, Purview
for governance.

More Jobs at Infiniti Research

Senior Data Scientist

Bengaluru

5 - 7 yrs

INR 30 - 35 Lacs

R Developer

Bengaluru

3 - 6 yrs

INR 25 - 27 Lacs

Ml Engineer

Bengaluru

4 - 8 yrs

INR 25 - 30 Lacs

Senior Business Analyst

Bengaluru

4 - 8 yrs

INR 25 - 30 Lacs

Senior Data Scientist

Bengaluru

5.0 - 7.0 yrs

INR 30 - 35 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.