Senior Data Engineer

codinglimits

7 years

0 Lacs

hyderabad telangana india

Posted:3 weeks ago| Platform:

Apply

Skills Required

data design etl databricks postgresql dynamodb logic analytics reporting pipeline storage indexing query checks backup strategies governance pyspark programming sql python azure aws processing json xml pdf latency spark automation versioning security

Work Mode

On-site

Job Type

Full Time

Job Description

JD:1 Data Engineer

Location: Bangalore or Hyderabad

Exp: 7-9+ years.

Key Responsibilities

Design and implement end-to-end ETL/ELT pipelines using Databricks and Snowflake across structured, semi-structured, and unstructured data formats.
Build and manage scalable, secure data platforms using PostgreSQL and DynamoDB tailored to application needs.
Develop real-time and batch ingestion pipelines from diverse sources including APIs, logs, files, and databases.
Apply transformation logic to clean, enrich, and normalize data for analytics and reporting use cases.
Optimize pipeline performance and storage costs using best practices in partitioning, indexing, and query tuning.
Implement robust data quality checks, access control, backup strategies, and governance policies aligned with enterprise standards.

Core skill

Proven expertise in building batch and streaming data pipelines using Databricks (PySpark) and Snowflake.
Strong programming skills in SQL, Python, and PySpark for scalable data processing.
In-depth knowledge of Azure and AWS data ecosystems, including Microsoft Fabric and distributed computing frameworks.
Proficient in processing structured (CSV, SQL), semi-structured (JSON, XML), and unstructured (PDF, logs) data.
Experience with designing low-latency, high-throughput pipelines using Spark and cloud-native tools.
Solid understanding of CI/CD, automation, schema versioning, and data security in production environments.

More Jobs at codinglimits

Oracle Recruitng Cloud

India

Experience: Not specified

Salary: Not disclosed

Senior Technical Lead - Generative AI

Noida, Uttar Pradesh, India

10.0 - 13.0 yrs

Salary: Not disclosed

SAP BTP UI5 FIORI CAPM

Sri Potti Sriramulu Nellore, Andhra Pradesh, India

4.0 - 4.0 yrs

Salary: Not disclosed

SAP GTS Consultant

Hyderabad, Telangana, India

Experience: Not specified

Salary: Not disclosed

HAM Solutions consultant

Hyderabad, Telangana, India

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.