Data Platform Engineer

Umanist NA

0 years

0 Lacs

hyderabad telangana india

Posted:17 hours ago| Platform:

Apply

Skills Required

data design etl aws databricks apache airflow storage relational nosql github jenkins terraform architecture code debugging testing pipeline analysis ai ml governance automation support documentation test coding certifications interface python pyspark sql processing azure gcp redshift development provisioning devops deployment compliance security optimization mongodb dynamodb integration

Work Mode

On-site

Job Type

Full Time

Job Description

Data Platform Engineer 241

Key Responsibilities

Develop and maintain end-to-end data pipelines for ingesting, wrangling, transforming, and joining data from diverse sources.
Design and implement scalable, high-performance ETL pipelines using tools such as AWS Glue, Databricks, Apache Airflow, and DataProc.
Develop data models, schemas, and storage solutions for relational, NoSQL, and lakehouse environments (Snowflake, BigQuery, Delta Lake).
Optimize data pipelines for performance and cost, including infrastructure considerations.
Implement CI/CD pipelines for data solutions and infrastructure using tools like GitHub Actions, Jenkins, Terraform, or similar.
Collaborate with teams to define architecture, design patterns, and reusable solutions for complex distributed systems.
Perform code reviews, debugging, testing, and validation to ensure high-quality and reliable data solutions.
Monitor pipeline performance, identify defects, perform root cause analysis (RCA), and implement preventive measures.
Work with AI/ML teams to enable LLM-based data classification, data enrichment, and data governance automation.
Support documentation, including design documents, architecture diagrams, source-target mappings, test cases, and release notes.
Mentor and guide team members on best practices, coding standards, and technical certifications.
Interface with stakeholders to clarify requirements, present design options, and provide guidance on implementation.

Required Skills & Expertise

Strong proficiency in Python, PySpark, and SQL for large-scale data processing.
Hands-on experience with cloud platforms (AWS, Azure, or GCP) and their data services (S3, Redshift, RDS, BigQuery, Lambda, Glue, DataProc, CloudFormation, CloudWatch).
Extensive experience in ETL pipeline development, data lakes, data warehouses, and distributed systems.
Knowledge of IaC tools such as Terraform for provisioning and managing cloud infrastructure.
Experience with CI/CD pipelines and DevOps practices for data and infrastructure deployment.
Familiarity with data governance, compliance, security, and performance optimization.
Experience with NoSQL databases (Document DB, MongoDB, DynamoDB) and analytical data formats (Parquet, Iceberg).
Working knowledge of LLM integration for data classification, RAG workflows, or AI-assisted data processing is a plus.

Skills: classification,design,infrastructure,cloud,aws,ci,pipelines,cd,data,architecture

More Jobs at Umanist NA

Business Intelligence Developer

Goregaon, Maharashtra, India

5.0 - 5.0 yrs

INR 15 - 18 Lacs

AI Intern

Kanpur, Uttar Pradesh, India

Experience: Not specified

Salary: Not disclosed

Relationship Manager

Ahmedabad, Gujarat, India

1.0 - 1.0 yrs

INR 3 - 4 Lacs

Relationship Manager

Gandhinagar, Gujarat, India

1.0 - 1.0 yrs

INR 3 - 4 Lacs

Relationship Manager

Surat, Gujarat, India

1.0 - 1.0 yrs

INR 3 - 4 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.