Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in hyderabad
>
Pennant Technologies
>
Big Data Engineer

Big Data Engineer

Pennant Technologies

5 - 10 years

5 - 9 Lacs

hyderabad

Posted:7 hours ago| Platform:

Apply

Skills Required

gcp analytical debugging agile sensors apache analytics sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Title:

Senior Big Data Engineer

Department:

Data & Analytics

Employment Type:

Full time

Role Overview

We are seeking an experienced Big Data Engineer to join our growing Data & Analytics team. In this role, you ll design, build, and optimize our modern data platform enabling scalable pipelines, reliable data lakes, and robust ML operations. You ll work closely with data scientists, analysts, and stakeholders to turn raw data into actionable insights.

Key Responsibilities

Data Platform Development & Maintenance
Design, deploy, and maintain ETL/ELT pipelines using Apache Airflow and dbt/meltano.
Architect and manage data lakes on HDFS and object storage (e.g., S3/HDFS).
Implement table formats and governance with Apache Iceberg and Unity Catalog/Apache ranger/Atlas.
Big Data Processing
Develop distributed data processing jobs in Apache Spark (Scala/Java/Python).
Optimize Spark applications for performance, cost, and scalability.
Good working knowledge in Databases (Postgres, Mongo, SQL, Sql server etc)
Collaboration with Data science teams
Integrate MLflow for experiment tracking, model registry, and seamless deployment.
Collaborate with data science teams to operationalize models in production.
Containerization & Orchestration
Containerize data services and workflows using Docker/kubenetes.
Orchestrate deployments on Kubernetes clusters (EKS/GKE/AKS).
Center of Excellence & Best Practices
Contribute to and govern the dbt Center of Excellence: define standards, code reviews, and documentation.
Advocate and enforce data engineering best practices: CI/CD, testing, monitoring, and observability.
Cross Functional Collaboration
Partner with Analytics, Product, and Engineering teams to understand requirements and deliver solutions.
Mentor junior engineers and share knowledge through code reviews and technical talks.

Required Skills & Experience

5+ years of professional experience in Big Data engineering or similar roles.
Airflow:
Hands on experience authoring DAGs, configuring operators/sensors, and managing scheduler/executor backends.
Apache Spark:
Proficient in building and tuning large scale ETL jobs (Scala/Python).
Apache Iceberg & Unity Catalog:
Familiarity with table formats, partitioning strategies, and catalog management.
Data Lake & HDFS:
Solid understanding of storage architectures, file formats (Parquet/ORC), and data partitioning.
MLflow:
Working knowledge of experiment tracking, model registry, and deployment workflows.
Kubernetes:
Skilled in deploying, scaling, and debugging containerized applications and stateful workloads.
dbt COE:
Experience with dbt for data modeling, CI/CD integration, and establishing a CoE for standards.
Strong programming skills in Python and/or Scala.
Experience with version control (Git), CI/CD pipelines (Jenkins/GitHub Actions), and monitoring tools (Prometheus, Grafana).

Nice to Have

Familiarity with cloud platforms (AWS, GCP, or Azure) and managed services (EMR, Dataproc, EKS, GKE).
Knowledge of additional data catalogs (e.g., AWS Glue, Hive Metastore).
Background in event driven architectures (Kafka, Pulsar).
Prior involvement in data governance or compliance initiatives.

Education & Certifications

Bachelor s or Master s degree in Computer Science, Engineering, or related field.
Relevant certifications (e.g., Any cloud certification AWS/GCP/Azure/Databricks Certified Big Data Specialty) are a plus.

Soft Skills

Excellent problem solving and analytical abilities.
Strong written and verbal communication; able to explain technical concepts to non technical stakeholders.
Proactive mindset with a passion for learning and driving innovation.
Collaborative team player who thrives in agile, fast paced environments.

More Jobs at Pennant Technologies

Social Media Marketing Lead

Hyderābād

8.0 - 10.0 yrs

INR 3 - 5 Lacs

Lead Business Solutions Consultant - Lending (Retail & Corporate)

Hyderābād

5.0 - 10.0 yrs

Salary: Not disclosed

Assistant Vice President (AVP) - AI/ML Technologies

Hyderābād

10.0 - 10.0 yrs

INR 6 - 7 Lacs

Senior Manager, Presales

Hyderābād

5.0 - 5.0 yrs

Salary: Not disclosed

Lead Engineer, Support

Visakhapatnam

8.0 - 13.0 yrs

INR 15 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

Pennant Technologies

Software Development

Hyderabad Telangana

Before You Leave... Find Your Perfect Job!

Login to

Please Verify Your Phone or Email

Confirm Action

Contact Us

Big Data Engineer

Experience & Salary

Skills Required

Work Mode

Job Type

Job Description

Job Title:

Department:

Employment Type:

Role Overview

Key Responsibilities

Data Platform Development & Maintenance

Big Data Processing

Collaboration with Data science teams

Containerization & Orchestration

Center of Excellence & Best Practices

Cross Functional Collaboration

Required Skills & Experience

Airflow:

Apache Spark:

Apache Iceberg & Unity Catalog:

Data Lake & HDFS:

MLflow:

Kubernetes:

dbt COE:

Nice to Have

Education & Certifications

Soft Skills

More Jobs at Pennant Technologies