Athena Consultancy Services logo

Big Data Engineer

Athena Consultancy Services

3 - 6 years

18 - 22 Lacs

Pune

Posted:1 week ago

Job Description

email-bushra@bharatjobs.com

JD - Big Data Engineer Key Responsibilities Design, develop, and optimize scalable data pipelines for batch and real-time processing. Build and maintain ETL/ELT workflows using modern data engineering frameworks. Process and analyze large datasets using distributed computing technologies. Develop and optimize Spark applications for high-volume data processing. Build streaming data solutions using Kafka and related technologies. Implement workflow orchestration and scheduling using Airflow. Work with data storage systems including HDFS, S3, Hive, and NoSQL databases. Ensure data quality, reliability, and observability across data platforms. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Monitor and troubleshoot production data pipelines and platform issues. Implement CI/CD practices and automation for data engineering workflows. Contribute to platform modernization initiatives involving containerization and cloud-native technologies. Required Technical Skills Programming Python Scala SQL Java is a plus Big Data Technologies Apache Spark (Core, SQL, Structured Streaming) Apache Hive Apache Kafka Apache NiFi Apache Airflow Hadoop Ecosystem (HDFS, YARN) Aerospike is a plus Kibana Grafana Elasticsearch DevOps & Platform Tools Git Jenkins Linux Bitbucket Jira Confluence Kubernetes Podman

Skills Required

Spark NoSQL Bitbucket SQL Apache Reliability Big Data Kafka

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Work from Office

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Big Data Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Athena Consultancy Services logo
Athena Consultancy Services

RecommendedJobs for You

SP Software logo

Bengaluru

Bristlecone logo

Bengaluru

Athena Consultancy Services logo

Big Data Engineer

Athena Consultancy Services

Pune

Evoke HR logo

Chennai, Bengaluru

B

Big Data Engineer

BRISKWIN IT SOLUTIONS

Chennai

HNM Systems logo

Hyderabad

Ambiquest Hyderabad logo

Big Data Engineer

Ambiquest Hyderabad

Chennai, Bengaluru

dunnhumby logo

Gurugram