Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in hyderabad
>
IDESLABS
>
Data Engineer - Hadoop + Spark

Data Engineer - Hadoop + Spark

IDESLABS

4 - 6 years

8 - 10 Lacs

hyderabad telangana india

Posted:1 week ago| Platform: Foundit logo

Apply

Skills Required

ci/cd

Work Mode

On-site

Job Type

Full Time

Job Description

Roles & Responsibilities:

Data Transformation and Aggregation

Design and develop Scala, Spark, and PySpark jobs for data transformation and aggregation.
Use Spark and Spark-SQL to read Parquet data and create Hive tables using Scala API.
Write unit tests for Spark transformations and helper methods.

Collaboration with Business Teams

Work closely with Business Analysts to review test results and obtain approvals.

Documentation and Knowledge Sharing

Prepare design and operational documentation for future reference.
Conduct peer code reviews to maintain code quality standards.

Code Quality and Best Practices

Ensure high-quality coding standards and act as a gatekeeper for code reviews.
Engage in hands-on coding, often in a pair-programming environment.

Agile Delivery and Teamwork

Work in highly collaborative teams to build reliable and scalable code.
Follow Agile methodologies and DevOps practices for efficient delivery.

Data Engineering and Application Development

Apply strong understanding of data structures, distributed processing, and data manipulation.
Develop applications using Hadoop tech stack and ensure timely and cost-effective delivery.
Implement automation to improve efficiency and reliability.

Production Support

Ensure smooth production deployments as per schedule and verify post-deployment success.
Address and resolve team queries related to application development and deployment.

Technology Exposure

Work with Hadoop ecosystem, Spark Streaming, Kafka, ML, Oracle, Hive, Impala, CI/CD, Git, Jenkins, and Cloudera Distribution.

More Jobs at IDESLABS

Looking For Intimidate Joiners For Netscaler

Bengaluru

6 - 11 yrs

INR 10 - 20 Lacs

SMG Professional

Bengaluru

3.0 - 8.0 yrs

INR 6 - 9 Lacs

Frontend React JS Developer

Hyderabad

5.0 - 10.0 yrs

INR 8 - 13 Lacs

PMO

Gurugram

6.0 - 11.0 yrs

INR 3 - 6 Lacs

Immediate Openings on SAP Modules PAN India (Contract) For ideslabs

Hyderabad, Pune, Bengaluru

6.0 - 10.0 yrs

INR 15 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.