Big Data Engineer

MyCareernet

0 years

0 Lacs

gurugram haryana india

Posted:12 hours ago| Platform:

Apply

Skills Required

data pyspark aws python scala etl linux analytics design metrics reports automate efficiency sql support unix scripting processing redshift storage pipeline software engineering coding code testing technology

Work Mode

On-site

Job Type

Full Time

Job Description

Company:Indian / Global Digital Organization

Key Skills:

Roles and Responsibilities:

Develop and deploy ETL and data warehousing solutions using Python libraries and Linux bash scripts on AWS EC2, with data stored in Redshift.
Collaborate with product and analytics teams to scope business needs, design metrics, and build reports/dashboards.
Automate and optimize existing data sets and ETL pipelines for efficiency and reliability.
Work with multi-terabyte data sets and write complex SQL queries to support analytics.
Design and implement ETL solutions integrating multiple data sources using Pentaho.
Utilize Linux/Unix scripting for data processing tasks.
Leverage AWS services (Redshift, S3, EC2) for storage, processing, and pipeline automation.
Follow software engineering best practices for coding standards, code reviews, source control, testing, and operations.

Skills Required:

Must-Have:

Hands-on experience with PySpark for big data processing
Strong knowledge of AWS services (Redshift, S3, EC2)
Proficiency in Python for data processing and automation
Strong SQL skills for working with RDBMS and multi-terabyte data sets

Nice-to-Have:

Experience with SCALA for distributed data processing
Knowledge of ETL tools such as Pentaho
Familiarity with Linux/Unix scripting for data operations
Exposure to data modeling, pipelines, and visualization

Education:

More Jobs at MyCareernet

Software Engineer

Pune, Maharashtra, India

5 - 8 yrs

Salary: Not disclosed

Quality Assurance Engineer

Hyderabad, Telangana, India

3 - 5 yrs

Salary: Not disclosed

Data Analyst

Pune, Maharashtra, India

Experience: Not specified

Salary: Not disclosed

Business Analyst

Mumbai, Maharashtra, India

Experience: Not specified

Salary: Not disclosed

Software Engineer - Java Full Stack

Hyderabad, Telangana, India

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.