Data Engineer

Sightspectrum

3 - 4 years

11 - 15 Lacs

chennai

Posted:None| Platform:

Apply

Skills Required

helm kubernetes python sql cloud storage bigquery cloud run gcp jenkins bigtable. pub/sub dataflow cloud functions

Work Mode

Remote

Job Type

Full Time

Job Description

Google Cloud Platform (GCP) Tools:(Mandatory) - BigQuery, Cloud Storage, Dataflow, Cloud Functions, Pub/Sub, Cloud Run, Cloud Composer (Airflow), Cloud Spanner, Bigtable.
Container Orchestration - Kubernetes (preferred on GKE) and Helm for managing and deploying containerized applications.
CI/CD and Automation- Jenkins for building CI/CD pipelines to automate deployment and testing of data pipelines.
Develop and Optimize Pipelines: Write efficient Python and SQL scripts to build data pipelines and ETL/ELT processes. Continuously monitor and optimize data workflows for performance and cost-effectiveness.
Data Integration and Orchestration: Design workflows to integrate data from various sources using GCP services, and orchestrate complex tasks with Cloud Composer (Apache Airflow)
Programming Languages- Proficient in Python for data processing and automation, SQL for querying and data manipulation. Experience with Java is a plus.
DevOps Tools - Familiarity with Terraform or Deployment Manager for Infrastructure as Code (IaC) to manage GCP resources.
Monitoring and Logging -Experience with Cloud Monitoring, Datadog, or other monitoring solutions to track pipeline performance and ensure operational efficiency.
Data Engineering Skills Expertise in ETL/ELT pipelines, data modeling, and data integration across large datasets.
Strong understanding of data warehousing and real-time data processing workflows.
Strong communication skills to work effectively with cross-functional teams and mentor junior developers. Proven ability to lead in an Agile environment.
3+ years of experience as a data engineer, with hands-on experience in Kubernetes, Helm, Python and Jenkins.
Strong experience building and optimizing data pipelines and services in any cloud platform.
Proficiency in Python and SQL. Familiarity with Java and Docker is a plus.

More Jobs at Sightspectrum

Big Data Developer

Pune, Chennai, Bengaluru

5.0 - 10.0 yrs

INR 13 - 23 Lacs

Big Data Engineer

Pune, Chennai

5.0 - 8.0 yrs

INR 5 - 12 Lacs

It Recruiter

Chennai

Experience: Not specified

INR 0 - 3 Lacs

IAAC Engineer

Kolkata, Pune, Chennai

7.0 - 10.0 yrs

INR 15 - 20 Lacs

Solution Architect

Pune, Chennai, Bengaluru

10.0 - 15.0 yrs

INR 20 - 32 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.