Data Engineer

Tata Consultancy Services

5 - 9 years

10 - 20 Lacs

pune bengaluru

Posted:2 hours ago| Platform:

Apply

Skills Required

pyspark scala aws python sql spark

Work Mode

Work from Office

Job Type

Full Time

Job Description

Role & responsibilities

Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility.
Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data.
Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency
Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle.
Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.

Preferred candidate profile

Candidate must have strong knowledge about Spark & Scala
Strong experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks.
Strong Expertise in SQL and/or NoSQL databases
In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses.
Having sound knowledge on
pyspark/spark framework
with
python/scala
as a programming language
Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams.
Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or Informatica.

More Jobs at Tata Consultancy Services

Tcs Hiring For GRC , Pan India

Pune, Bengaluru, Mumbai (All Areas)

7 - 12 yrs

INR 9 - 19 Lacs

Java / Spring boot with API Developer

Pune

5 - 10 yrs

INR 9 - 19 Lacs

SAP PP and QM Consultant

Pune

4 - 9 yrs

INR 8 - 18 Lacs

SAP BTP Consultant

Pune

4 - 9 yrs

INR 9 - 19 Lacs

Customer Service Associate

Pune

1 - 6 yrs

INR 1 - 5 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.