Posted:2 days ago|
Platform:
On-site
Full Time
Job Summary:
We are looking for a skilled Senior Data Engineer with strong expertise in Spark and Scala on the AWS platform. The ideal candidate should possess excellent problem-solving skills and hands-on experience in Spark-based data processing within a cloud-based ecosystem. This role offers the opportunity to independently execute diverse and complex engineering tasks, demonstrate a solid understanding of the end-to-end software development lifecycle, and collaborate effectively with stakeholders to deliver high-quality technical solutions.
Key Responsibilities:
Develop, analyze, debug, and enhance Spark-Scala programs.
Work on Spark batch processing jobs, with the ability to analyze/debug using Spark UI and logs.
Optimize performance of Spark applications and ensure scalability and reliability.
Manage data processing tasks using AWS S3, AWS EMR clusters, and other AWS services.
Leverage Hadoop ecosystem tools including HDFS, HBase, Hive, and MapReduce.
Write efficient and optimized SQL queries; experience with PostgreSQL and Couchbase or similar databases is preferred.
Utilize orchestration tools such as Kafka, NiFi, and Oozie.
Work with monitoring tools like Dynatrace and CloudWatch.
Contribute to the creation of High-Level Design (HLD) and Low-Level Design (LLD) documents and participate in reviews with architects.
Support development and lower environments setup, including local IDE configuration.
Follow defined coding standards, best practices, and quality processes.
Collaborate using Agile methodologies for development, review, and delivery.
Use supplementary programming languages like Python as needed.
Required Skills:
Mandatory:
Apache Spark
Scala
Big Data Hadoop Ecosystem
Spark SQL
Additional Preferred Skills:
Spring Core Framework
Core Java, Hibernate, Multithreading
AWS EMR, S3, CloudWatch
HDFS, HBase, Hive, MapReduce
PostgreSQL, Couchbase
Kafka, NiFi, Oozie
Dynatrace or other monitoring tools
Python (as supplementary language)
Agile Methodology
LTIMindtree
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Practice Python coding challenges to boost your skills
Start Practicing Python NowExperience: Not specified
Salary: Not disclosed
Experience: Not specified
Salary: Not disclosed
